tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm: add atomic VMA flags and set VM_MAYBE_GUARD as such

This patch adds the ability to atomically set VMA flags with only the mmap
read/VMA read lock held.

As this could be hugely problematic for VMA flags in general given that
all other accesses are non-atomic and serialised by the mmap/VMA locks, we
implement this with a strict allow-list - that is, only designated flags
are allowed to do this.

We make VM_MAYBE_GUARD one of these flags.

Link: https://lkml.kernel.org/r/97e57abed09f2663077ed7a36fb8206e243171a9.1763460113.git.lorenzo.stoakes@oracle.com
Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: David Hildenbrand (Red Hat) <david@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Andrei Vagin <avagin@gmail.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jann Horn <jannh@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: "Masami Hiramatsu (Google)" <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Lorenzo Stoakes and committed by
Andrew Morton 56882250 5dba5cc2

+44
+44
include/linux/mm.h
··· 518 518 /* This mask represents all the VMA flag bits used by mlock */ 519 519 #define VM_LOCKED_MASK (VM_LOCKED | VM_LOCKONFAULT) 520 520 521 + /* These flags can be updated atomically via VMA/mmap read lock. */ 522 + #define VM_ATOMIC_SET_ALLOWED VM_MAYBE_GUARD 523 + 521 524 /* Arch-specific flags to clear when updating VM flags on protection change */ 522 525 #ifndef VM_ARCH_CLEAR 523 526 # define VM_ARCH_CLEAR VM_NONE ··· 861 858 { 862 859 vma_start_write(vma); 863 860 __vm_flags_mod(vma, set, clear); 861 + } 862 + 863 + static inline bool __vma_flag_atomic_valid(struct vm_area_struct *vma, 864 + int bit) 865 + { 866 + const vm_flags_t mask = BIT(bit); 867 + 868 + /* Only specific flags are permitted */ 869 + if (WARN_ON_ONCE(!(mask & VM_ATOMIC_SET_ALLOWED))) 870 + return false; 871 + 872 + return true; 873 + } 874 + 875 + /* 876 + * Set VMA flag atomically. Requires only VMA/mmap read lock. Only specific 877 + * valid flags are allowed to do this. 878 + */ 879 + static inline void vma_flag_set_atomic(struct vm_area_struct *vma, int bit) 880 + { 881 + /* mmap read lock/VMA read lock must be held. */ 882 + if (!rwsem_is_locked(&vma->vm_mm->mmap_lock)) 883 + vma_assert_locked(vma); 884 + 885 + if (__vma_flag_atomic_valid(vma, bit)) 886 + set_bit(bit, &ACCESS_PRIVATE(vma, __vm_flags)); 887 + } 888 + 889 + /* 890 + * Test for VMA flag atomically. Requires no locks. Only specific valid flags 891 + * are allowed to do this. 892 + * 893 + * This is necessarily racey, so callers must ensure that serialisation is 894 + * achieved through some other means, or that races are permissible. 895 + */ 896 + static inline bool vma_flag_test_atomic(struct vm_area_struct *vma, int bit) 897 + { 898 + if (__vma_flag_atomic_valid(vma, bit)) 899 + return test_bit(bit, &vma->vm_flags); 900 + 901 + return false; 864 902 } 865 903 866 904 static inline void vma_set_anonymous(struct vm_area_struct *vma)