tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm: add mmap_action_map_kernel_pages[_full]()

A user can invoke mmap_action_map_kernel_pages() to specify that the
mapping should map kernel pages starting from desc->start of a specified
number of pages specified in an array.

In order to implement this, adjust mmap_action_prepare() to be able to
return an error code, as it makes sense to assert that the specified
parameters are valid as quickly as possible as well as updating the VMA
flags to include VMA_MIXEDMAP_BIT as necessary.

This provides an mmap_prepare equivalent of vm_insert_pages(). We
additionally update the existing vm_insert_pages() code to use
range_in_vma() and add a new range_in_vma_desc() helper function for the
mmap_prepare case, sharing the code between the two in range_is_subset().

We add both mmap_action_map_kernel_pages() and
mmap_action_map_kernel_pages_full() to allow for both partial and full VMA
mappings.

We update the documentation to reflect the new features.

Finally, we update the VMA tests accordingly to reflect the changes.

Link: https://lkml.kernel.org/r/926ac961690d856e67ec847bee2370ab3c6b9046.1774045440.git.ljs@kernel.org
Signed-off-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Bodo Stroesser <bostroesser@gmail.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Clemens Ladisch <clemens@ladisch.de>
Cc: David Hildenbrand <david@kernel.org>
Cc: David Howells <dhowells@redhat.com>
Cc: Dexuan Cui <decui@microsoft.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jann Horn <jannh@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Long Li <longli@microsoft.com>
Cc: Marc Dionne <marc.dionne@auristor.com>
Cc: "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Miquel Raynal <miquel.raynal@bootlin.com>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: Richard Weinberger <richard@nod.at>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Vignesh Raghavendra <vigneshr@ti.com>
Cc: Wei Liu <wei.liu@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Lorenzo Stoakes (Oracle) and committed by
Andrew Morton 62c65fd7 933f05f5

+160 -6
+8
Documentation/filesystems/mmap_prepare.rst
··· 156 156 * mmap_action_simple_ioremap() - Sets up an I/O remap from a specified 157 157 physical address and over a specified length. 158 158 159 + * mmap_action_map_kernel_pages() - Maps a specified array of `struct page` 160 + pointers in the VMA from a specific offset. 161 + 162 + * mmap_action_map_kernel_pages_full() - Maps a specified array of `struct 163 + page` pointers over the entire VMA. The caller must ensure there are 164 + sufficient entries in the page array to cover the entire range of the 165 + described VMA. 166 + 159 167 **NOTE:** The ``action`` field should never normally be manipulated directly, 160 168 rather you ought to use one of these helpers.
+93 -2
include/linux/mm.h
··· 2905 2905 * The caller must add any reference (e.g., from folio_try_get()) it might be 2906 2906 * holding itself to the result. 2907 2907 * 2908 - * Returns the expected folio refcount. 2908 + * Returns: the expected folio refcount. 2909 2909 */ 2910 2910 static inline int folio_expected_ref_count(const struct folio *folio) 2911 2911 { ··· 4348 4348 action->type = MMAP_SIMPLE_IO_REMAP; 4349 4349 } 4350 4350 4351 + /** 4352 + * mmap_action_map_kernel_pages - helper for mmap_prepare hook to specify that 4353 + * @num kernel pages contained in the @pages array should be mapped to userland 4354 + * starting at virtual address @start. 4355 + * @desc: The VMA descriptor for the VMA requiring kernel pags to be mapped. 4356 + * @start: The virtual address from which to map them. 4357 + * @pages: An array of struct page pointers describing the memory to map. 4358 + * @nr_pages: The number of entries in the @pages aray. 4359 + */ 4360 + static inline void mmap_action_map_kernel_pages(struct vm_area_desc *desc, 4361 + unsigned long start, struct page **pages, 4362 + unsigned long nr_pages) 4363 + { 4364 + struct mmap_action *action = &desc->action; 4365 + 4366 + action->type = MMAP_MAP_KERNEL_PAGES; 4367 + action->map_kernel.start = start; 4368 + action->map_kernel.pages = pages; 4369 + action->map_kernel.nr_pages = nr_pages; 4370 + action->map_kernel.pgoff = desc->pgoff; 4371 + } 4372 + 4373 + /** 4374 + * mmap_action_map_kernel_pages_full - helper for mmap_prepare hook to specify that 4375 + * kernel pages contained in the @pages array should be mapped to userland 4376 + * from @desc->start to @desc->end. 4377 + * @desc: The VMA descriptor for the VMA requiring kernel pags to be mapped. 4378 + * @pages: An array of struct page pointers describing the memory to map. 4379 + * 4380 + * The caller must ensure that @pages contains sufficient entries to cover the 4381 + * entire range described by @desc. 4382 + */ 4383 + static inline void mmap_action_map_kernel_pages_full(struct vm_area_desc *desc, 4384 + struct page **pages) 4385 + { 4386 + mmap_action_map_kernel_pages(desc, desc->start, pages, 4387 + vma_desc_pages(desc)); 4388 + } 4389 + 4351 4390 int mmap_action_prepare(struct vm_area_desc *desc); 4352 4391 int mmap_action_complete(struct vm_area_struct *vma, 4353 4392 struct mmap_action *action); ··· 4403 4364 return vma; 4404 4365 } 4405 4366 4367 + /** 4368 + * range_is_subset - Is the specified inner range a subset of the outer range? 4369 + * @outer_start: The start of the outer range. 4370 + * @outer_end: The exclusive end of the outer range. 4371 + * @inner_start: The start of the inner range. 4372 + * @inner_end: The exclusive end of the inner range. 4373 + * 4374 + * Returns: %true if [inner_start, inner_end) is a subset of [outer_start, 4375 + * outer_end), otherwise %false. 4376 + */ 4377 + static inline bool range_is_subset(unsigned long outer_start, 4378 + unsigned long outer_end, 4379 + unsigned long inner_start, 4380 + unsigned long inner_end) 4381 + { 4382 + return outer_start <= inner_start && inner_end <= outer_end; 4383 + } 4384 + 4385 + /** 4386 + * range_in_vma - is the specified [@start, @end) range a subset of the VMA? 4387 + * @vma: The VMA against which we want to check [@start, @end). 4388 + * @start: The start of the range we wish to check. 4389 + * @end: The exclusive end of the range we wish to check. 4390 + * 4391 + * Returns: %true if [@start, @end) is a subset of [@vma->vm_start, 4392 + * @vma->vm_end), %false otherwise. 4393 + */ 4406 4394 static inline bool range_in_vma(const struct vm_area_struct *vma, 4407 4395 unsigned long start, unsigned long end) 4408 4396 { 4409 - return (vma && vma->vm_start <= start && end <= vma->vm_end); 4397 + if (!vma) 4398 + return false; 4399 + 4400 + return range_is_subset(vma->vm_start, vma->vm_end, start, end); 4401 + } 4402 + 4403 + /** 4404 + * range_in_vma_desc - is the specified [@start, @end) range a subset of the VMA 4405 + * described by @desc, a VMA descriptor? 4406 + * @desc: The VMA descriptor against which we want to check [@start, @end). 4407 + * @start: The start of the range we wish to check. 4408 + * @end: The exclusive end of the range we wish to check. 4409 + * 4410 + * Returns: %true if [@start, @end) is a subset of [@desc->start, @desc->end), 4411 + * %false otherwise. 4412 + */ 4413 + static inline bool range_in_vma_desc(const struct vm_area_desc *desc, 4414 + unsigned long start, unsigned long end) 4415 + { 4416 + if (!desc) 4417 + return false; 4418 + 4419 + return range_is_subset(desc->start, desc->end, start, end); 4410 4420 } 4411 4421 4412 4422 #ifdef CONFIG_MMU ··· 4499 4411 int vm_insert_page(struct vm_area_struct *, unsigned long addr, struct page *); 4500 4412 int vm_insert_pages(struct vm_area_struct *vma, unsigned long addr, 4501 4413 struct page **pages, unsigned long *num); 4414 + int map_kernel_pages_prepare(struct vm_area_desc *desc); 4415 + int map_kernel_pages_complete(struct vm_area_struct *vma, 4416 + struct mmap_action *action); 4502 4417 int vm_map_pages(struct vm_area_struct *vma, struct page **pages, 4503 4418 unsigned long num); 4504 4419 int vm_map_pages_zero(struct vm_area_struct *vma, struct page **pages,
+7
include/linux/mm_types.h
··· 815 815 MMAP_REMAP_PFN, /* Remap PFN range. */ 816 816 MMAP_IO_REMAP_PFN, /* I/O remap PFN range. */ 817 817 MMAP_SIMPLE_IO_REMAP, /* I/O remap with guardrails. */ 818 + MMAP_MAP_KERNEL_PAGES, /* Map kernel page range from array. */ 818 819 }; 819 820 820 821 /* ··· 834 833 phys_addr_t start_phys_addr; 835 834 unsigned long size; 836 835 } simple_ioremap; 836 + struct { 837 + unsigned long start; 838 + struct page **pages; 839 + unsigned long nr_pages; 840 + pgoff_t pgoff; 841 + } map_kernel; 837 842 }; 838 843 enum mmap_action_type type; 839 844
+38 -4
mm/memory.c
··· 2484 2484 int vm_insert_pages(struct vm_area_struct *vma, unsigned long addr, 2485 2485 struct page **pages, unsigned long *num) 2486 2486 { 2487 - const unsigned long end_addr = addr + (*num * PAGE_SIZE) - 1; 2487 + const unsigned long nr_pages = *num; 2488 + const unsigned long end = addr + PAGE_SIZE * nr_pages; 2488 2489 2489 - if (addr < vma->vm_start || end_addr >= vma->vm_end) 2490 + if (!range_in_vma(vma, addr, end)) 2490 2491 return -EFAULT; 2491 2492 if (!(vma->vm_flags & VM_MIXEDMAP)) { 2492 - BUG_ON(mmap_read_trylock(vma->vm_mm)); 2493 - BUG_ON(vma->vm_flags & VM_PFNMAP); 2493 + VM_WARN_ON_ONCE(mmap_read_trylock(vma->vm_mm)); 2494 + VM_WARN_ON_ONCE(vma->vm_flags & VM_PFNMAP); 2494 2495 vm_flags_set(vma, VM_MIXEDMAP); 2495 2496 } 2496 2497 /* Defer page refcount checking till we're about to map that page. */ 2497 2498 return insert_pages(vma, addr, pages, num, vma->vm_page_prot); 2498 2499 } 2499 2500 EXPORT_SYMBOL(vm_insert_pages); 2501 + 2502 + int map_kernel_pages_prepare(struct vm_area_desc *desc) 2503 + { 2504 + const struct mmap_action *action = &desc->action; 2505 + const unsigned long addr = action->map_kernel.start; 2506 + unsigned long nr_pages, end; 2507 + 2508 + if (!vma_desc_test(desc, VMA_MIXEDMAP_BIT)) { 2509 + VM_WARN_ON_ONCE(mmap_read_trylock(desc->mm)); 2510 + VM_WARN_ON_ONCE(vma_desc_test(desc, VMA_PFNMAP_BIT)); 2511 + vma_desc_set_flags(desc, VMA_MIXEDMAP_BIT); 2512 + } 2513 + 2514 + nr_pages = action->map_kernel.nr_pages; 2515 + end = addr + PAGE_SIZE * nr_pages; 2516 + if (!range_in_vma_desc(desc, addr, end)) 2517 + return -EFAULT; 2518 + 2519 + return 0; 2520 + } 2521 + EXPORT_SYMBOL(map_kernel_pages_prepare); 2522 + 2523 + int map_kernel_pages_complete(struct vm_area_struct *vma, 2524 + struct mmap_action *action) 2525 + { 2526 + unsigned long nr_pages; 2527 + 2528 + nr_pages = action->map_kernel.nr_pages; 2529 + return insert_pages(vma, action->map_kernel.start, 2530 + action->map_kernel.pages, 2531 + &nr_pages, vma->vm_page_prot); 2532 + } 2533 + EXPORT_SYMBOL(map_kernel_pages_complete); 2500 2534 2501 2535 /** 2502 2536 * vm_insert_page - insert single page into user vma
+7
mm/util.c
··· 1448 1448 return io_remap_pfn_range_prepare(desc); 1449 1449 case MMAP_SIMPLE_IO_REMAP: 1450 1450 return simple_ioremap_prepare(desc); 1451 + case MMAP_MAP_KERNEL_PAGES: 1452 + return map_kernel_pages_prepare(desc); 1451 1453 } 1452 1454 1453 1455 WARN_ON_ONCE(1); ··· 1477 1475 case MMAP_REMAP_PFN: 1478 1476 err = remap_pfn_range_complete(vma, action); 1479 1477 break; 1478 + case MMAP_MAP_KERNEL_PAGES: 1479 + err = map_kernel_pages_complete(vma, action); 1480 + break; 1480 1481 case MMAP_IO_REMAP_PFN: 1481 1482 case MMAP_SIMPLE_IO_REMAP: 1482 1483 /* Should have been delegated. */ ··· 1500 1495 case MMAP_REMAP_PFN: 1501 1496 case MMAP_IO_REMAP_PFN: 1502 1497 case MMAP_SIMPLE_IO_REMAP: 1498 + case MMAP_MAP_KERNEL_PAGES: 1503 1499 WARN_ON_ONCE(1); /* nommu cannot handle these. */ 1504 1500 break; 1505 1501 } ··· 1520 1514 case MMAP_REMAP_PFN: 1521 1515 case MMAP_IO_REMAP_PFN: 1522 1516 case MMAP_SIMPLE_IO_REMAP: 1517 + case MMAP_MAP_KERNEL_PAGES: 1523 1518 WARN_ON_ONCE(1); /* nommu cannot handle this. */ 1524 1519 1525 1520 err = -EINVAL;
+7
tools/testing/vma/include/dup.h
··· 454 454 MMAP_REMAP_PFN, /* Remap PFN range. */ 455 455 MMAP_IO_REMAP_PFN, /* I/O remap PFN range. */ 456 456 MMAP_SIMPLE_IO_REMAP, /* I/O remap with guardrails. */ 457 + MMAP_MAP_KERNEL_PAGES, /* Map kernel page range from an array. */ 457 458 }; 458 459 459 460 /* ··· 473 472 phys_addr_t start_phys_addr; 474 473 unsigned long size; 475 474 } simple_ioremap; 475 + struct { 476 + unsigned long start; 477 + struct page **pages; 478 + unsigned long nr_pages; 479 + pgoff_t pgoff; 480 + } map_kernel; 476 481 }; 477 482 enum mmap_action_type type; 478 483