tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm: various small mmap_prepare cleanups

Patch series "mm: expand mmap_prepare functionality and usage", v4.

This series expands the mmap_prepare functionality, which is intended to
replace the deprecated f_op->mmap hook which has been the source of bugs
and security issues for some time.

This series starts with some cleanup of existing mmap_prepare logic, then
adds documentation for the mmap_prepare call to make it easier for
filesystem and driver writers to understand how it works.

It then importantly adds a vm_ops->mapped hook, a key feature that was
missing from mmap_prepare previously - this is invoked when a driver which
specifies mmap_prepare has successfully been mapped but not merged with
another VMA.

mmap_prepare is invoked prior to a merge being attempted, so you cannot
manipulate state such as reference counts as if it were a new mapping.

The vm_ops->mapped hook allows a driver to perform tasks required at this
stage, and provides symmetry against subsequent vm_ops->open,close calls.

The series uses this to correct the afs implementation which wrongly
manipulated reference count at mmap_prepare time.

It then adds an mmap_prepare equivalent of vm_iomap_memory() -
mmap_action_simple_ioremap(), then uses this to update a number of drivers.

It then splits out the mmap_prepare compatibility layer (which allows for
invocation of mmap_prepare hooks in an mmap() hook) in such a way as to
allow for more incremental implementation of mmap_prepare hooks.

It then uses this to extend mmap_prepare usage in drivers.

Finally it adds an mmap_prepare equivalent of vm_map_pages(), which lays
the foundation for future work which will extend mmap_prepare to DMA
coherent mappings.


This patch (of 21):

Rather than passing arbitrary fields, pass a vm_area_desc pointer to mmap
prepare functions to mmap prepare, and an action and vma pointer to mmap
complete in order to put all the action-specific logic in the function
actually doing the work.

Additionally, allow mmap prepare functions to return an error so we can
error out as soon as possible if there is something logically incorrect in
the input.

Update remap_pfn_range_prepare() to properly check the input range for the
CoW case.

Also remove io_remap_pfn_range_complete(), as we can simply set up the
fields correctly in io_remap_pfn_range_prepare() and use
remap_pfn_range_complete() for this.

While we're here, make remap_pfn_range_prepare_vma() a little neater, and
pass mmap_action directly to call_action_complete().

Then, update compat_vma_mmap() to perform its logic directly, as
__compat_vma_map() is not used by anything so we don't need to export it.

Also update compat_vma_mmap() to use vfs_mmap_prepare() rather than
calling the mmap_prepare op directly.

Finally, update the VMA userland tests to reflect the changes.

Link: https://lkml.kernel.org/r/cover.1774045440.git.ljs@kernel.org
Link: https://lkml.kernel.org/r/99f408e4694f44ab12bdc55fe0bd9685d3bd1117.1774045440.git.ljs@kernel.org
Signed-off-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Alexandre Torgue <alexandre.torgue@foss.st.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Bodo Stroesser <bostroesser@gmail.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Clemens Ladisch <clemens@ladisch.de>
Cc: David Hildenbrand <david@kernel.org>
Cc: David Howells <dhowells@redhat.com>
Cc: Dexuan Cui <decui@microsoft.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jann Horn <jannh@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Long Li <longli@microsoft.com>
Cc: Marc Dionne <marc.dionne@auristor.com>
Cc: "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Maxime Coquelin <mcoquelin.stm32@gmail.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Miquel Raynal <miquel.raynal@bootlin.com>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: Richard Weinberger <richard@nod.at>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vignesh Raghavendra <vigneshr@ti.com>
Cc: Wei Liu <wei.liu@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Lorenzo Stoakes (Oracle) and committed by
Andrew Morton 3e4bb270 71fdcf92

+130 -124
-2
include/linux/fs.h
··· 2058 2058 return true; 2059 2059 } 2060 2060 2061 - int __compat_vma_mmap(const struct file_operations *f_op, 2062 - struct file *file, struct vm_area_struct *vma); 2063 2061 int compat_vma_mmap(struct file *file, struct vm_area_struct *vma); 2064 2062 2065 2063 static inline int vfs_mmap(struct file *file, struct vm_area_struct *vma)
+3 -4
include/linux/mm.h
··· 4304 4304 mmap_action_ioremap(desc, desc->start, start_pfn, vma_desc_size(desc)); 4305 4305 } 4306 4306 4307 - void mmap_action_prepare(struct mmap_action *action, 4308 - struct vm_area_desc *desc); 4309 - int mmap_action_complete(struct mmap_action *action, 4310 - struct vm_area_struct *vma); 4307 + int mmap_action_prepare(struct vm_area_desc *desc); 4308 + int mmap_action_complete(struct vm_area_struct *vma, 4309 + struct mmap_action *action); 4311 4310 4312 4311 /* Look up the first VMA which exactly match the interval vm_start ... vm_end */ 4313 4312 static inline struct vm_area_struct *find_exact_vma(struct mm_struct *mm,
+17 -15
mm/internal.h
··· 1839 1839 void dup_mm_exe_file(struct mm_struct *mm, struct mm_struct *oldmm); 1840 1840 int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm); 1841 1841 1842 - void remap_pfn_range_prepare(struct vm_area_desc *desc, unsigned long pfn); 1843 - int remap_pfn_range_complete(struct vm_area_struct *vma, unsigned long addr, 1844 - unsigned long pfn, unsigned long size, pgprot_t pgprot); 1842 + int remap_pfn_range_prepare(struct vm_area_desc *desc); 1843 + int remap_pfn_range_complete(struct vm_area_struct *vma, 1844 + struct mmap_action *action); 1845 1845 1846 - static inline void io_remap_pfn_range_prepare(struct vm_area_desc *desc, 1847 - unsigned long orig_pfn, unsigned long size) 1846 + static inline int io_remap_pfn_range_prepare(struct vm_area_desc *desc) 1848 1847 { 1848 + struct mmap_action *action = &desc->action; 1849 + const unsigned long orig_pfn = action->remap.start_pfn; 1850 + const pgprot_t orig_pgprot = action->remap.pgprot; 1851 + const unsigned long size = action->remap.size; 1849 1852 const unsigned long pfn = io_remap_pfn_range_pfn(orig_pfn, size); 1853 + int err; 1850 1854 1851 - return remap_pfn_range_prepare(desc, pfn); 1852 - } 1855 + action->remap.start_pfn = pfn; 1856 + action->remap.pgprot = pgprot_decrypted(orig_pgprot); 1857 + err = remap_pfn_range_prepare(desc); 1858 + if (err) 1859 + return err; 1853 1860 1854 - static inline int io_remap_pfn_range_complete(struct vm_area_struct *vma, 1855 - unsigned long addr, unsigned long orig_pfn, unsigned long size, 1856 - pgprot_t orig_prot) 1857 - { 1858 - const unsigned long pfn = io_remap_pfn_range_pfn(orig_pfn, size); 1859 - const pgprot_t prot = pgprot_decrypted(orig_prot); 1860 - 1861 - return remap_pfn_range_complete(vma, addr, pfn, size, prot); 1861 + /* Remap does the actual work. */ 1862 + action->type = MMAP_REMAP_PFN; 1863 + return 0; 1862 1864 } 1863 1865 1864 1866 #ifdef CONFIG_MMU_NOTIFIER
+33 -20
mm/memory.c
··· 3099 3099 } 3100 3100 #endif 3101 3101 3102 - void remap_pfn_range_prepare(struct vm_area_desc *desc, unsigned long pfn) 3102 + int remap_pfn_range_prepare(struct vm_area_desc *desc) 3103 3103 { 3104 - /* 3105 - * We set addr=VMA start, end=VMA end here, so this won't fail, but we 3106 - * check it again on complete and will fail there if specified addr is 3107 - * invalid. 3108 - */ 3109 - get_remap_pgoff(vma_desc_is_cow_mapping(desc), desc->start, desc->end, 3110 - desc->start, desc->end, pfn, &desc->pgoff); 3111 - vma_desc_set_flags_mask(desc, VMA_REMAP_FLAGS); 3112 - } 3113 - 3114 - static int remap_pfn_range_prepare_vma(struct vm_area_struct *vma, unsigned long addr, 3115 - unsigned long pfn, unsigned long size) 3116 - { 3117 - unsigned long end = addr + PAGE_ALIGN(size); 3104 + const struct mmap_action *action = &desc->action; 3105 + const unsigned long start = action->remap.start; 3106 + const unsigned long end = start + action->remap.size; 3107 + const unsigned long pfn = action->remap.start_pfn; 3108 + const bool is_cow = vma_desc_is_cow_mapping(desc); 3118 3109 int err; 3119 3110 3120 - err = get_remap_pgoff(is_cow_mapping(vma->vm_flags), addr, end, 3121 - vma->vm_start, vma->vm_end, pfn, &vma->vm_pgoff); 3111 + err = get_remap_pgoff(is_cow, start, end, desc->start, desc->end, pfn, 3112 + &desc->pgoff); 3113 + if (err) 3114 + return err; 3115 + 3116 + vma_desc_set_flags_mask(desc, VMA_REMAP_FLAGS); 3117 + return 0; 3118 + } 3119 + 3120 + static int remap_pfn_range_prepare_vma(struct vm_area_struct *vma, 3121 + unsigned long addr, unsigned long pfn, 3122 + unsigned long size) 3123 + { 3124 + const unsigned long end = addr + PAGE_ALIGN(size); 3125 + const bool is_cow = is_cow_mapping(vma->vm_flags); 3126 + int err; 3127 + 3128 + err = get_remap_pgoff(is_cow, addr, end, vma->vm_start, vma->vm_end, 3129 + pfn, &vma->vm_pgoff); 3122 3130 if (err) 3123 3131 return err; 3124 3132 ··· 3159 3151 } 3160 3152 EXPORT_SYMBOL(remap_pfn_range); 3161 3153 3162 - int remap_pfn_range_complete(struct vm_area_struct *vma, unsigned long addr, 3163 - unsigned long pfn, unsigned long size, pgprot_t prot) 3154 + int remap_pfn_range_complete(struct vm_area_struct *vma, 3155 + struct mmap_action *action) 3164 3156 { 3165 - return do_remap_pfn_range(vma, addr, pfn, size, prot); 3157 + const unsigned long start = action->remap.start; 3158 + const unsigned long pfn = action->remap.start_pfn; 3159 + const unsigned long size = action->remap.size; 3160 + const pgprot_t prot = action->remap.pgprot; 3161 + 3162 + return do_remap_pfn_range(vma, start, pfn, size, prot); 3166 3163 } 3167 3164 3168 3165 /**
+54 -67
mm/util.c
··· 1164 1164 #endif 1165 1165 1166 1166 /** 1167 - * __compat_vma_mmap() - See description for compat_vma_mmap() 1168 - * for details. This is the same operation, only with a specific file operations 1169 - * struct which may or may not be the same as vma->vm_file->f_op. 1170 - * @f_op: The file operations whose .mmap_prepare() hook is specified. 1171 - * @file: The file which backs or will back the mapping. 1172 - * @vma: The VMA to apply the .mmap_prepare() hook to. 1173 - * Returns: 0 on success or error. 1174 - */ 1175 - int __compat_vma_mmap(const struct file_operations *f_op, 1176 - struct file *file, struct vm_area_struct *vma) 1177 - { 1178 - struct vm_area_desc desc = { 1179 - .mm = vma->vm_mm, 1180 - .file = file, 1181 - .start = vma->vm_start, 1182 - .end = vma->vm_end, 1183 - 1184 - .pgoff = vma->vm_pgoff, 1185 - .vm_file = vma->vm_file, 1186 - .vma_flags = vma->flags, 1187 - .page_prot = vma->vm_page_prot, 1188 - 1189 - .action.type = MMAP_NOTHING, /* Default */ 1190 - }; 1191 - int err; 1192 - 1193 - err = f_op->mmap_prepare(&desc); 1194 - if (err) 1195 - return err; 1196 - 1197 - mmap_action_prepare(&desc.action, &desc); 1198 - set_vma_from_desc(vma, &desc); 1199 - return mmap_action_complete(&desc.action, vma); 1200 - } 1201 - EXPORT_SYMBOL(__compat_vma_mmap); 1202 - 1203 - /** 1204 1167 * compat_vma_mmap() - Apply the file's .mmap_prepare() hook to an 1205 1168 * existing VMA and execute any requested actions. 1206 1169 * @file: The file which possesss an f_op->mmap_prepare() hook. ··· 1191 1228 */ 1192 1229 int compat_vma_mmap(struct file *file, struct vm_area_struct *vma) 1193 1230 { 1194 - return __compat_vma_mmap(file->f_op, file, vma); 1231 + struct vm_area_desc desc = { 1232 + .mm = vma->vm_mm, 1233 + .file = file, 1234 + .start = vma->vm_start, 1235 + .end = vma->vm_end, 1236 + 1237 + .pgoff = vma->vm_pgoff, 1238 + .vm_file = vma->vm_file, 1239 + .vma_flags = vma->flags, 1240 + .page_prot = vma->vm_page_prot, 1241 + 1242 + .action.type = MMAP_NOTHING, /* Default */ 1243 + }; 1244 + int err; 1245 + 1246 + err = vfs_mmap_prepare(file, &desc); 1247 + if (err) 1248 + return err; 1249 + 1250 + err = mmap_action_prepare(&desc); 1251 + if (err) 1252 + return err; 1253 + 1254 + set_vma_from_desc(vma, &desc); 1255 + return mmap_action_complete(vma, &desc.action); 1195 1256 } 1196 1257 EXPORT_SYMBOL(compat_vma_mmap); 1197 1258 ··· 1307 1320 } 1308 1321 } 1309 1322 1310 - static int mmap_action_finish(struct mmap_action *action, 1311 - const struct vm_area_struct *vma, int err) 1323 + static int mmap_action_finish(struct vm_area_struct *vma, 1324 + struct mmap_action *action, int err) 1312 1325 { 1313 1326 /* 1314 1327 * If an error occurs, unmap the VMA altogether and return an error. We ··· 1340 1353 /** 1341 1354 * mmap_action_prepare - Perform preparatory setup for an VMA descriptor 1342 1355 * action which need to be performed. 1343 - * @desc: The VMA descriptor to prepare for @action. 1344 - * @action: The action to perform. 1356 + * @desc: The VMA descriptor to prepare for its @desc->action. 1357 + * 1358 + * Returns: %0 on success, otherwise error. 1345 1359 */ 1346 - void mmap_action_prepare(struct mmap_action *action, 1347 - struct vm_area_desc *desc) 1360 + int mmap_action_prepare(struct vm_area_desc *desc) 1348 1361 { 1349 - switch (action->type) { 1362 + switch (desc->action.type) { 1350 1363 case MMAP_NOTHING: 1351 - break; 1364 + return 0; 1352 1365 case MMAP_REMAP_PFN: 1353 - remap_pfn_range_prepare(desc, action->remap.start_pfn); 1354 - break; 1366 + return remap_pfn_range_prepare(desc); 1355 1367 case MMAP_IO_REMAP_PFN: 1356 - io_remap_pfn_range_prepare(desc, action->remap.start_pfn, 1357 - action->remap.size); 1358 - break; 1368 + return io_remap_pfn_range_prepare(desc); 1359 1369 } 1370 + 1371 + WARN_ON_ONCE(1); 1372 + return -EINVAL; 1360 1373 } 1361 1374 EXPORT_SYMBOL(mmap_action_prepare); 1362 1375 1363 1376 /** 1364 1377 * mmap_action_complete - Execute VMA descriptor action. 1365 - * @action: The action to perform. 1366 1378 * @vma: The VMA to perform the action upon. 1379 + * @action: The action to perform. 1367 1380 * 1368 1381 * Similar to mmap_action_prepare(). 1369 1382 * 1370 1383 * Return: 0 on success, or error, at which point the VMA will be unmapped. 1371 1384 */ 1372 - int mmap_action_complete(struct mmap_action *action, 1373 - struct vm_area_struct *vma) 1385 + int mmap_action_complete(struct vm_area_struct *vma, 1386 + struct mmap_action *action) 1387 + 1374 1388 { 1375 1389 int err = 0; 1376 1390 ··· 1379 1391 case MMAP_NOTHING: 1380 1392 break; 1381 1393 case MMAP_REMAP_PFN: 1382 - err = remap_pfn_range_complete(vma, action->remap.start, 1383 - action->remap.start_pfn, action->remap.size, 1384 - action->remap.pgprot); 1394 + err = remap_pfn_range_complete(vma, action); 1385 1395 break; 1386 1396 case MMAP_IO_REMAP_PFN: 1387 - err = io_remap_pfn_range_complete(vma, action->remap.start, 1388 - action->remap.start_pfn, action->remap.size, 1389 - action->remap.pgprot); 1397 + /* Should have been delegated. */ 1398 + WARN_ON_ONCE(1); 1399 + err = -EINVAL; 1390 1400 break; 1391 1401 } 1392 1402 1393 - return mmap_action_finish(action, vma, err); 1403 + return mmap_action_finish(vma, action, err); 1394 1404 } 1395 1405 EXPORT_SYMBOL(mmap_action_complete); 1396 1406 #else 1397 - void mmap_action_prepare(struct mmap_action *action, 1398 - struct vm_area_desc *desc) 1407 + int mmap_action_prepare(struct vm_area_desc *desc) 1399 1408 { 1400 - switch (action->type) { 1409 + switch (desc->action.type) { 1401 1410 case MMAP_NOTHING: 1402 1411 break; 1403 1412 case MMAP_REMAP_PFN: ··· 1402 1417 WARN_ON_ONCE(1); /* nommu cannot handle these. */ 1403 1418 break; 1404 1419 } 1420 + 1421 + return 0; 1405 1422 } 1406 1423 EXPORT_SYMBOL(mmap_action_prepare); 1407 1424 1408 - int mmap_action_complete(struct mmap_action *action, 1409 - struct vm_area_struct *vma) 1425 + int mmap_action_complete(struct vm_area_struct *vma, 1426 + struct mmap_action *action) 1410 1427 { 1411 1428 int err = 0; 1412 1429 ··· 1423 1436 break; 1424 1437 } 1425 1438 1426 - return mmap_action_finish(action, vma, err); 1439 + return mmap_action_finish(vma, action, err); 1427 1440 } 1428 1441 EXPORT_SYMBOL(mmap_action_complete); 1429 1442 #endif
+14 -10
mm/vma.c
··· 2640 2640 vma_set_page_prot(vma); 2641 2641 } 2642 2642 2643 - static void call_action_prepare(struct mmap_state *map, 2644 - struct vm_area_desc *desc) 2643 + static int call_action_prepare(struct mmap_state *map, 2644 + struct vm_area_desc *desc) 2645 2645 { 2646 - struct mmap_action *action = &desc->action; 2646 + int err; 2647 2647 2648 - mmap_action_prepare(action, desc); 2648 + err = mmap_action_prepare(desc); 2649 + if (err) 2650 + return err; 2649 2651 2650 - if (action->hide_from_rmap_until_complete) 2652 + if (desc->action.hide_from_rmap_until_complete) 2651 2653 map->hold_file_rmap_lock = true; 2654 + return 0; 2652 2655 } 2653 2656 2654 2657 /* ··· 2675 2672 if (err) 2676 2673 return err; 2677 2674 2678 - call_action_prepare(map, desc); 2675 + err = call_action_prepare(map, desc); 2676 + if (err) 2677 + return err; 2679 2678 2680 2679 /* Update fields permitted to be changed. */ 2681 2680 map->pgoff = desc->pgoff; ··· 2732 2727 } 2733 2728 2734 2729 static int call_action_complete(struct mmap_state *map, 2735 - struct vm_area_desc *desc, 2730 + struct mmap_action *action, 2736 2731 struct vm_area_struct *vma) 2737 2732 { 2738 - struct mmap_action *action = &desc->action; 2739 2733 int ret; 2740 2734 2741 - ret = mmap_action_complete(action, vma); 2735 + ret = mmap_action_complete(vma, action); 2742 2736 2743 2737 /* If we held the file rmap we need to release it. */ 2744 2738 if (map->hold_file_rmap_lock) { ··· 2799 2795 __mmap_complete(&map, vma); 2800 2796 2801 2797 if (have_mmap_prepare && allocated_new) { 2802 - error = call_action_complete(&map, &desc, vma); 2798 + error = call_action_complete(&map, &desc.action, vma); 2803 2799 2804 2800 if (error) 2805 2801 return error;
+5 -2
tools/testing/vma/include/dup.h
··· 1277 1277 if (err) 1278 1278 return err; 1279 1279 1280 - mmap_action_prepare(&desc.action, &desc); 1280 + err = mmap_action_prepare(&desc); 1281 + if (err) 1282 + return err; 1283 + 1281 1284 set_vma_from_desc(vma, &desc); 1282 - return mmap_action_complete(&desc.action, vma); 1285 + return mmap_action_complete(vma, &desc.action); 1283 1286 } 1284 1287 1285 1288 static inline int compat_vma_mmap(struct file *file,
+4 -4
tools/testing/vma/include/stubs.h
··· 81 81 { 82 82 } 83 83 84 - static inline void mmap_action_prepare(struct mmap_action *action, 85 - struct vm_area_desc *desc) 84 + static inline int mmap_action_prepare(struct vm_area_desc *desc) 86 85 { 86 + return 0; 87 87 } 88 88 89 - static inline int mmap_action_complete(struct mmap_action *action, 90 - struct vm_area_struct *vma) 89 + static inline int mmap_action_complete(struct vm_area_struct *vma, 90 + struct mmap_action *action) 91 91 { 92 92 return 0; 93 93 }