Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm/huge_memory: fix folio isn't locked in softleaf_to_folio()

On arm64 server, we found folio that get from migration entry isn't locked
in softleaf_to_folio(). This issue triggers when mTHP splitting and
zap_nonpresent_ptes() races, and the root cause is lack of memory barrier
in softleaf_to_folio(). The race is as follows:

CPU0 CPU1

deferred_split_scan() zap_nonpresent_ptes()
lock folio
split_folio()
unmap_folio()
change ptes to migration entries
__split_folio_to_order() softleaf_to_folio()
set flags(including PG_locked) for tail pages folio = pfn_folio(softleaf_to_pfn(entry))
smp_wmb() VM_WARN_ON_ONCE(!folio_test_locked(folio))
prep_compound_page() for tail pages

In __split_folio_to_order(), smp_wmb() guarantees page flags of tail pages
are visible before the tail page becomes non-compound. smp_wmb() should
be paired with smp_rmb() in softleaf_to_folio(), which is missed. As a
result, if zap_nonpresent_ptes() accesses migration entry that stores tail
pfn, softleaf_to_folio() may see the updated compound_head of tail page
before page->flags.

This issue will trigger VM_WARN_ON_ONCE() in pfn_swap_entry_folio()
because of the race between folio split and zap_nonpresent_ptes()
leading to a folio incorrectly undergoing modification without a folio
lock being held.

This is a BUG_ON() before commit 93976a20345b ("mm: eliminate further
swapops predicates"), which in merged in v6.19-rc1.

To fix it, add missing smp_rmb() if the softleaf entry is migration entry
in softleaf_to_folio() and softleaf_to_page().

[tujinjiang@huawei.com: update function name and comments]
Link: https://lkml.kernel.org/r/20260321075214.3305564-1-tujinjiang@huawei.com
Link: https://lkml.kernel.org/r/20260319012541.4158561-1-tujinjiang@huawei.com
Fixes: e9b61f19858a ("thp: reintroduce split_huge_page()")
Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Cc: Barry Song <baohua@kernel.org>
Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nanyong Sun <sunnanyong@huawei.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Jinjiang Tu and committed by
Andrew Morton
4c5e7f0f 24f9515d

+21 -11
+21 -11
include/linux/leafops.h
··· 363 363 return swp_offset(entry) & SWP_PFN_MASK; 364 364 } 365 365 366 + static inline void softleaf_migration_sync(softleaf_t entry, 367 + struct folio *folio) 368 + { 369 + /* 370 + * Ensure we do not race with split, which might alter tail pages into new 371 + * folios and thus result in observing an unlocked folio. 372 + * This matches the write barrier in __split_folio_to_order(). 373 + */ 374 + smp_rmb(); 375 + 376 + /* 377 + * Any use of migration entries may only occur while the 378 + * corresponding page is locked 379 + */ 380 + VM_WARN_ON_ONCE(!folio_test_locked(folio)); 381 + } 382 + 366 383 /** 367 384 * softleaf_to_page() - Obtains struct page for PFN encoded within leaf entry. 368 385 * @entry: Leaf entry, softleaf_has_pfn(@entry) must return true. ··· 391 374 struct page *page = pfn_to_page(softleaf_to_pfn(entry)); 392 375 393 376 VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); 394 - /* 395 - * Any use of migration entries may only occur while the 396 - * corresponding page is locked 397 - */ 398 - VM_WARN_ON_ONCE(softleaf_is_migration(entry) && !PageLocked(page)); 377 + if (softleaf_is_migration(entry)) 378 + softleaf_migration_sync(entry, page_folio(page)); 399 379 400 380 return page; 401 381 } ··· 408 394 struct folio *folio = pfn_folio(softleaf_to_pfn(entry)); 409 395 410 396 VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); 411 - /* 412 - * Any use of migration entries may only occur while the 413 - * corresponding folio is locked. 414 - */ 415 - VM_WARN_ON_ONCE(softleaf_is_migration(entry) && 416 - !folio_test_locked(folio)); 397 + if (softleaf_is_migration(entry)) 398 + softleaf_migration_sync(entry, folio); 417 399 418 400 return folio; 419 401 }