tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm, swap: always lock and check the swap cache folio before use

Swap cache lookup only increases the reference count of the returned
folio. That's not enough to ensure a folio is stable in the swap cache,
so the folio could be removed from the swap cache at any time. The caller
should always lock and check the folio before using it.

We have just documented this in kerneldoc, now introduce a helper for swap
cache folio verification with proper sanity checks.

Also, sanitize a few current users to use this convention and the new
helper for easier debugging. They were not having observable problems
yet, only trivial issues like wasted CPU cycles on swapoff or reclaiming.
They would fail in some other way, but it is still better to always follow
this convention to make things robust and make later commits easier to do.

Link: https://lkml.kernel.org/r/20250916160100.31545-6-ryncsn@gmail.com
Signed-off-by: Kairui Song <kasong@tencent.com>
Acked-by: David Hildenbrand <david@redhat.com>
Acked-by: Chris Li <chrisl@kernel.org>
Acked-by: Nhat Pham <nphamcs@gmail.com>
Suggested-by: Chris Li <chrisl@kernel.org>
Reviewed-by: Barry Song <baohua@kernel.org>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: kernel test robot <oliver.sang@intel.com>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Yosry Ahmed <yosryahmed@google.com>
Cc: Zi Yan <ziy@nvidia.com>
Cc: SeongJae Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Kairui Song and committed by
Andrew Morton ae38eb21 3518b931

+41 -6
+1 -2
mm/memory.c
··· 4748 4748 * swapcache, we need to check that the page's swap has not 4749 4749 * changed. 4750 4750 */ 4751 - if (unlikely(!folio_test_swapcache(folio) || 4752 - page_swap_entry(page).val != entry.val)) 4751 + if (unlikely(!folio_matches_swap_entry(folio, entry))) 4753 4752 goto out_page; 4754 4753 4755 4754 if (unlikely(PageHWPoison(page))) {
+27
mm/swap.h
··· 52 52 return swp_offset(entry) & SWAP_ADDRESS_SPACE_MASK; 53 53 } 54 54 55 + /** 56 + * folio_matches_swap_entry - Check if a folio matches a given swap entry. 57 + * @folio: The folio. 58 + * @entry: The swap entry to check against. 59 + * 60 + * Context: The caller should have the folio locked to ensure it's stable 61 + * and nothing will move it in or out of the swap cache. 62 + * Return: true or false. 63 + */ 64 + static inline bool folio_matches_swap_entry(const struct folio *folio, 65 + swp_entry_t entry) 66 + { 67 + swp_entry_t folio_entry = folio->swap; 68 + long nr_pages = folio_nr_pages(folio); 69 + 70 + VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio); 71 + if (!folio_test_swapcache(folio)) 72 + return false; 73 + VM_WARN_ON_ONCE_FOLIO(!IS_ALIGNED(folio_entry.val, nr_pages), folio); 74 + return folio_entry.val == round_down(entry.val, nr_pages); 75 + } 76 + 55 77 void show_swap_cache_info(void); 56 78 void *get_shadow_from_swap_cache(swp_entry_t entry); 57 79 int add_to_swap_cache(struct folio *folio, swp_entry_t entry, ··· 164 142 static inline pgoff_t swap_cache_index(swp_entry_t entry) 165 143 { 166 144 return 0; 145 + } 146 + 147 + static inline bool folio_matches_swap_entry(const struct folio *folio, swp_entry_t entry) 148 + { 149 + return false; 167 150 } 168 151 169 152 static inline void show_swap_cache_info(void)
+5 -2
mm/swap_state.c
··· 79 79 * with reference count or locks. 80 80 * Return: Returns the found folio on success, NULL otherwise. The caller 81 81 * must lock and check if the folio still matches the swap entry before 82 - * use. 82 + * use (e.g. with folio_matches_swap_entry). 83 83 */ 84 84 struct folio *swap_cache_get_folio(swp_entry_t entry) 85 85 { ··· 346 346 for (;;) { 347 347 int err; 348 348 349 - /* Check the swap cache in case the folio is already there */ 349 + /* 350 + * Check the swap cache first, if a cached folio is found, 351 + * return it unlocked. The caller will lock and check it. 352 + */ 350 353 folio = swap_cache_get_folio(entry); 351 354 if (folio) 352 355 goto got_folio;
+8 -2
mm/swapfile.c
··· 240 240 * Offset could point to the middle of a large folio, or folio 241 241 * may no longer point to the expected offset before it's locked. 242 242 */ 243 - if (offset < swp_offset(folio->swap) || 244 - offset >= swp_offset(folio->swap) + nr_pages) { 243 + if (!folio_matches_swap_entry(folio, entry)) { 245 244 folio_unlock(folio); 246 245 folio_put(folio); 247 246 goto again; ··· 2002 2003 pte_t *pte, new_pte, old_pte; 2003 2004 bool hwpoisoned = false; 2004 2005 int ret = 1; 2006 + 2007 + /* 2008 + * If the folio is removed from swap cache by others, continue to 2009 + * unuse other PTEs. try_to_unuse may try again if we missed this one. 2010 + */ 2011 + if (!folio_matches_swap_entry(folio, entry)) 2012 + return 0; 2005 2013 2006 2014 swapcache = folio; 2007 2015 folio = ksm_might_need_to_copy(folio, vma, addr);