tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm, swap: wrap swap cache replacement with a helper

There are currently three swap cache users that are trying to replace an
existing folio with a new one: huge memory splitting, migration, and shmem
replacement. What they are doing is quite similar.

Introduce a common helper for this. In later commits, this can be easily
switched to use the swap table by updating this helper.

The newly added helper also makes the swap cache API better defined, and
make debugging easier by adding a few more debug checks.

Migration and shmem replace are meant to clone the folio, including
content, swap entry value, and flags. And splitting will adjust each sub
folio's swap entry according to order, which could be non-uniform in the
future. So document it clearly that it's the caller's responsibility to
set up the new folio's swap entries and flags before calling the helper.
The helper will just follow the new folio's entry value.

This also prepares for replacing high-order folios in the swap cache.
Currently, only splitting to order 0 is allowed for swap cache folios.
Using the new helper, we can handle high-order folio splitting better.

Link: https://lkml.kernel.org/r/20250916160100.31545-11-ryncsn@gmail.com
Signed-off-by: Kairui Song <kasong@tencent.com>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: David Hildenbrand <david@redhat.com>
Acked-by: Chris Li <chrisl@kernel.org>
Suggested-by: Chris Li <chrisl@kernel.org>
Cc: Baoquan He <bhe@redhat.com>
Cc: Barry Song <baohua@kernel.org>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: kernel test robot <oliver.sang@intel.com>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Yosry Ahmed <yosryahmed@google.com>
Cc: Zi Yan <ziy@nvidia.com>
Cc: SeongJae Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Kairui Song and committed by
Andrew Morton 094dc8b0 84a7a982

+44 -20
+1 -3
mm/huge_memory.c
··· 3798 3798 * NOTE: shmem in swap cache is not supported yet. 3799 3799 */ 3800 3800 if (swap_cache) { 3801 - __xa_store(&swap_cache->i_pages, 3802 - swap_cache_index(new_folio->swap), 3803 - new_folio, 0); 3801 + __swap_cache_replace_folio(folio, new_folio); 3804 3802 continue; 3805 3803 } 3806 3804
+3 -8
mm/migrate.c
··· 566 566 struct zone *oldzone, *newzone; 567 567 int dirty; 568 568 long nr = folio_nr_pages(folio); 569 - long entries, i; 570 569 571 570 if (!mapping) { 572 571 /* Take off deferred split queue while frozen and memcg set */ ··· 614 615 if (folio_test_swapcache(folio)) { 615 616 folio_set_swapcache(newfolio); 616 617 newfolio->private = folio_get_private(folio); 617 - entries = nr; 618 - } else { 619 - entries = 1; 620 618 } 621 619 622 620 /* Move dirty while folio refs frozen and newfolio not yet exposed */ ··· 623 627 folio_set_dirty(newfolio); 624 628 } 625 629 626 - /* Swap cache still stores N entries instead of a high-order entry */ 627 - for (i = 0; i < entries; i++) { 630 + if (folio_test_swapcache(folio)) 631 + __swap_cache_replace_folio(folio, newfolio); 632 + else 628 633 xas_store(&xas, newfolio); 629 - xas_next(&xas); 630 - } 631 634 632 635 /* 633 636 * Drop cache reference from old folio by unfreezing
+2 -9
mm/shmem.c
··· 2086 2086 struct folio *new, *old = *foliop; 2087 2087 swp_entry_t entry = old->swap; 2088 2088 struct address_space *swap_mapping = swap_address_space(entry); 2089 - pgoff_t swap_index = swap_cache_index(entry); 2090 - XA_STATE(xas, &swap_mapping->i_pages, swap_index); 2091 2089 int nr_pages = folio_nr_pages(old); 2092 - int error = 0, i; 2090 + int error = 0; 2093 2091 2094 2092 /* 2095 2093 * We have arrived here because our zones are constrained, so don't ··· 2116 2118 new->swap = entry; 2117 2119 folio_set_swapcache(new); 2118 2120 2119 - /* Swap cache still stores N entries instead of a high-order entry */ 2120 2121 xa_lock_irq(&swap_mapping->i_pages); 2121 - for (i = 0; i < nr_pages; i++) { 2122 - WARN_ON_ONCE(xas_store(&xas, new) != old); 2123 - xas_next(&xas); 2124 - } 2125 - 2122 + __swap_cache_replace_folio(old, new); 2126 2123 mem_cgroup_replace_folio(old, new); 2127 2124 shmem_update_stats(new, nr_pages); 2128 2125 shmem_update_stats(old, -nr_pages);
+5
mm/swap.h
··· 185 185 void swap_cache_del_folio(struct folio *folio); 186 186 void __swap_cache_del_folio(struct folio *folio, 187 187 swp_entry_t entry, void *shadow); 188 + void __swap_cache_replace_folio(struct folio *old, struct folio *new); 188 189 void swap_cache_clear_shadow(int type, unsigned long begin, 189 190 unsigned long end); 190 191 ··· 334 333 } 335 334 336 335 static inline void __swap_cache_del_folio(struct folio *folio, swp_entry_t entry, void *shadow) 336 + { 337 + } 338 + 339 + static inline void __swap_cache_replace_folio(struct folio *old, struct folio *new) 337 340 { 338 341 } 339 342
+33
mm/swap_state.c
··· 235 235 } 236 236 237 237 /** 238 + * __swap_cache_replace_folio - Replace a folio in the swap cache. 239 + * @old: The old folio to be replaced. 240 + * @new: The new folio. 241 + * 242 + * Replace an existing folio in the swap cache with a new folio. The 243 + * caller is responsible for setting up the new folio's flag and swap 244 + * entries. Replacement will take the new folio's swap entry value as 245 + * the starting offset to override all slots covered by the new folio. 246 + * 247 + * Context: Caller must ensure both folios are locked, also lock the 248 + * swap address_space that holds the old folio to avoid races. 249 + */ 250 + void __swap_cache_replace_folio(struct folio *old, struct folio *new) 251 + { 252 + swp_entry_t entry = new->swap; 253 + unsigned long nr_pages = folio_nr_pages(new); 254 + unsigned long offset = swap_cache_index(entry); 255 + unsigned long end = offset + nr_pages; 256 + 257 + XA_STATE(xas, &swap_address_space(entry)->i_pages, offset); 258 + 259 + VM_WARN_ON_ONCE(!folio_test_swapcache(old) || !folio_test_swapcache(new)); 260 + VM_WARN_ON_ONCE(!folio_test_locked(old) || !folio_test_locked(new)); 261 + VM_WARN_ON_ONCE(!entry.val); 262 + 263 + /* Swap cache still stores N entries instead of a high-order entry */ 264 + do { 265 + WARN_ON_ONCE(xas_store(&xas, new) != old); 266 + xas_next(&xas); 267 + } while (++offset < end); 268 + } 269 + 270 + /** 238 271 * swap_cache_clear_shadow - Clears a set of shadows in the swap cache. 239 272 * @type: Indicates the swap device. 240 273 * @begin: Beginning offset of the range.