Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm: rmap: support batched unmapping for file large folios

Similar to folio_referenced_one(), we can apply batched unmapping for file
large folios to optimize the performance of file folios reclamation.

Barry previously implemented batched unmapping for lazyfree anonymous
large folios[1] and did not further optimize anonymous large folios or
file-backed large folios at that stage. As for file-backed large folios,
the batched unmapping support is relatively straightforward, as we only
need to clear the consecutive (present) PTE entries for file-backed large
folios.

Note that it's not ready to support batched unmapping for uffd case, so
let's still fallback to per-page unmapping for the uffd case.

Performance testing:
Allocate 10G clean file-backed folios by mmap() in a memory cgroup, and
try to reclaim 8G file-backed folios via the memory.reclaim interface. I
can observe 75% performance improvement on my Arm64 32-core server (and
50%+ improvement on my X86 machine) with this patch.

W/o patch:
real 0m1.018s
user 0m0.000s
sys 0m1.018s

W/ patch:
real 0m0.249s
user 0m0.000s
sys 0m0.249s

[1] https://lore.kernel.org/all/20250214093015.51024-4-21cnbao@gmail.com/T/#u
Link: https://lkml.kernel.org/r/b53a16f67c93a3fe65e78092069ad135edf00eff.1770645603.git.baolin.wang@linux.alibaba.com
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: Ryan Roberts <ryan.roberts@arm.com>
Acked-by: Barry Song <baohua@kernel.org>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Jann Horn <jannh@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Baolin Wang and committed by
Andrew Morton
a67fe41e 07f440c2

+7 -3
+7 -3
mm/rmap.c
··· 1945 1945 end_addr = pmd_addr_end(addr, vma->vm_end); 1946 1946 max_nr = (end_addr - addr) >> PAGE_SHIFT; 1947 1947 1948 - /* We only support lazyfree batching for now ... */ 1949 - if (!folio_test_anon(folio) || folio_test_swapbacked(folio)) 1948 + /* We only support lazyfree or file folios batching for now ... */ 1949 + if (folio_test_anon(folio) && folio_test_swapbacked(folio)) 1950 1950 return 1; 1951 + 1951 1952 if (pte_unused(pte)) 1953 + return 1; 1954 + 1955 + if (userfaultfd_wp(vma)) 1952 1956 return 1; 1953 1957 1954 1958 return folio_pte_batch(folio, pvmw->pte, pte, max_nr); ··· 2317 2313 * 2318 2314 * See Documentation/mm/mmu_notifier.rst 2319 2315 */ 2320 - dec_mm_counter(mm, mm_counter_file(folio)); 2316 + add_mm_counter(mm, mm_counter_file(folio), -nr_pages); 2321 2317 } 2322 2318 discard: 2323 2319 if (unlikely(folio_test_hugetlb(folio))) {