tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm: remove unnecessary calls to lru_add_drain

There seem to be several categories of calls to lru_add_drain and
lru_add_drain_all.

The first are code paths that recently allocated, swapped in, or otherwise
processed a batch of pages, and want them all on the LRU. These drain
pages that were recently allocated, probably on the local CPU.

A second category are code paths that are actively trying to reclaim,
migrate, or offline memory. These often use lru_add_drain_all, to drain
the caches on all CPUs.

However, there also seem to be some other callers where we aren't really
doing either. They are calling lru_add_drain(), despite operating on
pages that may have been allocated long ago, and quite possibly on
different CPUs.

Those calls are not likely to be effective at anything but creating lock
contention on the LRU locks.

Remove the lru_add_drain calls in the latter category.

For detailed reasoning, see [1] and [2].

Link: https://lkml.kernel.org/r/dca2824e8e88e826c6b260a831d79089b5b9c79d.camel@surriel.com [1]
Link: https://lkml.kernel.org/r/xxfhcjaq2xxcl5adastz5omkytenq7izo2e5f4q7e3ns4z6lko@odigjjc7hqrg [2]
Link: https://lkml.kernel.org/r/20241219153253.3da9e8aa@fangorn
Signed-off-by: Rik van Riel <riel@surriel.com>
Suggested-by: David Hildenbrand <david@redhat.com>
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
Acked-by: David Hildenbrand <david@redhat.com>
Acked-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Chris Li <chrisl@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Rik van Riel and committed by
Andrew Morton 1aa43598 44d46b76

-6
-1
mm/memory.c
··· 2004 2004 struct mmu_notifier_range range; 2005 2005 struct mmu_gather tlb; 2006 2006 2007 - lru_add_drain(); 2008 2007 mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma->vm_mm, 2009 2008 address, end); 2010 2009 hugetlb_zap_begin(vma, &range.start, &range.end);
-2
mm/mmap.c
··· 1336 1336 goto destroy; 1337 1337 } 1338 1338 1339 - lru_add_drain(); 1340 1339 flush_cache_mm(mm); 1341 1340 tlb_gather_mmu_fullmm(&tlb, mm); 1342 1341 /* update_hiwater_rss(mm) here? but nobody should be looking */ ··· 1778 1779 vma, new_start, length, false, true)) 1779 1780 return -ENOMEM; 1780 1781 1781 - lru_add_drain(); 1782 1782 tlb_gather_mmu(&tlb, mm); 1783 1783 next = vma_next(&vmi); 1784 1784 if (new_end > old_start) {
-1
mm/swap_state.c
··· 317 317 struct folio_batch folios; 318 318 unsigned int refs[PAGEVEC_SIZE]; 319 319 320 - lru_add_drain(); 321 320 folio_batch_init(&folios); 322 321 for (int i = 0; i < nr; i++) { 323 322 struct folio *folio = page_folio(encoded_page_ptr(pages[i]));
-2
mm/vma.c
··· 430 430 struct mm_struct *mm = vma->vm_mm; 431 431 struct mmu_gather tlb; 432 432 433 - lru_add_drain(); 434 433 tlb_gather_mmu(&tlb, mm); 435 434 update_hiwater_rss(mm); 436 435 unmap_vmas(&tlb, mas, vma, vma->vm_start, vma->vm_end, vma->vm_end, ··· 1131 1132 * were isolated before we downgraded mmap_lock. 1132 1133 */ 1133 1134 mas_set(mas_detach, 1); 1134 - lru_add_drain(); 1135 1135 tlb_gather_mmu(&tlb, vms->vma->vm_mm); 1136 1136 update_hiwater_rss(vms->vma->vm_mm); 1137 1137 unmap_vmas(&tlb, mas_detach, vms->vma, vms->start, vms->end,