tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

filemap: optimize folio refount update in filemap_map_pages

There are two meaningless folio refcount update for order0 folio in
filemap_map_pages(). First, filemap_map_order0_folio() adds folio
refcount after the folio is mapped to pte. And then, filemap_map_pages()
drops a refcount grabbed by next_uptodate_folio(). We could remain the
refcount unchanged in this case.

As Matthew metenioned in [1], it is safe to call folio_unlock() before
calling folio_put() here, because the folio is in page cache with refcount
held, and truncation will wait for the unlock.

Optimize filemap_map_folio_range() with the same method too.

With this patch, we can get 8% performance gain for lmbench testcase
'lat_pagefault -P 1 file' in order0 folio case, the size of file is 512M.


Link: https://lkml.kernel.org/r/20250904132737.1250368-1-tujinjiang@huawei.com
Link: https://lore.kernel.org/all/aKcU-fzxeW3xT5Wv@casper.infradead.org/ [1]
Signed-off-by: Jinjiang Tu <tujinjiang@huawei.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Jinjiang Tu and committed by
Andrew Morton 0faa77af 24a3c7af

+14 -6
+14 -6
mm/filemap.c
··· 3665 3665 unsigned long addr, unsigned int nr_pages, 3666 3666 unsigned long *rss, unsigned short *mmap_miss) 3667 3667 { 3668 + unsigned int ref_from_caller = 1; 3668 3669 vm_fault_t ret = 0; 3669 3670 struct page *page = folio_page(folio, start); 3670 3671 unsigned int count = 0; ··· 3699 3698 if (count) { 3700 3699 set_pte_range(vmf, folio, page, count, addr); 3701 3700 *rss += count; 3702 - folio_ref_add(folio, count); 3701 + folio_ref_add(folio, count - ref_from_caller); 3702 + ref_from_caller = 0; 3703 3703 if (in_range(vmf->address, addr, count * PAGE_SIZE)) 3704 3704 ret = VM_FAULT_NOPAGE; 3705 3705 } ··· 3715 3713 if (count) { 3716 3714 set_pte_range(vmf, folio, page, count, addr); 3717 3715 *rss += count; 3718 - folio_ref_add(folio, count); 3716 + folio_ref_add(folio, count - ref_from_caller); 3717 + ref_from_caller = 0; 3719 3718 if (in_range(vmf->address, addr, count * PAGE_SIZE)) 3720 3719 ret = VM_FAULT_NOPAGE; 3721 3720 } 3722 3721 3723 3722 vmf->pte = old_ptep; 3723 + if (ref_from_caller) 3724 + /* Locked folios cannot get truncated. */ 3725 + folio_ref_dec(folio); 3724 3726 3725 3727 return ret; 3726 3728 } ··· 3737 3731 struct page *page = &folio->page; 3738 3732 3739 3733 if (PageHWPoison(page)) 3740 - return ret; 3734 + goto out; 3741 3735 3742 3736 /* See comment of filemap_map_folio_range() */ 3743 3737 if (!folio_test_workingset(folio)) ··· 3749 3743 * the fault-around logic. 3750 3744 */ 3751 3745 if (!pte_none(ptep_get(vmf->pte))) 3752 - return ret; 3746 + goto out; 3753 3747 3754 3748 if (vmf->address == addr) 3755 3749 ret = VM_FAULT_NOPAGE; 3756 3750 3757 3751 set_pte_range(vmf, folio, page, 1, addr); 3758 3752 (*rss)++; 3759 - folio_ref_inc(folio); 3753 + return ret; 3760 3754 3755 + out: 3756 + /* Locked folios cannot get truncated. */ 3757 + folio_ref_dec(folio); 3761 3758 return ret; 3762 3759 } 3763 3760 ··· 3820 3811 nr_pages, &rss, &mmap_miss); 3821 3812 3822 3813 folio_unlock(folio); 3823 - folio_put(folio); 3824 3814 } while ((folio = next_uptodate_folio(&xas, mapping, end_pgoff)) != NULL); 3825 3815 add_mm_counter(vma->vm_mm, folio_type, rss); 3826 3816 pte_unmap_unlock(vmf->pte, vmf->ptl);