Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

mm/huge_memory: refactor copy_huge_pmd() non-present logic

Right now we are inconsistent in our use of thp_migration_supported():

static inline bool thp_migration_supported(void)
{
return IS_ENABLED(CONFIG_ARCH_ENABLE_THP_MIGRATION);
}

And simply having arbitrary and ugly #ifdef
CONFIG_ARCH_ENABLE_THP_MIGRATION blocks in code.

This is exhibited in copy_huge_pmd(), which inserts a large #ifdef
CONFIG_ARCH_ENABLE_THP_MIGRATION block and an if-branch which is difficult
to follow

It's difficult to follow the logic of such a large function and the
non-present PMD logic is clearly separate as it sits in a giant if-branch.

Therefore this patch both separates out the logic and utilises
thp_migration_supported().

No functional change intended.

Link: https://lkml.kernel.org/r/6eaadc23ed512d370ede65561e34e96241c54b9d.1762812360.git.lorenzo.stoakes@oracle.com
Signed-off-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Reviewed-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Chengming Zhou <chengming.zhou@linux.dev>
Cc: Chris Li <chrisl@kernel.org>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Claudio Imbrenda <imbrenda@linux.ibm.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Gregory Price <gourry@gourry.net>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jann Horn <jannh@google.com>
Cc: Janosch Frank <frankja@linux.ibm.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Mathew Brost <matthew.brost@intel.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Miaohe Lin <linmiaohe@huawei.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Naoya Horiguchi <nao.horiguchi@gmail.com>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: SeongJae Park <sj@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Wei Xu <weixugc@google.com>
Cc: xu xin <xu.xin16@zte.com.cn>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Lorenzo Stoakes and committed by
Andrew Morton
e244d82d aa62204c

+59 -50
+59 -50
mm/huge_memory.c
··· 1699 1699 update_mmu_cache_pmd(vma, addr, pmd); 1700 1700 } 1701 1701 1702 + static void copy_huge_non_present_pmd( 1703 + struct mm_struct *dst_mm, struct mm_struct *src_mm, 1704 + pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, 1705 + struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, 1706 + pmd_t pmd, pgtable_t pgtable) 1707 + { 1708 + swp_entry_t entry = pmd_to_swp_entry(pmd); 1709 + struct folio *src_folio; 1710 + 1711 + VM_WARN_ON(!is_pmd_non_present_folio_entry(pmd)); 1712 + 1713 + if (is_writable_migration_entry(entry) || 1714 + is_readable_exclusive_migration_entry(entry)) { 1715 + entry = make_readable_migration_entry(swp_offset(entry)); 1716 + pmd = swp_entry_to_pmd(entry); 1717 + if (pmd_swp_soft_dirty(*src_pmd)) 1718 + pmd = pmd_swp_mksoft_dirty(pmd); 1719 + if (pmd_swp_uffd_wp(*src_pmd)) 1720 + pmd = pmd_swp_mkuffd_wp(pmd); 1721 + set_pmd_at(src_mm, addr, src_pmd, pmd); 1722 + } else if (is_device_private_entry(entry)) { 1723 + /* 1724 + * For device private entries, since there are no 1725 + * read exclusive entries, writable = !readable 1726 + */ 1727 + if (is_writable_device_private_entry(entry)) { 1728 + entry = make_readable_device_private_entry(swp_offset(entry)); 1729 + pmd = swp_entry_to_pmd(entry); 1730 + 1731 + if (pmd_swp_soft_dirty(*src_pmd)) 1732 + pmd = pmd_swp_mksoft_dirty(pmd); 1733 + if (pmd_swp_uffd_wp(*src_pmd)) 1734 + pmd = pmd_swp_mkuffd_wp(pmd); 1735 + set_pmd_at(src_mm, addr, src_pmd, pmd); 1736 + } 1737 + 1738 + src_folio = pfn_swap_entry_folio(entry); 1739 + VM_WARN_ON(!folio_test_large(src_folio)); 1740 + 1741 + folio_get(src_folio); 1742 + /* 1743 + * folio_try_dup_anon_rmap_pmd does not fail for 1744 + * device private entries. 1745 + */ 1746 + folio_try_dup_anon_rmap_pmd(src_folio, &src_folio->page, 1747 + dst_vma, src_vma); 1748 + } 1749 + 1750 + add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); 1751 + mm_inc_nr_ptes(dst_mm); 1752 + pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); 1753 + if (!userfaultfd_wp(dst_vma)) 1754 + pmd = pmd_swp_clear_uffd_wp(pmd); 1755 + set_pmd_at(dst_mm, addr, dst_pmd, pmd); 1756 + } 1757 + 1702 1758 int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, 1703 1759 pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, 1704 1760 struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma) ··· 1800 1744 ret = -EAGAIN; 1801 1745 pmd = *src_pmd; 1802 1746 1803 - #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION 1804 - if (unlikely(is_swap_pmd(pmd))) { 1805 - swp_entry_t entry = pmd_to_swp_entry(pmd); 1806 - 1807 - VM_WARN_ON(!is_pmd_non_present_folio_entry(pmd)); 1808 - 1809 - if (is_writable_migration_entry(entry) || 1810 - is_readable_exclusive_migration_entry(entry)) { 1811 - entry = make_readable_migration_entry(swp_offset(entry)); 1812 - pmd = swp_entry_to_pmd(entry); 1813 - if (pmd_swp_soft_dirty(*src_pmd)) 1814 - pmd = pmd_swp_mksoft_dirty(pmd); 1815 - if (pmd_swp_uffd_wp(*src_pmd)) 1816 - pmd = pmd_swp_mkuffd_wp(pmd); 1817 - set_pmd_at(src_mm, addr, src_pmd, pmd); 1818 - } else if (is_device_private_entry(entry)) { 1819 - /* 1820 - * For device private entries, since there are no 1821 - * read exclusive entries, writable = !readable 1822 - */ 1823 - if (is_writable_device_private_entry(entry)) { 1824 - entry = make_readable_device_private_entry(swp_offset(entry)); 1825 - pmd = swp_entry_to_pmd(entry); 1826 - 1827 - if (pmd_swp_soft_dirty(*src_pmd)) 1828 - pmd = pmd_swp_mksoft_dirty(pmd); 1829 - if (pmd_swp_uffd_wp(*src_pmd)) 1830 - pmd = pmd_swp_mkuffd_wp(pmd); 1831 - set_pmd_at(src_mm, addr, src_pmd, pmd); 1832 - } 1833 - 1834 - src_folio = pfn_swap_entry_folio(entry); 1835 - VM_WARN_ON(!folio_test_large(src_folio)); 1836 - 1837 - folio_get(src_folio); 1838 - /* 1839 - * folio_try_dup_anon_rmap_pmd does not fail for 1840 - * device private entries. 1841 - */ 1842 - folio_try_dup_anon_rmap_pmd(src_folio, &src_folio->page, 1843 - dst_vma, src_vma); 1844 - } 1845 - 1846 - add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); 1847 - mm_inc_nr_ptes(dst_mm); 1848 - pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); 1849 - if (!userfaultfd_wp(dst_vma)) 1850 - pmd = pmd_swp_clear_uffd_wp(pmd); 1851 - set_pmd_at(dst_mm, addr, dst_pmd, pmd); 1747 + if (unlikely(thp_migration_supported() && is_swap_pmd(pmd))) { 1748 + copy_huge_non_present_pmd(dst_mm, src_mm, dst_pmd, src_pmd, addr, 1749 + dst_vma, src_vma, pmd, pgtable); 1852 1750 ret = 0; 1853 1751 goto out_unlock; 1854 1752 } 1855 - #endif 1856 1753 1857 1754 if (unlikely(!pmd_trans_huge(pmd))) { 1858 1755 pte_free(dst_mm, pgtable);