tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

powerpc/code-patching: Speed up page mapping/unmapping

Since commit 591b4b268435 ("powerpc/code-patching: Pre-map patch area")
the patch area is premapped so intermediate page tables are already
allocated.

Use __set_pte_at() directly instead of the heavy map_kernel_page(),
at for unmapping just do a pte_clear() followed by a flush.

__set_pte_at() can be used directly without the filters in
set_pte_at() because we are mapping a normal page non executable.

Make sure gcc knows text_poke_area is page aligned in order to
optimise the flush.

This change reduces by 66% the time needed to activate ftrace on
an 8xx (588000 tb ticks instead of 1744000).

Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
[mpe: Add ptesync needed on radix to avoid spurious fault]
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/20220815114840.1468656-1-mpe@ellerman.id.au

authored by

Christophe Leroy and committed by
Michael Ellerman 8b4bb0ad fd20b60a

+19 -11
+19 -11
arch/powerpc/lib/code-patching.c
··· 94 94 static_branch_enable(&poking_init_done); 95 95 } 96 96 97 + static unsigned long get_patch_pfn(void *addr) 98 + { 99 + if (IS_ENABLED(CONFIG_MODULES) && is_vmalloc_or_module_addr(addr)) 100 + return vmalloc_to_pfn(addr); 101 + else 102 + return __pa_symbol(addr) >> PAGE_SHIFT; 103 + } 104 + 97 105 /* 98 106 * This can be called for kernel text or a module. 99 107 */ 100 108 static int map_patch_area(void *addr, unsigned long text_poke_addr) 101 109 { 102 - unsigned long pfn; 103 - 104 - if (IS_ENABLED(CONFIG_MODULES) && is_vmalloc_or_module_addr(addr)) 105 - pfn = vmalloc_to_pfn(addr); 106 - else 107 - pfn = __pa_symbol(addr) >> PAGE_SHIFT; 110 + unsigned long pfn = get_patch_pfn(addr); 108 111 109 112 return map_kernel_page(text_poke_addr, (pfn << PAGE_SHIFT), PAGE_KERNEL); 110 113 } ··· 152 149 int err; 153 150 u32 *patch_addr; 154 151 unsigned long text_poke_addr; 152 + pte_t *pte; 153 + unsigned long pfn = get_patch_pfn(addr); 155 154 156 - text_poke_addr = (unsigned long)__this_cpu_read(text_poke_area)->addr; 155 + text_poke_addr = (unsigned long)__this_cpu_read(text_poke_area)->addr & PAGE_MASK; 157 156 patch_addr = (u32 *)(text_poke_addr + offset_in_page(addr)); 158 157 159 - err = map_patch_area(addr, text_poke_addr); 160 - if (err) 161 - return err; 158 + pte = virt_to_kpte(text_poke_addr); 159 + __set_pte_at(&init_mm, text_poke_addr, pte, pfn_pte(pfn, PAGE_KERNEL), 0); 160 + /* See ptesync comment in radix__set_pte_at() */ 161 + if (radix_enabled()) 162 + asm volatile("ptesync": : :"memory"); 162 163 163 164 err = __patch_instruction(addr, instr, patch_addr); 164 165 165 - unmap_patch_area(text_poke_addr); 166 + pte_clear(&init_mm, text_poke_addr, pte); 167 + flush_tlb_kernel_range(text_poke_addr, text_poke_addr + PAGE_SIZE); 166 168 167 169 return err; 168 170 }