Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

arm64: mm: Provide level hint for flush_tlb_page()

Previously tlb invalidations issued by __flush_tlb_page() did not
contain a level hint. From the core API documentation, this function is
clearly only ever intended to target level 3 (PTE) tlb entries:

| 4) ``void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)``
|
| This time we need to remove the PAGE_SIZE sized translation
| from the TLB.

However, the arm64 documentation is more relaxed allowing any last level:

| this operation only invalidates a single, last-level page-table
| entry and therefore does not affect any walk-caches

It turns out that the function was actually being used to invalidate a
level 2 mapping via flush_tlb_fix_spurious_fault_pmd(). The bug was
benign because the level hint was not set so the HW would still
invalidate the PMD mapping, and also because the TLBF_NONOTIFY flag was
set, the bounds of the mapping were never used for anything else.

Now that we have the new and improved range-invalidation API, it is
trival to fix flush_tlb_fix_spurious_fault_pmd() to explicitly flush the
whole range (locally, without notification and last level only). So
let's do that, and then update __flush_tlb_page() to hint level 3.

Reviewed-by: Linu Cherian <linu.cherian@arm.com>
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
[catalin.marinas@arm.com: use "level 3" in the __flush_tlb_page() description]
[catalin.marinas@arm.com: tweak the commit message to include the core API text]
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>

authored by

Ryan Roberts and committed by
Catalin Marinas
752a0d1d 15397e3c

+5 -4
+3 -2
arch/arm64/include/asm/pgtable.h
··· 103 103 #define flush_tlb_fix_spurious_fault(vma, address, ptep) \ 104 104 __flush_tlb_page(vma, address, TLBF_NOBROADCAST | TLBF_NONOTIFY) 105 105 106 - #define flush_tlb_fix_spurious_fault_pmd(vma, address, pmdp) \ 107 - __flush_tlb_page(vma, address, TLBF_NOBROADCAST | TLBF_NONOTIFY) 106 + #define flush_tlb_fix_spurious_fault_pmd(vma, address, pmdp) \ 107 + __flush_tlb_range(vma, address, address + PMD_SIZE, PMD_SIZE, 2, \ 108 + TLBF_NOBROADCAST | TLBF_NONOTIFY | TLBF_NOWALKCACHE) 108 109 109 110 /* 110 111 * ZERO_PAGE is a global shared page that is always zero: used
+2 -2
arch/arm64/include/asm/tlbflush.h
··· 300 300 * __flush_tlb_page(vma, addr, flags) 301 301 * Invalidate a single user mapping for address 'addr' in the 302 302 * address space corresponding to 'vma->mm'. Note that this 303 - * operation only invalidates a single, last-level page-table entry 303 + * operation only invalidates a single level 3 page-table entry 304 304 * and therefore does not affect any walk-caches. flags may contain 305 305 * any combination of TLBF_NONOTIFY (don't call mmu notifiers), 306 306 * TLBF_NOSYNC (don't issue trailing dsb) and TLBF_NOBROADCAST ··· 591 591 unsigned long start = round_down(uaddr, PAGE_SIZE); 592 592 unsigned long end = start + PAGE_SIZE; 593 593 594 - __do_flush_tlb_range(vma, start, end, PAGE_SIZE, TLBI_TTL_UNKNOWN, 594 + __do_flush_tlb_range(vma, start, end, PAGE_SIZE, 3, 595 595 TLBF_NOWALKCACHE | flags); 596 596 } 597 597