Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

arm64: mm: cleanup __do_page_fault()

Patch series "arch/mm/fault: accelerate pagefault when badaccess", v2.

After VMA lock-based page fault handling enabled, if bad access met
under per-vma lock, it will fallback to mmap_lock-based handling,
so it leads to unnessary mmap lock and vma find again. A test from
lmbench shows 34% improve after this changes on arm64,

lat_sig -P 1 prot lat_sig 0.29194 -> 0.19198


This patch (of 7):

The __do_page_fault() only calls handle_mm_fault() after vm_flags checked,
and it is only called by do_page_fault(), let's squash it into
do_page_fault() to cleanup code.

Link: https://lkml.kernel.org/r/20240403083805.1818160-1-wangkefeng.wang@huawei.com
Link: https://lkml.kernel.org/r/20240403083805.1818160-2-wangkefeng.wang@huawei.com
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Kefeng Wang and committed by
Andrew Morton
6ea02ee4 3931b871

+7 -20
+7 -20
arch/arm64/mm/fault.c
··· 486 486 } 487 487 } 488 488 489 - #define VM_FAULT_BADMAP ((__force vm_fault_t)0x010000) 490 - #define VM_FAULT_BADACCESS ((__force vm_fault_t)0x020000) 491 - 492 - static vm_fault_t __do_page_fault(struct mm_struct *mm, 493 - struct vm_area_struct *vma, unsigned long addr, 494 - unsigned int mm_flags, unsigned long vm_flags, 495 - struct pt_regs *regs) 496 - { 497 - /* 498 - * Ok, we have a good vm_area for this memory access, so we can handle 499 - * it. 500 - * Check that the permissions on the VMA allow for the fault which 501 - * occurred. 502 - */ 503 - if (!(vma->vm_flags & vm_flags)) 504 - return VM_FAULT_BADACCESS; 505 - return handle_mm_fault(vma, addr, mm_flags, regs); 506 - } 507 - 508 489 static bool is_el0_instruction_abort(unsigned long esr) 509 490 { 510 491 return ESR_ELx_EC(esr) == ESR_ELx_EC_IABT_LOW; ··· 499 518 { 500 519 return (esr & ESR_ELx_WNR) && !(esr & ESR_ELx_CM); 501 520 } 521 + 522 + #define VM_FAULT_BADMAP ((__force vm_fault_t)0x010000) 523 + #define VM_FAULT_BADACCESS ((__force vm_fault_t)0x020000) 502 524 503 525 static int __kprobes do_page_fault(unsigned long far, unsigned long esr, 504 526 struct pt_regs *regs) ··· 601 617 goto done; 602 618 } 603 619 604 - fault = __do_page_fault(mm, vma, addr, mm_flags, vm_flags, regs); 620 + if (!(vma->vm_flags & vm_flags)) 621 + fault = VM_FAULT_BADACCESS; 622 + else 623 + fault = handle_mm_fault(vma, addr, mm_flags, regs); 605 624 606 625 /* Quick path to respond to signals */ 607 626 if (fault_signal_pending(fault, regs)) {