tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Merge branch 'bpf-report-arena-faults-to-bpf-streams'

Puranjay Mohan says:

====================
bpf: report arena faults to BPF streams

Changes in v6->v7:
v6: https://lore.kernel.org/all/20250908163638.23150-1-puranjay@kernel.org/
- Added comments about the usage of arena_reg in x86 and arm64 jits. (Alexei)
- Used clear_lo32() for clearing the lower 32-bits of user_vm_start. (Alexei)
- Moved update of the old tests to use __stderr to a separate commit (Eduard)
- Used test__skip() in prog_tests/stream.c (Eduard)
- Start a sub-test for read / write

Changes in v5->v6:
v5: https://lore.kernel.org/all/20250901193730.43543-1-puranjay@kernel.org/
- Introduces __stderr and __stdout for easy testing of bpf streams
(Eduard)
- Add more test cases for arena fault reporting (subprog and callback)
- Fix main_prog_aux usage and return main_prog from find_from_stack_cb
(Kumar)
- Properly fix the build issue reported by kernel test robot

Changes in v4->v5:
v4: https://lore.kernel.org/all/20250827153728.28115-1-puranjay@kernel.org/
- Added patch 2 to introducing main_prog_aux for easier access to
streams.
- Fixed bug in fault handlers when arena_reg == dst_reg
- Updated selftest to check test above edge case.
- Added comments about the usage of barrier_var() in code and commit
message.

Changes in v3->v4:
v3: https://lore.kernel.org/all/20250827150113.15763-1-puranjay@kernel.org/
- Fixed a build issue when CONFIG_BPF_JIT=y and # CONFIG_BPF_SYSCALL is
not set

Changes in v2->v3:
v2: https://lore.kernel.org/all/20250811111828.13836-1-puranjay@kernel.org/
- Improved the selftest to check the exact fault address
- Dropped BPF_NO_KFUNC_PROTOTYPES and bpf_arena_alloc/free_pages() usage
- Rebased on bpf-next/master

Changes in v1->v2:
v1: https://lore.kernel.org/all/20250806085847.18633-1-puranjay@kernel.org/
- Changed variable and mask names for consistency (Yonghong)
- Added Acked-by: Yonghong Song <yonghong.song@linux.dev> on two patches

This set adds the support of reporting page faults inside arena to BPF
stderr stream. The reported address is the one that a user would expect
to see if they pass it to bpf_printk();

Here is an example output from the stderr stream and bpf_printk()

ERROR: Arena WRITE access at unmapped address 0xdeaddead0000
CPU: 9 UID: 0 PID: 502 Comm: test_progs
Call trace:
bpf_stream_stage_dump_stack+0xc0/0x150
bpf_prog_report_arena_violation+0x98/0xf0
ex_handler_bpf+0x5c/0x78
fixup_exception+0xf8/0x160
__do_kernel_fault+0x40/0x188
do_bad_area+0x70/0x88
do_translation_fault+0x54/0x98
do_mem_abort+0x4c/0xa8
el1_abort+0x44/0x70
el1h_64_sync_handler+0x50/0x108
el1h_64_sync+0x6c/0x70
bpf_prog_a64a9778d31b8e88_stream_arena_write_fault+0x84/0xc8
*(page) = 1; @ stream.c:100
bpf_prog_test_run_syscall+0x100/0x328
__sys_bpf+0x508/0xb98
__arm64_sys_bpf+0x2c/0x48
invoke_syscall+0x50/0x120
el0_svc_common.constprop.0+0x48/0xf8
do_el0_svc+0x28/0x40
el0_svc+0x48/0xf8
el0t_64_sync_handler+0xa0/0xe8
el0t_64_sync+0x198/0x1a0

Same address is printed by bpf_printk():

1389.078831: bpf_trace_printk: Read Address: 0xdeaddead0000

To make this possible, some extra metadata has to be passed to the bpf
exception handler, so the bpf exception handling mechanism for both
x86-64 and arm64 have been improved in this set.

The streams selftest has been updated to test this new feature.
====================

Link: https://patch.msgid.link/20250911145808.58042-1-puranjay@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>

Alexei Starovoitov a578b54a 5d87e96a

+491 -112
+63 -22
arch/arm64/net/bpf_jit_comp.c
··· 1066 1066 emit(A64_RET(A64_LR), ctx); 1067 1067 } 1068 1068 1069 - #define BPF_FIXUP_OFFSET_MASK GENMASK(26, 0) 1069 + /* 1070 + * Metadata encoding for exception handling in JITed code. 1071 + * 1072 + * Format of `fixup` field in `struct exception_table_entry`: 1073 + * 1074 + * Bit layout of `fixup` (32-bit): 1075 + * 1076 + * +-----------+--------+-----------+-----------+----------+ 1077 + * | 31-27 | 26-22 | 21 | 20-16 | 15-0 | 1078 + * | | | | | | 1079 + * | FIXUP_REG | Unused | ARENA_ACC | ARENA_REG | OFFSET | 1080 + * +-----------+--------+-----------+-----------+----------+ 1081 + * 1082 + * - OFFSET (16 bits): Offset used to compute address for Load/Store instruction. 1083 + * - ARENA_REG (5 bits): Register that is used to calculate the address for load/store when 1084 + * accessing the arena region. 1085 + * - ARENA_ACCESS (1 bit): This bit is set when the faulting instruction accessed the arena region. 1086 + * - FIXUP_REG (5 bits): Destination register for the load instruction (cleared on fault) or set to 1087 + * DONT_CLEAR if it is a store instruction. 1088 + */ 1089 + 1090 + #define BPF_FIXUP_OFFSET_MASK GENMASK(15, 0) 1091 + #define BPF_FIXUP_ARENA_REG_MASK GENMASK(20, 16) 1092 + #define BPF_ARENA_ACCESS BIT(21) 1070 1093 #define BPF_FIXUP_REG_MASK GENMASK(31, 27) 1071 1094 #define DONT_CLEAR 5 /* Unused ARM64 register from BPF's POV */ 1072 1095 1073 1096 bool ex_handler_bpf(const struct exception_table_entry *ex, 1074 1097 struct pt_regs *regs) 1075 1098 { 1076 - off_t offset = FIELD_GET(BPF_FIXUP_OFFSET_MASK, ex->fixup); 1077 1099 int dst_reg = FIELD_GET(BPF_FIXUP_REG_MASK, ex->fixup); 1100 + s16 off = FIELD_GET(BPF_FIXUP_OFFSET_MASK, ex->fixup); 1101 + int arena_reg = FIELD_GET(BPF_FIXUP_ARENA_REG_MASK, ex->fixup); 1102 + bool is_arena = !!(ex->fixup & BPF_ARENA_ACCESS); 1103 + bool is_write = (dst_reg == DONT_CLEAR); 1104 + unsigned long addr; 1105 + 1106 + if (is_arena) { 1107 + addr = regs->regs[arena_reg] + off; 1108 + bpf_prog_report_arena_violation(is_write, addr, regs->pc); 1109 + } 1078 1110 1079 1111 if (dst_reg != DONT_CLEAR) 1080 1112 regs->regs[dst_reg] = 0; 1081 - regs->pc = (unsigned long)&ex->fixup - offset; 1113 + /* Skip the faulting instruction */ 1114 + regs->pc += AARCH64_INSN_SIZE; 1115 + 1082 1116 return true; 1083 1117 } 1084 1118 ··· 1122 1088 int dst_reg) 1123 1089 { 1124 1090 off_t ins_offset; 1125 - off_t fixup_offset; 1091 + s16 off = insn->off; 1092 + bool is_arena; 1093 + int arena_reg; 1126 1094 unsigned long pc; 1127 1095 struct exception_table_entry *ex; 1128 1096 ··· 1137 1101 BPF_MODE(insn->code) != BPF_PROBE_MEM32 && 1138 1102 BPF_MODE(insn->code) != BPF_PROBE_ATOMIC) 1139 1103 return 0; 1104 + 1105 + is_arena = (BPF_MODE(insn->code) == BPF_PROBE_MEM32) || 1106 + (BPF_MODE(insn->code) == BPF_PROBE_ATOMIC); 1140 1107 1141 1108 if (!ctx->prog->aux->extable || 1142 1109 WARN_ON_ONCE(ctx->exentry_idx >= ctx->prog->aux->num_exentries)) ··· 1159 1120 return -ERANGE; 1160 1121 1161 1122 /* 1162 - * Since the extable follows the program, the fixup offset is always 1163 - * negative and limited to BPF_JIT_REGION_SIZE. Store a positive value 1164 - * to keep things simple, and put the destination register in the upper 1165 - * bits. We don't need to worry about buildtime or runtime sort 1166 - * modifying the upper bits because the table is already sorted, and 1167 - * isn't part of the main exception table. 1168 - * 1169 - * The fixup_offset is set to the next instruction from the instruction 1170 - * that may fault. The execution will jump to this after handling the 1171 - * fault. 1172 - */ 1173 - fixup_offset = (long)&ex->fixup - (pc + AARCH64_INSN_SIZE); 1174 - if (!FIELD_FIT(BPF_FIXUP_OFFSET_MASK, fixup_offset)) 1175 - return -ERANGE; 1176 - 1177 - /* 1178 1123 * The offsets above have been calculated using the RO buffer but we 1179 1124 * need to use the R/W buffer for writes. 1180 1125 * switch ex to rw buffer for writing. ··· 1170 1147 if (BPF_CLASS(insn->code) != BPF_LDX) 1171 1148 dst_reg = DONT_CLEAR; 1172 1149 1173 - ex->fixup = FIELD_PREP(BPF_FIXUP_OFFSET_MASK, fixup_offset) | 1174 - FIELD_PREP(BPF_FIXUP_REG_MASK, dst_reg); 1150 + ex->fixup = FIELD_PREP(BPF_FIXUP_REG_MASK, dst_reg); 1151 + 1152 + if (is_arena) { 1153 + ex->fixup |= BPF_ARENA_ACCESS; 1154 + /* 1155 + * insn->src_reg/dst_reg holds the address in the arena region with upper 32-bits 1156 + * being zero because of a preceding addr_space_cast(r<n>, 0x0, 0x1) instruction. 1157 + * This address is adjusted with the addition of arena_vm_start (see the 1158 + * implementation of BPF_PROBE_MEM32 and BPF_PROBE_ATOMIC) before being used for the 1159 + * memory access. Pass the reg holding the unmodified 32-bit address to 1160 + * ex_handler_bpf. 1161 + */ 1162 + if (BPF_CLASS(insn->code) == BPF_LDX) 1163 + arena_reg = bpf2a64[insn->src_reg]; 1164 + else 1165 + arena_reg = bpf2a64[insn->dst_reg]; 1166 + 1167 + ex->fixup |= FIELD_PREP(BPF_FIXUP_OFFSET_MASK, off) | 1168 + FIELD_PREP(BPF_FIXUP_ARENA_REG_MASK, arena_reg); 1169 + } 1175 1170 1176 1171 ex->type = EX_TYPE_BPF; 1177 1172
+80 -5
arch/x86/net/bpf_jit_comp.c
··· 8 8 #include <linux/netdevice.h> 9 9 #include <linux/filter.h> 10 10 #include <linux/if_vlan.h> 11 + #include <linux/bitfield.h> 11 12 #include <linux/bpf.h> 12 13 #include <linux/memory.h> 13 14 #include <linux/sort.h> ··· 1389 1388 return 0; 1390 1389 } 1391 1390 1391 + /* 1392 + * Metadata encoding for exception handling in JITed code. 1393 + * 1394 + * Format of `fixup` and `data` fields in `struct exception_table_entry`: 1395 + * 1396 + * Bit layout of `fixup` (32-bit): 1397 + * 1398 + * +-----------+--------+-----------+---------+----------+ 1399 + * | 31 | 30-24 | 23-16 | 15-8 | 7-0 | 1400 + * | | | | | | 1401 + * | ARENA_ACC | Unused | ARENA_REG | DST_REG | INSN_LEN | 1402 + * +-----------+--------+-----------+---------+----------+ 1403 + * 1404 + * - INSN_LEN (8 bits): Length of faulting insn (max x86 insn = 15 bytes (fits in 8 bits)). 1405 + * - DST_REG (8 bits): Offset of dst_reg from reg2pt_regs[] (max offset = 112 (fits in 8 bits)). 1406 + * This is set to DONT_CLEAR if the insn is a store. 1407 + * - ARENA_REG (8 bits): Offset of the register that is used to calculate the 1408 + * address for load/store when accessing the arena region. 1409 + * - ARENA_ACCESS (1 bit): This bit is set when the faulting instruction accessed the arena region. 1410 + * 1411 + * Bit layout of `data` (32-bit): 1412 + * 1413 + * +--------------+--------+--------------+ 1414 + * | 31-16 | 15-8 | 7-0 | 1415 + * | | | | 1416 + * | ARENA_OFFSET | Unused | EX_TYPE_BPF | 1417 + * +--------------+--------+--------------+ 1418 + * 1419 + * - ARENA_OFFSET (16 bits): Offset used to calculate the address for load/store when 1420 + * accessing the arena region. 1421 + */ 1422 + 1392 1423 #define DONT_CLEAR 1 1424 + #define FIXUP_INSN_LEN_MASK GENMASK(7, 0) 1425 + #define FIXUP_REG_MASK GENMASK(15, 8) 1426 + #define FIXUP_ARENA_REG_MASK GENMASK(23, 16) 1427 + #define FIXUP_ARENA_ACCESS BIT(31) 1428 + #define DATA_ARENA_OFFSET_MASK GENMASK(31, 16) 1393 1429 1394 1430 bool ex_handler_bpf(const struct exception_table_entry *x, struct pt_regs *regs) 1395 1431 { 1396 - u32 reg = x->fixup >> 8; 1432 + u32 reg = FIELD_GET(FIXUP_REG_MASK, x->fixup); 1433 + u32 insn_len = FIELD_GET(FIXUP_INSN_LEN_MASK, x->fixup); 1434 + bool is_arena = !!(x->fixup & FIXUP_ARENA_ACCESS); 1435 + bool is_write = (reg == DONT_CLEAR); 1436 + unsigned long addr; 1437 + s16 off; 1438 + u32 arena_reg; 1439 + 1440 + if (is_arena) { 1441 + arena_reg = FIELD_GET(FIXUP_ARENA_REG_MASK, x->fixup); 1442 + off = FIELD_GET(DATA_ARENA_OFFSET_MASK, x->data); 1443 + addr = *(unsigned long *)((void *)regs + arena_reg) + off; 1444 + bpf_prog_report_arena_violation(is_write, addr, regs->ip); 1445 + } 1397 1446 1398 1447 /* jump over faulting load and clear dest register */ 1399 1448 if (reg != DONT_CLEAR) 1400 1449 *(unsigned long *)((void *)regs + reg) = 0; 1401 - regs->ip += x->fixup & 0xff; 1450 + regs->ip += insn_len; 1451 + 1402 1452 return true; 1403 1453 } 1404 1454 ··· 2122 2070 { 2123 2071 struct exception_table_entry *ex; 2124 2072 u8 *_insn = image + proglen + (start_of_ldx - temp); 2073 + u32 arena_reg, fixup_reg; 2125 2074 s64 delta; 2126 2075 2127 2076 if (!bpf_prog->aux->extable) ··· 2142 2089 2143 2090 ex->data = EX_TYPE_BPF; 2144 2091 2145 - ex->fixup = (prog - start_of_ldx) | 2146 - ((BPF_CLASS(insn->code) == BPF_LDX ? reg2pt_regs[dst_reg] : DONT_CLEAR) << 8); 2092 + /* 2093 + * src_reg/dst_reg holds the address in the arena region with upper 2094 + * 32-bits being zero because of a preceding addr_space_cast(r<n>, 2095 + * 0x0, 0x1) instruction. This address is adjusted with the addition 2096 + * of arena_vm_start (see the implementation of BPF_PROBE_MEM32 and 2097 + * BPF_PROBE_ATOMIC) before being used for the memory access. Pass 2098 + * the reg holding the unmodified 32-bit address to 2099 + * ex_handler_bpf(). 2100 + */ 2101 + if (BPF_CLASS(insn->code) == BPF_LDX) { 2102 + arena_reg = reg2pt_regs[src_reg]; 2103 + fixup_reg = reg2pt_regs[dst_reg]; 2104 + } else { 2105 + arena_reg = reg2pt_regs[dst_reg]; 2106 + fixup_reg = DONT_CLEAR; 2107 + } 2108 + 2109 + ex->fixup = FIELD_PREP(FIXUP_INSN_LEN_MASK, prog - start_of_ldx) | 2110 + FIELD_PREP(FIXUP_ARENA_REG_MASK, arena_reg) | 2111 + FIELD_PREP(FIXUP_REG_MASK, fixup_reg); 2112 + ex->fixup |= FIXUP_ARENA_ACCESS; 2113 + 2114 + ex->data |= FIELD_PREP(DATA_ARENA_OFFSET_MASK, insn->off); 2147 2115 } 2148 2116 break; 2149 2117 ··· 2282 2208 * End result: x86 insn "mov rbx, qword ptr [rax+0x14]" 2283 2209 * of 4 bytes will be ignored and rbx will be zero inited. 2284 2210 */ 2285 - ex->fixup = (prog - start_of_ldx) | (reg2pt_regs[dst_reg] << 8); 2211 + ex->fixup = FIELD_PREP(FIXUP_INSN_LEN_MASK, prog - start_of_ldx) | 2212 + FIELD_PREP(FIXUP_REG_MASK, reg2pt_regs[dst_reg]); 2286 2213 } 2287 2214 break; 2288 2215
+7
include/linux/bpf.h
··· 1633 1633 /* function name for valid attach_btf_id */ 1634 1634 const char *attach_func_name; 1635 1635 struct bpf_prog **func; 1636 + struct bpf_prog_aux *main_prog_aux; 1636 1637 void *jit_data; /* JIT specific data. arch dependent */ 1637 1638 struct bpf_jit_poke_descriptor *poke_tab; 1638 1639 struct bpf_kfunc_desc_tab *kfunc_tab; ··· 2881 2880 enum bpf_dynptr_type type, u32 offset, u32 size); 2882 2881 void bpf_dynptr_set_null(struct bpf_dynptr_kern *ptr); 2883 2882 void bpf_dynptr_set_rdonly(struct bpf_dynptr_kern *ptr); 2883 + void bpf_prog_report_arena_violation(bool write, unsigned long addr, unsigned long fault_ip); 2884 2884 2885 2885 #else /* !CONFIG_BPF_SYSCALL */ 2886 2886 static inline struct bpf_prog *bpf_prog_get(u32 ufd) ··· 3167 3165 } 3168 3166 3169 3167 static inline void bpf_dynptr_set_rdonly(struct bpf_dynptr_kern *ptr) 3168 + { 3169 + } 3170 + 3171 + static inline void bpf_prog_report_arena_violation(bool write, unsigned long addr, 3172 + unsigned long fault_ip) 3170 3173 { 3171 3174 } 3172 3175 #endif /* CONFIG_BPF_SYSCALL */
+30
kernel/bpf/arena.c
··· 633 633 return register_btf_kfunc_id_set(BPF_PROG_TYPE_UNSPEC, &common_kfunc_set); 634 634 } 635 635 late_initcall(kfunc_init); 636 + 637 + void bpf_prog_report_arena_violation(bool write, unsigned long addr, unsigned long fault_ip) 638 + { 639 + struct bpf_stream_stage ss; 640 + struct bpf_prog *prog; 641 + u64 user_vm_start; 642 + 643 + /* 644 + * The RCU read lock is held to safely traverse the latch tree, but we 645 + * don't need its protection when accessing the prog, since it will not 646 + * disappear while we are handling the fault. 647 + */ 648 + rcu_read_lock(); 649 + prog = bpf_prog_ksym_find(fault_ip); 650 + rcu_read_unlock(); 651 + if (!prog) 652 + return; 653 + 654 + /* Use main prog for stream access */ 655 + prog = prog->aux->main_prog_aux->prog; 656 + 657 + user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena); 658 + addr += clear_lo32(user_vm_start); 659 + 660 + bpf_stream_stage(ss, prog, BPF_STDERR, ({ 661 + bpf_stream_printk(ss, "ERROR: Arena %s access at unmapped address 0x%lx\n", 662 + write ? "WRITE" : "READ", addr); 663 + bpf_stream_dump_stack(ss); 664 + })); 665 + }
+3 -3
kernel/bpf/core.c
··· 120 120 121 121 fp->pages = size / PAGE_SIZE; 122 122 fp->aux = aux; 123 + fp->aux->main_prog_aux = aux; 123 124 fp->aux->prog = fp; 124 125 fp->jit_requested = ebpf_jit_enabled(); 125 126 fp->blinding_requested = bpf_jit_blinding_enabled(fp); ··· 3298 3297 rcu_read_unlock(); 3299 3298 if (!prog) 3300 3299 return true; 3301 - if (bpf_is_subprog(prog)) 3302 - return true; 3303 - ctxp->prog = prog; 3300 + /* Make sure we return the main prog if we found a subprog */ 3301 + ctxp->prog = prog->aux->main_prog_aux->prog; 3304 3302 return false; 3305 3303 } 3306 3304
+1
kernel/bpf/verifier.c
··· 21601 21601 func[i]->aux->func_info_cnt = prog->aux->func_info_cnt; 21602 21602 func[i]->aux->poke_tab = prog->aux->poke_tab; 21603 21603 func[i]->aux->size_poke_tab = prog->aux->size_poke_tab; 21604 + func[i]->aux->main_prog_aux = prog->aux; 21604 21605 21605 21606 for (j = 0; j < prog->aux->size_poke_tab; j++) { 21606 21607 struct bpf_jit_poke_descriptor *poke;
+49 -82
tools/testing/selftests/bpf/prog_tests/stream.c
··· 2 2 /* Copyright (c) 2025 Meta Platforms, Inc. and affiliates. */ 3 3 #include <test_progs.h> 4 4 #include <sys/mman.h> 5 - #include <regex.h> 6 5 7 6 #include "stream.skel.h" 8 7 #include "stream_fail.skel.h" ··· 15 16 { 16 17 RUN_TESTS(stream); 17 18 return; 18 - } 19 - 20 - struct { 21 - int prog_off; 22 - const char *errstr; 23 - } stream_error_arr[] = { 24 - { 25 - offsetof(struct stream, progs.stream_cond_break), 26 - "ERROR: Timeout detected for may_goto instruction\n" 27 - "CPU: [0-9]+ UID: 0 PID: [0-9]+ Comm: .*\n" 28 - "Call trace:\n" 29 - "([a-zA-Z_][a-zA-Z0-9_]*\\+0x[0-9a-fA-F]+/0x[0-9a-fA-F]+\n" 30 - "|[ \t]+[^\n]+\n)*", 31 - }, 32 - { 33 - offsetof(struct stream, progs.stream_deadlock), 34 - "ERROR: AA or ABBA deadlock detected for bpf_res_spin_lock\n" 35 - "Attempted lock = (0x[0-9a-fA-F]+)\n" 36 - "Total held locks = 1\n" 37 - "Held lock\\[ 0\\] = \\1\n" // Lock address must match 38 - "CPU: [0-9]+ UID: 0 PID: [0-9]+ Comm: .*\n" 39 - "Call trace:\n" 40 - "([a-zA-Z_][a-zA-Z0-9_]*\\+0x[0-9a-fA-F]+/0x[0-9a-fA-F]+\n" 41 - "|[ \t]+[^\n]+\n)*", 42 - }, 43 - }; 44 - 45 - static int match_regex(const char *pattern, const char *string) 46 - { 47 - int err, rc; 48 - regex_t re; 49 - 50 - err = regcomp(&re, pattern, REG_EXTENDED | REG_NEWLINE); 51 - if (err) 52 - return -1; 53 - rc = regexec(&re, string, 0, NULL, 0); 54 - regfree(&re); 55 - return rc == 0 ? 1 : 0; 56 - } 57 - 58 - void test_stream_errors(void) 59 - { 60 - LIBBPF_OPTS(bpf_test_run_opts, opts); 61 - LIBBPF_OPTS(bpf_prog_stream_read_opts, ropts); 62 - struct stream *skel; 63 - int ret, prog_fd; 64 - char buf[1024]; 65 - 66 - skel = stream__open_and_load(); 67 - if (!ASSERT_OK_PTR(skel, "stream__open_and_load")) 68 - return; 69 - 70 - for (int i = 0; i < ARRAY_SIZE(stream_error_arr); i++) { 71 - struct bpf_program **prog; 72 - 73 - prog = (struct bpf_program **)(((char *)skel) + stream_error_arr[i].prog_off); 74 - prog_fd = bpf_program__fd(*prog); 75 - ret = bpf_prog_test_run_opts(prog_fd, &opts); 76 - ASSERT_OK(ret, "ret"); 77 - ASSERT_OK(opts.retval, "retval"); 78 - 79 - #if !defined(__x86_64__) && !defined(__s390x__) && !defined(__aarch64__) 80 - ASSERT_TRUE(1, "Timed may_goto unsupported, skip."); 81 - if (i == 0) { 82 - ret = bpf_prog_stream_read(prog_fd, 2, buf, sizeof(buf), &ropts); 83 - ASSERT_EQ(ret, 0, "stream read"); 84 - continue; 85 - } 86 - #endif 87 - 88 - ret = bpf_prog_stream_read(prog_fd, BPF_STREAM_STDERR, buf, sizeof(buf), &ropts); 89 - ASSERT_GT(ret, 0, "stream read"); 90 - ASSERT_LE(ret, 1023, "len for buf"); 91 - buf[ret] = '\0'; 92 - 93 - ret = match_regex(stream_error_arr[i].errstr, buf); 94 - if (!ASSERT_TRUE(ret == 1, "regex match")) 95 - fprintf(stderr, "Output from stream:\n%s\n", buf); 96 - } 97 - 98 - stream__destroy(skel); 99 19 } 100 20 101 21 void test_stream_syscall(void) ··· 54 136 ASSERT_EQ(ret, 0, "no bytes stdout"); 55 137 ret = bpf_prog_stream_read(prog_fd, BPF_STREAM_STDERR, buf, 1, &ropts); 56 138 ASSERT_EQ(ret, 0, "no bytes stderr"); 139 + 140 + stream__destroy(skel); 141 + } 142 + 143 + static void test_address(struct bpf_program *prog, unsigned long *fault_addr_p) 144 + { 145 + LIBBPF_OPTS(bpf_test_run_opts, opts); 146 + LIBBPF_OPTS(bpf_prog_stream_read_opts, ropts); 147 + int ret, prog_fd; 148 + char fault_addr[64]; 149 + char buf[1024]; 150 + 151 + prog_fd = bpf_program__fd(prog); 152 + 153 + ret = bpf_prog_test_run_opts(prog_fd, &opts); 154 + ASSERT_OK(ret, "ret"); 155 + ASSERT_OK(opts.retval, "retval"); 156 + 157 + sprintf(fault_addr, "0x%lx", *fault_addr_p); 158 + 159 + ret = bpf_prog_stream_read(prog_fd, BPF_STREAM_STDERR, buf, sizeof(buf), &ropts); 160 + ASSERT_GT(ret, 0, "stream read"); 161 + ASSERT_LE(ret, 1023, "len for buf"); 162 + buf[ret] = '\0'; 163 + 164 + if (!ASSERT_HAS_SUBSTR(buf, fault_addr, "fault_addr")) { 165 + fprintf(stderr, "Output from stream:\n%s\n", buf); 166 + fprintf(stderr, "Fault Addr: %s\n", fault_addr); 167 + } 168 + } 169 + 170 + void test_stream_arena_fault_address(void) 171 + { 172 + struct stream *skel; 173 + 174 + #if !defined(__x86_64__) && !defined(__aarch64__) 175 + printf("%s:SKIP: arena fault reporting not supported\n", __func__); 176 + test__skip(); 177 + return; 178 + #endif 179 + 180 + skel = stream__open_and_load(); 181 + if (!ASSERT_OK_PTR(skel, "stream__open_and_load")) 182 + return; 183 + 184 + if (test__start_subtest("read_fault")) 185 + test_address(skel->progs.stream_arena_read_fault, &skel->bss->fault_addr); 186 + if (test__start_subtest("write_fault")) 187 + test_address(skel->progs.stream_arena_write_fault, &skel->bss->fault_addr); 57 188 58 189 stream__destroy(skel); 59 190 }
+10
tools/testing/selftests/bpf/progs/bpf_misc.h
··· 35 35 * inside the brackets. 36 36 * __msg_unpriv Same as __msg but for unprivileged mode. 37 37 * 38 + * __stderr Message expected to be found in bpf stderr stream. The 39 + * same regex rules apply like __msg. 40 + * __stderr_unpriv Same as __stderr but for unpriveleged mode. 41 + * __stdout Same as __stderr but for stdout stream. 42 + * __stdout_unpriv Same as __stdout but for unpriveleged mode. 43 + * 38 44 * __xlated Expect a line in a disassembly log after verifier applies rewrites. 39 45 * Multiple __xlated attributes could be specified. 40 46 * Regular expressions could be specified same way as in __msg. ··· 146 140 #define __caps_unpriv(caps) __attribute__((btf_decl_tag("comment:test_caps_unpriv=" EXPAND_QUOTE(caps)))) 147 141 #define __load_if_JITed() __attribute__((btf_decl_tag("comment:load_mode=jited"))) 148 142 #define __load_if_no_JITed() __attribute__((btf_decl_tag("comment:load_mode=no_jited"))) 143 + #define __stderr(msg) __attribute__((btf_decl_tag("comment:test_expect_stderr=" XSTR(__COUNTER__) "=" msg))) 144 + #define __stderr_unpriv(msg) __attribute__((btf_decl_tag("comment:test_expect_stderr_unpriv=" XSTR(__COUNTER__) "=" msg))) 145 + #define __stdout(msg) __attribute__((btf_decl_tag("comment:test_expect_stdout=" XSTR(__COUNTER__) "=" msg))) 146 + #define __stdout_unpriv(msg) __attribute__((btf_decl_tag("comment:test_expect_stdout_unpriv=" XSTR(__COUNTER__) "=" msg))) 149 147 150 148 /* Define common capabilities tested using __caps_unpriv */ 151 149 #define CAP_NET_ADMIN 12
+158
tools/testing/selftests/bpf/progs/stream.c
··· 5 5 #include <bpf/bpf_helpers.h> 6 6 #include "bpf_misc.h" 7 7 #include "bpf_experimental.h" 8 + #include "bpf_arena_common.h" 8 9 9 10 struct arr_elem { 10 11 struct bpf_res_spin_lock lock; ··· 18 17 __type(value, struct arr_elem); 19 18 } arrmap SEC(".maps"); 20 19 20 + struct { 21 + __uint(type, BPF_MAP_TYPE_ARENA); 22 + __uint(map_flags, BPF_F_MMAPABLE); 23 + __uint(max_entries, 1); /* number of pages */ 24 + } arena SEC(".maps"); 25 + 26 + struct elem { 27 + struct bpf_timer timer; 28 + }; 29 + 30 + struct { 31 + __uint(type, BPF_MAP_TYPE_ARRAY); 32 + __uint(max_entries, 1); 33 + __type(key, int); 34 + __type(value, struct elem); 35 + } array SEC(".maps"); 36 + 21 37 #define ENOSPC 28 22 38 #define _STR "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" 23 39 24 40 int size; 41 + u64 fault_addr; 42 + void *arena_ptr; 25 43 26 44 SEC("syscall") 27 45 __success __retval(0) ··· 57 37 } 58 38 59 39 SEC("syscall") 40 + __arch_x86_64 41 + __arch_arm64 42 + __arch_s390x 60 43 __success __retval(0) 44 + __stderr("ERROR: Timeout detected for may_goto instruction") 45 + __stderr("CPU: {{[0-9]+}} UID: 0 PID: {{[0-9]+}} Comm: {{.*}}") 46 + __stderr("Call trace:\n" 47 + "{{([a-zA-Z_][a-zA-Z0-9_]*\\+0x[0-9a-fA-F]+/0x[0-9a-fA-F]+\n" 48 + "|[ \t]+[^\n]+\n)*}}") 61 49 int stream_cond_break(void *ctx) 62 50 { 63 51 while (can_loop) ··· 75 47 76 48 SEC("syscall") 77 49 __success __retval(0) 50 + __stderr("ERROR: AA or ABBA deadlock detected for bpf_res_spin_lock") 51 + __stderr("{{Attempted lock = (0x[0-9a-fA-F]+)\n" 52 + "Total held locks = 1\n" 53 + "Held lock\\[ 0\\] = \\1}}") 54 + __stderr("...") 55 + __stderr("CPU: {{[0-9]+}} UID: 0 PID: {{[0-9]+}} Comm: {{.*}}") 56 + __stderr("Call trace:\n" 57 + "{{([a-zA-Z_][a-zA-Z0-9_]*\\+0x[0-9a-fA-F]+/0x[0-9a-fA-F]+\n" 58 + "|[ \t]+[^\n]+\n)*}}") 78 59 int stream_deadlock(void *ctx) 79 60 { 80 61 struct bpf_res_spin_lock *lock, *nlock; ··· 110 73 int stream_syscall(void *ctx) 111 74 { 112 75 bpf_stream_printk(BPF_STDOUT, "foo"); 76 + return 0; 77 + } 78 + 79 + SEC("syscall") 80 + __arch_x86_64 81 + __arch_arm64 82 + __success __retval(0) 83 + __stderr("ERROR: Arena WRITE access at unmapped address 0x{{.*}}") 84 + __stderr("CPU: {{[0-9]+}} UID: 0 PID: {{[0-9]+}} Comm: {{.*}}") 85 + __stderr("Call trace:\n" 86 + "{{([a-zA-Z_][a-zA-Z0-9_]*\\+0x[0-9a-fA-F]+/0x[0-9a-fA-F]+\n" 87 + "|[ \t]+[^\n]+\n)*}}") 88 + int stream_arena_write_fault(void *ctx) 89 + { 90 + struct bpf_arena *ptr = (void *)&arena; 91 + u64 user_vm_start; 92 + 93 + /* Prevent GCC bounds warning: casting &arena to struct bpf_arena * 94 + * triggers bounds checking since the map definition is smaller than struct 95 + * bpf_arena. barrier_var() makes the pointer opaque to GCC, preventing the 96 + * bounds analysis 97 + */ 98 + barrier_var(ptr); 99 + user_vm_start = ptr->user_vm_start; 100 + fault_addr = user_vm_start + 0x7fff; 101 + bpf_addr_space_cast(user_vm_start, 0, 1); 102 + asm volatile ( 103 + "r1 = %0;" 104 + "r2 = 1;" 105 + "*(u32 *)(r1 + 0x7fff) = r2;" 106 + : 107 + : "r" (user_vm_start) 108 + : "r1", "r2" 109 + ); 110 + return 0; 111 + } 112 + 113 + SEC("syscall") 114 + __arch_x86_64 115 + __arch_arm64 116 + __success __retval(0) 117 + __stderr("ERROR: Arena READ access at unmapped address 0x{{.*}}") 118 + __stderr("CPU: {{[0-9]+}} UID: 0 PID: {{[0-9]+}} Comm: {{.*}}") 119 + __stderr("Call trace:\n" 120 + "{{([a-zA-Z_][a-zA-Z0-9_]*\\+0x[0-9a-fA-F]+/0x[0-9a-fA-F]+\n" 121 + "|[ \t]+[^\n]+\n)*}}") 122 + int stream_arena_read_fault(void *ctx) 123 + { 124 + struct bpf_arena *ptr = (void *)&arena; 125 + u64 user_vm_start; 126 + 127 + /* Prevent GCC bounds warning: casting &arena to struct bpf_arena * 128 + * triggers bounds checking since the map definition is smaller than struct 129 + * bpf_arena. barrier_var() makes the pointer opaque to GCC, preventing the 130 + * bounds analysis 131 + */ 132 + barrier_var(ptr); 133 + user_vm_start = ptr->user_vm_start; 134 + fault_addr = user_vm_start + 0x7fff; 135 + bpf_addr_space_cast(user_vm_start, 0, 1); 136 + asm volatile ( 137 + "r1 = %0;" 138 + "r1 = *(u32 *)(r1 + 0x7fff);" 139 + : 140 + : "r" (user_vm_start) 141 + : "r1" 142 + ); 143 + return 0; 144 + } 145 + 146 + static __noinline void subprog(void) 147 + { 148 + int __arena *addr = (int __arena *)0xdeadbeef; 149 + 150 + arena_ptr = &arena; 151 + *addr = 1; 152 + } 153 + 154 + SEC("syscall") 155 + __arch_x86_64 156 + __arch_arm64 157 + __success __retval(0) 158 + __stderr("ERROR: Arena WRITE access at unmapped address 0x{{.*}}") 159 + __stderr("CPU: {{[0-9]+}} UID: 0 PID: {{[0-9]+}} Comm: {{.*}}") 160 + __stderr("Call trace:\n" 161 + "{{([a-zA-Z_][a-zA-Z0-9_]*\\+0x[0-9a-fA-F]+/0x[0-9a-fA-F]+\n" 162 + "|[ \t]+[^\n]+\n)*}}") 163 + int stream_arena_subprog_fault(void *ctx) 164 + { 165 + subprog(); 166 + return 0; 167 + } 168 + 169 + static __noinline int timer_cb(void *map, int *key, struct bpf_timer *timer) 170 + { 171 + int __arena *addr = (int __arena *)0xdeadbeef; 172 + 173 + arena_ptr = &arena; 174 + *addr = 1; 175 + return 0; 176 + } 177 + 178 + SEC("syscall") 179 + __arch_x86_64 180 + __arch_arm64 181 + __success __retval(0) 182 + __stderr("ERROR: Arena WRITE access at unmapped address 0x{{.*}}") 183 + __stderr("CPU: {{[0-9]+}} UID: 0 PID: {{[0-9]+}} Comm: {{.*}}") 184 + __stderr("Call trace:\n" 185 + "{{([a-zA-Z_][a-zA-Z0-9_]*\\+0x[0-9a-fA-F]+/0x[0-9a-fA-F]+\n" 186 + "|[ \t]+[^\n]+\n)*}}") 187 + int stream_arena_callback_fault(void *ctx) 188 + { 189 + struct bpf_timer *arr_timer; 190 + 191 + arr_timer = bpf_map_lookup_elem(&array, &(int){0}); 192 + if (!arr_timer) 193 + return 0; 194 + bpf_timer_init(arr_timer, &array, 1); 195 + bpf_timer_set_callback(arr_timer, timer_cb); 196 + bpf_timer_start(arr_timer, 0, 0); 113 197 return 0; 114 198 } 115 199
+90
tools/testing/selftests/bpf/test_loader.c
··· 38 38 #define TEST_TAG_JITED_PFX_UNPRIV "comment:test_jited_unpriv=" 39 39 #define TEST_TAG_CAPS_UNPRIV "comment:test_caps_unpriv=" 40 40 #define TEST_TAG_LOAD_MODE_PFX "comment:load_mode=" 41 + #define TEST_TAG_EXPECT_STDERR_PFX "comment:test_expect_stderr=" 42 + #define TEST_TAG_EXPECT_STDERR_PFX_UNPRIV "comment:test_expect_stderr_unpriv=" 43 + #define TEST_TAG_EXPECT_STDOUT_PFX "comment:test_expect_stdout=" 44 + #define TEST_TAG_EXPECT_STDOUT_PFX_UNPRIV "comment:test_expect_stdout_unpriv=" 41 45 42 46 /* Warning: duplicated in bpf_misc.h */ 43 47 #define POINTER_VALUE 0xbadcafe ··· 83 79 struct expected_msgs expect_msgs; 84 80 struct expected_msgs expect_xlated; 85 81 struct expected_msgs jited; 82 + struct expected_msgs stderr; 83 + struct expected_msgs stdout; 86 84 int retval; 87 85 bool execute; 88 86 __u64 caps; ··· 145 139 free_msgs(&spec->unpriv.expect_xlated); 146 140 free_msgs(&spec->priv.jited); 147 141 free_msgs(&spec->unpriv.jited); 142 + free_msgs(&spec->unpriv.stderr); 143 + free_msgs(&spec->priv.stderr); 144 + free_msgs(&spec->unpriv.stdout); 145 + free_msgs(&spec->priv.stdout); 148 146 149 147 free(spec->priv.name); 150 148 free(spec->unpriv.name); ··· 417 407 bool xlated_on_next_line = true; 418 408 bool unpriv_jit_on_next_line; 419 409 bool jit_on_next_line; 410 + bool stderr_on_next_line = true; 411 + bool unpriv_stderr_on_next_line = true; 412 + bool stdout_on_next_line = true; 413 + bool unpriv_stdout_on_next_line = true; 420 414 bool collect_jit = false; 421 415 int func_id, i, err = 0; 422 416 u32 arch_mask = 0; ··· 612 598 err = -EINVAL; 613 599 goto cleanup; 614 600 } 601 + } else if ((msg = skip_dynamic_pfx(s, TEST_TAG_EXPECT_STDERR_PFX))) { 602 + err = push_disasm_msg(msg, &stderr_on_next_line, 603 + &spec->priv.stderr); 604 + if (err) 605 + goto cleanup; 606 + } else if ((msg = skip_dynamic_pfx(s, TEST_TAG_EXPECT_STDERR_PFX_UNPRIV))) { 607 + err = push_disasm_msg(msg, &unpriv_stderr_on_next_line, 608 + &spec->unpriv.stderr); 609 + if (err) 610 + goto cleanup; 611 + } else if ((msg = skip_dynamic_pfx(s, TEST_TAG_EXPECT_STDOUT_PFX))) { 612 + err = push_disasm_msg(msg, &stdout_on_next_line, 613 + &spec->priv.stdout); 614 + if (err) 615 + goto cleanup; 616 + } else if ((msg = skip_dynamic_pfx(s, TEST_TAG_EXPECT_STDOUT_PFX_UNPRIV))) { 617 + err = push_disasm_msg(msg, &unpriv_stdout_on_next_line, 618 + &spec->unpriv.stdout); 619 + if (err) 620 + goto cleanup; 615 621 } 616 622 } 617 623 ··· 685 651 clone_msgs(&spec->priv.expect_xlated, &spec->unpriv.expect_xlated); 686 652 if (spec->unpriv.jited.cnt == 0) 687 653 clone_msgs(&spec->priv.jited, &spec->unpriv.jited); 654 + if (spec->unpriv.stderr.cnt == 0) 655 + clone_msgs(&spec->priv.stderr, &spec->unpriv.stderr); 656 + if (spec->unpriv.stdout.cnt == 0) 657 + clone_msgs(&spec->priv.stdout, &spec->unpriv.stdout); 688 658 } 689 659 690 660 spec->valid = true; ··· 748 710 if (!force && env.verbosity == VERBOSE_NONE) 749 711 return; 750 712 fprintf(stdout, "JITED:\n=============\n%s=============\n", jited); 713 + } 714 + 715 + static void emit_stderr(const char *stderr, bool force) 716 + { 717 + if (!force && env.verbosity == VERBOSE_NONE) 718 + return; 719 + fprintf(stdout, "STDERR:\n=============\n%s=============\n", stderr); 720 + } 721 + 722 + static void emit_stdout(const char *bpf_stdout, bool force) 723 + { 724 + if (!force && env.verbosity == VERBOSE_NONE) 725 + return; 726 + fprintf(stdout, "STDOUT:\n=============\n%s=============\n", bpf_stdout); 751 727 } 752 728 753 729 static void validate_msgs(char *log_buf, struct expected_msgs *msgs, ··· 986 934 return err; 987 935 } 988 936 937 + /* Read the bpf stream corresponding to the stream_id */ 938 + static int get_stream(int stream_id, int prog_fd, char *text, size_t text_sz) 939 + { 940 + LIBBPF_OPTS(bpf_prog_stream_read_opts, ropts); 941 + int ret; 942 + 943 + ret = bpf_prog_stream_read(prog_fd, stream_id, text, text_sz, &ropts); 944 + ASSERT_GT(ret, 0, "stream read"); 945 + text[ret] = '\0'; 946 + 947 + return ret; 948 + } 949 + 989 950 /* this function is forced noinline and has short generic name to look better 990 951 * in test_progs output (in case of a failure) 991 952 */ ··· 1173 1108 PRINT_FAIL("Unexpected retval: %d != %d\n", retval, subspec->retval); 1174 1109 goto tobj_cleanup; 1175 1110 } 1111 + 1112 + if (subspec->stderr.cnt) { 1113 + err = get_stream(2, bpf_program__fd(tprog), 1114 + tester->log_buf, tester->log_buf_sz); 1115 + if (err <= 0) { 1116 + PRINT_FAIL("Unexpected retval from get_stream(): %d, errno = %d\n", 1117 + err, errno); 1118 + goto tobj_cleanup; 1119 + } 1120 + emit_stderr(tester->log_buf, false /*force*/); 1121 + validate_msgs(tester->log_buf, &subspec->stderr, emit_stderr); 1122 + } 1123 + 1124 + if (subspec->stdout.cnt) { 1125 + err = get_stream(1, bpf_program__fd(tprog), 1126 + tester->log_buf, tester->log_buf_sz); 1127 + if (err <= 0) { 1128 + PRINT_FAIL("Unexpected retval from get_stream(): %d, errno = %d\n", 1129 + err, errno); 1130 + goto tobj_cleanup; 1131 + } 1132 + emit_stdout(tester->log_buf, false /*force*/); 1133 + validate_msgs(tester->log_buf, &subspec->stdout, emit_stdout); 1134 + } 1135 + 1176 1136 /* redo bpf_map__attach_struct_ops for each test */ 1177 1137 while (links_cnt > 0) 1178 1138 bpf_link__destroy(links[--links_cnt]);