powerpc64/bpf: fix handling of BPF stack in exception callback

Exception callback reuses the stack frame of exception boundary. When
exception boundary and exception callback programs have different BPF
stack depth, the current stack unwind in exception callback will fail.
Adjust the stack frame size of exception callback, in its prologue,
if its BPF stack depth is different from that of exception boundary.

Reported-by: bot+bpf-ci@kernel.org
Closes: https://lore.kernel.org/bpf/2a310e86a59eb4c44c3ac9e5647814469d9c955580c9c0f1b3d9ca4a44717a34@mail.kernel.org/
Fixes: 11d45eee9f42 ("powerpc64/bpf: Additional NVR handling for bpf_throw")
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260303181031.390073-6-hbathini@linux.ibm.com

authored by

Hari Bathini and committed by

Madhavan Srinivasan 2 months ago 51b8de4b 2d347d10

+25 -11

1 changed file

expand all

arch

powerpc

net

bpf_jit_comp64.c

+25 -11

arch/powerpc/net/bpf_jit_comp64.c

··· 32 32 * 33 33 * [ prev sp ] <------------- 34 34 * [ tail_call_info ] 8 | 35 - * [ nv gpr save area ] 6*8 + (12*8) | 35 + * [ nv gpr save area ] (6 * 8) | 36 + * [ addl. nv gpr save area] (12 * 8) | <--- exception boundary/callback program 36 37 * [ local_tmp_var ] 24 | 37 38 * fp (r31) --> [ ebpf stack space ] upto 512 | 38 39 * [ frame header ] 32/112 | 39 40 * sp (r1) ---> [ stack pointer ] -------------- 40 41 * 41 - * Additional (12*8) in 'nv gpr save area' only in case of 42 - * exception boundary. 42 + * Additional (12 * 8) in 'nv gpr save area' only in case of 43 + * exception boundary/callback. 43 44 */ 44 45 45 46 /* BPF non-volatile registers save area size */ ··· 52 51 * for additional non volatile registers(r14-r25) to be saved 53 52 * at exception boundary 54 53 */ 55 - #define BPF_PPC_EXC_STACK_SAVE (12*8) 54 + #define BPF_PPC_EXC_STACK_SAVE (12 * 8) 56 55 57 56 /* stack frame excluding BPF stack, ensure this is quadword aligned */ 58 57 #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ ··· 129 128 * [ ... ] | 130 129 * sp (r1) ---> [ stack pointer ] -------------- 131 130 * [ tail_call_info ] 8 132 - * [ nv gpr save area ] 6*8 + (12*8) 131 + * [ nv gpr save area ] (6 * 8) 132 + * [ addl. nv gpr save area] (12 * 8) <--- exception boundary/callback program 133 133 * [ local_tmp_var ] 24 134 134 * [ unused red zone ] 224 135 135 * 136 - * Additional (12*8) in 'nv gpr save area' only in case of 137 - * exception boundary. 136 + * Additional (12 * 8) in 'nv gpr save area' only in case of 137 + * exception boundary/callback. 138 138 */ 139 139 static int bpf_jit_stack_local(struct codegen_context *ctx) 140 140 { ··· 242 240 243 241 if (bpf_has_stack_frame(ctx) && !ctx->exception_cb) { 244 242 /* 245 - * exception_cb uses boundary frame after stack walk. 246 - * It can simply use redzone, this optimization reduces 247 - * stack walk loop by one level. 248 - * 249 243 * We need a stack frame, but we don't necessarily need to 250 244 * save/restore LR unless we call other functions 251 245 */ ··· 285 287 * program(main prog) as third arg 286 288 */ 287 289 EMIT(PPC_RAW_MR(_R1, _R5)); 290 + /* 291 + * Exception callback reuses the stack frame of exception boundary. 292 + * But BPF stack depth of exception callback and exception boundary 293 + * don't have to be same. If BPF stack depth is different, adjust the 294 + * stack frame size considering BPF stack depth of exception callback. 295 + * The non-volatile register save area remains unchanged. These non- 296 + * volatile registers are restored in exception callback's epilogue. 297 + */ 298 + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R5, 0)); 299 + EMIT(PPC_RAW_SUB(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_1), _R1)); 300 + EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_2), 301 + -BPF_PPC_EXC_STACKFRAME)); 302 + EMIT(PPC_RAW_CMPLDI(bpf_to_ppc(TMP_REG_2), ctx->stack_size)); 303 + PPC_BCC_CONST_SHORT(COND_EQ, 12); 304 + EMIT(PPC_RAW_MR(_R1, bpf_to_ppc(TMP_REG_1))); 305 + EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_EXC_STACKFRAME + ctx->stack_size))); 288 306 } 289 307 290 308 /*

Configure Feed

Configure Feed