Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Merge branch 'bpf-register-bounds-logic-and-testing-improvements'

Andrii Nakryiko says:

====================
BPF register bounds logic and testing improvements

This patch set adds a big set of manual and auto-generated test cases
validating BPF verifier's register bounds tracking and deduction logic. See
details in the last patch.

We start with building a tester that validates existing <range> vs <scalar>
verifier logic for range bounds. To make all this work, BPF verifier's logic
needed a bunch of improvements to handle some cases that previously were not
covered. This had no implications as to correctness of verifier logic, but it
was incomplete enough to cause significant disagreements with alternative
implementation of register bounds logic that tests in this patch set
implement. So we need BPF verifier logic improvements to make all the tests
pass. This is what we do in patches #3 through #9.

The end goal of this work, though, is to extend BPF verifier range state
tracking such as to allow to derive new range bounds when comparing non-const
registers. There is some more investigative work required to investigate and
fix existing potential issues with range tracking as part of ALU/ALU64
operations, so <range> x <range> part of v5 patch set ([0]) is dropped until
these issues are sorted out.

For now, we include preparatory refactorings and clean ups, that set up BPF
verifier code base to extend the logic to <range> vs <range> logic in
subsequent patch set. Patches #10-#16 perform preliminary refactorings without
functionally changing anything. But they do clean up check_cond_jmp_op() logic
and generalize a bunch of other pieces in is_branch_taken() logic.

[0] https://patchwork.kernel.org/project/netdevbpf/list/?series=797178&state=*

v5->v6:
- dropped <range> vs <range> patches (original patches #18 through #23) to
add more register range sanity checks and fix preexisting issues;
- comments improvements, addressing other feedback on first 17 patches
(Eduard, Alexei);
v4->v5:
- added entirety of verifier reg bounds tracking changes, now handling
<range> vs <range> cases (Alexei);
- added way more comments trying to explain why deductions added are
correct, hopefully they are useful and clarify things a bit (Daniel,
Shung-Hsi);
- added two preliminary selftests fixes necessary for RELEASE=1 build to
work again, it keeps breaking.
v3->v4:
- improvements to reg_bounds tester (progress report, split 32-bit and
64-bit ranges, fix various verbosity output issues, etc);
v2->v3:
- fix a subtle little-endianness assumption inside parge_reg_state() (CI);
v1->v2:
- fix compilation when building selftests with llvm-16 toolchain (CI).
====================

Link: https://lore.kernel.org/r/20231102033759.2541186-1-andrii@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>

+440 -389
+438 -384
kernel/bpf/verifier.c
··· 2324 2324 /* Uses signed min/max values to inform unsigned, and vice-versa */ 2325 2325 static void __reg32_deduce_bounds(struct bpf_reg_state *reg) 2326 2326 { 2327 + /* If upper 32 bits of u64/s64 range don't change, we can use lower 32 2328 + * bits to improve our u32/s32 boundaries. 2329 + * 2330 + * E.g., the case where we have upper 32 bits as zero ([10, 20] in 2331 + * u64) is pretty trivial, it's obvious that in u32 we'll also have 2332 + * [10, 20] range. But this property holds for any 64-bit range as 2333 + * long as upper 32 bits in that entire range of values stay the same. 2334 + * 2335 + * E.g., u64 range [0x10000000A, 0x10000000F] ([4294967306, 4294967311] 2336 + * in decimal) has the same upper 32 bits throughout all the values in 2337 + * that range. As such, lower 32 bits form a valid [0xA, 0xF] ([10, 15]) 2338 + * range. 2339 + * 2340 + * Note also, that [0xA, 0xF] is a valid range both in u32 and in s32, 2341 + * following the rules outlined below about u64/s64 correspondence 2342 + * (which equally applies to u32 vs s32 correspondence). In general it 2343 + * depends on actual hexadecimal values of 32-bit range. They can form 2344 + * only valid u32, or only valid s32 ranges in some cases. 2345 + * 2346 + * So we use all these insights to derive bounds for subregisters here. 2347 + */ 2348 + if ((reg->umin_value >> 32) == (reg->umax_value >> 32)) { 2349 + /* u64 to u32 casting preserves validity of low 32 bits as 2350 + * a range, if upper 32 bits are the same 2351 + */ 2352 + reg->u32_min_value = max_t(u32, reg->u32_min_value, (u32)reg->umin_value); 2353 + reg->u32_max_value = min_t(u32, reg->u32_max_value, (u32)reg->umax_value); 2354 + 2355 + if ((s32)reg->umin_value <= (s32)reg->umax_value) { 2356 + reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->umin_value); 2357 + reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->umax_value); 2358 + } 2359 + } 2360 + if ((reg->smin_value >> 32) == (reg->smax_value >> 32)) { 2361 + /* low 32 bits should form a proper u32 range */ 2362 + if ((u32)reg->smin_value <= (u32)reg->smax_value) { 2363 + reg->u32_min_value = max_t(u32, reg->u32_min_value, (u32)reg->smin_value); 2364 + reg->u32_max_value = min_t(u32, reg->u32_max_value, (u32)reg->smax_value); 2365 + } 2366 + /* low 32 bits should form a proper s32 range */ 2367 + if ((s32)reg->smin_value <= (s32)reg->smax_value) { 2368 + reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->smin_value); 2369 + reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->smax_value); 2370 + } 2371 + } 2372 + /* Special case where upper bits form a small sequence of two 2373 + * sequential numbers (in 32-bit unsigned space, so 0xffffffff to 2374 + * 0x00000000 is also valid), while lower bits form a proper s32 range 2375 + * going from negative numbers to positive numbers. E.g., let's say we 2376 + * have s64 range [-1, 1] ([0xffffffffffffffff, 0x0000000000000001]). 2377 + * Possible s64 values are {-1, 0, 1} ({0xffffffffffffffff, 2378 + * 0x0000000000000000, 0x00000000000001}). Ignoring upper 32 bits, 2379 + * we still get a valid s32 range [-1, 1] ([0xffffffff, 0x00000001]). 2380 + * Note that it doesn't have to be 0xffffffff going to 0x00000000 in 2381 + * upper 32 bits. As a random example, s64 range 2382 + * [0xfffffff0fffffff0; 0xfffffff100000010], forms a valid s32 range 2383 + * [-16, 16] ([0xfffffff0; 0x00000010]) in its 32 bit subregister. 2384 + */ 2385 + if ((u32)(reg->umin_value >> 32) + 1 == (u32)(reg->umax_value >> 32) && 2386 + (s32)reg->umin_value < 0 && (s32)reg->umax_value >= 0) { 2387 + reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->umin_value); 2388 + reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->umax_value); 2389 + } 2390 + if ((u32)(reg->smin_value >> 32) + 1 == (u32)(reg->smax_value >> 32) && 2391 + (s32)reg->smin_value < 0 && (s32)reg->smax_value >= 0) { 2392 + reg->s32_min_value = max_t(s32, reg->s32_min_value, (s32)reg->smin_value); 2393 + reg->s32_max_value = min_t(s32, reg->s32_max_value, (s32)reg->smax_value); 2394 + } 2395 + /* if u32 range forms a valid s32 range (due to matching sign bit), 2396 + * try to learn from that 2397 + */ 2398 + if ((s32)reg->u32_min_value <= (s32)reg->u32_max_value) { 2399 + reg->s32_min_value = max_t(s32, reg->s32_min_value, reg->u32_min_value); 2400 + reg->s32_max_value = min_t(s32, reg->s32_max_value, reg->u32_max_value); 2401 + } 2327 2402 /* Learn sign from signed bounds. 2328 2403 * If we cannot cross the sign boundary, then signed and unsigned bounds 2329 2404 * are the same, so combine. This works even in the negative case, e.g. ··· 2433 2358 2434 2359 static void __reg64_deduce_bounds(struct bpf_reg_state *reg) 2435 2360 { 2361 + /* If u64 range forms a valid s64 range (due to matching sign bit), 2362 + * try to learn from that. Let's do a bit of ASCII art to see when 2363 + * this is happening. Let's take u64 range first: 2364 + * 2365 + * 0 0x7fffffffffffffff 0x8000000000000000 U64_MAX 2366 + * |-------------------------------|--------------------------------| 2367 + * 2368 + * Valid u64 range is formed when umin and umax are anywhere in the 2369 + * range [0, U64_MAX], and umin <= umax. u64 case is simple and 2370 + * straightforward. Let's see how s64 range maps onto the same range 2371 + * of values, annotated below the line for comparison: 2372 + * 2373 + * 0 0x7fffffffffffffff 0x8000000000000000 U64_MAX 2374 + * |-------------------------------|--------------------------------| 2375 + * 0 S64_MAX S64_MIN -1 2376 + * 2377 + * So s64 values basically start in the middle and they are logically 2378 + * contiguous to the right of it, wrapping around from -1 to 0, and 2379 + * then finishing as S64_MAX (0x7fffffffffffffff) right before 2380 + * S64_MIN. We can try drawing the continuity of u64 vs s64 values 2381 + * more visually as mapped to sign-agnostic range of hex values. 2382 + * 2383 + * u64 start u64 end 2384 + * _______________________________________________________________ 2385 + * / \ 2386 + * 0 0x7fffffffffffffff 0x8000000000000000 U64_MAX 2387 + * |-------------------------------|--------------------------------| 2388 + * 0 S64_MAX S64_MIN -1 2389 + * / \ 2390 + * >------------------------------ -------------------------------> 2391 + * s64 continues... s64 end s64 start s64 "midpoint" 2392 + * 2393 + * What this means is that, in general, we can't always derive 2394 + * something new about u64 from any random s64 range, and vice versa. 2395 + * 2396 + * But we can do that in two particular cases. One is when entire 2397 + * u64/s64 range is *entirely* contained within left half of the above 2398 + * diagram or when it is *entirely* contained in the right half. I.e.: 2399 + * 2400 + * |-------------------------------|--------------------------------| 2401 + * ^ ^ ^ ^ 2402 + * A B C D 2403 + * 2404 + * [A, B] and [C, D] are contained entirely in their respective halves 2405 + * and form valid contiguous ranges as both u64 and s64 values. [A, B] 2406 + * will be non-negative both as u64 and s64 (and in fact it will be 2407 + * identical ranges no matter the signedness). [C, D] treated as s64 2408 + * will be a range of negative values, while in u64 it will be 2409 + * non-negative range of values larger than 0x8000000000000000. 2410 + * 2411 + * Now, any other range here can't be represented in both u64 and s64 2412 + * simultaneously. E.g., [A, C], [A, D], [B, C], [B, D] are valid 2413 + * contiguous u64 ranges, but they are discontinuous in s64. [B, C] 2414 + * in s64 would be properly presented as [S64_MIN, C] and [B, S64_MAX], 2415 + * for example. Similarly, valid s64 range [D, A] (going from negative 2416 + * to positive values), would be two separate [D, U64_MAX] and [0, A] 2417 + * ranges as u64. Currently reg_state can't represent two segments per 2418 + * numeric domain, so in such situations we can only derive maximal 2419 + * possible range ([0, U64_MAX] for u64, and [S64_MIN, S64_MAX] for s64). 2420 + * 2421 + * So we use these facts to derive umin/umax from smin/smax and vice 2422 + * versa only if they stay within the same "half". This is equivalent 2423 + * to checking sign bit: lower half will have sign bit as zero, upper 2424 + * half have sign bit 1. Below in code we simplify this by just 2425 + * casting umin/umax as smin/smax and checking if they form valid 2426 + * range, and vice versa. Those are equivalent checks. 2427 + */ 2428 + if ((s64)reg->umin_value <= (s64)reg->umax_value) { 2429 + reg->smin_value = max_t(s64, reg->smin_value, reg->umin_value); 2430 + reg->smax_value = min_t(s64, reg->smax_value, reg->umax_value); 2431 + } 2436 2432 /* Learn sign from signed bounds. 2437 2433 * If we cannot cross the sign boundary, then signed and unsigned bounds 2438 2434 * are the same, so combine. This works even in the negative case, e.g. ··· 2536 2390 } 2537 2391 } 2538 2392 2393 + static void __reg_deduce_mixed_bounds(struct bpf_reg_state *reg) 2394 + { 2395 + /* Try to tighten 64-bit bounds from 32-bit knowledge, using 32-bit 2396 + * values on both sides of 64-bit range in hope to have tigher range. 2397 + * E.g., if r1 is [0x1'00000000, 0x3'80000000], and we learn from 2398 + * 32-bit signed > 0 operation that s32 bounds are now [1; 0x7fffffff]. 2399 + * With this, we can substitute 1 as low 32-bits of _low_ 64-bit bound 2400 + * (0x100000000 -> 0x100000001) and 0x7fffffff as low 32-bits of 2401 + * _high_ 64-bit bound (0x380000000 -> 0x37fffffff) and arrive at a 2402 + * better overall bounds for r1 as [0x1'000000001; 0x3'7fffffff]. 2403 + * We just need to make sure that derived bounds we are intersecting 2404 + * with are well-formed ranges in respecitve s64 or u64 domain, just 2405 + * like we do with similar kinds of 32-to-64 or 64-to-32 adjustments. 2406 + */ 2407 + __u64 new_umin, new_umax; 2408 + __s64 new_smin, new_smax; 2409 + 2410 + /* u32 -> u64 tightening, it's always well-formed */ 2411 + new_umin = (reg->umin_value & ~0xffffffffULL) | reg->u32_min_value; 2412 + new_umax = (reg->umax_value & ~0xffffffffULL) | reg->u32_max_value; 2413 + reg->umin_value = max_t(u64, reg->umin_value, new_umin); 2414 + reg->umax_value = min_t(u64, reg->umax_value, new_umax); 2415 + /* u32 -> s64 tightening, u32 range embedded into s64 preserves range validity */ 2416 + new_smin = (reg->smin_value & ~0xffffffffULL) | reg->u32_min_value; 2417 + new_smax = (reg->smax_value & ~0xffffffffULL) | reg->u32_max_value; 2418 + reg->smin_value = max_t(s64, reg->smin_value, new_smin); 2419 + reg->smax_value = min_t(s64, reg->smax_value, new_smax); 2420 + 2421 + /* if s32 can be treated as valid u32 range, we can use it as well */ 2422 + if ((u32)reg->s32_min_value <= (u32)reg->s32_max_value) { 2423 + /* s32 -> u64 tightening */ 2424 + new_umin = (reg->umin_value & ~0xffffffffULL) | (u32)reg->s32_min_value; 2425 + new_umax = (reg->umax_value & ~0xffffffffULL) | (u32)reg->s32_max_value; 2426 + reg->umin_value = max_t(u64, reg->umin_value, new_umin); 2427 + reg->umax_value = min_t(u64, reg->umax_value, new_umax); 2428 + /* s32 -> s64 tightening */ 2429 + new_smin = (reg->smin_value & ~0xffffffffULL) | (u32)reg->s32_min_value; 2430 + new_smax = (reg->smax_value & ~0xffffffffULL) | (u32)reg->s32_max_value; 2431 + reg->smin_value = max_t(s64, reg->smin_value, new_smin); 2432 + reg->smax_value = min_t(s64, reg->smax_value, new_smax); 2433 + } 2434 + } 2435 + 2539 2436 static void __reg_deduce_bounds(struct bpf_reg_state *reg) 2540 2437 { 2541 2438 __reg32_deduce_bounds(reg); 2542 2439 __reg64_deduce_bounds(reg); 2440 + __reg_deduce_mixed_bounds(reg); 2543 2441 } 2544 2442 2545 2443 /* Attempts to improve var_off based on unsigned min/max information */ ··· 2604 2414 /* We might have learned new bounds from the var_off. */ 2605 2415 __update_reg_bounds(reg); 2606 2416 /* We might have learned something about the sign bit. */ 2417 + __reg_deduce_bounds(reg); 2607 2418 __reg_deduce_bounds(reg); 2608 2419 /* We might have learned some bits from the bounds. */ 2609 2420 __reg_bound_offset(reg); ··· 2637 2446 reg->smin_value = 0; 2638 2447 reg->smax_value = U32_MAX; 2639 2448 } 2640 - } 2641 - 2642 - static void __reg_combine_32_into_64(struct bpf_reg_state *reg) 2643 - { 2644 - /* special case when 64-bit register has upper 32-bit register 2645 - * zeroed. Typically happens after zext or <<32, >>32 sequence 2646 - * allowing us to use 32-bit bounds directly, 2647 - */ 2648 - if (tnum_equals_const(tnum_clear_subreg(reg->var_off), 0)) { 2649 - __reg_assign_32_into_64(reg); 2650 - } else { 2651 - /* Otherwise the best we can do is push lower 32bit known and 2652 - * unknown bits into register (var_off set from jmp logic) 2653 - * then learn as much as possible from the 64-bit tnum 2654 - * known and unknown bits. The previous smin/smax bounds are 2655 - * invalid here because of jmp32 compare so mark them unknown 2656 - * so they do not impact tnum bounds calculation. 2657 - */ 2658 - __mark_reg64_unbounded(reg); 2659 - } 2660 - reg_bounds_sync(reg); 2661 - } 2662 - 2663 - static bool __reg64_bound_s32(s64 a) 2664 - { 2665 - return a >= S32_MIN && a <= S32_MAX; 2666 - } 2667 - 2668 - static bool __reg64_bound_u32(u64 a) 2669 - { 2670 - return a >= U32_MIN && a <= U32_MAX; 2671 - } 2672 - 2673 - static void __reg_combine_64_into_32(struct bpf_reg_state *reg) 2674 - { 2675 - __mark_reg32_unbounded(reg); 2676 - if (__reg64_bound_s32(reg->smin_value) && __reg64_bound_s32(reg->smax_value)) { 2677 - reg->s32_min_value = (s32)reg->smin_value; 2678 - reg->s32_max_value = (s32)reg->smax_value; 2679 - } 2680 - if (__reg64_bound_u32(reg->umin_value) && __reg64_bound_u32(reg->umax_value)) { 2681 - reg->u32_min_value = (u32)reg->umin_value; 2682 - reg->u32_max_value = (u32)reg->umax_value; 2683 - } 2684 - reg_bounds_sync(reg); 2685 2449 } 2686 2450 2687 2451 /* Mark a register as having a completely unknown (scalar) value. */ ··· 6342 6196 * values are also truncated so we push 64-bit bounds into 6343 6197 * 32-bit bounds. Above were truncated < 32-bits already. 6344 6198 */ 6345 - if (size >= 4) 6346 - return; 6347 - __reg_combine_64_into_32(reg); 6199 + if (size < 4) { 6200 + __mark_reg32_unbounded(reg); 6201 + reg_bounds_sync(reg); 6202 + } 6348 6203 } 6349 6204 6350 6205 static void set_sext64_default_val(struct bpf_reg_state *reg, int size) ··· 14188 14041 })); 14189 14042 } 14190 14043 14191 - static int is_branch32_taken(struct bpf_reg_state *reg, u32 val, u8 opcode) 14044 + /* check if register is a constant scalar value */ 14045 + static bool is_reg_const(struct bpf_reg_state *reg, bool subreg32) 14192 14046 { 14193 - struct tnum subreg = tnum_subreg(reg->var_off); 14194 - s32 sval = (s32)val; 14195 - 14196 - switch (opcode) { 14197 - case BPF_JEQ: 14198 - if (tnum_is_const(subreg)) 14199 - return !!tnum_equals_const(subreg, val); 14200 - else if (val < reg->u32_min_value || val > reg->u32_max_value) 14201 - return 0; 14202 - else if (sval < reg->s32_min_value || sval > reg->s32_max_value) 14203 - return 0; 14204 - break; 14205 - case BPF_JNE: 14206 - if (tnum_is_const(subreg)) 14207 - return !tnum_equals_const(subreg, val); 14208 - else if (val < reg->u32_min_value || val > reg->u32_max_value) 14209 - return 1; 14210 - else if (sval < reg->s32_min_value || sval > reg->s32_max_value) 14211 - return 1; 14212 - break; 14213 - case BPF_JSET: 14214 - if ((~subreg.mask & subreg.value) & val) 14215 - return 1; 14216 - if (!((subreg.mask | subreg.value) & val)) 14217 - return 0; 14218 - break; 14219 - case BPF_JGT: 14220 - if (reg->u32_min_value > val) 14221 - return 1; 14222 - else if (reg->u32_max_value <= val) 14223 - return 0; 14224 - break; 14225 - case BPF_JSGT: 14226 - if (reg->s32_min_value > sval) 14227 - return 1; 14228 - else if (reg->s32_max_value <= sval) 14229 - return 0; 14230 - break; 14231 - case BPF_JLT: 14232 - if (reg->u32_max_value < val) 14233 - return 1; 14234 - else if (reg->u32_min_value >= val) 14235 - return 0; 14236 - break; 14237 - case BPF_JSLT: 14238 - if (reg->s32_max_value < sval) 14239 - return 1; 14240 - else if (reg->s32_min_value >= sval) 14241 - return 0; 14242 - break; 14243 - case BPF_JGE: 14244 - if (reg->u32_min_value >= val) 14245 - return 1; 14246 - else if (reg->u32_max_value < val) 14247 - return 0; 14248 - break; 14249 - case BPF_JSGE: 14250 - if (reg->s32_min_value >= sval) 14251 - return 1; 14252 - else if (reg->s32_max_value < sval) 14253 - return 0; 14254 - break; 14255 - case BPF_JLE: 14256 - if (reg->u32_max_value <= val) 14257 - return 1; 14258 - else if (reg->u32_min_value > val) 14259 - return 0; 14260 - break; 14261 - case BPF_JSLE: 14262 - if (reg->s32_max_value <= sval) 14263 - return 1; 14264 - else if (reg->s32_min_value > sval) 14265 - return 0; 14266 - break; 14267 - } 14268 - 14269 - return -1; 14047 + return reg->type == SCALAR_VALUE && 14048 + tnum_is_const(subreg32 ? tnum_subreg(reg->var_off) : reg->var_off); 14270 14049 } 14271 14050 14272 - 14273 - static int is_branch64_taken(struct bpf_reg_state *reg, u64 val, u8 opcode) 14051 + /* assuming is_reg_const() is true, return constant value of a register */ 14052 + static u64 reg_const_value(struct bpf_reg_state *reg, bool subreg32) 14274 14053 { 14275 - s64 sval = (s64)val; 14276 - 14277 - switch (opcode) { 14278 - case BPF_JEQ: 14279 - if (tnum_is_const(reg->var_off)) 14280 - return !!tnum_equals_const(reg->var_off, val); 14281 - else if (val < reg->umin_value || val > reg->umax_value) 14282 - return 0; 14283 - else if (sval < reg->smin_value || sval > reg->smax_value) 14284 - return 0; 14285 - break; 14286 - case BPF_JNE: 14287 - if (tnum_is_const(reg->var_off)) 14288 - return !tnum_equals_const(reg->var_off, val); 14289 - else if (val < reg->umin_value || val > reg->umax_value) 14290 - return 1; 14291 - else if (sval < reg->smin_value || sval > reg->smax_value) 14292 - return 1; 14293 - break; 14294 - case BPF_JSET: 14295 - if ((~reg->var_off.mask & reg->var_off.value) & val) 14296 - return 1; 14297 - if (!((reg->var_off.mask | reg->var_off.value) & val)) 14298 - return 0; 14299 - break; 14300 - case BPF_JGT: 14301 - if (reg->umin_value > val) 14302 - return 1; 14303 - else if (reg->umax_value <= val) 14304 - return 0; 14305 - break; 14306 - case BPF_JSGT: 14307 - if (reg->smin_value > sval) 14308 - return 1; 14309 - else if (reg->smax_value <= sval) 14310 - return 0; 14311 - break; 14312 - case BPF_JLT: 14313 - if (reg->umax_value < val) 14314 - return 1; 14315 - else if (reg->umin_value >= val) 14316 - return 0; 14317 - break; 14318 - case BPF_JSLT: 14319 - if (reg->smax_value < sval) 14320 - return 1; 14321 - else if (reg->smin_value >= sval) 14322 - return 0; 14323 - break; 14324 - case BPF_JGE: 14325 - if (reg->umin_value >= val) 14326 - return 1; 14327 - else if (reg->umax_value < val) 14328 - return 0; 14329 - break; 14330 - case BPF_JSGE: 14331 - if (reg->smin_value >= sval) 14332 - return 1; 14333 - else if (reg->smax_value < sval) 14334 - return 0; 14335 - break; 14336 - case BPF_JLE: 14337 - if (reg->umax_value <= val) 14338 - return 1; 14339 - else if (reg->umin_value > val) 14340 - return 0; 14341 - break; 14342 - case BPF_JSLE: 14343 - if (reg->smax_value <= sval) 14344 - return 1; 14345 - else if (reg->smin_value > sval) 14346 - return 0; 14347 - break; 14348 - } 14349 - 14350 - return -1; 14054 + return subreg32 ? tnum_subreg(reg->var_off).value : reg->var_off.value; 14351 14055 } 14352 14056 14353 - /* compute branch direction of the expression "if (reg opcode val) goto target;" 14354 - * and return: 14355 - * 1 - branch will be taken and "goto target" will be executed 14356 - * 0 - branch will not be taken and fall-through to next insn 14357 - * -1 - unknown. Example: "if (reg < 5)" is unknown when register value 14358 - * range [0,10] 14057 + /* 14058 + * <reg1> <op> <reg2>, currently assuming reg2 is a constant 14359 14059 */ 14360 - static int is_branch_taken(struct bpf_reg_state *reg, u64 val, u8 opcode, 14361 - bool is_jmp32) 14060 + static int is_scalar_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, 14061 + u8 opcode, bool is_jmp32) 14362 14062 { 14363 - if (__is_pointer_value(false, reg)) { 14364 - if (!reg_not_null(reg)) 14365 - return -1; 14063 + struct tnum t1 = is_jmp32 ? tnum_subreg(reg1->var_off) : reg1->var_off; 14064 + u64 umin1 = is_jmp32 ? (u64)reg1->u32_min_value : reg1->umin_value; 14065 + u64 umax1 = is_jmp32 ? (u64)reg1->u32_max_value : reg1->umax_value; 14066 + s64 smin1 = is_jmp32 ? (s64)reg1->s32_min_value : reg1->smin_value; 14067 + s64 smax1 = is_jmp32 ? (s64)reg1->s32_max_value : reg1->smax_value; 14068 + u64 uval = is_jmp32 ? (u32)tnum_subreg(reg2->var_off).value : reg2->var_off.value; 14069 + s64 sval = is_jmp32 ? (s32)uval : (s64)uval; 14366 14070 14367 - /* If pointer is valid tests against zero will fail so we can 14368 - * use this to direct branch taken. 14369 - */ 14370 - if (val != 0) 14371 - return -1; 14372 - 14373 - switch (opcode) { 14374 - case BPF_JEQ: 14071 + switch (opcode) { 14072 + case BPF_JEQ: 14073 + if (tnum_is_const(t1)) 14074 + return !!tnum_equals_const(t1, uval); 14075 + else if (uval < umin1 || uval > umax1) 14375 14076 return 0; 14376 - case BPF_JNE: 14077 + else if (sval < smin1 || sval > smax1) 14078 + return 0; 14079 + break; 14080 + case BPF_JNE: 14081 + if (tnum_is_const(t1)) 14082 + return !tnum_equals_const(t1, uval); 14083 + else if (uval < umin1 || uval > umax1) 14377 14084 return 1; 14378 - default: 14379 - return -1; 14380 - } 14085 + else if (sval < smin1 || sval > smax1) 14086 + return 1; 14087 + break; 14088 + case BPF_JSET: 14089 + if ((~t1.mask & t1.value) & uval) 14090 + return 1; 14091 + if (!((t1.mask | t1.value) & uval)) 14092 + return 0; 14093 + break; 14094 + case BPF_JGT: 14095 + if (umin1 > uval ) 14096 + return 1; 14097 + else if (umax1 <= uval) 14098 + return 0; 14099 + break; 14100 + case BPF_JSGT: 14101 + if (smin1 > sval) 14102 + return 1; 14103 + else if (smax1 <= sval) 14104 + return 0; 14105 + break; 14106 + case BPF_JLT: 14107 + if (umax1 < uval) 14108 + return 1; 14109 + else if (umin1 >= uval) 14110 + return 0; 14111 + break; 14112 + case BPF_JSLT: 14113 + if (smax1 < sval) 14114 + return 1; 14115 + else if (smin1 >= sval) 14116 + return 0; 14117 + break; 14118 + case BPF_JGE: 14119 + if (umin1 >= uval) 14120 + return 1; 14121 + else if (umax1 < uval) 14122 + return 0; 14123 + break; 14124 + case BPF_JSGE: 14125 + if (smin1 >= sval) 14126 + return 1; 14127 + else if (smax1 < sval) 14128 + return 0; 14129 + break; 14130 + case BPF_JLE: 14131 + if (umax1 <= uval) 14132 + return 1; 14133 + else if (umin1 > uval) 14134 + return 0; 14135 + break; 14136 + case BPF_JSLE: 14137 + if (smax1 <= sval) 14138 + return 1; 14139 + else if (smin1 > sval) 14140 + return 0; 14141 + break; 14381 14142 } 14382 14143 14383 - if (is_jmp32) 14384 - return is_branch32_taken(reg, val, opcode); 14385 - return is_branch64_taken(reg, val, opcode); 14144 + return -1; 14386 14145 } 14387 14146 14388 14147 static int flip_opcode(u32 opcode) ··· 14352 14299 return -1; 14353 14300 } 14354 14301 14355 - /* Adjusts the register min/max values in the case that the dst_reg is the 14356 - * variable register that we are working on, and src_reg is a constant or we're 14357 - * simply doing a BPF_K check. 14358 - * In JEQ/JNE cases we also adjust the var_off values. 14302 + /* compute branch direction of the expression "if (<reg1> opcode <reg2>) goto target;" 14303 + * and return: 14304 + * 1 - branch will be taken and "goto target" will be executed 14305 + * 0 - branch will not be taken and fall-through to next insn 14306 + * -1 - unknown. Example: "if (reg1 < 5)" is unknown when register value 14307 + * range [0,10] 14359 14308 */ 14360 - static void reg_set_min_max(struct bpf_reg_state *true_reg, 14361 - struct bpf_reg_state *false_reg, 14362 - u64 val, u32 val32, 14309 + static int is_branch_taken(struct bpf_reg_state *reg1, struct bpf_reg_state *reg2, 14310 + u8 opcode, bool is_jmp32) 14311 + { 14312 + u64 val; 14313 + 14314 + if (reg_is_pkt_pointer_any(reg1) && reg_is_pkt_pointer_any(reg2) && !is_jmp32) 14315 + return is_pkt_ptr_branch_taken(reg1, reg2, opcode); 14316 + 14317 + /* try to make sure reg2 is a constant SCALAR_VALUE */ 14318 + if (!is_reg_const(reg2, is_jmp32)) { 14319 + opcode = flip_opcode(opcode); 14320 + swap(reg1, reg2); 14321 + } 14322 + /* for now we expect reg2 to be a constant to make any useful decisions */ 14323 + if (!is_reg_const(reg2, is_jmp32)) 14324 + return -1; 14325 + val = reg_const_value(reg2, is_jmp32); 14326 + 14327 + if (__is_pointer_value(false, reg1)) { 14328 + if (!reg_not_null(reg1)) 14329 + return -1; 14330 + 14331 + /* If pointer is valid tests against zero will fail so we can 14332 + * use this to direct branch taken. 14333 + */ 14334 + if (val != 0) 14335 + return -1; 14336 + 14337 + switch (opcode) { 14338 + case BPF_JEQ: 14339 + return 0; 14340 + case BPF_JNE: 14341 + return 1; 14342 + default: 14343 + return -1; 14344 + } 14345 + } 14346 + 14347 + return is_scalar_branch_taken(reg1, reg2, opcode, is_jmp32); 14348 + } 14349 + 14350 + /* Adjusts the register min/max values in the case that the dst_reg and 14351 + * src_reg are both SCALAR_VALUE registers (or we are simply doing a BPF_K 14352 + * check, in which case we havea fake SCALAR_VALUE representing insn->imm). 14353 + * Technically we can do similar adjustments for pointers to the same object, 14354 + * but we don't support that right now. 14355 + */ 14356 + static void reg_set_min_max(struct bpf_reg_state *true_reg1, 14357 + struct bpf_reg_state *true_reg2, 14358 + struct bpf_reg_state *false_reg1, 14359 + struct bpf_reg_state *false_reg2, 14363 14360 u8 opcode, bool is_jmp32) 14364 14361 { 14365 - struct tnum false_32off = tnum_subreg(false_reg->var_off); 14366 - struct tnum false_64off = false_reg->var_off; 14367 - struct tnum true_32off = tnum_subreg(true_reg->var_off); 14368 - struct tnum true_64off = true_reg->var_off; 14369 - s64 sval = (s64)val; 14370 - s32 sval32 = (s32)val32; 14362 + struct tnum false_32off, false_64off; 14363 + struct tnum true_32off, true_64off; 14364 + u64 uval; 14365 + u32 uval32; 14366 + s64 sval; 14367 + s32 sval32; 14371 14368 14372 - /* If the dst_reg is a pointer, we can't learn anything about its 14373 - * variable offset from the compare (unless src_reg were a pointer into 14374 - * the same object, but we don't bother with that. 14375 - * Since false_reg and true_reg have the same type by construction, we 14376 - * only need to check one of them for pointerness. 14369 + /* If either register is a pointer, we can't learn anything about its 14370 + * variable offset from the compare (unless they were a pointer into 14371 + * the same object, but we don't bother with that). 14377 14372 */ 14378 - if (__is_pointer_value(false, false_reg)) 14373 + if (false_reg1->type != SCALAR_VALUE || false_reg2->type != SCALAR_VALUE) 14379 14374 return; 14375 + 14376 + /* we expect right-hand registers (src ones) to be constants, for now */ 14377 + if (!is_reg_const(false_reg2, is_jmp32)) { 14378 + opcode = flip_opcode(opcode); 14379 + swap(true_reg1, true_reg2); 14380 + swap(false_reg1, false_reg2); 14381 + } 14382 + if (!is_reg_const(false_reg2, is_jmp32)) 14383 + return; 14384 + 14385 + false_32off = tnum_subreg(false_reg1->var_off); 14386 + false_64off = false_reg1->var_off; 14387 + true_32off = tnum_subreg(true_reg1->var_off); 14388 + true_64off = true_reg1->var_off; 14389 + uval = false_reg2->var_off.value; 14390 + uval32 = (u32)tnum_subreg(false_reg2->var_off).value; 14391 + sval = (s64)uval; 14392 + sval32 = (s32)uval32; 14380 14393 14381 14394 switch (opcode) { 14382 14395 /* JEQ/JNE comparison doesn't change the register equivalence. ··· 14456 14337 */ 14457 14338 case BPF_JEQ: 14458 14339 if (is_jmp32) { 14459 - __mark_reg32_known(true_reg, val32); 14460 - true_32off = tnum_subreg(true_reg->var_off); 14340 + __mark_reg32_known(true_reg1, uval32); 14341 + true_32off = tnum_subreg(true_reg1->var_off); 14461 14342 } else { 14462 - ___mark_reg_known(true_reg, val); 14463 - true_64off = true_reg->var_off; 14343 + ___mark_reg_known(true_reg1, uval); 14344 + true_64off = true_reg1->var_off; 14464 14345 } 14465 14346 break; 14466 14347 case BPF_JNE: 14467 14348 if (is_jmp32) { 14468 - __mark_reg32_known(false_reg, val32); 14469 - false_32off = tnum_subreg(false_reg->var_off); 14349 + __mark_reg32_known(false_reg1, uval32); 14350 + false_32off = tnum_subreg(false_reg1->var_off); 14470 14351 } else { 14471 - ___mark_reg_known(false_reg, val); 14472 - false_64off = false_reg->var_off; 14352 + ___mark_reg_known(false_reg1, uval); 14353 + false_64off = false_reg1->var_off; 14473 14354 } 14474 14355 break; 14475 14356 case BPF_JSET: 14476 14357 if (is_jmp32) { 14477 - false_32off = tnum_and(false_32off, tnum_const(~val32)); 14478 - if (is_power_of_2(val32)) 14358 + false_32off = tnum_and(false_32off, tnum_const(~uval32)); 14359 + if (is_power_of_2(uval32)) 14479 14360 true_32off = tnum_or(true_32off, 14480 - tnum_const(val32)); 14361 + tnum_const(uval32)); 14481 14362 } else { 14482 - false_64off = tnum_and(false_64off, tnum_const(~val)); 14483 - if (is_power_of_2(val)) 14363 + false_64off = tnum_and(false_64off, tnum_const(~uval)); 14364 + if (is_power_of_2(uval)) 14484 14365 true_64off = tnum_or(true_64off, 14485 - tnum_const(val)); 14366 + tnum_const(uval)); 14486 14367 } 14487 14368 break; 14488 14369 case BPF_JGE: 14489 14370 case BPF_JGT: 14490 14371 { 14491 14372 if (is_jmp32) { 14492 - u32 false_umax = opcode == BPF_JGT ? val32 : val32 - 1; 14493 - u32 true_umin = opcode == BPF_JGT ? val32 + 1 : val32; 14373 + u32 false_umax = opcode == BPF_JGT ? uval32 : uval32 - 1; 14374 + u32 true_umin = opcode == BPF_JGT ? uval32 + 1 : uval32; 14494 14375 14495 - false_reg->u32_max_value = min(false_reg->u32_max_value, 14376 + false_reg1->u32_max_value = min(false_reg1->u32_max_value, 14496 14377 false_umax); 14497 - true_reg->u32_min_value = max(true_reg->u32_min_value, 14378 + true_reg1->u32_min_value = max(true_reg1->u32_min_value, 14498 14379 true_umin); 14499 14380 } else { 14500 - u64 false_umax = opcode == BPF_JGT ? val : val - 1; 14501 - u64 true_umin = opcode == BPF_JGT ? val + 1 : val; 14381 + u64 false_umax = opcode == BPF_JGT ? uval : uval - 1; 14382 + u64 true_umin = opcode == BPF_JGT ? uval + 1 : uval; 14502 14383 14503 - false_reg->umax_value = min(false_reg->umax_value, false_umax); 14504 - true_reg->umin_value = max(true_reg->umin_value, true_umin); 14384 + false_reg1->umax_value = min(false_reg1->umax_value, false_umax); 14385 + true_reg1->umin_value = max(true_reg1->umin_value, true_umin); 14505 14386 } 14506 14387 break; 14507 14388 } ··· 14512 14393 s32 false_smax = opcode == BPF_JSGT ? sval32 : sval32 - 1; 14513 14394 s32 true_smin = opcode == BPF_JSGT ? sval32 + 1 : sval32; 14514 14395 14515 - false_reg->s32_max_value = min(false_reg->s32_max_value, false_smax); 14516 - true_reg->s32_min_value = max(true_reg->s32_min_value, true_smin); 14396 + false_reg1->s32_max_value = min(false_reg1->s32_max_value, false_smax); 14397 + true_reg1->s32_min_value = max(true_reg1->s32_min_value, true_smin); 14517 14398 } else { 14518 14399 s64 false_smax = opcode == BPF_JSGT ? sval : sval - 1; 14519 14400 s64 true_smin = opcode == BPF_JSGT ? sval + 1 : sval; 14520 14401 14521 - false_reg->smax_value = min(false_reg->smax_value, false_smax); 14522 - true_reg->smin_value = max(true_reg->smin_value, true_smin); 14402 + false_reg1->smax_value = min(false_reg1->smax_value, false_smax); 14403 + true_reg1->smin_value = max(true_reg1->smin_value, true_smin); 14523 14404 } 14524 14405 break; 14525 14406 } ··· 14527 14408 case BPF_JLT: 14528 14409 { 14529 14410 if (is_jmp32) { 14530 - u32 false_umin = opcode == BPF_JLT ? val32 : val32 + 1; 14531 - u32 true_umax = opcode == BPF_JLT ? val32 - 1 : val32; 14411 + u32 false_umin = opcode == BPF_JLT ? uval32 : uval32 + 1; 14412 + u32 true_umax = opcode == BPF_JLT ? uval32 - 1 : uval32; 14532 14413 14533 - false_reg->u32_min_value = max(false_reg->u32_min_value, 14414 + false_reg1->u32_min_value = max(false_reg1->u32_min_value, 14534 14415 false_umin); 14535 - true_reg->u32_max_value = min(true_reg->u32_max_value, 14416 + true_reg1->u32_max_value = min(true_reg1->u32_max_value, 14536 14417 true_umax); 14537 14418 } else { 14538 - u64 false_umin = opcode == BPF_JLT ? val : val + 1; 14539 - u64 true_umax = opcode == BPF_JLT ? val - 1 : val; 14419 + u64 false_umin = opcode == BPF_JLT ? uval : uval + 1; 14420 + u64 true_umax = opcode == BPF_JLT ? uval - 1 : uval; 14540 14421 14541 - false_reg->umin_value = max(false_reg->umin_value, false_umin); 14542 - true_reg->umax_value = min(true_reg->umax_value, true_umax); 14422 + false_reg1->umin_value = max(false_reg1->umin_value, false_umin); 14423 + true_reg1->umax_value = min(true_reg1->umax_value, true_umax); 14543 14424 } 14544 14425 break; 14545 14426 } ··· 14550 14431 s32 false_smin = opcode == BPF_JSLT ? sval32 : sval32 + 1; 14551 14432 s32 true_smax = opcode == BPF_JSLT ? sval32 - 1 : sval32; 14552 14433 14553 - false_reg->s32_min_value = max(false_reg->s32_min_value, false_smin); 14554 - true_reg->s32_max_value = min(true_reg->s32_max_value, true_smax); 14434 + false_reg1->s32_min_value = max(false_reg1->s32_min_value, false_smin); 14435 + true_reg1->s32_max_value = min(true_reg1->s32_max_value, true_smax); 14555 14436 } else { 14556 14437 s64 false_smin = opcode == BPF_JSLT ? sval : sval + 1; 14557 14438 s64 true_smax = opcode == BPF_JSLT ? sval - 1 : sval; 14558 14439 14559 - false_reg->smin_value = max(false_reg->smin_value, false_smin); 14560 - true_reg->smax_value = min(true_reg->smax_value, true_smax); 14440 + false_reg1->smin_value = max(false_reg1->smin_value, false_smin); 14441 + true_reg1->smax_value = min(true_reg1->smax_value, true_smax); 14561 14442 } 14562 14443 break; 14563 14444 } ··· 14566 14447 } 14567 14448 14568 14449 if (is_jmp32) { 14569 - false_reg->var_off = tnum_or(tnum_clear_subreg(false_64off), 14450 + false_reg1->var_off = tnum_or(tnum_clear_subreg(false_64off), 14570 14451 tnum_subreg(false_32off)); 14571 - true_reg->var_off = tnum_or(tnum_clear_subreg(true_64off), 14452 + true_reg1->var_off = tnum_or(tnum_clear_subreg(true_64off), 14572 14453 tnum_subreg(true_32off)); 14573 - __reg_combine_32_into_64(false_reg); 14574 - __reg_combine_32_into_64(true_reg); 14454 + reg_bounds_sync(false_reg1); 14455 + reg_bounds_sync(true_reg1); 14575 14456 } else { 14576 - false_reg->var_off = false_64off; 14577 - true_reg->var_off = true_64off; 14578 - __reg_combine_64_into_32(false_reg); 14579 - __reg_combine_64_into_32(true_reg); 14457 + false_reg1->var_off = false_64off; 14458 + true_reg1->var_off = true_64off; 14459 + reg_bounds_sync(false_reg1); 14460 + reg_bounds_sync(true_reg1); 14580 14461 } 14581 - } 14582 - 14583 - /* Same as above, but for the case that dst_reg holds a constant and src_reg is 14584 - * the variable reg. 14585 - */ 14586 - static void reg_set_min_max_inv(struct bpf_reg_state *true_reg, 14587 - struct bpf_reg_state *false_reg, 14588 - u64 val, u32 val32, 14589 - u8 opcode, bool is_jmp32) 14590 - { 14591 - opcode = flip_opcode(opcode); 14592 - /* This uses zero as "not present in table"; luckily the zero opcode, 14593 - * BPF_JA, can't get here. 14594 - */ 14595 - if (opcode) 14596 - reg_set_min_max(true_reg, false_reg, val, val32, opcode, is_jmp32); 14597 14462 } 14598 14463 14599 14464 /* Regs are known to be equal, so intersect their min/max/var_off */ ··· 14809 14706 struct bpf_reg_state *regs = this_branch->frame[this_branch->curframe]->regs; 14810 14707 struct bpf_reg_state *dst_reg, *other_branch_regs, *src_reg = NULL; 14811 14708 struct bpf_reg_state *eq_branch_regs; 14709 + struct bpf_reg_state fake_reg = {}; 14812 14710 u8 opcode = BPF_OP(insn->code); 14813 14711 bool is_jmp32; 14814 14712 int pred = -1; ··· 14850 14746 verbose(env, "BPF_JMP/JMP32 uses reserved fields\n"); 14851 14747 return -EINVAL; 14852 14748 } 14749 + src_reg = &fake_reg; 14750 + src_reg->type = SCALAR_VALUE; 14751 + __mark_reg_known(src_reg, insn->imm); 14853 14752 } 14854 14753 14855 14754 is_jmp32 = BPF_CLASS(insn->code) == BPF_JMP32; 14856 - 14857 - if (BPF_SRC(insn->code) == BPF_K) { 14858 - pred = is_branch_taken(dst_reg, insn->imm, opcode, is_jmp32); 14859 - } else if (src_reg->type == SCALAR_VALUE && 14860 - is_jmp32 && tnum_is_const(tnum_subreg(src_reg->var_off))) { 14861 - pred = is_branch_taken(dst_reg, 14862 - tnum_subreg(src_reg->var_off).value, 14863 - opcode, 14864 - is_jmp32); 14865 - } else if (src_reg->type == SCALAR_VALUE && 14866 - !is_jmp32 && tnum_is_const(src_reg->var_off)) { 14867 - pred = is_branch_taken(dst_reg, 14868 - src_reg->var_off.value, 14869 - opcode, 14870 - is_jmp32); 14871 - } else if (dst_reg->type == SCALAR_VALUE && 14872 - is_jmp32 && tnum_is_const(tnum_subreg(dst_reg->var_off))) { 14873 - pred = is_branch_taken(src_reg, 14874 - tnum_subreg(dst_reg->var_off).value, 14875 - flip_opcode(opcode), 14876 - is_jmp32); 14877 - } else if (dst_reg->type == SCALAR_VALUE && 14878 - !is_jmp32 && tnum_is_const(dst_reg->var_off)) { 14879 - pred = is_branch_taken(src_reg, 14880 - dst_reg->var_off.value, 14881 - flip_opcode(opcode), 14882 - is_jmp32); 14883 - } else if (reg_is_pkt_pointer_any(dst_reg) && 14884 - reg_is_pkt_pointer_any(src_reg) && 14885 - !is_jmp32) { 14886 - pred = is_pkt_ptr_branch_taken(dst_reg, src_reg, opcode); 14887 - } 14888 - 14755 + pred = is_branch_taken(dst_reg, src_reg, opcode, is_jmp32); 14889 14756 if (pred >= 0) { 14890 14757 /* If we get here with a dst_reg pointer type it is because 14891 14758 * above is_branch_taken() special cased the 0 comparison. ··· 14904 14829 return -EFAULT; 14905 14830 other_branch_regs = other_branch->frame[other_branch->curframe]->regs; 14906 14831 14907 - /* detect if we are comparing against a constant value so we can adjust 14908 - * our min/max values for our dst register. 14909 - * this is only legit if both are scalars (or pointers to the same 14910 - * object, I suppose, see the PTR_MAYBE_NULL related if block below), 14911 - * because otherwise the different base pointers mean the offsets aren't 14912 - * comparable. 14913 - */ 14914 14832 if (BPF_SRC(insn->code) == BPF_X) { 14915 - struct bpf_reg_state *src_reg = &regs[insn->src_reg]; 14833 + reg_set_min_max(&other_branch_regs[insn->dst_reg], 14834 + &other_branch_regs[insn->src_reg], 14835 + dst_reg, src_reg, opcode, is_jmp32); 14916 14836 14917 14837 if (dst_reg->type == SCALAR_VALUE && 14918 - src_reg->type == SCALAR_VALUE) { 14919 - if (tnum_is_const(src_reg->var_off) || 14920 - (is_jmp32 && 14921 - tnum_is_const(tnum_subreg(src_reg->var_off)))) 14922 - reg_set_min_max(&other_branch_regs[insn->dst_reg], 14923 - dst_reg, 14924 - src_reg->var_off.value, 14925 - tnum_subreg(src_reg->var_off).value, 14926 - opcode, is_jmp32); 14927 - else if (tnum_is_const(dst_reg->var_off) || 14928 - (is_jmp32 && 14929 - tnum_is_const(tnum_subreg(dst_reg->var_off)))) 14930 - reg_set_min_max_inv(&other_branch_regs[insn->src_reg], 14931 - src_reg, 14932 - dst_reg->var_off.value, 14933 - tnum_subreg(dst_reg->var_off).value, 14934 - opcode, is_jmp32); 14935 - else if (!is_jmp32 && 14936 - (opcode == BPF_JEQ || opcode == BPF_JNE)) 14937 - /* Comparing for equality, we can combine knowledge */ 14938 - reg_combine_min_max(&other_branch_regs[insn->src_reg], 14939 - &other_branch_regs[insn->dst_reg], 14940 - src_reg, dst_reg, opcode); 14941 - if (src_reg->id && 14942 - !WARN_ON_ONCE(src_reg->id != other_branch_regs[insn->src_reg].id)) { 14943 - find_equal_scalars(this_branch, src_reg); 14944 - find_equal_scalars(other_branch, &other_branch_regs[insn->src_reg]); 14945 - } 14946 - 14838 + src_reg->type == SCALAR_VALUE && 14839 + !is_jmp32 && (opcode == BPF_JEQ || opcode == BPF_JNE)) { 14840 + /* Comparing for equality, we can combine knowledge */ 14841 + reg_combine_min_max(&other_branch_regs[insn->src_reg], 14842 + &other_branch_regs[insn->dst_reg], 14843 + src_reg, dst_reg, opcode); 14947 14844 } 14948 - } else if (dst_reg->type == SCALAR_VALUE) { 14845 + } else /* BPF_SRC(insn->code) == BPF_K */ { 14949 14846 reg_set_min_max(&other_branch_regs[insn->dst_reg], 14950 - dst_reg, insn->imm, (u32)insn->imm, 14951 - opcode, is_jmp32); 14847 + src_reg /* fake one */, 14848 + dst_reg, src_reg /* same fake one */, 14849 + opcode, is_jmp32); 14952 14850 } 14953 14851 14852 + if (BPF_SRC(insn->code) == BPF_X && 14853 + src_reg->type == SCALAR_VALUE && src_reg->id && 14854 + !WARN_ON_ONCE(src_reg->id != other_branch_regs[insn->src_reg].id)) { 14855 + find_equal_scalars(this_branch, src_reg); 14856 + find_equal_scalars(other_branch, &other_branch_regs[insn->src_reg]); 14857 + } 14954 14858 if (dst_reg->type == SCALAR_VALUE && dst_reg->id && 14955 14859 !WARN_ON_ONCE(dst_reg->id != other_branch_regs[insn->dst_reg].id)) { 14956 14860 find_equal_scalars(this_branch, dst_reg);
+1
tools/testing/selftests/bpf/prog_tests/btf.c
··· 5265 5265 #endif 5266 5266 5267 5267 assert(0); 5268 + return 0; 5268 5269 } 5269 5270 5270 5271 static void set_pprint_mapv(enum pprint_mapv_kind_t mapv_kind,
+1 -5
tools/testing/selftests/bpf/prog_tests/tc_opts.c
··· 2387 2387 const size_t prog_insn_cnt = sizeof(prog_insns) / sizeof(struct bpf_insn); 2388 2388 LIBBPF_OPTS(bpf_prog_load_opts, opts); 2389 2389 const size_t log_buf_sz = 256; 2390 - char *log_buf; 2390 + char log_buf[log_buf_sz]; 2391 2391 int fd = -1; 2392 2392 2393 - log_buf = malloc(log_buf_sz); 2394 - if (!ASSERT_OK_PTR(log_buf, "log_buf_alloc")) 2395 - return fd; 2396 2393 opts.log_buf = log_buf; 2397 2394 opts.log_size = log_buf_sz; 2398 2395 ··· 2399 2402 prog_insns, prog_insn_cnt, &opts); 2400 2403 ASSERT_STREQ(log_buf, "", "log_0"); 2401 2404 ASSERT_GE(fd, 0, "prog_fd"); 2402 - free(log_buf); 2403 2405 return fd; 2404 2406 } 2405 2407