tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

bpf: Use tnums for JEQ/JNE is_branch_taken logic

In the following toy program (reg states minimized for readability), R0
and R1 always have different values at instruction 6. This is obvious
when reading the program but cannot be guessed from ranges alone as
they overlap (R0 in [0; 0xc0000000], R1 in [1024; 0xc0000400]).

0: call bpf_get_prandom_u32#7 ; R0_w=scalar()
1: w0 = w0 ; R0_w=scalar(var_off=(0x0; 0xffffffff))
2: r0 >>= 30 ; R0_w=scalar(var_off=(0x0; 0x3))
3: r0 <<= 30 ; R0_w=scalar(var_off=(0x0; 0xc0000000))
4: r1 = r0 ; R1_w=scalar(var_off=(0x0; 0xc0000000))
5: r1 += 1024 ; R1_w=scalar(var_off=(0x400; 0xc0000000))
6: if r1 != r0 goto pc+1

Looking at tnums however, we can deduce that R1 is always different from
R0 because their tnums don't agree on known bits. This patch uses this
logic to improve is_scalar_branch_taken in case of BPF_JEQ and BPF_JNE.

This change has a tiny impact on complexity, which was measured with
the Cilium complexity CI test. That test covers 72 programs with
various build and load time configurations for a total of 970 test
cases. For 80% of test cases, the patch has no impact. On the other
test cases, the patch decreases complexity by only 0.08% on average. In
the best case, the verifier needs to walk 3% less instructions and, in
the worst case, 1.5% more. Overall, the patch has a small positive
impact, especially for our largest programs.

Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/be3ee70b6e489c49881cb1646114b1d861b5c334.1755694147.git.paul.chaignon@gmail.com

authored by

Paul Chaignon and committed by
Daniel Borkmann f41345f4 21aeabb6

+15
+3
include/linux/tnum.h
··· 51 51 /* Multiply two tnums, return @a * @b */ 52 52 struct tnum tnum_mul(struct tnum a, struct tnum b); 53 53 54 + /* Return true if the known bits of both tnums have the same value */ 55 + bool tnum_overlap(struct tnum a, struct tnum b); 56 + 54 57 /* Return a tnum representing numbers satisfying both @a and @b */ 55 58 struct tnum tnum_intersect(struct tnum a, struct tnum b); 56 59
+8
kernel/bpf/tnum.c
··· 143 143 return tnum_add(TNUM(acc_v, 0), acc_m); 144 144 } 145 145 146 + bool tnum_overlap(struct tnum a, struct tnum b) 147 + { 148 + u64 mu; 149 + 150 + mu = ~a.mask & ~b.mask; 151 + return (a.value & mu) == (b.value & mu); 152 + } 153 + 146 154 /* Note that if a and b disagree - i.e. one has a 'known 1' where the other has 147 155 * a 'known 0' - this will return a 'known 1' for that bit. 148 156 */
+4
kernel/bpf/verifier.c
··· 15897 15897 */ 15898 15898 if (tnum_is_const(t1) && tnum_is_const(t2)) 15899 15899 return t1.value == t2.value; 15900 + if (!tnum_overlap(t1, t2)) 15901 + return 0; 15900 15902 /* non-overlapping ranges */ 15901 15903 if (umin1 > umax2 || umax1 < umin2) 15902 15904 return 0; ··· 15923 15921 */ 15924 15922 if (tnum_is_const(t1) && tnum_is_const(t2)) 15925 15923 return t1.value != t2.value; 15924 + if (!tnum_overlap(t1, t2)) 15925 + return 1; 15926 15926 /* non-overlapping ranges */ 15927 15927 if (umin1 > umax2 || umax1 < umin2) 15928 15928 return 1;