tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

net: add skbuff_clear() helper

clang is unable to inline the memset() calls in net/core/skbuff.c
when initializing allocated sk_buff.

memset(skb, 0, offsetof(struct sk_buff, tail));

This is unfortunate, because:

1) calling external memset_orig() helper adds a call/ret and
typical setup cost.

2) offsetof(struct sk_buff, tail) == 0xb8 = 0x80 + 0x38

On x86_64, memset_orig() performs two 64 bytes clear,
then has to loop 7 times to clear the final 56 bytes.

skbuff_clear() makes sure the minimal and optimal code
is generated.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260109203836.1667441-1-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

authored by

Eric Dumazet and committed by
Jakub Kicinski 0391ab57 d7161b07

+22 -10
+22 -10
net/core/skbuff.c
··· 307 307 return skb; 308 308 } 309 309 310 + /* 311 + * Only clear those fields we need to clear, not those that we will 312 + * actually initialise later. Hence, don't put any more fields after 313 + * the tail pointer in struct sk_buff! 314 + */ 315 + static inline void skbuff_clear(struct sk_buff *skb) 316 + { 317 + /* Replace memset(skb, 0, offsetof(struct sk_buff, tail)) 318 + * with two smaller memset(), with a barrier() between them. 319 + * This forces the compiler to inline both calls. 320 + */ 321 + BUILD_BUG_ON(offsetof(struct sk_buff, tail) <= 128); 322 + memset(skb, 0, 128); 323 + barrier(); 324 + memset((void *)skb + 128, 0, offsetof(struct sk_buff, tail) - 128); 325 + } 326 + 310 327 /** 311 328 * napi_skb_cache_get_bulk - obtain a number of zeroed skb heads from the cache 312 329 * @skbs: pointer to an at least @n-sized array to fill with skb pointers ··· 374 357 skbs[i] = nc->skb_cache[base + i]; 375 358 376 359 kasan_mempool_unpoison_object(skbs[i], skbuff_cache_size); 377 - memset(skbs[i], 0, offsetof(struct sk_buff, tail)); 360 + skbuff_clear(skbs[i]); 378 361 } 379 362 380 363 nc->skb_count -= n; ··· 441 424 if (unlikely(!skb)) 442 425 return NULL; 443 426 444 - memset(skb, 0, offsetof(struct sk_buff, tail)); 427 + skbuff_clear(skb); 445 428 data = __slab_build_skb(data, &size); 446 429 __finalize_skb_around(skb, data, size); 447 430 ··· 493 476 if (unlikely(!skb)) 494 477 return NULL; 495 478 496 - memset(skb, 0, offsetof(struct sk_buff, tail)); 479 + skbuff_clear(skb); 497 480 __build_skb_around(skb, data, frag_size); 498 481 499 482 return skb; ··· 554 537 if (unlikely(!skb)) 555 538 return NULL; 556 539 557 - memset(skb, 0, offsetof(struct sk_buff, tail)); 540 + skbuff_clear(skb); 558 541 __build_skb_around(skb, data, frag_size); 559 542 560 543 return skb; ··· 713 696 */ 714 697 prefetchw(data + SKB_WITH_OVERHEAD(size)); 715 698 716 - /* 717 - * Only clear those fields we need to clear, not those that we will 718 - * actually initialise below. Hence, don't put any more fields after 719 - * the tail pointer in struct sk_buff! 720 - */ 721 - memset(skb, 0, offsetof(struct sk_buff, tail)); 699 + skbuff_clear(skb); 722 700 __build_skb_around(skb, data, size); 723 701 skb->pfmemalloc = pfmemalloc; 724 702