tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

net: always inline skb_frag_unref() and __skb_frag_unref()

clang is not inlining skb_frag_unref() and __skb_frag_unref()
in gro fast path.

It also does not inline gro_try_pull_from_frag0().

Using __always_inline fixes this issue, makes the
kernel faster _and_ smaller.

Also change __skb_frag_ref(), skb_frag_ref() and skb_page_unref()
to let them inlined for the last patch in this series.

$ scripts/bloat-o-meter -t vmlinux.0 vmlinux.1
add/remove: 2/6 grow/shrink: 1/2 up/down: 218/-511 (-293)
Function old new delta
gro_pull_from_frag0 - 188 +188
__pfx_gro_pull_from_frag0 - 16 +16
skb_shift 1125 1139 +14
__pfx_skb_frag_unref 16 - -16
__pfx_gro_try_pull_from_frag0 16 - -16
__pfx___skb_frag_unref 16 - -16
__skb_frag_unref 36 - -36
skb_frag_unref 59 - -59
dev_gro_receive 1608 1514 -94
napi_gro_frags 892 771 -121
gro_try_pull_from_frag0 153 - -153
Total: Before=22566192, After=22565899, chg -0.00%

Signed-off-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260122045720.1221017-2-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

authored by

Eric Dumazet and committed by
Jakub Kicinski c75734b5 3f7522f3

+5 -5
+5 -5
include/linux/skbuff_ref.h
··· 15 15 * 16 16 * Takes an additional reference on the paged fragment @frag. 17 17 */ 18 - static inline void __skb_frag_ref(skb_frag_t *frag) 18 + static __always_inline void __skb_frag_ref(skb_frag_t *frag) 19 19 { 20 20 get_netmem(skb_frag_netmem(frag)); 21 21 } ··· 27 27 * 28 28 * Takes an additional reference on the @f'th paged fragment of @skb. 29 29 */ 30 - static inline void skb_frag_ref(struct sk_buff *skb, int f) 30 + static __always_inline void skb_frag_ref(struct sk_buff *skb, int f) 31 31 { 32 32 __skb_frag_ref(&skb_shinfo(skb)->frags[f]); 33 33 } 34 34 35 35 bool napi_pp_put_page(netmem_ref netmem); 36 36 37 - static inline void skb_page_unref(netmem_ref netmem, bool recycle) 37 + static __always_inline void skb_page_unref(netmem_ref netmem, bool recycle) 38 38 { 39 39 #ifdef CONFIG_PAGE_POOL 40 40 if (recycle && napi_pp_put_page(netmem)) ··· 51 51 * Releases a reference on the paged fragment @frag 52 52 * or recycles the page via the page_pool API. 53 53 */ 54 - static inline void __skb_frag_unref(skb_frag_t *frag, bool recycle) 54 + static __always_inline void __skb_frag_unref(skb_frag_t *frag, bool recycle) 55 55 { 56 56 skb_page_unref(skb_frag_netmem(frag), recycle); 57 57 } ··· 63 63 * 64 64 * Releases a reference on the @f'th paged fragment of @skb. 65 65 */ 66 - static inline void skb_frag_unref(struct sk_buff *skb, int f) 66 + static __always_inline void skb_frag_unref(struct sk_buff *skb, int f) 67 67 { 68 68 struct skb_shared_info *shinfo = skb_shinfo(skb); 69 69