tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

tcp: cache RTAX_QUICKACK metric in a hot cache line

tcp_in_quickack_mode() is called from input path for small packets.

It calls __sk_dst_get() which reads sk->sk_dst_cache which has been
put in sock_read_tx group (for good reasons).

Then dst_metric(dst, RTAX_QUICKACK) also needs extra cache line misses.

Cache RTAX_QUICKACK in icsk->icsk_ack.dst_quick_ack to no longer pull
these cache lines for the cases a delayed ACK is scheduled.

After this patch TCP receive path does not longer access sock_read_tx
group.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Jason Xing <kerneljasonxing@gmail.com>
Reviewed-by: Neal Cardwell <ncardwell@google.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@amazon.com>
Link: https://patch.msgid.link/20250312083907.1931644-1-edumazet@google.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>

authored by

Eric Dumazet and committed by
Paolo Abeni 15492700 3c6b97a9

+8 -4
+2 -1
include/net/inet_connection_sock.h
··· 117 117 #define ATO_BITS 8 118 118 __u32 ato:ATO_BITS, /* Predicted tick of soft clock */ 119 119 lrcv_flowlabel:20, /* last received ipv6 flowlabel */ 120 - unused:4; 120 + dst_quick_ack:1, /* cache dst RTAX_QUICKACK */ 121 + unused:3; 121 122 unsigned long timeout; /* Currently scheduled timeout */ 122 123 __u32 lrcvtime; /* timestamp of last received data packet */ 123 124 __u16 last_seg_size; /* Size of last incoming segment */
+5 -1
net/core/sock.c
··· 2565 2565 u32 max_segs = 1; 2566 2566 2567 2567 sk->sk_route_caps = dst->dev->features; 2568 - if (sk_is_tcp(sk)) 2568 + if (sk_is_tcp(sk)) { 2569 + struct inet_connection_sock *icsk = inet_csk(sk); 2570 + 2569 2571 sk->sk_route_caps |= NETIF_F_GSO; 2572 + icsk->icsk_ack.dst_quick_ack = dst_metric(dst, RTAX_QUICKACK); 2573 + } 2570 2574 if (sk->sk_route_caps & NETIF_F_GSO) 2571 2575 sk->sk_route_caps |= NETIF_F_GSO_SOFTWARE; 2572 2576 if (unlikely(sk->sk_gso_disabled))
+1 -2
net/ipv4/tcp_input.c
··· 334 334 static bool tcp_in_quickack_mode(struct sock *sk) 335 335 { 336 336 const struct inet_connection_sock *icsk = inet_csk(sk); 337 - const struct dst_entry *dst = __sk_dst_get(sk); 338 337 339 - return (dst && dst_metric(dst, RTAX_QUICKACK)) || 338 + return icsk->icsk_ack.dst_quick_ack || 340 339 (icsk->icsk_ack.quick && !inet_csk_in_pingpong_mode(sk)); 341 340 } 342 341