Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

rqspinlock: Add support for timeouts

Introduce policy macro RES_CHECK_TIMEOUT which can be used to detect
when the timeout has expired for the slow path to return an error. It
depends on being passed two variables initialized to 0: ts, ret. The
'ts' parameter is of type rqspinlock_timeout.

This macro resolves to the (ret) expression so that it can be used in
statements like smp_cond_load_acquire to break the waiting loop
condition.

The 'spin' member is used to amortize the cost of checking time by
dispatching to the implementation every 64k iterations. The
'timeout_end' member is used to keep track of the timestamp that denotes
the end of the waiting period. The 'ret' parameter denotes the status of
the timeout, and can be checked in the slow path to detect timeouts
after waiting loops.

The 'duration' member is used to store the timeout duration for each
waiting loop. The default timeout value defined in the header
(RES_DEF_TIMEOUT) is 0.25 seconds.

This macro will be used as a condition for waiting loops in the slow
path. Since each waiting loop applies a fresh timeout using the same
rqspinlock_timeout, we add a new RES_RESET_TIMEOUT as well to ensure the
values can be easily reinitialized to the default state.

Reviewed-by: Barret Rhoden <brho@google.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20250316040541.108729-8-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>

authored by

Kumar Kartikeya Dwivedi and committed by
Alexei Starovoitov
14c48ee8 a926d099

+51
+6
include/asm-generic/rqspinlock.h
··· 10 10 #define __ASM_GENERIC_RQSPINLOCK_H 11 11 12 12 #include <linux/types.h> 13 + #include <vdso/time64.h> 13 14 14 15 struct qspinlock; 15 16 typedef struct qspinlock rqspinlock_t; 16 17 17 18 extern void resilient_queued_spin_lock_slowpath(rqspinlock_t *lock, u32 val); 19 + 20 + /* 21 + * Default timeout for waiting loops is 0.25 seconds 22 + */ 23 + #define RES_DEF_TIMEOUT (NSEC_PER_SEC / 4) 18 24 19 25 #endif /* __ASM_GENERIC_RQSPINLOCK_H */
+45
kernel/bpf/rqspinlock.c
··· 6 6 * (C) Copyright 2013-2014,2018 Red Hat, Inc. 7 7 * (C) Copyright 2015 Intel Corp. 8 8 * (C) Copyright 2015 Hewlett-Packard Enterprise Development LP 9 + * (C) Copyright 2024-2025 Meta Platforms, Inc. and affiliates. 9 10 * 10 11 * Authors: Waiman Long <longman@redhat.com> 11 12 * Peter Zijlstra <peterz@infradead.org> 13 + * Kumar Kartikeya Dwivedi <memxor@gmail.com> 12 14 */ 13 15 14 16 #include <linux/smp.h> ··· 24 22 #include <asm/qspinlock.h> 25 23 #include <trace/events/lock.h> 26 24 #include <asm/rqspinlock.h> 25 + #include <linux/timekeeping.h> 27 26 28 27 /* 29 28 * Include queued spinlock definitions and statistics code ··· 71 68 72 69 #include "../locking/mcs_spinlock.h" 73 70 71 + struct rqspinlock_timeout { 72 + u64 timeout_end; 73 + u64 duration; 74 + u16 spin; 75 + }; 76 + 77 + static noinline int check_timeout(struct rqspinlock_timeout *ts) 78 + { 79 + u64 time = ktime_get_mono_fast_ns(); 80 + 81 + if (!ts->timeout_end) { 82 + ts->timeout_end = time + ts->duration; 83 + return 0; 84 + } 85 + 86 + if (time > ts->timeout_end) 87 + return -ETIMEDOUT; 88 + 89 + return 0; 90 + } 91 + 92 + #define RES_CHECK_TIMEOUT(ts, ret) \ 93 + ({ \ 94 + if (!(ts).spin++) \ 95 + (ret) = check_timeout(&(ts)); \ 96 + (ret); \ 97 + }) 98 + 99 + /* 100 + * Initialize the 'spin' member. 101 + */ 102 + #define RES_INIT_TIMEOUT(ts) ({ (ts).spin = 1; }) 103 + 104 + /* 105 + * We only need to reset 'timeout_end', 'spin' will just wrap around as necessary. 106 + * Duration is defined for each spin attempt, so set it here. 107 + */ 108 + #define RES_RESET_TIMEOUT(ts, _duration) ({ (ts).timeout_end = 0; (ts).duration = _duration; }) 109 + 74 110 /* 75 111 * Per-CPU queue node structures; we can never have more than 4 nested 76 112 * contexts: task, softirq, hardirq, nmi. ··· 142 100 void __lockfunc resilient_queued_spin_lock_slowpath(rqspinlock_t *lock, u32 val) 143 101 { 144 102 struct mcs_spinlock *prev, *next, *node; 103 + struct rqspinlock_timeout ts; 145 104 u32 old, tail; 146 105 int idx; 147 106 148 107 BUILD_BUG_ON(CONFIG_NR_CPUS >= (1U << _Q_TAIL_CPU_BITS)); 108 + 109 + RES_INIT_TIMEOUT(ts); 149 110 150 111 /* 151 112 * Wait for in-progress pending->locked hand-overs with a bounded