tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Merge tag 'next.2025.05.17a' of git://git.kernel.org/pub/scm/linux/kernel/git/rcu/linux

Pull RCU updates from Joel Fernandes:
- Removed swake_up_one_online() workaround
- Reverted an incorrect rcuog wake-up fix from offline softirq
- Rust RCU Guard methods marked as inline
- Updated MAINTAINERS with Joel’s and Zqiang's new email address
- Replaced magic constant in rcu_seq_done_exact() with named constant
- Added warning mechanism to validate rcu_seq_done_exact()
- Switched SRCU polling API to use rcu_seq_done_exact()
- Commented on redundant delta check in rcu_seq_done_exact()
- Made ->gpwrap tests in rcutorture more frequent
- Fixed reuse of ARM64 images in rcutorture
- rcutorture improved to check Kconfig and reader conflict handling
- Extracted logic from rcu_torture_one_read() for clarity
- Updated LWN RCU API documentation links
- Enabled --do-rt in torture.sh for CONFIG_PREEMPT_RT
- Added tests for SRCU up/down reader primitives
- Added comments and delays checks in rcutorture
- Deprecated srcu_read_lock_lite() and srcu_read_unlock_lite() via checkpatch
- Added --do-normal and --do-no-normal to torture.sh
- Added RCU Rust binding tests to torture.sh
- Reduced CPU overcommit and removed MAXSMP/CPUMASK_OFFSTACK in TREE01
- Replaced kmalloc() with kcalloc() in rcuscale
- Refined listRCU example code for stale data elimination
- Fixed hardirq count bug for x86 in cpu_stall_cputime
- Added safety checks in rcu/nocb for offloaded rdp access
- Other miscellaneous changes

* tag 'next.2025.05.17a' of git://git.kernel.org/pub/scm/linux/kernel/git/rcu/linux: (27 commits)
rcutorture: Fix issue with re-using old images on ARM64
rcutorture: Remove MAXSMP and CPUMASK_OFFSTACK from TREE01
rcutorture: Reduce TREE01 CPU overcommit
torture: Check for "Call trace:" as well as "Call Trace:"
rcutorture: Perform more frequent testing of ->gpwrap
torture: Add testing of RCU's Rust bindings to torture.sh
torture: Add --do-{,no-}normal to torture.sh
checkpatch: Deprecate srcu_read_lock_lite() and srcu_read_unlock_lite()
rcutorture: Comment invocations of tick_dep_set_task()
rcu/nocb: Add Safe checks for access offloaded rdp
rcuscale: using kcalloc() to relpace kmalloc()
doc/RCU/listRCU: refine example code for eliminating stale data
doc: Update LWN RCU API links in whatisRCU.rst
Revert "rcu/nocb: Fix rcuog wake-up from offline softirq"
rust: sync: rcu: Mark Guard methods as inline
rcu/cpu_stall_cputime: fix the hardirq count for x86 architecture
rcu: Remove swake_up_one_online() bandaid
MAINTAINERS: Update Zqiang's email address
rcutorture: Make torture.sh --do-rt use CONFIG_PREEMPT_RT
srcu: Use rcu_seq_done_exact() for polling API
...

Linus Torvalds 07046958 66fd704d

+403 -124
+6 -4
Documentation/RCU/listRCU.rst
··· 334 334 to accomplish this would be to add a ``deleted`` flag and a ``lock`` spinlock to the 335 335 ``audit_entry`` structure, and modify audit_filter_task() as follows:: 336 336 337 - static enum audit_state audit_filter_task(struct task_struct *tsk) 337 + static struct audit_entry *audit_filter_task(struct task_struct *tsk, char **key) 338 338 { 339 339 struct audit_entry *e; 340 340 enum audit_state state; ··· 346 346 if (e->deleted) { 347 347 spin_unlock(&e->lock); 348 348 rcu_read_unlock(); 349 - return AUDIT_BUILD_CONTEXT; 349 + return NULL; 350 350 } 351 351 rcu_read_unlock(); 352 352 if (state == AUDIT_STATE_RECORD) 353 353 *key = kstrdup(e->rule.filterkey, GFP_ATOMIC); 354 - return state; 354 + /* As long as e->lock is held, e is valid and 355 + * its value is not stale */ 356 + return e; 355 357 } 356 358 } 357 359 rcu_read_unlock(); 358 - return AUDIT_BUILD_CONTEXT; 360 + return NULL; 359 361 } 360 362 361 363 The ``audit_del_rule()`` function would need to set the ``deleted`` flag under the
+3
Documentation/RCU/whatisRCU.rst
··· 15 15 | 2014 Big API Table https://lwn.net/Articles/609973/ 16 16 | 6. The RCU API, 2019 Edition https://lwn.net/Articles/777036/ 17 17 | 2019 Big API Table https://lwn.net/Articles/777165/ 18 + | 7. The RCU API, 2024 Edition https://lwn.net/Articles/988638/ 19 + | 2024 Background Information https://lwn.net/Articles/988641/ 20 + | 2024 Big API Table https://lwn.net/Articles/988666/ 18 21 19 22 For those preferring video: 20 23
+25
Documentation/admin-guide/kernel-parameters.txt
··· 5672 5672 are zero, rcutorture acts as if is interpreted 5673 5673 they are all non-zero. 5674 5674 5675 + rcutorture.gpwrap_lag= [KNL] 5676 + Enable grace-period wrap lag testing. Setting 5677 + to false prevents the gpwrap lag test from 5678 + running. Default is true. 5679 + 5680 + rcutorture.gpwrap_lag_gps= [KNL] 5681 + Set the value for grace-period wrap lag during 5682 + active lag testing periods. This controls how many 5683 + grace periods differences we tolerate between 5684 + rdp and rnp's gp_seq before setting overflow flag. 5685 + The default is always set to 8. 5686 + 5687 + rcutorture.gpwrap_lag_cycle_mins= [KNL] 5688 + Set the total cycle duration for gpwrap lag 5689 + testing in minutes. This is the total time for 5690 + one complete cycle of active and inactive 5691 + testing periods. Default is 30 minutes. 5692 + 5693 + rcutorture.gpwrap_lag_active_mins= [KNL] 5694 + Set the duration for which gpwrap lag is active 5695 + within each cycle, in minutes. During this time, 5696 + the grace-period wrap lag will be set to the 5697 + value specified by gpwrap_lag_gps. Default is 5698 + 5 minutes. 5699 + 5675 5700 rcutorture.irqreader= [KNL] 5676 5701 Run RCU readers from irq handlers, or, more 5677 5702 accurately, from a timer handler. Not all RCU
+4 -4
MAINTAINERS
··· 1682 1682 M: Arve Hjønnevåg <arve@android.com> 1683 1683 M: Todd Kjos <tkjos@android.com> 1684 1684 M: Martijn Coenen <maco@android.com> 1685 - M: Joel Fernandes <joel@joelfernandes.org> 1685 + M: Joel Fernandes <joelagnelf@nvidia.com> 1686 1686 M: Christian Brauner <christian@brauner.io> 1687 1687 M: Carlos Llamas <cmllamas@google.com> 1688 1688 M: Suren Baghdasaryan <surenb@google.com> ··· 13746 13746 M: "Paul E. McKenney" <paulmck@kernel.org> 13747 13747 R: Akira Yokosawa <akiyks@gmail.com> 13748 13748 R: Daniel Lustig <dlustig@nvidia.com> 13749 - R: Joel Fernandes <joel@joelfernandes.org> 13749 + R: Joel Fernandes <joelagnelf@nvidia.com> 13750 13750 L: linux-kernel@vger.kernel.org 13751 13751 L: linux-arch@vger.kernel.org 13752 13752 L: lkmm@lists.linux.dev ··· 20514 20514 M: "Paul E. McKenney" <paulmck@kernel.org> 20515 20515 M: Frederic Weisbecker <frederic@kernel.org> (kernel/rcu/tree_nocb.h) 20516 20516 M: Neeraj Upadhyay <neeraj.upadhyay@kernel.org> (kernel/rcu/tasks.h) 20517 - M: Joel Fernandes <joel@joelfernandes.org> 20517 + M: Joel Fernandes <joelagnelf@nvidia.com> 20518 20518 M: Josh Triplett <josh@joshtriplett.org> 20519 20519 M: Boqun Feng <boqun.feng@gmail.com> 20520 20520 M: Uladzislau Rezki <urezki@gmail.com> 20521 20521 R: Steven Rostedt <rostedt@goodmis.org> 20522 20522 R: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> 20523 20523 R: Lai Jiangshan <jiangshanlai@gmail.com> 20524 - R: Zqiang <qiang.zhang1211@gmail.com> 20524 + R: Zqiang <qiang.zhang@linux.dev> 20525 20525 L: rcu@vger.kernel.org 20526 20526 S: Supported 20527 20527 W: http://www.rdrop.com/users/paulmck/RCU/
+17 -1
kernel/rcu/rcu.h
··· 57 57 /* Low-order bit definition for polled grace-period APIs. */ 58 58 #define RCU_GET_STATE_COMPLETED 0x1 59 59 60 + /* A complete grace period count */ 61 + #define RCU_SEQ_GP (RCU_SEQ_STATE_MASK + 1) 62 + 60 63 extern int sysctl_sched_rt_runtime; 61 64 62 65 /* ··· 160 157 * Given a snapshot from rcu_seq_snap(), determine whether or not a 161 158 * full update-side operation has occurred, but do not allow the 162 159 * (ULONG_MAX / 2) safety-factor/guard-band. 160 + * 161 + * The token returned by get_state_synchronize_rcu_full() is based on 162 + * rcu_state.gp_seq but it is tested in poll_state_synchronize_rcu_full() 163 + * against the root rnp->gp_seq. Since rcu_seq_start() is first called 164 + * on rcu_state.gp_seq and only later reflected on the root rnp->gp_seq, 165 + * it is possible that rcu_seq_snap(rcu_state.gp_seq) returns 2 full grace 166 + * periods ahead of the root rnp->gp_seq. To prevent false-positives with the 167 + * full polling API that a wrap around instantly completed the GP, when nothing 168 + * like that happened, adjust for the 2 GPs in the ULONG_CMP_LT(). 163 169 */ 164 170 static inline bool rcu_seq_done_exact(unsigned long *sp, unsigned long s) 165 171 { 166 172 unsigned long cur_s = READ_ONCE(*sp); 167 173 168 - return ULONG_CMP_GE(cur_s, s) || ULONG_CMP_LT(cur_s, s - (3 * RCU_SEQ_STATE_MASK + 1)); 174 + return ULONG_CMP_GE(cur_s, s) || ULONG_CMP_LT(cur_s, s - (2 * RCU_SEQ_GP)); 169 175 } 170 176 171 177 /* ··· 584 572 unsigned long c_old, 585 573 unsigned long c); 586 574 void rcu_gp_set_torture_wait(int duration); 575 + void rcu_set_gpwrap_lag(unsigned long lag); 576 + int rcu_get_gpwrap_count(int cpu); 587 577 #else 588 578 static inline void rcutorture_get_gp_data(int *flags, unsigned long *gp_seq) 589 579 { ··· 603 589 do { } while (0) 604 590 #endif 605 591 static inline void rcu_gp_set_torture_wait(int duration) { } 592 + static inline void rcu_set_gpwrap_lag(unsigned long lag) { } 593 + static inline int rcu_get_gpwrap_count(int cpu) { return 0; } 606 594 #endif 607 595 unsigned long long rcutorture_gather_gp_seqs(void); 608 596 void rcutorture_format_gp_seqs(unsigned long long seqs, char *cp, size_t len);
+1 -1
kernel/rcu/rcuscale.c
··· 762 762 } 763 763 764 764 for (i = 0; i < kfree_alloc_num; i++) { 765 - alloc_ptr = kmalloc(kfree_mult * sizeof(struct kfree_obj), GFP_KERNEL); 765 + alloc_ptr = kcalloc(kfree_mult, sizeof(struct kfree_obj), GFP_KERNEL); 766 766 if (!alloc_ptr) 767 767 return -ENOMEM; 768 768
+159 -47
kernel/rcu/rcutorture.c
··· 115 115 torture_param(int, object_debug, 0, "Enable debug-object double call_rcu() testing"); 116 116 torture_param(int, onoff_holdoff, 0, "Time after boot before CPU hotplugs (s)"); 117 117 torture_param(int, onoff_interval, 0, "Time between CPU hotplugs (jiffies), 0=disable"); 118 + torture_param(bool, gpwrap_lag, true, "Enable grace-period wrap lag testing"); 119 + torture_param(int, gpwrap_lag_gps, 8, "Value to set for set_gpwrap_lag during an active testing period."); 120 + torture_param(int, gpwrap_lag_cycle_mins, 30, "Total cycle duration for gpwrap lag testing (in minutes)"); 121 + torture_param(int, gpwrap_lag_active_mins, 5, "Duration for which gpwrap lag is active within each cycle (in minutes)"); 118 122 torture_param(int, nocbs_nthreads, 0, "Number of NOCB toggle threads, 0 to disable"); 119 123 torture_param(int, nocbs_toggle, 1000, "Time between toggling nocb state (ms)"); 120 124 torture_param(int, preempt_duration, 0, "Preemption duration (ms), zero to disable"); ··· 417 413 bool (*reader_blocked)(void); 418 414 unsigned long long (*gather_gp_seqs)(void); 419 415 void (*format_gp_seqs)(unsigned long long seqs, char *cp, size_t len); 416 + void (*set_gpwrap_lag)(unsigned long lag); 417 + int (*get_gpwrap_count)(int cpu); 420 418 long cbflood_max; 421 419 int irq_capable; 422 420 int can_boost; ··· 625 619 : NULL, 626 620 .gather_gp_seqs = rcutorture_gather_gp_seqs, 627 621 .format_gp_seqs = rcutorture_format_gp_seqs, 622 + .set_gpwrap_lag = rcu_set_gpwrap_lag, 623 + .get_gpwrap_count = rcu_get_gpwrap_count, 628 624 .irq_capable = 1, 629 625 .can_boost = IS_ENABLED(CONFIG_RCU_BOOST), 630 626 .extendables = RCUTORTURE_MAX_EXTEND, ··· 2172 2164 return &rtrsp[j]; 2173 2165 } 2174 2166 2175 - /* 2176 - * Do one read-side critical section, returning false if there was 2177 - * no data to read. Can be invoked both from process context and 2178 - * from a timer handler. 2179 - */ 2180 - static bool rcu_torture_one_read(struct torture_random_state *trsp, long myid) 2181 - { 2182 - bool checkpolling = !(torture_random(trsp) & 0xfff); 2167 + struct rcu_torture_one_read_state { 2168 + bool checkpolling; 2183 2169 unsigned long cookie; 2184 2170 struct rcu_gp_oldstate cookie_full; 2185 - int i; 2186 2171 unsigned long started; 2187 - unsigned long completed; 2188 - int newstate; 2189 2172 struct rcu_torture *p; 2190 - int pipe_count; 2191 - bool preempted = false; 2192 - int readstate = 0; 2193 - struct rt_read_seg rtseg[RCUTORTURE_RDR_MAX_SEGS] = { { 0 } }; 2194 - struct rt_read_seg *rtrsp = &rtseg[0]; 2195 - struct rt_read_seg *rtrsp1; 2173 + int readstate; 2174 + struct rt_read_seg rtseg[RCUTORTURE_RDR_MAX_SEGS]; 2175 + struct rt_read_seg *rtrsp; 2196 2176 unsigned long long ts; 2177 + }; 2197 2178 2198 - WARN_ON_ONCE(!rcu_is_watching()); 2199 - newstate = rcutorture_extend_mask(readstate, trsp); 2200 - rcutorture_one_extend(&readstate, newstate, myid < 0, trsp, rtrsp++); 2201 - if (checkpolling) { 2179 + static void init_rcu_torture_one_read_state(struct rcu_torture_one_read_state *rtorsp, 2180 + struct torture_random_state *trsp) 2181 + { 2182 + memset(rtorsp, 0, sizeof(*rtorsp)); 2183 + rtorsp->checkpolling = !(torture_random(trsp) & 0xfff); 2184 + rtorsp->rtrsp = &rtorsp->rtseg[0]; 2185 + } 2186 + 2187 + /* 2188 + * Set up the first segment of a series of overlapping read-side 2189 + * critical sections. The caller must have actually initiated the 2190 + * outermost read-side critical section. 2191 + */ 2192 + static bool rcu_torture_one_read_start(struct rcu_torture_one_read_state *rtorsp, 2193 + struct torture_random_state *trsp, long myid) 2194 + { 2195 + if (rtorsp->checkpolling) { 2202 2196 if (cur_ops->get_gp_state && cur_ops->poll_gp_state) 2203 - cookie = cur_ops->get_gp_state(); 2197 + rtorsp->cookie = cur_ops->get_gp_state(); 2204 2198 if (cur_ops->get_gp_state_full && cur_ops->poll_gp_state_full) 2205 - cur_ops->get_gp_state_full(&cookie_full); 2199 + cur_ops->get_gp_state_full(&rtorsp->cookie_full); 2206 2200 } 2207 - started = cur_ops->get_gp_seq(); 2208 - ts = rcu_trace_clock_local(); 2209 - p = rcu_dereference_check(rcu_torture_current, 2201 + rtorsp->started = cur_ops->get_gp_seq(); 2202 + rtorsp->ts = rcu_trace_clock_local(); 2203 + rtorsp->p = rcu_dereference_check(rcu_torture_current, 2210 2204 !cur_ops->readlock_held || cur_ops->readlock_held()); 2211 - if (p == NULL) { 2205 + if (rtorsp->p == NULL) { 2212 2206 /* Wait for rcu_torture_writer to get underway */ 2213 - rcutorture_one_extend(&readstate, 0, myid < 0, trsp, rtrsp); 2207 + rcutorture_one_extend(&rtorsp->readstate, 0, myid < 0, trsp, rtorsp->rtrsp); 2214 2208 return false; 2215 2209 } 2216 - if (p->rtort_mbtest == 0) 2210 + if (rtorsp->p->rtort_mbtest == 0) 2217 2211 atomic_inc(&n_rcu_torture_mberror); 2218 - rcu_torture_reader_do_mbchk(myid, p, trsp); 2219 - rtrsp = rcutorture_loop_extend(&readstate, myid < 0, trsp, rtrsp); 2212 + rcu_torture_reader_do_mbchk(myid, rtorsp->p, trsp); 2213 + return true; 2214 + } 2215 + 2216 + /* 2217 + * Complete the last segment of a series of overlapping read-side 2218 + * critical sections and check for errors. 2219 + */ 2220 + static void rcu_torture_one_read_end(struct rcu_torture_one_read_state *rtorsp, 2221 + struct torture_random_state *trsp, long myid) 2222 + { 2223 + int i; 2224 + unsigned long completed; 2225 + int pipe_count; 2226 + bool preempted = false; 2227 + struct rt_read_seg *rtrsp1; 2228 + 2220 2229 preempt_disable(); 2221 - pipe_count = READ_ONCE(p->rtort_pipe_count); 2230 + pipe_count = READ_ONCE(rtorsp->p->rtort_pipe_count); 2222 2231 if (pipe_count > RCU_TORTURE_PIPE_LEN) { 2223 2232 // Should not happen in a correct RCU implementation, 2224 2233 // happens quite often for torture_type=busted. ··· 2243 2218 } 2244 2219 completed = cur_ops->get_gp_seq(); 2245 2220 if (pipe_count > 1) { 2246 - do_trace_rcu_torture_read(cur_ops->name, &p->rtort_rcu, 2247 - ts, started, completed); 2221 + do_trace_rcu_torture_read(cur_ops->name, &rtorsp->p->rtort_rcu, 2222 + rtorsp->ts, rtorsp->started, completed); 2248 2223 rcu_ftrace_dump(DUMP_ALL); 2249 2224 } 2250 2225 __this_cpu_inc(rcu_torture_count[pipe_count]); 2251 - completed = rcutorture_seq_diff(completed, started); 2226 + completed = rcutorture_seq_diff(completed, rtorsp->started); 2252 2227 if (completed > RCU_TORTURE_PIPE_LEN) { 2253 2228 /* Should not happen, but... */ 2254 2229 completed = RCU_TORTURE_PIPE_LEN; 2255 2230 } 2256 2231 __this_cpu_inc(rcu_torture_batch[completed]); 2257 2232 preempt_enable(); 2258 - if (checkpolling) { 2233 + if (rtorsp->checkpolling) { 2259 2234 if (cur_ops->get_gp_state && cur_ops->poll_gp_state) 2260 - WARN_ONCE(cur_ops->poll_gp_state(cookie), 2235 + WARN_ONCE(cur_ops->poll_gp_state(rtorsp->cookie), 2261 2236 "%s: Cookie check 2 failed %s(%d) %lu->%lu\n", 2262 2237 __func__, 2263 2238 rcu_torture_writer_state_getname(), 2264 2239 rcu_torture_writer_state, 2265 - cookie, cur_ops->get_gp_state()); 2240 + rtorsp->cookie, cur_ops->get_gp_state()); 2266 2241 if (cur_ops->get_gp_state_full && cur_ops->poll_gp_state_full) 2267 - WARN_ONCE(cur_ops->poll_gp_state_full(&cookie_full), 2242 + WARN_ONCE(cur_ops->poll_gp_state_full(&rtorsp->cookie_full), 2268 2243 "%s: Cookie check 6 failed %s(%d) online %*pbl\n", 2269 2244 __func__, 2270 2245 rcu_torture_writer_state_getname(), ··· 2273 2248 } 2274 2249 if (cur_ops->reader_blocked) 2275 2250 preempted = cur_ops->reader_blocked(); 2276 - rcutorture_one_extend(&readstate, 0, myid < 0, trsp, rtrsp); 2277 - WARN_ON_ONCE(readstate); 2251 + rcutorture_one_extend(&rtorsp->readstate, 0, myid < 0, trsp, rtorsp->rtrsp); 2252 + WARN_ON_ONCE(rtorsp->readstate); 2278 2253 // This next splat is expected behavior if leakpointer, especially 2279 2254 // for CONFIG_RCU_STRICT_GRACE_PERIOD=y kernels. 2280 - WARN_ON_ONCE(leakpointer && READ_ONCE(p->rtort_pipe_count) > 1); 2255 + WARN_ON_ONCE(leakpointer && READ_ONCE(rtorsp->p->rtort_pipe_count) > 1); 2281 2256 2282 2257 /* If error or close call, record the sequence of reader protections. */ 2283 2258 if ((pipe_count > 1 || completed > 1) && !xchg(&err_segs_recorded, 1)) { 2284 2259 i = 0; 2285 - for (rtrsp1 = &rtseg[0]; rtrsp1 < rtrsp; rtrsp1++) 2260 + for (rtrsp1 = &rtorsp->rtseg[0]; rtrsp1 < rtorsp->rtrsp; rtrsp1++) 2286 2261 err_segs[i++] = *rtrsp1; 2287 2262 rt_read_nsegs = i; 2288 2263 rt_read_preempted = preempted; 2289 2264 } 2265 + } 2290 2266 2267 + /* 2268 + * Do one read-side critical section, returning false if there was 2269 + * no data to read. Can be invoked both from process context and 2270 + * from a timer handler. 2271 + */ 2272 + static bool rcu_torture_one_read(struct torture_random_state *trsp, long myid) 2273 + { 2274 + int newstate; 2275 + struct rcu_torture_one_read_state rtors; 2276 + 2277 + WARN_ON_ONCE(!rcu_is_watching()); 2278 + init_rcu_torture_one_read_state(&rtors, trsp); 2279 + newstate = rcutorture_extend_mask(rtors.readstate, trsp); 2280 + rcutorture_one_extend(&rtors.readstate, newstate, myid < 0, trsp, rtors.rtrsp++); 2281 + if (!rcu_torture_one_read_start(&rtors, trsp, myid)) { 2282 + rcutorture_one_extend(&rtors.readstate, 0, myid < 0, trsp, rtors.rtrsp); 2283 + return false; 2284 + } 2285 + rtors.rtrsp = rcutorture_loop_extend(&rtors.readstate, myid < 0, trsp, rtors.rtrsp); 2286 + rcu_torture_one_read_end(&rtors, trsp, myid); 2291 2287 return true; 2292 2288 } 2293 2289 ··· 2353 2307 set_user_nice(current, MAX_NICE); 2354 2308 if (irqreader && cur_ops->irq_capable) 2355 2309 timer_setup_on_stack(&t, rcu_torture_timer, 0); 2356 - tick_dep_set_task(current, TICK_DEP_BIT_RCU); 2310 + tick_dep_set_task(current, TICK_DEP_BIT_RCU); // CPU bound, so need tick. 2357 2311 do { 2358 2312 if (irqreader && cur_ops->irq_capable) { 2359 2313 if (!timer_pending(&t)) ··· 2440 2394 int i; 2441 2395 long pipesummary[RCU_TORTURE_PIPE_LEN + 1] = { 0 }; 2442 2396 long batchsummary[RCU_TORTURE_PIPE_LEN + 1] = { 0 }; 2397 + long n_gpwraps = 0; 2443 2398 struct rcu_torture *rtcp; 2444 2399 static unsigned long rtcv_snap = ULONG_MAX; 2445 2400 static bool splatted; ··· 2451 2404 pipesummary[i] += READ_ONCE(per_cpu(rcu_torture_count, cpu)[i]); 2452 2405 batchsummary[i] += READ_ONCE(per_cpu(rcu_torture_batch, cpu)[i]); 2453 2406 } 2407 + if (cur_ops->get_gpwrap_count) 2408 + n_gpwraps += cur_ops->get_gpwrap_count(cpu); 2454 2409 } 2455 2410 for (i = RCU_TORTURE_PIPE_LEN; i >= 0; i--) { 2456 2411 if (pipesummary[i] != 0) ··· 2484 2435 data_race(n_barrier_attempts), 2485 2436 data_race(n_rcu_torture_barrier_error)); 2486 2437 pr_cont("read-exits: %ld ", data_race(n_read_exits)); // Statistic. 2487 - pr_cont("nocb-toggles: %ld:%ld\n", 2438 + pr_cont("nocb-toggles: %ld:%ld ", 2488 2439 atomic_long_read(&n_nocb_offload), atomic_long_read(&n_nocb_deoffload)); 2440 + pr_cont("gpwraps: %ld\n", n_gpwraps); 2489 2441 2490 2442 pr_alert("%s%s ", torture_type, TORTURE_FLAG); 2491 2443 if (atomic_read(&n_rcu_torture_mberror) || ··· 3086 3036 cver = READ_ONCE(rcu_torture_current_version); 3087 3037 gps = cur_ops->get_gp_seq(); 3088 3038 rfp->rcu_launder_gp_seq_start = gps; 3089 - tick_dep_set_task(current, TICK_DEP_BIT_RCU); 3039 + tick_dep_set_task(current, TICK_DEP_BIT_RCU); // CPU bound, so need tick. 3090 3040 while (time_before(jiffies, stopat) && 3091 3041 !shutdown_time_arrived() && 3092 3042 !READ_ONCE(rcu_fwd_emergency_stop) && !torture_must_stop()) { ··· 3657 3607 3658 3608 static enum cpuhp_state rcutor_hp; 3659 3609 3610 + static struct hrtimer gpwrap_lag_timer; 3611 + static bool gpwrap_lag_active; 3612 + 3613 + /* Timer handler for toggling RCU grace-period sequence overflow test lag value */ 3614 + static enum hrtimer_restart rcu_gpwrap_lag_timer(struct hrtimer *timer) 3615 + { 3616 + ktime_t next_delay; 3617 + 3618 + if (gpwrap_lag_active) { 3619 + pr_alert("rcu-torture: Disabling gpwrap lag (value=0)\n"); 3620 + cur_ops->set_gpwrap_lag(0); 3621 + gpwrap_lag_active = false; 3622 + next_delay = ktime_set((gpwrap_lag_cycle_mins - gpwrap_lag_active_mins) * 60, 0); 3623 + } else { 3624 + pr_alert("rcu-torture: Enabling gpwrap lag (value=%d)\n", gpwrap_lag_gps); 3625 + cur_ops->set_gpwrap_lag(gpwrap_lag_gps); 3626 + gpwrap_lag_active = true; 3627 + next_delay = ktime_set(gpwrap_lag_active_mins * 60, 0); 3628 + } 3629 + 3630 + if (torture_must_stop_irq()) 3631 + return HRTIMER_NORESTART; 3632 + 3633 + hrtimer_forward_now(timer, next_delay); 3634 + return HRTIMER_RESTART; 3635 + } 3636 + 3637 + static int rcu_gpwrap_lag_init(void) 3638 + { 3639 + if (!gpwrap_lag) 3640 + return 0; 3641 + 3642 + if (gpwrap_lag_cycle_mins <= 0 || gpwrap_lag_active_mins <= 0) { 3643 + pr_alert("rcu-torture: lag timing parameters must be positive\n"); 3644 + return -EINVAL; 3645 + } 3646 + 3647 + hrtimer_setup(&gpwrap_lag_timer, rcu_gpwrap_lag_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); 3648 + gpwrap_lag_active = false; 3649 + hrtimer_start(&gpwrap_lag_timer, 3650 + ktime_set((gpwrap_lag_cycle_mins - gpwrap_lag_active_mins) * 60, 0), HRTIMER_MODE_REL); 3651 + 3652 + return 0; 3653 + } 3654 + 3655 + static void rcu_gpwrap_lag_cleanup(void) 3656 + { 3657 + hrtimer_cancel(&gpwrap_lag_timer); 3658 + cur_ops->set_gpwrap_lag(0); 3659 + gpwrap_lag_active = false; 3660 + } 3660 3661 static void 3661 3662 rcu_torture_cleanup(void) 3662 3663 { ··· 3877 3776 torture_cleanup_end(); 3878 3777 if (cur_ops->gp_slow_unregister) 3879 3778 cur_ops->gp_slow_unregister(NULL); 3779 + 3780 + if (gpwrap_lag && cur_ops->set_gpwrap_lag) 3781 + rcu_gpwrap_lag_cleanup(); 3880 3782 } 3881 3783 3882 3784 static void rcu_torture_leak_cb(struct rcu_head *rhp) ··· 4376 4272 } 4377 4273 if (object_debug) 4378 4274 rcu_test_debug_objects(); 4379 - torture_init_end(); 4275 + 4380 4276 if (cur_ops->gp_slow_register && !WARN_ON_ONCE(!cur_ops->gp_slow_unregister)) 4381 4277 cur_ops->gp_slow_register(&rcu_fwd_cb_nodelay); 4278 + 4279 + if (gpwrap_lag && cur_ops->set_gpwrap_lag) { 4280 + firsterr = rcu_gpwrap_lag_init(); 4281 + if (torture_init_error(firsterr)) 4282 + goto unwind; 4283 + } 4284 + 4285 + torture_init_end(); 4382 4286 return 0; 4383 4287 4384 4288 unwind:
+1 -1
kernel/rcu/srcutree.c
··· 1589 1589 bool poll_state_synchronize_srcu(struct srcu_struct *ssp, unsigned long cookie) 1590 1590 { 1591 1591 if (cookie != SRCU_GET_STATE_COMPLETED && 1592 - !rcu_seq_done(&ssp->srcu_sup->srcu_gp_seq, cookie)) 1592 + !rcu_seq_done_exact(&ssp->srcu_sup->srcu_gp_seq, cookie)) 1593 1593 return false; 1594 1594 // Ensure that the end of the SRCU grace period happens before 1595 1595 // any subsequent code that the caller might execute.
+46 -38
kernel/rcu/tree.c
··· 80 80 static DEFINE_PER_CPU_SHARED_ALIGNED(struct rcu_data, rcu_data) = { 81 81 .gpwrap = true, 82 82 }; 83 + 84 + int rcu_get_gpwrap_count(int cpu) 85 + { 86 + struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); 87 + 88 + return READ_ONCE(rdp->gpwrap_count); 89 + } 90 + EXPORT_SYMBOL_GPL(rcu_get_gpwrap_count); 91 + 83 92 static struct rcu_state rcu_state = { 84 93 .level = { &rcu_state.node[0] }, 85 94 .gp_state = RCU_GP_IDLE, ··· 766 757 smp_store_release(per_cpu_ptr(&rcu_data.rcu_urgent_qs, cpu), true); 767 758 } 768 759 760 + static unsigned long seq_gpwrap_lag = ULONG_MAX / 4; 761 + 762 + /** 763 + * rcu_set_gpwrap_lag - Set RCU GP sequence overflow lag value. 764 + * @lag_gps: Set overflow lag to this many grace period worth of counters 765 + * which is used by rcutorture to quickly force a gpwrap situation. 766 + * @lag_gps = 0 means we reset it back to the boot-time value. 767 + */ 768 + void rcu_set_gpwrap_lag(unsigned long lag_gps) 769 + { 770 + unsigned long lag_seq_count; 771 + 772 + lag_seq_count = (lag_gps == 0) 773 + ? ULONG_MAX / 4 774 + : lag_gps << RCU_SEQ_CTR_SHIFT; 775 + WRITE_ONCE(seq_gpwrap_lag, lag_seq_count); 776 + } 777 + EXPORT_SYMBOL_GPL(rcu_set_gpwrap_lag); 778 + 769 779 /* 770 780 * When trying to report a quiescent state on behalf of some other CPU, 771 781 * it is our responsibility to check for and handle potential overflow ··· 795 767 static void rcu_gpnum_ovf(struct rcu_node *rnp, struct rcu_data *rdp) 796 768 { 797 769 raw_lockdep_assert_held_rcu_node(rnp); 798 - if (ULONG_CMP_LT(rcu_seq_current(&rdp->gp_seq) + ULONG_MAX / 4, 799 - rnp->gp_seq)) 770 + if (ULONG_CMP_LT(rcu_seq_current(&rdp->gp_seq) + seq_gpwrap_lag, 771 + rnp->gp_seq)) { 800 772 WRITE_ONCE(rdp->gpwrap, true); 773 + WRITE_ONCE(rdp->gpwrap_count, READ_ONCE(rdp->gpwrap_count) + 1); 774 + } 801 775 if (ULONG_CMP_LT(rdp->rcu_iw_gp_seq + ULONG_MAX / 4, rnp->gp_seq)) 802 776 rdp->rcu_iw_gp_seq = rnp->gp_seq + ULONG_MAX / 4; 803 777 } ··· 830 800 } 831 801 return 0; 832 802 } 803 + 804 + #ifndef arch_irq_stat_cpu 805 + #define arch_irq_stat_cpu(cpu) 0 806 + #endif 833 807 834 808 /* 835 809 * Returns positive if the specified CPU has passed through a quiescent state ··· 970 936 rsrp->cputime_irq = kcpustat_field(kcsp, CPUTIME_IRQ, cpu); 971 937 rsrp->cputime_softirq = kcpustat_field(kcsp, CPUTIME_SOFTIRQ, cpu); 972 938 rsrp->cputime_system = kcpustat_field(kcsp, CPUTIME_SYSTEM, cpu); 973 - rsrp->nr_hardirqs = kstat_cpu_irqs_sum(rdp->cpu); 974 - rsrp->nr_softirqs = kstat_cpu_softirqs_sum(rdp->cpu); 975 - rsrp->nr_csw = nr_context_switches_cpu(rdp->cpu); 939 + rsrp->nr_hardirqs = kstat_cpu_irqs_sum(cpu) + arch_irq_stat_cpu(cpu); 940 + rsrp->nr_softirqs = kstat_cpu_softirqs_sum(cpu); 941 + rsrp->nr_csw = nr_context_switches_cpu(cpu); 976 942 rsrp->jiffies = jiffies; 977 943 rsrp->gp_seq = rdp->gp_seq; 978 944 } ··· 1094 1060 return needmore; 1095 1061 } 1096 1062 1097 - static void swake_up_one_online_ipi(void *arg) 1098 - { 1099 - struct swait_queue_head *wqh = arg; 1100 - 1101 - swake_up_one(wqh); 1102 - } 1103 - 1104 - static void swake_up_one_online(struct swait_queue_head *wqh) 1105 - { 1106 - int cpu = get_cpu(); 1107 - 1108 - /* 1109 - * If called from rcutree_report_cpu_starting(), wake up 1110 - * is dangerous that late in the CPU-down hotplug process. The 1111 - * scheduler might queue an ignored hrtimer. Defer the wake up 1112 - * to an online CPU instead. 1113 - */ 1114 - if (unlikely(cpu_is_offline(cpu))) { 1115 - int target; 1116 - 1117 - target = cpumask_any_and(housekeeping_cpumask(HK_TYPE_RCU), 1118 - cpu_online_mask); 1119 - 1120 - smp_call_function_single(target, swake_up_one_online_ipi, 1121 - wqh, 0); 1122 - put_cpu(); 1123 - } else { 1124 - put_cpu(); 1125 - swake_up_one(wqh); 1126 - } 1127 - } 1128 - 1129 1063 /* 1130 1064 * Awaken the grace-period kthread. Don't do a self-awaken (unless in an 1131 1065 * interrupt or softirq handler, in which case we just might immediately ··· 1118 1116 return; 1119 1117 WRITE_ONCE(rcu_state.gp_wake_time, jiffies); 1120 1118 WRITE_ONCE(rcu_state.gp_wake_seq, READ_ONCE(rcu_state.gp_seq)); 1121 - swake_up_one_online(&rcu_state.gp_wq); 1119 + swake_up_one(&rcu_state.gp_wq); 1122 1120 } 1123 1121 1124 1122 /* ··· 1800 1798 struct rcu_data *rdp; 1801 1799 struct rcu_node *rnp = rcu_get_root(); 1802 1800 bool start_new_poll; 1801 + unsigned long old_gp_seq; 1803 1802 1804 1803 WRITE_ONCE(rcu_state.gp_activity, jiffies); 1805 1804 raw_spin_lock_irq_rcu_node(rnp); ··· 1828 1825 */ 1829 1826 start_new_poll = rcu_sr_normal_gp_init(); 1830 1827 /* Record GP times before starting GP, hence rcu_seq_start(). */ 1828 + old_gp_seq = rcu_state.gp_seq; 1831 1829 rcu_seq_start(&rcu_state.gp_seq); 1830 + /* Ensure that rcu_seq_done_exact() guardband doesn't give false positives. */ 1831 + WARN_ON_ONCE(IS_ENABLED(CONFIG_PROVE_RCU) && 1832 + rcu_seq_done_exact(&old_gp_seq, rcu_seq_snap(&rcu_state.gp_seq))); 1833 + 1832 1834 ASSERT_EXCLUSIVE_WRITER(rcu_state.gp_seq); 1833 1835 trace_rcu_grace_period(rcu_state.name, rcu_state.gp_seq, TPS("start")); 1834 1836 rcu_poll_gp_seq_start(&rcu_state.gp_seq_polled_snap);
+2 -1
kernel/rcu/tree.h
··· 168 168 u64 cputime_irq; /* Accumulated cputime of hard irqs */ 169 169 u64 cputime_softirq;/* Accumulated cputime of soft irqs */ 170 170 u64 cputime_system; /* Accumulated cputime of kernel tasks */ 171 - unsigned long nr_hardirqs; /* Accumulated number of hard irqs */ 171 + u64 nr_hardirqs; /* Accumulated number of hard irqs */ 172 172 unsigned int nr_softirqs; /* Accumulated number of soft irqs */ 173 173 unsigned long long nr_csw; /* Accumulated number of task switches */ 174 174 unsigned long jiffies; /* Track jiffies value */ ··· 183 183 bool core_needs_qs; /* Core waits for quiescent state. */ 184 184 bool beenonline; /* CPU online at least once. */ 185 185 bool gpwrap; /* Possible ->gp_seq wrap. */ 186 + unsigned int gpwrap_count; /* Count of GP sequence wrap. */ 186 187 bool cpu_started; /* RCU watching this onlining CPU. */ 187 188 struct rcu_node *mynode; /* This CPU's leaf of hierarchy */ 188 189 unsigned long grpmask; /* Mask to apply to leaf qsmask. */
+1 -1
kernel/rcu/tree_exp.h
··· 200 200 if (rnp->parent == NULL) { 201 201 raw_spin_unlock_irqrestore_rcu_node(rnp, flags); 202 202 if (wake) 203 - swake_up_one_online(&rcu_state.expedited_wq); 203 + swake_up_one(&rcu_state.expedited_wq); 204 204 205 205 break; 206 206 }
+2 -8
kernel/rcu/tree_nocb.h
··· 216 216 raw_spin_unlock_irqrestore(&rdp_gp->nocb_gp_lock, flags); 217 217 if (needwake) { 218 218 trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("DoWake")); 219 - swake_up_one_online(&rdp_gp->nocb_gp_wq); 219 + swake_up_one(&rdp_gp->nocb_gp_wq); 220 220 } 221 221 222 222 return needwake; ··· 554 554 rcu_nocb_unlock(rdp); 555 555 wake_nocb_gp_defer(rdp, RCU_NOCB_WAKE_LAZY, 556 556 TPS("WakeLazy")); 557 - } else if (!irqs_disabled_flags(flags) && cpu_online(rdp->cpu)) { 557 + } else if (!irqs_disabled_flags(flags)) { 558 558 /* ... if queue was empty ... */ 559 559 rcu_nocb_unlock(rdp); 560 560 wake_nocb_gp(rdp, false); 561 561 trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, 562 562 TPS("WakeEmpty")); 563 563 } else { 564 - /* 565 - * Don't do the wake-up upfront on fragile paths. 566 - * Also offline CPUs can't call swake_up_one_online() from 567 - * (soft-)IRQs. Rely on the final deferred wake-up from 568 - * rcutree_report_cpu_dead() 569 - */ 570 564 rcu_nocb_unlock(rdp); 571 565 wake_nocb_gp_defer(rdp, RCU_NOCB_WAKE, 572 566 TPS("WakeEmptyIsDeferred"));
+1 -1
kernel/rcu/tree_plugin.h
··· 29 29 (IS_ENABLED(CONFIG_HOTPLUG_CPU) && lockdep_is_cpus_held()) || 30 30 lockdep_is_held(&rdp->nocb_lock) || 31 31 lockdep_is_held(&rcu_state.nocb_mutex) || 32 - (!(IS_ENABLED(CONFIG_PREEMPT_COUNT) && preemptible()) && 32 + ((!(IS_ENABLED(CONFIG_PREEMPT_COUNT) && preemptible()) || softirq_count()) && 33 33 rdp == this_cpu_ptr(&rcu_data)) || 34 34 rcu_current_is_nocb_kthread(rdp)), 35 35 "Unsafe read of RCU_NOCB offloaded state"
+2 -2
kernel/rcu/tree_stall.h
··· 435 435 rsr.cputime_system = kcpustat_field(kcsp, CPUTIME_SYSTEM, cpu); 436 436 437 437 pr_err("\t hardirqs softirqs csw/system\n"); 438 - pr_err("\t number: %8ld %10d %12lld\n", 439 - kstat_cpu_irqs_sum(cpu) - rsrp->nr_hardirqs, 438 + pr_err("\t number: %8lld %10d %12lld\n", 439 + kstat_cpu_irqs_sum(cpu) + arch_irq_stat_cpu(cpu) - rsrp->nr_hardirqs, 440 440 kstat_cpu_softirqs_sum(cpu) - rsrp->nr_softirqs, 441 441 nr_context_switches_cpu(cpu) - rsrp->nr_csw); 442 442 pr_err("\tcputime: %8lld %10lld %12lld ==> %d(ms)\n",
+5
rust/kernel/sync/rcu.rs
··· 17 17 18 18 impl Guard { 19 19 /// Acquires the RCU read side lock and returns a guard. 20 + #[inline] 20 21 pub fn new() -> Self { 21 22 // SAFETY: An FFI call with no additional requirements. 22 23 unsafe { bindings::rcu_read_lock() }; ··· 26 25 } 27 26 28 27 /// Explicitly releases the RCU read side lock. 28 + #[inline] 29 29 pub fn unlock(self) {} 30 30 } 31 31 32 32 impl Default for Guard { 33 + #[inline] 33 34 fn default() -> Self { 34 35 Self::new() 35 36 } 36 37 } 37 38 38 39 impl Drop for Guard { 40 + #[inline] 39 41 fn drop(&mut self) { 40 42 // SAFETY: By the type invariants, the RCU read side is locked, so it is ok to unlock it. 41 43 unsafe { bindings::rcu_read_unlock() }; ··· 46 42 } 47 43 48 44 /// Acquires the RCU read side lock. 45 + #[inline] 49 46 pub fn read_lock() -> Guard { 50 47 Guard::new() 51 48 }
+2
scripts/checkpatch.pl
··· 839 839 "kunmap" => "kunmap_local", 840 840 "kmap_atomic" => "kmap_local_page", 841 841 "kunmap_atomic" => "kunmap_local", 842 + "srcu_read_lock_lite" => "srcu_read_lock_fast", 843 + "srcu_read_unlock_lite" => "srcu_read_unlock_fast", 842 844 ); 843 845 844 846 #Create a search pattern for all these strings to speed up a loop below
+1 -1
tools/testing/selftests/rcutorture/bin/console-badness.sh
··· 10 10 # 11 11 # Authors: Paul E. McKenney <paulmck@kernel.org> 12 12 13 - grep -E 'Badness|WARNING:|Warn|BUG|===========|BUG: KCSAN:|Call Trace:|Oops:|detected stalls on CPUs/tasks:|self-detected stall on CPU|Stall ended before state dump start|\?\?\? Writer stall state|rcu_.*kthread starved for|!!!' | 13 + grep -E 'Badness|WARNING:|Warn|BUG|===========|BUG: KCSAN:|Call Trace:|Call trace:|Oops:|detected stalls on CPUs/tasks:|self-detected stall on CPU|Stall ended before state dump start|\?\?\? Writer stall state|rcu_.*kthread starved for|!!!' | 14 14 grep -v 'ODEBUG: ' | 15 15 grep -v 'This means that this is a DEBUG kernel and it is' | 16 16 grep -v 'Warning: unable to open an initial console' |
+1 -1
tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
··· 73 73 cp $T/KcList $resdir/ConfigFragment 74 74 75 75 base_resdir=`echo $resdir | sed -e 's/\.[0-9]\+$//'` 76 - if test "$base_resdir" != "$resdir" && test -f $base_resdir/bzImage && test -f $base_resdir/vmlinux 76 + if test "$base_resdir" != "$resdir" && (test -f $base_resdir/bzImage || test -f $base_resdir/Image) && test -f $base_resdir/vmlinux 77 77 then 78 78 # Rerunning previous test, so use that test's kernel. 79 79 QEMU="`identify_qemu $base_resdir/vmlinux`"
+1 -1
tools/testing/selftests/rcutorture/bin/parse-console.sh
··· 148 148 summary="$summary KCSAN: $n_kcsan" 149 149 fi 150 150 fi 151 - n_calltrace=`grep -c 'Call Trace:' $file` 151 + n_calltrace=`grep -Ec 'Call Trace:|Call trace:' $file` 152 152 if test "$n_calltrace" -ne 0 153 153 then 154 154 summary="$summary Call Traces: $n_calltrace"
+41 -1
tools/testing/selftests/rcutorture/bin/srcu_lockdep.sh
··· 39 39 shift 40 40 done 41 41 42 - err= 43 42 nerrs=0 43 + 44 + # Test lockdep's handling of deadlocks. 44 45 for d in 0 1 45 46 do 46 47 for t in 0 1 2 ··· 53 52 tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 5s --configs "SRCU-P" --kconfig "CONFIG_FORCE_NEED_SRCU_NMI_SAFE=y" --bootargs "rcutorture.test_srcu_lockdep=$val rcutorture.reader_flavor=0x2" --trust-make --datestamp "$ds/$val" > "$T/kvm.sh.out" 2>&1 54 53 ret=$? 55 54 mv "$T/kvm.sh.out" "$RCUTORTURE/res/$ds/$val" 55 + if ! grep -q '^CONFIG_PROVE_LOCKING=y' .config 56 + then 57 + echo "rcu_torture_init_srcu_lockdep:Error: CONFIG_PROVE_LOCKING disabled in rcutorture SRCU-P scenario" 58 + nerrs=$((nerrs+1)) 59 + err=1 60 + fi 56 61 if test "$d" -ne 0 && test "$ret" -eq 0 57 62 then 58 63 err=1 ··· 78 71 done 79 72 done 80 73 done 74 + 75 + # Test lockdep-enabled testing of mixed SRCU readers. 76 + for val in 0x1 0xf 77 + do 78 + err= 79 + tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 5s --configs "SRCU-P" --kconfig "CONFIG_FORCE_NEED_SRCU_NMI_SAFE=y" --bootargs "rcutorture.reader_flavor=$val" --trust-make --datestamp "$ds/$val" > "$T/kvm.sh.out" 2>&1 80 + ret=$? 81 + mv "$T/kvm.sh.out" "$RCUTORTURE/res/$ds/$val" 82 + if ! grep -q '^CONFIG_PROVE_LOCKING=y' .config 83 + then 84 + echo "rcu_torture_init_srcu_lockdep:Error: CONFIG_PROVE_LOCKING disabled in rcutorture SRCU-P scenario" 85 + nerrs=$((nerrs+1)) 86 + err=1 87 + fi 88 + if test "$val" -eq 0xf && test "$ret" -eq 0 89 + then 90 + err=1 91 + echo -n Unexpected success for > "$RCUTORTURE/res/$ds/$val/kvm.sh.err" 92 + fi 93 + if test "$val" -eq 0x1 && test "$ret" -ne 0 94 + then 95 + err=1 96 + echo -n Unexpected failure for > "$RCUTORTURE/res/$ds/$val/kvm.sh.err" 97 + fi 98 + if test -n "$err" 99 + then 100 + grep "rcu_torture_init_srcu_lockdep: test_srcu_lockdep = " "$RCUTORTURE/res/$ds/$val/SRCU-P/console.log" | sed -e 's/^.*rcu_torture_init_srcu_lockdep://' >> "$RCUTORTURE/res/$ds/$val/kvm.sh.err" 101 + cat "$RCUTORTURE/res/$ds/$val/kvm.sh.err" 102 + nerrs=$((nerrs+1)) 103 + fi 104 + done 105 + 106 + # Set up exit code. 81 107 if test "$nerrs" -ne 0 82 108 then 83 109 exit 1
+81 -8
tools/testing/selftests/rcutorture/bin/torture.sh
··· 51 51 do_rcuscale=yes 52 52 do_refscale=yes 53 53 do_kvfree=yes 54 + do_normal=yes 55 + explicit_normal=no 54 56 do_kasan=yes 55 57 do_kcsan=no 56 58 do_clocksourcewd=yes 57 59 do_rt=yes 58 60 do_rcutasksflavors=yes 59 61 do_srcu_lockdep=yes 62 + do_rcu_rust=no 60 63 61 64 # doyesno - Helper function for yes/no arguments 62 65 function doyesno () { ··· 90 87 echo " --do-rcutorture / --do-no-rcutorture / --no-rcutorture" 91 88 echo " --do-refscale / --do-no-refscale / --no-refscale" 92 89 echo " --do-rt / --do-no-rt / --no-rt" 90 + echo " --do-rcu-rust / --do-no-rcu-rust / --no-rcu-rust" 93 91 echo " --do-scftorture / --do-no-scftorture / --no-scftorture" 94 92 echo " --do-srcu-lockdep / --do-no-srcu-lockdep / --no-srcu-lockdep" 95 93 echo " --duration [ <minutes> | <hours>h | <days>d ]" ··· 132 128 do_refscale=yes 133 129 do_rt=yes 134 130 do_kvfree=yes 131 + do_normal=yes 132 + explicit_normal=no 135 133 do_kasan=yes 136 134 do_kcsan=yes 137 135 do_clocksourcewd=yes ··· 167 161 do_refscale=no 168 162 do_rt=no 169 163 do_kvfree=no 164 + do_normal=no 165 + explicit_normal=no 170 166 do_kasan=no 171 167 do_kcsan=no 172 168 do_clocksourcewd=no 173 169 do_srcu_lockdep=no 170 + ;; 171 + --do-normal|--do-no-normal|--no-normal) 172 + do_normal=`doyesno "$1" --do-normal` 173 + explicit_normal=yes 174 174 ;; 175 175 --do-rcuscale|--do-no-rcuscale|--no-rcuscale) 176 176 do_rcuscale=`doyesno "$1" --do-rcuscale` ··· 192 180 ;; 193 181 --do-rt|--do-no-rt|--no-rt) 194 182 do_rt=`doyesno "$1" --do-rt` 183 + ;; 184 + --do-rcu-rust|--do-no-rcu-rust|--no-rcu-rust) 185 + do_rcu_rust=`doyesno "$1" --do-rcu-rust` 195 186 ;; 196 187 --do-scftorture|--do-no-scftorture|--no-scftorture) 197 188 do_scftorture=`doyesno "$1" --do-scftorture` ··· 256 241 257 242 echo " --- " $scriptname $args | tee -a $T/log 258 243 echo " --- Results directory: " $ds | tee -a $T/log 244 + 245 + if test "$do_normal" = "no" && test "$do_kasan" = "no" && test "$do_kcsan" = "no" 246 + then 247 + # Match old scripts so that "--do-none --do-rcutorture" does 248 + # normal rcutorture testing, but no KASAN or KCSAN testing. 249 + if test $explicit_normal = yes 250 + then 251 + echo " --- Everything disabled, so explicit --do-normal overridden" | tee -a $T/log 252 + fi 253 + do_normal=yes 254 + fi 259 255 260 256 # Calculate rcutorture defaults and apportion time 261 257 if test -z "$configs_rcutorture" ··· 358 332 local kcsan_kmake_tag= 359 333 local flavor=$1 360 334 shift 361 - curflavor=$flavor 362 - torture_one "$@" 363 - mv $T/last-resdir $T/last-resdir-nodebug || : 335 + if test "$do_normal" = "yes" 336 + then 337 + curflavor=$flavor 338 + torture_one "$@" 339 + mv $T/last-resdir $T/last-resdir-nodebug || : 340 + fi 364 341 if test "$do_kasan" = "yes" 365 342 then 366 343 curflavor=${flavor}-kasan ··· 477 448 478 449 if test "$do_rt" = "yes" 479 450 then 480 - # With all post-boot grace periods forced to normal. 481 - torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcupdate.rcu_normal=1" 482 - torture_set "rcurttorture" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --trust-make 451 + # In both runs, disable testing of RCU priority boosting because 452 + # -rt doesn't like its interaction with testing of callback 453 + # flooding. 454 + 455 + # With all post-boot grace periods forced to normal (default for PREEMPT_RT). 456 + torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcutorture.test_boost=0 rcutorture.preempt_duration=0" 457 + torture_set "rcurttorture" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --kconfig "CONFIG_PREEMPT_RT=y CONFIG_EXPERT=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=y CONFIG_RCU_NOCB_CPU=y" --trust-make 483 458 484 459 # With all post-boot grace periods forced to expedited. 485 - torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcupdate.rcu_expedited=1" 486 - torture_set "rcurttorture-exp" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --trust-make 460 + torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcutorture.test_boost=0 rcupdate.rcu_normal_after_boot=0 rcupdate.rcu_expedited=1 rcutorture.preempt_duration=0" 461 + torture_set "rcurttorture-exp" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --kconfig "CONFIG_PREEMPT_RT=y CONFIG_EXPERT=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_FULL=y CONFIG_RCU_NOCB_CPU=y" --trust-make 462 + fi 463 + 464 + if test "$do_rcu_rust" = "yes" 465 + then 466 + echo " --- do-rcu-rust:" Start `date` | tee -a $T/log 467 + rrdir="tools/testing/selftests/rcutorture/res/$ds/results-rcu-rust" 468 + mkdir -p "$rrdir" 469 + echo " --- make LLVM=1 rustavailable " | tee -a $rrdir/log > $rrdir/rustavailable.out 470 + make LLVM=1 rustavailable > $T/rustavailable.out 2>&1 471 + retcode=$? 472 + echo $retcode > $rrdir/rustavailable.exitcode 473 + cat $T/rustavailable.out | tee -a $rrdir/log >> $rrdir/rustavailable.out 2>&1 474 + buildphase=rustavailable 475 + if test "$retcode" -eq 0 476 + then 477 + echo " --- Running 'make mrproper' in order to run kunit." | tee -a $rrdir/log > $rrdir/mrproper.out 478 + make mrproper > $rrdir/mrproper.out 2>&1 479 + retcode=$? 480 + echo $retcode > $rrdir/mrproper.exitcode 481 + buildphase=mrproper 482 + fi 483 + if test "$retcode" -eq 0 484 + then 485 + echo " --- Running rust_doctests_kernel." | tee -a $rrdir/log > $rrdir/rust_doctests_kernel.out 486 + ./tools/testing/kunit/kunit.py run --make_options LLVM=1 --make_options CLIPPY=1 --arch arm64 --kconfig_add CONFIG_SMP=y --kconfig_add CONFIG_WERROR=y --kconfig_add CONFIG_RUST=y rust_doctests_kernel >> $rrdir/rust_doctests_kernel.out 2>&1 487 + # @@@ Remove "--arch arm64" in order to test on native architecture? 488 + # @@@ Analyze $rrdir/rust_doctests_kernel.out contents? 489 + retcode=$? 490 + echo $retcode > $rrdir/rust_doctests_kernel.exitcode 491 + buildphase=rust_doctests_kernel 492 + fi 493 + if test "$retcode" -eq 0 494 + then 495 + echo "rcu-rust($retcode)" $rrdir >> $T/successes 496 + echo Success >> $rrdir/log 497 + else 498 + echo "rcu-rust($retcode)" $rrdir >> $T/failures 499 + echo " --- rcu-rust Test summary:" >> $rrdir/log 500 + echo " --- Summary: Exit code $retcode from $buildphase, see $rrdir/$buildphase.out" >> $rrdir/log 501 + fi 487 502 fi 488 503 489 504 if test "$do_srcu_lockdep" = "yes"
-2
tools/testing/selftests/rcutorture/configs/rcu/TREE01
··· 8 8 CONFIG_NO_HZ_FULL=n 9 9 CONFIG_RCU_TRACE=y 10 10 CONFIG_HOTPLUG_CPU=y 11 - CONFIG_MAXSMP=y 12 - CONFIG_CPUMASK_OFFSTACK=y 13 11 CONFIG_RCU_NOCB_CPU=y 14 12 CONFIG_DEBUG_LOCK_ALLOC=n 15 13 CONFIG_RCU_BOOST=n
+1 -1
tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot
··· 1 - maxcpus=8 nr_cpus=43 1 + maxcpus=8 nr_cpus=17 2 2 rcutree.gp_preinit_delay=3 3 3 rcutree.gp_init_delay=3 4 4 rcutree.gp_cleanup_delay=3