tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

locking/lock_events: Use this_cpu_add() when necessary

The kernel test robot has reported that the use of __this_cpu_add()
causes bug messages like:

BUG: using __this_cpu_add() in preemptible [00000000] code: ...

Given the imprecise nature of the count and the possibility of resetting
the count and doing the measurement again, this is not really a big
problem to use the unprotected __this_cpu_*() functions.

To make the preemption checking code happy, the this_cpu_*() functions
will be used if CONFIG_DEBUG_PREEMPT is defined.

The imprecise nature of the locking counts are also documented with
the suggestion that we should run the measurement a few times with the
counts reset in between to get a better picture of what is going on
under the hood.

Fixes: a8654596f0371 ("locking/rwsem: Enable lock event counting")
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

authored by

Waiman Long and committed by
Linus Torvalds 51816e9e 0a72ef89

+40 -2
+40 -2
kernel/locking/lock_events.h
··· 31 31 DECLARE_PER_CPU(unsigned long, lockevents[lockevent_num]); 32 32 33 33 /* 34 + * The purpose of the lock event counting subsystem is to provide a low 35 + * overhead way to record the number of specific locking events by using 36 + * percpu counters. It is the percpu sum that matters, not specifically 37 + * how many of them happens in each cpu. 38 + * 39 + * It is possible that the same percpu counter may be modified in both 40 + * the process and interrupt contexts. For architectures that perform 41 + * percpu operation with multiple instructions, it is possible to lose 42 + * count if a process context percpu update is interrupted in the middle 43 + * and the same counter is updated in the interrupt context. Therefore, 44 + * the generated percpu sum may not be precise. The error, if any, should 45 + * be small and insignificant. 46 + * 47 + * For those architectures that do multi-instruction percpu operation, 48 + * preemption in the middle and moving the task to another cpu may cause 49 + * a larger error in the count. Again, this will be few and far between. 50 + * Given the imprecise nature of the count and the possibility of resetting 51 + * the count and doing the measurement again, this is not really a big 52 + * problem. 53 + * 54 + * To get a better picture of what is happening under the hood, it is 55 + * suggested that a few measurements should be taken with the counts 56 + * reset in between to stamp out outliner because of these possible 57 + * error conditions. 58 + * 59 + * To minimize overhead, we use __this_cpu_*() in all cases except when 60 + * CONFIG_DEBUG_PREEMPT is defined. In this particular case, this_cpu_*() 61 + * will be used to avoid the appearance of unwanted BUG messages. 62 + */ 63 + #ifdef CONFIG_DEBUG_PREEMPT 64 + #define lockevent_percpu_inc(x) this_cpu_inc(x) 65 + #define lockevent_percpu_add(x, v) this_cpu_add(x, v) 66 + #else 67 + #define lockevent_percpu_inc(x) __this_cpu_inc(x) 68 + #define lockevent_percpu_add(x, v) __this_cpu_add(x, v) 69 + #endif 70 + 71 + /* 34 72 * Increment the PV qspinlock statistical counters 35 73 */ 36 74 static inline void __lockevent_inc(enum lock_events event, bool cond) 37 75 { 38 76 if (cond) 39 - __this_cpu_inc(lockevents[event]); 77 + lockevent_percpu_inc(lockevents[event]); 40 78 } 41 79 42 80 #define lockevent_inc(ev) __lockevent_inc(LOCKEVENT_ ##ev, true) ··· 82 44 83 45 static inline void __lockevent_add(enum lock_events event, int inc) 84 46 { 85 - __this_cpu_add(lockevents[event], inc); 47 + lockevent_percpu_add(lockevents[event], inc); 86 48 } 87 49 88 50 #define lockevent_add(ev, c) __lockevent_add(LOCKEVENT_ ##ev, c)