tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

watchdog: update saved interrupts during check

Currently, arch_touch_nmi_watchdog() causes an early return that skips
updating hrtimer_interrupts_saved. This leads to stale comparisons and
delayed lockup detection.

I found this issue because in our system the serial console is fairly
chatty. For example, the 8250 console driver frequently calls
touch_nmi_watchdog() via console_write(). If a CPU locks up after a timer
interrupt but before next watchdog check, we see the following sequence:

* watchdog_hardlockup_check() saves counter (e.g., 1000)
* Timer runs and updates the counter (1001)
* touch_nmi_watchdog() is called
* CPU locks up
* 10s pass: check() notices touch, returns early, skips update
* 10s pass: check() saves counter (1001)
* 10s pass: check() finally detects lockup

This delays detection to 30 seconds. With this fix, we detect the lockup
in 20 seconds.

Link: https://lkml.kernel.org/r/20260312-hardlockup-watchdog-fixes-v2-2-45bd8a0cc7ed@google.com
Signed-off-by: Mayank Rungta <mrungta@google.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Li Huafei <lihuafei1@huawei.com>
Cc: Max Kellermann <max.kellermann@ionos.com>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Stephane Erainan <eranian@google.com>
Cc: Wang Jinchao <wangjinchao600@gmail.com>
Cc: Yunhui Cui <cuiyunhui@bytedance.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

authored by

Mayank Rungta and committed by
Andrew Morton 746bb7fa 3e811cae

+13 -5
+13 -5
kernel/watchdog.c
··· 159 159 per_cpu(watchdog_hardlockup_touched, cpu) = true; 160 160 } 161 161 162 - static bool is_hardlockup(unsigned int cpu) 162 + static void watchdog_hardlockup_update(unsigned int cpu) 163 163 { 164 164 int hrint = atomic_read(&per_cpu(hrtimer_interrupts, cpu)); 165 - 166 - if (per_cpu(hrtimer_interrupts_saved, cpu) == hrint) 167 - return true; 168 165 169 166 /* 170 167 * NOTE: we don't need any fancy atomic_t or READ_ONCE/WRITE_ONCE ··· 169 172 * written/read by a single CPU. 170 173 */ 171 174 per_cpu(hrtimer_interrupts_saved, cpu) = hrint; 175 + } 172 176 173 - return false; 177 + static bool is_hardlockup(unsigned int cpu) 178 + { 179 + int hrint = atomic_read(&per_cpu(hrtimer_interrupts, cpu)); 180 + 181 + if (per_cpu(hrtimer_interrupts_saved, cpu) != hrint) { 182 + watchdog_hardlockup_update(cpu); 183 + return false; 184 + } 185 + 186 + return true; 174 187 } 175 188 176 189 static void watchdog_hardlockup_kick(void) ··· 198 191 unsigned long flags; 199 192 200 193 if (per_cpu(watchdog_hardlockup_touched, cpu)) { 194 + watchdog_hardlockup_update(cpu); 201 195 per_cpu(watchdog_hardlockup_touched, cpu) = false; 202 196 return; 203 197 }