tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

sched/fair: Move checking for nohz cpus after time check

Current code does.
- Read nohz.nr_cpus
- Check if the time has passed to do NOHZ idle balance

Instead do this.
- Check if the time has passed to do NOHZ idle balance
- Read nohz.nr_cpus

This will skip the read most of the time in normal system usage.
i.e when there are nohz.nr_cpus (system is not 100% busy).

Note that when there are no idle CPUs(100% busy), even if the flag gets
set to NOHZ_STATS_KICK | NOHZ_NEXT_KICK, find_new_ilb will fail and
there will be no NOHZ idle balance. In such cases there will be a very
narrow window where, kick_ilb will be called un-necessarily.
However current functionality is still retained.

Note: This patch doesn't solve any cacheline overheads. No improvement
in performance apart from saving a few cycles of reading nohz.nr_cpus

Reviewed-and-tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Link: https://patch.msgid.link/20260115073524.376643-2-sshegde@linux.ibm.com

authored by

Shrikanth Hegde and committed by
Peter Zijlstra 6b67c8a7 553255cc

+16 -7
+16 -7
kernel/sched/fair.c
··· 12441 12441 */ 12442 12442 nohz_balance_exit_idle(rq); 12443 12443 12444 + if (READ_ONCE(nohz.has_blocked_load) && 12445 + time_after(now, READ_ONCE(nohz.next_blocked))) 12446 + flags = NOHZ_STATS_KICK; 12447 + 12448 + /* 12449 + * Most of the time system is not 100% busy. i.e nohz.nr_cpus > 0 12450 + * Skip the read if time is not due. 12451 + * 12452 + * If none are in tickless mode, there maybe a narrow window 12453 + * (28 jiffies, HZ=1000) where flags maybe set and kick_ilb called. 12454 + * But idle load balancing is not done as find_new_ilb fails. 12455 + * That's very rare. So read nohz.nr_cpus only if time is due. 12456 + */ 12457 + if (time_before(now, nohz.next_balance)) 12458 + goto out; 12459 + 12444 12460 /* 12445 12461 * None are in tickless mode and hence no need for NOHZ idle load 12446 12462 * balancing: 12447 12463 */ 12448 12464 if (likely(!atomic_read(&nohz.nr_cpus))) 12449 12465 return; 12450 - 12451 - if (READ_ONCE(nohz.has_blocked_load) && 12452 - time_after(now, READ_ONCE(nohz.next_blocked))) 12453 - flags = NOHZ_STATS_KICK; 12454 - 12455 - if (time_before(now, nohz.next_balance)) 12456 - goto out; 12457 12466 12458 12467 if (rq->nr_running >= 2) { 12459 12468 flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK;