tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

hrtimer: Avoid unnecessary SMP function calls in clock_was_set()

Setting of clocks triggers an unconditional SMP function call on all online
CPUs to reprogram the clock event device.

However, only some clocks have their offsets updated and therefore
potentially require a reprogram. That's CLOCK_REALTIME and CLOCK_TAI and in
the case of resume (delayed sleep time injection) also CLOCK_BOOTTIME.

Instead of sending an IPI unconditionally, check each per CPU hrtimer base
whether it has active timers in the affected clock bases which are
indicated by the caller in the @bases argument of clock_was_set().

If that's not the case, skip the IPI and update the offsets remotely which
ensures that any subsequently armed timers on the affected clocks are
evaluated with the correct offsets.

[ tglx: Adopted to the new bases argument, removed the softirq_active
check, added comment, fixed up stale comment ]

Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/20210713135158.787536542@linutronix.de

authored by

Marcelo Tosatti and committed by
Thomas Gleixner 81d741d3 17a1b882

+33 -2
+33 -2
kernel/time/hrtimer.c
··· 882 882 */ 883 883 void clock_was_set(unsigned int bases) 884 884 { 885 + cpumask_var_t mask; 886 + int cpu; 887 + 885 888 if (!hrtimer_hres_active() && !tick_nohz_active) 886 889 goto out_timerfd; 887 890 888 - /* Retrigger the CPU local events everywhere */ 889 - on_each_cpu(retrigger_next_event, NULL, 1); 891 + if (!zalloc_cpumask_var(&mask, GFP_KERNEL)) { 892 + on_each_cpu(retrigger_next_event, NULL, 1); 893 + goto out_timerfd; 894 + } 895 + 896 + /* Avoid interrupting CPUs if possible */ 897 + cpus_read_lock(); 898 + for_each_online_cpu(cpu) { 899 + struct hrtimer_cpu_base *cpu_base = &per_cpu(hrtimer_bases, cpu); 900 + unsigned long flags; 901 + 902 + raw_spin_lock_irqsave(&cpu_base->lock, flags); 903 + /* 904 + * Only send the IPI when there are timers queued in one of 905 + * the affected clock bases. Otherwise update the base 906 + * remote to ensure that the next enqueue of a timer on 907 + * such a clock base will see the correct offsets. 908 + */ 909 + if (cpu_base->active_bases & bases) 910 + cpumask_set_cpu(cpu, mask); 911 + else 912 + hrtimer_update_base(cpu_base); 913 + raw_spin_unlock_irqrestore(&cpu_base->lock, flags); 914 + } 915 + 916 + preempt_disable(); 917 + smp_call_function_many(mask, retrigger_next_event, NULL, 1); 918 + preempt_enable(); 919 + cpus_read_unlock(); 920 + free_cpumask_var(mask); 890 921 891 922 out_timerfd: 892 923 timerfd_clock_was_set();