tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

ring-buffer: Enforce read ordering of trace_buffer cpumask and buffers

On CPU hotplug, if it is the first time a trace_buffer sees a CPU, a
ring_buffer_per_cpu will be allocated and its corresponding bit toggled
in the cpumask. Many readers check this cpumask to know if they can
safely read the ring_buffer_per_cpu but they are doing so without memory
ordering and may observe the cpumask bit set while having NULL buffer
pointer.

Enforce the memory read ordering by sending an IPI to all online CPUs.
The hotplug path is a slow-path anyway and it saves us from adding read
barriers in numerous call sites.

Link: https://patch.msgid.link/20260401053659.3458961-1-vdonnefort@google.com
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Suggested-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>

authored by

Vincent Donnefort and committed by
Steven Rostedt (Google) 20ad8b08 23d1cfc0

+18 -1
+18 -1
kernel/trace/ring_buffer.c
··· 7722 7722 return 0; 7723 7723 } 7724 7724 7725 + static void rb_cpu_sync(void *data) 7726 + { 7727 + /* Not really needed, but documents what is happening */ 7728 + smp_rmb(); 7729 + } 7730 + 7725 7731 /* 7726 7732 * We only allocate new buffers, never free them if the CPU goes down. 7727 7733 * If we were to free the buffer, then the user would lose any trace that was in ··· 7766 7760 cpu); 7767 7761 return -ENOMEM; 7768 7762 } 7769 - smp_wmb(); 7763 + 7764 + /* 7765 + * Ensure trace_buffer readers observe the newly allocated 7766 + * ring_buffer_per_cpu before they check the cpumask. Instead of using a 7767 + * read barrier for all readers, send an IPI. 7768 + */ 7769 + if (unlikely(system_state == SYSTEM_RUNNING)) { 7770 + on_each_cpu(rb_cpu_sync, NULL, 1); 7771 + /* Not really needed, but documents what is happening */ 7772 + smp_wmb(); 7773 + } 7774 + 7770 7775 cpumask_set_cpu(cpu, buffer->cpumask); 7771 7776 return 0; 7772 7777 }