tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

ring-buffer: Make wake once of ring_buffer_wait() more robust

The default behavior of ring_buffer_wait() when passed a NULL "cond"
parameter is to exit the function the first time it is woken up. The
current implementation uses a counter that starts at zero and when it is
greater than one it exits the wait_event_interruptible().

But this relies on the internal working of wait_event_interruptible() as
that code basically has:

if (cond)
return;
prepare_to_wait();
if (!cond)
schedule();
finish_wait();

That is, cond is called twice before it sleeps. The default cond of
ring_buffer_wait() needs to account for that and wait for its counter to
increment twice before exiting.

Instead, use the seq/atomic_inc logic that is used by the tracing code
that calls this function. Add an atomic_t seq to rb_irq_work and when cond
is NULL, have the default callback take a descriptor as its data that
holds the rbwork and the value of the seq when it started.

The wakeups will now increment the rbwork->seq and the cond callback will
simply check if that number is different, and no longer have to rely on
the implementation of wait_event_interruptible().

Link: https://lore.kernel.org/linux-trace-kernel/20240315063115.6cb5d205@gandalf.local.home

Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Fixes: 7af9ded0c2ca ("ring-buffer: Use wait_event_interruptible() in ring_buffer_wait()")
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>

Steven Rostedt (Google) b70f2938 f1e30cb6

+21 -13
+21 -13
kernel/trace/ring_buffer.c
··· 384 384 struct irq_work work; 385 385 wait_queue_head_t waiters; 386 386 wait_queue_head_t full_waiters; 387 + atomic_t seq; 387 388 bool waiters_pending; 388 389 bool full_waiters_pending; 389 390 bool wakeup_full; ··· 754 753 { 755 754 struct rb_irq_work *rbwork = container_of(work, struct rb_irq_work, work); 756 755 756 + /* For waiters waiting for the first wake up */ 757 + (void)atomic_fetch_inc_release(&rbwork->seq); 758 + 757 759 wake_up_all(&rbwork->waiters); 758 760 if (rbwork->full_waiters_pending || rbwork->wakeup_full) { 759 761 /* Only cpu_buffer sets the above flags */ ··· 885 881 return false; 886 882 } 887 883 884 + struct rb_wait_data { 885 + struct rb_irq_work *irq_work; 886 + int seq; 887 + }; 888 + 888 889 /* 889 890 * The default wait condition for ring_buffer_wait() is to just to exit the 890 891 * wait loop the first time it is woken up. 891 892 */ 892 893 static bool rb_wait_once(void *data) 893 894 { 894 - long *once = data; 895 + struct rb_wait_data *rdata = data; 896 + struct rb_irq_work *rbwork = rdata->irq_work; 895 897 896 - /* wait_event() actually calls this twice before scheduling*/ 897 - if (*once > 1) 898 - return true; 899 - 900 - (*once)++; 901 - return false; 898 + return atomic_read_acquire(&rbwork->seq) != rdata->seq; 902 899 } 903 900 904 901 /** ··· 920 915 struct ring_buffer_per_cpu *cpu_buffer; 921 916 struct wait_queue_head *waitq; 922 917 struct rb_irq_work *rbwork; 923 - long once = 0; 918 + struct rb_wait_data rdata; 924 919 int ret = 0; 925 - 926 - if (!cond) { 927 - cond = rb_wait_once; 928 - data = &once; 929 - } 930 920 931 921 /* 932 922 * Depending on what the caller is waiting for, either any ··· 943 943 waitq = &rbwork->full_waiters; 944 944 else 945 945 waitq = &rbwork->waiters; 946 + 947 + /* Set up to exit loop as soon as it is woken */ 948 + if (!cond) { 949 + cond = rb_wait_once; 950 + rdata.irq_work = rbwork; 951 + rdata.seq = atomic_read_acquire(&rbwork->seq); 952 + data = &rdata; 953 + } 946 954 947 955 ret = wait_event_interruptible((*waitq), 948 956 rb_wait_cond(rbwork, buffer, cpu, full, cond, data));