Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

io-wq: write next_work before dropping acct_lock

Commit 361aee450c6e ("io-wq: add intermediate work step between pending
list and active work") closed a race between a cancellation and the work
being removed from the wq for execution. To ensure the request is
always reachable by the cancellation, we need to move it within the wq
lock, which also synchronizes the cancellation. But commit
42abc95f05bf ("io-wq: decouple work_list protection from the big
wqe->lock") replaced the wq lock here and accidentally reintroduced the
race by releasing the acct_lock too early.

In other words:

worker | cancellation
work = io_get_next_work() |
raw_spin_unlock(&acct->lock); |
|
| io_acct_cancel_pending_work
| io_wq_worker_cancel()
worker->next_work = work

Using acct_lock is still enough since we synchronize on it on
io_acct_cancel_pending_work.

Fixes: 42abc95f05bf ("io-wq: decouple work_list protection from the big wqe->lock")
Signed-off-by: Gabriel Krisman Bertazi <krisman@suse.de>
Link: https://lore.kernel.org/r/20240416021054.3940-2-krisman@suse.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>

authored by

Gabriel Krisman Bertazi and committed by
Jens Axboe
068c27e3 c4ce0ab2

+8 -5
+8 -5
io_uring/io-wq.c
··· 564 564 * clear the stalled flag. 565 565 */ 566 566 work = io_get_next_work(acct, worker); 567 - raw_spin_unlock(&acct->lock); 568 567 if (work) { 569 - __io_worker_busy(wq, worker); 570 - 571 568 /* 572 569 * Make sure cancelation can find this, even before 573 570 * it becomes the active work. That avoids a window ··· 575 578 raw_spin_lock(&worker->lock); 576 579 worker->next_work = work; 577 580 raw_spin_unlock(&worker->lock); 578 - } else { 579 - break; 580 581 } 582 + 583 + raw_spin_unlock(&acct->lock); 584 + 585 + if (!work) 586 + break; 587 + 588 + __io_worker_busy(wq, worker); 589 + 581 590 io_assign_current_work(worker, work); 582 591 __set_current_state(TASK_RUNNING); 583 592