tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

io_uring: fix spurious drain flushing

io_queue_deferred() is not tolerant to spurious calls not completing
some requests. You can have an inflight drain-marked request and another
request that came after and got queued into the drain list. Now, if
io_queue_deferred() is called before the first request completes, it'll
check the 2nd req with req_need_defer(), find that there is no drain
flag set, and queue it for execution.

To make io_queue_deferred() work, it should at least check sequences for
the first request, and then we need also need to check if there is
another drain request creating another bubble.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/972bde11b7d4ef25b3f5e3fd34f80e4d2aa345b8.1746788718.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

authored by

Pavel Begunkov and committed by
Jens Axboe fde04c7e f979c205

+13 -1
+13 -1
io_uring/io_uring.c
··· 559 559 io_req_task_work_add(req); 560 560 } 561 561 562 + static bool io_drain_defer_seq(struct io_kiocb *req, u32 seq) 563 + { 564 + struct io_ring_ctx *ctx = req->ctx; 565 + 566 + return seq + READ_ONCE(ctx->cq_extra) != ctx->cached_cq_tail; 567 + } 568 + 562 569 static __cold noinline void io_queue_deferred(struct io_ring_ctx *ctx) 563 570 { 571 + bool drain_seen = false, first = true; 572 + 564 573 spin_lock(&ctx->completion_lock); 565 574 while (!list_empty(&ctx->defer_list)) { 566 575 struct io_defer_entry *de = list_first_entry(&ctx->defer_list, 567 576 struct io_defer_entry, list); 568 577 569 - if (req_need_defer(de->req, de->seq)) 578 + drain_seen |= de->req->flags & REQ_F_IO_DRAIN; 579 + if ((drain_seen || first) && io_drain_defer_seq(de->req, de->seq)) 570 580 break; 581 + 571 582 list_del_init(&de->list); 572 583 io_req_task_queue(de->req); 573 584 kfree(de); 585 + first = false; 574 586 } 575 587 spin_unlock(&ctx->completion_lock); 576 588 }