tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

io_uring: fix IOPOLL with passthrough I/O

A previous commit improving IOPOLL made an incorrect assumption that
task_work isn't used with IOPOLL. This can cause crashes when doing
passthrough I/O on nvme, where queueing the completion task_work will
trample on the same memory that holds the completed list of requests.

Fix it up by shuffling the members around, so we're not sharing any
parts that end up getting used in this path.

Fixes: 3c7d76d6128a ("io_uring: IOPOLL polling improvements")
Reported-by: Yi Zhang <yi.zhang@redhat.com>
Link: https://lore.kernel.org/linux-block/CAHj4cs_SLPj9v9w5MgfzHKy+983enPx3ZQY2kMuMJ1202DBefw@mail.gmail.com/
Tested-by: Yi Zhang <yi.zhang@redhat.com>
Cc: Ming Lei <ming.lei@redhat.com>
Reviewed-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Jens Axboe 697a5284 d6406c45

+7 -9
+4 -7
include/linux/io_uring_types.h
··· 713 713 atomic_t refs; 714 714 bool cancel_seq_set; 715 715 716 - /* 717 - * IOPOLL doesn't use task_work, so use the ->iopoll_node list 718 - * entry to manage pending iopoll requests. 719 - */ 720 716 union { 721 717 struct io_task_work io_task_work; 722 - struct list_head iopoll_node; 718 + /* For IOPOLL setup queues, with hybrid polling */ 719 + u64 iopoll_start; 723 720 }; 724 721 725 722 union { ··· 725 728 * poll 726 729 */ 727 730 struct hlist_node hash_node; 728 - /* For IOPOLL setup queues, with hybrid polling */ 729 - u64 iopoll_start; 731 + /* IOPOLL completion handling */ 732 + struct list_head iopoll_node; 730 733 /* for private io_kiocb freeing */ 731 734 struct rcu_head rcu_head; 732 735 };
+3 -2
io_uring/rw.c
··· 1296 1296 struct io_comp_batch *iob, unsigned int poll_flags) 1297 1297 { 1298 1298 struct io_ring_ctx *ctx = req->ctx; 1299 - u64 runtime, sleep_time; 1299 + u64 runtime, sleep_time, iopoll_start; 1300 1300 int ret; 1301 1301 1302 + iopoll_start = READ_ONCE(req->iopoll_start); 1302 1303 sleep_time = io_hybrid_iopoll_delay(ctx, req); 1303 1304 ret = io_uring_classic_poll(req, iob, poll_flags); 1304 - runtime = ktime_get_ns() - req->iopoll_start - sleep_time; 1305 + runtime = ktime_get_ns() - iopoll_start - sleep_time; 1305 1306 1306 1307 /* 1307 1308 * Use minimum sleep time if we're polling devices with different