tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

io_uring: remove io_preinit_req()

Apart from setting ->ctx, io_preinit_req() zeroes a bunch of fields of a
request, from which only ->file_node is mandatory. Remove the function
and zero the entire request on first allocation. With that, we also need
to initialise ->ctx every time, which might be a good thing for
performance as now we're likely overwriting the entire cache line, and
so it can write combined and avoid RMW.

Suggested-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/ba5485dc913f1e275862ce88f5169d4ac4a33836.1746533807.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

authored by

Pavel Begunkov and committed by
Jens Axboe 9c2ff3f9 78967aab

+3 -19
+2 -19
io_uring/io_uring.c
··· 928 928 } 929 929 930 930 /* 931 - * Don't initialise the fields below on every allocation, but do that in 932 - * advance and keep them valid across allocations. 933 - */ 934 - static void io_preinit_req(struct io_kiocb *req, struct io_ring_ctx *ctx) 935 - { 936 - req->ctx = ctx; 937 - req->buf_node = NULL; 938 - req->file_node = NULL; 939 - req->link = NULL; 940 - req->async_data = NULL; 941 - /* not necessary, but safer to zero */ 942 - memset(&req->cqe, 0, sizeof(req->cqe)); 943 - memset(&req->big_cqe, 0, sizeof(req->big_cqe)); 944 - } 945 - 946 - /* 947 931 * A request might get retired back into the request caches even before opcode 948 932 * handlers and io_issue_sqe() are done with it, e.g. inline completion path. 949 933 * Because of that, io_alloc_req() should be called only under ->uring_lock ··· 936 952 __cold bool __io_alloc_req_refill(struct io_ring_ctx *ctx) 937 953 __must_hold(&ctx->uring_lock) 938 954 { 939 - gfp_t gfp = GFP_KERNEL | __GFP_NOWARN; 955 + gfp_t gfp = GFP_KERNEL | __GFP_NOWARN | __GFP_ZERO; 940 956 void *reqs[IO_REQ_ALLOC_BATCH]; 941 957 int ret; 942 958 ··· 957 973 while (ret--) { 958 974 struct io_kiocb *req = reqs[ret]; 959 975 960 - io_preinit_req(req, ctx); 961 976 io_req_add_to_cache(req, ctx); 962 977 } 963 978 return true; ··· 2032 2049 int personality; 2033 2050 u8 opcode; 2034 2051 2035 - /* req is partially pre-initialised, see io_preinit_req() */ 2052 + req->ctx = ctx; 2036 2053 req->opcode = opcode = READ_ONCE(sqe->opcode); 2037 2054 /* same numerical values with corresponding REQ_F_*, safe to copy */ 2038 2055 sqe_flags = READ_ONCE(sqe->flags);
+1
io_uring/notif.c
··· 112 112 113 113 if (unlikely(!io_alloc_req(ctx, &notif))) 114 114 return NULL; 115 + notif->ctx = ctx; 115 116 notif->opcode = IORING_OP_NOP; 116 117 notif->flags = 0; 117 118 notif->file = NULL;