tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

io_uring: remove async request cache

io_req_complete_post() was a sole user of ->locked_free_list, but
since we just gutted the function, the cache is not used anymore and
can be removed.

->locked_free_list served as an asynhronous counterpart of the main
request (i.e. struct io_kiocb) cache for all unlocked cases like io-wq.
Now they're all forced to be completed into the main cache directly,
off of the normal completion path or via io_free_req().

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/7bffccd213e370abd4de480e739d8b08ab6c1326.1712331455.git.asml.silence@gmail.com
Reviewed-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

authored by

Pavel Begunkov and committed by
Jens Axboe d9713ad3 de96e9ae

-26
-4
include/linux/io_uring_types.h
··· 346 346 347 347 spinlock_t completion_lock; 348 348 349 - /* IRQ completion list, under ->completion_lock */ 350 - unsigned int locked_free_nr; 351 - struct io_wq_work_list locked_free_list; 352 - 353 349 struct list_head io_buffers_comp; 354 350 struct list_head cq_overflow_list; 355 351 struct io_hash_table cancel_table;
-22
io_uring/io_uring.c
··· 334 334 init_llist_head(&ctx->work_llist); 335 335 INIT_LIST_HEAD(&ctx->tctx_list); 336 336 ctx->submit_state.free_list.next = NULL; 337 - INIT_WQ_LIST(&ctx->locked_free_list); 338 337 INIT_HLIST_HEAD(&ctx->waitid_list); 339 338 #ifdef CONFIG_FUTEX 340 339 INIT_HLIST_HEAD(&ctx->futex_list); ··· 987 988 memset(&req->big_cqe, 0, sizeof(req->big_cqe)); 988 989 } 989 990 990 - static void io_flush_cached_locked_reqs(struct io_ring_ctx *ctx, 991 - struct io_submit_state *state) 992 - { 993 - spin_lock(&ctx->completion_lock); 994 - wq_list_splice(&ctx->locked_free_list, &state->free_list); 995 - ctx->locked_free_nr = 0; 996 - spin_unlock(&ctx->completion_lock); 997 - } 998 - 999 991 /* 1000 992 * A request might get retired back into the request caches even before opcode 1001 993 * handlers and io_issue_sqe() are done with it, e.g. inline completion path. ··· 999 1009 gfp_t gfp = GFP_KERNEL | __GFP_NOWARN; 1000 1010 void *reqs[IO_REQ_ALLOC_BATCH]; 1001 1011 int ret; 1002 - 1003 - /* 1004 - * If we have more than a batch's worth of requests in our IRQ side 1005 - * locked cache, grab the lock and move them over to our submission 1006 - * side cache. 1007 - */ 1008 - if (data_race(ctx->locked_free_nr) > IO_COMPL_BATCH) { 1009 - io_flush_cached_locked_reqs(ctx, &ctx->submit_state); 1010 - if (!io_req_cache_empty(ctx)) 1011 - return true; 1012 - } 1013 1012 1014 1013 ret = kmem_cache_alloc_bulk(req_cachep, gfp, ARRAY_SIZE(reqs), reqs); 1015 1014 ··· 2633 2654 int nr = 0; 2634 2655 2635 2656 mutex_lock(&ctx->uring_lock); 2636 - io_flush_cached_locked_reqs(ctx, &ctx->submit_state); 2637 2657 2638 2658 while (!io_req_cache_empty(ctx)) { 2639 2659 req = io_extract_req(ctx);