tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

io_uring: delay sqarray static branch disablement

io_key_has_sqarray static branch can be easily switched on/off by the
user every time patching the kernel. That can be very disruptive as it
might require heavy synchronisation across all CPUs. Use deferred static
keys, which can rate-limit it by deferring, batching and potentially
effectively eliminating dec+inc pairs.

Fixes: 9b296c625ac1d ("io_uring: static_key for !IORING_SETUP_NO_SQARRAY")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

authored by

Pavel Begunkov and committed by
Jens Axboe 56112578 6b34f8ed

+4 -4
+4 -4
io_uring/io_uring.c
··· 119 119 static void io_queue_sqe(struct io_kiocb *req, unsigned int extra_flags); 120 120 static void __io_req_caches_free(struct io_ring_ctx *ctx); 121 121 122 - static __read_mostly DEFINE_STATIC_KEY_FALSE(io_key_has_sqarray); 122 + static __read_mostly DEFINE_STATIC_KEY_DEFERRED_FALSE(io_key_has_sqarray, HZ); 123 123 124 124 struct kmem_cache *req_cachep; 125 125 static struct workqueue_struct *iou_wq __ro_after_init; ··· 1978 1978 unsigned mask = ctx->sq_entries - 1; 1979 1979 unsigned head = ctx->cached_sq_head++ & mask; 1980 1980 1981 - if (static_branch_unlikely(&io_key_has_sqarray) && 1981 + if (static_branch_unlikely(&io_key_has_sqarray.key) && 1982 1982 (!(ctx->flags & IORING_SETUP_NO_SQARRAY))) { 1983 1983 head = READ_ONCE(ctx->sq_array[head]); 1984 1984 if (unlikely(head >= ctx->sq_entries)) { ··· 2173 2173 io_rings_free(ctx); 2174 2174 2175 2175 if (!(ctx->flags & IORING_SETUP_NO_SQARRAY)) 2176 - static_branch_dec(&io_key_has_sqarray); 2176 + static_branch_slow_dec_deferred(&io_key_has_sqarray); 2177 2177 2178 2178 percpu_ref_exit(&ctx->refs); 2179 2179 free_uid(ctx->user); ··· 2946 2946 ctx->clock_offset = 0; 2947 2947 2948 2948 if (!(ctx->flags & IORING_SETUP_NO_SQARRAY)) 2949 - static_branch_inc(&io_key_has_sqarray); 2949 + static_branch_deferred_inc(&io_key_has_sqarray); 2950 2950 2951 2951 if ((ctx->flags & IORING_SETUP_DEFER_TASKRUN) && 2952 2952 !(ctx->flags & IORING_SETUP_IOPOLL))