tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

blk-mq: Defer freeing flush queue to SRCU callback

The freeing of the flush queue/request in blk_mq_exit_hctx() can race with
tag iterators that may still be accessing it. To prevent a potential
use-after-free, the deallocation should be deferred until after a grace
period. With this way, we can replace the big tags->lock in tags iterator
code path with srcu for solving the issue.

This patch introduces an SRCU-based deferred freeing mechanism for the
flush queue.

The changes include:
- Adding a `rcu_head` to `struct blk_flush_queue`.
- Creating a new callback function, `blk_free_flush_queue_callback`,
to handle the actual freeing.
- Replacing the direct call to `blk_free_flush_queue()` in
`blk_mq_exit_hctx()` with `call_srcu()`, using the `tags_srcu`
instance to ensure synchronization with tag iterators.

Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Yu Kuai <yukuai3@huawei.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

authored by

Ming Lei and committed by
Jens Axboe 135b8521 ad0d05db

+11 -1
+10 -1
block/blk-mq.c
··· 3912 3912 spin_unlock_irqrestore(&tags->lock, flags); 3913 3913 } 3914 3914 3915 + static void blk_free_flush_queue_callback(struct rcu_head *head) 3916 + { 3917 + struct blk_flush_queue *fq = 3918 + container_of(head, struct blk_flush_queue, rcu_head); 3919 + 3920 + blk_free_flush_queue(fq); 3921 + } 3922 + 3915 3923 /* hctx->ctxs will be freed in queue's release handler */ 3916 3924 static void blk_mq_exit_hctx(struct request_queue *q, 3917 3925 struct blk_mq_tag_set *set, ··· 3939 3931 if (set->ops->exit_hctx) 3940 3932 set->ops->exit_hctx(hctx, hctx_idx); 3941 3933 3942 - blk_free_flush_queue(hctx->fq); 3934 + call_srcu(&set->tags_srcu, &hctx->fq->rcu_head, 3935 + blk_free_flush_queue_callback); 3943 3936 hctx->fq = NULL; 3944 3937 3945 3938 xa_erase(&q->hctx_table, hctx_idx);
+1
block/blk.h
··· 41 41 struct list_head flush_queue[2]; 42 42 unsigned long flush_data_in_flight; 43 43 struct request *flush_rq; 44 + struct rcu_head rcu_head; 44 45 }; 45 46 46 47 bool is_flush_rq(struct request *req);