tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

sched_ext: Resolve caller's scheduler in scx_bpf_destroy_dsq() / scx_bpf_dsq_nr_queued()

scx_bpf_create_dsq() resolves the calling scheduler via scx_prog_sched(aux)
and inserts the new DSQ into that scheduler's dsq_hash. Its inverse
scx_bpf_destroy_dsq() and the query helper scx_bpf_dsq_nr_queued() were
hard-coded to rcu_dereference(scx_root), so a sub-scheduler could only
destroy or query DSQs in the root scheduler's hash - never its own. If the
root had a DSQ with the same id, the sub-sched silently destroyed it and the
root aborted on the next dispatch ("invalid DSQ ID 0x0..").

Take a const struct bpf_prog_aux *aux via KF_IMPLICIT_ARGS and resolve the
scheduler with scx_prog_sched(aux), matching scx_bpf_create_dsq().

Fixes: ebeca1f930ea ("sched_ext: Introduce cgroup sub-sched support")
Reported-by: Chris Mason <clm@meta.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

Tejun Heo cc2a387d 80afd4c8

+9 -8
+9 -8
kernel/sched/ext.c
··· 8722 8722 /** 8723 8723 * scx_bpf_dsq_nr_queued - Return the number of queued tasks 8724 8724 * @dsq_id: id of the DSQ 8725 + * @aux: implicit BPF argument to access bpf_prog_aux hidden from BPF progs 8725 8726 * 8726 8727 * Return the number of tasks in the DSQ matching @dsq_id. If not found, 8727 8728 * -%ENOENT is returned. 8728 8729 */ 8729 - __bpf_kfunc s32 scx_bpf_dsq_nr_queued(u64 dsq_id) 8730 + __bpf_kfunc s32 scx_bpf_dsq_nr_queued(u64 dsq_id, const struct bpf_prog_aux *aux) 8730 8731 { 8731 8732 struct scx_sched *sch; 8732 8733 struct scx_dispatch_q *dsq; ··· 8735 8734 8736 8735 preempt_disable(); 8737 8736 8738 - sch = rcu_dereference_sched(scx_root); 8737 + sch = scx_prog_sched(aux); 8739 8738 if (unlikely(!sch)) { 8740 8739 ret = -ENODEV; 8741 8740 goto out; ··· 8767 8766 /** 8768 8767 * scx_bpf_destroy_dsq - Destroy a custom DSQ 8769 8768 * @dsq_id: DSQ to destroy 8769 + * @aux: implicit BPF argument to access bpf_prog_aux hidden from BPF progs 8770 8770 * 8771 8771 * Destroy the custom DSQ identified by @dsq_id. Only DSQs created with 8772 8772 * scx_bpf_create_dsq() can be destroyed. The caller must ensure that the DSQ is 8773 8773 * empty and no further tasks are dispatched to it. Ignored if called on a DSQ 8774 8774 * which doesn't exist. Can be called from any online scx_ops operations. 8775 8775 */ 8776 - __bpf_kfunc void scx_bpf_destroy_dsq(u64 dsq_id) 8776 + __bpf_kfunc void scx_bpf_destroy_dsq(u64 dsq_id, const struct bpf_prog_aux *aux) 8777 8777 { 8778 8778 struct scx_sched *sch; 8779 8779 8780 - rcu_read_lock(); 8781 - sch = rcu_dereference(scx_root); 8780 + guard(rcu)(); 8781 + sch = scx_prog_sched(aux); 8782 8782 if (sch) 8783 8783 destroy_dsq(sch, dsq_id); 8784 - rcu_read_unlock(); 8785 8784 } 8786 8785 8787 8786 /** ··· 9535 9534 BTF_ID_FLAGS(func, scx_bpf_task_set_slice, KF_IMPLICIT_ARGS | KF_RCU); 9536 9535 BTF_ID_FLAGS(func, scx_bpf_task_set_dsq_vtime, KF_IMPLICIT_ARGS | KF_RCU); 9537 9536 BTF_ID_FLAGS(func, scx_bpf_kick_cpu, KF_IMPLICIT_ARGS) 9538 - BTF_ID_FLAGS(func, scx_bpf_dsq_nr_queued) 9539 - BTF_ID_FLAGS(func, scx_bpf_destroy_dsq) 9537 + BTF_ID_FLAGS(func, scx_bpf_dsq_nr_queued, KF_IMPLICIT_ARGS) 9538 + BTF_ID_FLAGS(func, scx_bpf_destroy_dsq, KF_IMPLICIT_ARGS) 9540 9539 BTF_ID_FLAGS(func, scx_bpf_dsq_peek, KF_IMPLICIT_ARGS | KF_RCU_PROTECTED | KF_RET_NULL) 9541 9540 BTF_ID_FLAGS(func, scx_bpf_dsq_reenq, KF_IMPLICIT_ARGS) 9542 9541 BTF_ID_FLAGS(func, scx_bpf_reenqueue_local___v2, KF_IMPLICIT_ARGS)