tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

sched_ext: Remove scx_ops_cpu_preempt static_key

scx_ops_cpu_preempt is used to encode whether ops.cpu_acquire/release() are
implemented into a static_key. These tests aren't hot enough for static_key
usage to make any meaningful difference and are made to use a static_key
mostly because there was no reason not to. However, global static_keys can't
work with the planned hierarchical multiple scheduler support. Remove the
static_key and instead use an internal ops flag SCX_OPS_HAS_CPU_PREEMPT to
record and test whether ops.cpu_acquire/release() are implemented.

In repeated hackbench runs before and after static_keys removal on an AMD
Ryzen 3900X, I couldn't tell any measurable performance difference.

Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: Changwoo Min <changwoo@igalia.com>
Acked-by: Andrea Righi <arighi@nvidia.com>

Tejun Heo 54d2e717 cc39454c

+8 -5
+8 -5
kernel/sched/ext.c
··· 173 173 SCX_OPS_SWITCH_PARTIAL | 174 174 SCX_OPS_BUILTIN_IDLE_PER_NODE | 175 175 SCX_OPS_HAS_CGROUP_WEIGHT, 176 + 177 + /* high 8 bits are internal, don't include in SCX_OPS_ALL_FLAGS */ 178 + __SCX_OPS_INTERNAL_MASK = 0xffLLU << 56, 179 + 180 + SCX_OPS_HAS_CPU_PREEMPT = 1LLU << 56, 176 181 }; 177 182 178 183 /* argument container for ops.init_task() */ ··· 929 924 static bool scx_warned_zero_slice; 930 925 931 926 DEFINE_STATIC_KEY_FALSE(scx_ops_allow_queued_wakeup); 932 - static DEFINE_STATIC_KEY_FALSE(scx_ops_cpu_preempt); 933 927 934 928 static struct static_key_false scx_has_op[SCX_OPI_END] = 935 929 { [0 ... SCX_OPI_END-1] = STATIC_KEY_FALSE_INIT }; ··· 2935 2931 rq->scx.flags |= SCX_RQ_IN_BALANCE; 2936 2932 rq->scx.flags &= ~(SCX_RQ_BAL_PENDING | SCX_RQ_BAL_KEEP); 2937 2933 2938 - if (static_branch_unlikely(&scx_ops_cpu_preempt) && 2934 + if ((scx_ops.flags & SCX_OPS_HAS_CPU_PREEMPT) && 2939 2935 unlikely(rq->scx.cpu_released)) { 2940 2936 /* 2941 2937 * If the previous sched_class for the current CPU was not SCX, ··· 3164 3160 */ 3165 3161 smp_store_release(&rq->scx.pnt_seq, rq->scx.pnt_seq + 1); 3166 3162 #endif 3167 - if (!static_branch_unlikely(&scx_ops_cpu_preempt)) 3163 + if (!(scx_ops.flags & SCX_OPS_HAS_CPU_PREEMPT)) 3168 3164 return; 3169 3165 3170 3166 /* ··· 4729 4725 for (i = SCX_OPI_BEGIN; i < SCX_OPI_END; i++) 4730 4726 static_branch_disable(&scx_has_op[i]); 4731 4727 static_branch_disable(&scx_ops_allow_queued_wakeup); 4732 - static_branch_disable(&scx_ops_cpu_preempt); 4733 4728 scx_idle_disable(); 4734 4729 synchronize_rcu(); 4735 4730 ··· 5370 5367 if (ops->flags & SCX_OPS_ALLOW_QUEUED_WAKEUP) 5371 5368 static_branch_enable(&scx_ops_allow_queued_wakeup); 5372 5369 if (scx_ops.cpu_acquire || scx_ops.cpu_release) 5373 - static_branch_enable(&scx_ops_cpu_preempt); 5370 + scx_ops.flags |= SCX_OPS_HAS_CPU_PREEMPT; 5374 5371 5375 5372 /* 5376 5373 * Lock out forks, cgroup on/offlining and moves before opening the