sched: Factor out update_other_load_avgs() from __update_blocked_others()

RT, DL, thermal and irq load and utilization metrics need to be decayed and
updated periodically and before consumption to keep the numbers reasonable.
This is currently done from __update_blocked_others() as a part of the fair
class load balance path. Let's factor it out to update_other_load_avgs().
Pure refactor. No functional changes.

This will be used by the new BPF extensible scheduling class to ensure that
the above metrics are properly maintained.

v2: Refreshed on top of tip:sched/core.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: David Vernet <dvernet@meta.com>

Tejun Heo 2 years ago 96fd6c65 4f9c7ca8

+26 -13

3 changed files

expand all

kernel

sched

fair.c

sched.h

syscalls.c

+3 -13

kernel/sched/fair.c

··· 9352 9352 9353 9353 static bool __update_blocked_others(struct rq *rq, bool *done) 9354 9354 { 9355 - const struct sched_class *curr_class; 9356 - u64 now = rq_clock_pelt(rq); 9357 - unsigned long hw_pressure; 9358 - bool decayed; 9355 + bool updated; 9359 9356 9360 9357 /* 9361 9358 * update_load_avg() can call cpufreq_update_util(). Make sure that RT, 9362 9359 * DL and IRQ signals have been updated before updating CFS. 9363 9360 */ 9364 - curr_class = rq->curr->sched_class; 9365 - 9366 - hw_pressure = arch_scale_hw_pressure(cpu_of(rq)); 9367 - 9368 - decayed = update_rt_rq_load_avg(now, rq, curr_class == &rt_sched_class) | 9369 - update_dl_rq_load_avg(now, rq, curr_class == &dl_sched_class) | 9370 - update_hw_load_avg(now, rq, hw_pressure) | 9371 - update_irq_load_avg(rq, 0); 9361 + updated = update_other_load_avgs(rq); 9372 9362 9373 9363 if (others_have_blocked(rq)) 9374 9364 *done = false; 9375 9365 9376 - return decayed; 9366 + return updated; 9377 9367 } 9378 9368 9379 9369 #ifdef CONFIG_FAIR_GROUP_SCHED

kernel/sched/sched.h

··· 3074 3074 3075 3075 #ifdef CONFIG_SMP 3076 3076 3077 + bool update_other_load_avgs(struct rq *rq); 3078 + 3077 3079 unsigned long effective_cpu_util(int cpu, unsigned long util_cfs, 3078 3080 unsigned long *min, 3079 3081 unsigned long *max); ··· 3119 3117 return READ_ONCE(rq->avg_rt.util_avg); 3120 3118 } 3121 3119 3120 + #else /* !CONFIG_SMP */ 3121 + static inline bool update_other_load_avgs(struct rq *rq) { return false; } 3122 3122 #endif /* CONFIG_SMP */ 3123 3123 3124 3124 #ifdef CONFIG_UCLAMP_TASK

+19

kernel/sched/syscalls.c

··· 260 260 261 261 #ifdef CONFIG_SMP 262 262 /* 263 + * Load avg and utiliztion metrics need to be updated periodically and before 264 + * consumption. This function updates the metrics for all subsystems except for 265 + * the fair class. @rq must be locked and have its clock updated. 266 + */ 267 + bool update_other_load_avgs(struct rq *rq) 268 + { 269 + u64 now = rq_clock_pelt(rq); 270 + const struct sched_class *curr_class = rq->curr->sched_class; 271 + unsigned long hw_pressure = arch_scale_hw_pressure(cpu_of(rq)); 272 + 273 + lockdep_assert_rq_held(rq); 274 + 275 + return update_rt_rq_load_avg(now, rq, curr_class == &rt_sched_class) | 276 + update_dl_rq_load_avg(now, rq, curr_class == &dl_sched_class) | 277 + update_hw_load_avg(now, rq, hw_pressure) | 278 + update_irq_load_avg(rq, 0); 279 + } 280 + 281 + /* 263 282 * This function computes an effective utilization for the given CPU, to be 264 283 * used for frequency selection given the linear relation: f = u * f_max. 265 284 *

Configure Feed

Configure Feed