tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

sched/fair: Propagate load for throttled cfs_rq

Before task based throttle model, propagating load will stop at a
throttled cfs_rq and that propagate will happen on unthrottle time by
update_load_avg().

Now that there is no update_load_avg() on unthrottle for throttled
cfs_rq and all load tracking is done by task related operations, let the
propagate happen immediately.

While at it, add a comment to explain why cfs_rqs that are not affected
by throttle have to be added to leaf cfs_rq list in
propagate_entity_cfs_rq() per my understanding of commit 0258bdfaff5b
("sched/fair: Fix unfairness caused by missing load decay").

Signed-off-by: Aaron Lu <ziqianlu@bytedance.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Chengming Zhou <chengming.zhou@linux.dev>

authored by

Aaron Lu and committed by
Peter Zijlstra fe8d238e 5b726e9b

+18 -8
+18 -8
kernel/sched/fair.c
··· 5729 5729 return cfs_bandwidth_used() && cfs_rq->throttled; 5730 5730 } 5731 5731 5732 + static inline bool cfs_rq_pelt_clock_throttled(struct cfs_rq *cfs_rq) 5733 + { 5734 + return cfs_bandwidth_used() && cfs_rq->pelt_clock_throttled; 5735 + } 5736 + 5732 5737 /* check whether cfs_rq, or any parent, is throttled */ 5733 5738 static inline int throttled_hierarchy(struct cfs_rq *cfs_rq) 5734 5739 { ··· 6724 6719 static inline int cfs_rq_throttled(struct cfs_rq *cfs_rq) 6725 6720 { 6726 6721 return 0; 6722 + } 6723 + 6724 + static inline bool cfs_rq_pelt_clock_throttled(struct cfs_rq *cfs_rq) 6725 + { 6726 + return false; 6727 6727 } 6728 6728 6729 6729 static inline int throttled_hierarchy(struct cfs_rq *cfs_rq) ··· 13161 13151 { 13162 13152 struct cfs_rq *cfs_rq = cfs_rq_of(se); 13163 13153 13164 - if (cfs_rq_throttled(cfs_rq)) 13165 - return; 13166 - 13167 - if (!throttled_hierarchy(cfs_rq)) 13154 + /* 13155 + * If a task gets attached to this cfs_rq and before being queued, 13156 + * it gets migrated to another CPU due to reasons like affinity 13157 + * change, make sure this cfs_rq stays on leaf cfs_rq list to have 13158 + * that removed load decayed or it can cause faireness problem. 13159 + */ 13160 + if (!cfs_rq_pelt_clock_throttled(cfs_rq)) 13168 13161 list_add_leaf_cfs_rq(cfs_rq); 13169 13162 13170 13163 /* Start to propagate at parent */ ··· 13178 13165 13179 13166 update_load_avg(cfs_rq, se, UPDATE_TG); 13180 13167 13181 - if (cfs_rq_throttled(cfs_rq)) 13182 - break; 13183 - 13184 - if (!throttled_hierarchy(cfs_rq)) 13168 + if (!cfs_rq_pelt_clock_throttled(cfs_rq)) 13185 13169 list_add_leaf_cfs_rq(cfs_rq); 13186 13170 } 13187 13171 }