psi: fix possible trigger missing in the window

When a new threshold breaching stall happens after a psi event was
generated and within the window duration, the new event is not
generated because the events are rate-limited to one per window. If
after that no new stall is recorded then the event will not be
generated even after rate-limiting duration has passed. This is
happening because with no new stall, window_update will not be called
even though threshold was previously breached. To fix this, record
threshold breaching occurrence and generate the event once window
duration is passed.

Suggested-by: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Zhaoyang Huang <zhaoyang.huang@unisoc.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Suren Baghdasaryan <surenb@google.com>
Link: https://lore.kernel.org/r/1643093818-19835-1-git-send-email-huangzhaoyang@gmail.com

authored by

Zhaoyang Huang and committed by

Peter Zijlstra 4 years ago e6df4ead 5c7b1aaf

+31 -14

2 changed files

expand all

include

linux

psi_types.h

kernel

sched

psi.c

include/linux/psi_types.h

··· 144 144 145 145 /* Refcounting to prevent premature destruction */ 146 146 struct kref refcount; 147 + 148 + /* Deferred event(s) from previous ratelimit window */ 149 + bool pending_event; 147 150 }; 148 151 149 152 struct psi_group {

+28 -14

kernel/sched/psi.c

··· 523 523 static u64 update_triggers(struct psi_group *group, u64 now) 524 524 { 525 525 struct psi_trigger *t; 526 - bool new_stall = false; 526 + bool update_total = false; 527 527 u64 *total = group->total[PSI_POLL]; 528 528 529 529 /* ··· 532 532 */ 533 533 list_for_each_entry(t, &group->triggers, node) { 534 534 u64 growth; 535 + bool new_stall; 535 536 536 - /* Check for stall activity */ 537 - if (group->polling_total[t->state] == total[t->state]) 537 + new_stall = group->polling_total[t->state] != total[t->state]; 538 + 539 + /* Check for stall activity or a previous threshold breach */ 540 + if (!new_stall && !t->pending_event) 538 541 continue; 539 - 540 542 /* 541 - * Multiple triggers might be looking at the same state, 542 - * remember to update group->polling_total[] once we've 543 - * been through all of them. Also remember to extend the 544 - * polling time if we see new stall activity. 543 + * Check for new stall activity, as well as deferred 544 + * events that occurred in the last window after the 545 + * trigger had already fired (we want to ratelimit 546 + * events without dropping any). 545 547 */ 546 - new_stall = true; 548 + if (new_stall) { 549 + /* 550 + * Multiple triggers might be looking at the same state, 551 + * remember to update group->polling_total[] once we've 552 + * been through all of them. Also remember to extend the 553 + * polling time if we see new stall activity. 554 + */ 555 + update_total = true; 547 556 548 - /* Calculate growth since last update */ 549 - growth = window_update(&t->win, now, total[t->state]); 550 - if (growth < t->threshold) 551 - continue; 557 + /* Calculate growth since last update */ 558 + growth = window_update(&t->win, now, total[t->state]); 559 + if (growth < t->threshold) 560 + continue; 552 561 562 + t->pending_event = true; 563 + } 553 564 /* Limit event signaling to once per window */ 554 565 if (now < t->last_event_time + t->win.size) 555 566 continue; ··· 569 558 if (cmpxchg(&t->event, 0, 1) == 0) 570 559 wake_up_interruptible(&t->event_wait); 571 560 t->last_event_time = now; 561 + /* Reset threshold breach flag once event got generated */ 562 + t->pending_event = false; 572 563 } 573 564 574 - if (new_stall) 565 + if (update_total) 575 566 memcpy(group->polling_total, total, 576 567 sizeof(group->polling_total)); 577 568 ··· 1138 1125 t->last_event_time = 0; 1139 1126 init_waitqueue_head(&t->event_wait); 1140 1127 kref_init(&t->refcount); 1128 + t->pending_event = false; 1141 1129 1142 1130 mutex_lock(&group->trigger_lock); 1143 1131

Configure Feed

Configure Feed