recaptime.dev / phorge
@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.) hq.recaptime.dev/wiki/Phorge
phorge phabricator
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Improve taskmaster behavior on empty queues

Summary:
Right now, taskmasters on empty queues sleep for 30 seconds. With a default setup (4 taskmasters), this averages out to 7.5 seconds between the time you do anything that queues something and the time that the taskmasters start work on it.

On instances, which currently launch a smaller number of taskmasters, this wait is even longer.

Instead, sleep for the number of seconds that there are taskmasters, with a random offset. This makes the average wait to start a task from an empty queue 1 second, and the average maximum load of an empty queue also one query per second.

On loaded instances this doesn't matter, but this should dramatically improve behavior for less-loaded instances without any real tradeoffs.

Test Plan: Started several taskmasters, saw them jitter out of sync and then use short sleeps to give an empty queue about a 1s delay.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D11772

epriestley 2cd77b5b 3a8cd60b

+18 -2
+18 -2
src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php
··· 3 3 final class PhabricatorTaskmasterDaemon extends PhabricatorDaemon { 4 4 5 5 protected function run() { 6 - $sleep = 0; 6 + $taskmaster_count = PhabricatorEnv::getEnvConfig('phd.start-taskmasters'); 7 + $offset = mt_rand(0, $taskmaster_count - 1); 8 + 7 9 do { 8 10 $tasks = id(new PhabricatorWorkerLeaseQuery()) 9 11 ->setLimit(1) ··· 40 42 41 43 $sleep = 0; 42 44 } else { 43 - $sleep = min($sleep + 1, 30); 45 + // When there's no work, sleep for as many seconds as there are 46 + // active taskmasters. 47 + 48 + // On average, this starts tasks added to an empty queue after one 49 + // second. This keeps responsiveness high even on small instances 50 + // without much work to do. 51 + 52 + // It also means an empty queue has an average load of one query 53 + // per second even if there are a very large number of taskmasters 54 + // launched. 55 + 56 + // The first time we sleep, we add a random offset to try to spread 57 + // the sleep times out somewhat evenly. 58 + $sleep = $taskmaster_count + $offset; 59 + $offset = 0; 44 60 } 45 61 46 62 $this->sleep($sleep);