recaptime.dev / phorge
@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.) hq.recaptime.dev/wiki/Phorge
phorge phabricator
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Reverse meaning of task priority column

Summary:
Ref T6615. Mixing ASC and DESC ordering on a multipart key makes it dramatically less effective (or perhaps totally ineffective).

Reverse the meaning of the `priority` column so it goes in the same direction as the `id` column (both ascending, lower values execute sooner).

Test Plan:
- Queued 1.2M tasks with `bin/worker flood`.
- Processed ~1 task/second with `bin/phd debug taskmaster` before patch.
- Applied patch, took ~5 seconds for ~1.2M rows.
- Processed ~100-200 tasks/second with `bin/phd debug taskmaster` after patch.
- "Next in Queue" query on daemon page dropped from 1.5s to <1ms.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: aklapper, 20after4, epriestley

Maniphest Tasks: T6615

Differential Revision: https://secure.phabricator.com/D10895

epriestley b5f7e9ee 7e1c3121

+19 -11
+2
resources/sql/autopatches/20141123.taskpriority.1.sql
··· 1 + UPDATE {$NAMESPACE}_worker.worker_activetask 2 + SET priority = 5000 - priority;
+2
resources/sql/autopatches/20141123.taskpriority.2.sql
··· 1 + UPDATE {$NAMESPACE}_worker.worker_archivetask 2 + SET priority = 5000 - priority;
+8 -4
src/infrastructure/daemon/workers/PhabricatorWorker.php
··· 9 9 private static $runAllTasksInProcess = false; 10 10 private $queuedTasks = array(); 11 11 12 - const PRIORITY_ALERTS = 4000; 13 - const PRIORITY_DEFAULT = 3000; 14 - const PRIORITY_BULK = 2000; 15 - const PRIORITY_IMPORT = 1000; 12 + // NOTE: Lower priority numbers execute first. The priority numbers have to 13 + // have the same ordering that IDs do (lowest first) so MySQL can use a 14 + // multipart key across both of them efficiently. 15 + 16 + const PRIORITY_ALERTS = 1000; 17 + const PRIORITY_DEFAULT = 2000; 18 + const PRIORITY_BULK = 3000; 19 + const PRIORITY_IMPORT = 4000; 16 20 17 21 18 22 /* -( Configuring Retries and Failures )----------------------------------- */
+5 -5
src/infrastructure/daemon/workers/__tests__/PhabricatorWorkerTestCase.php
··· 160 160 $this->expectNextLease($task1); 161 161 } 162 162 163 - public function testLeasedIsHighestPriority() { 164 - $task1 = $this->scheduleTask(array(), 1); 165 - $task2 = $this->scheduleTask(array(), 1); 166 - $task3 = $this->scheduleTask(array(), 2); 163 + public function testLeasedIsLowestPriority() { 164 + $task1 = $this->scheduleTask(array(), 2); 165 + $task2 = $this->scheduleTask(array(), 2); 166 + $task3 = $this->scheduleTask(array(), 1); 167 167 168 168 $this->expectNextLease( 169 169 $task3, 170 - 'Tasks with a higher priority should be scheduled first.'); 170 + 'Tasks with a lower priority should be scheduled first.'); 171 171 $this->expectNextLease( 172 172 $task1, 173 173 'Tasks with the same priority should be FIFO.');
+2 -2
src/infrastructure/daemon/workers/query/PhabricatorWorkerLeaseQuery.php
··· 215 215 switch ($phase) { 216 216 case self::PHASE_UNLEASED: 217 217 // When selecting new tasks, we want to consume them in order of 218 - // decreasing priority (and then FIFO). 219 - return qsprintf($conn_w, 'ORDER BY priority DESC, id ASC'); 218 + // increasing priority (and then FIFO). 219 + return qsprintf($conn_w, 'ORDER BY priority ASC, id ASC'); 220 220 case self::PHASE_EXPIRED: 221 221 // When selecting failed tasks, we want to consume them in roughly 222 222 // FIFO order of their failures, which is not necessarily their original