recaptime.dev / phorge
@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.) hq.recaptime.dev/wiki/Phorge
phorge phabricator
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Do initial repository imports at a lower priority and finish importing commits before starting new ones

Summary:
Fixes T11677. This makes two minor adjustments to the repository import daemons:

- The first step ("Message") now queues at a slightly-lower-than-default (for already-imported repositories) or very-low (for newly importing repositories) priority level.
- The other steps now queue at "default" priority level. This is actually what they already did, but without this change their behavior would be to inherit the priority level of their parents.

This has two effects:

- When adding new repositories to an existing install, they shouldn't block other things from happening anymore.
- The daemons will tend to start one commit and run through all of its steps before starting another commit. This makes progress through the queue more even and predictable.
- Before, they did ALL the message tasks, then ALL the change tasks, etc. This works fine but is confusing/uneven/less-predictable because each type of task takes a different amount of time.

Test Plan:
- Added a new repository.
- Saw all of its "message" steps queue at priority 4000.
- Saw followups queue at priority 2000.
- Saw progress generally "finish what you started" -- go through the queue one commit at a time, instead of one type of task at a time.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11677

Differential Revision: https://secure.phabricator.com/D16585

epriestley db2425b3 8941bbfc

+35 -1
+26 -1
src/applications/repository/engine/PhabricatorRepositoryDiscoveryEngine.php
··· 682 682 683 683 $data['commitID'] = $commit->getID(); 684 684 685 - PhabricatorWorker::scheduleTask($class, $data); 685 + // If the repository is importing for the first time, we schedule tasks 686 + // at IMPORT priority, which is very low. Making progress on importing a 687 + // new repository for the first time is less important than any other 688 + // daemon task. 689 + 690 + // If the repostitory has finished importing and we're just catching up 691 + // on recent commits, we schedule discovery at COMMIT priority, which is 692 + // slightly below the default priority. 693 + 694 + // Note that followup tasks and triggered tasks (like those generated by 695 + // Herald or Harbormaster) will queue at DEFAULT priority, so that each 696 + // commit tends to fully import before we start the next one. This tends 697 + // to give imports fairly predictable progress. See T11677 for some 698 + // discussion. 699 + 700 + if ($repository->isImporting()) { 701 + $task_priority = PhabricatorWorker::PRIORITY_IMPORT; 702 + } else { 703 + $task_priority = PhabricatorWorker::PRIORITY_COMMIT; 704 + } 705 + 706 + $options = array( 707 + 'priority' => $task_priority, 708 + ); 709 + 710 + PhabricatorWorker::scheduleTask($class, $data, $options); 686 711 } 687 712 688 713 private function isInitialImport(array $refs) {
+8
src/applications/repository/worker/commitmessageparser/PhabricatorRepositoryCommitMessageParserWorker.php
··· 47 47 $this->getFollowupTaskClass(), 48 48 array( 49 49 'commitID' => $commit->getID(), 50 + ), 51 + array( 52 + // We queue followup tasks at default priority so that the queue 53 + // finishes work it has started before starting more work. If 54 + // followups are queued at the same priority level, we do all 55 + // message parses first, then all change parses, etc. This makes 56 + // progress uneven. See T11677 for discussion. 57 + 'priority' => PhabricatorWorker::PRIORITY_DEFAULT, 50 58 )); 51 59 } 52 60 }
+1
src/infrastructure/daemon/workers/PhabricatorWorker.php
··· 16 16 17 17 const PRIORITY_ALERTS = 1000; 18 18 const PRIORITY_DEFAULT = 2000; 19 + const PRIORITY_COMMIT = 2500; 19 20 const PRIORITY_BULK = 3000; 20 21 const PRIORITY_IMPORT = 4000; 21 22