@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.)
hq.recaptime.dev/wiki/Phorge
phorge
phabricator
Add basic documentation for using "bin/worker" to manage imports of large repositories
Summary: Ref T13591. Provide some guidance on the most common cases for wanting to interact with the worker queue.
Test Plan: Read documentation.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21536
+83
+83
src/docs/user/field/worker_queue.diviner
+83
src/docs/user/field/worker_queue.diviner
···
1
+
@title Managing the Worker Queue
2
+
@group fieldmanual
3
+
4
+
Advanced guide to managing the background worker task queue.
5
+
6
+
Overview
7
+
========
8
+
9
+
Phabricator uses daemonized worker processes to execute some tasks (like
10
+
importing repositories and sending mail) in the background.
11
+
12
+
In most cases, this queue will automatically execute tasks in an appropriate
13
+
order. However, in some cases you may want to exercise greater control over
14
+
which tasks execute, when, and at what priority.
15
+
16
+
Reference: Priority Levels
17
+
==========================
18
+
19
+
Tasks queued by Phabricator use these default priority levels:
20
+
21
+
| Priority | Name | Tasks |
22
+
|---|---|---|
23
+
| 1000 | `ALERTS` | Time-sensitive notifications and email. |
24
+
| 2000 | `DEFAULT` | Normal publishing and processing. |
25
+
| 2500 | `COMMIT` | Import of commits in existing repositories. |
26
+
| 3000 | `BULK` | Edits applied via "Bulk Edit" interface. |
27
+
| 3500 | `INDEX` | Search engine index updates. |
28
+
| 4000 | `IMPORT` | Import of commits in new repositories. |
29
+
30
+
Tasks with smaller priority numbers execute before tasks with larger priority
31
+
numbers (for example, a task with priority 1000 will execute before a task
32
+
with priority 2000).
33
+
34
+
Any positive integer is a valid priority level, and if you adjust the priority
35
+
of tasks with `bin/worker priority` you may select any level even if
36
+
Phabricator would never naturally queue tasks at that level. For example, you
37
+
may adjust tasks to priority `5678`, which will make them execute after all
38
+
other types of natural tasks.
39
+
40
+
Although tasks usually execute in priority order, task execution order is not
41
+
strictly a function of priority, and task priority does not guarantee execution
42
+
order.
43
+
44
+
Large Repository Imports
45
+
========================
46
+
47
+
The most common case where you may want to make an adjustment to the default
48
+
behavior of the worker queue is when importing a very large repository like
49
+
the Linux kernel.
50
+
51
+
Although Phabricator will automatically process imports of new repositories at
52
+
a lower priority level than all other non-import tasks, you may still run into
53
+
issues like these:
54
+
55
+
- You may also want to import one or more //other// new repositories, and
56
+
would prefer they import at a higher priority.
57
+
- You may find overall repository performance is impacted by the large
58
+
repository import.
59
+
60
+
You can manually change the priority of tasks with `bin/worker priority`. For
61
+
example, if your copy of the Linux repository is `R123` and you'd like it to
62
+
import at a lower priority than all other tasks (including other imports of
63
+
new repositories), you can run a command like this:
64
+
65
+
```
66
+
phabricator/ $ ./bin/worker priority --priority 5000 --container R123
67
+
```
68
+
69
+
This means: set all tasks associated with container `R123` (in this example,
70
+
the Linux repository) to priority 5000 (which is lower than any natural
71
+
priority).
72
+
73
+
You can delay tasks until later with `bin/worker delay`, which allows you to
74
+
schedule tasks to execute at night or over the weekend. For example, to
75
+
pause an import for 6 hours, run a command like this:
76
+
77
+
```
78
+
phabricator/ $ ./bin/worker delay --until "6 hours" --container R123
79
+
```
80
+
81
+
The selected tasks will not execute until 6 hours from the time this command
82
+
is issued. You can also provide an explicit date, or "now" to let tasks begin
83
+
execution immediately.