@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.)
hq.recaptime.dev/wiki/Phorge
phorge
phabricator
Unprototype Harbormaster
Summary: Ref T8089. There's still some cleanup but none of it needs to block this.
Test Plan: Viewed `/applications/`, read documentation.
Reviewers: chad
Reviewed By: chad
Subscribers: devurandom, swisspol
Maniphest Tasks: T8089
Differential Revision: https://secure.phabricator.com/D13811
+239
-4
+9
-4
src/applications/harbormaster/application/PhabricatorHarbormasterApplication.php
+9
-4
src/applications/harbormaster/application/PhabricatorHarbormasterApplication.php
···
36
36
);
37
37
}
38
38
39
-
public function isPrototype() {
40
-
return true;
39
+
public function getRemarkupRules() {
40
+
return array(
41
+
new HarbormasterRemarkupRule(),
42
+
);
41
43
}
42
44
43
-
public function getRemarkupRules() {
45
+
public function getHelpDocumentationArticles(PhabricatorUser $viewer) {
44
46
return array(
45
-
new HarbormasterRemarkupRule(),
47
+
array(
48
+
'name' => pht('Harbormaster User Guide'),
49
+
'href' => PhabricatorEnv::getDoclink('Harbormaster User Guide'),
50
+
),
46
51
);
47
52
}
48
53
+230
src/docs/user/userguide/harbormaster.diviner
+230
src/docs/user/userguide/harbormaster.diviner
···
1
+
@title Harbormaster User Guide
2
+
@group userguide
3
+
4
+
Guide to Harbormaster, a build and continuous integration application.
5
+
6
+
Overview
7
+
========
8
+
9
+
WARNING: Harbormaster is still very rough. Read this document carefully to
10
+
understand what it can and can not do and what to expect in the future.
11
+
12
+
The Harbormaster application provides build and continuous integration support
13
+
for Phabricator.
14
+
15
+
Harbormaster is not a mature application. You should expect it to have major
16
+
missing capabilities and to change substantially over time. The current version
17
+
of Harbormaster can perform some basic build tasks, but has many limitations
18
+
and is not a complete build platform.
19
+
20
+
In particular, you should be aware of these common limitations:
21
+
22
+
- **Creating Build Plans**: Harbormaster ships with only very basic, crude
23
+
tools for writing build plans. There are no default integrations with
24
+
`arc unit` or systems like Jenkins. Build plans are likely to change
25
+
substantially over time.
26
+
- **Triggering Builds**: Harbormaster can only trigger builds through Herald
27
+
rules. It can not currently run periodic builds.
28
+
- **Executing Builds**: Harbormaster can only execute builds in a remote
29
+
system, like Jenkins. It can not currently host builds.
30
+
- **Change Handoff**: Change handoff is covered in rough edges and tradeoffs.
31
+
32
+
33
+
Harbormaster Basics
34
+
===================
35
+
36
+
Use Harbormaster to run builds or tests on code reviews and commits. In general,
37
+
the Harbormaster workflow looks like this today:
38
+
39
+
- You create a new "Build Plan" which describes how to build a project (which
40
+
tests to run, which commands to execute, etc).
41
+
- You configure Harbormaster to trigger the plan when relevant code reviews
42
+
are created or relevant commits are pushed or discovered.
43
+
- Harbormaster executes the plan and reports the results, allowing you to see
44
+
if a change or commit breaks tests.
45
+
46
+
The remainder of this document walks through these steps in more detail.
47
+
48
+
49
+
Concepts and Terminology
50
+
========================
51
+
52
+
Harbormaster uses these concepts to describe builds:
53
+
54
+
- **Build Step**: Describes a single step in a build process, like running a
55
+
command.
56
+
- **Build Plan**: A collection of build steps which describe a build process.
57
+
You'll create build plans to tell Harbormaster which commands it needs to
58
+
run to perform a build.
59
+
- **Buildable**: A reference to an object from another application which can
60
+
have builds run against it. In the upstream, code reviews (from
61
+
Differential) and commits (from Diffusion) are buildable.
62
+
- **Build**: Created by running a build plan against a buildable. Collects
63
+
results from running build commands and shows build progress, status and
64
+
results. A build describes what happened when an entire build plan was
65
+
run.
66
+
- **Build Target**: Builds are made up of build targets, which are created
67
+
automatically when Harbormaster runs the individual steps in a build. A
68
+
build target describes what happened when a specific build step was run.
69
+
70
+
71
+
Creating a Build Plan
72
+
=====================
73
+
74
+
NOTE: Build plans are currently crude and subject to change in future versions
75
+
of Harbormaster.
76
+
77
+
A build plan tells Harbormaster how to run a build: which commands to run,
78
+
services to call, and so on. Builds start with a build plan.
79
+
80
+
To create a build plan, navigate to {nav Harbormaster > Manage Build Plans >
81
+
New Build Plan}.
82
+
83
+
Build plans are composed of "Build Steps". Each step describes an individual
84
+
action (like running a command) and the sequence of steps as a whole comprise
85
+
the plan. For example, you might want to run one command to build a binary,
86
+
then a second command to execute unit tests. Add steps to your build plan
87
+
with {nav Add Build Step}.
88
+
89
+
Currently, the only useful type of build step is "Make HTTP Request", which you
90
+
can use to make a call to an external build system like Jenkins. Today, most
91
+
plans should therefor look something like this:
92
+
93
+
- Use a "Make HTTP Request" step to tell Jenkins or some other similar
94
+
external build system about the code.
95
+
- Have the build step "Wait for Message" after the external system is
96
+
notified.
97
+
- Write custom code on the build server to respond to the request, run a
98
+
build, then report the results back to Phabricator by calling the
99
+
`harbormaster.sendmessage` Conduit API.
100
+
101
+
You'll need to write a nontrivial amount of code to get this working today.
102
+
In the future, Harbormaster will become more powerful and have more builtin
103
+
support for interacting with build systems.
104
+
105
+
106
+
Triggering Builds
107
+
=================
108
+
109
+
NOTE: Harbormaster can not currently watch a branch (like "build 'master' every
110
+
time it changes") or run periodic builds (like "build every hour"). These
111
+
capabilities may be added in the future.
112
+
113
+
You can run builds manually by using {nav Run Plan Manually} from the detail
114
+
screen of a build plan. This will execute a manual build immediately, and can
115
+
be used to test that plans work properly.
116
+
117
+
To trigger a build automatically, write a Herald rule which executes the "Run
118
+
build plans" action. The simplest rule would just use the "Always" condition
119
+
and run a single build plan, but you can use more complex conditions to control
120
+
which plans run on which code.
121
+
122
+
This action is available for commits and revisions, as either can be built
123
+
with Harbormaster. This action is only available for "Project" or "Global"
124
+
rules.
125
+
126
+
Change Handoff
127
+
==============
128
+
129
+
NOTE: Change handoff is currently very rough. It may improve in the future.
130
+
131
+
If you want to build code reviews in an external system, it will need to be
132
+
able to construct a working copy with the changes before it can build them.
133
+
134
+
There are three ways to do this:
135
+
136
+
- **Automatic Staging Areas**: Recommended. This is the simplest and
137
+
cleanest way to hand changes to an external build system.
138
+
- **Manual Staging Areas**: Recommended if you can not use automatic
139
+
staging areas. This is a simple way to hand changes to an external build
140
+
system, but not as clean as automatic staging areas.
141
+
- **`arc patch`**: Not recommended. This mechanism is the most difficult to
142
+
configure and debug, and is not nearly as reliable as handoff via staging
143
+
areas.
144
+
145
+
With staging areas, `arc` pushes a copy of the local changes somewhere as a
146
+
side effect of running `arc diff`. In Git, it pushes changes to a tag like
147
+
`phabricator/diff/123` in a designated remote.
148
+
149
+
The build system can then interact with this copy using normal VCS commands.
150
+
This is simpler to configure, use, troubleshoot and work with than `arc patch`.
151
+
152
+
With `arc patch`, the build system downloads patches from Phabricator and
153
+
applies them to a local working copy. This is more complex and more error-prone
154
+
than staging areas.
155
+
156
+
**Automatic Staging Areas**: This is the recommended mechanism for change
157
+
handoff. This mechanism has not been built yet, so you can not use it.
158
+
159
+
**Manual Staging Areas**: If you can not use automatic staging areas, manual
160
+
staging areas are the next best approach. Manual staging areas are only
161
+
supported under Git, but work with both hosted and imported repositories.
162
+
163
+
Manual staging areas work like this:
164
+
165
+
- You configure a staging area for the repository you want to be able to
166
+
run builds for. A staging area is just a remote repository that you're
167
+
designating for temporary storage.
168
+
- Once a staging area is configured, `arc diff` will automatically push a
169
+
copy of the changes to the staging area as a side effect when creating
170
+
and updating reviews.
171
+
- Your build system can pull changes directly from the configured staging
172
+
area.
173
+
174
+
Configure a staging area by navigating to {nav Diffusion >
175
+
(Choose a Repository) > Edit Repository > Edit Staging}. You'll enter the
176
+
remote URI of a repository to use as a staging area, and `arc diff` will push
177
+
changes to tags like `phabricator/diff/123`.
178
+
179
+
There are several ways to select a staging area:
180
+
181
+
- You can use the repository itself as its own staging area, but this will
182
+
clog it up with a lot of tags that users probably don't care about. This is
183
+
simplest to configure but will be disruptive and potentially confusing to
184
+
users.
185
+
- You can create a single staging repository and have all other
186
+
repositories use it as a staging area. This is simple to configure and
187
+
won't disrupt or confuse users, but you won't be able to set granular
188
+
permissions on the staging repository: all the staged changes in a
189
+
repository are visible to anyone who has access to the repository, even if
190
+
they came from a repository that the viewer does not have access to.
191
+
- You can create a staging repository for each standard repository. This will
192
+
give you the most control, but is also the most time consuming to configure.
193
+
- You can use a hybrid approach and have several staging repositories, each
194
+
of which is used for one or more standard repositories. This will let you
195
+
strike a balance between setup costs and granularity.
196
+
- Using automatic staging areas avoids all this complexity by using the
197
+
repository as its own staging area but hiding the tags from users.
198
+
199
+
Once you've configured a staging area, have your build system clone the staging
200
+
area repository and do a checkout of the relevant tag in order to perform a
201
+
build.
202
+
203
+
**`arc patch`**: You can also have the build system pull changes out of
204
+
Phabricator as patches and apply them with `arc patch`. This mechanism is the
205
+
most complex to configure and debug, and is much less reliable than using
206
+
staging areas. It is not recommended.
207
+
208
+
To use `arc patch`-based handoff, install PHP on your build server and set up
209
+
`arc`. Create a "bot" user for your build system and generate a Conduit token
210
+
in {nav Settings > Conduit API Tokens}. Then have your build system clone the
211
+
repository and run `arc patch` to apply the changes:
212
+
213
+
$ arc patch --conduit-token <token> --diff <diff-id>
214
+
215
+
This will usually work, but is more complex and less reliable than using a
216
+
staging area.
217
+
218
+
219
+
Troubleshooting
220
+
===============
221
+
222
+
You can troubleshoot Harbormaster by using `bin/harbormaster` from the command
223
+
line. Run it as `bin/harbormaster help` for details.
224
+
225
+
In particular, you can run manual builds in the foreground from the CLI to see
226
+
more details about what they're doing:
227
+
228
+
phabricator/ $ ./bin/harbormaster build D123 --plan 456 --trace
229
+
230
+
This may help you understand or debug issues with a build plan.