@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.)
hq.recaptime.dev/wiki/Phorge
phorge
phabricator
Update repository hosting documentation for all the issues users have hit
Summary:
Ref T4151. Addresses these issues:
- Mentions `diffusion.ssh-user`.
- Mentions `/etc/shadow` and `!!`.
- Mentions `/etc/passwd` and shell.
- Mentions `sshd -d -d -d`.
- Mentions `Defaults requiretty`.
- Adds `AllowUsers` to default configuration.
- Mentions `sudo -E ...` as a troubleshooting step.
- Mentions multiple VCS binaries.
- Fixes `sshd` paths to be absolute.
- Fixes example path in `sshd_config` template.
- Mentions `GIT_CURL_VERBOSE`.
- Walks users through cloning.
- Adds documentation for custom hooks.
- Mentions that only `daemon-user` interacts with repositories.
- Added general troubleshooting guide.
I didn't fix these:
- Weird one-time issue with `sudoers.d/`. We tell you to edit `/etc/sudoers` directly anyway.
- Insane `#includedir` magic, as above.
- Confusion around `vcs-user` for HTTP, since I think this is fairly clear.
- Confusion around parent directory permissions -- not sure about this one, `sshd` normally runs as root?
I added an `ssh-shell` as a safer alternative to `/bin/sh`. I need to test this a bit more.
Test Plan:
- Read documentation.
- Will test `ssh-shell`.
Reviewers: btrahan, chad
Reviewed By: chad
Subscribers: bluehawk, mbishopim3, epriestley
Maniphest Tasks: T4151
Differential Revision: https://secure.phabricator.com/D8586
+223
-12
+2
-1
resources/sshd/sshd_config.phabricator.example
+2
-1
resources/sshd/sshd_config.phabricator.example
···
3
3
4
4
# NOTE: Edit these to the correct values for your setup.
5
5
6
-
AuthorizedKeysCommand /etc/phabricator-ssh-hook.sh
6
+
AuthorizedKeysCommand /usr/libexec/phabricator-ssh-hook.sh
7
7
AuthorizedKeysCommandUser vcs-user
8
+
AllowUsers vcs-user
8
9
9
10
# You may need to tweak these options, but mostly they just turn off everything
10
11
# dangerous.
+52
src/docs/user/userguide/diffusion_hooks.diviner
+52
src/docs/user/userguide/diffusion_hooks.diviner
···
1
+
@title Diffusion User Guide: Commit Hooks
2
+
@group userguide
3
+
4
+
Guide to commit hooks in hosted repositories.
5
+
6
+
= Overview =
7
+
8
+
Phabricator installs pre-receive/pre-commit hooks in hosted repositories
9
+
automatically. They enforce a few rules automatically (like preventing
10
+
dangerous changes unless a repository is configured to allow them). They can
11
+
also enforce more complex rules via Herald, using the "Commit Hook:
12
+
Branches/Tags/Bookmarks" and "Commit Hook: Commit Content" rule types.
13
+
14
+
Herald rules are flexible, and can express many of the most common hooks that
15
+
are often installed on repositories (like protecting branches, restricting
16
+
access to repositories, and requiring review).
17
+
18
+
However, if Herald isn't powerful enough to enforce everything you want to
19
+
check, you can install additional custom hooks. These work mostly like normal
20
+
hooks, but with a few differences.
21
+
22
+
= Installing Custom Hooks =
23
+
24
+
With hosted repositories, you can install hooks by dropping them into the
25
+
relevant directory of the repository on disk:
26
+
27
+
- **SVN** Put hooks in `hooks/pre-commit-phabricator.d/`.
28
+
- **Git** Put hooks in `hooks/pre-receive-phabricator.d/`.
29
+
- **Mercurial** Phabricator does not currently support custom hooks in
30
+
Mercurial.
31
+
32
+
These hooks act like normal `pre-commit` or `pre-receive` hooks:
33
+
34
+
- Executables in these directories will be run one at a time, in alphabetical
35
+
order.
36
+
- They'll be passed the arguments and environment that normal hooks are
37
+
passed.
38
+
- They should emit output and return codes like normal hooks do.
39
+
- These hooks will run only after all the Herald rules have passed and
40
+
Phabricator is otherwise ready to accept the commit or push.
41
+
42
+
These additional variables will be available in the environment, in addition
43
+
to the variables the VCS normally provides:
44
+
45
+
- `PHABRICATOR_REPOSITORY` The callsign of the repository the hook is
46
+
executing for.
47
+
- `PHABRICATOR_USER` The Phabricator username that the session is
48
+
authenticated under.
49
+
- `PHABRICATOR_REMOTE_ADDRESS` The connection's remote address (that is,
50
+
the IP address of whoever is pushing or committing).
51
+
- `PHABRICATOR_REMOTE_PROTOCOL` The protocol the connection is using (for
52
+
example, "ssh" or "http").
+169
-11
src/docs/user/userguide/diffusion_hosting.diviner
+169
-11
src/docs/user/userguide/diffusion_hosting.diviner
···
9
9
access to them over HTTP and SSH. This document describes how to configure
10
10
repository hosting.
11
11
12
-
NOTE: This feature is new and has some rough edges. Let us know if you run into
13
-
issues (see @{article:Give Feedback! Get Support!}).
14
-
15
12
= Understanding Supported Protocols =
16
13
17
14
Phabricator supports hosting over these protocols:
···
53
50
The system accounts are:
54
51
55
52
- The user the daemons run as. We'll call this `daemon-user`. For more
56
-
information on the daemons, see @{article:Managing Daemons with phd}.
53
+
information on the daemons, see @{article:Managing Daemons with phd}. This
54
+
user is the only user which will interact with the repositories directly.
55
+
Other accounts will `sudo` to this account in order to perform VCS
56
+
operations.
57
57
- The user the webserver runs as. We'll call this `www-user`. If you do not
58
58
plan to make repositories available over HTTP, you do not need to perform
59
59
any special configuration for this user.
···
91
91
Adding these commands to `sudoers` will allow the daemon and webserver users to
92
92
write to repositories as the daemon user.
93
93
94
-
Finally, once you've configured `sudoers`, set `phd.user` to the `daemon-user`:
94
+
Before saving and closing `/etc/sudoers`, look for this line:
95
+
96
+
Defaults requiretty
97
+
98
+
If it's present, comment it out by putting a `#` at the beginning of the line.
99
+
With this option enabled, VCS SSH sessions won't be able to use `sudo`.
100
+
101
+
If you're planning to use SSH, you should also edit `/etc/passwd` and
102
+
`/etc/shadow` to make sure the `vcs-user` account is set up correctly.
103
+
104
+
- Open `/etc/shadow` and find the line for the `vcs-user` account.
105
+
- The second field (which is the password field) must not be set to
106
+
`!!`. This value will prevent login. If it is set to `!!`, edit it
107
+
and set it to `NP` ("no password") instead.
108
+
- Open `/etc/passwd` and find the line for the `vcs-user` account.
109
+
- The last field (which is the login shell) must be set to a real shell.
110
+
If it is set to something like `/bin/false`, then `sshd` will not be able
111
+
to execute commands. Instead, you should set it to a real shell, like
112
+
`/bin/sh`.
113
+
114
+
Finally, once you've configured `/etc/sudoers`, `/etc/shadow` and `/etc/passwd`,
115
+
set `phd.user` to the `daemon-user`:
95
116
96
117
phabricator/ $ ./bin/config set phd.user daemon-user
97
118
119
+
If you're using a `vcs-user`, you should also configure that here:
120
+
121
+
phabricator/ $ ./bin/config set diffusion.ssh-user vcs-user
122
+
98
123
= Configuring HTTP =
99
124
100
125
If you plan to use authenticated HTTP, you need to set
101
126
`diffusion.allow-http-auth` in Config. If you don't plan to use HTTP, or plan to
102
127
use only anonymous HTTP, you can leave this setting disabled.
128
+
129
+
If you plan to use authenticated HTTP, you'll also need to configure a VCS
130
+
password in "Settings" -> "VCS Password". This is a different password than
131
+
your main Phabricator password primarily for security reasons.
103
132
104
133
Otherwise, if you've configured system accounts above, you're all set. No
105
134
additional server configuration is required to make HTTP work.
···
111
140
112
141
- You'll move the normal `sshd` daemon to another port, like `222`. When
113
142
connecting to the machine to administrate it, you'll use this alternate
114
-
port.
143
+
port to get a normal login shell.
115
144
- You'll run a highly restricted `sshd` on port 22, with a special locked-down
116
145
configuration that uses Phabricator to authorize users and execute commands.
117
146
- The `sshd` on port 22 **MUST** be 6.2 or newer, because Phabricator relies
···
128
157
129
158
To smoke-test a configuration, just start another `sshd` using the `-f` flag:
130
159
131
-
sudo sshd -f /path/to/config_file.edited
160
+
sudo /path/to/sshd -f /path/to/config_file.edited
132
161
133
162
You can then connect and make sure the edited config file is valid before
134
163
replacing your primary configuration file.
···
177
206
`phabricator/resources/sshd/sshd_config.phabricator.example` to somewhere like
178
207
`/etc/ssh/sshd_config.phabricator`.
179
208
180
-
Open the file and edit the `AuthorizedKeysCommand` and
181
-
`AuthorizedKeysCommandUser` settings to be correct for your system.
209
+
Open the file and edit the `AuthorizedKeysCommand`,
210
+
`AuthorizedKeysCommandUser`, and `AllowUsers` settings to be correct for your
211
+
system.
182
212
183
213
**Start SSHD**: Now, start the Phabricator `sshd`:
184
214
185
-
sudo sshd -f /path/to/sshd_config.phabricator
215
+
sudo /path/to/sshd -f /path/to/sshd_config.phabricator
186
216
187
217
If you did everything correctly, you should be able to run this:
188
218
···
193
223
{"result":"orbital","error_code":null,"error_info":null}
194
224
195
225
(If you get an authentication error, make sure you added your public key in
196
-
**Settings > SSH Public Keys**.)
226
+
**Settings > SSH Public Keys**.) If you're having trouble, check the
227
+
troubleshooting section below.
197
228
198
229
= Authentication Over HTTP =
199
230
···
205
236
206
237
To authenticate over SSH, users should add **SSH Public Keys** in the
207
238
**Settings** screen.
239
+
240
+
= Cloning a Repository =
241
+
242
+
If you've already set up a hosted repository, you can try cloning it now. To
243
+
do this, browse to the repository's main screen in Diffusion. You should see
244
+
clone commands at the top of the page.
245
+
246
+
To clone the repository, just run the appropriate command.
247
+
248
+
If you don't see the commands or running them doesn't work, see below for tips
249
+
on troubleshooting.
250
+
251
+
= Troubleshooting HTTP =
252
+
253
+
Some general tips for troubleshooting problems with HTTP:
254
+
255
+
- Make sure `diffusion.allow-http-auth` is enabled in your Phabricator config.
256
+
- Make sure HTTP serving is enabled for the repository you're trying to clone.
257
+
You can find this in "Edit Repository" -> "Hosting".
258
+
- Make sure you've configured a VCS password. This is separate from your main
259
+
account password. You can configure this in "Settings" -> "VCS Password".
260
+
- Make sure the main repository screen in Diffusion shows a clone/checkout
261
+
command for HTTP. If it doesn't, something above isn't set up correctly:
262
+
double-check your configuration. You should see a `svn checkout http://...`,
263
+
`git clone http://...` or `hg clone http://...` command. Run that command
264
+
verbatim to clone the repository.
265
+
266
+
If you're using Git, using `GIT_CURL_VERBOSE` may help assess login failures.
267
+
To do so, specify it on the command line before the `git clone` command, like
268
+
this:
269
+
270
+
$ GIT_CURL_VERBOSE=1 git clone ...
271
+
272
+
This will make `git` print out a lot more information. Particularly, the line
273
+
with the HTTP response is likely to be useful:
274
+
275
+
< HTTP/1.1 403 Invalid credentials.
276
+
277
+
In many cases, this can give you more information about what's wrong.
278
+
279
+
= Troubleshooting SSH =
280
+
281
+
Some general tips for troubleshooting problems with SSH:
282
+
283
+
- Check that you've configured `diffusion.ssh-user`.
284
+
- Check that you've configured `phd.user`.
285
+
- Make sure SSH serving is enabled for the repository you're trying to clone.
286
+
You can find this in "Edit Repository" -> "Hosting".
287
+
- Make sure you've added an SSH public key to your account. You can do this
288
+
in "Settings" -> "SSH Keys".
289
+
- Make sure the main repository screen in Diffusion shows a clone/checkout
290
+
command for SSH. If it doesn't, something above isn't set up correctly.
291
+
You should see an `svn checkout svn+ssh://...`, `git clone ssh://...` or
292
+
`hg clone ssh://...` command. Run that command verbatim to clone the
293
+
repository.
294
+
295
+
To troubleshoot SSH setup: connect to the server with `ssh`, without running
296
+
a command. You may need to use the `-T` flag. You should see a message like
297
+
this one:
298
+
299
+
$ ssh -T dweller@secure.phabricator.com
300
+
phabricator-ssh-exec: Welcome to Phabricator.
301
+
302
+
You are logged in as alincoln.
303
+
304
+
You haven't specified a command to run. This means you're requesting an
305
+
interactive shell, but Phabricator does not provide an interactive shell over
306
+
SSH.
307
+
308
+
Usually, you should run a command like `git clone` or `hg push` rather than
309
+
connecting directly with SSH.
310
+
311
+
Supported commands are: conduit, git-receive-pack, git-upload-pack, hg,
312
+
svnserve.
313
+
314
+
If you see this message, all your SSH stuff is configured correctly. **If you
315
+
get a login shell instead, you've missed some major setup step: review the
316
+
documentation above.** If you get some other sort of error, double check these
317
+
settings:
318
+
319
+
- You're connecting as the `vcs-user`.
320
+
- The `vcs-user` has `NP` in `/etc/shadow`.
321
+
- The `vcs-user` has `/bin/sh` or some other valid shell in `/etc/passwd`.
322
+
- Your SSH key is correct, and you've added it to Phabricator in the Settings
323
+
panel.
324
+
325
+
If you can get this far, but can't execute VCS commands like `git clone`, there
326
+
is probably an issue with your `sudoers` configuration. Check:
327
+
328
+
- Your `sudoers` file is set up as instructed above.
329
+
- You've commented out `Defaults requiretty` in `sudoers`.
330
+
- You don't have multiple copies of the VCS binaries (like `git-upload-pack`)
331
+
on your system. You may have granted sudo access to one, while the VCS user
332
+
is trying to run a different one.
333
+
- You've configured `phd.user`.
334
+
- The `phd.user` has read and write access to the repositories.
335
+
336
+
It may also be helpful to run `sshd` in debug mode:
337
+
338
+
$ /path/to/sshd -d -d -d -f /path/to/sshd_config.phabricator
339
+
340
+
This will run it in the foreground and emit a large amount of debugging
341
+
information.
342
+
343
+
Finally, you can usually test that `sudoers` is configured correctly by
344
+
doing something like this:
345
+
346
+
$ su vcs-user
347
+
$ sudo -E -n -u daemon-user -- /path/to/some/vcs-binary --help
348
+
349
+
That will try to run the binary via `sudo` in a manner similar to the way that
350
+
Phabricator will run it. This can give you better error messages about issues
351
+
with `sudoers` configuration.
352
+
353
+
= Miscellaneous Troubleshooting =
354
+
355
+
- If you're getting an error about `svnlook` not being found, add the path
356
+
where `svnlook` is located to the Phabricator configuration
357
+
`environment.append-paths` (even if it already appears in PATH). This issue
358
+
is caused by SVN wiping the environment (including PATH) when invoking
359
+
commit hooks.
360
+
361
+
= Next Steps =
362
+
363
+
Once hosted repositories are set up:
364
+
365
+
- learn about commit hooks with @{Diffusion User Guide: Commit Hooks}.