CLI app for developers prototyping atproto functionality
docs: add human test plan for test-oauth-client implementation
Generated by the test-analyst agent after all 49 automated ACs were
verified covered. Documents the one manual gate (AC4.1 interactive
wait-for-Ctrl-C) plus visual colour/verbosity checks and an optional
real-external-client end-to-end flow.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
+106
+106
docs/test-plans/2026-04-16-test-oauth-client.md
+106
docs/test-plans/2026-04-16-test-oauth-client.md
···
1
+
# Human test plan — `atproto-devtool test oauth client`
2
+
3
+
Implementation plan: [`docs/implementation-plans/2026-04-16-test-oauth-client/`](../implementation-plans/2026-04-16-test-oauth-client/)
4
+
5
+
All 49 automatable acceptance criteria are covered by `cargo test`. This plan
6
+
covers the one branch that automated tests cannot exercise cleanly (AC4.1
7
+
interactive wait-for-Ctrl-C) plus a handful of visual / ergonomic checks
8
+
that a human eye catches faster than a snapshot.
9
+
10
+
## Prerequisites
11
+
12
+
- Rust toolchain per `rust-toolchain.toml` installed.
13
+
- Working copy at `/home/str4d/dev/ai/atproto-devtool/.worktrees/test-oauth-client/`.
14
+
- `cargo test`, `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` all clean.
15
+
- Terminal capable of ANSI colour (e.g., `xterm-256color`) for colour checks.
16
+
- Port `4444` free on `127.0.0.1` for the interactive-mode run.
17
+
18
+
## Phase 1: Static-mode smoke tests
19
+
20
+
| Step | Action | Expected |
21
+
|---|---|---|
22
+
| 1.1 | `cargo run -- test oauth client http://localhost` | Exit 0. Report shows discovery / metadata / JWKS rows all `[SKIP]` with reason "metadata is implicit for loopback clients" or equivalent. |
23
+
| 1.2 | `cargo run -- test oauth client ftp://example.com/x` | Non-zero exit. Rendered miette error references `oauth_client::target::unsupported_scheme` and mentions the `ftp` scheme. |
24
+
| 1.3 | `cargo run -- test oauth client https://example.invalid/metadata.json` | Exit 2. Rendered report contains a `[NET]` row for discovery; downstream rows `[SKIP]` with reason `blocked by oauth_client::discovery::metadata_document_fetchable`. |
25
+
| 1.4 | `cargo run -- test oauth client --help` | Help text contains `<TARGET>` argument. Help does **NOT** contain `--port` or `--public-base-url`. |
26
+
27
+
## Phase 2: Interactive mode — AC4.1 (the authoritative manual gate)
28
+
29
+
Automated tests never reach `tokio::signal::ctrl_c()` because they use
30
+
`DriveRpInProcess`. This phase verifies the `WaitForExternalClient` path.
31
+
32
+
| Step | Action | Expected |
33
+
|---|---|---|
34
+
| 2.1 | `cargo run -- test oauth client http://localhost/client.json interactive --port 4444` | Process starts and stays running. Stdout contains lines starting with `Fake atproto authorization server is running.`, `Handle:`, `DID:`, `Base:`. The `Base:` line contains `http://127.0.0.1:4444`. |
35
+
| 2.2 | With 2.1 still running, in another shell: `curl -s http://127.0.0.1:4444/.well-known/oauth-authorization-server \| python3 -m json.tool` | JSON body contains `"issuer": "http://127.0.0.1:4444"`, `"pushed_authorization_request_endpoint"`, `"require_pushed_authorization_requests": true`, `"code_challenge_methods_supported": ["S256"]`, `"dpop_signing_alg_values_supported": ["ES256"]`. |
36
+
| 2.3 | Send `SIGINT` (Ctrl-C) to the interactive process from 2.1 | Process exits cleanly; a rendered report prints before exit. Exit code follows AC8 semantics (0, 1, or 2 depending on what requests were captured). No tokio panic, no orphaned-thread warnings. |
37
+
| 2.4 | `cargo run -- test oauth client http://localhost interactive --public-base-url https://funnel.example.com`, then Ctrl-C | `Base:` line references `https://funnel.example.com`. Served metadata (see `curl` in 2.2) would use the public URL. |
38
+
| 2.5 | First shell: `nc -l 127.0.0.1 4445` (holds the port). Second shell: `cargo run -- test oauth client http://localhost/client.json interactive --port 4445` | Non-zero exit. Error message reports the bind failure and references port 4445. No synthetic identity lines printed. |
39
+
40
+
## Phase 3: Colour / verbosity ergonomics
41
+
42
+
| Step | Action | Expected |
43
+
|---|---|---|
44
+
| 3.1 | Colour-capable terminal: `cargo run -- test oauth client http://localhost` | Output contains ANSI colour escapes (green `[OK]`, red `[FAIL]`, etc., visible as coloured text). |
45
+
| 3.2 | `NO_COLOR=1 cargo run -- test oauth client http://localhost` | Output is plain text; no ANSI escape sequences visible. |
46
+
| 3.3 | `cargo run -- test oauth client http://localhost --no-color` | Same as 3.2 — no ANSI. |
47
+
| 3.4 | `cargo run -- test oauth client http://localhost --verbose 2>&1 \| grep -c DEBUG` | Non-zero count. Same command without `--verbose` shows no DEBUG lines. |
48
+
49
+
## Phase 4: End-to-end against a real external OAuth client (optional smoke)
50
+
51
+
Purpose: exercises the complete `DriveRpInProcess` path that automated tests
52
+
cover via the mocked RP seam. Confirms a real external OAuth client could
53
+
drive the fake AS if pointed at it.
54
+
55
+
1. Start the fake AS: `cargo run -- test oauth client http://localhost/client.json interactive --port 4444`
56
+
2. In a second terminal, walk the well-known endpoints:
57
+
- `curl -s http://127.0.0.1:4444/.well-known/did.json | python3 -m json.tool` — confirm `id` matches `did:web:127.0.0.1%3A4444`, `service[0].serviceEndpoint` is `http://127.0.0.1:4444`.
58
+
- `curl -s http://127.0.0.1:4444/.well-known/oauth-protected-resource | python3 -m json.tool` — confirm `resource` and `authorization_servers` array present.
59
+
- Fetch AS metadata as in step 2.2.
60
+
3. Point a real atproto OAuth client library (e.g., the TypeScript
61
+
`@atproto/oauth-client-node` examples, configured to trust `http://`
62
+
loopback) at the printed `Base:` URL and walk the full PAR → authorize
63
+
→ token → refresh flow.
64
+
4. Send `SIGINT` to the fake AS.
65
+
5. Verify the rendered report emits one row per request observed. PAR /
66
+
authorize / token hits the client performed should render `[OK]`;
67
+
anything absent should `[SKIP]` with an appropriate `blocked by` reason.
68
+
No authentication bypass paths emit `[OK]`.
69
+
70
+
## Traceability
71
+
72
+
| AC | Automated | Manual |
73
+
|---|---|---|
74
+
| AC1.1 | `oauth_client_discovery::https_confidential_happy_discovery` | — |
75
+
| AC1.2 | `oauth_client_discovery::loopback_root_produces_skip_rows` | Phase 1.1 (informal sanity) |
76
+
| AC1.3 | `oauth_client_discovery::loopback_127_0_0_1` | — |
77
+
| AC1.4 | `commands::test::oauth::client::pipeline::tests::ftp_scheme_rejected` | Phase 1.2 |
78
+
| AC1.5 | `oauth_client_discovery::https_404_produces_network_error` | Phase 1.3 |
79
+
| AC1.6 | `oauth_client_discovery::https_not_json_produces_spec_violation` | — |
80
+
| AC1.7 | `oauth_client_discovery::loopback_with_port_produces_same_skip_rows` | — |
81
+
| AC2.1–AC2.9 | `oauth_client_metadata::{confidential_happy, public_happy, native_happy, dpop_bound_false, confidential_missing_jwks, public_with_token_endpoint_auth, native_redirect_scheme_mismatch, scope_grammar_invalid, loopback_skips_all_metadata_checks}` | — |
82
+
| AC3.1–AC3.8 | `oauth_client_jwks::{inline_es256_happy_jwks_passes, uri_es256_happy_jwks_passes, uri_unreachable_produces_network_error, duplicate_kids_produces_spec_violation, missing_alg_produces_spec_violation, wrong_use_produces_spec_violation, weak_alg_rs1_produces_spec_violation, public_client_skips_all_jwks}` | — |
83
+
| **AC4.1** | Spawn-and-print half exercised in every `oauth_client_ac_coverage` test via `spawn_fake_as` | **Phase 2.1–2.3** |
84
+
| AC4.2 | `oauth_client_interactive::serves_as_metadata_document` | Phase 2.2 |
85
+
| AC4.3 | `oauth_client_interactive::public_base_url_rewrites_served_urls` | Phase 2.4 |
86
+
| AC4.4 | `oauth_client_interactive::{serves_did_json_document, serves_prm_document, serves_as_metadata_document}` | Phase 2.2 + Phase 4 step 2 |
87
+
| AC4.5 | `oauth_client_ac_coverage::ac4_5_par_request_logged_verbatim_with_timestamp` | — |
88
+
| AC4.6 | `oauth_client_interactive::bind_unbindable_port_returns_error` | Phase 2.5 |
89
+
| AC5.1 | `oauth_client_ac_coverage::ac5_1_static_stages_render_before_interactive` | — |
90
+
| AC5.2 | `oauth_client_ac_coverage::ac5_2_all_static_pass_runs_full_interactive_inventory` | — |
91
+
| AC5.3 | `oauth_client_interactive::interactive_partial_static_failure_blocks` | — |
92
+
| AC5.4 | `oauth_client_ac_coverage::ac5_4_non_gating_static_failure_leaves_interactive_inventory_intact` | — |
93
+
| AC5.5 | `oauth_client_interactive::interactive_partial_static_failure_blocks` | — |
94
+
| AC6.1–AC6.4 | `oauth_client_ac_coverage::{ac6_1_*, ac6_2_*, ac6_3_*, ac6_4_*}` | — |
95
+
| AC6.5–AC6.6 | `oauth_client_broken_rp::{ac6_5_*, ac6_6_*}` | — |
96
+
| AC7.1–AC7.3 | `oauth_client_ac_coverage::{ac7_1_*, ac7_2_*, ac7_3_*}` | — |
97
+
| AC7.4–AC7.6 | `oauth_client_broken_rp::{ac7_4_*, ac7_5_*, ac7_6_*}` | — |
98
+
| AC8.1 | `oauth_client_endtoend::exit_zero_when_all_pass` | Phase 1.1 |
99
+
| AC8.2 | `oauth_client_endtoend::exit_one_on_any_spec_violation` | — |
100
+
| AC8.3 | `oauth_client_endtoend::exit_two_on_network_error_alone` | Phase 1.3 |
101
+
| AC8.4 | `oauth_client_endtoend::advisory_and_skipped_do_not_affect_exit` | — |
102
+
| AC8.5 | `oauth_client_cli::verbose_flag_accepted` | Phase 3.4 |
103
+
| AC8.6 | `oauth_client_cli::{no_color_env_var_suppresses_color, no_color_flag_suppresses_color}` | Phase 3.1–3.3 |
104
+
| AC8.7 | `oauth_client_check_id_coverage::every_check_id_appears_in_at_least_one_snapshot` | — |
105
+
| AC8.8 | `oauth_client_check_id_coverage::every_diagnostic_code_appears_in_at_least_one_snapshot` | — |
106
+
| AC8.9 | `oauth_client_cli::{interactive_help_has_port_and_public_base_url, static_help_does_not_have_port_or_public_base_url}` | Phase 1.4 |