CLI app for developers prototyping atproto functionality
docs: add labeler report stage design plan
Closes a gap in the test labeler conformance suite: existing stages
exercise only the read side of a labeler. Design introduces a fifth
report stage covering com.atproto.moderation.createReport under
service-auth JWTs across three modes (self-mint for dev-loop, PDS
getServiceAuth for labeler-integration, PDS-proxied for production
smoke test). Eight implementation phases, 10 stage checks, no new
crate dependencies.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
authored by
Jack Grigg
Claude Opus 4.7
and committed by
Tangled
84178c97
b66dd194
+386
+386
docs/design-plans/2026-04-17-labeler-report-stage.md
+386
docs/design-plans/2026-04-17-labeler-report-stage.md
···
1
+
# Labeler report stage design
2
+
3
+
## Summary
4
+
5
+
The `test labeler` pipeline today exercises the read side of a labeler — verifying that its DID document is correctly formed, that its HTTP endpoint speaks atproto, that its label-subscription WebSocket stream is alive, and that its signing keys are cryptographically valid. What it cannot yet do is prove that the labeler's authenticated write path works: specifically, that it correctly enforces the service-auth JWT protocol for `com.atproto.moderation.createReport`, the XRPC method through which clients submit moderation reports to a labeler. This stage closes that gap.
6
+
7
+
The design introduces three operating modes that map to three different deployment milestones a labeler author moves through. In the self-mint mode — suited to local development — the tool generates a temporary cryptographic identity, stands up a miniature DID document server on localhost, and mints its own JWTs. This makes the tool entirely self-contained for negative tests (wrong audience, wrong method, expired token, etc.) without requiring any real atproto account. In the PDS service-auth mode — suited to integration testing against a real labeler — the tool authenticates to the user's Personal Data Server, asks it to mint a JWT bound to the labeler, and delivers that JWT directly. This catches real-world quirks such as the PDS's specific JWT algorithm choices and nonce handling that a hand-crafted self-mint JWT cannot replicate. In the PDS-proxied mode — a production smoke test — the tool sends the report to the PDS and lets it forward the request, exactly as the Bluesky app does in production. This verifies the full network path: that the PDS can reach the labeler and that the labeler accepts JWTs issued on behalf of real accounts. The three modes are additive and gated: each requires progressively more context (locality of the labeler, PDS credentials, and explicit opt-in to actually submitting reports), so the tool degrades gracefully and reports precisely why any particular check was skipped.
8
+
9
+
## Definition of Done
10
+
11
+
**Primary deliverable.** A fifth `report` stage in the `test labeler` pipeline that exercises the authenticated `com.atproto.moderation.createReport` path end-to-end, following the same module-level `run()` + `*StageOutput { facts, results }` pattern the existing identity / http / subscription / crypto stages use. The stage lives in `src/commands/test/labeler/create_report.rs` (named after the XRPC method under test, to disambiguate from the existing `src/commands/test/labeler/report.rs` which already holds the stage-agnostic `CheckResult` / `CheckStatus` types).
12
+
13
+
**Three operating modes, all in v1.** Each mode verifies a different deployment milestone:
14
+
15
+
- **Self-mint (local dev loop).** The tool spins up an ephemeral HTTP server on `127.0.0.1:0` (OS-assigned port), serves a throwaway did:web DID document, generates an ES256K or ES256 keypair (chosen via `--self-mint-curve`, default ES256K for maximum overlap with real atproto accounts), and mints service-auth JWTs itself. Powers every negative check plus `report::self_mint_accepted`. Viable only when the labeler can reach this machine (local labeler, LAN labeler, or `--force-self-mint` override).
16
+
- **PDS `getServiceAuth` + direct POST (labeler-integration check).** The tool calls `com.atproto.server.createSession` and then `com.atproto.server.getServiceAuth` on the user's PDS, and POSTs the returned JWT directly to the labeler. Powers `report::pds_service_auth_accepted`. Catches PDS-specific JWT quirks (algorithm choice, `jti` nonce, ES256K handling) that self-mint cannot replicate. Viable whenever the tool can reach both PDS and labeler.
17
+
- **PDS-proxied (production smoke test).** The tool POSTs `com.atproto.moderation.createReport` to the user's *PDS* with header `atproto-proxy: <labeler-did>#atproto_labeler`; the PDS mints and forwards. Powers `report::pds_proxied_accepted`. End-to-end path matching the real Bluesky app flow — verifies the PDS can reach and authenticate to the labeler. Viable when the user's PDS can reach the labeler.
18
+
19
+
**Stage gating behavior.** Four gating axes interact: `contract_advertised` (from identity-parsed `LabelerPolicies`), `self_mint_viable` (hostname heuristic or `--force-self-mint`), `pds_creds` (`--handle` + `--app-password` both supplied), and `commit` (`--commit-report`). Per-check gating:
20
+
21
+
| Check ID | Runs when | Skipped reason when gated off |
22
+
|---|---|---|
23
+
| `report::contract_published` | Always | n/a — severity depends on `commit` |
24
+
| `report::unauthenticated_rejected` | `contract_advertised` | "labeler does not advertise report acceptance" |
25
+
| `report::malformed_bearer_rejected` | `contract_advertised` | same |
26
+
| `report::wrong_aud_rejected` | `contract_advertised` && `self_mint_viable` | "self-mint required; labeler endpoint appears non-local (override with --force-self-mint)" |
27
+
| `report::wrong_lxm_rejected` | same | same |
28
+
| `report::expired_rejected` | same | same |
29
+
| `report::rejected_shape_returns_400` | same | same |
30
+
| `report::self_mint_accepted` | `contract_advertised` && `self_mint_viable` && `commit` | self-mint-viable reason OR "commit gated behind --commit-report" |
31
+
| `report::pds_service_auth_accepted` | `contract_advertised` && `pds_creds` && `commit` | "requires --handle, --app-password, and --commit-report" |
32
+
| `report::pds_proxied_accepted` | same | same |
33
+
34
+
`report::contract_published` severity: `Pass` when `reason_types` and `subject_types` are both non-empty. When the contract is missing: `Skipped` (entire stage, with all other checks Skipped "labeler does not advertise report acceptance") if `commit == false`; `SpecViolation` (diagnostic `labeler::report::contract_missing`, all other checks Skipped "blocked by `report::contract_published`") if `commit == true`.
35
+
36
+
The "never short-circuit, always emit one row per stable ID" invariant is preserved in every case — every run produces exactly 10 `report::*` rows.
37
+
38
+
**Stage checks (stable IDs, 10 total).**
39
+
40
+
- `report::contract_published` — validates identity-parsed `LabelerPolicies` has non-empty `reasonTypes` and `subjectTypes`. Severity interacts with `--commit-report`.
41
+
- `report::unauthenticated_rejected` — POST without `Authorization` → expect 401. `SpecViolation` on fail.
42
+
- `report::malformed_bearer_rejected` — POST with `Authorization: Bearer not-a-jwt` → expect 401. `SpecViolation` on fail.
43
+
- `report::wrong_aud_rejected` — self-mint JWT with `aud` set to a bogus bare DID → expect 401. `SpecViolation` on fail.
44
+
- `report::wrong_lxm_rejected` — self-mint JWT with `lxm = "com.atproto.server.getSession"` → expect 401. `SpecViolation` on fail.
45
+
- `report::expired_rejected` — self-mint JWT with `exp` 300s in the past → expect 401. `SpecViolation` on fail.
46
+
- `report::rejected_shape_returns_400` — self-mint JWT with valid claims but unadvertised `reasonType` → expect 400 `InvalidRequest`. `Advisory` on fail.
47
+
- `report::self_mint_accepted` — self-mint JWT with advertised shape, sentinel-reason, local-or-non-local pollution policy → expect 2xx with `createReport#output` body shape. `SpecViolation` on fail.
48
+
- `report::pds_service_auth_accepted` — PDS-minted JWT via `getServiceAuth`, delivered directly → expect 2xx. `SpecViolation` on fail.
49
+
- `report::pds_proxied_accepted` — POSTed to PDS with `atproto-proxy` header; PDS forwards → expect 2xx. `SpecViolation` on fail.
50
+
51
+
**CLI surface.** New flags added to `LabelerCmd`:
52
+
53
+
- `--handle <h>` and `--app-password <p>` — `clap(requires = "…")` enforces both-or-neither at parse time. When supplied, enables modes 2 and 3.
54
+
- `--report-subject-did <did>` — override the computed default subject.
55
+
- `--commit-report` — opt-in to all committing checks and assert reporting conformance (contract-missing becomes a SpecViolation).
56
+
- `--force-self-mint` — override the locality heuristic; force self-mint checks to run regardless of labeler endpoint classification.
57
+
- `--self-mint-curve {es256,es256k}` — select the self-mint key curve; default `es256k`.
58
+
59
+
Exit-code semantics unchanged: any `SpecViolation` → 1, else any `NetworkError` → 2, else 0.
60
+
61
+
**Pollution avoidance.** When a positive check actually POSTs AND the labeler endpoint is classified non-local:
62
+
- Prefer `reasonOther` from `com.atproto.moderation.defs#reasonType` if advertised; fall back to the lex-first advertised entry.
63
+
- Prefer `record` subject type with a hard-coded AT-URI (a to-be-created explanation post) if advertised; fall back to `account` subject type pointing at the reporter's own DID.
64
+
- `reason` field contains a stable sentinel — `"atproto-devtool conformance test <UTC-RFC3339> <run-id>"` — so moderators can identify and dismiss.
65
+
66
+
For local labelers, use lex-first `reasonType` and `account` subject with the reporter's own DID — pollution is a non-concern in a developer's own queue.
67
+
68
+
**Quality bar.**
69
+
70
+
- Rich miette diagnostics with stable `code = "labeler::report::..."` strings for every new error. The tool must be useful for debugging a broken labeler, not just pass/fail.
71
+
- Every `report::*` check ID and diagnostic code pinned in insta snapshots.
72
+
- One new network seam `CreateReportTee` (mirroring `RawHttpTee`), production impl `RealCreateReportTee`, test fake `FakeCreateReportTee` in `tests/common/mod.rs`.
73
+
- Integration test binary `tests/labeler_report.rs` wired into the same fake-based pattern as other per-stage integration tests.
74
+
- `AnySigningKey` newtype introduced alongside existing `AnyVerifyingKey` in `src/common/identity.rs`; no new crate dependencies (JWT hand-rolled on existing `k256`/`p256`/`sha2`/`base64`; ephemeral did:web server hand-rolled on `tokio::net::TcpListener`).
75
+
- All existing tests continue to pass; no regressions in identity / http / subscription / crypto output.
76
+
77
+
**Explicitly out of scope for v1.**
78
+
79
+
- Verifying label state changes after a submitted report (no post-commit polling of `queryLabels`).
80
+
- `report::zero_retention_probe` (low signal, deferred).
81
+
- `report::clock_skew_accepted` (the reference implementation has zero skew tolerance by default; check would be noise, deferred).
82
+
- PUT / DELETE "negate" API testing (labeler-implementation vocabulary, not protocol).
83
+
- Issuer modes other than ephemeral localhost did:web (self-mint) and the user's own PDS (PDS-mediated modes) — no configurable external did:web, no user-supplied raw signing keys, no `did:key` issuer.
84
+
85
+
## Acceptance Criteria
86
+
87
+
### labeler-report-stage.AC1: `report::contract_published` behavior
88
+
89
+
- **labeler-report-stage.AC1.1 Success:** Labeler advertises non-empty `reasonTypes` and `subjectTypes` → check emits `Pass`.
90
+
- **labeler-report-stage.AC1.2 Success (stage-skip):** No `--commit-report`, contract missing → every `report::*` check emits `Skipped` with reason "labeler does not advertise report acceptance".
91
+
- **labeler-report-stage.AC1.3 Failure:** `--commit-report` set, contract missing → `report::contract_published` emits `SpecViolation` with diagnostic `labeler::report::contract_missing`; all other checks emit `Skipped` with reason "blocked by `report::contract_published`".
92
+
- **labeler-report-stage.AC1.4 Edge:** Empty arrays (`reasonTypes: []`) treated identically to absent field.
93
+
94
+
### labeler-report-stage.AC2: No-JWT negative checks
95
+
96
+
- **labeler-report-stage.AC2.1 Success:** `unauthenticated_rejected` emits `Pass` when labeler returns 401 with non-empty atproto error envelope for unauthenticated POST.
97
+
- **labeler-report-stage.AC2.2 Failure:** `unauthenticated_rejected` emits `SpecViolation` (diagnostic `labeler::report::unauthenticated_accepted`) when labeler returns 2xx.
98
+
- **labeler-report-stage.AC2.3 Success:** `malformed_bearer_rejected` emits `Pass` when labeler returns 401 for garbage Bearer.
99
+
- **labeler-report-stage.AC2.4 Failure:** `malformed_bearer_rejected` emits `SpecViolation` (diagnostic `labeler::report::malformed_bearer_accepted`) when labeler accepts garbage Bearer.
100
+
- **labeler-report-stage.AC2.5 Edge:** 401 with empty or missing `error` envelope field still treated as `Pass` on status alone; summary text notes the non-conformant response shape.
101
+
102
+
### labeler-report-stage.AC3: Self-mint negative checks
103
+
104
+
- **labeler-report-stage.AC3.1 Success:** `wrong_aud_rejected` emits `Pass` when labeler returns 401 for fresh-signed JWT with mutated `aud`.
105
+
- **labeler-report-stage.AC3.2 Failure:** emits `SpecViolation` (diagnostic `labeler::report::wrong_aud_accepted`) when labeler returns 2xx for wrong aud.
106
+
- **labeler-report-stage.AC3.3 Success/Failure pair:** `wrong_lxm_rejected` behaves analogously for mutated `lxm`; diagnostic `labeler::report::wrong_lxm_accepted`.
107
+
- **labeler-report-stage.AC3.4 Success/Failure pair:** `expired_rejected` behaves analogously for past-expiry JWT; diagnostic `labeler::report::expired_accepted`.
108
+
- **labeler-report-stage.AC3.5 Success:** `rejected_shape_returns_400` emits `Pass` when labeler returns 400 `InvalidRequest` for unadvertised `reasonType`.
109
+
- **labeler-report-stage.AC3.6 Advisory:** `rejected_shape_returns_400` emits `Advisory` (diagnostic `labeler::report::shape_not_400`) when labeler returns 401 or 500 (rejection for wrong reason).
110
+
- **labeler-report-stage.AC3.7 Skip:** Every self-mint negative check emits `Skipped` with reason naming the `--force-self-mint` override when heuristic classifies labeler non-local.
111
+
- **labeler-report-stage.AC3.8 Override:** `--force-self-mint` bypasses the heuristic; all self-mint checks run regardless of hostname.
112
+
113
+
### labeler-report-stage.AC4: Self-mint positive check
114
+
115
+
- **labeler-report-stage.AC4.1 Success (local labeler):** `self_mint_accepted` emits `Pass` using lex-first `reasonType` and account subject = reporter DID when labeler returns 2xx.
116
+
- **labeler-report-stage.AC4.2 Success (non-local labeler):** `self_mint_accepted` emits `Pass` using `reasonOther` (if advertised) and `record` subject with hard-coded AT-URI (if advertised) when labeler returns 2xx.
117
+
- **labeler-report-stage.AC4.3 Failure:** emits `SpecViolation` (diagnostic `labeler::report::self_mint_rejected`) when labeler returns non-2xx.
118
+
- **labeler-report-stage.AC4.4 Skip (no commit):** emits `Skipped` with reason naming the `--commit-report` gate.
119
+
- **labeler-report-stage.AC4.5 Skip (not viable):** emits `Skipped` with the self-mint-unreachable reason when heuristic trips.
120
+
- **labeler-report-stage.AC4.6 Sentinel:** The `reason` field in the submitted POST body contains the stable sentinel string `"atproto-devtool conformance test <RFC3339> <run-id>"`.
121
+
122
+
### labeler-report-stage.AC5: PDS `getServiceAuth` mode
123
+
124
+
- **labeler-report-stage.AC5.1 Success:** `pds_service_auth_accepted` emits `Pass` when `createSession` + `getServiceAuth` + labeler POST all succeed.
125
+
- **labeler-report-stage.AC5.2 Failure (labeler-side):** emits `SpecViolation` (diagnostic `labeler::report::pds_service_auth_rejected`) when labeler returns non-2xx for the PDS-minted JWT.
126
+
- **labeler-report-stage.AC5.3 Failure (PDS-side):** emits `NetworkError` when PDS is unreachable, credentials are rejected, or `getServiceAuth` returns an error.
127
+
- **labeler-report-stage.AC5.4 Skip:** emits `Skipped` with reason "requires --handle, --app-password, and --commit-report" when any of the three are missing.
128
+
129
+
### labeler-report-stage.AC6: PDS-proxied mode
130
+
131
+
- **labeler-report-stage.AC6.1 Success:** `pds_proxied_accepted` emits `Pass` when the proxied POST returns 2xx from the PDS.
132
+
- **labeler-report-stage.AC6.2 Failure (labeler-side):** emits `SpecViolation` (diagnostic `labeler::report::pds_proxied_rejected`) when the PDS surfaces a labeler-side rejection (status/error envelope indicating downstream 4xx/5xx).
133
+
- **labeler-report-stage.AC6.3 Failure (PDS-side):** emits `NetworkError` when PDS is unreachable or rejects the proxy attempt itself.
134
+
- **labeler-report-stage.AC6.4 Skip:** emits `Skipped` ("requires --handle, --app-password, and --commit-report") when any of the three are missing.
135
+
136
+
### labeler-report-stage.AC7: Never-short-circuit and row-count invariants
137
+
138
+
- **labeler-report-stage.AC7.1 Row count:** Every `test labeler` run that reaches the report stage emits exactly 10 `report::*` `CheckResult` rows, regardless of flag or environment combinations.
139
+
- **labeler-report-stage.AC7.2 Row order:** Row order is stable and matches the DoD list top-to-bottom.
140
+
141
+
### labeler-report-stage.AC8: CLI flag handling
142
+
143
+
- **labeler-report-stage.AC8.1 Both-or-neither:** `--handle` without `--app-password` (and vice versa) produces a clap parse error before any stage runs.
144
+
- **labeler-report-stage.AC8.2 Curve selection:** `--self-mint-curve es256` advertises a P-256 key in the did:web DID doc; `es256k` advertises secp256k1; the minted JWT's `alg` header matches.
145
+
- **labeler-report-stage.AC8.3 Subject override:** `--report-subject-did <did>` replaces the computed default subject in the body of committing checks.
146
+
- **labeler-report-stage.AC8.4 Exit codes:** Exit 1 on any `SpecViolation`; exit 2 on any `NetworkError` absent `SpecViolation`; exit 0 otherwise — unchanged from existing stages.
147
+
148
+
## Glossary
149
+
150
+
- **atproto**: The AT Protocol — an open, federated social networking protocol developed by Bluesky. Defines the wire formats, lexicons, identity system, and service roles (PDS, relay, labeler, etc.) that power Bluesky and compatible applications.
151
+
- **labeler**: An atproto service that receives moderation reports and emits signed labels (annotations) on accounts or content. Labelers are independent services; the Bluesky moderation infrastructure is one example but users can subscribe to any labeler.
152
+
- **DID (Decentralized Identifier)**: A W3C standard identifier of the form `did:<method>:<id>`. In atproto, every account and service has a DID as its stable, globally unique identity anchor. A DID resolves to a DID document describing the entity's keys and service endpoints.
153
+
- **did:web**: A DID method that resolves via HTTPS — `did:web:example.com` resolves by fetching `https://example.com/.well-known/did.json`. The self-mint mode uses an ephemeral `did:web` on `127.0.0.1:<port>`.
154
+
- **did:plc**: The primary DID method used for atproto user accounts. Identities are controlled through a signed log of operations stored at `plc.directory`. Unlike `did:web`, it does not require the subject to host their own infrastructure.
155
+
- **XRPC**: The RPC protocol used by atproto. Calls are HTTP requests to `GET /xrpc/<nsid>` (queries) or `POST /xrpc/<nsid>` (procedures), where the NSID identifies the Lexicon method.
156
+
- **PDS (Personal Data Server)**: The server that hosts a user's atproto repository — their posts, follows, and other records. The PDS also acts as an authentication intermediary: it can mint service-auth JWTs on behalf of a logged-in user and proxy XRPC calls to downstream services.
157
+
- **Ozone**: The reference open-source labeler and moderation server implementation from Bluesky. Many independent labelers run Ozone. The error envelope vocabulary documented in the design (`BadJwt`, `BadJwtAudience`, etc.) originates from Ozone's `@atproto/xrpc-server`.
158
+
- **service-auth JWT**: A short-lived JSON Web Token that atproto services use to authenticate machine-to-machine calls. Unlike a user's access token, a service-auth JWT carries an `aud` (audience, the target service's DID) and an `lxm` (the specific Lexicon method being called), tightly scoping what the token may be used for.
159
+
- **atproto-proxy header**: An HTTP request header (`atproto-proxy: <did>#<service-id>`) sent by a client to its PDS asking the PDS to forward the request to the named service on the client's behalf. The PDS resolves the DID, mints an appropriate service-auth JWT, and proxies the call.
160
+
- **`lxm` claim**: A custom JWT claim in atproto's service-auth spec containing the NSID (namespaced identifier) of the Lexicon method the token authorizes. A labeler must reject tokens whose `lxm` does not match the method being invoked.
161
+
- **`iss` / `aud` claims**: Standard JWT claims for issuer and audience. In atproto service-auth, `iss` is the caller's DID and `aud` is the target service's DID. A labeler must reject tokens with an `aud` that does not match its own DID.
162
+
- **`jti` claim**: Standard JWT claim for a unique token identifier. atproto service-auth includes a random `jti` nonce to prevent replay attacks.
163
+
- **`com.atproto.moderation.createReport`**: The XRPC procedure that clients call to submit a moderation report to a labeler. This is the method under test throughout the design.
164
+
- **`LabelerPolicies`**: The structured data published in a labeler's DID document (and reflected in its atproto records) declaring which `reasonTypes` and `subjectTypes` it accepts for reports. The stage treats this as the "contract" the labeler has advertised.
165
+
- **`reasonTypes` / `subjectTypes`**: Fields within `LabelerPolicies`. `reasonTypes` lists the moderation reason categories the labeler accepts (e.g., `reasonSpam`, `reasonOther`); `subjectTypes` lists what can be reported (`account`, `record`).
166
+
- **Lexicon**: The atproto schema definition language. Lexicons define the types, fields, and constraints for every XRPC method and record type. `com.atproto.moderation.defs#reasonType` is a Lexicon-defined enum of moderation reason categories.
167
+
- **insta**: A Rust snapshot-testing library. Tests call `insta::assert_snapshot!()` and the library writes `.snap` files that are committed to the repository. Reviewing and approving output changes is done via `cargo insta review`.
168
+
- **miette**: A Rust diagnostics library that renders rich, human-readable error messages with source spans, labels, and diagnostic codes. All user-facing errors in this codebase carry a stable `code = "..."` string so users can search for documentation on a specific error.
169
+
- **clap**: The Rust command-line argument parsing library used by this tool. The design uses clap's `requires` attribute to enforce that `--handle` and `--app-password` must be supplied together.
170
+
- **ES256 / ES256K**: JWT algorithm identifiers for ECDSA signatures. ES256 uses the P-256 elliptic curve (NIST); ES256K uses secp256k1 (the same curve used by Bitcoin). atproto accounts predominantly use ES256K; P-256 is increasingly common for hardware-backed keys.
171
+
- **low-s ECDSA normalization**: ECDSA signatures have two mathematically valid forms (high-s and low-s). atproto requires the low-s form. The `AnySigningKey::sign` method normalizes to low-s to match the convention already established in `AnySignature`.
172
+
- **sentinel reason**: A stable, recognizable string embedded in the `reason` field of test report submissions — `"atproto-devtool conformance test <RFC3339> <run-id>"` — so that labeler operators can identify and dismiss reports submitted by the conformance tool.
173
+
- **multibase**: A self-describing encoding prefix scheme (e.g., `z` prefix for base58btc). atproto DID documents encode public keys as multibase-prefixed, multicodec-prefixed byte strings. The existing `AnyVerifyingKey` type in `src/common/identity.rs` already handles multibase decoding.
174
+
- **`k256` / `p256`**: Rust crates implementing secp256k1 and P-256 elliptic curve cryptography respectively. Used directly for signing and verification rather than through a higher-level JWT library, avoiding a new dependency.
175
+
- **did:web on localhost**: A `did:web` whose hostname is `127.0.0.1:<port>`. The self-mint mode relies on this: the tool binds a port, serves a DID document there, and constructs a `did:web:127.0.0.1:<port>` identity. The labeler must be able to reach `127.0.0.1` to resolve and verify the identity, which is why this mode is gated on the labeler itself being local.
176
+
177
+
## Architecture
178
+
179
+
The report stage is a new module `src/commands/test/labeler/create_report.rs`. It exposes `pub async fn run(...) -> CreateReportStageOutput` and is invoked as the final stage in `pipeline::run_pipeline` after the existing `crypto::run(...)` call. Placing it last matches the existing "reads before writes" ordering and keeps side-effectful POSTs at the tail of the pipeline.
180
+
181
+
Stage inputs:
182
+
183
+
- `identity_facts: &IdentityFacts` (labeler DID, endpoint URL, `labeler_policies`). Required; stage emits all 10 checks as `Skipped` with reason "blocked by identity stage" when `None`.
184
+
- `report_tee: &dyn CreateReportTee` (the new I/O seam for POSTing createReport to the labeler).
185
+
- `http: &dyn HttpClient` (reused for PDS calls in modes 2 and 3).
186
+
- `self_mint_signer: Option<&SelfMintSigner>` — always present in production (the baseline); `None` only in tests that exercise the "self-mint not viable, no override" path by construction.
187
+
- `pds_credentials: Option<&PdsCredentials>` — present when `--handle` + `--app-password` supplied.
188
+
- `report_subject_override: Option<&Did>`, `commit_report: bool`, `self_mint_viable: bool` — CLI-derived booleans and optional overrides.
189
+
190
+
Stage output: `CreateReportStageOutput { facts: Option<CreateReportFacts>, results: Vec<CheckResult> }`. `CreateReportFacts` is minimal — one `Option<bool>` per positive check (`self_mint_succeeded`, `pds_service_auth_succeeded`, `pds_proxied_succeeded`) for possible future consumer stages.
191
+
192
+
**Key components introduced:**
193
+
194
+
- **`CreateReportTee`** trait (new test seam): `async fn post_create_report(&self, auth: Option<&str>, body: &CreateReportRequest) -> Result<RawCreateReportResponse, CreateReportStageError>`. Real impl wraps `reqwest::Client`; fake impl in `tests/common/mod.rs` records the last request and returns scripted responses keyed by call index.
195
+
- **`SelfMintSigner`** concrete struct: owns an `AnySigningKey`, an issuer DID, and a `DidDocServer` handle. Exposes `sign_jwt(claims: JwtClaims) -> String` for the stage to build both valid and mutated JWTs.
196
+
- **`DidDocServer`** RAII type: binds `127.0.0.1:0`, spawns a tokio task serving a single JSON response at `/.well-known/did.json`, shuts down on drop. Hand-rolled on `tokio::net::TcpListener` — no web framework dependency.
197
+
- **`AnySigningKey`** newtype enum in `src/common/identity.rs`: variants `Secp256k1(k256::ecdsa::SigningKey)`, `P256(p256::ecdsa::SigningKey)`. Mirrors the existing `AnyVerifyingKey`. Method `sign(msg: &[u8]) -> AnySignature` produces low-s-normalized ECDSA signatures.
198
+
- **`PdsJwtFetcher`** concrete type: wraps an `&dyn HttpClient` and a PDS endpoint URL; exposes `create_session(...)` and `mint_valid(aud, lxm)` for mode 2.
199
+
- **`PdsProxiedPoster`** concrete type: wraps an `&dyn HttpClient` and PDS endpoint; exposes `post_via_proxy(labeler_did, body, access_jwt)` for mode 3.
200
+
201
+
**JWT construction (self-mint, hand-rolled).** Header: `{"typ":"JWT","alg":"ES256K"}` or `ES256`. Compact form: `base64url(header) + "." + base64url(claims) + "." + base64url(sig)`. Claims shape:
202
+
203
+
```rust
204
+
pub struct JwtClaims {
205
+
pub iss: Did, // did:web:127.0.0.1:PORT for self-mint
206
+
pub aud: Did, // bare labeler DID (no #atproto_labeler fragment)
207
+
pub exp: i64,
208
+
pub iat: i64,
209
+
pub lxm: Nsid, // com.atproto.moderation.createReport
210
+
pub jti: String, // 16 random bytes hex
211
+
// nbf deliberately omitted — not in atproto service-auth spec.
212
+
}
213
+
```
214
+
215
+
Negative-test claim mutations are applied to a `valid_claims_template()` before signing, so each mutated JWT is freshly signed by the self-mint key. This isolates the labeler's rejection to the mutated claim, regardless of whether the labeler checks claims or signature first.
216
+
217
+
**PDS call shapes (modes 2 and 3).** Mode 2 issues two XRPC calls (`com.atproto.server.createSession`, `com.atproto.server.getServiceAuth`) then hands the returned JWT to `CreateReportTee` verbatim. Mode 3 issues one XRPC call — POST `com.atproto.moderation.createReport` to the PDS with `Authorization: Bearer <access_jwt>` and `atproto-proxy: <labeler-did>#atproto_labeler` — and reads the PDS's response as the outcome. Tool never handles the service-auth JWT in mode 3.
218
+
219
+
**Self-mint viability heuristic.** Applied to the labeler endpoint URL's hostname:
220
+
221
+
- Classified local (self-mint viable): `localhost`, `127.0.0.1`, `::1`, hostnames ending in `.local`, RFC 1918 ranges (`10.*`, `172.16.*`–`172.31.*`, `192.168.*`).
222
+
- Classified non-local: everything else.
223
+
- `--force-self-mint` overrides the classification.
224
+
225
+
## Existing Patterns
226
+
227
+
Investigation verified the following patterns, and the design follows them strictly:
228
+
229
+
- **Stage shape:** module-level `pub async fn run(...)` returning `*StageOutput { facts: Option<*Facts>, results: Vec<CheckResult> }`. Established by `src/commands/test/labeler/identity.rs` (`run` + `IdentityFacts`), `http.rs` (`run` + `HttpStageOutput`), `subscription.rs`, and `crypto.rs`.
230
+
- **Never short-circuit invariant:** every check in a stage emits exactly one `CheckResult` per run, with "blocked by …" `Skipped` reasons when prerequisites fail. Enforced by per-stage loops in `pipeline::run_pipeline` (`src/commands/test/labeler/pipeline.rs:197-381`).
231
+
- **Check ID namespacing:** `identity::xxx`, `http::yyy`, `subscription::zzz`, `crypto::www`. New IDs use `report::...`.
232
+
- **Miette diagnostic codes:** stable `code = "labeler::<stage>::<subject>"` strings (e.g., `labeler::identity::labeler_service_present` at `src/commands/test/labeler/identity.rs:64`). Pinned in `tests/snapshots/` — renaming one is a breaking change.
233
+
- **Single new I/O trait per stage:** `RawHttpTee` in http, `WebSocketClient + FrameStream` in subscription. New stage gets `CreateReportTee`. Incidental HTTP reuses the existing `HttpClient` trait from `src/common/identity.rs:282`.
234
+
- **Newtype enums for crypto primitives:** `AnyVerifyingKey` (`src/common/identity.rs:112-117`) wraps `k256` and `p256` verifying keys; `AnySignature` (same file, line 150) wraps their signatures. New `AnySigningKey` follows the identical shape for signing keys.
235
+
- **Integration test structure:** one `tests/labeler_<stage>.rs` binary per stage, fixtures under `tests/fixtures/labeler/<stage>/`, shared fakes in `tests/common/mod.rs`. New stage gets `tests/labeler_report.rs` and `tests/fixtures/labeler/report/`.
236
+
- **Clap derive with `requires`:** argument dependencies enforced at parse time, as already practiced in the workspace's CLI conventions.
237
+
238
+
One new pattern is introduced — the ephemeral `DidDocServer` (single-endpoint tokio HTTP server). Scoped narrowly to this stage. Not positioned as a reusable framework; if future stages need similar facilities, they can be generalized then.
239
+
240
+
## Implementation Phases
241
+
242
+
<!-- START_PHASE_1 -->
243
+
### Phase 1: Shared primitives
244
+
245
+
**Goal:** Land the crate-wide primitives that later phases depend on, without yet introducing any report-stage code.
246
+
247
+
**Components:**
248
+
- `AnySigningKey` enum in `src/common/identity.rs` — variants `Secp256k1(k256::ecdsa::SigningKey)` and `P256(p256::ecdsa::SigningKey)`; method `sign(msg: &[u8]) -> AnySignature` producing low-s-normalized signatures matching the existing `AnySignature` conventions.
249
+
- New module `src/common/jwt.rs` — `JwtHeader`, `JwtClaims` structs; `encode_compact(header, claims, signer) -> String` helper; `decode_compact(token) -> (JwtHeader, JwtClaims, sig_bytes)` helper for round-trip verification in tests.
250
+
- Sentinel-reason builder in `src/commands/test/labeler/create_report/sentinel.rs` (small module; produces `"atproto-devtool conformance test <RFC3339> <run-id>"`).
251
+
- Local-labeler viability heuristic — function in `src/common/identity.rs` classifying a `&url::Url`'s hostname against the rules above, returning `bool`.
252
+
253
+
**Dependencies:** None (first phase).
254
+
255
+
**Done when:** `cargo build` succeeds; unit tests pass for (a) AnySigningKey sign/verify round-trip in both curves against AnyVerifyingKey, (b) JWT encode+decode round-trip, (c) hostname classifier over a table of cases including RFC 1918 ranges and `.local` suffixes. No ACs directly covered — this phase is infrastructure.
256
+
<!-- END_PHASE_1 -->
257
+
258
+
<!-- START_PHASE_2 -->
259
+
### Phase 2: Self-mint JWT infrastructure
260
+
261
+
**Goal:** Produce a `SelfMintSigner` that can mint a JWT the labeler can resolve and verify end-to-end.
262
+
263
+
**Components:**
264
+
- `DidDocServer` type in `src/commands/test/labeler/create_report/did_doc_server.rs` — RAII struct that binds `127.0.0.1:0`, spawns a tokio task serving one JSON body at `/.well-known/did.json`, and closes on drop. Hand-rolled HTTP/1.1 reply on `tokio::net::TcpListener`.
265
+
- `SelfMintSigner` struct in `src/commands/test/labeler/create_report/self_mint.rs` — owns an `AnySigningKey`, the issuer `Did` (built from the port), and a `DidDocServer` handle; exposes `sign_jwt(claims: JwtClaims) -> String` and `issuer_did() -> &Did`.
266
+
- Curve-selection parser for `--self-mint-curve` (clap `ValueEnum`).
267
+
268
+
**Dependencies:** Phase 1 (AnySigningKey, JWT helpers).
269
+
270
+
**Done when:** Integration test verifies a `SelfMintSigner` round-trip — a second `tokio::task` fetches the DID doc over HTTP, extracts the multibase key, and verifies a JWT signed by the signer using `AnyVerifyingKey`. Test runs in both curves.
271
+
<!-- END_PHASE_2 -->
272
+
273
+
<!-- START_PHASE_3 -->
274
+
### Phase 3: `CreateReportTee` seam
275
+
276
+
**Goal:** Stand up the test seam for the labeler POST so all later functionality phases have a mockable entry point.
277
+
278
+
**Components:**
279
+
- `CreateReportTee` trait in `src/commands/test/labeler/create_report.rs` module root.
280
+
- `RealCreateReportTee` concrete type wrapping `reqwest::Client` + labeler endpoint URL.
281
+
- `FakeCreateReportTee` in `tests/common/mod.rs` — scripted per-call-index responses, records last request (`auth` header + body) for assertion by tests.
282
+
- `RawCreateReportResponse` struct (status, headers, raw body, content-type) for preserved diagnostics.
283
+
284
+
**Dependencies:** None beyond existing crate; can proceed in parallel with Phase 2.
285
+
286
+
**Done when:** Smoke test proves `FakeCreateReportTee` can serve a scripted sequence of responses and record the last request's Authorization header and body for inspection.
287
+
<!-- END_PHASE_3 -->
288
+
289
+
<!-- START_PHASE_4 -->
290
+
### Phase 4: Stage scaffolding and contract check
291
+
292
+
**Goal:** Wire the new stage into the pipeline with the `report::contract_published` check operational in all four gating outcomes.
293
+
294
+
**Components:**
295
+
- Module `src/commands/test/labeler/create_report.rs` with `pub async fn run(...) -> CreateReportStageOutput`, `CreateReportStageOutput`, `CreateReportFacts`, `CreateReportStageError` (thiserror + miette).
296
+
- `report::contract_published` check logic using `IdentityFacts::labeler_policies`; diagnostic `labeler::report::contract_missing` when `commit_report == true` and contract is empty.
297
+
- Pipeline integration: new call after `crypto::run(...)` in `src/commands/test/labeler/pipeline.rs` around line 381; new field on `LabelerOptions` for the `CreateReportTee` ref.
298
+
- CLI plumbing in `src/commands/test/labeler.rs`: `--commit-report`, `--self-mint-curve`, `--force-self-mint`, `--report-subject-did` (other flags arrive in Phase 7).
299
+
- Initial insta snapshots for: contract present + commit false; contract present + commit true; contract missing + commit false (whole-stage skip); contract missing + commit true (SpecViolation + 9 blocked rows).
300
+
301
+
**Dependencies:** Phase 3 (CreateReportTee plumbed into LabelerOptions).
302
+
303
+
**Done when:** Four new integration tests pass in `tests/labeler_report.rs` covering every contract × commit combination. Snapshots pinned. Covers **labeler-report-stage.AC1.1** through **AC1.4**.
304
+
<!-- END_PHASE_4 -->
305
+
306
+
<!-- START_PHASE_5 -->
307
+
### Phase 5: No-JWT negative checks
308
+
309
+
**Goal:** Cover the two checks that don't depend on self-mint infrastructure (no JWT is constructed) — they can run against any reachable labeler.
310
+
311
+
**Components:**
312
+
- `report::unauthenticated_rejected` implementation — POST with `auth: None`.
313
+
- `report::malformed_bearer_rejected` implementation — POST with `auth: Some("not-a-jwt")`.
314
+
- Response assertion helper: status must be 401 AND response body must be parseable as the atproto error envelope with a non-empty `error` field. Diagnostic codes `labeler::report::unauthenticated_accepted`, `labeler::report::malformed_bearer_accepted`.
315
+
- Fixtures: 401 with typical envelope, 200 (failure path), non-envelope 401 response.
316
+
317
+
**Dependencies:** Phase 4.
318
+
319
+
**Done when:** Integration tests cover both checks' success and failure paths. Covers **labeler-report-stage.AC2.1** through **AC2.4**.
320
+
<!-- END_PHASE_5 -->
321
+
322
+
<!-- START_PHASE_6 -->
323
+
### Phase 6: Self-mint negative checks
324
+
325
+
**Goal:** The four checks requiring a valid-signature JWT with mutated claims or body.
326
+
327
+
**Components:**
328
+
- `report::wrong_aud_rejected` — mutate `aud` to `did:plc:0000000000000000000000000`.
329
+
- `report::wrong_lxm_rejected` — mutate `lxm` to `com.atproto.server.getSession`.
330
+
- `report::expired_rejected` — mutate `exp` to `now - 300`, `iat` to `now - 360`.
331
+
- `report::rejected_shape_returns_400` — valid claims but body with a `reasonType` not in `labeler_policies.reasonTypes`; assert status 400 with `error: "InvalidRequest"`.
332
+
- Self-mint viability gating applied — when `self_mint_viable == false`, all four emit `Skipped` with the override hint.
333
+
- Diagnostic codes `labeler::report::wrong_aud_accepted`, `wrong_lxm_accepted`, `expired_accepted`, `shape_not_400`.
334
+
335
+
**Dependencies:** Phase 2 (SelfMintSigner), Phase 4 (stage wiring).
336
+
337
+
**Done when:** Integration tests cover each check's success/failure paths and the viability-heuristic skip path. Covers **labeler-report-stage.AC3.1** through **AC3.6**.
338
+
<!-- END_PHASE_6 -->
339
+
340
+
<!-- START_PHASE_7 -->
341
+
### Phase 7: Self-mint positive check + pollution avoidance
342
+
343
+
**Goal:** Land `report::self_mint_accepted` with the pollution-avoidance rules applied based on labeler locality.
344
+
345
+
**Components:**
346
+
- `report::self_mint_accepted` implementation — well-formed JWT, advertised `reasonType`/`subject`, sentinel `reason`, expect 2xx + `createReport#output` body shape.
347
+
- Pollution-avoidance helper module: `choose_reason_type(policies, local) -> Nsid` (prefers `reasonOther` when non-local), `choose_subject(policies, local, reporter_did, subject_override) -> Subject` (prefers `record` with hard-coded AT-URI when non-local and advertised).
348
+
- Placeholder constant `const CONFORMANCE_REPORT_SUBJECT_URI: &str = "<TBD: at://did:plc:... — release-gate>"` with a release-checklist entry in the implementation plan.
349
+
- Diagnostic code `labeler::report::self_mint_rejected`.
350
+
- `--commit-report` gating: emits `Skipped` with advisory reason when false.
351
+
352
+
**Dependencies:** Phase 6 (self-mint plumbing).
353
+
354
+
**Done when:** Integration tests cover: successful 2xx (local labeler, lex-first reasonType, account subject); successful 2xx (non-local, reasonOther, record subject); 4xx failure emits SpecViolation; commit-false skip; self-mint-unviable skip. Covers **labeler-report-stage.AC4.1** through **AC4.6**.
355
+
<!-- END_PHASE_7 -->
356
+
357
+
<!-- START_PHASE_8 -->
358
+
### Phase 8: PDS modes and end-to-end integration
359
+
360
+
**Goal:** Land the two PDS-backed positive checks, the remaining CLI flags, and the complete end-to-end snapshot coverage for the 10-check contract.
361
+
362
+
**Components:**
363
+
- `PdsJwtFetcher` concrete type — calls `com.atproto.server.createSession` and `com.atproto.server.getServiceAuth` via the existing `HttpClient` seam.
364
+
- `PdsProxiedPoster` concrete type — POSTs `com.atproto.moderation.createReport` to the PDS with `atproto-proxy` header via `HttpClient`.
365
+
- `report::pds_service_auth_accepted` — uses `PdsJwtFetcher` to mint JWT, then `CreateReportTee` to POST to labeler.
366
+
- `report::pds_proxied_accepted` — uses `PdsProxiedPoster` to POST via PDS.
367
+
- Diagnostic codes `labeler::report::pds_service_auth_rejected`, `labeler::report::pds_proxied_rejected`.
368
+
- CLI flags: `--handle`, `--app-password` (with clap `requires`).
369
+
- End-to-end snapshot fixtures: `all_pass_local_labeler/` (7 applicable checks pass, PDS checks Skipped), `all_pass_full_suite/` (all 10 checks pass), `all_fail_misconfigured_labeler/` (negatives pass, positives fail).
370
+
371
+
**Dependencies:** Phase 7.
372
+
373
+
**Done when:** Integration tests cover each PDS mode's success/failure/skip paths; end-to-end snapshots confirm the full 10-row output contract across representative runtime configurations. Covers **labeler-report-stage.AC5.1** through **AC5.5** and **AC6.1** through **AC6.5**. Also verifies **AC7.1** (always-10-rows invariant) and **AC8** (CLI-flag validation) via `tests/labeler_cli.rs` extensions.
374
+
<!-- END_PHASE_8 -->
375
+
376
+
## Additional Considerations
377
+
378
+
**Pre-release TODO: hard-coded subject AT-URI.** Phase 7 introduces `CONFORMANCE_REPORT_SUBJECT_URI` as a placeholder constant. Before v1 ships, the user must (a) publish an explanation post on an atproto account explaining what atproto-devtool is, (b) capture its AT-URI and CID, and (c) replace the placeholder constant. The implementation plan must include this as an explicit release-gate checklist item, not code work.
379
+
380
+
**Error envelope assertion is deliberately loose.** Research confirmed that labelers using `@atproto/xrpc-server` return specific error names (`AuthenticationRequired`, `BadJwt`, `BadJwtAudience`, `BadJwtLexiconMethod`, `JwtExpired`, `BadJwtSignature`). Non-xrpc-server labelers may use different names while still being conformant. Negative checks assert "status = 401 AND envelope has a non-empty `error` field," not specific error strings. A SecondaryAdvisory surfaces the specific `error` value when it doesn't match Ozone's vocabulary — useful for debugging without causing false SpecViolations.
381
+
382
+
**PDS call failures distinguish from labeler failures.** When mode 2 or mode 3 fails at the PDS call itself (unreachable, invalid credentials, PDS error), the check emits `NetworkError`, not `SpecViolation` — the labeler never got tested. Diagnostic text clarifies which hop failed. This means exit code 2 from a PDS-side failure, exit code 1 from a labeler rejection; the tool's existing exit-code precedence rule holds.
383
+
384
+
**Snapshot churn during development.** Every new `report::*` check ID and diagnostic code is part of the public snapshot contract. Breaking one is a CLI-surface change. The implementation plan should note that snapshot reviews (`cargo insta review`) are the expected workflow during development, and that any phase introducing new IDs must pin them before being considered done.
385
+
386
+
**No cross-run state.** Each `test labeler` invocation is a fresh run; no persisted state between runs. The `<run-id>` in the sentinel reason is generated per invocation and used only within that run.