this repo has no description
docs: document relay hot-path bench
+76
-13
+1
.gitignore
+1
.gitignore
+52
-5
README.md
+52
-5
README.md
···
2
2
3
3
SDK-level benchmarks for AT Protocol relay infrastructure.
4
4
5
-
measures the two CPU-bound bottlenecks a relay hits on every incoming commit: **decode** (CBOR + CAR + DAG-CBOR + CID verification) and **signature verification** (ECDSA). same corpora across all SDKs, verified work parity via block counts, error counts, and entry counts.
5
+
measures the CPU-bound work a relay hits on incoming commits: **decode** (CBOR + CAR + DAG-CBOR + CID verification), **signature verification** (ECDSA), and the combined **relay hot path**. same corpora across SDKs where possible, verified work parity via block counts, error counts, entry counts, and signature validation counts.
6
6
7
7
## what this measures
8
8
···
43
43
| python ([atproto](https://github.com/MarshalX/atproto)) | 33,842 | 163.0 | 9.98 | 0 |
44
44
| go (indigo) | 15,587 | 75.6 | 9.98 | 0 |
45
45
46
-
note: indigo appears in both tables. its number is the same because it always verifies — there is no option to disable it in go-car v1.
46
+
note: indigo appears in both tables. its number is the same because the benchmark uses go-car/v2's default `NewBlockReader`, which verifies block hashes.
47
47
48
48
run-to-run variance is ~30-40%. compare ratios within a single `just bench` run, not across runs.
49
49
···
55
55
|-----|---------------------|-------|
56
56
| zig (zat) | yes (v0.2.1+) | `car.read()` verifies by default; `readWithOptions(.{ .verify_block_hashes = false })` to skip |
57
57
| rust (rsky, rs-car-sync) | yes | `CarReader::new(&mut cursor, true)` — second arg enables verification |
58
-
| go (indigo, go-car v1) | yes (always) | no option to disable in v1 |
58
+
| go (indigo, go-car/v2 default reader) | yes | this benchmark does not use `WithTrustedCAR(true)` |
59
59
| rust (jacquard, iroh-car) | no | not implemented |
60
60
| rust (raw) | no | not implemented |
61
61
| go (raw) | no | not implemented |
···
87
87
- decode every block as DAG-CBOR
88
88
89
89
**what zat and indigo do that isn't obvious:**
90
-
- enforce size limits (2MB max on blocks field, max block count) — zat matches indigo's limits as of v0.2.2
90
+
- enforce size limits (2MB max on blocks field, max block count) — zat has matched indigo's limits since v0.2.2
91
91
92
92
**what none of them do:**
93
93
- DAG-CBOR deterministic encoding validation (sorted keys, minimal integers) — indigo's refmt doesn't check this either
···
177
177
178
178
captured by connecting to the firehose, extracting signed commit blocks from CAR data, and resolving each DID via PLC directory to get the signing key. entries that fail verification are dropped during capture.
179
179
180
+
## relay hot path
181
+
182
+
the relay hot-path benchmark combines the decode and signature work into one CPU-only pass over raw firehose frames with pre-resolved signing keys. it approximates what a relay does for each incoming commit after identity resolution has already populated a DID key cache:
183
+
184
+
1. CBOR decode the frame header and payload
185
+
2. parse the CAR and verify CID hashes
186
+
3. find the commit block
187
+
4. strip `sig`, re-encode the unsigned commit as DAG-CBOR
188
+
5. verify the commit signature with the cached public key
189
+
190
+
the corpus is captured from the live firehose, then each entry is validated before it is written. benchmark runs do not hit the network.
191
+
192
+
### current zig result
193
+
194
+
_3,173 firehose frames (16.0 MB), 5 measured passes, macOS arm64 (M3 Max)_
195
+
196
+
| variant | frames/sec (median) | MB/s | live firehose headroom | verified | errors |
197
+
|---------|--------------------:|-----:|-----------------------:|---------:|------:|
198
+
| zig relay hot path (arena reuse) | 9,793 | 49.5 | 16x | 15,865 | 0 |
199
+
| zig relay hot path (alloc per frame) | 10,194 | 51.6 | 17x | 15,865 | 0 |
200
+
201
+
stage breakdown from the median arena-reuse pass:
202
+
203
+
| stage | time/frame | share |
204
+
|-------|-----------:|------:|
205
+
| CBOR decode | <1µs | 0.3% |
206
+
| CAR parse + CID verification | 3µs | 3.4% |
207
+
| signature verification | 98µs | 96.2% |
208
+
209
+
signature verification dominates the combined path. this is why the standalone decode benchmarks show very large SDK differences, while the full relay hot path is bounded by ECDSA throughput once commit signatures are included.
210
+
211
+
```sh
212
+
just capture-relay # capture frames + resolve signing keys
213
+
just bench-relay # run zig relay hot-path benchmark
214
+
```
215
+
180
216
## corpus format
181
217
182
218
the fixture file (`fixtures/firehose-frames.bin`) uses a simple length-prefixed binary format:
···
205
241
206
242
| lang | SDK | version | CBOR engine | CAR engine |
207
243
|------|-----|---------|-------------|------------|
208
-
| zig | [zat](https://tangled.sh/@zzstoatzz.io/zat) v0.2.2 + [k256](https://tangled.sh/@zzstoatzz.io/k256) v0.0.4 | — | hand-rolled | hand-rolled (+ SHA-256 CID verify, size limits) |
244
+
| zig | [zat](https://tangled.sh/@zzstoatzz.io/zat) v0.2.18 + [k256](https://tangled.sh/@zzstoatzz.io/k256) v0.0.4 | — | hand-rolled | hand-rolled (+ SHA-256 CID verify, size limits) |
209
245
| rust | [rsky](https://github.com/blacksky-algorithms/rsky) stack | — | [ciborium](https://crates.io/crates/ciborium) (header) + [serde_ipld_dagcbor](https://crates.io/crates/serde_ipld_dagcbor) (body) | [rs-car-sync](https://crates.io/crates/rs-car-sync) (+ SHA-256 CID verify) |
210
246
| rust | raw (minicbor + bumpalo) | — | [minicbor](https://crates.io/crates/minicbor) (zero-copy) | hand-rolled (sync) |
211
247
| rust | [jacquard](https://tangled.sh/@nonbinary.computer/jacquard) | 0.9 | [ciborium](https://crates.io/crates/ciborium) (header) + [serde_ipld_dagcbor](https://crates.io/crates/serde_ipld_dagcbor) (body) | [iroh-car](https://crates.io/crates/iroh-car) (async) |
···
269
305
just bench-sigs-rust
270
306
just bench-sigs-go
271
307
308
+
# relay hot path
309
+
just capture-relay # capture frames + resolve signing keys
310
+
just bench-relay # run zig combined relay hot-path benchmark
311
+
272
312
# trust chain verification
273
313
just verify pfrazee.com # run all three implementations
274
314
just chart pfrazee.com # run + generate SVG charts to docs/
275
315
```
276
316
317
+
the Zig package currently targets Zig 0.15.x (`zig/build.zig.zon` sets `minimum_zig_version = "0.15.2"`). if your default `zig` is a 0.16 development build, point the just recipes at a 0.15 binary:
318
+
319
+
```sh
320
+
ZIG=/path/to/zig-0.15 just bench-relay
321
+
```
322
+
277
323
## methodology
278
324
279
325
- `just capture` connects to the live firehose for ~10 seconds, filters for commits with ops via CBOR header peek (uses zat's CBOR decoder — see fairness notes), writes a length-prefixed corpus
326
+
- `just capture-relay` captures raw firehose frames and resolves signing keys up front, so `just bench-relay` is CPU-only and does not hit PLC/PDS services
280
327
- each benchmark decodes every frame fully: header → payload → CAR → decode every block as DAG-CBOR
281
328
- zat and indigo additionally SHA-256 verify every block CID
282
329
- 2 warmup passes, 5 measured passes over the full corpus
+23
-8
justfile
+23
-8
justfile
···
1
1
# atproto firehose benchmarks
2
+
zig := env_var_or_default('ZIG', 'zig')
2
3
3
4
# capture ~10s of firehose traffic + jetstream event
4
5
capture:
5
-
cd zig && zig build run-capture
6
+
cd zig && {{zig}} build run-capture
6
7
7
8
# run all benchmarks
8
9
bench: _ensure-fixtures
···
13
14
@echo "corpus: firehose-frames.bin ($(wc -c < fixtures/firehose-frames.bin | tr -d ' ') bytes)"
14
15
@echo ""
15
16
@echo "--------------------------------------------"
16
-
cd zig && zig build run-bench -Doptimize=ReleaseFast
17
+
cd zig && {{zig}} build run-bench -Doptimize=ReleaseFast
17
18
@echo "--------------------------------------------"
18
19
cd rust-rsky && cargo run --release 2>&1
19
20
@echo "--------------------------------------------"
···
30
31
31
32
# run a single language's bench
32
33
bench-zig: _ensure-fixtures
33
-
cd zig && zig build run-bench -Doptimize=ReleaseFast
34
+
cd zig && {{zig}} build run-bench -Doptimize=ReleaseFast
34
35
35
36
bench-rust: _ensure-fixtures
36
37
cd rust && cargo run --release
···
55
56
# run full AT Protocol trust chain verification with timing (all three implementations)
56
57
verify *ARGS:
57
58
@echo ""
58
-
cd zig && zig build run-verify -Doptimize=ReleaseFast -- {{ARGS}}
59
+
cd zig && {{zig}} build run-verify -Doptimize=ReleaseFast -- {{ARGS}}
59
60
@echo "--------------------------------------------"
60
61
cd go-verify && go run . {{ARGS}}
61
62
@echo "--------------------------------------------"
···
69
70
70
71
# capture signed commits + public keys from firehose (~10s)
71
72
capture-sigs:
72
-
cd zig && zig build run-capture-sigs
73
+
cd zig && {{zig}} build run-capture-sigs
73
74
74
75
# run all sig verify benchmarks
75
76
bench-sigs: _ensure-sigs-fixtures
···
80
81
@echo "corpus: sig-verify-corpus.bin ($(wc -c < fixtures/sig-verify-corpus.bin | tr -d ' ') bytes)"
81
82
@echo ""
82
83
@echo "--------------------------------------------"
83
-
cd zig && zig build run-bench-sigs -Doptimize=ReleaseFast
84
+
cd zig && {{zig}} build run-bench-sigs -Doptimize=ReleaseFast
84
85
@echo "--------------------------------------------"
85
86
cd rust-sigs && cargo run --release 2>&1
86
87
@echo "--------------------------------------------"
···
88
89
@echo "============================================"
89
90
90
91
bench-sigs-zig: _ensure-sigs-fixtures
91
-
cd zig && zig build run-bench-sigs -Doptimize=ReleaseFast
92
+
cd zig && {{zig}} build run-bench-sigs -Doptimize=ReleaseFast
93
+
94
+
# --- relay hot path ---
95
+
96
+
# capture raw frames with pre-resolved signing keys for full relay hot-path benchmark
97
+
capture-relay:
98
+
cd zig && {{zig}} build run-capture-relay
99
+
100
+
# run decode + CAR CID verification + commit signature verification benchmark
101
+
bench-relay: _ensure-relay-fixtures
102
+
cd zig && {{zig}} build run-bench-relay -Doptimize=ReleaseFast
92
103
93
104
bench-sigs-rust: _ensure-sigs-fixtures
94
105
cd rust-sigs && cargo run --release
···
100
111
101
112
# build all (no run)
102
113
build:
103
-
cd zig && zig build
114
+
cd zig && {{zig}} build
104
115
cd rust && cargo build --release
105
116
cd rust-raw && cargo build --release
106
117
cd rust-rsky && cargo build --release
···
126
137
_ensure-sigs-fixtures:
127
138
@test -f fixtures/sig-verify-corpus.bin \
128
139
|| (echo "sig corpus not found — run 'just capture-sigs' first" && exit 1)
140
+
141
+
_ensure-relay-fixtures:
142
+
@test -f fixtures/relay-corpus.bin \
143
+
|| (echo "relay corpus not found — run 'just capture-relay' first" && exit 1)