wip: benchmarks for testing different p2p sync strategies using a pds as a relay
plans
+93
+92
plans/pds-roundtrip-optimization.md
+92
plans/pds-roundtrip-optimization.md
···
1
+
# PDS Round-Trip Optimization Plan
2
+
3
+
## Problem
4
+
5
+
pds-yrs (strategy 4) degrades significantly with file count in remote/PDS mode due to excessive HTTP round-trips during merge.
6
+
7
+
**Benchmark data (v1, 2 collaborators):**
8
+
9
+
| Files | git strategies (ms) | pds-yrs (ms) | Ratio |
10
+
|-------|--------------------:|-------------:|------:|
11
+
| 10 | ~7,000 | ~3,800 | 0.5x |
12
+
| 50 | ~7,000 | ~9,000 | 1.3x |
13
+
| 200 | ~7,300 | ~29,000 | 4.0x |
14
+
15
+
At 50 files or fewer, pds-yrs is competitive. At 200 files, it's 4x slower.
16
+
17
+
## Root Cause Analysis
18
+
19
+
### Save (efficient: ~3 round-trips)
20
+
1. `upload_blob()` — single pack blob containing all files
21
+
2. `put_record()` — manifest record with pack refs
22
+
3. Optional `get_record()` for existing state
23
+
24
+
### Load (efficient: ~2 round-trips)
25
+
1. `get_record()` — fetch manifest
26
+
2. `get_blob()` — fetch pack blob (cached across files)
27
+
28
+
### Merge (bottleneck: O(N × sites) round-trips)
29
+
`merge_text_file()` in `merge.rs` calls `yrs_pds::file_entry_to_doc()` per file per site. Each call downloads the pack blob separately — **no pack cache** in the merge path, unlike load. For 200 files × 2 sites = 400 blob downloads, mostly redundant since files in the same site share the same pack blob.
30
+
31
+
## Options
32
+
33
+
### Option A: Zip Snapshot Mode (new mode, biggest win)
34
+
35
+
Bundle the entire site (content files + .yrs sidecar data) into a single zip blob. Save/load/merge each become 1-2 round-trips regardless of file count.
36
+
37
+
**Save:** zip directory → `upload_blob()` → `put_record()` = 2 round-trips
38
+
**Load:** `get_record()` → `get_blob()` → unzip = 2 round-trips
39
+
**Merge:** `get_record()` × N sites → `get_blob()` × N sites → unzip all → merge in memory → zip → save = 2N + 2 round-trips
40
+
41
+
**Tradeoffs:**
42
+
- Pro: O(1) round-trips per site instead of O(files)
43
+
- Pro: Simple to implement — zip/unzip is well-supported
44
+
- Pro: Natural chunking boundary for large sites (zip > 40MB → chunk)
45
+
- Con: No incremental updates — re-uploads entire site each save
46
+
- Con: Doesn't help real-time sync mode (which needs per-file granularity)
47
+
- Con: AT Protocol record size limits still apply (~1MB for manifest, but zip blob can be larger since blobs have separate limits)
48
+
49
+
**Implementation:** New `--mode zip` flag on save/load/merge commands. Site manifest gets `format: "zip"` field. Reuses existing `pack.rs` chunking for large zips.
50
+
51
+
### Option B: Add Pack Cache to Merge (quick fix, medium win)
52
+
53
+
Mirror load.rs's existing pack cache in the merge path. When `file_entry_to_doc()` needs a pack blob, check a `HashMap<CID, Vec<u8>>` first. Since all files in a site typically share 1-2 pack blobs, this reduces 200 blob downloads to 2.
54
+
55
+
**Expected improvement:** For 200 files × 2 sites: from ~400 round-trips to ~4 round-trips (2 pack blobs × 2 sites). Should bring pds-yrs 200-file merge from ~29s down to ~7-8s (in line with git strategies).
56
+
57
+
**Tradeoffs:**
58
+
- Pro: Minimal code change (~20 lines in merge.rs)
59
+
- Pro: No API or format changes
60
+
- Pro: Immediate win for all existing sites
61
+
- Con: Still O(sites) round-trips, not O(1)
62
+
- Con: Memory usage scales with total pack blob size
63
+
64
+
**Implementation:** Add `pack_cache: HashMap<String, Vec<u8>>` parameter to `merge_sites()` and thread it through to `file_entry_to_doc()`. Populate on first fetch, reuse for subsequent files in the same pack.
65
+
66
+
### Option C: Pre-fetch All Blobs Before Merge
67
+
68
+
Before entering the per-file merge loop, scan all FileEntries across all sites, collect unique pack blob CIDs, and download them all upfront (potentially in parallel with tokio).
69
+
70
+
**Tradeoffs:**
71
+
- Pro: Can parallelize downloads (multiple blobs concurrently)
72
+
- Pro: Clean separation between I/O and merge logic
73
+
- Con: More code than Option B for similar result
74
+
- Con: Downloads blobs that might not be needed (if merge short-circuits)
75
+
- Con: Requires async runtime for parallel downloads
76
+
77
+
**Implementation:** New `prefetch_packs()` function that returns `HashMap<CID, Vec<u8>>`. Call before `merge_sites()`. Optionally use `tokio::spawn` for concurrent downloads.
78
+
79
+
## Recommended Order
80
+
81
+
1. **Option B first** — quick fix, biggest bang for buck, unblocks 200-file scenarios
82
+
2. **Option A second** — for batch/non-streaming use cases, eliminates the problem entirely
83
+
3. **Option C optional** — only if parallel downloads needed beyond what B provides
84
+
85
+
## Files to Modify
86
+
87
+
- `references/pds-yrs/src/merge.rs` — Options B and C
88
+
- `references/pds-yrs/src/yrs_pds.rs` — add pack cache parameter to `file_entry_to_doc()`
89
+
- `references/pds-yrs/src/save.rs` — Option A (zip mode)
90
+
- `references/pds-yrs/src/load.rs` — Option A (zip mode)
91
+
- `references/pds-yrs/src/types.rs` — Option A (format field on manifest)
92
+
- `references/pds-yrs/Cargo.toml` — Option A (zip dependency)