wip: benchmarks for testing different p2p sync strategies using a pds as a relay
readme.md
+982
+166
README.md
+166
README.md
···
1
+
# Merge Strategy Benchmark Report
2
+
3
+
**Date:** 2026-03-13
4
+
**PDS:** bluesky-pds.t1cc.commoninternet.net (v0.4.208)
5
+
**Modes:** Local (no network) and Remote (real PDS round-trips)
6
+
7
+
## Strategies
8
+
9
+
1. **plain-git** — Standard `git merge` 3-way merge
10
+
2. **yrs-diff** — git merge with git-yrs-merge CRDT driver (diff-only mode)
11
+
3. **yrs-sidecar** — git merge with git-yrs-merge CRDT driver + .yrs/ sidecar files
12
+
4. **yrs-on-pds** — Pure Yrs CRDT merge (local: in-memory from .pds-sim/; PDS: pds-yrs save/merge/load)
13
+
14
+
## Default Matrix (10/50/200 files, 2 collaborators)
15
+
16
+
### Local
17
+
18
+
| Files | Conflict | 1-git (ms) | 1-conflicts | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) |
19
+
|-------|----------|-----------|-------------|-------------|----------------|------------|
20
+
| 10 | low | 34 | 0 | 62 | 108 | 5 |
21
+
| 10 | medium | 32 | 1 | 57 | 106 | 3 |
22
+
| 10 | high | 30 | 0 | 85 | 122 | 3 |
23
+
| 50 | low | 39 | 0 | 60 | 91 | 23 |
24
+
| 50 | medium | 40 | 1 | 61 | 114 | 14 |
25
+
| 50 | high | 42 | 1 | 109 | 147 | 17 |
26
+
| 200 | low | 53 | 0 | 81 | 125 | 82 |
27
+
| 200 | medium | 62 | 1 | 117 | 208 | 65 |
28
+
| 200 | high | 65 | 0 | 264 | 472 | 68 |
29
+
30
+
### Remote (PDS)
31
+
32
+
| Files | Conflict | 1-git (ms) | 1-conflicts | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) |
33
+
|-------|----------|-----------|-------------|-------------|----------------|------------|
34
+
| 10 | low | 7094 | 0 | 6729 | 6702 | 3783 |
35
+
| 10 | medium | 6823 | 0 | 6776 | 6738 | 3924 |
36
+
| 10 | high | 6877 | 1 | 6764 | 8074 | 4471 |
37
+
| 50 | low | 7033 | 0 | 7089 | 7424 | 9316 |
38
+
| 50 | medium | 6999 | 0 | 7423 | 7973 | 9224 |
39
+
| 50 | high | 6882 | 0 | 7049 | 6963 | 9378 |
40
+
| 200 | low | 7000 | 0 | 7406 | 7388 | 28381 |
41
+
| 200 | medium | 6826 | 0 | 9745 | 7370 | 28995 |
42
+
| 200 | high | 7503 | 1 | 7327 | 7672 | 27605 |
43
+
44
+
## Stress Test (1000 files, 2 collaborators)
45
+
46
+
### Local
47
+
48
+
| Files | Conflict | 1-git (ms) | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) |
49
+
|-------|----------|-----------|-------------|----------------|------------|
50
+
| 1000 | low | 141 | 433 | 470 | 691 |
51
+
| 1000 | medium | 157 | 340 | 443 | 520 |
52
+
| 1000 | high | 97 | 356 | 334 | 579 |
53
+
54
+
### Remote (PDS)
55
+
56
+
All 3 scenarios failed with `413 Payload Too Large` — 1000-file sites exceed AT Protocol record size limits. Strategy 4 (pds-yrs) stores all file metadata in a single record, which hits the ~1MB limit. Strategies 1-3 (git-remote-pds) pack the full git repo into a single record, which also exceeds limits at this scale.
57
+
58
+
## File Size Sweep (50 files, 2 collaborators, medium conflict)
59
+
60
+
### Local
61
+
62
+
| Avg Size | 1-git (ms) | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) |
63
+
|----------|-----------|-------------|----------------|------------|
64
+
| 1 KB | 51 | 104 | 241 | 56 |
65
+
| 10 KB | 191 | 270 | 219 | 128 |
66
+
| 50 KB | 117 | 568 | 649 | 455 |
67
+
68
+
### Remote (PDS)
69
+
70
+
| Avg Size | 1-git (ms) | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) |
71
+
|----------|-----------|-------------|----------------|------------|
72
+
| 1 KB | error (500) | - | - | - |
73
+
| 10 KB | 6800 | 7448 | 7156 | 11436 |
74
+
| 50 KB | error (413) | - | - | - |
75
+
76
+
The 1KB scenario hit a transient 500 error; the 50KB scenario exceeded payload limits on pds-yrs save.
77
+
78
+
## Multi-Collaborator (5 collaborators)
79
+
80
+
### Local
81
+
82
+
| Files | Conflict | 1-git (ms) | 1-conflicts | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) |
83
+
|-------|----------|-----------|-------------|-------------|----------------|------------|
84
+
| 50 | low | 173 | 0 | 410 | 527 | 106 |
85
+
| 200 | low | 311 | 0 | 734 | 746 | 253 |
86
+
| 50 | medium | 113 | 1 | 554 | 544 | 44 |
87
+
| 200 | medium | 237 | 0 | 618 | 778 | 214 |
88
+
| 50 | high | 39 | 1 | 775 | 824 | 46 |
89
+
| 200 | high | 89 | 1 | 1700 | 1201 | 166 |
90
+
91
+
### Remote (PDS)
92
+
93
+
| Files | Conflict | 1-git (ms) | 1-conflicts | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) |
94
+
|-------|----------|-----------|-------------|-------------|----------------|------------|
95
+
| 50 | low | 9841 | 0 | 9238 | 11476 | 8912 |
96
+
| 200 | low | 9851 | 1 | 11497 | 10358 | 31053 |
97
+
| 50 | medium | 9258 | 0 | 11243 | 10379 | 8788 |
98
+
| 200 | medium | 10572 | 1 | 10372 | 10110 | 29553 |
99
+
| 50 | high | 9594 | 2 | 10401 | 10122 | 8630 |
100
+
| 200 | high | 9759 | 4 | 11703 | 10867 | 29131 |
101
+
102
+
## Guaranteed Conflict (2 collaborators, same lines edited)
103
+
104
+
### Local
105
+
106
+
| Files | 1-git (ms) | 1-conflicts | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) |
107
+
|-------|-----------|-------------|-------------|----------------|------------|
108
+
| 10 | 327 | 10 | 1456 | 614 | 6 |
109
+
| 50 | 156 | 20 | 961 | 1082 | 30 |
110
+
| 200 | 154 | 20 | 1048 | 1082 | 115 |
111
+
112
+
### Remote (PDS)
113
+
114
+
| Files | 1-git (ms) | 1-conflicts | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) |
115
+
|-------|-----------|-------------|-------------|----------------|------------|
116
+
| 10 | 6634 | 10 | 6950 | 7166 | 5281 |
117
+
| 50 | 6900 | 20 | 7301 | 7296 | 10650 |
118
+
| 200 | 6741 | 20 | 8120 | 8299 | 28950 |
119
+
120
+
## Key Findings
121
+
122
+
### Local performance
123
+
124
+
- **Plain git is fastest** for pure merge speed (27-65ms for small/medium repos)
125
+
- **Yrs CRDT merge (strategy 4)** is competitive locally (2-82ms) since it's pure in-memory merge with no git overhead
126
+
- **git-yrs-merge strategies** (2, 3) add 2-10x overhead vs plain git due to CRDT driver invocations, but **eliminate all merge conflicts** — 0 conflicts across every scenario
127
+
- **Plain git fails with conflicts** in medium/high conflict scenarios; CRDT strategies always succeed
128
+
129
+
### Remote (PDS) performance
130
+
131
+
- **Network dominates** — all strategies take 6-10 seconds for small repos, regardless of local merge cost
132
+
- **Git strategies (1-3) are bottlenecked by push+clone** at ~6-7s baseline for 2 collaborators, ~9-11s for 5 collaborators
133
+
- **pds-yrs (strategy 4) scales worse with file count** — 3.5s at 10 files, 9s at 50, 28s at 200. Each file requires separate blob operations during save/merge
134
+
- **Git strategies scale better at high file counts** because git packs everything into a single push/fetch vs pds-yrs doing per-file blob ops
135
+
- At **50 files or fewer, pds-yrs is competitive or faster** than git strategies
136
+
- At **200 files, git strategies are 3-4x faster** than pds-yrs due to O(n) blob round-trips
137
+
138
+
### PDS limits
139
+
140
+
- AT Protocol record size limits (~1MB) prevent pds-yrs from handling 1000-file sites or 50KB average file sizes in a single record
141
+
- git-remote-pds also hits limits at 1000 files with large edits
142
+
- Pack blob format helps (bundles file data into single blob upload), but the record metadata itself grows with file count
143
+
144
+
### Conflict resolution
145
+
146
+
- Plain git: conflicts in 8 of 24 PDS scenarios (33%) — requires manual resolution
147
+
- All CRDT strategies (2, 3, 4): **zero conflicts across all 48 scenarios** — automatic resolution via Yrs CRDT merge
148
+
149
+
## Usage
150
+
151
+
```bash
152
+
# Local mode (default)
153
+
merge-bench run # default matrix
154
+
merge-bench run --quick # fast smoke test
155
+
merge-bench run --stress # 1000 files
156
+
merge-bench run --filesize # 1KB/10KB/50KB sweep
157
+
merge-bench run --multi-collab # 5 collaborators
158
+
merge-bench run --conflict # guaranteed conflicts
159
+
160
+
# Remote PDS mode (reads testuser.toml)
161
+
merge-bench run --pds # default matrix against real PDS
162
+
merge-bench run --quick --pds # quick PDS test
163
+
merge-bench run --conflict --pds # conflict test against PDS
164
+
```
165
+
166
+
Results are written to `bench-results-local/` and `bench-results-pds/` respectively.
+101
about.md
+101
about.md
···
1
+
# merge-bench
2
+
3
+
Benchmark harness comparing merge strategies for collaborative text editing via AT Protocol PDS.
4
+
5
+
## Why
6
+
7
+
Lichen uses Yrs CRDTs to enable conflict-free collaborative editing on sites stored in AT Protocol PDS repositories. There are several possible architectures for how collaborators merge their changes:
8
+
9
+
1. **Plain git** -- standard 3-way merge, conflicts require manual resolution
10
+
2. **git + yrs-merge (diff)** -- git merge with a CRDT merge driver that resolves conflicts automatically using Yrs document state
11
+
3. **git + yrs-merge (sidecar)** -- same as above, but with `.yrs/` sidecar files tracked in git for richer CRDT state
12
+
4. **Yrs-on-PDS** -- pure CRDT merge via `pds-yrs`, no git involved; each collaborator saves Yrs state to PDS and merges happen server-side
13
+
14
+
This tool generates synthetic repositories with configurable parameters (file count, file size, collaborator count, conflict rate) and measures each strategy's merge time, conflict count, and success rate.
15
+
16
+
Both **local** (no network, pure computation) and **remote** (real PDS round-trips via `git-remote-pds` and `pds-yrs`) modes are supported.
17
+
18
+
## Usage
19
+
20
+
```bash
21
+
cargo build
22
+
```
23
+
24
+
### Run benchmarks
25
+
26
+
```bash
27
+
# Local mode (default) -- measures pure merge cost
28
+
merge-bench run # default matrix (10/50/200 files x low/med/high conflict)
29
+
merge-bench run --quick # quick smoke test (2 scenarios)
30
+
merge-bench run --stress # 1000 files
31
+
merge-bench run --filesize # file size sweep (1KB/10KB/50KB)
32
+
merge-bench run --multi-collab # 5 collaborators
33
+
merge-bench run --conflict # guaranteed conflicts (same lines edited)
34
+
35
+
# Remote PDS mode -- includes real network round-trips
36
+
merge-bench run --pds # default matrix against real PDS
37
+
merge-bench run --conflict --pds # conflict test against real PDS
38
+
39
+
# Group related runs into a named set
40
+
merge-bench run --set v1 # bench-results/v1/default-local/
41
+
merge-bench run --pds --set v1 # bench-results/v1/default-pds/
42
+
merge-bench run --stress --set v1 # bench-results/v1/stress-local/
43
+
```
44
+
45
+
### Generate a repo manually
46
+
47
+
```bash
48
+
merge-bench gen --files 50 --avg-size 1000 --collaborators 3 --edits 10 --conflict-rate medium --output /tmp/test-repo
49
+
```
50
+
51
+
### Format existing results
52
+
53
+
```bash
54
+
merge-bench report --input bench-results/v1/default-local/results.json --format markdown
55
+
merge-bench report --input bench-results/v1/default-pds/results.json --format csv
56
+
```
57
+
58
+
## Output structure
59
+
60
+
Each run produces a folder with:
61
+
- `results.json` -- raw benchmark data
62
+
- `report.md` -- formatted markdown table
63
+
- `results.csv` -- CSV for spreadsheet import
64
+
65
+
Runs are grouped by set name under `bench-results/`:
66
+
67
+
```
68
+
bench-results/
69
+
v1/
70
+
default-local/
71
+
default-pds/
72
+
stress-local/
73
+
conflict-local/
74
+
conflict-pds/
75
+
...
76
+
```
77
+
78
+
## PDS mode setup
79
+
80
+
Remote mode requires credentials in `testuser.toml`:
81
+
82
+
```toml
83
+
pds = "https://your-pds.example.com"
84
+
handle = "user.your-pds.example.com"
85
+
password = "your-password"
86
+
did = "did:plc:..."
87
+
```
88
+
89
+
It also requires `git-remote-pds` and `pds-yrs` binaries to be built.
90
+
91
+
## Reports
92
+
93
+
See [reports/benchmark-report-2026-03-13.md](reports/benchmark-report-2026-03-13.md) for the first full benchmark comparing local vs PDS performance across all strategies.
94
+
95
+
## Key findings
96
+
97
+
- CRDT strategies (2, 3, 4) **eliminate all merge conflicts** -- 0 conflicts across every scenario tested
98
+
- Plain git fails with conflicts in 33% of PDS scenarios
99
+
- Locally, plain git is fastest (27-65ms) but Yrs CRDT merge is competitive (2-82ms)
100
+
- Over PDS, network dominates: ~7s baseline for git strategies, ~3.5s for pds-yrs at small scale
101
+
- pds-yrs scales worse with file count (O(n) blob round-trips) -- at 200 files it takes ~28s vs ~7s for git strategies
+18
bench-results/v1/conflict-pds/report.md
+18
bench-results/v1/conflict-pds/report.md
···
1
+
# Merge Strategy Benchmark Results
2
+
3
+
Generated: 2026-03-13T16:30:24.995814622+00:00
4
+
5
+
| Files | Collabs | Conflict | 1-git (ms) | 1-conflicts | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) | 2-ok | 3-ok | 4-ok |
6
+
|-------|---------|----------|-----------|-------------|-------------|----------------|------------|------|------|------|
7
+
| 10 | 2 | guaranteed | 6363 | 10 | 6902 | 7004 | 4335 | Y | Y | Y |
8
+
| 50 | 2 | guaranteed | 6723 | 20 | 7853 | 10435 | 9840 | Y | Y | Y |
9
+
| 200 | 2 | guaranteed | 6782 | 20 | 7308 | 7997 | 30904 | Y | Y | Y |
10
+
11
+
### Legend
12
+
13
+
- **1-git**: Plain git 3-way merge
14
+
- **2-diff**: git-yrs-merge in diff-only mode
15
+
- **3-sidecar**: git-yrs-merge with .yrs/ sidecars
16
+
- **4-yrs**: Yrs CRDT merge (simulated, no network)
17
+
- **ok**: merge completed successfully (Y/N)
18
+
- **conflicts**: number of files with conflict markers (strategy 1 only)
+13
bench-results/v1/conflict-pds/results.csv
+13
bench-results/v1/conflict-pds/results.csv
···
1
+
files,conflict_rate,collaborators,edits,strategy,time_ms,conflicts,success
2
+
10,guaranteed,2,10,1-plain-git,6363,10,false
3
+
10,guaranteed,2,10,2-yrs-diff,6902,0,true
4
+
10,guaranteed,2,10,3-yrs-sidecar,7004,0,true
5
+
10,guaranteed,2,10,4-yrs-on-pds,4335,0,true
6
+
50,guaranteed,2,20,1-plain-git,6723,20,false
7
+
50,guaranteed,2,20,2-yrs-diff,7853,0,true
8
+
50,guaranteed,2,20,3-yrs-sidecar,10435,0,true
9
+
50,guaranteed,2,20,4-yrs-on-pds,9840,0,true
10
+
200,guaranteed,2,20,1-plain-git,6782,20,false
11
+
200,guaranteed,2,20,2-yrs-diff,7308,0,true
12
+
200,guaranteed,2,20,3-yrs-sidecar,7997,0,true
13
+
200,guaranteed,2,20,4-yrs-on-pds,30904,0,true
+125
bench-results/v1/conflict-pds/results.json
+125
bench-results/v1/conflict-pds/results.json
···
1
+
[
2
+
{
3
+
"file_count": 10,
4
+
"avg_file_size": 1000,
5
+
"collaborators": 2,
6
+
"edits_per_branch": 10,
7
+
"conflict_rate": "guaranteed",
8
+
"results": [
9
+
{
10
+
"strategy": "1-plain-git",
11
+
"merge_time_ms": 6363,
12
+
"conflicts_reported": 10,
13
+
"files_processed": 30,
14
+
"success": false,
15
+
"error": ""
16
+
},
17
+
{
18
+
"strategy": "2-yrs-diff",
19
+
"merge_time_ms": 6902,
20
+
"conflicts_reported": 0,
21
+
"files_processed": 51,
22
+
"success": true,
23
+
"error": null
24
+
},
25
+
{
26
+
"strategy": "3-yrs-sidecar",
27
+
"merge_time_ms": 7004,
28
+
"conflicts_reported": 0,
29
+
"files_processed": 51,
30
+
"success": true,
31
+
"error": null
32
+
},
33
+
{
34
+
"strategy": "4-yrs-on-pds",
35
+
"merge_time_ms": 4335,
36
+
"conflicts_reported": 0,
37
+
"files_processed": 10,
38
+
"success": true,
39
+
"error": null
40
+
}
41
+
]
42
+
},
43
+
{
44
+
"file_count": 50,
45
+
"avg_file_size": 1000,
46
+
"collaborators": 2,
47
+
"edits_per_branch": 20,
48
+
"conflict_rate": "guaranteed",
49
+
"results": [
50
+
{
51
+
"strategy": "1-plain-git",
52
+
"merge_time_ms": 6723,
53
+
"conflicts_reported": 20,
54
+
"files_processed": 90,
55
+
"success": false,
56
+
"error": ""
57
+
},
58
+
{
59
+
"strategy": "2-yrs-diff",
60
+
"merge_time_ms": 7853,
61
+
"conflicts_reported": 0,
62
+
"files_processed": 191,
63
+
"success": true,
64
+
"error": null
65
+
},
66
+
{
67
+
"strategy": "3-yrs-sidecar",
68
+
"merge_time_ms": 10435,
69
+
"conflicts_reported": 0,
70
+
"files_processed": 191,
71
+
"success": true,
72
+
"error": null
73
+
},
74
+
{
75
+
"strategy": "4-yrs-on-pds",
76
+
"merge_time_ms": 9840,
77
+
"conflicts_reported": 0,
78
+
"files_processed": 50,
79
+
"success": true,
80
+
"error": null
81
+
}
82
+
]
83
+
},
84
+
{
85
+
"file_count": 200,
86
+
"avg_file_size": 1000,
87
+
"collaborators": 2,
88
+
"edits_per_branch": 20,
89
+
"conflict_rate": "guaranteed",
90
+
"results": [
91
+
{
92
+
"strategy": "1-plain-git",
93
+
"merge_time_ms": 6782,
94
+
"conflicts_reported": 20,
95
+
"files_processed": 240,
96
+
"success": false,
97
+
"error": ""
98
+
},
99
+
{
100
+
"strategy": "2-yrs-diff",
101
+
"merge_time_ms": 7308,
102
+
"conflicts_reported": 0,
103
+
"files_processed": 641,
104
+
"success": true,
105
+
"error": null
106
+
},
107
+
{
108
+
"strategy": "3-yrs-sidecar",
109
+
"merge_time_ms": 7997,
110
+
"conflicts_reported": 0,
111
+
"files_processed": 641,
112
+
"success": true,
113
+
"error": null
114
+
},
115
+
{
116
+
"strategy": "4-yrs-on-pds",
117
+
"merge_time_ms": 30904,
118
+
"conflicts_reported": 0,
119
+
"files_processed": 200,
120
+
"success": true,
121
+
"error": null
122
+
}
123
+
]
124
+
}
125
+
]
+24
bench-results/v1/default-pds/report.md
+24
bench-results/v1/default-pds/report.md
···
1
+
# Merge Strategy Benchmark Results
2
+
3
+
Generated: 2026-03-13T16:28:03.201090705+00:00
4
+
5
+
| Files | Collabs | Conflict | 1-git (ms) | 1-conflicts | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) | 2-ok | 3-ok | 4-ok |
6
+
|-------|---------|----------|-----------|-------------|-------------|----------------|------------|------|------|------|
7
+
| 10 | 2 | low | 6676 | 0 | 7649 | 8303 | 3793 | Y | Y | Y |
8
+
| 10 | 2 | medium | 6728 | 1 | 7005 | 6643 | 4066 | Y | Y | Y |
9
+
| 10 | 2 | high | 6666 | 1 | 7363 | 6482 | 3717 | Y | Y | Y |
10
+
| 50 | 2 | low | 7228 | 0 | 7006 | 6901 | 9042 | Y | Y | Y |
11
+
| 50 | 2 | medium | 6618 | 0 | 6845 | 6965 | 8737 | Y | Y | Y |
12
+
| 50 | 2 | high | 6984 | 1 | 7957 | 6972 | 9386 | Y | Y | Y |
13
+
| 200 | 2 | low | 6966 | 0 | 7064 | 7567 | 29411 | Y | Y | Y |
14
+
| 200 | 2 | medium | 11105 | 0 | 7335 | 7056 | 27647 | Y | Y | Y |
15
+
| 200 | 2 | high | 7294 | 1 | 7792 | 7370 | 29289 | Y | Y | Y |
16
+
17
+
### Legend
18
+
19
+
- **1-git**: Plain git 3-way merge
20
+
- **2-diff**: git-yrs-merge in diff-only mode
21
+
- **3-sidecar**: git-yrs-merge with .yrs/ sidecars
22
+
- **4-yrs**: Yrs CRDT merge (simulated, no network)
23
+
- **ok**: merge completed successfully (Y/N)
24
+
- **conflicts**: number of files with conflict markers (strategy 1 only)
+37
bench-results/v1/default-pds/results.csv
+37
bench-results/v1/default-pds/results.csv
···
1
+
files,conflict_rate,collaborators,edits,strategy,time_ms,conflicts,success
2
+
10,low,2,5,1-plain-git,6676,0,true
3
+
10,low,2,5,2-yrs-diff,7649,0,true
4
+
10,low,2,5,3-yrs-sidecar,8303,0,true
5
+
10,low,2,5,4-yrs-on-pds,3793,0,true
6
+
10,medium,2,5,1-plain-git,6728,1,false
7
+
10,medium,2,5,2-yrs-diff,7005,0,true
8
+
10,medium,2,5,3-yrs-sidecar,6643,0,true
9
+
10,medium,2,5,4-yrs-on-pds,4066,0,true
10
+
10,high,2,5,1-plain-git,6666,1,false
11
+
10,high,2,5,2-yrs-diff,7363,0,true
12
+
10,high,2,5,3-yrs-sidecar,6482,0,true
13
+
10,high,2,5,4-yrs-on-pds,3717,0,true
14
+
50,low,2,10,1-plain-git,7228,0,true
15
+
50,low,2,10,2-yrs-diff,7006,0,true
16
+
50,low,2,10,3-yrs-sidecar,6901,0,true
17
+
50,low,2,10,4-yrs-on-pds,9042,0,true
18
+
50,medium,2,10,1-plain-git,6618,0,true
19
+
50,medium,2,10,2-yrs-diff,6845,0,true
20
+
50,medium,2,10,3-yrs-sidecar,6965,0,true
21
+
50,medium,2,10,4-yrs-on-pds,8737,0,true
22
+
50,high,2,10,1-plain-git,6984,1,false
23
+
50,high,2,10,2-yrs-diff,7957,0,true
24
+
50,high,2,10,3-yrs-sidecar,6972,0,true
25
+
50,high,2,10,4-yrs-on-pds,9386,0,true
26
+
200,low,2,20,1-plain-git,6966,0,true
27
+
200,low,2,20,2-yrs-diff,7064,0,true
28
+
200,low,2,20,3-yrs-sidecar,7567,0,true
29
+
200,low,2,20,4-yrs-on-pds,29411,0,true
30
+
200,medium,2,20,1-plain-git,11105,0,true
31
+
200,medium,2,20,2-yrs-diff,7335,0,true
32
+
200,medium,2,20,3-yrs-sidecar,7056,0,true
33
+
200,medium,2,20,4-yrs-on-pds,27647,0,true
34
+
200,high,2,20,1-plain-git,7294,1,false
35
+
200,high,2,20,2-yrs-diff,7792,0,true
36
+
200,high,2,20,3-yrs-sidecar,7370,0,true
37
+
200,high,2,20,4-yrs-on-pds,29289,0,true
+371
bench-results/v1/default-pds/results.json
+371
bench-results/v1/default-pds/results.json
···
1
+
[
2
+
{
3
+
"file_count": 10,
4
+
"avg_file_size": 1000,
5
+
"collaborators": 2,
6
+
"edits_per_branch": 5,
7
+
"conflict_rate": "low",
8
+
"results": [
9
+
{
10
+
"strategy": "1-plain-git",
11
+
"merge_time_ms": 6676,
12
+
"conflicts_reported": 0,
13
+
"files_processed": 10,
14
+
"success": true,
15
+
"error": null
16
+
},
17
+
{
18
+
"strategy": "2-yrs-diff",
19
+
"merge_time_ms": 7649,
20
+
"conflicts_reported": 0,
21
+
"files_processed": 39,
22
+
"success": true,
23
+
"error": null
24
+
},
25
+
{
26
+
"strategy": "3-yrs-sidecar",
27
+
"merge_time_ms": 8303,
28
+
"conflicts_reported": 0,
29
+
"files_processed": 37,
30
+
"success": true,
31
+
"error": null
32
+
},
33
+
{
34
+
"strategy": "4-yrs-on-pds",
35
+
"merge_time_ms": 3793,
36
+
"conflicts_reported": 0,
37
+
"files_processed": 10,
38
+
"success": true,
39
+
"error": null
40
+
}
41
+
]
42
+
},
43
+
{
44
+
"file_count": 10,
45
+
"avg_file_size": 1000,
46
+
"collaborators": 2,
47
+
"edits_per_branch": 5,
48
+
"conflict_rate": "medium",
49
+
"results": [
50
+
{
51
+
"strategy": "1-plain-git",
52
+
"merge_time_ms": 6728,
53
+
"conflicts_reported": 1,
54
+
"files_processed": 12,
55
+
"success": false,
56
+
"error": ""
57
+
},
58
+
{
59
+
"strategy": "2-yrs-diff",
60
+
"merge_time_ms": 7005,
61
+
"conflicts_reported": 0,
62
+
"files_processed": 39,
63
+
"success": true,
64
+
"error": null
65
+
},
66
+
{
67
+
"strategy": "3-yrs-sidecar",
68
+
"merge_time_ms": 6643,
69
+
"conflicts_reported": 0,
70
+
"files_processed": 38,
71
+
"success": true,
72
+
"error": null
73
+
},
74
+
{
75
+
"strategy": "4-yrs-on-pds",
76
+
"merge_time_ms": 4066,
77
+
"conflicts_reported": 0,
78
+
"files_processed": 10,
79
+
"success": true,
80
+
"error": null
81
+
}
82
+
]
83
+
},
84
+
{
85
+
"file_count": 10,
86
+
"avg_file_size": 1000,
87
+
"collaborators": 2,
88
+
"edits_per_branch": 5,
89
+
"conflict_rate": "high",
90
+
"results": [
91
+
{
92
+
"strategy": "1-plain-git",
93
+
"merge_time_ms": 6666,
94
+
"conflicts_reported": 1,
95
+
"files_processed": 12,
96
+
"success": false,
97
+
"error": ""
98
+
},
99
+
{
100
+
"strategy": "2-yrs-diff",
101
+
"merge_time_ms": 7363,
102
+
"conflicts_reported": 0,
103
+
"files_processed": 38,
104
+
"success": true,
105
+
"error": null
106
+
},
107
+
{
108
+
"strategy": "3-yrs-sidecar",
109
+
"merge_time_ms": 6482,
110
+
"conflicts_reported": 0,
111
+
"files_processed": 39,
112
+
"success": true,
113
+
"error": null
114
+
},
115
+
{
116
+
"strategy": "4-yrs-on-pds",
117
+
"merge_time_ms": 3717,
118
+
"conflicts_reported": 0,
119
+
"files_processed": 10,
120
+
"success": true,
121
+
"error": null
122
+
}
123
+
]
124
+
},
125
+
{
126
+
"file_count": 50,
127
+
"avg_file_size": 1000,
128
+
"collaborators": 2,
129
+
"edits_per_branch": 10,
130
+
"conflict_rate": "low",
131
+
"results": [
132
+
{
133
+
"strategy": "1-plain-git",
134
+
"merge_time_ms": 7228,
135
+
"conflicts_reported": 0,
136
+
"files_processed": 50,
137
+
"success": true,
138
+
"error": null
139
+
},
140
+
{
141
+
"strategy": "2-yrs-diff",
142
+
"merge_time_ms": 7006,
143
+
"conflicts_reported": 0,
144
+
"files_processed": 167,
145
+
"success": true,
146
+
"error": null
147
+
},
148
+
{
149
+
"strategy": "3-yrs-sidecar",
150
+
"merge_time_ms": 6901,
151
+
"conflicts_reported": 0,
152
+
"files_processed": 166,
153
+
"success": true,
154
+
"error": null
155
+
},
156
+
{
157
+
"strategy": "4-yrs-on-pds",
158
+
"merge_time_ms": 9042,
159
+
"conflicts_reported": 0,
160
+
"files_processed": 50,
161
+
"success": true,
162
+
"error": null
163
+
}
164
+
]
165
+
},
166
+
{
167
+
"file_count": 50,
168
+
"avg_file_size": 1000,
169
+
"collaborators": 2,
170
+
"edits_per_branch": 10,
171
+
"conflict_rate": "medium",
172
+
"results": [
173
+
{
174
+
"strategy": "1-plain-git",
175
+
"merge_time_ms": 6618,
176
+
"conflicts_reported": 0,
177
+
"files_processed": 50,
178
+
"success": true,
179
+
"error": null
180
+
},
181
+
{
182
+
"strategy": "2-yrs-diff",
183
+
"merge_time_ms": 6845,
184
+
"conflicts_reported": 0,
185
+
"files_processed": 168,
186
+
"success": true,
187
+
"error": null
188
+
},
189
+
{
190
+
"strategy": "3-yrs-sidecar",
191
+
"merge_time_ms": 6965,
192
+
"conflicts_reported": 0,
193
+
"files_processed": 167,
194
+
"success": true,
195
+
"error": null
196
+
},
197
+
{
198
+
"strategy": "4-yrs-on-pds",
199
+
"merge_time_ms": 8737,
200
+
"conflicts_reported": 0,
201
+
"files_processed": 50,
202
+
"success": true,
203
+
"error": null
204
+
}
205
+
]
206
+
},
207
+
{
208
+
"file_count": 50,
209
+
"avg_file_size": 1000,
210
+
"collaborators": 2,
211
+
"edits_per_branch": 10,
212
+
"conflict_rate": "high",
213
+
"results": [
214
+
{
215
+
"strategy": "1-plain-git",
216
+
"merge_time_ms": 6984,
217
+
"conflicts_reported": 1,
218
+
"files_processed": 52,
219
+
"success": false,
220
+
"error": ""
221
+
},
222
+
{
223
+
"strategy": "2-yrs-diff",
224
+
"merge_time_ms": 7957,
225
+
"conflicts_reported": 0,
226
+
"files_processed": 165,
227
+
"success": true,
228
+
"error": null
229
+
},
230
+
{
231
+
"strategy": "3-yrs-sidecar",
232
+
"merge_time_ms": 6972,
233
+
"conflicts_reported": 0,
234
+
"files_processed": 167,
235
+
"success": true,
236
+
"error": null
237
+
},
238
+
{
239
+
"strategy": "4-yrs-on-pds",
240
+
"merge_time_ms": 9386,
241
+
"conflicts_reported": 0,
242
+
"files_processed": 50,
243
+
"success": true,
244
+
"error": null
245
+
}
246
+
]
247
+
},
248
+
{
249
+
"file_count": 200,
250
+
"avg_file_size": 1000,
251
+
"collaborators": 2,
252
+
"edits_per_branch": 20,
253
+
"conflict_rate": "low",
254
+
"results": [
255
+
{
256
+
"strategy": "1-plain-git",
257
+
"merge_time_ms": 6966,
258
+
"conflicts_reported": 0,
259
+
"files_processed": 200,
260
+
"success": true,
261
+
"error": null
262
+
},
263
+
{
264
+
"strategy": "2-yrs-diff",
265
+
"merge_time_ms": 7064,
266
+
"conflicts_reported": 0,
267
+
"files_processed": 637,
268
+
"success": true,
269
+
"error": null
270
+
},
271
+
{
272
+
"strategy": "3-yrs-sidecar",
273
+
"merge_time_ms": 7567,
274
+
"conflicts_reported": 0,
275
+
"files_processed": 637,
276
+
"success": true,
277
+
"error": null
278
+
},
279
+
{
280
+
"strategy": "4-yrs-on-pds",
281
+
"merge_time_ms": 29411,
282
+
"conflicts_reported": 0,
283
+
"files_processed": 200,
284
+
"success": true,
285
+
"error": null
286
+
}
287
+
]
288
+
},
289
+
{
290
+
"file_count": 200,
291
+
"avg_file_size": 1000,
292
+
"collaborators": 2,
293
+
"edits_per_branch": 20,
294
+
"conflict_rate": "medium",
295
+
"results": [
296
+
{
297
+
"strategy": "1-plain-git",
298
+
"merge_time_ms": 11105,
299
+
"conflicts_reported": 0,
300
+
"files_processed": 200,
301
+
"success": true,
302
+
"error": null
303
+
},
304
+
{
305
+
"strategy": "2-yrs-diff",
306
+
"merge_time_ms": 7335,
307
+
"conflicts_reported": 0,
308
+
"files_processed": 639,
309
+
"success": true,
310
+
"error": null
311
+
},
312
+
{
313
+
"strategy": "3-yrs-sidecar",
314
+
"merge_time_ms": 7056,
315
+
"conflicts_reported": 0,
316
+
"files_processed": 637,
317
+
"success": true,
318
+
"error": null
319
+
},
320
+
{
321
+
"strategy": "4-yrs-on-pds",
322
+
"merge_time_ms": 27647,
323
+
"conflicts_reported": 0,
324
+
"files_processed": 200,
325
+
"success": true,
326
+
"error": null
327
+
}
328
+
]
329
+
},
330
+
{
331
+
"file_count": 200,
332
+
"avg_file_size": 1000,
333
+
"collaborators": 2,
334
+
"edits_per_branch": 20,
335
+
"conflict_rate": "high",
336
+
"results": [
337
+
{
338
+
"strategy": "1-plain-git",
339
+
"merge_time_ms": 7294,
340
+
"conflicts_reported": 1,
341
+
"files_processed": 202,
342
+
"success": false,
343
+
"error": ""
344
+
},
345
+
{
346
+
"strategy": "2-yrs-diff",
347
+
"merge_time_ms": 7792,
348
+
"conflicts_reported": 0,
349
+
"files_processed": 637,
350
+
"success": true,
351
+
"error": null
352
+
},
353
+
{
354
+
"strategy": "3-yrs-sidecar",
355
+
"merge_time_ms": 7370,
356
+
"conflicts_reported": 0,
357
+
"files_processed": 636,
358
+
"success": true,
359
+
"error": null
360
+
},
361
+
{
362
+
"strategy": "4-yrs-on-pds",
363
+
"merge_time_ms": 29289,
364
+
"conflicts_reported": 0,
365
+
"files_processed": 200,
366
+
"success": true,
367
+
"error": null
368
+
}
369
+
]
370
+
}
371
+
]
+17
bench-results/v1/filesize-pds/report.md
+17
bench-results/v1/filesize-pds/report.md
···
1
+
# Merge Strategy Benchmark Results
2
+
3
+
Generated: 2026-03-13T16:30:07.934536033+00:00
4
+
5
+
| Files | Collabs | Conflict | 1-git (ms) | 1-conflicts | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) | 2-ok | 3-ok | 4-ok |
6
+
|-------|---------|----------|-----------|-------------|-------------|----------------|------------|------|------|------|
7
+
| 50 | 2 | medium | 6514 | 1 | 6870 | 6937 | 9475 | Y | Y | Y |
8
+
| 50 | 2 | medium | 7582 | 0 | 7528 | 7309 | 9252 | Y | Y | Y |
9
+
10
+
### Legend
11
+
12
+
- **1-git**: Plain git 3-way merge
13
+
- **2-diff**: git-yrs-merge in diff-only mode
14
+
- **3-sidecar**: git-yrs-merge with .yrs/ sidecars
15
+
- **4-yrs**: Yrs CRDT merge (simulated, no network)
16
+
- **ok**: merge completed successfully (Y/N)
17
+
- **conflicts**: number of files with conflict markers (strategy 1 only)
+9
bench-results/v1/filesize-pds/results.csv
+9
bench-results/v1/filesize-pds/results.csv
···
1
+
files,conflict_rate,collaborators,edits,strategy,time_ms,conflicts,success
2
+
50,medium,2,10,1-plain-git,6514,1,false
3
+
50,medium,2,10,2-yrs-diff,6870,0,true
4
+
50,medium,2,10,3-yrs-sidecar,6937,0,true
5
+
50,medium,2,10,4-yrs-on-pds,9475,0,true
6
+
50,medium,2,10,1-plain-git,7582,0,true
7
+
50,medium,2,10,2-yrs-diff,7528,0,true
8
+
50,medium,2,10,3-yrs-sidecar,7309,0,true
9
+
50,medium,2,10,4-yrs-on-pds,9252,0,true
+84
bench-results/v1/filesize-pds/results.json
+84
bench-results/v1/filesize-pds/results.json
···
1
+
[
2
+
{
3
+
"file_count": 50,
4
+
"avg_file_size": 1000,
5
+
"collaborators": 2,
6
+
"edits_per_branch": 10,
7
+
"conflict_rate": "medium",
8
+
"results": [
9
+
{
10
+
"strategy": "1-plain-git",
11
+
"merge_time_ms": 6514,
12
+
"conflicts_reported": 1,
13
+
"files_processed": 52,
14
+
"success": false,
15
+
"error": ""
16
+
},
17
+
{
18
+
"strategy": "2-yrs-diff",
19
+
"merge_time_ms": 6870,
20
+
"conflicts_reported": 0,
21
+
"files_processed": 169,
22
+
"success": true,
23
+
"error": null
24
+
},
25
+
{
26
+
"strategy": "3-yrs-sidecar",
27
+
"merge_time_ms": 6937,
28
+
"conflicts_reported": 0,
29
+
"files_processed": 168,
30
+
"success": true,
31
+
"error": null
32
+
},
33
+
{
34
+
"strategy": "4-yrs-on-pds",
35
+
"merge_time_ms": 9475,
36
+
"conflicts_reported": 0,
37
+
"files_processed": 50,
38
+
"success": true,
39
+
"error": null
40
+
}
41
+
]
42
+
},
43
+
{
44
+
"file_count": 50,
45
+
"avg_file_size": 10000,
46
+
"collaborators": 2,
47
+
"edits_per_branch": 10,
48
+
"conflict_rate": "medium",
49
+
"results": [
50
+
{
51
+
"strategy": "1-plain-git",
52
+
"merge_time_ms": 7582,
53
+
"conflicts_reported": 0,
54
+
"files_processed": 50,
55
+
"success": true,
56
+
"error": null
57
+
},
58
+
{
59
+
"strategy": "2-yrs-diff",
60
+
"merge_time_ms": 7528,
61
+
"conflicts_reported": 0,
62
+
"files_processed": 169,
63
+
"success": true,
64
+
"error": null
65
+
},
66
+
{
67
+
"strategy": "3-yrs-sidecar",
68
+
"merge_time_ms": 7309,
69
+
"conflicts_reported": 0,
70
+
"files_processed": 169,
71
+
"success": true,
72
+
"error": null
73
+
},
74
+
{
75
+
"strategy": "4-yrs-on-pds",
76
+
"merge_time_ms": 9252,
77
+
"conflicts_reported": 0,
78
+
"files_processed": 50,
79
+
"success": true,
80
+
"error": null
81
+
}
82
+
]
83
+
}
84
+
]
+15
bench-results/v1/stress-pds/report.md
+15
bench-results/v1/stress-pds/report.md
···
1
+
# Merge Strategy Benchmark Results
2
+
3
+
Generated: 2026-03-13T16:30:14.428726422+00:00
4
+
5
+
| Files | Collabs | Conflict | 1-git (ms) | 1-conflicts | 2-diff (ms) | 3-sidecar (ms) | 4-yrs (ms) | 2-ok | 3-ok | 4-ok |
6
+
|-------|---------|----------|-----------|-------------|-------------|----------------|------------|------|------|------|
7
+
8
+
### Legend
9
+
10
+
- **1-git**: Plain git 3-way merge
11
+
- **2-diff**: git-yrs-merge in diff-only mode
12
+
- **3-sidecar**: git-yrs-merge with .yrs/ sidecars
13
+
- **4-yrs**: Yrs CRDT merge (simulated, no network)
14
+
- **ok**: merge completed successfully (Y/N)
15
+
- **conflicts**: number of files with conflict markers (strategy 1 only)
+1
bench-results/v1/stress-pds/results.csv
+1
bench-results/v1/stress-pds/results.csv
···
1
+
files,conflict_rate,collaborators,edits,strategy,time_ms,conflicts,success
+1
bench-results/v1/stress-pds/results.json
+1
bench-results/v1/stress-pds/results.json
···
1
+
[]