an atproto pds written in F# (.NET 9) 🦒
pds
fsharp
giraffe
dotnet
atproto
1================================================================================
2*roadmap.txt* PDSharp: F#/Giraffe PDS Implementation Roadmap
3================================================================================
4PHASE 1: IMPLEMENTATION (Build)
5================================================================================
6Milestone A: Giraffe XRPC Shell
7--------------------------------------------------------------------------------
8- [x] Implement /xrpc/com.atproto.server.describeServer (GET)
9- [x] Implement NSID-based routing with structured error responses
10DoD: describeServer responds with stable JSON
11--------------------------------------------------------------------------------
12Milestone B: Identity + Crypto Primitives
13--------------------------------------------------------------------------------
14- [x] DID document fetch/parse for signing key and PDS endpoint
15- [x] SHA-256 hashing, ECDSA sign/verify (p256 + k256), low-S enforcement
16DoD: Sign and verify atproto commit hash with low-S
17--------------------------------------------------------------------------------
18Milestone C: DAG-CBOR + CID
19--------------------------------------------------------------------------------
20- [x] Canonical DAG-CBOR encode/decode with IPLD link tagging
21- [x] CID creation/parsing (multicodec dag-cbor, sha2-256)
22DoD: Record JSON → stable DAG-CBOR bytes → deterministic CID
23--------------------------------------------------------------------------------
24Milestone D: MST Implementation
25--------------------------------------------------------------------------------
26- [x] Merkle Search Tree per repository spec
27- [x] Key depth = leading zero bits in SHA-256(key) counted in 2-bit chunks
28- [x] Node encoding: (l, e[p,k,v,t]) with key prefix compression
29DoD: Insert/update/delete yields reproducible root CID
30--------------------------------------------------------------------------------
31Milestone E: Commit + BlockStore + putRecord
32--------------------------------------------------------------------------------
33- [x] BlockStore: cid → bytes, indexed by DID/rev/head
34- [x] Commit signing: UnsignedCommit → DAG-CBOR → sha256 → ECDSA sign
35- [x] Implement com.atproto.repo.putRecord/createRecord
36DoD: Write and read records by path/AT-URI
37--------------------------------------------------------------------------------
38Milestone F: CAR Export + Sync Endpoints
39--------------------------------------------------------------------------------
40- [x] CARv1 writer (roots = commit CID, blocks stream)
41- [x] Implement: sync.getRepo, sync.getBlocks, sync.getBlob
42DoD: External services can fetch repo snapshot + blocks
43--------------------------------------------------------------------------------
44Milestone G: subscribeRepos Firehose
45--------------------------------------------------------------------------------
46- [x] Monotonic sequence number + commit event generation
47- [x] WebSocket streaming for subscribeRepos
48DoD: Relay/client receives commit events after writes
49--------------------------------------------------------------------------------
50Milestone H: Account + Sessions
51--------------------------------------------------------------------------------
52- [x] Implement: server.createAccount, server.createSession, refreshSession
53- [x] Password/app-password hashing + JWT issuance
54DoD: Authenticate and write records with accessJwt
55--------------------------------------------------------------------------------
56Milestone I: Lexicon Validation + Conformance
57--------------------------------------------------------------------------------
58- [x] Lexicon validation for writes (app.bsky.* records)
59- [x] Conformance testing: diff CIDs/CARs/signatures vs reference PDS
60DoD: Same inputs → same outputs for repo/sync surfaces
61--------------------------------------------------------------------------------
62Milestone J: Persistence + Backups
63--------------------------------------------------------------------------------
64Deliverables:
65 - BackupOps module in Core (scheduler unit / cron / scripts, plus Litestream config)
66Backups (SQLite)
67 [ ] Set PDS_SQLITE_DISABLE_WAL_AUTO_CHECKPOINT=true (Litestream-friendly)
68 [ ] Run a scheduled backup/replication job that:
69 - finds recently updated DBs
70 - backs up /pds/actors/* and PDS-wide DBs
71 - runs on SIGTERM during deploys (avoid missing last writes)
72Backups (Blobs)
73 [ ] Configurable Options (app settings):
74 (A) Disk blobs: include /pds/blocks in backups
75 (B) S3-compatible blobstore: rely on object-store durability
76Guardrails
77 [ ] Uptime check: https://<pds>/xrpc/_health
78 [ ] Alert if “latest backup” is older than N minutes.
79 [ ] Alert on disk pressure for /pds.
80DoD:
81 - You can restore onto a fresh host and pass the P3 verification checklist.
82 - Backups run automatically and are observable (“last successful backup”).
83 - Backup set is explicitly documented (DBs + blobs decision).
84================================================================================
85PHASE 2: DEPLOYMENT (Self-Host)
86================================================================================
87Milestone J: Topology + Domain Planning
88--------------------------------------------------------------------------------
89- Choose PDS hostname (pds.example.com) vs handle domain (example.com)
90- Obtain domain, DNS access, VPS with static IP, reverse proxy
91DoD: Clear plan for PDS location, handle, and DID resolution
92--------------------------------------------------------------------------------
93Milestone K: DNS + TLS + Reverse Proxy
94--------------------------------------------------------------------------------
95- DNS A/AAAA records for PDS hostname
96- TLS certs (ACME) via Caddy
97DoD: https://<pds-hostname> responds with valid cert
98--------------------------------------------------------------------------------
99Milestone L: Deploy PDSharp
100--------------------------------------------------------------------------------
101- Deploy built PDS with persistence (SQLite/Postgres + blob storage)
102- Verify /xrpc/com.atproto.server.describeServer
103DoD: describeServer returns capabilities payload
104--------------------------------------------------------------------------------
105Milestone M: Account Creation
106--------------------------------------------------------------------------------
107- Create account using admin tooling
108- Verify authentication: createSession
109DoD: Obtain session and perform authenticated write
110--------------------------------------------------------------------------------
111Milestone N: Smoke Test Repo + Blobs
112--------------------------------------------------------------------------------
113- Write record via putRecord
114- Upload blob, verify retrieval via sync.getBlob
115DoD: Posts appear in clients, media loads reliably
116--------------------------------------------------------------------------------
117Milestone O: Account Migration
118--------------------------------------------------------------------------------
119- Export/import from bsky.social
120- Update DID service endpoint
121- Verify handle/DID resolution
122DoD: Handle unchanged, DID points to your PDS
123--------------------------------------------------------------------------------
124Milestone P: Reliability
125--------------------------------------------------------------------------------
126- Backups: repo storage + database + blobs
127- Restore drill on fresh instance
128- Monitoring: uptime checks for describeServer + getBlob
129DoD: Restore from backup passes smoke tests
130--------------------------------------------------------------------------------
131Milestone Q: Updates + Security
132--------------------------------------------------------------------------------
133- Update cadence with rollback plan
134- Rate limits and access controls at proxy
135- Log retention and disk growth alerts
136DoD: Update smoothly, maintain stable federation
137================================================================================
138QUICK CHECKLIST
139================================================================================
140[x] describeServer endpoint working
141[x] Crypto primitives (sha256, ECDSA p256/k256, low-S)
142[x] DAG-CBOR + CID generation correct
143[x] MST producing deterministic root CIDs
144[x] putRecord + blockstore operational
145[x] CAR export + sync endpoints
146[x] subscribeRepos firehose
147[x] Authentication (createAccount, createSession)
148[ ] Lexicon validation
149[ ] Domain + TLS configured
150[ ] PDS deployed and reachable
151[ ] Account created, session works
152[ ] Writes + blobs verified
153[ ] Backups + monitoring in place
154================================================================================
155REFERENCES
156================================================================================
157https://atproto.com/guides/self-hosting
158https://github.com/bluesky-social/pds
159https://atproto.com/specs/repository
160https://atproto.com/specs/sync
161https://atproto.com/specs/blob
162https://docs.bsky.app/docs/api/com-atproto-server-describe-server
163https://docs.bsky.app/docs/api/com-atproto-server-create-session
164https://docs.bsky.app/docs/api/com-atproto-repo-put-record
165https://docs.bsky.app/docs/api/com-atproto-sync-get-blob