atproto user agency toolkit for individuals and groups
1# Policy Engine Design
2
3## 1. Current State Analysis
4
5### What Exists
6
7The policy system is spread across four locations, each with its own notion of "this DID should be replicated":
8
9**Source 1: Config REPLICATE_DIDS** (`src/config.ts`)
10- Comma-separated DID list from environment variable
11- Cannot be modified at runtime
12- Always replicated, regardless of policy engine state
13- No parameters (uses global defaults)
14
15**Source 2: admin_tracked_dids** (`src/replication/sync-storage.ts`)
16- SQLite table populated by `addDid()` and the offer-acceptance flow
17- Mutable at runtime via XRPC endpoints
18- Also used as the promotion target when mutual offers are detected: `offerDid()` publishes an offer, and when mutual agreement is discovered, `addDid()` inserts the DID here
19- No parameters attached to the row itself
20
21**Source 3: PolicyEngine** (`src/policy/engine.ts`)
22- In-memory `Policy[]` with JSON serialization
23- Loaded from `POLICY_FILE` or created empty
24- `p2p:`-prefixed policies are auto-generated by OfferManager when mutual offers are detected
25- Can be manually populated via presets (reciprocal, saas, groupGovernance)
26- Pure evaluation: `evaluate(did)` merges all matching policies into an `EffectivePolicy`
27
28**Source 4: offered_dids / incoming_offers** (`src/replication/sync-storage.ts`)
29- `offered_dids`: DIDs we have published an offer for but that are not yet actively replicated (awaiting mutual consent)
30- `incoming_offers`: offers from remote peers, awaiting user acceptance
31- These are staging tables that feed into Source 2 upon mutual agreement
32
33### What's Working
34
35The consent-gated flow is functional end-to-end:
361. User clicks "Add" on a DID -> publishes `org.p2pds.replication.offer` record to their PDS
372. Remote peer receives push notification -> incoming_offers row created
383. Remote user clicks "Accept" -> publishes reciprocal offer, triggers offer discovery
394. Mutual agreement detected -> DID promoted to admin_tracked_dids, P2P policy created, sync begins
405. Revocation: either side deletes offer -> peer notified -> data purged
41
42The PolicyEngine correctly merges multiple overlapping policies (max minCopies, min interval, etc.) and the ReplicationManager respects per-DID sync intervals.
43
44### What's Messy
45
46**1. Three-source merging creates implicit priority rules.**
47`getReplicateDids()` unions config DIDs, admin DIDs, and policy explicit DIDs. Config DIDs and admin DIDs always replicate regardless of policy evaluation. This means the policy engine is advisory for some DIDs but authoritative for others, and the logic for which is which is buried in `getReplicateDids()`.
48
49**2. admin_tracked_dids conflates manual tracking with agreement-driven tracking.**
50When a user manually adds a DID via the UI, it goes into admin_tracked_dids. When a mutual offer promotes a DID, it also goes into admin_tracked_dids. There's no way to distinguish "I manually archived this DID" from "this DID was auto-promoted from a P2P agreement." This matters for revocation: revoking a P2P agreement should remove the DID, but removing a manual archive should not touch any offer records.
51
52**3. Policy is stateless; agreement state lives elsewhere.**
53The P2P policy (`p2p:did:plc:xxx`) is a plain `Policy` object with no lifecycle metadata: no creation timestamp, no agreement state, no link back to the offers that generated it. OfferManager maintains this linkage implicitly by re-scanning offers during `discoverAndSync()`, but if the process restarts, P2P policies must be rediscovered from scratch.
54
55**4. No violation/degradation model.**
56The current system is binary: either a DID is replicated or it isn't. There's no notion of "this agreement is in trouble" (peer stopped syncing, challenges failing, peer unreachable). The challenge-response system produces reliability scores, but nothing acts on them.
57
58**5. UI shows raw state, not semantic status.**
59The app shows sync states (pending/syncing/synced/error/tombstoned) and DID sources (config/user/policy), but not agreement-level status. A user can't see "your reciprocal replication agreement with @alice.bsky.social is healthy" -- they see a DID, a sync timestamp, and a source label.
60
61---
62
63## 2. Proposed Architecture
64
65### Core Principle: Policy as the Single Source of Truth
66
67Every replicated DID must be justified by a policy. The policy engine becomes the authoritative answer to:
68- "Should this DID be replicated?" (yes/no)
69- "How should it be replicated?" (parameters)
70- "Why is it being replicated?" (policy type + provenance)
71- "What is the state of that replication relationship?" (lifecycle)
72
73Config REPLICATE_DIDS and admin_tracked_dids stop being independent sources. Instead, they become policy *generators*: inputs that produce policy objects which flow through the same evaluation pipeline as everything else.
74
75### Architecture Layers
76
77```
78+---------------------------------------------------------+
79| UI / XRPC Layer |
80| Shows policies, agreements, lifecycle states, actions |
81+---------------------------------------------------------+
82 |
83+---------------------------------------------------------+
84| Policy Engine |
85| - Stores all policies (SQLite-backed) |
86| - Evaluates effective policy per DID |
87| - Manages lifecycle state transitions |
88| - Emits events on state changes |
89+---------------------------------------------------------+
90 | | |
91+-----------------+ +------------------+ +--------------+
92| Policy Sources | | Lifecycle Driver | | Health Monitor|
93| - Config DIDs | | - Offer flow | | - Sync status |
94| - Manual pins | | - Acceptance | | - Challenges |
95| - P2P offers | | - Revocation | | - Reachability|
96| - File presets | | - Expiration | | - Grace periods|
97+-----------------+ +------------------+ +--------------+
98```
99
100### Policy Storage
101
102Policies move from in-memory-only to SQLite-backed with the following schema:
103
104```sql
105CREATE TABLE policies (
106 id TEXT PRIMARY KEY,
107 type TEXT NOT NULL, -- 'reciprocal', 'archive', 'config', 'saas', 'group'
108 state TEXT NOT NULL DEFAULT 'active', -- lifecycle state
109 consent TEXT NOT NULL DEFAULT 'unconsented', -- 'reciprocal', 'consented', 'unconsented', 'revoked'
110 name TEXT NOT NULL,
111 target_json TEXT NOT NULL, -- JSON: PolicyTarget
112 replication_json TEXT NOT NULL, -- JSON: ReplicationGoals
113 sync_json TEXT NOT NULL, -- JSON: SyncConfig
114 retention_json TEXT NOT NULL, -- JSON: RetentionConfig
115 priority INTEGER NOT NULL DEFAULT 50,
116 enabled INTEGER NOT NULL DEFAULT 1,
117
118 -- Lifecycle metadata
119 created_at TEXT NOT NULL,
120 activated_at TEXT,
121 suspended_at TEXT,
122 terminated_at TEXT,
123 expires_at TEXT, -- optional TTL
124
125 -- Provenance (who/what created this policy)
126 created_by TEXT, -- DID of the user who created it, or 'system'
127 source TEXT, -- 'config', 'file', 'offer', 'manual', 'api'
128
129 -- Agreement linkage (for P2P policies)
130 counterparty_did TEXT, -- the other party in a bilateral agreement
131 local_offer_uri TEXT, -- AT URI of our offer record
132 remote_offer_uri TEXT -- AT URI of their offer record
133);
134```
135
136This replaces the current in-memory `Policy[]` and the separate `admin_tracked_dids`, `offered_dids` tables. (The `incoming_offers` table remains -- it represents *pending* proposals that have not yet become policies.)
137
138---
139
140## 3. Consent Model
141
142Consent and reciprocity are orthogonal dimensions. Every replication relationship can be characterized along both axes:
143
144| | **Consensual** | **Non-consensual** |
145|--|---------------|-------------------|
146| **Reciprocal** | Both parties explicitly agree to archive each other. The current offer/accept flow. | Both parties independently archive each other without coordination. An emergent state, not a policy type — detected when two independent archive policies point at each other. |
147| **Non-reciprocal** | Subject grants permission to be archived (open consent or directed consent), but the archiver does not share their data back. | Archiver replicates public data without the subject's explicit permission. The data is public anyway (it's on their PDS), but the subject hasn't opted in. |
148
149### Consent mechanisms
150
151**Explicit consent (directed):** The current offer/accept flow. Account A offers to replicate Account B. Account B accepts, granting consent and optionally reciprocating. Consent is recorded as `org.p2pds.replication.offer` records in both repos.
152
153**Open consent:** An account publishes a standing record (`org.p2pds.replication.consent`) declaring "anyone may archive my data." Any p2pds node that discovers this record can create an archive policy without needing a per-peer handshake. Open consent can be revoked at any time by deleting the record.
154
155**No consent (unilateral):** An archiver replicates public atproto data without any consent signal from the subject. This is always technically possible (the data is public), but the policy engine should distinguish it so the UI can surface the consent status clearly.
156
157### How consent flows through the system
158
1591. **Archiver initiates:** User clicks "Archive" on a DID → policy engine checks for consent signals.
1602. **Consent check:** Is there an open consent record? Is there an existing directed offer from the subject? If yes → archive with consent. If no → archive without consent (unilateral).
1613. **Reciprocity check:** Does the subject also archive the archiver? If yes → the relationship is reciprocal. The policy engine detects this and can surface it in the UI ("reciprocal") even if neither side explicitly requested reciprocity.
1624. **Consent revocation:** Subject deletes their consent/offer record → the archiver's policy transitions. For reciprocal policies, this terminates the agreement. For unilateral archives, the data remains (it's public) but the consent status updates to "no consent."
163
164### Consent status as policy metadata
165
166Rather than consent being a policy *type*, it's a property of every policy:
167
168```typescript
169type ConsentStatus =
170 | "reciprocal" // Both parties consented and archive each other
171 | "consented" // Subject explicitly consented (open or directed)
172 | "unconsented" // No consent signal from subject
173 | "revoked"; // Subject previously consented, then revoked
174```
175
176This means the policy types become simpler — they describe *what* is happening, while consent describes the *relationship*:
177
178## 4. Policy Type Taxonomy
179
180### 4.1 Archive (`archive`)
181
182The fundamental operation: replicate a DID's data on this node. Every replication relationship starts as an archive.
183
184**Consent variants:**
185- **Consented archive:** Subject has an open consent record or accepted a directed offer. The archiver can prove permission.
186- **Unconsented archive:** No consent signal. The data is public, so this is always possible, but the UI makes the distinction visible.
187
188**Parameters:**
189- `minCopies`: 1 (just this node)
190- `intervalSec`: 300 (default)
191- `priority`: 40
192- `consent`: ConsentStatus
193
194**Provenance:** Created by user action ("Archive" button), config, or API.
195
196**Invariant:** Active as long as the archiver wants it. If consent was present and gets revoked, the archive can continue (data is public) but consent status changes to `revoked`.
197
198### 4.2 Reciprocal (`reciprocal`)
199
200A reciprocal policy is an archive with a bilateral consent agreement: both parties archive each other and both have consented. This is the current offer/accept flow.
201
202**How it emerges:**
2031. Account A offers to archive Account B (publishes offer, creates a `proposed` reciprocal policy)
2042. Account B accepts (publishes reciprocal offer, creating their own reciprocal policy)
2053. Both policies transition to `active` when mutual offers are detected
206
207**Can also emerge organically:** If Account A creates an archive policy for B, and Account B independently creates an archive policy for A, the policy engine detects the mutual relationship and can prompt: "You're both archiving each other. Convert to a reciprocal agreement?"
208
209**Parameters:**
210- `minCopies`: 2 (at least two copies across the agreement)
211- `intervalSec`: 600 (default)
212- `priority`: 50 (higher than unilateral archive)
213- `counterpartyDid`: the other party's DID
214- `consent`: always `"reciprocal"`
215
216**Provenance:** Created by the offer/accept consent flow. Both sides hold a reciprocal policy pointing at each other.
217
218**Invariant:** Valid only when both parties have active offer records. If either side revokes consent, the policy terminates. The remaining party can choose to continue as a unilateral archive (downgrade to `archive` with `consent: "revoked"`).
219
220### 4.3 Config (`config`)
221
222DIDs specified via the `REPLICATE_DIDS` environment variable. Infrastructure-level archives that persist across restarts and can't be modified from the UI.
223
224**Parameters:**
225- `minCopies`: 1
226- `intervalSec`: 300
227- `priority`: 30 (lowest — infrastructure baseline)
228- `consent`: typically `"unconsented"` (infra doesn't negotiate)
229
230**Provenance:** Generated at startup from config. Policy IDs are `config:{did}`.
231
232**Invariant:** Exists as long as the DID is in the config. Recreated on every startup.
233
234### 4.4 Open Consent Record (`org.p2pds.replication.consent`)
235
236Not a policy type, but a mechanism that feeds into policy creation. An account publishes this record to declare open consent:
237
238```json
239{
240 "$type": "org.p2pds.replication.consent",
241 "scope": "any",
242 "createdAt": "2026-02-16T..."
243}
244```
245
246When a p2pds node discovers this record (during archival or via firehose), it can upgrade any unconsented archive of that DID to `consent: "consented"`. It also enables the UI to show "this account welcomes archiving" when a user is deciding whether to archive someone.
247
248Future extensions: `scope` could be `"any"`, `"followers"`, or an explicit DID list for directed open consent.
249
250### 4.5 Future: SaaS (`saas`)
251
252An operator guarantees replication for a set of accounts under an SLA. Not implemented now, but the type system accommodates it.
253
254**Parameters:**
255- `minCopies`: 3+ (SLA-driven)
256- `intervalSec`: 60 (aggressive)
257- `priority`: 80+ (high)
258- SLA metadata: uptime target, response time, penalty terms
259
260### 4.6 Future: Group / Mutual Aid (`group`)
261
262A community collectively decides which accounts to archive. This is where true mutual aid lives — community-level coordination rather than bilateral agreements. Requires quorum/voting mechanisms.
263
264**Parameters:**
265- `minCopies`: varies by group decision
266- `memberDids`: the governance group
267- `approvalThreshold`: N-of-M required
268- `priority`: 60
269
270---
271
272## 4. Lifecycle State Machine
273
274Every policy goes through a lifecycle. The states and transitions depend on the policy type, but the core state machine is shared:
275
276```
277 +-----------+
278 create --> | proposed |----> [reject] ----> terminated
279 +-----------+
280 |
281 [activate]
282 |
283 v
284 +-----------+
285 | active |<---- [resume]
286 +-----------+
287 | |
288 [suspend]| |[terminate]
289 v |
290 +-----------+|
291 | suspended ||
292 +-----------+|
293 | |
294 [resume] | |
295 (back up) |
296 v
297 +-----------+
298 | terminated|
299 +-----------+
300 |
301 [purge data]
302 |
303 +-----------+
304 | purged |
305 +-----------+
306```
307
308### State Definitions
309
310| State | Meaning | Data Kept? | Sync Active? |
311|-------|---------|------------|--------------|
312| `proposed` | Policy created but not yet activated (e.g., offer sent, awaiting acceptance) | No data yet | No |
313| `active` | Policy is live; DID is being replicated according to parameters | Yes | Yes |
314| `suspended` | Temporarily paused (health check failure, peer unreachable, user pause) | Yes (retained) | No |
315| `terminated` | Policy ended (revocation, offer deleted, manual removal) | Retained briefly | No |
316| `purged` | Data has been deleted; policy record kept for audit trail | No | No |
317
318### State Transitions by Policy Type
319
320**Reciprocal:**
321```
322User clicks "Replicate" on a DID
323 -> proposed (offer published, awaiting consent from counterparty)
324
325Counterparty consents (accepts offer, publishes reciprocal offer)
326 -> active, consent: reciprocal (both sides syncing)
327
328Challenge failures exceed threshold
329 -> suspended (grace period, retry)
330
331Grace period expires with no recovery
332 -> terminated (notify counterparty)
333
334Either side revokes consent
335 -> terminated (notify counterparty, begin data retention countdown)
336
337Data retention period expires
338 -> purged
339```
340
341**Archive:**
342```
343User clicks "Archive" on a DID
344 -> active (immediate; consent status set based on subject's consent signals)
345 -> consent: consented (if open consent record exists)
346 -> consent: unconsented (if no consent signal)
347
348Subject publishes open consent record
349 -> consent status upgrades to "consented"
350
351Subject revokes open consent
352 -> consent status changes to "revoked" (archive continues, data is public)
353
354User pauses
355 -> suspended
356
357User resumes
358 -> active
359
360User removes
361 -> terminated -> purged (immediate, no retention needed)
362```
363
364**Config:**
365```
366Startup, DID in REPLICATE_DIDS
367 -> active (immediate)
368
369DID removed from config + restart
370 -> (policy simply not recreated)
371```
372
373### Grace Periods and Escalation
374
375When a reciprocal policy enters `suspended` state, the system follows this escalation path:
376
3771. **Soft suspension** (0-1 hour): Sync paused, challenge retries continue. No user notification. Peer may just be temporarily offline.
3782. **Warning** (1-6 hours): UI shows warning badge. Push notification to the user. Continue monitoring.
3793. **Hard suspension** (6-24 hours): UI shows alert. Data retained but no sync attempts. Peer gets a "your agreement is at risk" signal.
3804. **Termination** (24+ hours): Agreement terminated. Begin data retention countdown (configurable, default 7 days).
381
382This is configuration within the policy itself, not hardcoded. The reciprocal preset provides sensible defaults, but users can adjust thresholds.
383
384```typescript
385interface ViolationPolicy {
386 /** Seconds of peer unreachability before soft suspension. */
387 softSuspensionAfterSec: number; // default: 3600 (1 hour)
388 /** Seconds before warning escalation. */
389 warningAfterSec: number; // default: 21600 (6 hours)
390 /** Seconds before hard suspension. */
391 hardSuspensionAfterSec: number; // default: 86400 (24 hours)
392 /** Seconds before automatic termination. */
393 terminationAfterSec: number; // default: 86400 (24 hours)
394 /** Seconds to retain data after termination before purge. */
395 dataRetentionAfterTerminationSec: number; // default: 604800 (7 days)
396 /** Challenge failure count before suspension. */
397 challengeFailureThreshold: number; // default: 3
398}
399```
400
401---
402
403## 5. UI Language Mapping
404
405The policy type and lifecycle state determine what the user sees. This section maps each state to display language for the primary UI contexts.
406
407### 5.1 Action Buttons
408
409| Context | Current Language | Proposed Language | Notes |
410|---------|-----------------|-------------------|-------|
411| Adding a DID — request reciprocal | "Add" | "Replicate" | Initiates a reciprocal proposal (publishes offer, awaits consent) |
412| Adding a DID — unilateral archive | (not distinct) | "Archive" | Creates an archive policy; no consent required |
413| Responding to an incoming offer | "Accept" | "Accept & Replicate" | Makes it clear: you consent and replicate them back |
414| Declining an incoming offer | "Reject" | "Decline" | Less hostile than "reject" |
415| Stopping a reciprocal agreement | "Remove" | "Revoke" | Revokes your consent; agreement ends for both sides |
416| Stopping a unilateral archive | "Remove" | "Remove" | Simple removal, no consent implications |
417| Temporarily pausing | (not available) | "Pause" | Suspends without terminating |
418
419### 5.2 Status Labels
420
421| Policy State | Consent | User-Facing Label | Icon/Badge | Description Text |
422|-------------|---------|-------------------|------------|------------------|
423| proposed | — | "Waiting for consent" | Clock | "You've offered to replicate this account. Waiting for them to accept." |
424| active | reciprocal | "Reciprocal" | Green shield | "You and {handle} are replicating each other's data." |
425| active | consented | "Archiving (consented)" | Green dot | "Archiving with {handle}'s consent." |
426| active | unconsented | "Archiving" | Blue dot | "Archiving public data." |
427| active | revoked | "Archiving (consent revoked)" | Yellow dot | "{handle} revoked consent. Data is still public." |
428| suspended (soft) | any | "Paused" | Yellow dot | "Sync paused. Retrying automatically." |
429| suspended (warning) | reciprocal | "Connection issues" | Orange warning | "{handle}'s node hasn't responded in {duration}." |
430| suspended (hard) | reciprocal | "Agreement at risk" | Red warning | "No response for {duration}. Agreement will end in {time remaining}." |
431| terminated | any | "Ended" | Gray X | "This replication relationship has ended." |
432| purged | any | (not shown) | — | Removed from active UI; available in history |
433
434### 5.3 DID Source Labels
435
436Replace the current "config"/"user"/"policy" source labels with policy-aware labels:
437
438| Policy Type | Consent | Source Label |
439|-------------|---------|-------------|
440| reciprocal | reciprocal | "Reciprocal with {handle}" |
441| archive | consented | "Archived (consented)" |
442| archive | unconsented | "Archived" |
443| archive | revoked | "Archived (consent revoked)" |
444| config | any | "Infrastructure" |
445| saas | any | "SLA: {policy name}" |
446| group | any | "Group: {group name}" |
447
448### 5.4 Notification Language
449
450| Event | Notification Text |
451|-------|-------------------|
452| Incoming offer | "{handle} wants to replicate your data. Accept to replicate each other." |
453| Offer accepted | "{handle} accepted. You're now replicating each other's data." |
454| Consent revoked by subject | "{handle} revoked archiving consent. Your archive continues (public data)." |
455| Reciprocal agreement revoked by peer | "{handle} revoked your replication agreement. Your data was removed from their node." |
456| Soft suspension | (no notification — auto-retry is silent) |
457| Warning | "{handle}'s node hasn't synced in {duration}." |
458| Termination by timeout | "Agreement with {handle} ended due to inactivity." |
459| Accidental reciprocity detected | "You and {handle} are both archiving each other. Formalize as a reciprocal agreement?" |
460
461### 5.5 App Sections
462
463Restructure the app around policies and consent status:
464
465```
466Reciprocal Agreements
467 @alice.bsky.social — Replicating (last sync: 2m ago) [Revoke]
468 @bob.example.com — Waiting for consent [Cancel]
469
470Archives
471 @carol.bsky.social — Archiving (consented) [Remove]
472 did:plc:xyz123 — Archiving [Remove]
473
474Infrastructure
475 did:plc:abc789 — Replicating (from config)
476
477Incoming Requests
478 @dave.bsky.social wants to replicate your data. [Accept & Replicate] [Decline]
479```
480
481---
482
483## 6. Migration Path
484
485### Phase 0: No-op foundation (prepare the ground)
486
487**Goal:** Add the `policies` table and policy persistence without changing any behavior.
488
4891. Add SQLite `policies` table schema to SyncStorage (or a new PolicyStorage class).
4902. Extend the `Policy` type with lifecycle fields (`state`, `createdAt`, `type`, `source`, `counterpartyDid`).
4913. PolicyEngine gains `persist()` and `loadFromDb()` methods alongside the existing in-memory operations.
4924. On startup, load policies from DB. Existing `POLICY_FILE` loading still works and seeds the DB.
493
494**Migration:** On first startup with the new schema, the `policies` table is empty. No behavior change. Existing in-memory policies continue to work.
495
496### Phase 1: Config and archive policies
497
498**Goal:** Config DIDs and admin_tracked_dids generate policy objects.
499
5001. At startup, for each DID in `REPLICATE_DIDS`, create a `config:{did}` policy if it doesn't exist.
5012. When `addDid()` is called, instead of inserting into `admin_tracked_dids`, create a `archive` policy.
5023. `getReplicateDids()` changes to: query all policies where `state = 'active'` and `enabled = 1`, collect target DIDs. No more three-source merging.
5034. `getDidSource()` changes to: look up the policy type for the DID.
5045. `removeDid()` changes to: transition the policy to `terminated`/`purged`.
505
506**Migration:** On first startup after this change, scan `admin_tracked_dids` and create `archive` policies for each entry. After successful migration, the `admin_tracked_dids` table becomes unused (keep it around briefly for rollback safety).
507
508### Phase 2: Reciprocal lifecycle
509
510**Goal:** The offer flow creates and manages reciprocal policies with lifecycle states.
511
5121. `offerDid()` creates a `reciprocal` policy in `proposed` state (replaces `offered_dids` insertion).
5132. `acceptOffer()` transitions the policy to `active` (replaces the addDid() promotion).
5143. `revokeReplication()` transitions to `terminated`.
5154. `runOfferDiscovery()` no longer promotes offered_dids to admin_tracked_dids. Instead, it transitions `proposed` policies to `active` when mutual agreement is detected.
5165. Broken-agreement detection transitions `active` policies to `terminated`.
517
518**Migration:** On first startup, scan `offered_dids` and create `proposed` reciprocal policies. Scan `admin_tracked_dids` entries that have corresponding P2P policies and convert them to `active` reciprocal policies.
519
520### Phase 3: Health monitoring and suspension
521
522**Goal:** Policies can be suspended based on health signals.
523
5241. Integrate challenge-response results: if a peer fails N consecutive challenges, suspend the policy.
5252. Integrate sync failure tracking: if sync fails repeatedly for a DID, suspend.
5263. Implement grace period escalation (soft -> warning -> hard -> termination).
5274. Implement data retention countdown after termination.
5285. Add health status to the policy evaluation output so the UI can display it.
529
530### Phase 4: UI restructuring
531
532**Goal:** The dashboard is organized by policy, not by raw DID list.
533
5341. API returns policies with lifecycle state, counterparty info, health status.
5352. Dashboard groups DIDs by policy type.
5363. Action buttons reflect policy semantics ("End agreement" vs "Remove archive").
5374. Notifications use policy-aware language.
538
539---
540
541## 7. Implementation Plan
542
543### Phase 0: Foundation (estimated: 1-2 sessions)
544
545Files to modify:
546- `src/policy/types.ts` -- extend Policy with lifecycle fields
547- `src/policy/engine.ts` -- add persistence, load from DB
548- `src/replication/sync-storage.ts` -- add policies table schema (or new file `src/policy/storage.ts`)
549- `src/start.ts` -- load policies from DB on startup, seed from file
550
551Tests:
552- PolicyEngine persistence round-trip
553- Schema migration on fresh DB
554
555### Phase 1: Config and Archive Policies (estimated: 2-3 sessions)
556
557Files to modify:
558- `src/start.ts` -- generate config policies at startup
559- `src/replication/replication-manager.ts` -- `addDid()` creates archive policy, `getReplicateDids()` reads from policy engine, `removeDid()` transitions policy
560- `src/xrpc/app.ts` -- update API responses to use policy-based source info
561- `src/policy/presets.ts` -- add `archive()` and `configArchive()` preset factories
562
563Migration script:
564- `src/policy/migrate.ts` -- scan admin_tracked_dids, create archive policies
565
566Tests:
567- addDid creates archive policy
568- getReplicateDids returns policy-driven list
569- Config DIDs generate config policies
570- Removal transitions policy state
571
572### Phase 2: Reciprocal Lifecycle (estimated: 3-4 sessions)
573
574Files to modify:
575- `src/replication/replication-manager.ts` -- `offerDid()`, `acceptOffer()`, `revokeReplication()` operate on policies
576- `src/replication/offer-manager.ts` -- `syncPolicies()` transitions policy states instead of add/remove
577- `src/policy/engine.ts` -- lifecycle transition methods (`propose()`, `activate()`, `suspend()`, `terminate()`)
578- `src/policy/types.ts` -- add ViolationPolicy, lifecycle state enum
579
580Migration:
581- Convert offered_dids to proposed policies
582- Convert P2P-origin admin_tracked_dids to active reciprocal policies
583
584Tests:
585- Full offer -> accept -> active -> revoke -> terminated lifecycle
586- Broken agreement detection transitions to terminated
587- Policy state persists across restarts
588
589### Phase 3: Health and Suspension (estimated: 2-3 sessions)
590
591Files to modify:
592- `src/policy/engine.ts` -- health evaluation, suspension logic
593- `src/replication/challenge-response/challenge-scheduler.ts` -- report results to policy engine
594- `src/replication/replication-manager.ts` -- sync failure reporting to policy engine
595- `src/policy/types.ts` -- ViolationPolicy defaults in presets
596
597Tests:
598- Challenge failures trigger suspension
599- Grace period escalation
600- Termination after timeout
601- Recovery from suspension when peer comes back
602
603### Phase 4: UI Restructuring (estimated: 2-3 sessions)
604
605Files to modify:
606- `src/xrpc/app.ts` -- new API shape (policies with lifecycle state)
607- `src/index.ts` -- dashboard HTML rendering
608- UI template (inline HTML in index.ts or separate template)
609
610Tests:
611- API returns correct policy groupings
612- Status labels match lifecycle state
613- Action buttons match policy type
614
615---
616
617## 8. Detailed Type Definitions (Proposed)
618
619```typescript
620/** Policy lifecycle states. */
621type PolicyState =
622 | "proposed" // Created, awaiting activation (e.g., offer sent)
623 | "active" // Live, DID is being replicated
624 | "suspended" // Temporarily paused (health issue, user pause)
625 | "terminated" // Ended, data retained briefly
626 | "purged"; // Data deleted, record kept for history
627
628/** What kind of policy this is. */
629type PolicyType =
630 | "reciprocal" // Bilateral agreement
631 | "archive" // Unilateral personal archive
632 | "config" // Infrastructure, from env var
633 | "saas" // SLA compliance (future)
634 | "group"; // Governance group (future)
635
636/** How the policy was created. */
637type PolicySource =
638 | "config" // REPLICATE_DIDS env var
639 | "file" // POLICY_FILE JSON
640 | "offer" // P2P offer flow
641 | "manual" // User action in the UI
642 | "api"; // Programmatic API call
643
644/** Consent status of a replication relationship. */
645type ConsentStatus =
646 | "reciprocal" // Both parties consented and archive each other
647 | "consented" // Subject explicitly consented (open or directed)
648 | "unconsented" // No consent signal from subject
649 | "revoked"; // Subject previously consented, then revoked
650
651/** Extended policy with lifecycle and consent. */
652interface PolicyV2 extends Policy {
653 type: PolicyType;
654 state: PolicyState;
655 source: PolicySource;
656 consent: ConsentStatus;
657
658 // Timestamps
659 createdAt: string;
660 activatedAt: string | null;
661 suspendedAt: string | null;
662 terminatedAt: string | null;
663 expiresAt: string | null;
664
665 // Provenance
666 createdBy: string; // DID or 'system'
667
668 // Agreement linkage (reciprocal only)
669 counterpartyDid: string | null;
670 localOfferUri: string | null;
671 remoteOfferUri: string | null;
672
673 // Violation handling
674 violation: ViolationPolicy;
675
676 // Health snapshot (computed, not stored)
677 health?: PolicyHealth;
678}
679
680/** Health status computed from sync/challenge data. */
681interface PolicyHealth {
682 /** Overall health: green/yellow/orange/red */
683 level: "healthy" | "degraded" | "warning" | "critical";
684 /** Last successful sync timestamp. */
685 lastSyncAt: string | null;
686 /** Consecutive sync failures. */
687 consecutiveSyncFailures: number;
688 /** Latest challenge result (if applicable). */
689 latestChallengeResult: "pass" | "fail" | "pending" | null;
690 /** Peer reliability score (0-1). */
691 peerReliability: number | null;
692 /** Seconds until next escalation (if degraded). */
693 escalationInSec: number | null;
694}
695```
696
697---
698
699## 9. Open Questions
700
701**Q1: What happens when an archive subject revokes consent?**
702If a subject had open consent and revokes it, should existing archives be terminated or just marked `consent: "revoked"`? The data is public regardless — revoking consent doesn't make the data private. But it does signal "I don't want you archiving me."
703
704Recommendation: Default behavior is to mark `consent: "revoked"` but keep the archive. The UI clearly shows the consent status change. The archiver can choose to respect the revocation and delete, or continue. For reciprocal policies, revocation terminates the agreement (since reciprocity requires bilateral consent).
705
706**Q2: How should the UI present unconsented archives?**
707Unconsented archiving is always technically possible (public data). But we should be thoughtful about the UX. Should the UI show a warning? Should it be a separate flow from consented archiving?
708
709Recommendation: The "Archive" action always works. If the subject has open consent, the UI shows a green consent badge. If not, the UI shows a neutral "no consent signal" indicator — not a warning, but visible. The distinction matters for trust and transparency, not for blocking the action.
710
711**Q3: How does open consent interact with reciprocal detection?**
712If Account A archives Account B (who has open consent), and Account B independently archives Account A, this is technically reciprocal but wasn't negotiated. Should the policy engine auto-detect this and offer to upgrade?
713
714Recommendation: Yes. The policy engine periodically checks for "accidental reciprocity" — two independent archive policies pointing at each other. It surfaces a prompt: "You and {handle} are both archiving each other. Would you like to formalize this as a reciprocal agreement?" This is opt-in, not automatic.
715
716**Q4: How long should terminated reciprocal data be retained?**
717The current implementation purges immediately on revocation. A grace period (e.g., 7 days) would allow the user to re-establish the agreement without re-syncing everything. But it also means storing data the peer has explicitly asked you to delete.
718
719Recommendation: Default 7-day retention for reciprocal termination (configurable to 0 for immediate purge). The data is content-addressed and already public (it's on their PDS), so retaining it briefly is not a privacy concern. The peer record itself says "terminated" — we're not claiming to still serve it.
720
721**Q5: Should the policy engine emit events?**
722State transitions (proposed → active, active → suspended, etc.) could emit events that other components subscribe to (e.g., UI notifications, sync scheduling changes, gossipsub announcements).
723
724Recommendation: Yes, but as a simple callback/listener pattern, not a full event bus. The PolicyEngine gains an `onStateChange(callback)` method. ReplicationManager and the UI layer register listeners.
725
726**Q6: What happens to policies when the user logs out?**
727Currently, logout disconnects OAuth and prevents offer management. Should policies survive logout? Reciprocal policies depend on offer records on the PDS, which require an active session to manage.
728
729Recommendation: Policies survive logout. Active policies continue to replicate (sync doesn't require authentication). But the user can't create, accept, or revoke offers without logging back in. The UI shows "Log in to manage agreements."
730
731**Q7: Policy versioning and schema evolution.**
732The current PolicySet has `version: 1`. As PolicyV2 adds fields, how do we handle upgrades?
733
734Recommendation: Version the `policies` table schema, not the policy objects. Use SQLite `ALTER TABLE ADD COLUMN` with defaults for additive changes. For breaking changes, write a migration function. The JSON fields (target_json, etc.) are forward-compatible by nature — unknown keys are ignored.