# Policy Engine Design ## 1. Current State Analysis ### What Exists The policy system is spread across four locations, each with its own notion of "this DID should be replicated": **Source 1: Config REPLICATE_DIDS** (`src/config.ts`) - Comma-separated DID list from environment variable - Cannot be modified at runtime - Always replicated, regardless of policy engine state - No parameters (uses global defaults) **Source 2: admin_tracked_dids** (`src/replication/sync-storage.ts`) - SQLite table populated by `addDid()` and the offer-acceptance flow - Mutable at runtime via XRPC endpoints - Also used as the promotion target when mutual offers are detected: `offerDid()` publishes an offer, and when mutual agreement is discovered, `addDid()` inserts the DID here - No parameters attached to the row itself **Source 3: PolicyEngine** (`src/policy/engine.ts`) - In-memory `Policy[]` with JSON serialization - Loaded from `POLICY_FILE` or created empty - `p2p:`-prefixed policies are auto-generated by OfferManager when mutual offers are detected - Can be manually populated via presets (reciprocal, saas, groupGovernance) - Pure evaluation: `evaluate(did)` merges all matching policies into an `EffectivePolicy` **Source 4: offered_dids / incoming_offers** (`src/replication/sync-storage.ts`) - `offered_dids`: DIDs we have published an offer for but that are not yet actively replicated (awaiting mutual consent) - `incoming_offers`: offers from remote peers, awaiting user acceptance - These are staging tables that feed into Source 2 upon mutual agreement ### What's Working The consent-gated flow is functional end-to-end: 1. User clicks "Add" on a DID -> publishes `org.p2pds.replication.offer` record to their PDS 2. Remote peer receives push notification -> incoming_offers row created 3. Remote user clicks "Accept" -> publishes reciprocal offer, triggers offer discovery 4. Mutual agreement detected -> DID promoted to admin_tracked_dids, P2P policy created, sync begins 5. Revocation: either side deletes offer -> peer notified -> data purged The PolicyEngine correctly merges multiple overlapping policies (max minCopies, min interval, etc.) and the ReplicationManager respects per-DID sync intervals. ### What's Messy **1. Three-source merging creates implicit priority rules.** `getReplicateDids()` unions config DIDs, admin DIDs, and policy explicit DIDs. Config DIDs and admin DIDs always replicate regardless of policy evaluation. This means the policy engine is advisory for some DIDs but authoritative for others, and the logic for which is which is buried in `getReplicateDids()`. **2. admin_tracked_dids conflates manual tracking with agreement-driven tracking.** When a user manually adds a DID via the UI, it goes into admin_tracked_dids. When a mutual offer promotes a DID, it also goes into admin_tracked_dids. There's no way to distinguish "I manually archived this DID" from "this DID was auto-promoted from a P2P agreement." This matters for revocation: revoking a P2P agreement should remove the DID, but removing a manual archive should not touch any offer records. **3. Policy is stateless; agreement state lives elsewhere.** The P2P policy (`p2p:did:plc:xxx`) is a plain `Policy` object with no lifecycle metadata: no creation timestamp, no agreement state, no link back to the offers that generated it. OfferManager maintains this linkage implicitly by re-scanning offers during `discoverAndSync()`, but if the process restarts, P2P policies must be rediscovered from scratch. **4. No violation/degradation model.** The current system is binary: either a DID is replicated or it isn't. There's no notion of "this agreement is in trouble" (peer stopped syncing, challenges failing, peer unreachable). The challenge-response system produces reliability scores, but nothing acts on them. **5. UI shows raw state, not semantic status.** The app shows sync states (pending/syncing/synced/error/tombstoned) and DID sources (config/user/policy), but not agreement-level status. A user can't see "your reciprocal replication agreement with @alice.bsky.social is healthy" -- they see a DID, a sync timestamp, and a source label. --- ## 2. Proposed Architecture ### Core Principle: Policy as the Single Source of Truth Every replicated DID must be justified by a policy. The policy engine becomes the authoritative answer to: - "Should this DID be replicated?" (yes/no) - "How should it be replicated?" (parameters) - "Why is it being replicated?" (policy type + provenance) - "What is the state of that replication relationship?" (lifecycle) Config REPLICATE_DIDS and admin_tracked_dids stop being independent sources. Instead, they become policy *generators*: inputs that produce policy objects which flow through the same evaluation pipeline as everything else. ### Architecture Layers ``` +---------------------------------------------------------+ | UI / XRPC Layer | | Shows policies, agreements, lifecycle states, actions | +---------------------------------------------------------+ | +---------------------------------------------------------+ | Policy Engine | | - Stores all policies (SQLite-backed) | | - Evaluates effective policy per DID | | - Manages lifecycle state transitions | | - Emits events on state changes | +---------------------------------------------------------+ | | | +-----------------+ +------------------+ +--------------+ | Policy Sources | | Lifecycle Driver | | Health Monitor| | - Config DIDs | | - Offer flow | | - Sync status | | - Manual pins | | - Acceptance | | - Challenges | | - P2P offers | | - Revocation | | - Reachability| | - File presets | | - Expiration | | - Grace periods| +-----------------+ +------------------+ +--------------+ ``` ### Policy Storage Policies move from in-memory-only to SQLite-backed with the following schema: ```sql CREATE TABLE policies ( id TEXT PRIMARY KEY, type TEXT NOT NULL, -- 'reciprocal', 'archive', 'config', 'saas', 'group' state TEXT NOT NULL DEFAULT 'active', -- lifecycle state consent TEXT NOT NULL DEFAULT 'unconsented', -- 'reciprocal', 'consented', 'unconsented', 'revoked' name TEXT NOT NULL, target_json TEXT NOT NULL, -- JSON: PolicyTarget replication_json TEXT NOT NULL, -- JSON: ReplicationGoals sync_json TEXT NOT NULL, -- JSON: SyncConfig retention_json TEXT NOT NULL, -- JSON: RetentionConfig priority INTEGER NOT NULL DEFAULT 50, enabled INTEGER NOT NULL DEFAULT 1, -- Lifecycle metadata created_at TEXT NOT NULL, activated_at TEXT, suspended_at TEXT, terminated_at TEXT, expires_at TEXT, -- optional TTL -- Provenance (who/what created this policy) created_by TEXT, -- DID of the user who created it, or 'system' source TEXT, -- 'config', 'file', 'offer', 'manual', 'api' -- Agreement linkage (for P2P policies) counterparty_did TEXT, -- the other party in a bilateral agreement local_offer_uri TEXT, -- AT URI of our offer record remote_offer_uri TEXT -- AT URI of their offer record ); ``` This replaces the current in-memory `Policy[]` and the separate `admin_tracked_dids`, `offered_dids` tables. (The `incoming_offers` table remains -- it represents *pending* proposals that have not yet become policies.) --- ## 3. Consent Model Consent and reciprocity are orthogonal dimensions. Every replication relationship can be characterized along both axes: | | **Consensual** | **Non-consensual** | |--|---------------|-------------------| | **Reciprocal** | Both parties explicitly agree to archive each other. The current offer/accept flow. | Both parties independently archive each other without coordination. An emergent state, not a policy type — detected when two independent archive policies point at each other. | | **Non-reciprocal** | Subject grants permission to be archived (open consent or directed consent), but the archiver does not share their data back. | Archiver replicates public data without the subject's explicit permission. The data is public anyway (it's on their PDS), but the subject hasn't opted in. | ### Consent mechanisms **Explicit consent (directed):** The current offer/accept flow. Account A offers to replicate Account B. Account B accepts, granting consent and optionally reciprocating. Consent is recorded as `org.p2pds.replication.offer` records in both repos. **Open consent:** An account publishes a standing record (`org.p2pds.replication.consent`) declaring "anyone may archive my data." Any p2pds node that discovers this record can create an archive policy without needing a per-peer handshake. Open consent can be revoked at any time by deleting the record. **No consent (unilateral):** An archiver replicates public atproto data without any consent signal from the subject. This is always technically possible (the data is public), but the policy engine should distinguish it so the UI can surface the consent status clearly. ### How consent flows through the system 1. **Archiver initiates:** User clicks "Archive" on a DID → policy engine checks for consent signals. 2. **Consent check:** Is there an open consent record? Is there an existing directed offer from the subject? If yes → archive with consent. If no → archive without consent (unilateral). 3. **Reciprocity check:** Does the subject also archive the archiver? If yes → the relationship is reciprocal. The policy engine detects this and can surface it in the UI ("reciprocal") even if neither side explicitly requested reciprocity. 4. **Consent revocation:** Subject deletes their consent/offer record → the archiver's policy transitions. For reciprocal policies, this terminates the agreement. For unilateral archives, the data remains (it's public) but the consent status updates to "no consent." ### Consent status as policy metadata Rather than consent being a policy *type*, it's a property of every policy: ```typescript type ConsentStatus = | "reciprocal" // Both parties consented and archive each other | "consented" // Subject explicitly consented (open or directed) | "unconsented" // No consent signal from subject | "revoked"; // Subject previously consented, then revoked ``` This means the policy types become simpler — they describe *what* is happening, while consent describes the *relationship*: ## 4. Policy Type Taxonomy ### 4.1 Archive (`archive`) The fundamental operation: replicate a DID's data on this node. Every replication relationship starts as an archive. **Consent variants:** - **Consented archive:** Subject has an open consent record or accepted a directed offer. The archiver can prove permission. - **Unconsented archive:** No consent signal. The data is public, so this is always possible, but the UI makes the distinction visible. **Parameters:** - `minCopies`: 1 (just this node) - `intervalSec`: 300 (default) - `priority`: 40 - `consent`: ConsentStatus **Provenance:** Created by user action ("Archive" button), config, or API. **Invariant:** Active as long as the archiver wants it. If consent was present and gets revoked, the archive can continue (data is public) but consent status changes to `revoked`. ### 4.2 Reciprocal (`reciprocal`) A reciprocal policy is an archive with a bilateral consent agreement: both parties archive each other and both have consented. This is the current offer/accept flow. **How it emerges:** 1. Account A offers to archive Account B (publishes offer, creates a `proposed` reciprocal policy) 2. Account B accepts (publishes reciprocal offer, creating their own reciprocal policy) 3. Both policies transition to `active` when mutual offers are detected **Can also emerge organically:** If Account A creates an archive policy for B, and Account B independently creates an archive policy for A, the policy engine detects the mutual relationship and can prompt: "You're both archiving each other. Convert to a reciprocal agreement?" **Parameters:** - `minCopies`: 2 (at least two copies across the agreement) - `intervalSec`: 600 (default) - `priority`: 50 (higher than unilateral archive) - `counterpartyDid`: the other party's DID - `consent`: always `"reciprocal"` **Provenance:** Created by the offer/accept consent flow. Both sides hold a reciprocal policy pointing at each other. **Invariant:** Valid only when both parties have active offer records. If either side revokes consent, the policy terminates. The remaining party can choose to continue as a unilateral archive (downgrade to `archive` with `consent: "revoked"`). ### 4.3 Config (`config`) DIDs specified via the `REPLICATE_DIDS` environment variable. Infrastructure-level archives that persist across restarts and can't be modified from the UI. **Parameters:** - `minCopies`: 1 - `intervalSec`: 300 - `priority`: 30 (lowest — infrastructure baseline) - `consent`: typically `"unconsented"` (infra doesn't negotiate) **Provenance:** Generated at startup from config. Policy IDs are `config:{did}`. **Invariant:** Exists as long as the DID is in the config. Recreated on every startup. ### 4.4 Open Consent Record (`org.p2pds.replication.consent`) Not a policy type, but a mechanism that feeds into policy creation. An account publishes this record to declare open consent: ```json { "$type": "org.p2pds.replication.consent", "scope": "any", "createdAt": "2026-02-16T..." } ``` When a p2pds node discovers this record (during archival or via firehose), it can upgrade any unconsented archive of that DID to `consent: "consented"`. It also enables the UI to show "this account welcomes archiving" when a user is deciding whether to archive someone. Future extensions: `scope` could be `"any"`, `"followers"`, or an explicit DID list for directed open consent. ### 4.5 Future: SaaS (`saas`) An operator guarantees replication for a set of accounts under an SLA. Not implemented now, but the type system accommodates it. **Parameters:** - `minCopies`: 3+ (SLA-driven) - `intervalSec`: 60 (aggressive) - `priority`: 80+ (high) - SLA metadata: uptime target, response time, penalty terms ### 4.6 Future: Group / Mutual Aid (`group`) A community collectively decides which accounts to archive. This is where true mutual aid lives — community-level coordination rather than bilateral agreements. Requires quorum/voting mechanisms. **Parameters:** - `minCopies`: varies by group decision - `memberDids`: the governance group - `approvalThreshold`: N-of-M required - `priority`: 60 --- ## 4. Lifecycle State Machine Every policy goes through a lifecycle. The states and transitions depend on the policy type, but the core state machine is shared: ``` +-----------+ create --> | proposed |----> [reject] ----> terminated +-----------+ | [activate] | v +-----------+ | active |<---- [resume] +-----------+ | | [suspend]| |[terminate] v | +-----------+| | suspended || +-----------+| | | [resume] | | (back up) | v +-----------+ | terminated| +-----------+ | [purge data] | +-----------+ | purged | +-----------+ ``` ### State Definitions | State | Meaning | Data Kept? | Sync Active? | |-------|---------|------------|--------------| | `proposed` | Policy created but not yet activated (e.g., offer sent, awaiting acceptance) | No data yet | No | | `active` | Policy is live; DID is being replicated according to parameters | Yes | Yes | | `suspended` | Temporarily paused (health check failure, peer unreachable, user pause) | Yes (retained) | No | | `terminated` | Policy ended (revocation, offer deleted, manual removal) | Retained briefly | No | | `purged` | Data has been deleted; policy record kept for audit trail | No | No | ### State Transitions by Policy Type **Reciprocal:** ``` User clicks "Replicate" on a DID -> proposed (offer published, awaiting consent from counterparty) Counterparty consents (accepts offer, publishes reciprocal offer) -> active, consent: reciprocal (both sides syncing) Challenge failures exceed threshold -> suspended (grace period, retry) Grace period expires with no recovery -> terminated (notify counterparty) Either side revokes consent -> terminated (notify counterparty, begin data retention countdown) Data retention period expires -> purged ``` **Archive:** ``` User clicks "Archive" on a DID -> active (immediate; consent status set based on subject's consent signals) -> consent: consented (if open consent record exists) -> consent: unconsented (if no consent signal) Subject publishes open consent record -> consent status upgrades to "consented" Subject revokes open consent -> consent status changes to "revoked" (archive continues, data is public) User pauses -> suspended User resumes -> active User removes -> terminated -> purged (immediate, no retention needed) ``` **Config:** ``` Startup, DID in REPLICATE_DIDS -> active (immediate) DID removed from config + restart -> (policy simply not recreated) ``` ### Grace Periods and Escalation When a reciprocal policy enters `suspended` state, the system follows this escalation path: 1. **Soft suspension** (0-1 hour): Sync paused, challenge retries continue. No user notification. Peer may just be temporarily offline. 2. **Warning** (1-6 hours): UI shows warning badge. Push notification to the user. Continue monitoring. 3. **Hard suspension** (6-24 hours): UI shows alert. Data retained but no sync attempts. Peer gets a "your agreement is at risk" signal. 4. **Termination** (24+ hours): Agreement terminated. Begin data retention countdown (configurable, default 7 days). This is configuration within the policy itself, not hardcoded. The reciprocal preset provides sensible defaults, but users can adjust thresholds. ```typescript interface ViolationPolicy { /** Seconds of peer unreachability before soft suspension. */ softSuspensionAfterSec: number; // default: 3600 (1 hour) /** Seconds before warning escalation. */ warningAfterSec: number; // default: 21600 (6 hours) /** Seconds before hard suspension. */ hardSuspensionAfterSec: number; // default: 86400 (24 hours) /** Seconds before automatic termination. */ terminationAfterSec: number; // default: 86400 (24 hours) /** Seconds to retain data after termination before purge. */ dataRetentionAfterTerminationSec: number; // default: 604800 (7 days) /** Challenge failure count before suspension. */ challengeFailureThreshold: number; // default: 3 } ``` --- ## 5. UI Language Mapping The policy type and lifecycle state determine what the user sees. This section maps each state to display language for the primary UI contexts. ### 5.1 Action Buttons | Context | Current Language | Proposed Language | Notes | |---------|-----------------|-------------------|-------| | Adding a DID — request reciprocal | "Add" | "Replicate" | Initiates a reciprocal proposal (publishes offer, awaits consent) | | Adding a DID — unilateral archive | (not distinct) | "Archive" | Creates an archive policy; no consent required | | Responding to an incoming offer | "Accept" | "Accept & Replicate" | Makes it clear: you consent and replicate them back | | Declining an incoming offer | "Reject" | "Decline" | Less hostile than "reject" | | Stopping a reciprocal agreement | "Remove" | "Revoke" | Revokes your consent; agreement ends for both sides | | Stopping a unilateral archive | "Remove" | "Remove" | Simple removal, no consent implications | | Temporarily pausing | (not available) | "Pause" | Suspends without terminating | ### 5.2 Status Labels | Policy State | Consent | User-Facing Label | Icon/Badge | Description Text | |-------------|---------|-------------------|------------|------------------| | proposed | — | "Waiting for consent" | Clock | "You've offered to replicate this account. Waiting for them to accept." | | active | reciprocal | "Reciprocal" | Green shield | "You and {handle} are replicating each other's data." | | active | consented | "Archiving (consented)" | Green dot | "Archiving with {handle}'s consent." | | active | unconsented | "Archiving" | Blue dot | "Archiving public data." | | active | revoked | "Archiving (consent revoked)" | Yellow dot | "{handle} revoked consent. Data is still public." | | suspended (soft) | any | "Paused" | Yellow dot | "Sync paused. Retrying automatically." | | suspended (warning) | reciprocal | "Connection issues" | Orange warning | "{handle}'s node hasn't responded in {duration}." | | suspended (hard) | reciprocal | "Agreement at risk" | Red warning | "No response for {duration}. Agreement will end in {time remaining}." | | terminated | any | "Ended" | Gray X | "This replication relationship has ended." | | purged | any | (not shown) | — | Removed from active UI; available in history | ### 5.3 DID Source Labels Replace the current "config"/"user"/"policy" source labels with policy-aware labels: | Policy Type | Consent | Source Label | |-------------|---------|-------------| | reciprocal | reciprocal | "Reciprocal with {handle}" | | archive | consented | "Archived (consented)" | | archive | unconsented | "Archived" | | archive | revoked | "Archived (consent revoked)" | | config | any | "Infrastructure" | | saas | any | "SLA: {policy name}" | | group | any | "Group: {group name}" | ### 5.4 Notification Language | Event | Notification Text | |-------|-------------------| | Incoming offer | "{handle} wants to replicate your data. Accept to replicate each other." | | Offer accepted | "{handle} accepted. You're now replicating each other's data." | | Consent revoked by subject | "{handle} revoked archiving consent. Your archive continues (public data)." | | Reciprocal agreement revoked by peer | "{handle} revoked your replication agreement. Your data was removed from their node." | | Soft suspension | (no notification — auto-retry is silent) | | Warning | "{handle}'s node hasn't synced in {duration}." | | Termination by timeout | "Agreement with {handle} ended due to inactivity." | | Accidental reciprocity detected | "You and {handle} are both archiving each other. Formalize as a reciprocal agreement?" | ### 5.5 App Sections Restructure the app around policies and consent status: ``` Reciprocal Agreements @alice.bsky.social — Replicating (last sync: 2m ago) [Revoke] @bob.example.com — Waiting for consent [Cancel] Archives @carol.bsky.social — Archiving (consented) [Remove] did:plc:xyz123 — Archiving [Remove] Infrastructure did:plc:abc789 — Replicating (from config) Incoming Requests @dave.bsky.social wants to replicate your data. [Accept & Replicate] [Decline] ``` --- ## 6. Migration Path ### Phase 0: No-op foundation (prepare the ground) **Goal:** Add the `policies` table and policy persistence without changing any behavior. 1. Add SQLite `policies` table schema to SyncStorage (or a new PolicyStorage class). 2. Extend the `Policy` type with lifecycle fields (`state`, `createdAt`, `type`, `source`, `counterpartyDid`). 3. PolicyEngine gains `persist()` and `loadFromDb()` methods alongside the existing in-memory operations. 4. On startup, load policies from DB. Existing `POLICY_FILE` loading still works and seeds the DB. **Migration:** On first startup with the new schema, the `policies` table is empty. No behavior change. Existing in-memory policies continue to work. ### Phase 1: Config and archive policies **Goal:** Config DIDs and admin_tracked_dids generate policy objects. 1. At startup, for each DID in `REPLICATE_DIDS`, create a `config:{did}` policy if it doesn't exist. 2. When `addDid()` is called, instead of inserting into `admin_tracked_dids`, create a `archive` policy. 3. `getReplicateDids()` changes to: query all policies where `state = 'active'` and `enabled = 1`, collect target DIDs. No more three-source merging. 4. `getDidSource()` changes to: look up the policy type for the DID. 5. `removeDid()` changes to: transition the policy to `terminated`/`purged`. **Migration:** On first startup after this change, scan `admin_tracked_dids` and create `archive` policies for each entry. After successful migration, the `admin_tracked_dids` table becomes unused (keep it around briefly for rollback safety). ### Phase 2: Reciprocal lifecycle **Goal:** The offer flow creates and manages reciprocal policies with lifecycle states. 1. `offerDid()` creates a `reciprocal` policy in `proposed` state (replaces `offered_dids` insertion). 2. `acceptOffer()` transitions the policy to `active` (replaces the addDid() promotion). 3. `revokeReplication()` transitions to `terminated`. 4. `runOfferDiscovery()` no longer promotes offered_dids to admin_tracked_dids. Instead, it transitions `proposed` policies to `active` when mutual agreement is detected. 5. Broken-agreement detection transitions `active` policies to `terminated`. **Migration:** On first startup, scan `offered_dids` and create `proposed` reciprocal policies. Scan `admin_tracked_dids` entries that have corresponding P2P policies and convert them to `active` reciprocal policies. ### Phase 3: Health monitoring and suspension **Goal:** Policies can be suspended based on health signals. 1. Integrate challenge-response results: if a peer fails N consecutive challenges, suspend the policy. 2. Integrate sync failure tracking: if sync fails repeatedly for a DID, suspend. 3. Implement grace period escalation (soft -> warning -> hard -> termination). 4. Implement data retention countdown after termination. 5. Add health status to the policy evaluation output so the UI can display it. ### Phase 4: UI restructuring **Goal:** The dashboard is organized by policy, not by raw DID list. 1. API returns policies with lifecycle state, counterparty info, health status. 2. Dashboard groups DIDs by policy type. 3. Action buttons reflect policy semantics ("End agreement" vs "Remove archive"). 4. Notifications use policy-aware language. --- ## 7. Implementation Plan ### Phase 0: Foundation (estimated: 1-2 sessions) Files to modify: - `src/policy/types.ts` -- extend Policy with lifecycle fields - `src/policy/engine.ts` -- add persistence, load from DB - `src/replication/sync-storage.ts` -- add policies table schema (or new file `src/policy/storage.ts`) - `src/start.ts` -- load policies from DB on startup, seed from file Tests: - PolicyEngine persistence round-trip - Schema migration on fresh DB ### Phase 1: Config and Archive Policies (estimated: 2-3 sessions) Files to modify: - `src/start.ts` -- generate config policies at startup - `src/replication/replication-manager.ts` -- `addDid()` creates archive policy, `getReplicateDids()` reads from policy engine, `removeDid()` transitions policy - `src/xrpc/app.ts` -- update API responses to use policy-based source info - `src/policy/presets.ts` -- add `archive()` and `configArchive()` preset factories Migration script: - `src/policy/migrate.ts` -- scan admin_tracked_dids, create archive policies Tests: - addDid creates archive policy - getReplicateDids returns policy-driven list - Config DIDs generate config policies - Removal transitions policy state ### Phase 2: Reciprocal Lifecycle (estimated: 3-4 sessions) Files to modify: - `src/replication/replication-manager.ts` -- `offerDid()`, `acceptOffer()`, `revokeReplication()` operate on policies - `src/replication/offer-manager.ts` -- `syncPolicies()` transitions policy states instead of add/remove - `src/policy/engine.ts` -- lifecycle transition methods (`propose()`, `activate()`, `suspend()`, `terminate()`) - `src/policy/types.ts` -- add ViolationPolicy, lifecycle state enum Migration: - Convert offered_dids to proposed policies - Convert P2P-origin admin_tracked_dids to active reciprocal policies Tests: - Full offer -> accept -> active -> revoke -> terminated lifecycle - Broken agreement detection transitions to terminated - Policy state persists across restarts ### Phase 3: Health and Suspension (estimated: 2-3 sessions) Files to modify: - `src/policy/engine.ts` -- health evaluation, suspension logic - `src/replication/challenge-response/challenge-scheduler.ts` -- report results to policy engine - `src/replication/replication-manager.ts` -- sync failure reporting to policy engine - `src/policy/types.ts` -- ViolationPolicy defaults in presets Tests: - Challenge failures trigger suspension - Grace period escalation - Termination after timeout - Recovery from suspension when peer comes back ### Phase 4: UI Restructuring (estimated: 2-3 sessions) Files to modify: - `src/xrpc/app.ts` -- new API shape (policies with lifecycle state) - `src/index.ts` -- dashboard HTML rendering - UI template (inline HTML in index.ts or separate template) Tests: - API returns correct policy groupings - Status labels match lifecycle state - Action buttons match policy type --- ## 8. Detailed Type Definitions (Proposed) ```typescript /** Policy lifecycle states. */ type PolicyState = | "proposed" // Created, awaiting activation (e.g., offer sent) | "active" // Live, DID is being replicated | "suspended" // Temporarily paused (health issue, user pause) | "terminated" // Ended, data retained briefly | "purged"; // Data deleted, record kept for history /** What kind of policy this is. */ type PolicyType = | "reciprocal" // Bilateral agreement | "archive" // Unilateral personal archive | "config" // Infrastructure, from env var | "saas" // SLA compliance (future) | "group"; // Governance group (future) /** How the policy was created. */ type PolicySource = | "config" // REPLICATE_DIDS env var | "file" // POLICY_FILE JSON | "offer" // P2P offer flow | "manual" // User action in the UI | "api"; // Programmatic API call /** Consent status of a replication relationship. */ type ConsentStatus = | "reciprocal" // Both parties consented and archive each other | "consented" // Subject explicitly consented (open or directed) | "unconsented" // No consent signal from subject | "revoked"; // Subject previously consented, then revoked /** Extended policy with lifecycle and consent. */ interface PolicyV2 extends Policy { type: PolicyType; state: PolicyState; source: PolicySource; consent: ConsentStatus; // Timestamps createdAt: string; activatedAt: string | null; suspendedAt: string | null; terminatedAt: string | null; expiresAt: string | null; // Provenance createdBy: string; // DID or 'system' // Agreement linkage (reciprocal only) counterpartyDid: string | null; localOfferUri: string | null; remoteOfferUri: string | null; // Violation handling violation: ViolationPolicy; // Health snapshot (computed, not stored) health?: PolicyHealth; } /** Health status computed from sync/challenge data. */ interface PolicyHealth { /** Overall health: green/yellow/orange/red */ level: "healthy" | "degraded" | "warning" | "critical"; /** Last successful sync timestamp. */ lastSyncAt: string | null; /** Consecutive sync failures. */ consecutiveSyncFailures: number; /** Latest challenge result (if applicable). */ latestChallengeResult: "pass" | "fail" | "pending" | null; /** Peer reliability score (0-1). */ peerReliability: number | null; /** Seconds until next escalation (if degraded). */ escalationInSec: number | null; } ``` --- ## 9. Open Questions **Q1: What happens when an archive subject revokes consent?** If a subject had open consent and revokes it, should existing archives be terminated or just marked `consent: "revoked"`? The data is public regardless — revoking consent doesn't make the data private. But it does signal "I don't want you archiving me." Recommendation: Default behavior is to mark `consent: "revoked"` but keep the archive. The UI clearly shows the consent status change. The archiver can choose to respect the revocation and delete, or continue. For reciprocal policies, revocation terminates the agreement (since reciprocity requires bilateral consent). **Q2: How should the UI present unconsented archives?** Unconsented archiving is always technically possible (public data). But we should be thoughtful about the UX. Should the UI show a warning? Should it be a separate flow from consented archiving? Recommendation: The "Archive" action always works. If the subject has open consent, the UI shows a green consent badge. If not, the UI shows a neutral "no consent signal" indicator — not a warning, but visible. The distinction matters for trust and transparency, not for blocking the action. **Q3: How does open consent interact with reciprocal detection?** If Account A archives Account B (who has open consent), and Account B independently archives Account A, this is technically reciprocal but wasn't negotiated. Should the policy engine auto-detect this and offer to upgrade? Recommendation: Yes. The policy engine periodically checks for "accidental reciprocity" — two independent archive policies pointing at each other. It surfaces a prompt: "You and {handle} are both archiving each other. Would you like to formalize this as a reciprocal agreement?" This is opt-in, not automatic. **Q4: How long should terminated reciprocal data be retained?** The current implementation purges immediately on revocation. A grace period (e.g., 7 days) would allow the user to re-establish the agreement without re-syncing everything. But it also means storing data the peer has explicitly asked you to delete. Recommendation: Default 7-day retention for reciprocal termination (configurable to 0 for immediate purge). The data is content-addressed and already public (it's on their PDS), so retaining it briefly is not a privacy concern. The peer record itself says "terminated" — we're not claiming to still serve it. **Q5: Should the policy engine emit events?** State transitions (proposed → active, active → suspended, etc.) could emit events that other components subscribe to (e.g., UI notifications, sync scheduling changes, gossipsub announcements). Recommendation: Yes, but as a simple callback/listener pattern, not a full event bus. The PolicyEngine gains an `onStateChange(callback)` method. ReplicationManager and the UI layer register listeners. **Q6: What happens to policies when the user logs out?** Currently, logout disconnects OAuth and prevents offer management. Should policies survive logout? Reciprocal policies depend on offer records on the PDS, which require an active session to manage. Recommendation: Policies survive logout. Active policies continue to replicate (sync doesn't require authentication). But the user can't create, accept, or revoke offers without logging back in. The UI shows "Log in to manage agreements." **Q7: Policy versioning and schema evolution.** The current PolicySet has `version: 1`. As PolicyV2 adds fields, how do we handle upgrades? Recommendation: Version the `policies` table schema, not the policy objects. Use SQLite `ALTER TABLE ADD COLUMN` with defaults for additive changes. For breaking changes, write a migration function. The JSON fields (target_json, etc.) are forward-compatible by nature — unknown keys are ignored.