[READ ONLY MIRROR] Spark Social AppView Server
github.com/sprksocial/server
atproto
deno
hono
lexicon
1# Spark AppView
2
3An AT Protocol AppView implementation that provides a comprehensive view of the
4Spark lexicon.
5
6## Features
7
8- **Real-time sync**: Subscribes to AT Protocol relay for live data ingestion
9- **Rich API**: XRPC endpoints for feeds, profiles, audio, stories, and social
10 graph
11- **MongoDB storage**: Efficient document-based storage with Mongoose ODM
12- **Pipeline architecture**: Clean separation between skeleton, hydration, and
13 presentation layers
14
15## Quick Start
16
17### Docker Compose (Recommended)
18
19```bash
20deno task docker-dev
21```
22
23This starts MongoDB and the AppView in Docker containers with hot reloading at
24`http://localhost:4000`.
25
26### Local Development
27
281. **Prerequisites**: Deno 2.x, MongoDB 8.x
29
302. **Environment setup**: Create `.env` file (see Configuration below)
31
323. **Start services**:
33
34```bash
35deno task dev
36```
37
38This runs three parallel services:
39
40- MongoDB (`dev:db`)
41- API server (`dev:api`) on port 4000
42- Ingester (`dev:ingest`) for real-time sync
43
44## Architecture
45
46### Key Directories
47
48- `api/` - XRPC endpoint handlers using pipeline pattern
49- `data-plane/` - Database layer, indexing plugins, and subscription logic
50- `hydration/` - Data enrichment layer (actors, feeds, graphs)
51- `views/` - Presentation layer transforming hydrated data to API responses
52- `lexicons/` - AT Protocol lexicon definitions (JSON)
53- `lex/` - Generated TypeScript types from lexicons
54- `utils/` - Shared utilities (transformers, logger, retry logic)
55
56### Data Flow
57
58```
59AT Protocol Relay → Ingester → MongoDB ← Data Plane ← Pipeline ← API Endpoints
60 (Firehose) (ingest.ts) (Raw queries) (4 stages) (XRPC)
61```
62
63### Request Pipeline (api/)
64
65Every API endpoint follows a 4-stage pipeline pattern:
66
67```
68 Client Request
69 │
70 ▼
71┌─────────────────────────────────────────────────────────────────┐
72│ 1. SKELETON │
73│ • Query parameters → minimal data identifiers (URIs, DIDs) │
74│ • Fast database queries for structure only │
75│ • Returns: { postUris: [...], authorDids: [...] } │
76└────────────────────────────────┬────────────────────────────────┘
77 ▼
78┌─────────────────────────────────────────────────────────────────┐
79│ 2. HYDRATION (hydration/) │
80│ • Skeleton → Data Plane → rich data from MongoDB │
81│ • Batch fetches: actors, posts, likes, blocks, etc. │
82│ • Returns: HydrationState with all related records │
83└────────────────────────────────┬────────────────────────────────┘
84 ▼
85┌─────────────────────────────────────────────────────────────────┐
86│ 3. RULES │
87│ • Apply business logic (filtering, sorting, permissions) │
88│ • Modify skeleton based on hydrated data │
89│ • Returns: Modified skeleton │
90└────────────────────────────────┬────────────────────────────────┘
91 ▼
92┌─────────────────────────────────────────────────────────────────┐
93│ 4. PRESENTATION (views/) │
94│ • Skeleton + Hydration → API response format │
95│ • Transform internal models to lexicon types │
96│ • Apply CDN URLs, format dates, handle takedowns │
97│ • Returns: JSON response matching lexicon schema │
98└────────────────────────────────┬────────────────────────────────┘
99 ▼
100 Client Response
101```
102
103### Layer Responsibilities
104
105**Data Plane** (`data-plane/`)
106
107- Direct MongoDB access through Mongoose models
108- Route handlers: `actors`, `feeds`, `follows`, `likes`, `blocks`, etc.
109- No business logic, pure data operations
110- Used only by Hydrator
111
112**Hydrator** (`hydration/`)
113
114- Orchestrates Data Plane queries
115- Batches requests for efficiency
116- Maintains viewer context (permissions, blocks)
117- Returns `HydrationState` with all data needed for presentation
118
119**Views** (`views/`)
120
121- Pure transformation functions
122- No database access
123- Applies CDN URLs, formats responses
124- Enforces lexicon schemas
125
126## Configuration
127
128Create a `.env` file:
129
130```bash
131# Database
132SPRK_DB_URI=mongodb://mongo:mongo@localhost:27017
133SPRK_DB_NAME=dev
134
135NODE_ENV=development
136SPRK_PORT=4000
137SPRK_PUBLIC_URL=https://example.com
138SPRK_SERVER_DID=did:web:example.com
139
140# openssl ecparam -name secp256k1 -genkey -noout -outform DER | tail -c +8 | head -c 32 | xxd -p -c 32
141SPRK_PRIVATE_KEY=your_private_key_hex
142SPRK_ADMIN_PASSWORDS=password1,password2
143SPRK_MOD_SERVICE_DID=did:web:mod.bsky.app
144
145SPRK_VERSION=0.1.0
146SPRK_INDEXED_AT_EPOCH=2025-01-01T00:00:00Z
147```