···11# HappyView
2233-HappyView is a lexicon-driven ATProto AppView. Upload lexicon definitions at runtime and HappyView dynamically generates XRPC query and procedure endpoints, indexes records from the network via Jetstream, and proxies writes to users' PDSes --- no restart required.
33+HappyView is the best way to build an [AppView](https://atproto.com/guides/glossary#app-view) for the [AT Protocol](https://atproto.com). Upload your [lexicon](reference/glossary#at-protocol-terms) schemas and get a fully functional AppView, complete with [XRPC](reference/glossary#at-protocol-terms) endpoints, OAuth, real-time network sync, and historical [backfill](guides/backfill), without writing a single line of server code.
44+55+Building an AppView from scratch means wiring up firehose connections, record storage, XRPC routing, OAuth flows, and PDS write proxying before you can even think about your application. HappyView handles all of that. Define your data model with lexicons, add custom logic with Lua scripts when you need it, and ship your app.
66+77+## Features
88+99+- 📜 **Lexicon-Driven**: Upload your lexicon schemas and HappyView generates fully functional XRPC query and procedure endpoints automatically, no code required
1010+- 🔄 **Real-Time Sync**: Records stream in from the AT Protocol network in real-time via [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap), with cryptographic verification and backfill via the admin API
1111+- 🔐 **OAuth Built In**: [AIP](https://github.com/graze-social/aip) handles authentication, and writes are proxied back to the user's PDS, so there's no session management needed
1212+- 🌙 **Lua Scripting**: Add custom query and procedure logic with Lua scripts that have full access to the record database
1313+- 🗄️ **Automatic Indexing**: HappyView indexes relevant records into PostgreSQL as they arrive, ready to query
1414+- 🌐 **Network Lexicons**: Fetch lexicon schemas directly from the AT Protocol network via DNS authority resolution
1515+- ⚡ **Hot Reloading**: Upload or update lexicons at runtime, and new endpoints are available immediately with no restart
1616+- 🛠️ **Admin Dashboard**: Manage lexicons, monitor record stats, and run backfill jobs through a built-in admin API
41755-## How it fits together
1818+## Design Principles
61977-```
88-Jetstream ──> HappyView ──> PostgreSQL
99- │
1010- ┌───────────┼───────────┐
1111- │ │ │
1212- Clients AIP PDSes
1313- (OAuth) (user repos)
1414-```
2020+- **Schema-first**: Your Lexicons are the source of truth. Upload a schema and HappyView derives endpoints, indexing rules, and network sync from it. You describe _what_ your data looks like; HappyView figures out the rest.
2121+2222+- **Zero boilerplate**: HappyView handles AppView infrastructure (firehose, backfill, OAuth, PDS proxying) for you. You should be writing application logic from minute one, not plumbing.
2323+2424+- **Runtime-configurable**: Lexicons can be added, updated, and removed without restarting the server. New endpoints and sync rules take effect immediately, so you can iterate on your data model in real time.
15251616-- **Jetstream** pushes real-time record events. HappyView subscribes to the collections defined by uploaded lexicons and indexes records into Postgres.
1717-- **AIP** (ATProto Identity Provider) handles OAuth 2.1 with PKCE. HappyView validates tokens by calling AIP's `/oauth/userinfo` endpoint.
1818-- **PDSes** store user data. HappyView proxies writes and blob uploads to each user's PDS using DPoP-authenticated requests.
1919-- **Clients** talk to HappyView's XRPC and admin APIs. Any ATProto-compatible client can connect.
2626+- **Protocol-native**: HappyView works with _any_ PDS, resolves DIDs through the directory, and follows AT Protocol conventions. It's a first-class citizen of the network, not a wrapper around it.
20272121-## Docs
2828+## Next Steps
22292323-- [Quickstart](quickstart.md) - get a local instance running
2424-- [Configuration](configuration.md) - environment variables reference
2525-- [Deployment](deployment.md) - Docker, production, TLS
2626-- [Lexicons](lexicons.md) - uploading and managing lexicon definitions
2727-- [Network Lexicons](network-lexicons.md) - loading lexicons from the ATProto network
2828-- [Backfill](backfill.md) - bulk-indexing historical records
2929-- [XRPC API](xrpc-api.md) - query and procedure endpoints
3030-- [Admin API](admin-api.md) - manage lexicons, backfills, and admins
3131-- [Architecture](architecture.md) - internals for contributors
3030+- [Quickstart](getting-started/deployment/railway): Deploy HappyView on Railway or run it locally
3131+- [Lexicons](guides/lexicons): Upload lexicon schemas and start indexing records
3232+- [Lua Scripting](guides/scripting): Write custom query and procedure logic
+61-24
docs/admin-api.md
docs/reference/admin-api.md
···11# Admin API
2233-All admin endpoints live under `/admin` and require an AIP-issued Bearer token from a DID that exists in the `admins` table.
33+The admin API lets you manage lexicons, monitor records, run backfill jobs, and control admin access. All endpoints live under `/admin` and require an [AIP](https://github.com/graze-social/aip)-issued Bearer token from a DID that exists in the `admins` table. You can also manage all of this through the [web dashboard](../getting-started/dashboard).
4455## Auth
6677-Admin auth works the same as user auth — the Bearer token is validated against AIP's `/oauth/userinfo` endpoint to retrieve the caller's DID. That DID is then checked against the `admins` table.
77+Admin auth works the same as user auth: the Bearer token is validated against AIP's `/oauth/userinfo` endpoint to retrieve the caller's DID. That DID is then checked against the `admins` table.
8899**Auto-bootstrap**: If the `admins` table is empty, the first authenticated request automatically inserts the caller as the initial admin.
10101111Non-admin DIDs receive a `403 Forbidden` response.
12121313+All error responses return JSON with an `error` field:
1414+1515+```json
1616+{
1717+ "error": "description of what went wrong"
1818+}
1919+```
2020+2121+| Status | Meaning |
2222+|--------|---------|
2323+| `400 Bad Request` | Invalid input (missing required fields, malformed lexicon JSON) |
2424+| `401 Unauthorized` | Missing or invalid Bearer token. See [AIP documentation](https://github.com/graze-social/aip) for token issues |
2525+| `403 Forbidden` | Authenticated DID is not in the admins table |
2626+| `404 Not Found` | Lexicon, admin, or backfill job not found |
2727+1328```sh
1429# All examples assume $TOKEN is an AIP-issued access token for an admin DID
1530AUTH="Authorization: Bearer $TOKEN"
1631```
1717-1818----
19322033## Lexicons
2134···3043 -H "$AUTH" \
3144 -H "Content-Type: application/json" \
3245 -d '{
3333- "lexicon_json": { "lexicon": 1, "id": "example.record", "defs": { "main": { "type": "record", "key": "tid", "record": { "type": "object", "properties": { "title": { "type": "string" } } } } } },
4646+ "lexicon_json": { "lexicon": 1, "id": "xyz.statusphere.status", "defs": { "main": { "type": "record", "key": "tid", "record": { "type": "object", "required": ["status", "createdAt"], "properties": { "status": { "type": "string", "maxGraphemes": 1 }, "createdAt": { "type": "string", "format": "datetime" } } } } } },
3447 "backfill": true,
3548 "target_collection": null
3649 }'
···46594760```json
4861{
4949- "id": "example.record",
6262+ "id": "xyz.statusphere.status",
5063 "revision": 1
5164}
5265```
···6679```json
6780[
6881 {
6969- "id": "example.record",
8282+ "id": "xyz.statusphere.status",
7083 "revision": 1,
7184 "lexicon_type": "record",
7285 "backfill": true,
···8396```
84978598```sh
8686-curl http://localhost:3000/admin/lexicons/example.record -H "$AUTH"
9999+curl http://localhost:3000/admin/lexicons/xyz.statusphere.status -H "$AUTH"
87100```
8810189102**Response**: `200 OK` with full lexicon details including raw JSON.
···95108```
9610997110```sh
9898-curl -X DELETE http://localhost:3000/admin/lexicons/example.record -H "$AUTH"
111111+curl -X DELETE http://localhost:3000/admin/lexicons/xyz.statusphere.status -H "$AUTH"
99112```
100113101114**Response**: `204 No Content`
102102-103103----
104115105116## Network Lexicons
106117107107-Network lexicons are fetched from the ATProto network via DNS TXT resolution and kept updated via Jetstream. See [Network Lexicons](network-lexicons.md) for background.
118118+Network lexicons are fetched from the AT Protocol network via DNS TXT resolution and kept updated via Tap. See [Lexicons - Network lexicons](../guides/lexicons#network-lexicons) for background.
108119109120### Add a network lexicon
110121···117128 -H "$AUTH" \
118129 -H "Content-Type: application/json" \
119130 -d '{
120120- "nsid": "games.gamesgamesgamesgames.game",
131131+ "nsid": "xyz.statusphere.status",
121132 "target_collection": null
122133 }'
123134```
···133144134145```json
135146{
136136- "nsid": "games.gamesgamesgamesgames.game",
147147+ "nsid": "xyz.statusphere.status",
137148 "authority_did": "did:plc:authority",
138149 "revision": 1
139150}
···154165```json
155166[
156167 {
157157- "nsid": "games.gamesgamesgamesgames.game",
168168+ "nsid": "xyz.statusphere.status",
158169 "authority_did": "did:plc:authority",
159170 "target_collection": null,
160171 "last_fetched_at": "2025-01-01T00:00:00Z",
···170181```
171182172183```sh
173173-curl -X DELETE http://localhost:3000/admin/network-lexicons/games.gamesgamesgamesgames.game \
184184+curl -X DELETE http://localhost:3000/admin/network-lexicons/xyz.statusphere.status \
174185 -H "$AUTH"
175186```
176187···178189179190**Response**: `204 No Content`
180191181181----
182182-183192## Stats
184193185194### Record counts
···197206```json
198207{
199208 "total_records": 12345,
200200- "collections": [{ "collection": "example.record", "count": 500 }]
209209+ "collections": [{ "collection": "xyz.statusphere.status", "count": 500 }]
201210}
202211```
203212204204----
213213+## Tap Stats
214214+215215+Aggregate stats from the [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap) instance. Useful for monitoring backfill progress. See [Backfill - Job lifecycle](../guides/backfill#job-lifecycle) for context.
216216+217217+### Get Tap stats
218218+219219+```
220220+GET /admin/tap/stats
221221+```
222222+223223+```sh
224224+curl http://localhost:3000/admin/tap/stats -H "$AUTH"
225225+```
226226+227227+**Response**: `200 OK`
228228+229229+```json
230230+{
231231+ "repo_count": 5234,
232232+ "record_count": 1048576,
233233+ "outbox_buffer": 42
234234+}
235235+```
236236+237237+| Field | Type | Description |
238238+| -------------- | ------ | -------------------------------------------------------- |
239239+| `repo_count` | number | Total repos Tap is tracking |
240240+| `record_count` | number | Total records Tap has indexed |
241241+| `outbox_buffer`| number | Pending events awaiting delivery (high = Tap is busy) |
242242+243243+Returns `502 Bad Gateway` if Tap is unreachable.
205244206245## Backfill
207246···215254curl -X POST http://localhost:3000/admin/backfill \
216255 -H "$AUTH" \
217256 -H "Content-Type: application/json" \
218218- -d '{ "collection": "example.record" }'
257257+ -d '{ "collection": "xyz.statusphere.status" }'
219258```
220259221260| Field | Type | Required | Description |
···248287[
249288 {
250289 "id": "550e8400-e29b-41d4-a716-446655440000",
251251- "collection": "example.record",
290290+ "collection": "xyz.statusphere.status",
252291 "did": null,
253292 "status": "completed",
254293 "total_repos": 42,
···261300 }
262301]
263302```
264264-265265----
266303267304## Admin management
268305
-170
docs/architecture.md
···11-# Architecture
22-33-Guide for contributors working on HappyView itself.
44-55-## Module overview
66-77-```
88-src/
99- main.rs Startup: config, DB, migrations, spawn workers, start server
1010- lib.rs AppState struct, module declarations
1111- config.rs Environment variable loading
1212- error.rs AppError enum (Auth, BadRequest, Forbidden, Internal, NotFound, PdsError)
1313- server.rs Axum router: fixed routes + admin nest + XRPC catch-all
1414- lexicon.rs ParsedLexicon, LexiconRegistry (Arc<RwLock<HashMap>>)
1515- profile.rs DID document resolution, PDS discovery, profile fetching
1616- jetstream.rs Jetstream WebSocket listener, record indexing
1717- backfill.rs Background worker for bulk record ingestion
1818- auth/
1919- middleware.rs Claims extractor (validates Bearer token via AIP /oauth/userinfo)
2020- admin/
2121- mod.rs Admin route definitions
2222- auth.rs AdminAuth extractor (Claims + DID lookup + auto-bootstrap)
2323- admins.rs Admin CRUD handlers
2424- lexicons.rs Lexicon CRUD handlers
2525- stats.rs Record count stats
2626- backfill.rs Backfill job creation and status
2727- types.rs Request/response structs for admin endpoints
2828- repo/
2929- mod.rs Re-exports
3030- dpop.rs DPoP JWT proof generation (ES256/P-256)
3131- pds.rs PDS proxy helpers (JSON POST, blob POST, response forwarding)
3232- session.rs ATP session fetching from AIP
3333- upload_blob.rs Blob upload handler
3434- media.rs Media blob URL enrichment
3535- at_uri.rs AT URI parsing
3636- xrpc/
3737- mod.rs Re-exports
3838- query.rs Dynamic GET handler (single record + list with pagination)
3939- procedure.rs Dynamic POST handler (create vs put auto-detection)
4040-```
4141-4242-## Request flow
4343-4444-### Reads (queries)
4545-4646-```
4747-Client GET /xrpc/{method}?params
4848- -> xrpc::xrpc_get()
4949- -> LexiconRegistry lookup (must be Query type)
5050- -> SQL query on records table (collection from target_collection)
5151- -> Media blob URL enrichment
5252- -> JSON response
5353-```
5454-5555-### Writes (procedures)
5656-5757-```
5858-Client POST /xrpc/{method} + Bearer token
5959- -> Claims extractor validates token via AIP /oauth/userinfo
6060- -> xrpc::xrpc_post()
6161- -> LexiconRegistry lookup (must be Procedure type)
6262- -> Fetch ATP session from AIP /api/atprotocol/session
6363- -> Generate DPoP proof (ES256)
6464- -> Proxy to user's PDS (createRecord or putRecord)
6565- -> Upsert record locally
6666- -> Forward PDS response
6767-```
6868-6969-### Admin endpoints
7070-7171-```
7272-Client request + Bearer token
7373- -> AdminAuth extractor:
7474- 1. Claims validation via AIP
7575- 2. DID lookup in admins table (auto-bootstrap if empty)
7676- 3. 403 if not admin
7777- -> Admin handler
7878- -> JSON response
7979-```
8080-8181-## Data flow
8282-8383-### Real-time indexing
8484-8585-```
8686-Jetstream WebSocket
8787- -> Filter by wantedCollections (from record-type lexicons)
8888- -> commit events:
8989- create/update -> UPSERT into records table
9090- delete -> DELETE from records table
9191- -> Cursor tracked in AtomicI64 (rewinds 5s on reconnect)
9292-```
9393-9494-### Backfill
9595-9696-```
9797-backfill_jobs table (status = pending)
9898- -> Worker picks up job
9999- -> Relay listReposByCollection -> list of DIDs
100100- -> For each DID (8 concurrent):
101101- PLC directory -> PDS endpoint
102102- PDS listRecords -> UPSERT into records table
103103- -> Update job status and counters
104104-```
105105-106106-## Database schema
107107-108108-### `records`
109109-110110-| Column | Type | Description |
111111-|--------|------|-------------|
112112-| `uri` | text (PK) | AT URI (`at://did/collection/rkey`) |
113113-| `did` | text | Author DID |
114114-| `collection` | text | Lexicon NSID |
115115-| `rkey` | text | Record key |
116116-| `record` | jsonb | Record value |
117117-| `cid` | text | Content identifier |
118118-| `indexed_at` | timestamptz | When HappyView indexed this record |
119119-120120-### `lexicons`
121121-122122-| Column | Type | Description |
123123-|--------|------|-------------|
124124-| `id` | text (PK) | Lexicon NSID |
125125-| `revision` | integer | Incremented on upsert |
126126-| `lexicon_json` | jsonb | Raw lexicon definition |
127127-| `lexicon_type` | text | record, query, procedure, definitions |
128128-| `backfill` | boolean | Whether to backfill on upload |
129129-| `target_collection` | text | For queries/procedures: which record collection |
130130-| `created_at` | timestamptz | |
131131-| `updated_at` | timestamptz | |
132132-133133-### `admins`
134134-135135-| Column | Type | Description |
136136-|--------|------|-------------|
137137-| `id` | uuid (PK) | |
138138-| `did` | text (unique) | Admin's ATProto DID |
139139-| `created_at` | timestamptz | |
140140-| `last_used_at` | timestamptz | Updated on each authenticated request |
141141-142142-### `backfill_jobs`
143143-144144-| Column | Type | Description |
145145-|--------|------|-------------|
146146-| `id` | uuid (PK) | |
147147-| `collection` | text | Target collection (null = all) |
148148-| `did` | text | Target DID (null = all) |
149149-| `status` | text | pending, running, completed, failed |
150150-| `total_repos` | integer | |
151151-| `processed_repos` | integer | |
152152-| `total_records` | integer | |
153153-| `error` | text | Error message if failed |
154154-| `started_at` | timestamptz | |
155155-| `completed_at` | timestamptz | |
156156-| `created_at` | timestamptz | |
157157-158158-## Testing
159159-160160-```sh
161161-# Unit tests (no database needed)
162162-cargo test --lib
163163-164164-# All tests including e2e (requires Postgres)
165165-docker compose -f docker-compose.test.yml up -d
166166-TEST_DATABASE_URL=postgres://happyview:happyview@localhost:5433/happyview_test cargo test
167167-docker compose -f docker-compose.test.yml down
168168-```
169169-170170-E2e tests use `wiremock` to mock external services (AIP, PLC directory, PDSes) and a real Postgres database for full integration coverage.
-78
docs/backfill.md
···11-# Backfill
22-33-Backfill bulk-indexes existing records from the ATProto network into HappyView's database. It discovers repos via the relay and fetches records directly from each user's PDS.
44-55-## When backfill runs
66-77-- **Automatically** when a record-type lexicon is uploaded with `backfill: true` (the default)
88-- **Manually** via `POST /admin/backfill`
99-1010-## How it works
1111-1212-1. **Determine target collections**: uses the specified collection, or all record lexicons with `backfill: true`
1313-2. **Discover DIDs**: calls the relay's `com.atproto.sync.listReposByCollection` to find repos that contain records for each collection (paginated, 1000 per page)
1414-3. **Fetch records**: for each DID, resolves the PDS endpoint via PLC directory, then calls `com.atproto.repo.listRecords` on the PDS (paginated, 100 per page)
1515-4. **Upsert**: inserts or updates records in Postgres
1616-1717-## Creating a backfill job
1818-1919-### Backfill all collections
2020-2121-```sh
2222-curl -X POST http://localhost:3000/admin/backfill \
2323- -H "Authorization: Bearer $TOKEN" \
2424- -H "Content-Type: application/json" \
2525- -d '{}'
2626-```
2727-2828-### Backfill a specific collection
2929-3030-```sh
3131-curl -X POST http://localhost:3000/admin/backfill \
3232- -H "Authorization: Bearer $TOKEN" \
3333- -H "Content-Type: application/json" \
3434- -d '{ "collection": "games.gamesgamesgamesgames.game" }'
3535-```
3636-3737-### Backfill a specific DID
3838-3939-```sh
4040-curl -X POST http://localhost:3000/admin/backfill \
4141- -H "Authorization: Bearer $TOKEN" \
4242- -H "Content-Type: application/json" \
4343- -d '{
4444- "collection": "games.gamesgamesgamesgames.game",
4545- "did": "did:plc:abc123"
4646- }'
4747-```
4848-4949-## Monitoring progress
5050-5151-```sh
5252-curl http://localhost:3000/admin/backfill/status \
5353- -H "Authorization: Bearer $TOKEN"
5454-```
5555-5656-```json
5757-[
5858- {
5959- "id": "550e8400-e29b-41d4-a716-446655440000",
6060- "collection": "games.gamesgamesgamesgames.game",
6161- "did": null,
6262- "status": "running",
6363- "total_repos": 42,
6464- "processed_repos": 15,
6565- "total_records": 350,
6666- "error": null,
6767- "started_at": "2025-01-01T00:01:00Z",
6868- "completed_at": null,
6969- "created_at": "2025-01-01T00:00:00Z"
7070- }
7171-]
7272-```
7373-7474-Job statuses: `pending` -> `running` -> `completed` or `failed`.
7575-7676-## Concurrency
7777-7878-The backfill worker processes one job at a time, polling for pending jobs every 5 seconds. Within a job, up to 8 PDSes are fetched concurrently.
-31
docs/configuration.md
···11-# Configuration
22-33-HappyView is configured via environment variables. A `.env` file in the project root is loaded automatically on startup.
44-55-## Environment variables
66-77-| Variable | Required | Default | Description |
88-|----------|----------|---------|-------------|
99-| `DATABASE_URL` | yes | --- | Postgres connection string |
1010-| `AIP_URL` | yes | --- | AIP instance URL for OAuth token validation |
1111-| `HOST` | no | `0.0.0.0` | Bind host |
1212-| `PORT` | no | `3000` | Bind port |
1313-| `JETSTREAM_URL` | no | `wss://jetstream2.us-west.bsky.network/subscribe` | Jetstream WebSocket URL |
1414-| `RELAY_URL` | no | `https://bsky.network` | Relay URL for backfill repo discovery |
1515-| `PLC_URL` | no | `https://plc.directory` | PLC directory URL for DID resolution |
1616-| `RUST_LOG` | no | `happyview=debug,tower_http=debug` | Log filter (uses `tracing_subscriber::EnvFilter`) |
1717-1818-## Example `.env`
1919-2020-```sh
2121-DATABASE_URL=postgres://happyview:happyview@localhost/happyview
2222-AIP_URL=http://localhost:8080
2323-2424-# Optional overrides
2525-# HOST=0.0.0.0
2626-# PORT=3000
2727-# JETSTREAM_URL=wss://jetstream2.us-west.bsky.network/subscribe
2828-# RELAY_URL=https://bsky.network
2929-# PLC_URL=https://plc.directory
3030-# RUST_LOG=happyview=debug,tower_http=debug
3131-```
-61
docs/deployment.md
···11-# Deployment
22-33-## Docker
44-55-Build the image:
66-77-```sh
88-docker build -t happyview .
99-```
1010-1111-### Local development with Docker Compose
1212-1313-```sh
1414-docker compose up
1515-```
1616-1717-This starts Postgres, AIP, and HappyView together. See `docker-compose.yml` for the full configuration.
1818-1919-### Production Compose example
2020-2121-```yaml
2222-services:
2323- postgres:
2424- image: postgres:17
2525- environment:
2626- POSTGRES_USER: happyview
2727- POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
2828- POSTGRES_DB: happyview
2929- volumes:
3030- - pgdata:/var/lib/postgresql/data
3131-3232- happyview:
3333- image: happyview:latest
3434- ports:
3535- - "3000:3000"
3636- environment:
3737- DATABASE_URL: "postgres://happyview:${POSTGRES_PASSWORD}@postgres/happyview"
3838- AIP_URL: "https://aip.example.com"
3939- depends_on:
4040- postgres:
4141- condition: service_healthy
4242-4343-volumes:
4444- pgdata:
4545-```
4646-4747-## Railway / Fly.io / generic
4848-4949-1. Provision a Postgres database
5050-2. Set `DATABASE_URL` and `AIP_URL` environment variables
5151-3. Deploy the Docker image or build from source
5252-4. HappyView listens on `PORT` (default `3000`)
5353-5. Health check: `GET /health` returns `ok`
5454-5555-## Database
5656-5757-Migrations run automatically on startup via `sqlx::migrate!()`. No manual migration step is needed.
5858-5959-## TLS
6060-6161-HappyView does not terminate TLS. Put it behind a reverse proxy (nginx, Caddy, Cloudflare Tunnel, etc.) for HTTPS.
+65
docs/getting-started/authentication.md
···11+# Authentication
22+33+HappyView uses [AT Protocol OAuth](https://atproto.com/specs/oauth) for authentication, handled by an external [AIP](https://github.com/graze-social/aip) instance. HappyView does not store credentials or issue tokens: all OAuth is delegated to AIP.
44+55+## Which endpoints require auth?
66+77+| Endpoint type | Auth required? |
88+|---------------|---------------|
99+| Queries (`GET /xrpc/{method}`) | No |
1010+| Procedures (`POST /xrpc/{method}`) | Yes |
1111+| Admin API (`/admin/*`) | Yes (must be an admin) |
1212+| Health check (`GET /health`) | No |
1313+1414+Authenticated requests must include an `Authorization` header with a token issued by AIP:
1515+1616+```
1717+Authorization: Bearer <token>
1818+```
1919+2020+## Getting a token from the dashboard
2121+2222+The easiest way to get a token for CLI or curl usage is through the [web dashboard](dashboard):
2323+2424+1. Open the dashboard and log in with your AT Protocol identity
2525+2. Open your browser's developer tools (F12 or Cmd+Shift+I)
2626+3. Go to **Application** (Chrome) or **Storage** (Firefox) > **Session Storage**
2727+4. Find the entry for your dashboard's URL
2828+5. Copy the value of the `session` key: this contains your access token
2929+3030+You can then use it in curl:
3131+3232+```sh
3333+export TOKEN="your-token-here"
3434+curl http://localhost:3000/admin/lexicons \
3535+ -H "Authorization: Bearer $TOKEN"
3636+```
3737+3838+Tokens expire based on AIP's configuration. When a token expires, log in again through the dashboard to get a new one.
3939+4040+## Programmatic access
4141+4242+For scripts or applications that need to authenticate programmatically, you'll need to implement the AT Protocol OAuth flow against your AIP instance. This involves:
4343+4444+1. Registering an OAuth client with AIP
4545+2. Redirecting the user to AIP's authorization endpoint
4646+3. Exchanging the authorization code for an access token
4747+4. Using that token with HappyView
4848+4949+See the [AIP documentation](https://github.com/graze-social/aip) for endpoint details and the [ATProto OAuth spec](https://atproto.com/specs/oauth) for the full protocol.
5050+5151+## How token validation works
5252+5353+When HappyView receives an authenticated request, it forwards the token to AIP's `/oauth/userinfo` endpoint. AIP responds with the user's DID, which HappyView uses to:
5454+5555+- Identify who is making the request
5656+- Proxy writes to the correct PDS
5757+- Check admin permissions (for admin endpoints)
5858+5959+Token validation happens on every request; there is no local token caching.
6060+6161+## Admin access
6262+6363+Admin endpoints require the authenticated user's DID to exist in the `admins` table. If the table is empty (fresh deployment), the first authenticated request to any admin endpoint auto-bootstraps that user as the initial admin.
6464+6565+To add more admins, use `POST /admin/admins` or the [dashboard](dashboard). See [Admin API](../reference/admin-api#admin-management) for details.
+33
docs/getting-started/configuration.md
···11+# Configuration
22+33+HappyView is configured via environment variables. A `.env` file in the project root is loaded automatically on startup. See [Deployment](deployment/docker) for local setup or [Production Deployment](../reference/production-deployment) for production setup.
44+55+## Environment variables
66+77+| Variable | Required | Default | Description |
88+|----------|----------|---------|-------------|
99+| `DATABASE_URL` | yes | --- | Postgres connection string |
1010+| `AIP_URL` | yes | --- | [AIP](https://github.com/graze-social/aip) instance URL for OAuth token validation |
1111+| `HOST` | no | `0.0.0.0` | Bind host |
1212+| `PORT` | no | `3000` | Bind port |
1313+| `TAP_URL` | no | `http://localhost:2480` | [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap) instance URL for real-time record streaming and backfill |
1414+| `TAP_ADMIN_PASSWORD` | no | --- | Shared secret for authenticating with Tap's admin endpoints |
1515+| `RELAY_URL` | no | `https://bsky.network` | Relay URL for [backfill](../guides/backfill) repo discovery |
1616+| `PLC_URL` | no | `https://plc.directory` | [PLC directory](https://github.com/did-method-plc/did-method-plc) URL for DID resolution |
1717+| `RUST_LOG` | no | `happyview=debug,tower_http=debug` | Log filter (uses `tracing_subscriber::EnvFilter`) |
1818+1919+## Example `.env`
2020+2121+```sh
2222+DATABASE_URL=postgres://happyview:happyview@localhost/happyview
2323+AIP_URL=http://localhost:8080
2424+2525+# Optional overrides
2626+# HOST=0.0.0.0
2727+# PORT=3000
2828+# TAP_URL=http://localhost:2480
2929+# TAP_ADMIN_PASSWORD=your-secret-here
3030+# RELAY_URL=https://bsky.network
3131+# PLC_URL=https://plc.directory
3232+# RUST_LOG=happyview=debug,tower_http=debug
3333+```
+38
docs/getting-started/dashboard.md
···11+# Dashboard
22+33+HappyView ships with a web dashboard that provides a visual interface for everything the [admin API](../reference/admin-api) offers: managing lexicons, viewing indexed records, and monitoring backfill jobs. It runs as a separate Next.js application alongside the Rust backend.
44+55+## Logging in for the first time
66+77+The dashboard uses AT Protocol OAuth via AIP. If no admins exist in the database yet, the first authenticated request to any admin endpoint automatically bootstraps that user as an admin.
88+99+## Adding a lexicon
1010+1111+Navigate to **Lexicons > Add Lexicon** and choose **Local** or **Network**.
1212+1313+**Local** lexicons are defined by you. The editor shows two side-by-side panels (stacked on mobile):
1414+1515+- **Lexicon JSON** (left): define your lexicon schema
1616+- **Lua Script** (right): write the handler for query/procedure types
1717+1818+The Lua panel only appears when the lexicon's `defs.main.type` is `query` or `procedure`. For record-type lexicons, only the JSON panel is shown.
1919+2020+A default Lua script is auto-generated when you first set the type to query or procedure. The template updates automatically when the type changes, but once you manually edit the script your changes are preserved.
2121+2222+Toggle **Enable backfill** to index historical records when uploading a record-type lexicon.
2323+2424+**Network** lexicons are fetched from the AT Protocol network. Enter an NSID (e.g. `xyz.statusphere.status`) and HappyView resolves the schema automatically. If found, the lexicon JSON is displayed in a read-only editor. Click **Add** to track it. Network lexicons are kept up to date via Tap. See [Lexicons - Network lexicons](../guides/lexicons#network-lexicons) for how resolution works.
2525+2626+### JSON editor
2727+2828+The JSON editor provides real-time validation against the AT Protocol Lexicon v1 schema:
2929+3030+- Validation for Lexicon format
3131+- Auto-complete for definition types (`record`, `query`, `procedure`, `subscription`), property types (`string`, `integer`, `boolean`, `ref`, `union`, `blob`, `cid-link`, etc.), and schema structure (`defs`, `main`, `properties`, `required`)
3232+- Enforces the required top-level shape: `lexicon`, `id`, and `defs.main`
3333+3434+### Lua editor
3535+3636+The Lua editor provides context-aware code completions, including suggestions for the `Record`, `db`, `input`, and `params` APIs as well as Lua keywords, builtins, and standard library functions. It also offers snippet templates for common constructs like `if`, `for`, and `function`.
3737+3838+See [Lua Scripting](../guides/scripting) for the full runtime reference and examples.
+49
docs/getting-started/deployment/docker.md
···11+# Local Development with Docker
22+33+This guide runs the full HappyView stack locally using Docker Compose: Postgres, [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap), HappyView, and the web dashboard.
44+55+## Prerequisites
66+77+- [Docker](https://docs.docker.com/get-docker/) and Docker Compose
88+- An [AIP](https://github.com/graze-social/aip) instance for OAuth. The Docker Compose config points at the public AIP instance at `aip.gamesgamesgamesgames.games` by default.
99+1010+:::warning
1111+This public AIP instance is provided for development convenience only. Production deployments should run their own AIP instance or risk being blocked. See the [AIP documentation](https://github.com/graze-social/aip) for setup.
1212+:::
1313+1414+## 1. Clone and configure
1515+1616+```sh
1717+git clone https://github.com/graze-social/happyview.git
1818+cd happyview
1919+cp .env.example .env
2020+```
2121+2222+Set `TAP_ADMIN_PASSWORD` in your `.env`. This shared secret is used by both Tap and HappyView:
2323+2424+```sh
2525+TAP_ADMIN_PASSWORD=your-secret-here
2626+```
2727+2828+The `docker-compose.yml` configures everything else (database URLs, service connections) automatically.
2929+3030+## 2. Start the stack
3131+3232+```sh
3333+docker compose up
3434+```
3535+3636+This starts:
3737+3838+| Service | Port | Description |
3939+| ------------- | ---- | ---------------------------------------------------- |
4040+| **postgres** | 5432 | PostgreSQL 17 (databases for both HappyView and Tap) |
4141+| **tap** | 2480 | Firehose consumer, backfill worker |
4242+| **happyview** | 3000 | HappyView API server |
4343+| **web** | 3001 | Next.js dashboard |
4444+4545+HappyView runs migrations automatically on startup. The first build will take a few minutes while Rust compiles.
4646+4747+## Next steps
4848+4949+Your HappyView stack is running. Follow the [Statusphere tutorial](../../tutorials/statusphere) to upload lexicons, add custom query logic, and start indexing records from the network.
+55
docs/getting-started/deployment/other.md
···11+# Local Development from Source
22+33+This guide runs HappyView directly with `cargo run`, with you managing Postgres, AIP, and Tap separately. If you'd rather use Docker Compose to run everything together, see [Local Development with Docker](docker).
44+55+## Prerequisites
66+77+- Rust (stable)
88+- PostgreSQL 17+
99+- A running [AIP](https://github.com/graze-social/aip) instance (handles OAuth). See the [AIP documentation](https://github.com/graze-social/aip) for setup.
1010+- A running [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap) instance (delivers real-time records and handles backfill). See the [Tap documentation](https://github.com/bluesky-social/indigo/tree/main/cmd/tap) for setup.
1111+1212+## 1. Clone and configure
1313+1414+```sh
1515+git clone https://github.com/graze-social/happyview.git
1616+cd happyview
1717+cp .env.example .env
1818+```
1919+2020+Edit `.env` to point at your running services:
2121+2222+```sh
2323+DATABASE_URL=postgres://happyview:happyview@localhost/happyview
2424+AIP_URL=http://localhost:8080
2525+TAP_URL=http://localhost:2480
2626+TAP_ADMIN_PASSWORD=your-secret-here
2727+```
2828+2929+See [Configuration](../configuration) for all available variables.
3030+3131+## 2. Create the database
3232+3333+```sh
3434+createdb happyview
3535+```
3636+3737+Or if using a Postgres user with a password:
3838+3939+```sh
4040+psql -c "CREATE DATABASE happyview;" -U postgres
4141+```
4242+4343+HappyView runs migrations automatically on startup, so no manual migration step is needed.
4444+4545+## 3. Start HappyView
4646+4747+```sh
4848+cargo run
4949+```
5050+5151+HappyView starts on port 3000 (configurable via the `PORT` environment variable).
5252+5353+## Next steps
5454+5555+Your HappyView instance is running. Follow the [Statusphere tutorial](../../tutorials/statusphere) to upload lexicons, add custom query logic, and start indexing records from the network.
+20
docs/getting-started/deployment/railway.md
···11+# Deploy on Railway
22+33+The fastest way to get HappyView running is with Railway. This template deploys HappyView, [AIP](https://github.com/graze-social/aip) (OAuth provider), [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap) (real-time data and backfill), and Postgres with a single click:
44+55+[](https://railway.com/deploy/I1jvZl?referralCode=0QOgj_)
66+77+## Required configuration
88+99+After deploying the template, you'll need to configure a few things before the stack works properly:
1010+1111+1. **Set your admin DID.** In the AIP service variables, set `ADMIN_DIDS` to your AT Protocol DID (e.g. `did:plc:abc123...`). You can find your DID by looking up your handle on [Internect](https://internect.info/).
1212+1313+2. **Generate AIP signing keys.** The `OAUTH_SIGNING_KEYS` and `ATPROTO_OAUTH_SIGNING_KEYS` variables require multibase-encoded P-256 private keys. See the [AIP Signing Keys documentation](https://github.com/graze-social/aip/blob/main/CONFIGURATION.md#signing-keys) for generation instructions.
1414+1515+3. **Assign public domains.** In the Railway dashboard, add a public domain to both the HappyView and AIP services. The services need publicly accessible URLs to handle OAuth callbacks and XRPC requests.
1616+ :::note
1717+ Your instances can use custom domains or Railway's generated URLs with no additional configuration. The domains are injected automatically to the containers.
1818+ :::
1919+2020+4. Access your HappyView dashboard at the instance's public URL.
+62
docs/getting-started/quickstart.md
···11+# Quickstart
22+33+This page walks you through the fastest path to a working HappyView instance. By the end, you'll have an AppView that indexes records from the AT Protocol network and serves XRPC endpoints.
44+55+## 1. Deploy HappyView
66+77+Pick whichever option fits your situation:
88+99+| Option | Best for |
1010+|--------|----------|
1111+| [**Railway**](deployment/railway) | Fastest path — one-click deploy of the full stack (HappyView + AIP + Tap + Postgres) |
1212+| [**Docker Compose**](deployment/docker) | Local development with the full stack running in containers |
1313+| [**From source**](deployment/other) | Running HappyView with `cargo run` and managing dependencies yourself |
1414+1515+If you're just trying HappyView for the first time, start with Railway.
1616+1717+## 2. Log in to the dashboard
1818+1919+Open your HappyView instance in a browser. The built-in [dashboard](dashboard) is served at the root URL.
2020+2121+Click **Log in** and authenticate with your AT Protocol identity. On a fresh deployment with no admins configured, the first authenticated request to any admin endpoint automatically bootstraps that user as an admin.
2222+2323+## 3. Add your first lexicon
2424+2525+Lexicons tell HappyView what data to index and what endpoints to serve. The quickest way to get started is to add one from the network:
2626+2727+1. In the dashboard, go to **Lexicons > Add Lexicon > Network**
2828+2. Enter an NSID (e.g. `xyz.statusphere.status`)
2929+3. HappyView resolves the schema from the AT Protocol network and shows a preview
3030+4. Click **Add**
3131+3232+HappyView immediately starts indexing records for that collection. A backfill job is created to fetch historical records, and new records stream in via Tap in real time.
3333+3434+You can also upload lexicons manually via the dashboard or the [admin API](../reference/admin-api). See [Lexicons](../guides/lexicons) for the full details.
3535+3636+## 4. Verify records are being indexed
3737+3838+Go to the **Dashboard** home page. The stat cards show the total record count and a breakdown by collection. You can also browse indexed records on the **Records** page.
3939+4040+To check backfill progress, go to the **Backfill** page. The Tap stats cards show how many repos and records Tap has processed.
4141+4242+## 5. Query your data
4343+4444+Once you have a record lexicon indexed, add a query lexicon to expose a read endpoint. Go to **Lexicons > Add Lexicon > Local** and create a query lexicon with `target_collection` set to your record collection's NSID.
4545+4646+Without a Lua script, HappyView generates a default query endpoint that supports `limit`, `cursor`, `did`, and `uri` parameters:
4747+4848+```
4949+GET /xrpc/xyz.statusphere.listStatuses?limit=5
5050+```
5151+5252+For custom query logic, attach a [Lua script](../guides/scripting).
5353+5454+## Next steps
5555+5656+You now have a working AppView. Here's where to go from here:
5757+5858+- [**Statusphere tutorial**](../tutorials/statusphere): end-to-end walkthrough building a complete AppView with record, query, and procedure lexicons
5959+- [**Lexicons guide**](../guides/lexicons): target collections, backfill flag, network lexicons
6060+- [**Lua Scripting**](../guides/scripting): custom query and procedure logic
6161+- [**Configuration**](configuration): environment variables and tuning
6262+- [**Authentication**](authentication): how OAuth works and how to get API tokens
+33
docs/guides/backfill.md
···11+# Backfill
22+33+When you add a new record-type lexicon, HappyView starts indexing new records from that moment via [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap). But what about records that already exist on the network? That's what backfill does: HappyView discovers repos via the relay and delegates the actual record fetching to Tap.
44+55+## When backfill runs
66+77+- **Automatically** when a record-type lexicon is uploaded with `backfill: true` (the default). See [Lexicons - Backfill flag](lexicons#backfill-flag).
88+- **Manually** via `POST /admin/backfill` or the [dashboard](../getting-started/dashboard). You can scope a manual backfill to a specific collection, a specific DID, or both.
99+1010+See the [admin API](../reference/admin-api#backfill) for endpoint details.
1111+1212+## How it works
1313+1414+1. **Determine target collections**: uses the specified collection, or all record lexicons with `backfill: true`
1515+2. **Discover DIDs**: HappyView calls the relay's `com.atproto.sync.listReposByCollection` to find repos that contain records for each target collection (paginated, 1000 per page)
1616+3. **Delegate to Tap**: HappyView sends discovered DIDs to Tap in batches of 1000 via its `/repos/add` endpoint
1717+4. **Tap fetches records**: Tap handles the actual record fetching from each user's PDS and delivers them to HappyView via the WebSocket channel
1818+1919+## Job lifecycle
2020+2121+HappyView marks a backfill job as "completed" once it finishes discovering repos and handing DIDs off to Tap (steps 1-3). This does **not** mean Tap has finished processing all the records. Tap works through them asynchronously after the handoff.
2222+2323+To see whether Tap is still working through the backlog, check the Tap stats on the dashboard's Backfill page or via `GET /admin/tap/stats`. The **outbox buffer** indicates how many events are still queued for delivery; a high number means Tap is actively processing.
2424+2525+## Re-running backfills
2626+2727+Backfill jobs are idempotent. Running a backfill for a collection that's already been backfilled will re-discover repos and send them to Tap again. Tap deduplicates repos it already knows about, so re-running a backfill is safe and useful for catching repos that were added to the network since the last run.
2828+2929+## Next steps
3030+3131+- [Lexicons](lexicons#backfill-flag): Control whether lexicons trigger backfill on upload
3232+- [Admin API](../reference/admin-api#backfill): Full reference for backfill endpoints
3333+- [Admin API - Tap Stats](../reference/admin-api#tap-stats): Monitor Tap's processing progress
+82
docs/guides/lexicons.md
···11+# Lexicons
22+33+Lexicons are the core building block of HappyView. They're [AT Protocol schema definitions](https://atproto.com/specs/lexicon) that describe your data model, and HappyView uses them to decide which records to index from the network and what XRPC endpoints to serve.
44+55+You don't write route handlers or database queries; you upload a lexicon and HappyView generates the infrastructure from it. There are two ways to add lexicons: uploading them via the [admin API](../reference/admin-api#lexicons) or [dashboard](../getting-started/dashboard), or fetching them directly from the AT Protocol network via [DNS authority resolution](#network-lexicons).
66+77+## Supported lexicon types
88+99+| Type | Effect |
1010+| ------------- | ------------------------------------------------------------------------------ |
1111+| `record` | Syncs the collection filter to Tap and indexes records into Postgres |
1212+| `query` | Registers a `GET /xrpc/{nsid}` endpoint that queries indexed records |
1313+| `procedure` | Registers a `POST /xrpc/{nsid}` endpoint that proxies writes to the user's PDS |
1414+| `definitions` | Stored but does not generate routes or subscriptions |
1515+1616+A typical setup has three lexicons working together: a **record** lexicon that defines the data and triggers indexing, a **query** lexicon that exposes a read endpoint, and a **procedure** lexicon that exposes a write endpoint. The [Statusphere tutorial](../tutorials/statusphere) walks through this pattern end-to-end.
1717+1818+## Target collection
1919+2020+Query and procedure lexicons don't store data themselves. They operate on records stored by a record-type lexicon. The `target_collection` field tells HappyView which record collection to read from or write to. Without it, default queries and procedures won't know which DB records to operate on.
2121+2222+For example, a query lexicon `xyz.statusphere.listStatuses` would set `target_collection` to `xyz.statusphere.status` to read from that record collection.
2323+2424+See the [admin API](../reference/admin-api#upload--upsert-a-lexicon) for how to set `target_collection` when uploading.
2525+2626+:::note
2727+The `target_collection` is available in Lua scripts as the `collection` global, but it is not required if your endpoint uses a Lua script.
2828+:::
2929+3030+## Backfill flag
3131+3232+When uploading a record-type lexicon, HappyView automatically creates a backfill job to discover existing records. If you only want to index new records going forward, you can set `backfill` to `false`.
3333+3434+## Tap collection filters
3535+3636+When record-type lexicons change (uploaded or deleted), HappyView automatically syncs the updated collection filter to Tap. HappyView always includes `com.atproto.lexicon.schema` in the filter to track network lexicon updates.
3737+3838+## Network lexicons
3939+4040+If a lexicon has already been published, you don't need to upload the JSON manually. Point HappyView at the NSID and it fetches the lexicon directly from the network. Network lexicons are kept updated automatically via Tap. If the publisher updates their schema, your instance will pick up the change.
4141+4242+### NSID authority resolution
4343+4444+Lexicons are stored as records themselves with the `com.atproto.lexicon.schema` NSID and the rkey set to the lexicon's NSID. To find which repo holds a lexicon, HappyView resolves the NSID's authority:
4545+4646+1. Extract the authority from the NSID (all segments except the last). For example, `xyz.statusphere.status` has authority `xyz.statusphere`.
4747+2. Reverse the authority segments to form a domain: `statusphere.xyz`.
4848+3. Look up the DNS TXT record at `_lexicon.{domain}` (e.g. `_lexicon.statusphere.xyz`).
4949+4. Parse the TXT record for a `did=<DID>` value.
5050+5. Resolve the DID to a PDS endpoint via the PLC directory.
5151+5252+:::note
5353+The spec states that resolution must be **non-hierarchical**. Each authority requires its own explicit TXT record. If you have multiple levels of authority (e.g. `xyz.statusphere.status` and `xyz.statusphere.actor.profile`), each level must have an explicit TXT record.
5454+:::
5555+5656+### Fetching
5757+5858+Once the authority DID and PDS endpoint are known, HappyView calls `com.atproto.repo.getRecord` with:
5959+6060+- `repo` = the authority DID
6161+- `collection` = `com.atproto.lexicon.schema`
6262+- `rkey` = the NSID
6363+6464+The `value` field of the response is the raw lexicon JSON.
6565+6666+### Live updates via Tap
6767+6868+Tap always subscribes to `com.atproto.lexicon.schema` alongside the dynamic record collections. When a record event arrives:
6969+7070+- **create/update**: If the event's DID and rkey match a tracked network lexicon (`authority_did` and `nsid`), the lexicon is parsed, upserted into the `lexicons` table and in-memory registry, and collection filters are updated if it's a record type.
7171+- **delete**: The lexicon is removed from the `lexicons` table and registry.
7272+7373+### Startup re-fetch
7474+7575+On every startup, HappyView re-fetches all network lexicons from their respective PDSes. This ensures consistency even if events were missed while offline. Failures are logged as warnings but don't block startup.
7676+7777+## Next steps
7878+7979+- [Lua Scripting](scripting): Add custom query and procedure logic to your endpoints
8080+- [XRPC API](../reference/xrpc-api): Understand how the generated endpoints behave
8181+- [Backfill](backfill): Learn how historical records are indexed
8282+- [Admin API](../reference/admin-api): Full reference for lexicon management endpoints
+314
docs/guides/scripting.md
···11+# Lua Scripting
22+33+Without Lua scripts, HappyView's query endpoints return raw records and procedure endpoints proxy simple creates and updates. Lua scripts let you go much further:
44+55+- Add filtering logic
66+- Transform responses
77+- Validate input
88+- Compose multi-record operations
99+- Build entirely custom behavior
1010+1111+Scripts are attached to query and procedure lexicons and run in a sandboxed Lua VM with access to the [Record API](#record-api), a [read-only database API](#database-api), and a set of [context globals](#context-globals).
1212+1313+## Script structure
1414+1515+Every script must define a `handle()` function. HappyView calls it when the XRPC endpoint is hit and returns its result as JSON to the client.
1616+1717+```lua
1818+function handle()
1919+ -- your logic here
2020+ return { key = "value" }
2121+end
2222+```
2323+2424+You can define helper functions and variables outside `handle()`. They're evaluated once when the script loads, then `handle()` is called per request.
2525+2626+## Sandbox
2727+2828+Scripts run in a restricted environment. The following standard Lua modules are **removed** and unavailable:
2929+3030+`os`, `io`, `debug`, `package`, `require`, `dofile`, `loadfile`, `load`, `collectgarbage`
3131+3232+An instruction limit of 1,000,000 prevents infinite loops. Exceeding it terminates the script with an error.
3333+3434+## Context globals
3535+3636+These globals are set automatically before `handle()` is called.
3737+3838+### Procedure globals
3939+4040+| Global | Type | Description |
4141+| ------------ | ------ | ------------------------------------------------------- |
4242+| `method` | string | The XRPC method name (e.g. `xyz.statusphere.setStatus`) |
4343+| `input` | table | Parsed JSON request body |
4444+| `caller_did` | string | DID of the authenticated user |
4545+| `collection` | string | Target collection NSID |
4646+4747+### Query globals
4848+4949+| Global | Type | Description |
5050+| ------------ | ------ | ------------------------------------------------ |
5151+| `method` | string | The XRPC method name |
5252+| `params` | table | Query string parameters (all values are strings) |
5353+| `collection` | string | Target collection NSID |
5454+5555+Queries are unauthenticated: there is no `caller_did` or `input`.
5656+5757+## Utility globals
5858+5959+Available in both queries and procedures:
6060+6161+| Function | Returns | Description |
6262+| -------------- | ------- | ------------------------------------------------------------------- |
6363+| `now()` | string | Current UTC timestamp in ISO 8601 format |
6464+| `log(message)` | — | Log a message (appears in server logs at debug level) |
6565+| `TID()` | string | Generate a fresh AT Protocol TID (13-character sortable identifier) |
6666+6767+## Record API
6868+6969+The `Record` API is only available in **procedure** scripts. It handles creating, updating, loading, and deleting AT Protocol records. Writes are proxied to the caller's PDS and indexed locally.
7070+7171+### Constructor
7272+7373+```lua
7474+local r = Record("xyz.statusphere.status", { status = "\ud83d\ude0a", createdAt = now() })
7575+```
7676+7777+Creates a new record instance for the given collection. The optional second argument sets initial field values. The record's `_key_type` is automatically set from the lexicon's `key` definition. Default values from the schema are populated for any missing fields.
7878+7979+### Static methods
8080+8181+```lua
8282+-- Save multiple records in parallel
8383+Record.save_all({ record1, record2, record3 })
8484+8585+-- Load a record from the local database by AT URI
8686+local r = Record.load("at://did:plc:abc/xyz.statusphere.status/abc123")
8787+-- Returns nil if not found
8888+8989+-- Load multiple records in parallel
9090+local records = Record.load_all({ uri1, uri2 })
9191+-- Returns nil entries for URIs not found
9292+```
9393+9494+### Instance methods
9595+9696+```lua
9797+-- Save (creates or updates depending on whether _uri is set)
9898+r:save()
9999+100100+-- Delete from PDS and local database
101101+r:delete()
102102+103103+-- Set the record key type (tid, any, nsid, or literal:*)
104104+r:set_key_type("tid")
105105+106106+-- Set a specific record key
107107+r:set_rkey("my-key")
108108+109109+-- Auto-generate a record key based on _key_type
110110+local key = r:generate_rkey()
111111+```
112112+113113+**Key type behavior for `generate_rkey()`:**
114114+115115+| Key type | Generated rkey |
116116+| --------------- | --------------------------------- |
117117+| `tid` | Sortable timestamp-based ID |
118118+| `any` | Same as `tid` |
119119+| `literal:value` | The literal value after the colon |
120120+| `nsid` | Error — use `set_rkey()` instead |
121121+122122+### Instance fields
123123+124124+These fields are set automatically and are read-only (writes raise an error):
125125+126126+| Field | Type | Description |
127127+| ------------- | ------- | ----------------------------------------------------------- |
128128+| `_uri` | string? | AT URI — set after `save()`, cleared after `delete()` |
129129+| `_cid` | string? | Content hash — set after `save()`, cleared after `delete()` |
130130+| `_key_type` | string? | Record key type from the lexicon definition |
131131+| `_rkey` | string? | Record key — set via `set_rkey()` or `generate_rkey()` |
132132+| `_collection` | string | Collection NSID (always set) |
133133+| `_schema` | table? | Schema definition from the lexicon (used for validation) |
134134+135135+### Schema validation
136136+137137+When a record has a schema (loaded from the lexicon):
138138+139139+- **On save:** required fields are checked, and missing required fields raise an error
140140+- **On construction:** default values from schema properties are auto-populated
141141+- **On save:** only fields defined in the schema's `properties` are sent to the PDS
142142+143143+### Save behavior
144144+145145+`r:save()` auto-detects create vs update:
146146+147147+- If `_uri` is nil → calls `createRecord` on the PDS
148148+- If `_uri` is set → calls `putRecord` on the PDS
149149+150150+After a successful save, `_uri` and `_cid` are updated on the record instance.
151151+152152+## Database API
153153+154154+The `db` table provides read-only access to indexed records. Available in both queries and procedures.
155155+156156+### db.query
157157+158158+```lua
159159+local result = db.query({
160160+ collection = "xyz.statusphere.status", -- required
161161+ did = "did:plc:abc", -- optional: filter by DID
162162+ limit = 20, -- optional: max 100, default 20
163163+ offset = 0, -- optional: for pagination
164164+})
165165+166166+-- result.records — array of record tables (each includes a "uri" field)
167167+-- result.cursor — present when more records exist
168168+```
169169+170170+### db.get
171171+172172+```lua
173173+local record = db.get("at://did:plc:abc/xyz.statusphere.status/abc123")
174174+-- Returns the record table or nil
175175+-- The returned table includes a "uri" field
176176+```
177177+178178+### db.count
179179+180180+```lua
181181+local n = db.count("xyz.statusphere.status")
182182+local n = db.count("xyz.statusphere.status", "did:plc:abc") -- filter by DID
183183+```
184184+185185+## Standard libraries
186186+187187+The following Lua 5.4 standard library modules are available:
188188+189189+<details>
190190+<summary>
191191+`string`
192192+</summary>
193193+- [`byte`](https://lua.org/manual/5.4/manual.html#pdf-string.byte)
194194+- [`char`](https://lua.org/manual/5.4/manual.html#pdf-string.char)
195195+- [`find`](https://lua.org/manual/5.4/manual.html#pdf-string.find)
196196+- [`format`](https://lua.org/manual/5.4/manual.html#pdf-string.format)
197197+- [`gmatch`](https://lua.org/manual/5.4/manual.html#pdf-string.gmatch)
198198+- [`gsub`](https://lua.org/manual/5.4/manual.html#pdf-string.gsub)
199199+- [`len`](https://lua.org/manual/5.4/manual.html#pdf-string.len)
200200+- [`lower`](https://lua.org/manual/5.4/manual.html#pdf-string.lower)
201201+- [`match`](https://lua.org/manual/5.4/manual.html#pdf-string.match)
202202+- [`rep`](https://lua.org/manual/5.4/manual.html#pdf-string.rep)
203203+- [`reverse`](https://lua.org/manual/5.4/manual.html#pdf-string.reverse)
204204+- [`sub`](https://lua.org/manual/5.4/manual.html#pdf-string.sub)
205205+- [`upper`](https://lua.org/manual/5.4/manual.html#pdf-string.upper)
206206+</details>
207207+208208+<details>
209209+<summary>
210210+`table`
211211+</summary>
212212+- [`concat`](https://lua.org/manual/5.4/manual.html#pdf-table.concat)
213213+- [`insert`](https://lua.org/manual/5.4/manual.html#pdf-table.insert)
214214+- [`remove`](https://lua.org/manual/5.4/manual.html#pdf-table.remove)
215215+- [`sort`](https://lua.org/manual/5.4/manual.html#pdf-table.sort)
216216+- [`unpack`](https://lua.org/manual/5.4/manual.html#pdf-table.unpack)
217217+</details>
218218+219219+<details>
220220+<summary>
221221+`math`
222222+</summary>
223223+- [`abs`](https://lua.org/manual/5.4/manual.html#pdf-math.abs)
224224+- [`ceil`](https://lua.org/manual/5.4/manual.html#pdf-math.ceil)
225225+- [`floor`](https://lua.org/manual/5.4/manual.html#pdf-math.floor)
226226+- [`max`](https://lua.org/manual/5.4/manual.html#pdf-math.max)
227227+- [`min`](https://lua.org/manual/5.4/manual.html#pdf-math.min)
228228+- [`random`](https://lua.org/manual/5.4/manual.html#pdf-math.random)
229229+- [`sqrt`](https://lua.org/manual/5.4/manual.html#pdf-math.sqrt)
230230+- [`huge`](https://lua.org/manual/5.4/manual.html#pdf-math.huge)
231231+- [`pi`](https://lua.org/manual/5.4/manual.html#pdf-math.pi)
232232+</details>
233233+234234+<details>
235235+<summary>
236236+Standard builtins
237237+</summary>
238238+- [`print`](https://lua.org/manual/5.4/manual.html#pdf-print)
239239+- [`tostring`](https://lua.org/manual/5.4/manual.html#pdf-tostring)
240240+- [`tonumber`](https://lua.org/manual/5.4/manual.html#pdf-tonumber)
241241+- [`type`](https://lua.org/manual/5.4/manual.html#pdf-type)
242242+- [`pairs`](https://lua.org/manual/5.4/manual.html#pdf-pairs)
243243+- [`ipairs`](https://lua.org/manual/5.4/manual.html#pdf-ipairs)
244244+- [`next`](https://lua.org/manual/5.4/manual.html#pdf-next)
245245+- [`select`](https://lua.org/manual/5.4/manual.html#pdf-select)
246246+- [`unpack`](https://lua.org/manual/5.4/manual.html#pdf-table.unpack)
247247+- [`error`](https://lua.org/manual/5.4/manual.html#pdf-error)
248248+- [`pcall`](https://lua.org/manual/5.4/manual.html#pdf-pcall)
249249+- [`xpcall`](https://lua.org/manual/5.4/manual.html#pdf-xpcall)
250250+- [`assert`](https://lua.org/manual/5.4/manual.html#pdf-assert)
251251+- [`setmetatable`](https://lua.org/manual/5.4/manual.html#pdf-setmetatable)
252252+- [`getmetatable`](https://lua.org/manual/5.4/manual.html#pdf-getmetatable)
253253+- [`rawget`](https://lua.org/manual/5.4/manual.html#pdf-rawget)
254254+- [`rawset`](https://lua.org/manual/5.4/manual.html#pdf-rawset)
255255+- [`rawequal`](https://lua.org/manual/5.4/manual.html#pdf-rawequal)
256256+</details>
257257+258258+## Debugging
259259+260260+### Logging
261261+262262+Use `log()` to trace script execution. Output appears in the server logs at **debug** level with the field `lua_log`:
263263+264264+```lua
265265+function handle()
266266+ log("handle called with params: " .. tostring(params.limit))
267267+ local result = db.query({ collection = collection, limit = params.limit })
268268+ log("query returned " .. #result.records .. " records")
269269+ return result
270270+end
271271+```
272272+273273+To see log output, make sure your `RUST_LOG` environment variable includes debug level for HappyView (the default `happyview=debug` works). See [Configuration](../getting-started/configuration).
274274+275275+### Error messages
276276+277277+When a script fails, the client receives a generic `500` response:
278278+279279+- `{"error": "script execution failed"}`: covers syntax errors, runtime errors, missing `handle()` function, and errors raised with `error()`
280280+- `{"error": "script exceeded execution time limit"}`: the script hit the 1,000,000 instruction limit
281281+282282+The **full error message** is logged server-side at error level. Check the server logs to see the actual Lua error, including line numbers and stack traces.
283283+284284+### Common mistakes
285285+286286+- **Missing `handle()` function**: Every script must define a global `handle()` function. If it's missing or misspelled, the script fails silently with "script execution failed".
287287+- **Calling `error()` for expected conditions**: Lua's `error()` triggers a 500 response. For expected conditions like "record not found", return a structured error response instead: `return { error = "not found" }`.
288288+- **Infinite loops**: The sandbox enforces a 1,000,000 instruction limit. If your script processes large data sets, paginate with `db.query()` limits instead of loading everything at once.
289289+- **Forgetting `params` values are strings**: All query string parameters arrive as strings. Use `tonumber(params.limit)` if you need a number.
290290+291291+## Example scripts
292292+293293+See the example script references for complete, ready-to-use scripts:
294294+295295+**Queries:**
296296+- [Get a record](../reference/scripts/get-record) — fetch a single record by AT URI
297297+- [Paginated list](../reference/scripts/paginated-list) — list records with cursor-based pagination and DID filtering
298298+- [List or fetch](../reference/scripts/list-or-fetch) — combined single-record lookup and paginated listing
299299+- [Expanded query](../reference/scripts/expanded-query) — list statuses with user profiles in a single response
300300+301301+**Procedures:**
302302+- [Create a record](../reference/scripts/create-record) — simple write that saves input as a record
303303+- [Upsert a record](../reference/scripts/upsert-record) — create or update using a deterministic rkey
304304+- [Update or delete](../reference/scripts/update-or-delete) — single endpoint handling create, update, and delete
305305+- [Batch save](../reference/scripts/batch-save) — create multiple records in parallel with `Record.save_all()`
306306+- [Sidecar records](../reference/scripts/sidecar-records) — create linked records across collections with a shared rkey
307307+- [Cascading delete](../reference/scripts/cascading-delete) — delete a record and all related records
308308+- [Complex mutations](../reference/scripts/complex-mutations) — load, transform, and save a record with multiple field changes
309309+310310+## Next steps
311311+312312+- [Lexicons](lexicons): Understand how record, query, and procedure lexicons work together
313313+- [XRPC API](../reference/xrpc-api): See how endpoints behave with and without Lua scripts
314314+- [Dashboard](../getting-started/dashboard#lua-editor): Use the web editor with context-aware completions
-115
docs/lexicons.md
···11-# Lexicons
22-33-Lexicons are ATProto schema definitions that tell HappyView which records to index and what XRPC endpoints to serve. HappyView supports uploading lexicons at runtime via the admin API.
44-55-## Supported lexicon types
66-77-| Type | Effect |
88-|------|--------|
99-| `record` | Subscribes to Jetstream for that collection and indexes records into Postgres |
1010-| `query` | Registers a `GET /xrpc/{nsid}` endpoint that queries indexed records |
1111-| `procedure` | Registers a `POST /xrpc/{nsid}` endpoint that proxies writes to the user's PDS |
1212-| `definitions` | Stored but does not generate routes or subscriptions |
1313-1414-## Uploading a lexicon
1515-1616-```sh
1717-curl -X POST http://localhost:3000/admin/lexicons \
1818- -H "Authorization: Bearer $TOKEN" \
1919- -H "Content-Type: application/json" \
2020- -d '{
2121- "lexicon_json": {
2222- "lexicon": 1,
2323- "id": "games.gamesgamesgamesgames.game",
2424- "defs": {
2525- "main": {
2626- "type": "record",
2727- "key": "tid",
2828- "record": {
2929- "type": "object",
3030- "properties": {
3131- "title": { "type": "string" }
3232- }
3333- }
3434- }
3535- }
3636- },
3737- "backfill": true
3838- }'
3939-```
4040-4141-Re-uploading the same lexicon ID increments its revision number.
4242-4343-## The `target_collection` field
4444-4545-Query and procedure lexicons need to know which record collection they operate on. Set `target_collection` to the NSID of the record lexicon:
4646-4747-```sh
4848-curl -X POST http://localhost:3000/admin/lexicons \
4949- -H "Authorization: Bearer $TOKEN" \
5050- -H "Content-Type: application/json" \
5151- -d '{
5252- "lexicon_json": {
5353- "lexicon": 1,
5454- "id": "games.gamesgamesgamesgames.listGames",
5555- "defs": {
5656- "main": {
5757- "type": "query",
5858- "parameters": {
5959- "type": "params",
6060- "properties": {
6161- "limit": { "type": "integer" }
6262- }
6363- },
6464- "output": { "encoding": "application/json" }
6565- }
6666- }
6767- },
6868- "target_collection": "games.gamesgamesgamesgames.game"
6969- }'
7070-```
7171-7272-Without `target_collection`, queries and procedures won't know which DB records to read from.
7373-7474-## The `backfill` flag
7575-7676-When `backfill` is `true` (the default), uploading a record-type lexicon triggers a backfill job that discovers existing repos via the relay and fetches historical records from their PDSes.
7777-7878-Set `backfill: false` if you only want to index new records going forward.
7979-8080-## Jetstream collection filters
8181-8282-When record-type lexicons change (uploaded or deleted), HappyView automatically reconnects to Jetstream with an updated collection filter. If no record lexicons exist, the Jetstream listener idles without connecting.
8383-8484-## Example: full setup
8585-8686-Upload the record lexicon, then a query and a procedure that target it:
8787-8888-```sh
8989-# 1. Record lexicon (triggers Jetstream subscription + backfill)
9090-curl -X POST http://localhost:3000/admin/lexicons \
9191- -H "Authorization: Bearer $TOKEN" \
9292- -H "Content-Type: application/json" \
9393- -d '{
9494- "lexicon_json": { "lexicon": 1, "id": "games.gamesgamesgamesgames.game", "defs": { "main": { "type": "record", "key": "tid", "record": { "type": "object", "properties": { "title": { "type": "string" } } } } } },
9595- "backfill": true
9696- }'
9797-9898-# 2. Query lexicon
9999-curl -X POST http://localhost:3000/admin/lexicons \
100100- -H "Authorization: Bearer $TOKEN" \
101101- -H "Content-Type: application/json" \
102102- -d '{
103103- "lexicon_json": { "lexicon": 1, "id": "games.gamesgamesgamesgames.listGames", "defs": { "main": { "type": "query", "output": { "encoding": "application/json" } } } },
104104- "target_collection": "games.gamesgamesgamesgames.game"
105105- }'
106106-107107-# 3. Procedure lexicon
108108-curl -X POST http://localhost:3000/admin/lexicons \
109109- -H "Authorization: Bearer $TOKEN" \
110110- -H "Content-Type: application/json" \
111111- -d '{
112112- "lexicon_json": { "lexicon": 1, "id": "games.gamesgamesgamesgames.createGame", "defs": { "main": { "type": "procedure", "input": { "encoding": "application/json" }, "output": { "encoding": "application/json" } } } },
113113- "target_collection": "games.gamesgamesgamesgames.game"
114114- }'
115115-```
-57
docs/network-lexicons.md
···11-# Network Lexicons
22-33-Network lexicons are lexicon definitions that HappyView fetches directly from the ATProto network rather than being uploaded manually via the admin API. An admin specifies an NSID, HappyView resolves the authority's repo, fetches the lexicon record, and keeps it updated via Jetstream.
44-55-## How it works
66-77-### NSID authority resolution
88-99-Lexicon records live in repos as `com.atproto.lexicon.schema` with the rkey set to the NSID. To find which repo holds a lexicon, HappyView resolves the NSID's authority:
1010-1111-1. Extract the authority from the NSID (all segments except the last). For example, `games.gamesgamesgamesgames.game` has authority `games.gamesgamesgamesgames`.
1212-2. Reverse the authority segments to form a domain: `gamesgamesgamesgames.games`.
1313-3. Look up the DNS TXT record at `_lexicon.{domain}` (e.g. `_lexicon.gamesgamesgamesgames.games`).
1414-4. Parse the TXT record for a `did=<DID>` value.
1515-5. Resolve the DID to a PDS endpoint via the PLC directory.
1616-1717-Resolution is **non-hierarchical** --- each authority requires its own explicit TXT record.
1818-1919-### Fetching
2020-2121-Once the authority DID and PDS endpoint are known, HappyView calls `com.atproto.repo.getRecord` with:
2222-- `repo` = the authority DID
2323-- `collection` = `com.atproto.lexicon.schema`
2424-- `rkey` = the NSID
2525-2626-The `value` field of the response is the raw lexicon JSON.
2727-2828-### Live updates via Jetstream
2929-3030-Jetstream always subscribes to `com.atproto.lexicon.schema` alongside the dynamic record collections. When a commit event arrives:
3131-3232-- **create/update**: If the event's DID and rkey match a tracked network lexicon (`authority_did` and `nsid`), the lexicon is parsed, upserted into the `lexicons` table and in-memory registry, and Jetstream is notified if it's a record type.
3333-- **delete**: The lexicon is removed from the `lexicons` table and registry.
3434-3535-### Startup re-fetch
3636-3737-On every startup, HappyView re-fetches all network lexicons from their respective PDSes. This ensures consistency even if Jetstream events were missed while offline. Failures are logged as warnings but don't block startup.
3838-3939-## Admin API
4040-4141-See [Admin API - Network Lexicons](admin-api.md#network-lexicons) for endpoint details.
4242-4343-### Quick reference
4444-4545-```sh
4646-# Add a network lexicon
4747-curl -X POST http://localhost:3000/admin/network-lexicons \
4848- -H "$AUTH" -H "Content-Type: application/json" \
4949- -d '{ "nsid": "games.gamesgamesgamesgames.game" }'
5050-5151-# List tracked network lexicons
5252-curl http://localhost:3000/admin/network-lexicons -H "$AUTH"
5353-5454-# Remove a network lexicon
5555-curl -X DELETE http://localhost:3000/admin/network-lexicons/games.gamesgamesgamesgames.game \
5656- -H "$AUTH"
5757-```
-90
docs/quickstart.md
···11-# Quickstart
22-33-## Deploy on Railway
44-55-The fastest way to get HappyView running is with Railway. This template deploys HappyView, AIP, Tap, and Postgres with a single click:
66-77-[](https://railway.com/deploy/I1jvZl?referralCode=0QOgj_)
88-99-### Required configuration
1010-1111-After deploying the template, you'll need to configure a few things before the stack works properly:
1212-1313-1. **Set your admin DID.** In the AIP service variables, set `ADMIN_DIDS` to your AT Protocol DID (e.g. `did:plc:abc123...`). You can find your DID by looking up your handle on [Internect](https://internect.info/).
1414-1515-2. **Generate AIP signing keys.** The `OAUTH_SIGNING_KEYS` and `ATPROTO_OAUTH_SIGNING_KEYS` variables require multibase-encoded P-256 private keys. See the [AIP Signing Keys documentation](https://github.com/graze-social/aip/blob/main/CONFIGURATION.md#signing-keys) for generation instructions.
1616-1717-3. **Generate public URLs.** The services won't work until HappyView and AIP have public domains assigned in Railway.
1818-1919-## Local development
2020-2121-### Prerequisites
2222-2323-- Rust (stable)
2424-- PostgreSQL 17+
2525-- A running [AIP](https://github.com/graze-social/aip) instance
2626-2727-### 1. Clone and configure
2828-2929-```sh
3030-git clone https://github.com/graze-social/happyview.git
3131-cd happyview
3232-cp .env.example .env
3333-```
3434-3535-Edit `.env`:
3636-3737-```sh
3838-DATABASE_URL=postgres://happyview:happyview@localhost/happyview
3939-AIP_URL=http://localhost:8080
4040-```
4141-4242-See [Configuration](configuration.md) for all available variables.
4343-4444-### 2. Start Postgres and run migrations
4545-4646-```sh
4747-docker compose up -d postgres
4848-cargo run
4949-```
5050-5151-Migrations run automatically on startup.
5252-5353-### 3. Upload a lexicon
5454-5555-The first authenticated request to an admin endpoint auto-creates you as the initial admin. Authenticate with an AIP-issued Bearer token:
5656-5757-```sh
5858-curl -X POST http://localhost:3000/admin/lexicons \
5959- -H "Authorization: Bearer $TOKEN" \
6060- -H "Content-Type: application/json" \
6161- -d '{
6262- "lexicon_json": {
6363- "lexicon": 1,
6464- "id": "games.gamesgamesgamesgames.game",
6565- "defs": {
6666- "main": {
6767- "type": "record",
6868- "key": "tid",
6969- "record": {
7070- "type": "object",
7171- "properties": {
7272- "title": { "type": "string" }
7373- }
7474- }
7575- }
7676- }
7777- },
7878- "backfill": true
7979- }'
8080-```
8181-8282-HappyView now subscribes to `games.gamesgamesgamesgames.game` on Jetstream and starts indexing records.
8383-8484-### 4. Query records
8585-8686-```sh
8787-curl http://localhost:3000/xrpc/games.gamesgamesgamesgames.listGames?limit=10
8888-```
8989-9090-See [XRPC API](xrpc-api.md) for query and procedure details.
+211
docs/reference/architecture.md
···11+# Architecture
22+33+Guide for contributors working on HappyView itself. For a user-facing overview, see the [Introduction](/).
44+55+## System overview
66+77+```mermaid
88+graph LR
99+ Application
1010+1111+ Application -->|"GET /xrpc/{method}"| Query
1212+ Application -->|"POST /xrpc/{method}"| Procedure
1313+1414+ subgraph HappyView
1515+ Query["Query Handler<br/><small>Lua Script (Optional)</small>"]
1616+ Procedure["Procedure Handler<br/><small>Lua Script (Optional)</small>"]
1717+ end
1818+1919+ Procedure --> DB
2020+ Query --> DB
2121+2222+ Procedure -->|proxy write| PDS["User PDS"]
2323+2424+ DB[("PostgreSQL<br/><small>records · lexicons</small>")]
2525+2626+ Tap["Tap<br/><small>WebSocket</small>"] -->|record events| DB
2727+ Relay["Relay<br/><small>Firehose</small>"] --> Tap
2828+```
2929+3030+Reads flow top-down through the query handler to Postgres. Writes flow through the procedure handler to the user's PDS, then HappyView indexes the record locally. All record data enters the system through Tap, which handles both real-time firehose events and historical backfill. HappyView syncs collection filters to Tap and discovers repos via the relay for backfill, but Tap performs all record fetching.
3131+3232+## Module overview
3333+3434+```
3535+src/
3636+ main.rs Startup: config, DB, migrations, spawn Tap worker, start server
3737+ lib.rs AppState struct, module declarations
3838+ config.rs Environment variable loading
3939+ error.rs AppError enum (Auth, BadRequest, Forbidden, Internal, NotFound, PdsError)
4040+ server.rs Axum router: fixed routes + admin nest + XRPC catch-all + static files
4141+ lexicon.rs ParsedLexicon, LexiconRegistry (Arc<RwLock<HashMap>>)
4242+ profile.rs DID document resolution, PDS discovery, profile fetching
4343+ tap.rs Tap WebSocket listener, collection filter sync, backfill delegation
4444+ aip.rs AIP reverse proxy
4545+ resolve.rs NSID authority resolution (DNS TXT → DID → PDS)
4646+ auth/
4747+ mod.rs Re-exports
4848+ middleware.rs Claims extractor (validates Bearer token via AIP /oauth/userinfo)
4949+ jwks.rs JWKS key fetching
5050+ admin/
5151+ mod.rs Admin route definitions
5252+ auth.rs AdminAuth extractor (Claims + DID lookup + auto-bootstrap)
5353+ admins.rs Admin CRUD handlers
5454+ lexicons.rs Lexicon CRUD handlers
5555+ network_lexicons.rs Network lexicon tracking (add, list, remove)
5656+ records.rs Record listing handler
5757+ stats.rs Record count stats
5858+ backfill.rs Backfill job creation (relay discovery + Tap delegation)
5959+ types.rs Request/response structs for admin endpoints
6060+ lua/
6161+ mod.rs Re-exports
6262+ context.rs Lua context globals (method, params, input, caller_did, collection)
6363+ db_api.rs Lua database API (db.query, db.get, db.count)
6464+ execute.rs Script execution and sandbox setup
6565+ record.rs Lua Record API (constructor, save, delete, load)
6666+ sandbox.rs Restricted Lua environment (removed modules, instruction limit)
6767+ tid.rs TID generation for Lua scripts
6868+ repo/
6969+ mod.rs Re-exports
7070+ dpop.rs DPoP JWT proof generation (ES256/P-256)
7171+ pds.rs PDS proxy helpers (JSON POST, blob POST, response forwarding)
7272+ session.rs ATP session fetching from AIP
7373+ upload_blob.rs Blob upload handler
7474+ xrpc/
7575+ mod.rs Re-exports
7676+ query.rs Dynamic GET handler (Lua script or default: single record + list)
7777+ procedure.rs Dynamic POST handler (Lua script or default: create vs put)
7878+```
7979+8080+## Request flow
8181+8282+### Reads (queries)
8383+8484+```
8585+Client GET /xrpc/{method}?params
8686+ -> xrpc::xrpc_get()
8787+ -> LexiconRegistry lookup (must be Query type)
8888+ -> If Lua script attached: execute script (has access to db API)
8989+ -> Else: default SQL query on records table (collection from target_collection)
9090+ -> JSON response
9191+```
9292+9393+### Writes (procedures)
9494+9595+```
9696+Client POST /xrpc/{method} + Bearer token
9797+ -> Claims extractor validates token via AIP /oauth/userinfo
9898+ -> xrpc::xrpc_post()
9999+ -> LexiconRegistry lookup (must be Procedure type)
100100+ -> If Lua script attached: execute script (has access to Record API)
101101+ -> Else: default create/update (auto-detect based on uri field)
102102+ -> Fetch ATP session from AIP /api/atprotocol/session
103103+ -> Generate DPoP proof (ES256)
104104+ -> Proxy to user's PDS (createRecord or putRecord)
105105+ -> Upsert record locally
106106+ -> Forward PDS response
107107+```
108108+109109+### Admin endpoints
110110+111111+```
112112+Client request + Bearer token
113113+ -> AdminAuth extractor:
114114+ 1. Claims validation via AIP
115115+ 2. DID lookup in admins table (auto-bootstrap if empty)
116116+ 3. 403 if not admin
117117+ -> Admin handler
118118+ -> JSON response
119119+```
120120+121121+## Data flow
122122+123123+### Real-time indexing
124124+125125+```
126126+Tap WebSocket connection (tap::spawn)
127127+ -> Collection filters synced to Tap on startup and lexicon changes
128128+ -> Record events:
129129+ create/update -> UPSERT into records table
130130+ delete -> DELETE from records table
131131+ -> Lexicon schema events (com.atproto.lexicon.schema):
132132+ -> Update tracked network lexicons in DB and registry
133133+ -> Reconnects automatically on errors or collection filter changes
134134+```
135135+136136+### Backfill
137137+138138+```
139139+POST /admin/backfill
140140+ -> Create backfill_jobs record (status = running)
141141+ -> Relay listReposByCollection -> list of DIDs
142142+ -> Send DIDs to Tap in batches of 1000 (POST /repos/add)
143143+ -> Mark job as completed
144144+ -> Tap fetches records asynchronously and delivers via WebSocket
145145+```
146146+147147+## Database schema
148148+149149+### `records`
150150+151151+| Column | Type | Description |
152152+| ------------ | ----------- | ----------------------------------- |
153153+| `uri` | text (PK) | AT URI (`at://did/collection/rkey`) |
154154+| `did` | text | Author DID |
155155+| `collection` | text | Lexicon NSID |
156156+| `rkey` | text | Record key |
157157+| `record` | jsonb | Record value |
158158+| `cid` | text | Content identifier |
159159+| `indexed_at` | timestamptz | When HappyView indexed this record |
160160+161161+### `lexicons`
162162+163163+| Column | Type | Description |
164164+| ------------------- | ----------- | ----------------------------------------------- |
165165+| `id` | text (PK) | Lexicon NSID |
166166+| `revision` | integer | Incremented on upsert |
167167+| `lexicon_json` | jsonb | Raw lexicon definition |
168168+| `lexicon_type` | text | record, query, procedure, definitions |
169169+| `backfill` | boolean | Whether to backfill on upload |
170170+| `target_collection` | text | For queries/procedures: which record collection |
171171+| `created_at` | timestamptz | |
172172+| `updated_at` | timestamptz | |
173173+174174+### `admins`
175175+176176+| Column | Type | Description |
177177+| -------------- | ------------- | ------------------------------------- |
178178+| `id` | uuid (PK) | |
179179+| `did` | text (unique) | Admin's AT Protocol DID |
180180+| `created_at` | timestamptz | |
181181+| `last_used_at` | timestamptz | Updated on each authenticated request |
182182+183183+### `backfill_jobs`
184184+185185+| Column | Type | Description |
186186+| ----------------- | ----------- | ----------------------------------- |
187187+| `id` | uuid (PK) | |
188188+| `collection` | text | Target collection (null = all) |
189189+| `did` | text | Target DID (null = all) |
190190+| `status` | text | pending, running, completed, failed |
191191+| `total_repos` | integer | |
192192+| `processed_repos` | integer | |
193193+| `total_records` | integer | |
194194+| `error` | text | Error message if failed |
195195+| `started_at` | timestamptz | |
196196+| `completed_at` | timestamptz | |
197197+| `created_at` | timestamptz | |
198198+199199+## Testing
200200+201201+```sh
202202+# Unit tests (no database needed)
203203+cargo test --lib
204204+205205+# All tests including end-to-end (requires Postgres)
206206+docker compose -f docker-compose.test.yml up -d
207207+TEST_DATABASE_URL=postgres://happyview:happyview@localhost:5433/happyview_test cargo test
208208+docker compose -f docker-compose.test.yml down
209209+```
210210+211211+End-to-end tests use `wiremock` to mock external services (AIP, PLC directory, PDSes) and a real Postgres database for full integration coverage.
+43
docs/reference/glossary.md
···11+# Glossary
22+33+Key terms used throughout the HappyView documentation. For a broader introduction to the AT Protocol, see the [official ATProto glossary](https://atproto.com/guides/glossary).
44+55+## AT Protocol terms
66+77+**AppView** — A backend service that indexes AT Protocol records and serves them through an API. HappyView is an AppView. See the [ATProto docs](https://atproto.com/guides/glossary#app-view) for more.
88+99+**DID** (Decentralized Identifier) — A persistent, globally unique identifier for an account (e.g. `did:plc:abc123`).
1010+1111+**Firehose** — A real-time stream of all record events (creates, updates, deletes) across the AT Protocol network. HappyView consumes this via [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap).
1212+1313+**Handle** — A human-readable name for an account (e.g. `user.bsky.social`). Handles resolve to a DID via DNS or the PLC directory.
1414+1515+**Lexicon** — A schema definition for AT Protocol data types and API methods. Lexicons define what records look like, what endpoints exist, and what parameters they accept. See [Lexicons](../guides/lexicons).
1616+1717+**NSID** (Namespaced Identifier) — A reverse-DNS identifier for a lexicon (e.g. `xyz.statusphere.status`). The authority is everything except the last segment.
1818+1919+**PDS** (Personal Data Server) — The server that hosts a user's data. Users can be on any PDS — there's no single server. HappyView proxies writes back to each user's PDS.
2020+2121+**PLC directory** — A public service (e.g. `plc.directory`) that maps DIDs to their DID documents, which contain the user's PDS endpoint and other metadata.
2222+2323+**Record** — A single piece of data in an AT Protocol repository, identified by an AT URI (e.g. `at://did:plc:abc/xyz.statusphere.status/abc123`).
2424+2525+**Relay** — A network service that aggregates repository data from many PDSes. HappyView queries the relay during [backfill](../guides/backfill) to discover which repos contain records for a given collection, then delegates the actual record fetching to Tap.
2626+2727+**rkey** (Record Key) — The unique key for a record within a collection and repo. These are most commonly TIDs (timestamp-based) or NSIDs.
2828+2929+**TID** (Timestamp Identifier) — A 13-character sortable identifier used as a record key. Generated from the current timestamp.
3030+3131+**XRPC** — The HTTP-based RPC protocol used by the AT Protocol. Query methods map to GET requests, procedure methods map to POST requests. See [XRPC API](xrpc-api).
3232+3333+## HappyView-specific terms
3434+3535+**AIP** — [Authentication and Identity Provider](https://github.com/graze-social/aip). An external service that handles AT Protocol OAuth for HappyView. Issues Bearer tokens used for authentication.
3636+3737+**Backfill** — The process of bulk-indexing existing records from the network. HappyView discovers repos via the relay and delegates record fetching to Tap. Runs when a new record-type lexicon is uploaded or triggered manually. See [Backfill](../guides/backfill).
3838+3939+**Network lexicon** — A lexicon fetched directly from the AT Protocol network via DNS authority resolution, rather than uploaded manually. See [Lexicons - Network lexicons](../guides/lexicons#network-lexicons).
4040+4141+**Tap** — A [firehose consumer and backfill worker](https://github.com/bluesky-social/indigo/tree/main/cmd/tap) that handles real-time record streaming, cryptographic verification, and historical record fetching. HappyView connects to Tap via WebSocket to receive record events, and delegates backfill work to Tap via its HTTP API.
4242+4343+**Target collection** — The record collection that a query or procedure lexicon operates on. Set via the `target_collection` field when uploading a lexicon.
+70
docs/reference/production-deployment.md
···11+# Deployment
22+33+HappyView requires a Postgres database and an [AIP](https://github.com/graze-social/aip) instance for OAuth. The [Quickstart](../getting-started/deployment/railway) covers the fastest path with Railway. This page covers other deployment options.
44+55+## Docker
66+77+Build the image:
88+99+```sh
1010+docker build -t happyview .
1111+```
1212+1313+For local development, see [Docker deployment](../getting-started/deployment/docker).
1414+1515+### Production Compose example
1616+1717+:::note
1818+This example omits [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap), which is required for real-time record streaming and backfill. See the full `docker-compose.yml` in the repository for a complete configuration including Tap.
1919+:::
2020+2121+```yaml
2222+services:
2323+ postgres:
2424+ image: postgres:17
2525+ environment:
2626+ POSTGRES_USER: happyview
2727+ POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
2828+ POSTGRES_DB: happyview
2929+ volumes:
3030+ - pgdata:/var/lib/postgresql/data
3131+3232+ happyview:
3333+ image: happyview:latest
3434+ ports:
3535+ - "3000:3000"
3636+ environment:
3737+ DATABASE_URL: "postgres://happyview:${POSTGRES_PASSWORD}@postgres/happyview"
3838+ AIP_URL: "https://aip.example.com"
3939+ depends_on:
4040+ postgres:
4141+ condition: service_healthy
4242+4343+volumes:
4444+ pgdata:
4545+```
4646+4747+## Railway / Fly.io / other platforms
4848+4949+The general process for any hosting platform:
5050+5151+1. Provision a Postgres 17+ database
5252+2. Deploy an [AIP](https://github.com/graze-social/aip) instance (handles OAuth for your AppView)
5353+3. Set `DATABASE_URL` and `AIP_URL` environment variables (see [Configuration](../getting-started/configuration) for all options)
5454+4. Deploy the Docker image or build from source
5555+5. HappyView listens on `PORT` (default `3000`)
5656+6. Health check: `GET /health` returns `ok`
5757+5858+For Railway specifically, the [Quickstart](../getting-started/deployment/railway) template handles all of this with a single click.
5959+6060+## Database
6161+6262+Migrations run automatically on startup via `sqlx::migrate!()`. No manual migration step is needed.
6363+6464+## TLS
6565+6666+HappyView does not terminate TLS. Put it behind a reverse proxy (nginx, Caddy, Cloudflare Tunnel, etc.) for HTTPS.
6767+6868+## Logging
6969+7070+HappyView uses the `RUST_LOG` environment variable to control log output. The default (`happyview=debug,tower_http=debug`) logs all HappyView activity and HTTP requests. For production, consider `happyview=info,tower_http=info` to reduce noise. See [Configuration](../getting-started/configuration) for details.
+46
docs/reference/scripts/batch-save.md
···11+# Procedure: Batch Save
22+33+Use `Record.save_all()` to create multiple records in parallel.
44+55+**Lexicon type:** procedure
66+77+```lua
88+function handle()
99+ local records = {}
1010+ for _, item in ipairs(input.items) do
1111+ local r = Record(collection, item)
1212+ records[#records + 1] = r
1313+ end
1414+ Record.save_all(records)
1515+1616+ local uris = {}
1717+ for _, r in ipairs(records) do
1818+ uris[#uris + 1] = r._uri
1919+ end
2020+ return { uris = uris }
2121+end
2222+```
2323+2424+## How it works
2525+2626+1. Iterate over `input.items` and create a [`Record`](../../guides/scripting#record-api) instance for each item.
2727+2. Call [`Record.save_all()`](../../guides/scripting#static-methods) to save all records in parallel, rather than one at a time.
2828+3. Collect the resulting AT URIs and return them.
2929+3030+## Usage
3131+3232+```sh
3333+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.batchCreate \
3434+ -H "Authorization: Bearer $TOKEN" \
3535+ -H "Content-Type: application/json" \
3636+ -d '{
3737+ "items": [
3838+ { "text": "First", "createdAt": "2025-01-01T00:00:00Z" },
3939+ { "text": "Second", "createdAt": "2025-01-01T00:01:00Z" }
4040+ ]
4141+ }'
4242+```
4343+4444+## Use case
4545+4646+Batch saving is useful when a single user action should create multiple records (e.g. importing data, multi-step forms). `save_all` is significantly faster than calling `r:save()` in a loop because the PDS writes happen concurrently.
+74
docs/reference/scripts/cascading-delete.md
···11+# Procedure: Cascading Delete
22+33+Delete a record and all related records across collections.
44+55+**Lexicon type:** procedure
66+77+```lua
88+function handle()
99+ if not input.uri then
1010+ return { error = "uri is required" }
1111+ end
1212+1313+ -- Load the primary record
1414+ local primary = Record.load(input.uri)
1515+ if not primary then
1616+ return { error = "not found" }
1717+ end
1818+1919+ -- Find related records that reference this URI
2020+ local comments = db.query({
2121+ collection = "xyz.statusphere.comment",
2222+ did = caller_did,
2323+ limit = 100,
2424+ })
2525+2626+ -- Collect records to delete
2727+ local to_delete = { primary }
2828+ for _, comment in ipairs(comments.records) do
2929+ if comment.postUri == input.uri then
3030+ local r = Record.load(comment.uri)
3131+ if r then
3232+ to_delete[#to_delete + 1] = r
3333+ end
3434+ end
3535+ end
3636+3737+ -- Delete all matched records
3838+ for _, r in ipairs(to_delete) do
3939+ r:delete()
4040+ end
4141+4242+ return {
4343+ deleted = #to_delete,
4444+ }
4545+end
4646+```
4747+4848+## How it works
4949+5050+1. Load the primary record by URI. Return early if it doesn't exist.
5151+2. Query for related records, in this example comments by the same user that reference the primary record's URI.
5252+3. Load each related record with [`Record.load`](../../guides/scripting#static-methods) to get a deletable `Record` instance.
5353+4. Delete everything. Each `r:delete()` removes the record from the user's PDS and the local index.
5454+5555+## Usage
5656+5757+```sh
5858+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.deletePost \
5959+ -H "Authorization: Bearer $TOKEN" \
6060+ -H "Content-Type: application/json" \
6161+ -d '{ "uri": "at://did:plc:abc/xyz.statusphere.post/abc123" }'
6262+```
6363+6464+```json
6565+{
6666+ "deleted": 4
6767+}
6868+```
6969+7070+## Use case
7171+7272+Cascading deletes are useful when your data model has parent-child relationships across collections. For example, deleting a post should also clean up its comments, reactions, or metadata records. This keeps the user's repo and the local index consistent.
7373+7474+Note that this only deletes records owned by `caller_did`. AT Protocol records can only be deleted by their owner. If the related records could have more than 100 matches, paginate through all of them before deleting.
+84
docs/reference/scripts/complex-mutations.md
···11+# Procedure: Complex Mutations
22+33+Load an existing record, apply multiple transformations, and save it back.
44+55+**Lexicon type:** procedure
66+77+```lua
88+function handle()
99+ if not input.uri then
1010+ return { error = "uri is required" }
1111+ end
1212+1313+ local r = Record.load(input.uri)
1414+ if not r then
1515+ return { error = "not found" }
1616+ end
1717+1818+ -- Increment a counter
1919+ r.likeCount = (r.likeCount or 0) + 1
2020+2121+ -- Merge tags, deduplicating and capping at 10
2222+ r.tags = r.tags or {}
2323+ if input.tags then
2424+ for _, tag in ipairs(input.tags) do
2525+ local found = false
2626+ for _, t in ipairs(r.tags) do
2727+ if t == tag then
2828+ found = true
2929+ break
3030+ end
3131+ end
3232+ if not found then
3333+ r.tags[#r.tags + 1] = tag
3434+ end
3535+ end
3636+ -- Keep only the last 10
3737+ while #r.tags > 10 do
3838+ table.remove(r.tags, 1)
3939+ end
4040+ end
4141+4242+ -- Normalize a string field
4343+ if input.title then
4444+ r.title = string.gsub(input.title, "^%s+", "")
4545+ r.title = string.gsub(r.title, "%s+$", "")
4646+ end
4747+4848+ -- Set a computed field
4949+ r.updatedAt = now()
5050+5151+ r:save()
5252+5353+ return { uri = r._uri, cid = r._cid }
5454+end
5555+```
5656+5757+## How it works
5858+5959+1. Load the existing record with [`Record.load`](../../guides/scripting#static-methods). This gives you a mutable `Record` instance with all the current field values.
6060+2. Apply transformations directly on the record's fields:
6161+ - **Increment a counter**: use `or 0` to handle the field being `nil` on first access.
6262+ - **Merge tags**: iterate over `input.tags`, skip duplicates already in `r.tags`, append new ones, then trim the list to 10.
6363+ - **Normalize a string**: use `string.gsub` to trim whitespace.
6464+ - **Set a timestamp**: use [`now()`](../../guides/scripting#utility-globals) for UTC ISO 8601.
6565+3. Call `r:save()`. Since `_uri` is set (from the load), this calls `putRecord` to update the record on the user's PDS.
6666+6767+## Usage
6868+6969+```sh
7070+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.updatePost \
7171+ -H "Authorization: Bearer $TOKEN" \
7272+ -H "Content-Type: application/json" \
7373+ -d '{
7474+ "uri": "at://did:plc:abc/xyz.statusphere.post/abc123",
7575+ "tags": ["tutorial", "atproto"],
7676+ "title": " My Post Title "
7777+ }'
7878+```
7979+8080+## Use case
8181+8282+This pattern is useful when updates involve more than simple field replacement: counters, bounded lists, string normalization, or computed fields. All mutations happen in memory before the single `r:save()` call, so there's no partial save: either all changes are written or none are.
8383+8484+If the record has a schema, HappyView only sends fields defined in the schema's `properties` to the PDS on save. Extra fields you set on the record instance are ignored.
+32
docs/reference/scripts/create-record.md
···11+# Procedure: Create a Record
22+33+The simplest write: take the request body, save it as a record, and return the URI.
44+55+**Lexicon type:** procedure
66+77+```lua
88+function handle()
99+ local r = Record(collection, input)
1010+ r:save()
1111+ return { uri = r._uri, cid = r._cid }
1212+end
1313+```
1414+1515+## How it works
1616+1717+1. Create a new [`Record`](../../guides/scripting#record-api) instance from the target collection, populated with the fields from the request body.
1818+2. Call `r:save()`, which creates the record on the caller's PDS and indexes it locally.
1919+3. Return the AT URI and CID of the newly created record.
2020+2121+## Usage
2222+2323+```sh
2424+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.createRecord \
2525+ -H "Authorization: Bearer $TOKEN" \
2626+ -H "Content-Type: application/json" \
2727+ -d '{ "text": "Hello world", "createdAt": "2025-01-01T00:00:00Z" }'
2828+```
2929+3030+## Use case
3131+3232+This is the simplest possible write procedure. It works well when the client is responsible for populating all record fields and no server-side validation or transformation is needed.
+83
docs/reference/scripts/expanded-query.md
···11+# Query: Expanded Query with Profiles
22+33+List statuses and include the profile of each user who created one.
44+55+**Lexicon type:** query
66+77+```lua
88+function handle()
99+ local limit = tonumber(params.limit) or 20
1010+ if limit > 100 then limit = 100 end
1111+1212+ local result = db.query({
1313+ collection = "xyz.statusphere.status",
1414+ did = params.did,
1515+ limit = limit,
1616+ offset = tonumber(params.cursor) or 0,
1717+ })
1818+1919+ -- Collect unique DIDs from the statuses
2020+ local seen = {}
2121+ local profile_uris = {}
2222+ for _, status in ipairs(result.records) do
2323+ local did = string.match(status.uri, "at://([^/]+)/")
2424+ if did and not seen[did] then
2525+ seen[did] = true
2626+ profile_uris[#profile_uris + 1] = "at://" .. did .. "/app.bsky.actor.profile/self"
2727+ end
2828+ end
2929+3030+ -- Load all profiles in parallel
3131+ local profiles = {}
3232+ if #profile_uris > 0 then
3333+ local loaded = Record.load_all(profile_uris)
3434+ for i, profile in ipairs(loaded) do
3535+ if profile then
3636+ profiles[#profiles + 1] = profile
3737+ end
3838+ end
3939+ end
4040+4141+ return {
4242+ statuses = result.records,
4343+ profiles = profiles,
4444+ cursor = result.cursor,
4545+ }
4646+end
4747+```
4848+4949+## How it works
5050+5151+1. Query statuses from the target collection with pagination, same as a normal list query.
5252+2. Extract the unique DIDs from the returned status URIs using `string.match`.
5353+3. Build an AT URI for each DID's `app.bsky.actor.profile/self` record (this is where Bluesky profiles live).
5454+4. Load all profiles in parallel with [`Record.load_all`](../../guides/scripting#static-methods). Profiles that aren't indexed locally return `nil` and are skipped.
5555+5. Return statuses and profiles as separate keys, with the cursor from the status query.
5656+5757+## Usage
5858+5959+```
6060+GET /xrpc/xyz.statusphere.listStatusesWithProfiles?limit=10
6161+GET /xrpc/xyz.statusphere.listStatusesWithProfiles?did=did:plc:abc
6262+GET /xrpc/xyz.statusphere.listStatusesWithProfiles?cursor=20&limit=20
6363+```
6464+6565+```json
6666+{
6767+ "statuses": [
6868+ { "uri": "at://did:plc:abc/xyz.statusphere.status/3abc123", "status": "😊", "createdAt": "..." },
6969+ { "uri": "at://did:plc:def/xyz.statusphere.status/3def456", "status": "🌟", "createdAt": "..." }
7070+ ],
7171+ "profiles": [
7272+ { "uri": "at://did:plc:abc/app.bsky.actor.profile/self", "displayName": "Alice", "avatar": "..." },
7373+ { "uri": "at://did:plc:def/app.bsky.actor.profile/self", "displayName": "Bob", "avatar": "..." }
7474+ ],
7575+ "cursor": "10"
7676+}
7777+```
7878+7979+## Use case
8080+8181+This pattern avoids N+1 queries (fetching each author's profile individually) on the client side. Instead of fetching statuses and then making a separate request for each user's profile, the client gets everything in one call. The deduplication step ensures each profile is loaded only once even if multiple statuses are from the same user.
8282+8383+Note that `Record.load_all` reads from HappyView's local index. Profiles only appear if `app.bsky.actor.profile` is also being indexed. If a profile hasn't been indexed yet, it's silently omitted from the response.
+36
docs/reference/scripts/get-record.md
···11+# Query: Get a Single Record
22+33+Fetch a single record by its AT URI.
44+55+**Lexicon type:** query
66+77+```lua
88+function handle()
99+ if not params.uri then
1010+ return { error = "uri parameter is required" }
1111+ end
1212+1313+ local record = db.get(params.uri)
1414+ if not record then
1515+ return { error = "not found" }
1616+ end
1717+1818+ return { record = record }
1919+end
2020+```
2121+2222+## How it works
2323+2424+1. Check that the `uri` query parameter is present. Return a structured error if missing.
2525+2. Look up the record with [`db.get`](../../guides/scripting#dbget), which returns the record table or `nil`.
2626+3. Return the record wrapped in an object.
2727+2828+## Usage
2929+3030+```
3131+GET /xrpc/xyz.statusphere.getRecord?uri=at://did:plc:abc/xyz.statusphere.record/abc123
3232+```
3333+3434+## Use case
3535+3636+A focused read endpoint for detail views or record verification. Returns structured error responses instead of calling `error()`, so the client gets a 200 with an error field it can handle gracefully rather than a 500.
+41
docs/reference/scripts/list-or-fetch.md
···11+# Query: List or Fetch Records
22+33+This query handles both single-record lookups (when a `uri` param is provided) and paginated listing.
44+55+**Lexicon type:** query
66+77+```lua
88+function handle()
99+ if params.uri then
1010+ local record = db.get(params.uri)
1111+ if not record then
1212+ return { error = "record not found" }
1313+ end
1414+ return { record = record }
1515+ end
1616+1717+ return db.query({
1818+ collection = collection,
1919+ did = params.did,
2020+ limit = tonumber(params.limit) or 20,
2121+ offset = tonumber(params.cursor) or 0,
2222+ })
2323+end
2424+```
2525+2626+## How it works
2727+2828+1. If a `uri` query parameter is provided, fetch that single record with [`db.get`](../../guides/scripting#dbget) and return it. If it doesn't exist, return a structured error (using `error()` would trigger a 500 response).
2929+2. Otherwise, list records from the target collection using [`db.query`](../../guides/scripting#dbquery), with optional filtering by `did` and pagination via `limit`/`offset`. Since query parameters arrive as strings, `tonumber()` converts them to numbers.
3030+3131+## Usage
3232+3333+```
3434+GET /xrpc/xyz.statusphere.listRecords?limit=10
3535+GET /xrpc/xyz.statusphere.listRecords?did=did:plc:abc
3636+GET /xrpc/xyz.statusphere.listRecords?uri=at://did:plc:abc/xyz.statusphere.record/abc123
3737+```
3838+3939+## Use case
4040+4141+This is a good default query script when you want a single endpoint that serves double duty: list browsing for feeds/timelines and direct record fetching for detail views.
+40
docs/reference/scripts/paginated-list.md
···11+# Query: Paginated List
22+33+List records from a collection with cursor-based pagination and an optional DID filter.
44+55+**Lexicon type:** query
66+77+```lua
88+function handle()
99+ local limit = tonumber(params.limit) or 20
1010+ if limit > 100 then limit = 100 end
1111+1212+ local result = db.query({
1313+ collection = collection,
1414+ did = params.did,
1515+ limit = limit,
1616+ offset = tonumber(params.cursor) or 0,
1717+ })
1818+1919+ return result
2020+end
2121+```
2222+2323+## How it works
2424+2525+1. Parse `limit` from the query string, defaulting to 20 and capping at 100.
2626+2. Call [`db.query`](../../guides/scripting#dbquery) with the target collection, optional DID filter, and offset-based pagination.
2727+3. Return the result directly. `db.query` returns `{ records = [...], cursor = "..." }` where `cursor` is present when more records exist.
2828+2929+## Usage
3030+3131+```
3232+GET /xrpc/xyz.statusphere.listStatuses
3333+GET /xrpc/xyz.statusphere.listStatuses?limit=50
3434+GET /xrpc/xyz.statusphere.listStatuses?did=did:plc:abc&limit=10
3535+GET /xrpc/xyz.statusphere.listStatuses?cursor=20&limit=20
3636+```
3737+3838+## Use case
3939+4040+A straightforward list endpoint for feeds, timelines, or browsing records by collection. The `cursor` value returned by `db.query` is an offset. Clients pass it back as the `cursor` parameter to fetch the next page. Since all query parameters arrive as strings, use `tonumber()` to convert `limit` and `cursor` to numbers.
+74
docs/reference/scripts/sidecar-records.md
···11+# Procedure: Create Sidecar Records
22+33+Create two records with different collection NSIDs but the same rkey, linking them together by key.
44+55+**Lexicon type:** procedure
66+77+```lua
88+function handle()
99+ local rkey = TID()
1010+1111+ local post = Record("xyz.statusphere.post", {
1212+ text = input.text,
1313+ createdAt = now(),
1414+ })
1515+ post:set_rkey(rkey)
1616+1717+ local metadata = Record("xyz.statusphere.postMetadata", {
1818+ lang = input.lang or "en",
1919+ source = input.source or "web",
2020+ createdAt = now(),
2121+ })
2222+ metadata:set_rkey(rkey)
2323+2424+ Record.save_all({ post, metadata })
2525+2626+ return {
2727+ post = { uri = post._uri, cid = post._cid },
2828+ metadata = { uri = metadata._uri, cid = metadata._cid },
2929+ }
3030+end
3131+```
3232+3333+## How it works
3434+3535+1. Generate a single [`TID()`](../../guides/scripting#utility-globals) to use as the rkey for both records.
3636+2. Create a `Record` for each collection and call `r:set_rkey()` with the shared rkey.
3737+3. Save both records in parallel with [`Record.save_all()`](../../guides/scripting#static-methods).
3838+4. Return both URIs so the client knows the identity of each record.
3939+4040+## Usage
4141+4242+```sh
4343+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.createPost \
4444+ -H "Authorization: Bearer $TOKEN" \
4545+ -H "Content-Type: application/json" \
4646+ -d '{ "text": "Hello world", "lang": "en", "source": "web" }'
4747+```
4848+4949+The response includes URIs for both the post and its metadata:
5050+5151+```json
5252+{
5353+ "post": {
5454+ "uri": "at://did:plc:abc/xyz.statusphere.post/3abc123",
5555+ "cid": "bafyrei..."
5656+ },
5757+ "metadata": {
5858+ "uri": "at://did:plc:abc/xyz.statusphere.postMetadata/3abc123",
5959+ "cid": "bafyrei..."
6060+ }
6161+}
6262+```
6363+6464+## Use case
6565+6666+Sidecar records are useful when you want to associate related data across collections without embedding everything in a single record. Because they share an rkey, you can derive one URI from the other:
6767+6868+```
6969+at:// did:plc:abc /xyz.statusphere.post /3abc123
7070+at:// did:plc:abc /xyz.statusphere.postMetadata /3abc123
7171+ ^^^^^^^ same rkey
7272+```
7373+7474+This is a common AT Protocol pattern for keeping a primary record lean while storing auxiliary data (metadata, reactions, settings) in a companion collection.
+61
docs/reference/scripts/update-or-delete.md
···11+# Procedure: Update or Delete
22+33+A single endpoint that handles create, update, and delete based on the input fields.
44+55+**Lexicon type:** procedure
66+77+```lua
88+function handle()
99+ if input.delete and input.uri then
1010+ local r = Record.load(input.uri)
1111+ if r then r:delete() end
1212+ return { success = true }
1313+ end
1414+1515+ if input.uri then
1616+ -- Update existing
1717+ local r = Record.load(input.uri)
1818+ if not r then error("not found") end
1919+ r.status = input.status
2020+ r:save()
2121+ return { uri = r._uri, cid = r._cid }
2222+ end
2323+2424+ -- Create new
2525+ local r = Record(collection, input)
2626+ r:save()
2727+ return { uri = r._uri, cid = r._cid }
2828+end
2929+```
3030+3131+## How it works
3232+3333+1. If `input.delete` is truthy and `input.uri` is provided, load the record with [`Record.load`](../../guides/scripting#static-methods) and delete it.
3434+2. If only `input.uri` is provided, load the existing record with [`Record.load`](../../guides/scripting#static-methods), update its fields, and save it back. Since `_uri` is already set, `r:save()` calls `putRecord` instead of `createRecord`.
3535+3. If neither condition matches, create a new record from the input.
3636+3737+## Usage
3838+3939+```sh
4040+# Create
4141+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.setRecord \
4242+ -H "Authorization: Bearer $TOKEN" \
4343+ -H "Content-Type: application/json" \
4444+ -d '{ "status": "hello" }'
4545+4646+# Update
4747+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.setRecord \
4848+ -H "Authorization: Bearer $TOKEN" \
4949+ -H "Content-Type: application/json" \
5050+ -d '{ "uri": "at://did:plc:abc/xyz.statusphere.record/abc123", "status": "updated" }'
5151+5252+# Delete
5353+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.setRecord \
5454+ -H "Authorization: Bearer $TOKEN" \
5555+ -H "Content-Type: application/json" \
5656+ -d '{ "uri": "at://did:plc:abc/xyz.statusphere.record/abc123", "delete": true }'
5757+```
5858+5959+## Use case
6060+6161+This pattern reduces the number of endpoints your app needs by multiplexing create, update, and delete through a single procedure. The presence of `uri` and `delete` fields in the input determines the action.
+60
docs/reference/scripts/upsert-record.md
···11+# Procedure: Upsert a Record
22+33+Create a new record, or update an existing one if the client provides its rkey.
44+55+**Lexicon type:** procedure
66+77+```lua
88+function handle()
99+ local rkey = input.rkey or TID()
1010+ local uri = "at://" .. caller_did .. "/" .. collection .. "/" .. rkey
1111+1212+ local r = Record.load(uri)
1313+ if r then
1414+ -- Update existing record
1515+ r.status = input.status
1616+ r.updatedAt = now()
1717+ r:save()
1818+ else
1919+ -- Create new record
2020+ r = Record(collection, {
2121+ status = input.status,
2222+ createdAt = now(),
2323+ updatedAt = now(),
2424+ })
2525+ r:set_rkey(rkey)
2626+ r:save()
2727+ end
2828+2929+ return { uri = r._uri, cid = r._cid }
3030+end
3131+```
3232+3333+## How it works
3434+3535+1. Use the client-provided `input.rkey` if present, otherwise generate a new [`TID()`](../../guides/scripting#utility-globals). This means omitting `rkey` always creates, while providing one enables updates.
3636+2. Build the AT URI from the caller's DID, the target collection, and the rkey, then try to load it with [`Record.load`](../../guides/scripting#static-methods).
3737+3. If the record exists, update its fields and save. Since `_uri` is already set, `r:save()` calls `putRecord`.
3838+4. If it doesn't exist, create a new record, set the rkey explicitly with `r:set_rkey()`, and save. This calls `createRecord` with the specified rkey.
3939+4040+## Usage
4141+4242+```sh
4343+# Create: no rkey, so a new TID is generated
4444+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.setStatus \
4545+ -H "Authorization: Bearer $TOKEN" \
4646+ -H "Content-Type: application/json" \
4747+ -d '{ "status": "hello" }'
4848+# → { "uri": "at://did:plc:abc/xyz.statusphere.status/3abc123", "cid": "bafyrei..." }
4949+5050+# Update: pass the rkey back to update the same record
5151+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.setStatus \
5252+ -H "Authorization: Bearer $TOKEN" \
5353+ -H "Content-Type: application/json" \
5454+ -d '{ "rkey": "3abc123", "status": "updated" }'
5555+# → { "uri": "at://did:plc:abc/xyz.statusphere.status/3abc123", "cid": "bafyrei..." }
5656+```
5757+5858+## Use case
5959+6060+This is useful when the client knows whether it's creating or editing, but you want a single endpoint for both. The client omits `rkey` for new records and includes it when editing an existing one. The rkey from the initial create response acts as the record's stable identifier for future updates.
+100
docs/reference/troubleshooting.md
···11+# Troubleshooting
22+33+Common issues and how to resolve them.
44+55+## XRPC endpoint returns 404
66+77+**Symptom**: `GET /xrpc/your.method.name` returns `{"error": "method not found"}`.
88+99+**Causes**:
1010+1111+- The lexicon hasn't been uploaded yet. Check with `GET /admin/lexicons` or the [dashboard](../getting-started/dashboard).
1212+- The lexicon's `defs.main.type` doesn't match the HTTP method. Queries are `GET`, procedures are `POST`.
1313+- The NSID in the URL doesn't match the `id` field in the uploaded lexicon JSON.
1414+1515+## Queries return empty results
1616+1717+**Symptom**: The XRPC query endpoint returns `{"records": []}` even though records should exist.
1818+1919+**Causes**:
2020+2121+- The query lexicon is missing a `target_collection`. Without it, the query doesn't know which records to read. See [Lexicons - target_collection](../guides/lexicons#target-collection).
2222+- The record-type lexicon hasn't finished backfilling. Check backfill status with `GET /admin/backfill/status` or the dashboard.
2323+- Records exist on the network but HappyView hasn't indexed them yet. Tap only picks up new events from when the collection filter was added. Use [backfill](../guides/backfill) for historical records.
2424+2525+## Procedure returns 401 Unauthorized
2626+2727+**Symptom**: `POST /xrpc/your.method.name` returns `{"error": "..."}` with status 401.
2828+2929+**Causes**:
3030+3131+- The `Authorization: Bearer <token>` header is missing or malformed.
3232+- The token has expired or is invalid. Tokens are validated against AIP's `/oauth/userinfo` endpoint.
3333+- AIP is unreachable. Check that `AIP_URL` is set correctly and the AIP service is running.
3434+3535+For AIP-specific issues, see the [AIP documentation](https://github.com/graze-social/aip).
3636+3737+## Admin endpoints return 403 Forbidden
3838+3939+**Symptom**: Admin API calls return `{"error": "forbidden"}`.
4040+4141+**Causes**:
4242+4343+- Your DID is not in the admins table. Ask an existing admin to add you via `POST /admin/admins`.
4444+- If this is a fresh deployment with no admins, the first authenticated request to any admin endpoint automatically bootstraps you as admin. Make sure you're sending a valid Bearer token.
4545+4646+## Lua script errors
4747+4848+**Symptom**: An XRPC endpoint returns `{"error": "script execution failed"}` or `{"error": "script exceeded execution time limit"}`.
4949+5050+**What to do**:
5151+5252+1. Check the server logs: the full error message is logged at error level but not exposed to the client.
5353+2. Use `log("message")` in your script to trace execution. Output appears in server logs at debug level (requires `RUST_LOG` to include debug).
5454+3. If you hit the execution limit, your script likely has an infinite loop or is processing too much data. See [Lua Scripting - Sandbox](../guides/scripting#sandbox).
5555+5656+See [Lua Scripting - Debugging](../guides/scripting#debugging) for more.
5757+5858+## Backfill job stuck in "pending" or "running"
5959+6060+**Symptom**: A backfill job doesn't progress or stays in `pending`.
6161+6262+**Causes**:
6363+6464+- The backfill worker processes one job at a time. If another job is running, yours will wait.
6565+- The relay (`RELAY_URL`) may be unreachable or slow to respond. Check connectivity.
6666+- Individual PDS fetches can fail silently. The worker logs warnings and continues. Check server logs for details.
6767+6868+See [Backfill](../guides/backfill) for how the process works.
6969+7070+## Records not appearing in real time
7171+7272+**Symptom**: New records created on the network don't show up in queries.
7373+7474+**Causes**:
7575+7676+- HappyView receives real-time events via [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap). Make sure Tap is running and connected to HappyView. See the [Tap documentation](https://github.com/bluesky-social/indigo/tree/main/cmd/tap) for configuration.
7777+- No record-type lexicon exists for the collection. HappyView only indexes collections that have a corresponding record-type lexicon.
7878+- The Tap connection hasn't synced the new collection filter after a lexicon change. This should happen automatically. Check server logs for connection errors.
7979+8080+## OAuth or login issues
8181+8282+OAuth is handled entirely by [AIP](https://github.com/graze-social/aip). If users can't log in or tokens aren't working:
8383+8484+1. Verify AIP is running and reachable at the configured `AIP_URL`.
8585+2. Check that AIP has valid signing keys configured (`OAUTH_SIGNING_KEYS`).
8686+3. Check that both HappyView and AIP have public URLs assigned (required for OAuth callbacks).
8787+8888+See the [AIP documentation](https://github.com/graze-social/aip) for setup and debugging.
8989+9090+## Database connection errors
9191+9292+**Symptom**: HappyView fails to start or returns 500 errors.
9393+9494+**Causes**:
9595+9696+- `DATABASE_URL` is not set or points to an unreachable Postgres instance.
9797+- The database user doesn't have sufficient permissions. HappyView needs to create tables (migrations run automatically on startup).
9898+- Postgres version is too old. HappyView requires Postgres 17+.
9999+100100+See [Configuration](../getting-started/configuration) for environment variable details.
+208
docs/reference/xrpc-api.md
···11+# XRPC API
22+33+[XRPC](https://atproto.com/specs/xrpc) is the HTTP-based RPC protocol used by the AT Protocol. HappyView dynamically registers XRPC endpoints based on your uploaded [lexicons](../guides/lexicons): query lexicons become `GET /xrpc/{nsid}` routes, procedure lexicons become `POST /xrpc/{nsid}` routes.
44+55+If a query or procedure lexicon has a [Lua script](../guides/scripting) attached, the script handles the request. Otherwise, HappyView uses built-in default behavior (described below).
66+77+## Auth
88+99+- **Queries** (`GET /xrpc/{method}`): unauthenticated
1010+- **Procedures** (`POST /xrpc/{method}`): require an AIP-issued `Authorization: Bearer <token>` header
1111+- **getProfile**: requires auth
1212+- **uploadBlob**: requires auth
1313+1414+## Fixed endpoints
1515+1616+These endpoints are always available regardless of which lexicons are loaded.
1717+1818+### Health check
1919+2020+```
2121+GET /health
2222+```
2323+2424+```sh
2525+curl http://localhost:3000/health
2626+```
2727+2828+**Response**: `200 OK` with body `ok`
2929+3030+### Get profile
3131+3232+```
3333+GET /xrpc/app.bsky.actor.getProfile
3434+```
3535+3636+Returns the authenticated user's profile, resolved from their PDS via PLC directory lookup.
3737+3838+```sh
3939+curl http://localhost:3000/xrpc/app.bsky.actor.getProfile \
4040+ -H "Authorization: Bearer $TOKEN"
4141+```
4242+4343+**Response**: `200 OK`
4444+4545+```json
4646+{
4747+ "did": "did:plc:abc123",
4848+ "handle": "user.bsky.social",
4949+ "displayName": "User Name",
5050+ "description": "Bio text",
5151+ "avatarURL": "https://pds.example.com/xrpc/com.atproto.sync.getBlob?did=did:plc:abc123&cid=bafyabc"
5252+}
5353+```
5454+5555+### Upload blob
5656+5757+```
5858+POST /xrpc/com.atproto.repo.uploadBlob
5959+```
6060+6161+Proxies a blob upload to the authenticated user's PDS. Maximum size: 50MB.
6262+6363+```sh
6464+curl -X POST http://localhost:3000/xrpc/com.atproto.repo.uploadBlob \
6565+ -H "Authorization: Bearer $TOKEN" \
6666+ -H "Content-Type: image/png" \
6767+ --data-binary @image.png
6868+```
6969+7070+**Response**: proxied from the user's PDS.
7171+7272+## Dynamic query endpoints
7373+7474+Query endpoints are generated from lexicons with `type: "query"`. Without a [Lua script](../guides/scripting), they support two built-in modes depending on whether a `uri` parameter is provided.
7575+7676+### Single record
7777+7878+```
7979+GET /xrpc/{method}?uri={at-uri}
8080+```
8181+8282+```sh
8383+curl "http://localhost:3000/xrpc/xyz.statusphere.listStatuses?uri=at%3A%2F%2Fdid%3Aplc%3Aabc%2Fxyz.statusphere.status%2Fabc123"
8484+```
8585+8686+**Response**: `200 OK`
8787+8888+```json
8989+{
9090+ "record": {
9191+ "uri": "at://did:plc:abc/xyz.statusphere.status/abc123",
9292+ "$type": "xyz.statusphere.status",
9393+ "status": "\ud83d\ude0a",
9494+ "createdAt": "2025-01-01T12:00:00Z"
9595+ }
9696+}
9797+```
9898+9999+Media blobs are automatically enriched with a `url` field pointing to the user's PDS.
100100+101101+### List records
102102+103103+```
104104+GET /xrpc/{method}?limit=20&cursor=0&did=optional
105105+```
106106+107107+| Param | Type | Default | Description |
108108+|-------|------|---------|-------------|
109109+| `limit` | integer | 20 | Max records to return (max 100) |
110110+| `cursor` | string | `0` | Pagination cursor (opaque, pass from previous response) |
111111+| `did` | string | --- | Filter records by DID |
112112+113113+```sh
114114+curl "http://localhost:3000/xrpc/xyz.statusphere.listStatuses?limit=10&did=did:plc:abc"
115115+```
116116+117117+**Response**: `200 OK`
118118+119119+```json
120120+{
121121+ "records": [
122122+ {
123123+ "uri": "at://did:plc:abc/xyz.statusphere.status/abc123",
124124+ "status": "\ud83d\ude0a",
125125+ "createdAt": "2025-01-01T12:00:00Z"
126126+ }
127127+ ],
128128+ "cursor": "10"
129129+}
130130+```
131131+132132+The `cursor` field is present only when more records exist.
133133+134134+## Dynamic procedure endpoints
135135+136136+Procedure endpoints are generated from lexicons with `type: "procedure"`. Without a [Lua script](../guides/scripting), HappyView auto-detects create vs update based on whether the request body contains a `uri` field.
137137+138138+### Create a record
139139+140140+```
141141+POST /xrpc/{method}
142142+```
143143+144144+When the body does **not** contain a `uri` field, a new record is created.
145145+146146+```sh
147147+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.setStatus \
148148+ -H "Authorization: Bearer $TOKEN" \
149149+ -H "Content-Type: application/json" \
150150+ -d '{ "status": "\ud83d\ude0a", "createdAt": "2025-01-01T12:00:00Z" }'
151151+```
152152+153153+HappyView proxies this to the user's PDS as `com.atproto.repo.createRecord`, then indexes the created record locally.
154154+155155+### Update a record
156156+157157+When the body **contains** a `uri` field, the existing record is updated.
158158+159159+```sh
160160+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.setStatus \
161161+ -H "Authorization: Bearer $TOKEN" \
162162+ -H "Content-Type: application/json" \
163163+ -d '{
164164+ "uri": "at://did:plc:abc/xyz.statusphere.status/abc123",
165165+ "status": "\ud83c\udf1f",
166166+ "createdAt": "2025-01-01T13:00:00Z"
167167+ }'
168168+```
169169+170170+HappyView proxies this to the user's PDS as `com.atproto.repo.putRecord`, then upserts the record locally.
171171+172172+**Response** for both: proxied from the user's PDS.
173173+174174+## Errors
175175+176176+All error responses return JSON with an `error` field:
177177+178178+```json
179179+{
180180+ "error": "description of what went wrong"
181181+}
182182+```
183183+184184+| Status | Meaning | Common causes |
185185+|--------|---------|---------------|
186186+| `400 Bad Request` | Invalid input | Missing required fields, malformed JSON, invalid AT URI |
187187+| `401 Unauthorized` | Authentication failed | Missing or invalid Bearer token. See [AIP documentation](https://github.com/graze-social/aip) for token issues |
188188+| `404 Not Found` | Method or record not found | XRPC method has no matching lexicon, or the requested record doesn't exist |
189189+| `500 Internal Server Error` | Server-side failure | Lua script error, database error, or upstream PDS failure |
190190+191191+### Lua script errors
192192+193193+When a Lua script fails, the response is `500` with one of:
194194+195195+- `{"error": "script execution failed"}`: syntax error, runtime error, or missing `handle()` function
196196+- `{"error": "script exceeded execution time limit"}`: the script hit the 1,000,000 instruction limit
197197+198198+The full error details are logged server-side but not exposed to the client. See [Lua Scripting - Debugging](../guides/scripting#debugging) for how to diagnose script issues.
199199+200200+### PDS errors
201201+202202+When a procedure proxies a write to the user's PDS and the PDS returns an error, HappyView forwards the PDS response status code and body directly to the client.
203203+204204+## Next steps
205205+206206+- [Lua Scripting](../guides/scripting): Override the default query and procedure behavior with custom logic
207207+- [Lexicons](../guides/lexicons): Understand how lexicons generate these endpoints
208208+- [Admin API](admin-api): Manage lexicons and monitor your instance
+305
docs/tutorials/statusphere.md
···11+# Tutorial: Statusphere with HappyView
22+33+[Statusphere](https://github.com/bluesky-social/statusphere-example-app) is an example AT Protocol application where users set their current status as a single emoji. It's a great way to learn how HappyView works because the data model is simple but the queries are interesting.
44+55+In this tutorial, you'll set up HappyView to act as the AppView for Statusphere. By the end, you'll have automatically indexed records and automatically generated XPRC endpoints.
66+77+:::tip
88+This tutorial assumes you have a running HappyView instance. If you don't, start with the [Quickstart](../getting-started/deployment/railway) or one of the local development guides ([Docker](../getting-started/deployment/docker), [from source](../getting-started/deployment/other)).
99+:::
1010+1111+## The Statusphere lexicon
1212+1313+Statusphere uses a single record type, `xyz.statusphere.status`. Each record has two fields:
1414+1515+- `status`: a single emoji
1616+- `createdAt`: a timestamp
1717+1818+Users can set their status as many times as they want. Each status is a new record in their repository, keyed by a TID (timestamp-based identifier). The most recent record is their "current" status.
1919+2020+For more background on how the app works, see the [ATProto Statusphere guide](https://atproto.com/guides/applications).
2121+2222+## Step 1: Upload the record lexicon
2323+2424+First, upload the `xyz.statusphere.status` lexicon to HappyView. This tells HappyView to start indexing Statusphere records from across the network as they're created, updated, or deleted.
2525+2626+The examples below use `$TOKEN` as a placeholder for an AIP-issued access token. See [Authentication](../getting-started/authentication) for how to get one.
2727+2828+```sh
2929+curl -X POST http://localhost:3000/admin/lexicons \
3030+ -H "Authorization: Bearer $TOKEN" \
3131+ -H "Content-Type: application/json" \
3232+ -d '{
3333+ "lexicon_json": {
3434+ "lexicon": 1,
3535+ "id": "xyz.statusphere.status",
3636+ "defs": {
3737+ "main": {
3838+ "type": "record",
3939+ "key": "tid",
4040+ "record": {
4141+ "type": "object",
4242+ "required": ["status", "createdAt"],
4343+ "properties": {
4444+ "status": { "type": "string", "maxGraphemes": 1 },
4545+ "createdAt": { "type": "string", "format": "datetime" }
4646+ }
4747+ }
4848+ }
4949+ }
5050+ },
5151+ "backfill": true
5252+ }'
5353+```
5454+5555+HappyView now subscribes to `xyz.statusphere.status` via Tap. The `backfill` flag tells HappyView to also index existing status records from the network. You can monitor progress with `GET /admin/backfill/status` or the [dashboard](../getting-started/dashboard).
5656+5757+:::tip
5858+Since the `xyz.statusphere.status` lexicon is [published on the AT Protocol network](../guides/lexicons#network-lexicons), you can also add it as a network lexicon instead of uploading the JSON manually:
5959+6060+```sh
6161+curl -X POST http://localhost:3000/admin/network-lexicons \
6262+ -H "Authorization: Bearer $TOKEN" \
6363+ -H "Content-Type: application/json" \
6464+ -d '{ "nsid": "xyz.statusphere.status" }'
6565+```
6666+6767+:::
6868+6969+## Step 2: Verify records are being indexed
7070+7171+Once the backfill starts processing, you should see records appearing. Check the stats:
7272+7373+```sh
7474+curl http://localhost:3000/admin/stats \
7575+ -H "Authorization: Bearer $TOKEN"
7676+```
7777+7878+```json
7979+{
8080+ "total_records": 1234,
8181+ "collections": [{ "collection": "xyz.statusphere.status", "count": 1234 }]
8282+}
8383+```
8484+8585+## Step 3: Add a query lexicon for listing statuses
8686+8787+Now add a query endpoint to read the indexed data. Upload a query lexicon with `target_collection` pointing at the record collection from Step 1:
8888+8989+```sh
9090+curl -X POST http://localhost:3000/admin/lexicons \
9191+ -H "Authorization: Bearer $TOKEN" \
9292+ -H "Content-Type: application/json" \
9393+ -d '{
9494+ "lexicon_json": {
9595+ "lexicon": 1,
9696+ "id": "xyz.statusphere.listStatuses",
9797+ "defs": {
9898+ "main": {
9999+ "type": "query",
100100+ "output": { "encoding": "application/json" }
101101+ }
102102+ }
103103+ },
104104+ "target_collection": "xyz.statusphere.status"
105105+ }'
106106+```
107107+108108+This creates a `GET /xrpc/xyz.statusphere.listStatuses` endpoint. Without a Lua script, it uses HappyView's built-in default behavior: listing records with `limit`, `cursor`, and `did` parameters, or fetching a single record by `uri`. Try it:
109109+110110+```sh
111111+curl "http://localhost:3000/xrpc/xyz.statusphere.listStatuses?limit=5"
112112+```
113113+114114+```json
115115+{
116116+ "records": [
117117+ {
118118+ "uri": "at://did:plc:abc/xyz.statusphere.status/3abc123",
119119+ "status": "\ud83d\ude0a",
120120+ "createdAt": "2025-01-01T12:00:00Z"
121121+ },
122122+ {
123123+ "uri": "at://did:plc:def/xyz.statusphere.status/3def456",
124124+ "status": "\ud83c\udf1f",
125125+ "createdAt": "2025-01-01T11:30:00Z"
126126+ }
127127+ ],
128128+ "cursor": "5"
129129+}
130130+```
131131+132132+See [XRPC API](../reference/xrpc-api) for the full default query behavior.
133133+134134+## Step 4: Enhance the query with a Lua script
135135+136136+The default query behavior works, but let's customize it with a [Lua script](../guides/scripting). Here's a script that handles single-record lookups by URI and paginated listing with an optional DID filter:
137137+138138+```lua
139139+function handle()
140140+ if params.uri then
141141+ local record = db.get(params.uri)
142142+ if not record then
143143+ return { error = "not found" }
144144+ end
145145+ return { record = record }
146146+ end
147147+148148+ return db.query({
149149+ collection = collection,
150150+ did = params.did,
151151+ limit = tonumber(params.limit) or 20,
152152+ offset = tonumber(params.cursor) or 0,
153153+ })
154154+end
155155+```
156156+157157+Re-upload the lexicon with parameters defined in the schema and the script attached:
158158+159159+```sh
160160+LEXICON='{
161161+ "lexicon": 1,
162162+ "id": "xyz.statusphere.listStatuses",
163163+ "defs": {
164164+ "main": {
165165+ "type": "query",
166166+ "parameters": {
167167+ "type": "params",
168168+ "properties": {
169169+ "uri": { "type": "string" },
170170+ "did": { "type": "string" },
171171+ "limit": { "type": "integer" },
172172+ "cursor": { "type": "string" }
173173+ }
174174+ },
175175+ "output": { "encoding": "application/json" }
176176+ }
177177+ }
178178+}'
179179+180180+SCRIPT='function handle()
181181+ if params.uri then
182182+ local record = db.get(params.uri)
183183+ if not record then
184184+ return { error = "not found" }
185185+ end
186186+ return { record = record }
187187+ end
188188+189189+ return db.query({
190190+ collection = collection,
191191+ did = params.did,
192192+ limit = tonumber(params.limit) or 20,
193193+ offset = tonumber(params.cursor) or 0,
194194+ })
195195+end'
196196+197197+curl -X POST http://localhost:3000/admin/lexicons \
198198+ -H "Authorization: Bearer $TOKEN" \
199199+ -H "Content-Type: application/json" \
200200+ -d "{
201201+ \"lexicon_json\": $LEXICON,
202202+ \"target_collection\": \"xyz.statusphere.status\",
203203+ \"script\": \"$SCRIPT\"
204204+ }"
205205+```
206206+207207+The endpoint now uses your custom logic. Filter by a specific user:
208208+209209+```sh
210210+curl "http://localhost:3000/xrpc/xyz.statusphere.listStatuses?did=did:plc:abc&limit=1"
211211+```
212212+213213+Fetch a single record by URI:
214214+215215+```sh
216216+curl "http://localhost:3000/xrpc/xyz.statusphere.listStatuses?uri=at://did:plc:abc/xyz.statusphere.status/3abc123"
217217+```
218218+219219+## Step 5: Add a procedure lexicon for setting status
220220+221221+Add a write endpoint so users can set their status through your AppView. This creates a `POST /xrpc/xyz.statusphere.setStatus` endpoint that proxies writes to the user's PDS.
222222+223223+The Lua script auto-fills `createdAt` and uses the authenticated user's DID:
224224+225225+```lua
226226+function handle()
227227+ local r = Record(collection, {
228228+ status = input.status,
229229+ createdAt = now(),
230230+ })
231231+ r:save()
232232+ return { uri = r._uri, cid = r._cid }
233233+end
234234+```
235235+236236+Upload the procedure lexicon with this script:
237237+238238+```sh
239239+LEXICON='{
240240+ "lexicon": 1,
241241+ "id": "xyz.statusphere.setStatus",
242242+ "defs": {
243243+ "main": {
244244+ "type": "procedure",
245245+ "input": { "encoding": "application/json" },
246246+ "output": { "encoding": "application/json" }
247247+ }
248248+ }
249249+}'
250250+251251+SCRIPT='function handle()
252252+ local r = Record(collection, {
253253+ status = input.status,
254254+ createdAt = now(),
255255+ })
256256+ r:save()
257257+ return { uri = r._uri, cid = r._cid }
258258+end'
259259+260260+curl -X POST http://localhost:3000/admin/lexicons \
261261+ -H "Authorization: Bearer $TOKEN" \
262262+ -H "Content-Type: application/json" \
263263+ -d "{
264264+ \"lexicon_json\": $LEXICON,
265265+ \"target_collection\": \"xyz.statusphere.status\",
266266+ \"script\": \"$SCRIPT\"
267267+ }"
268268+```
269269+270270+## Step 6: Test the procedure endpoint
271271+272272+Set a status (requires authentication):
273273+274274+```sh
275275+curl -X POST http://localhost:3000/xrpc/xyz.statusphere.setStatus \
276276+ -H "Authorization: Bearer $TOKEN" \
277277+ -H "Content-Type: application/json" \
278278+ -d '{ "status": "\ud83d\ude80" }'
279279+```
280280+281281+```json
282282+{
283283+ "uri": "at://did:plc:yourDID/xyz.statusphere.status/3xyz789",
284284+ "cid": "bafyreiabc123..."
285285+}
286286+```
287287+288288+The record is created on your PDS and immediately indexed by HappyView.
289289+290290+## What you've built
291291+292292+With three lexicon uploads and a few lines of Lua, you have a complete Statusphere AppView:
293293+294294+- **Real-time indexing** of `xyz.statusphere.status` records from the entire AT Protocol network
295295+- **Historical backfill** of existing status records
296296+- **A query endpoint** (`xyz.statusphere.listStatuses`) with filtering, pagination, and single-record lookups
297297+- **A write endpoint** (`xyz.statusphere.setStatus`) that creates records on the user's PDS and indexes them locally
298298+299299+## Next steps
300300+301301+- [Lua Scripting](../guides/scripting): Explore the full Record and database APIs to build more complex queries
302302+- [Lexicons](../guides/lexicons): Learn about network lexicons, the backfill flag, and target collections
303303+- [XRPC API](../reference/xrpc-api): Understand how the generated endpoints behave
304304+- [Statusphere example app](https://github.com/bluesky-social/statusphere-example-app): See the full Statusphere frontend
305305+- [ATProto Statusphere guide](https://atproto.com/guides/applications): Deep dive into how the app works at the protocol level
-171
docs/xrpc-api.md
···11-# XRPC API
22-33-HappyView dynamically registers XRPC endpoints based on uploaded lexicons. Query lexicons become `GET /xrpc/{nsid}` routes, procedure lexicons become `POST /xrpc/{nsid}` routes.
44-55-## Auth
66-77-- **Queries** (`GET /xrpc/{method}`): unauthenticated
88-- **Procedures** (`POST /xrpc/{method}`): require an AIP-issued `Authorization: Bearer <token>` header
99-- **getProfile**: requires auth
1010-- **uploadBlob**: requires auth
1111-1212----
1313-1414-## Fixed endpoints
1515-1616-### Health check
1717-1818-```
1919-GET /health
2020-```
2121-2222-```sh
2323-curl http://localhost:3000/health
2424-```
2525-2626-**Response**: `200 OK` with body `ok`
2727-2828-### Get profile
2929-3030-```
3131-GET /xrpc/app.bsky.actor.getProfile
3232-```
3333-3434-Returns the authenticated user's profile, resolved from their PDS via PLC directory lookup.
3535-3636-```sh
3737-curl http://localhost:3000/xrpc/app.bsky.actor.getProfile \
3838- -H "Authorization: Bearer $TOKEN"
3939-```
4040-4141-**Response**: `200 OK`
4242-4343-```json
4444-{
4545- "did": "did:plc:abc123",
4646- "handle": "user.bsky.social",
4747- "displayName": "User Name",
4848- "description": "Bio text",
4949- "avatarURL": "https://pds.example.com/xrpc/com.atproto.sync.getBlob?did=did:plc:abc123&cid=bafyabc"
5050-}
5151-```
5252-5353-### Upload blob
5454-5555-```
5656-POST /xrpc/com.atproto.repo.uploadBlob
5757-```
5858-5959-Proxies a blob upload to the authenticated user's PDS. Maximum size: 50MB.
6060-6161-```sh
6262-curl -X POST http://localhost:3000/xrpc/com.atproto.repo.uploadBlob \
6363- -H "Authorization: Bearer $TOKEN" \
6464- -H "Content-Type: image/png" \
6565- --data-binary @image.png
6666-```
6767-6868-**Response**: proxied from the user's PDS.
6969-7070----
7171-7272-## Dynamic query endpoints
7373-7474-Query endpoints are generated from lexicons with `type: "query"`. They support two modes depending on whether a `uri` parameter is provided.
7575-7676-### Single record
7777-7878-```
7979-GET /xrpc/{method}?uri={at-uri}
8080-```
8181-8282-```sh
8383-curl "http://localhost:3000/xrpc/games.gamesgamesgamesgames.listGames?uri=at%3A%2F%2Fdid%3Aplc%3Aabc%2Fgames.gamesgamesgamesgames.game%2Fabc123"
8484-```
8585-8686-**Response**: `200 OK`
8787-8888-```json
8989-{
9090- "record": {
9191- "uri": "at://did:plc:abc/games.gamesgamesgamesgames.game/abc123",
9292- "$type": "games.gamesgamesgamesgames.game",
9393- "title": "My Game"
9494- }
9595-}
9696-```
9797-9898-Media blobs are automatically enriched with a `url` field pointing to the user's PDS.
9999-100100-### List records
101101-102102-```
103103-GET /xrpc/{method}?limit=20&cursor=0&did=optional
104104-```
105105-106106-| Param | Type | Default | Description |
107107-|-------|------|---------|-------------|
108108-| `limit` | integer | 20 | Max records to return (max 100) |
109109-| `cursor` | string | `0` | Pagination cursor (opaque, pass from previous response) |
110110-| `did` | string | --- | Filter records by DID |
111111-112112-```sh
113113-curl "http://localhost:3000/xrpc/games.gamesgamesgamesgames.listGames?limit=10&did=did:plc:abc"
114114-```
115115-116116-**Response**: `200 OK`
117117-118118-```json
119119-{
120120- "records": [
121121- {
122122- "uri": "at://did:plc:abc/games.gamesgamesgamesgames.game/abc123",
123123- "title": "My Game"
124124- }
125125- ],
126126- "cursor": "10"
127127-}
128128-```
129129-130130-The `cursor` field is present only when more records exist.
131131-132132----
133133-134134-## Dynamic procedure endpoints
135135-136136-Procedure endpoints are generated from lexicons with `type: "procedure"`. HappyView auto-detects create vs update based on whether the request body contains a `uri` field.
137137-138138-### Create a record
139139-140140-```
141141-POST /xrpc/{method}
142142-```
143143-144144-When the body does **not** contain a `uri` field, a new record is created.
145145-146146-```sh
147147-curl -X POST http://localhost:3000/xrpc/games.gamesgamesgamesgames.createGame \
148148- -H "Authorization: Bearer $TOKEN" \
149149- -H "Content-Type: application/json" \
150150- -d '{ "title": "My Game" }'
151151-```
152152-153153-HappyView proxies this to the user's PDS as `com.atproto.repo.createRecord`, then indexes the created record locally.
154154-155155-### Update a record
156156-157157-When the body **contains** a `uri` field, the existing record is updated.
158158-159159-```sh
160160-curl -X POST http://localhost:3000/xrpc/games.gamesgamesgamesgames.createGame \
161161- -H "Authorization: Bearer $TOKEN" \
162162- -H "Content-Type: application/json" \
163163- -d '{
164164- "uri": "at://did:plc:abc/games.gamesgamesgamesgames.game/abc123",
165165- "title": "Updated Title"
166166- }'
167167-```
168168-169169-HappyView proxies this to the user's PDS as `com.atproto.repo.putRecord`, then upserts the record locally.
170170-171171-**Response** for both: proxied from the user's PDS.