this repo has no description
Update docs for Elixir appview rewrite [CL-290]
Replace all references to Rust appview (Axum, SQLite, tower_governor,
opake-appview binary) with the Elixir/Phoenix implementation (Bandit,
PostgreSQL, Hammer, OTP supervision tree). Update README, ARCHITECTURE,
and appview docs with new setup instructions and module structure.
+116
-97
+11
-4
README.md
+11
-4
README.md
···
42
42
cargo build --release
43
43
```
44
44
45
-
Produces two binaries: `target/release/opake` (CLI) and `target/release/opake-appview` (indexer/API server).
45
+
Produces `target/release/opake` (CLI). The AppView is a separate Elixir/Phoenix app in `appview/`.
46
46
47
47
## Usage
48
48
···
158
158
159
159
## AppView
160
160
161
-
The AppView is a separate binary (`opake-appview`) that indexes grants and keyrings from the AT Protocol firehose and serves them via a REST API. It enables grant discovery — "what's been shared with me?" — without scanning every PDS in the network.
161
+
The AppView is an Elixir/Phoenix service (`appview/`) that indexes grants and keyrings from the AT Protocol firehose and serves them via a REST API. It enables grant discovery — "what's been shared with me?" — without scanning every PDS in the network.
162
+
163
+
```sh
164
+
cd appview
165
+
docker compose up -d # start postgres
166
+
mix setup && mix phx.server # dev server on :6100
167
+
docker compose --profile full up --build # production-like (postgres + appview)
168
+
```
162
169
163
170
See [docs/appview.md](docs/appview.md) for configuration, authentication, API endpoints, and deployment.
164
171
165
172
## Architecture
166
173
167
-
Four crates + a web frontend:
174
+
Three Rust crates, an Elixir service, and a web frontend:
168
175
169
176
- **`opake-core`** — platform-agnostic library (compiles to WASM). Encryption, records, XRPC client, document operations, `Storage` trait.
170
177
- **`opake-cli`** — CLI binary. `FileStorage` (filesystem-backed), command dispatch.
171
-
- **`opake-appview`** — Axum-based indexer and REST API. Jetstream firehose consumer, SQLite storage, DID-scoped Ed25519 auth.
172
178
- **`opake-derive`** — Proc-macro crate. `RedactedDebug` derive macro for secret-safe Debug output.
179
+
- **`appview/`** — Elixir/Phoenix indexer and REST API. Jetstream firehose consumer (WebSockex), PostgreSQL storage (Ecto), DID-scoped Ed25519 auth via Erlang `:crypto`.
173
180
- **`web/`** — React SPA (Vite + TanStack Router + Tailwind/daisyUI). Uses `opake-core` via WASM. `IndexedDbStorage` (IndexedDB-backed) implements the same `Storage` trait as the CLI.
174
181
175
182
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the encryption model, crate structure, and design decisions. See [docs/FLOWS.md](docs/FLOWS.md) for sequence diagrams of every operation.
+37
-36
docs/ARCHITECTURE.md
+37
-36
docs/ARCHITECTURE.md
···
12
12
end
13
13
14
14
subgraph Server ["AppView (self-hosted)"]
15
-
AppView["opake-appview"]
16
-
SQLite["SQLite"]
15
+
AppView["opake-appview<br/>(Elixir/Phoenix)"]
16
+
Postgres["PostgreSQL"]
17
17
end
18
18
19
19
subgraph Network ["AT Protocol Network"]
···
33
33
Web -->|inbox query| AppView
34
34
35
35
AppView -->|subscribe| Jetstream
36
-
AppView --> SQLite
36
+
AppView --> Postgres
37
37
Jetstream -.->|events from| OwnPDS
38
38
Jetstream -.->|events from| OtherPDS
39
39
···
146
146
logout.rs Remove account
147
147
set_default.rs Switch default account
148
148
149
-
opake-appview/ Indexer + REST API for grant/keyring discovery
150
-
src/
151
-
main.rs Clap app (run/index/serve/status subcommands)
152
-
config.rs AppView config (appview.toml)
153
-
state.rs AppState (Database, indexer status, key cache)
154
-
error.rs Typed error hierarchy (thiserror)
155
-
indexer.rs Event loop: firehose → parse → store
156
-
api/
157
-
mod.rs Axum router (public + protected routes, rate limiting)
158
-
auth.rs DID-scoped Ed25519 auth middleware
159
-
key_cache.rs Signing key cache with TTL
160
-
health.rs GET /api/health (unauthenticated)
161
-
inbox.rs GET /api/inbox (grants by recipient DID)
162
-
keyrings.rs GET /api/keyrings (memberships by DID)
163
-
types.rs API response types
164
-
db/
165
-
mod.rs Database wrapper (SQLite, WAL mode)
166
-
schema.rs Table definitions
167
-
cursor.rs Firehose cursor persistence
168
-
grants.rs Grant upsert/query/delete
169
-
keyrings.rs Keyring member upsert/query/delete
170
-
firehose/
171
-
mod.rs Re-exports
172
-
subscribe.rs WebSocket connection to Jetstream
173
-
events.rs Event parsing → IndexableEvent
174
-
commands/
175
-
mod.rs Shared helpers (build_state, serve_http)
176
-
run.rs Indexer + API (default)
177
-
index.rs Indexer only
178
-
serve.rs API only
179
-
status.rs Print cursor + stats
180
-
181
149
opake-derive/ Proc-macro crate (RedactedDebug derive)
182
150
src/
183
151
lib.rs #[derive(RedactedDebug)] + #[redact] attribute
···
211
179
tests/
212
180
lib/
213
181
indexeddb-storage.test.ts Storage contract tests (fake-indexeddb)
182
+
183
+
appview/ Elixir/Phoenix indexer + REST API (replaces Rust appview)
184
+
lib/
185
+
opake_appview/
186
+
application.ex OTP supervision tree (Repo, KeyCache, Endpoint, Consumer)
187
+
indexer.ex Event dispatch, cursor saving, connection state (ETS)
188
+
release.ex Release tasks (create_db, migrate, rollback, status)
189
+
repo.ex Ecto Repo
190
+
auth/
191
+
plug.ex Opake-Ed25519 header verification (Plug)
192
+
key_cache.ex GenServer + ETS, 5-min TTL per DID
193
+
key_fetcher.ex DID → PDS → publicKey → signingKey resolution
194
+
base64.ex Flexible base64 decode (padded/unpadded)
195
+
jetstream/
196
+
consumer.ex WebSockex client with exponential backoff
197
+
event.ex Jetstream JSON → tagged tuples
198
+
queries/
199
+
cursor_queries.ex Singleton cursor upsert/load
200
+
grant_queries.ex Grant CRUD + inbox pagination
201
+
keyring_queries.ex Keyring member CRUD + membership pagination
202
+
pagination.ex Shared cursor-based pagination helpers
203
+
schemas/
204
+
cursor.ex Singleton cursor (id=1)
205
+
grant.ex Grant (uri PK)
206
+
keyring_member.ex Keyring member (composite PK)
207
+
opake_appview_web/
208
+
router.ex /api/health (public), /api/inbox + /api/keyrings (auth'd)
209
+
endpoint.ex Bandit HTTP, API-only (no sessions/static)
210
+
plugs/rate_limit.ex Hammer ETS rate limiting per IP
211
+
controllers/
212
+
health_controller.ex Indexer status + cursor lag
213
+
inbox_controller.ex Grants by recipient DID
214
+
keyrings_controller.ex Keyrings by member DID
215
+
pagination_helpers.ex Shared param parsing (did, limit, cursor)
214
216
```
215
217
216
218
The boundary is strict: `opake-core` never touches the filesystem, stdin, or any platform-specific API. All I/O happens through the `Storage` trait — `FileStorage` (CLI, filesystem) and `IndexedDbStorage` (web, IndexedDB) implement the same contract with platform-specific backends. This keeps `opake-core` compilable to WASM, which the web frontend uses via `wasm-pack`.
···
352
354
```
353
355
~/.config/opake/
354
356
config.toml CLI config (default DID, account map)
355
-
appview.toml AppView config (jetstream URL, listen addr, db path)
356
357
accounts/
357
358
<did>/
358
359
session.json JWT tokens
+68
-57
docs/appview.md
+68
-57
docs/appview.md
···
2
2
3
3
The AppView indexes `app.opake.grant` and `app.opake.keyring` records from the AT Protocol firehose and serves them via a REST API. It enables the `inbox` command — "what's been shared with me?" — without scanning every PDS in the network.
4
4
5
+
Built with Elixir/Phoenix. Source lives in `appview/`.
6
+
5
7
## Running Modes
6
8
9
+
Controlled by environment variables, not subcommands:
10
+
11
+
| Mode | Config | Effect |
12
+
|------|--------|--------|
13
+
| `run` (default) | — | Indexer + API |
14
+
| `serve` | `INDEXER_ENABLED=false` | API only (no Jetstream) |
15
+
| `index` | `PHX_SERVER=false` | Indexer only (no HTTP) |
16
+
17
+
Status check via release eval:
18
+
7
19
```bash
8
-
opake-appview run # indexer + API (default)
9
-
opake-appview index # indexer only (write-only, no HTTP)
10
-
opake-appview serve # API only (read-only, no Jetstream)
11
-
opake-appview status # print cursor + stats, exit
20
+
bin/opake_appview eval "OpakeAppview.Release.status()"
12
21
```
13
22
14
-
Running with no subcommand is equivalent to `run`.
23
+
## Development
15
24
16
-
### Flags
25
+
```bash
26
+
cd appview
27
+
docker compose up -d # start postgres
28
+
mix setup # deps + create DB + migrate
29
+
mix phx.server # dev server on :6100
30
+
mix test # run tests
31
+
```
32
+
33
+
## Production (Docker)
34
+
35
+
```bash
36
+
cd appview
37
+
docker compose --profile full up --build
38
+
```
17
39
18
-
| Flag | Effect |
19
-
|------|--------|
20
-
| `-v` / `-vv` / `-vvv` | Logging: info / debug / trace |
21
-
| `--config-dir <path>` | Override data directory containing `appview.toml` |
40
+
This starts postgres and the appview container. The entrypoint auto-creates the database and runs migrations.
22
41
23
42
## Configuration
24
43
25
-
TOML file at `~/.config/opake/appview.toml` (or `$XDG_CONFIG_HOME/opake/appview.toml`).
44
+
### Development
26
45
27
-
Override: set `OPAKE_DATA_DIR` to the directory containing `appview.toml`, or use `--config-dir`.
46
+
`config/dev.exs` — defaults to local postgres (`postgres:postgres@localhost/opake_appview_dev`) and the public Jetstream relay.
28
47
29
-
```toml
30
-
jetstream_url = "wss://jetstream2.us-east.bsky.network/subscribe"
31
-
listen = "127.0.0.1:6100"
32
-
db_path = "~/.config/opake/appview.db"
33
-
```
48
+
### Production (environment variables)
34
49
35
-
| Field | Required | Notes |
36
-
|-------|----------|-------|
37
-
| `jetstream_url` | yes | Must start with `ws://` or `wss://` |
38
-
| `listen` | yes | `host:port` for the HTTP server |
39
-
| `db_path` | yes | SQLite path. `~` is expanded. |
50
+
| Variable | Required | Default | Notes |
51
+
|----------|----------|---------|-------|
52
+
| `DATABASE_URL` | yes | — | Ecto URL, e.g. `ecto://user:pass@host/db` |
53
+
| `SECRET_KEY_BASE` | yes | — | 64+ char random string |
54
+
| `JETSTREAM_URL` | no | dev default | Must start with `ws://` or `wss://` |
55
+
| `PORT` | no | `6100` | HTTP listen port |
56
+
| `PHX_HOST` | no | `localhost` | Hostname for URL generation |
57
+
| `PHX_SERVER` | no | `true` | Set to `false` to disable HTTP |
58
+
| `INDEXER_ENABLED` | no | `true` | Set to `false` to disable Jetstream consumer |
59
+
| `POOL_SIZE` | no | `10` | Postgres connection pool size |
60
+
| `ECTO_IPV6` | no | `false` | Enable IPv6 for Postgres connections |
40
61
41
62
## Authentication
42
63
43
64
All API endpoints except `/api/health` require authentication via DID-scoped Ed25519 signatures.
44
-
45
-
Users prove they control a DID by signing with their opake Ed25519 signing key.
46
65
47
66
**Header format:**
48
67
```
···
60
79
```
61
80
62
81
**Verification flow:**
63
-
1. Parse header — extract DID, timestamp, signature
82
+
1. Parse header — extract DID, timestamp, signature (split from right, DIDs contain colons)
64
83
2. Reject if timestamp is >60 seconds from now (replay protection)
65
84
3. Reject if `?did=` parameter doesn't match authenticated DID (scope enforcement)
66
85
4. Fetch `app.opake.publicKey/self` from the user's PDS
67
86
5. Extract `signingKey` (Ed25519) from the record
68
-
6. Verify signature with `ed25519-dalek`
69
-
7. Cache verified key for 5 minutes
70
-
71
-
**CLI side:**
72
-
```rust
73
-
let timestamp = Utc::now().timestamp();
74
-
let message = format!("GET:/api/inbox:{timestamp}:{did}");
75
-
let signature = signing_key.sign(message.as_bytes());
76
-
let header = format!("Opake-Ed25519 {did}:{timestamp}:{}", BASE64.encode(signature.to_bytes()));
77
-
```
87
+
6. Verify signature with Erlang `:crypto` (Ed25519)
88
+
7. Cache verified key in ETS for 5 minutes
78
89
79
90
## API Endpoints
80
91
···
85
96
```json
86
97
{
87
98
"indexerConnected": true,
88
-
"cursorTime": "2026-03-02T12:00:00+00:00",
99
+
"cursorTime": "2026-03-02T12:00:00.000000Z",
89
100
"cursorAgeSecs": 5
90
101
}
91
102
```
···
107
118
"uri": "at://did:plc:owner/app.opake.grant/3abc",
108
119
"ownerDid": "did:plc:owner",
109
120
"documentUri": "at://did:plc:owner/app.opake.document/3xyz",
110
-
"permissions": "read",
111
-
"note": "photos from the trip",
112
121
"createdAt": "2026-03-01T12:00:00Z"
113
122
}
114
123
],
115
-
"cursor": "2026-03-01T12:00:01Z::at://did:plc:owner/app.opake.grant/3abc"
124
+
"cursor": "2026-03-01T12:00:01.000000Z::at://did:plc:owner/app.opake.grant/3abc"
116
125
}
117
-
118
126
```
119
127
120
128
### `GET /api/keyrings?did=<did>&limit=<n>&cursor=<cursor>`
···
127
135
{
128
136
"uri": "at://did:plc:owner/app.opake.keyring/3def",
129
137
"ownerDid": "did:plc:owner",
130
-
"name": "family-photos",
131
-
"indexedAt": "2026-03-01T12:00:00Z"
138
+
"indexedAt": "2026-03-01T12:00:00.000000Z"
132
139
}
133
140
],
134
141
"cursor": "..."
···
137
144
138
145
## Rate Limiting
139
146
140
-
All endpoints are rate-limited per IP via `tower_governor`. Limits: 10 requests/second sustained, 30 burst. Requests beyond the limit receive `429 Too Many Requests`.
147
+
All endpoints are rate-limited per IP via Hammer (ETS backend). Limits: 30 requests/second burst. Requests beyond the limit receive `429 Too Many Requests`.
141
148
142
-
IP extraction checks `X-Forwarded-For`, `X-Real-Ip`, and falls back to peer address — works correctly behind Traefik or similar reverse proxies.
149
+
IP extraction checks `X-Forwarded-For`, `X-Real-Ip`, and falls back to peer address — works correctly behind reverse proxies.
143
150
144
151
## Horizontal Scaling
145
152
146
-
SQLite WAL mode supports one writer + many readers. For horizontal scaling:
153
+
PostgreSQL supports concurrent reads and writes natively. For horizontal scaling:
147
154
148
-
- **One `index` process** — writes to the database
149
-
- **N `serve` processes** — read-only, behind a load balancer
155
+
- **One process with `INDEXER_ENABLED=true`** — consumes the firehose
156
+
- **N processes with `INDEXER_ENABLED=false`** — read-only API servers behind a load balancer
150
157
151
-
All processes point at the same `db_path`. WAL mode handles concurrent reads during writes.
158
+
All processes share the same `DATABASE_URL`.
152
159
153
-
For setups beyond a single machine, migrate to Postgres (future work).
160
+
## Release Tasks
161
+
162
+
Available via `bin/opake_appview eval`:
163
+
164
+
```bash
165
+
# Create the database
166
+
bin/opake_appview eval "OpakeAppview.Release.create_db()"
154
167
155
-
## Status Command
168
+
# Run pending migrations
169
+
bin/opake_appview eval "OpakeAppview.Release.migrate()"
156
170
157
-
Quick operational check without starting a server:
171
+
# Print cursor position, lag, and indexed record counts
172
+
bin/opake_appview eval "OpakeAppview.Release.status()"
158
173
159
-
```
160
-
$ opake-appview status
161
-
Cursor: 2026-03-02T12:00:00+00:00
162
-
Lag: 5s
163
-
Grants: 42
164
-
Keyrings: 3
174
+
# Rollback to a specific migration version
175
+
bin/opake_appview eval "OpakeAppview.Release.rollback(OpakeAppview.Repo, 20260310000001)"
165
176
```