WIP! A BB-style forum, on the ATmosphere!
We're still working... we'll be back soon when we have something to show off!
node
typescript
hono
htmx
atproto
1# ATProto Forum Software — Project Plan
2
3**atBB** — A BB-style forum, on the ATmosphere!
4Domain: `atbb.space` (owned) | License: AGPL-3.0 | Org: [atBB-Community](https://github.com/atBB-Community)
5
6---
7
8## Architecture Overview
9
10```
11┌─────────────┐ ┌──────────────┐ ┌─────────────────┐
12│ Forum UI │────▶│ AppView │────▶│ Firehose / │
13│ (Web App) │◀────│ (Node/TS) │◀────│ User PDS nodes │
14└─────────────┘ └──────┬───────┘ └─────────────────┘
15 │
16 ┌──────▼───────┐
17 │ Forum DID │
18 │ (Service │
19 │ Account) │
20 └──────────────┘
21```
22
23**Core principle:** User-generated content (topics, posts, reactions) lives on each user's PDS. Forum metadata (categories, permissions, branding, mod actions) lives on a dedicated Forum Service Account (its own DID/PDS).
24
25---
26
27## High-Level Plan (Full Vision)
28
29### 1. Identity & Ownership
30
31- Each forum instance has a **dedicated DID** (service account) that owns forum-level records.
32- Users authenticate via **AT Proto OAuth**. Their DID is their identity across all forums.
33- A user's membership, posts, and reputation are portable — leave a forum, your content stays in your PDS (though the AppView can stop surfacing it).
34
35### 2. Custom Lexicon Namespace
36
37Namespace: `space.atbb.*` (domain `atbb.space` is owned and under control)
38
39#### Prior Art Audit
40
41**Lexicon repo** ([atBB-Community/lexicon](https://github.com/atBB-Community/lexicon)):
42- Source of truth is YAML (JSON generated via `build_lexicons.sh` + `yq`)
43- Two lexicons defined today:
44 - `space.atbb.forum.forum` — key: `literal:self`, fields: `name` (required), `description` (optional). Note the nested `forum.forum` namespace.
45 - `space.atbb.post` — key: `tid`, fields: `text` (required), `createdAt` (required), `forum` (optional forumRef → strongRef), `reply` (optional replyRef with root + parent → strongRef)
46- **Design decision:** No separate `topic` type. A post without a `reply` ref acts as the topic/thread starter. This follows the same root/parent reply-chain pattern as Bluesky's `app.bsky.feed.post`.
47- Vendors `com.atproto.repo.strongRef` locally
48
49**Rust AppView** ([atBB-Community/atBB](https://github.com/atBB-Community/atBB)):
50- Axum 0.7, SQLx 0.8 (dual Postgres + SQLite), Tera templates, HTMX
51- `jetstream-oxide` for firehose subscription, `atrium-api` 0.24 for AT Proto SDK
52- DB schema: `forums` (id, rkey, cid, name, description) and `posts` (id, rkey, cid) — minimal columns, no user/thread indexing yet
53- Hardcoded test forum data in `server.rs`; routes: `/healthz/*`, `/v1/forums`, web UI at `/`
54- Docker build with `cargo-chef` + `sccache`, CI publishes to GHCR
55- Rust toolchain pinned to 1.82.0
56
57**Monorepo** (`malpercio-dev/atbb-monorepo`):
58- Git submodules under `prior-art/`: atBB, lexicon, and `at-delegation` (the delegation pattern discussed in Future Roadmap)
59
60#### Lexicon Plan
61
62Existing lexicons to **keep as-is** (reuse directly):
63- `space.atbb.forum.forum` — forum metadata
64- `space.atbb.post` — unified post/reply record (no separate topic type)
65
66New lexicons to **define for MVP**:
67| Lexicon | Owner | Description |
68|---|---|---|
69| `space.atbb.forum.category` | Forum DID | Subforum / category definition |
70| `space.atbb.forum.role` | Forum DID | Role definitions (admin, mod, member, custom) |
71| `space.atbb.membership` | User DID | User's membership in a forum + assigned role |
72| `space.atbb.reaction` | User DID | Upvote/like/emoji reaction on a post |
73| `space.atbb.modAction` | Forum DID | Moderation action (ban, mute, pin, lock, delete) |
74
75Deferred to post-MVP:
76| Lexicon | Description |
77|---|---|
78| `space.atbb.crossPost` | Reference to content in another forum |
79
80### 3. Permissions Model
81
82AT Proto has **no native ACLs**, so permissions are enforced at the AppView layer:
83
84- **Write side:** Users can always write records to their PDS. The AppView decides whether to index/surface them.
85- **Read side:** AppView checks role assignments before serving content from restricted categories.
86- **Mod actions:** Stored as records on the Forum DID. The AppView treats these as authoritative (e.g., a ban record means "don't surface this user's content").
87- **Role hierarchy:** Owner → Admin → Moderator → Member → Guest. Configurable per-category.
88
89**MVP shortcut — AppView as trusted intermediary:** The AppView holds the Forum DID's signing keys directly. When an admin or moderator performs an action (create category, ban user, etc.), the AppView verifies their role via its own index, then writes the record to the Forum DID's PDS on their behalf. This is a single point of trust but avoids the need for a delegation layer at launch.
90
91**Future — AT Protocol Privilege Delegation:** The `at.delegation` namespace and Delegation Gateway Server (DGS) pattern would replace this shortcut. Admins and moderators would be Permitted Actors with scoped write access to specific collections on the Forum DID's repo (e.g., a moderator gets delegation for `space.atbb.modAction` but not `space.atbb.category`). This removes the AppView as a key-holding bottleneck and enables multi-server moderation teams. See dedicated section in Future Roadmap.
92
93### 4. Federation & Discovery
94
95- **Shared identity:** Since users have a single DID across forums, any AppView can query "which forums does DID X belong to" by reading their `space.atbb.membership` records. This enables "common forums" features like Discord's mutual servers.
96- **Forum directory:** A well-known registry (could be a shared record collection or a simple directory service) lets forums opt into discoverability.
97- **Cross-posting:** A `crossPost` record references content by AT-URI. The target forum's AppView can choose to index it (with attribution) or ignore it.
98
99### 5. Self-Hosted Model
100
101- Ship as a **Docker Compose stack**: AppView + Postgres + Web UI + optional bundled PDS for the forum service account.
102- Config file defines: forum DID, domain, branding, category structure, default roles.
103- Operator runs their own instance, points DNS at it. No central authority.
104
105---
106
107## Detailed MVP Plan
108
109### MVP Goal
110A single self-hosted forum where users can authenticate with their AT Proto identity, browse categories, create topics, reply, and where admins can perform basic moderation.
111
112### MVP Scope (In / Out)
113
114| In Scope | Out of Scope |
115|---|---|
116| Single forum instance | Multi-forum hosting |
117| AT Proto OAuth login | Cross-posting |
118| Categories (admin-created) | Forum discovery / directory |
119| Create topic, reply (unified `space.atbb.post` — topics are posts without reply ref) | Nested threading |
120| Basic roles: admin, member | Custom roles, per-category permissions |
121| Mod actions: ban user, lock topic, delete post (hide from index) | Rich moderation tools (warnings, temp bans, audit log) |
122| Firehose subscription + indexing | Full-text search |
123| Basic web UI | Rich UI (avatars, signatures, user profiles) |
124| Docker Compose deployment | Helm charts, cloud-native deploy |
125
126### MVP Technical Stack
127
128- **AppView:** Node.js + TypeScript
129- **Framework:** Express or Hono
130- **Database:** PostgreSQL (indexed forum state)
131- **AT Proto libraries:** `@atproto/api`, `@atproto/lexicon`, `@atproto/repo`
132- **Firehose consumer:** `@atproto/sync` (subscribe to repos)
133- **Web UI:** Lightweight — React or even server-rendered (your call, can defer)
134- **Deployment:** Docker Compose
135
136### MVP Milestones
137
138#### Phase 0: Foundation (Week 1–2)
139- [x] Audit existing `space.atbb.*` lexicons — **Result:** 2 existing (`forum.forum`, `post`), 5 new needed for MVP. No separate topic type; unified post model with reply refs.
140- [x] Review prior Rust AppView — **Result:** Axum/SQLx scaffold with jetstream-oxide firehose, minimal DB schema. Reference for route structure and Docker setup; MVP will be Node/TS rewrite.
141- [x] Define new lexicons in YAML: `forum.category`, `forum.role`, `membership`, `reaction`, `modAction` — **Result:** All 5 defined in `packages/lexicon/lexicons/`, with YAML→JSON→TypeScript build pipeline via `@atproto/lex-cli`.
142- [x] Set up project scaffolding: monorepo with `packages/lexicon`, `apps/appview`, `apps/web` — **Result:** Turborepo + pnpm workspaces, devenv for Nix toolchain. AppView (Hono JSON API, port 3000), Web (Hono JSX + HTMX, port 3001). Apps live in `apps/`, shared libraries in `packages/`.
143- [x] Create Forum Service Account (generate DID, set up PDS or use existing hosting) — **Complete:** Forum DID configured in `.env`, PDS credentials set (ATB-5)
144- [x] Spike: write a test record to a PDS, read it back via AT Proto API — **Complete:** Validated write/read/list/delete for all MVP record types (ATB-6)
145
146#### Phase 1: AppView Core (Week 3–4)
147- [x] Implement firehose subscription — connect to relay, filter for `space.atbb.*` records — **Complete:** Production-ready implementation in `apps/appview/src/lib/firehose.ts` with Jetstream WebSocket client, cursor persistence, circuit breaker, and exponential backoff reconnection logic (ATB-9)
148- [x] Build indexer: parse incoming records, write to Postgres — **Complete:** Full implementation in `apps/appview/src/lib/indexer.ts` handles all record types (posts, forums, categories, memberships, modActions) with transaction support. Reactions handlers stubbed pending schema table addition (ATB-10)
149- [x] Database schema: `forums`, `categories`, `users`, `memberships`, `posts` (unified — thread starters have no parent_uri), `mod_actions` — **Complete:** 7 tables defined in `packages/db/src/schema.ts` using Drizzle ORM (includes `firehose_cursor` for subscription state). Migrations in `packages/db/drizzle/` (ATB-7)
150- [x] API endpoints (read path) — **Complete:** Implemented factory pattern with database queries via Drizzle ORM. Routes use dependency injection to access AppContext (ATB-11):
151 - `GET /api/forum` — queries singleton forum record (`rkey='self'`) from `forums` table (`apps/appview/src/routes/forum.ts`)
152 - `GET /api/categories` — lists all categories ordered by `sort_order` (`apps/appview/src/routes/categories.ts`)
153 - `GET /api/topics/:id` — fetches topic post + all replies where `rootPostId = :id`, joins with users, filters deleted posts (`apps/appview/src/routes/topics.ts`)
154 - Factory functions (`createForumRoutes`, `createCategoriesRoutes`, `createTopicsRoutes`) accept `AppContext` parameter for database access
155 - BigInt IDs serialized as strings for JSON compatibility
156 - Defensive parsing with try-catch returns 400 Bad Request for invalid IDs
157 - Comprehensive error handling with try-catch on all database queries
158 - Global error handler in create-app.ts for unhandled errors
159 - Helper functions for serialization (serializeBigInt, serializeDate, serializeAuthor, parseBigIntParam)
160- [x] API endpoints (write path — proxy to user's PDS) — **DONE** (ATB-12):
161 - `POST /api/topics` — create `space.atbb.post` record with `forumRef` but no `reply` ref. Validates text (1-300 graphemes), writes to user's PDS via OAuth agent, returns {uri, cid, rkey} with 201 status. Fire-and-forget design (firehose indexes asynchronously). (`apps/appview/src/routes/topics.ts:13-119`)
162 - `POST /api/posts` — create `space.atbb.post` record with both `forumRef` and `reply` ref. Validates text, parses rootPostId/parentPostId, validates parent belongs to same thread, writes to user's PDS, returns {uri, cid, rkey}. (`apps/appview/src/routes/posts.ts:13-119`)
163 - Helper functions for validation: `validatePostText()` (1-300 graphemes using `@atproto/api` UnicodeString, with type guard for non-string input), `getForumByUri()`, `getPostsByIds()` (bulk lookup with Map), `validateReplyParent()` (thread boundary validation). (`apps/appview/src/routes/helpers.ts:65-190`)
164 - Error handling: Type guards prevent crashes, JSON parsing wrapped in try-catch (400 for malformed), catch blocks re-throw TypeError/ReferenceError (don't swallow programming bugs), network errors (503) vs server errors (500) properly classified. No silent data fabrication (returns null).
165 - Tests: 16 integration tests for POST /api/topics (includes 5 PDS error scenarios), 14 integration tests for POST /api/posts (includes 5 PDS error scenarios), 16 unit tests for helpers. **134 total appview tests passing** (29 new tests for ATB-12). Three comprehensive review rounds completed.
166- [x] **ATB-23: Boards Hierarchy** (EXPANDED from "add categoryUri column") — **Complete:** 2026-02-14
167 - Implemented 3-level hierarchy: Forum → Categories → Boards → Topics
168 - Added `space.atbb.forum.board` lexicon (YAML + generated types)
169 - Schema changes: `boards` table (bigserial id, did, rkey, cid, name, description, categoryUri, sortOrder, indexed_at), added `boardUri` and `boardId` columns to `posts` table with foreign key constraint, migrations `0002_sturdy_maestro.sql` (boards table) and `0003_brief_mariko_yashida.sql` (posts columns)
170 - Posts now link to boards (primary) and forums (redundant for backward compatibility)
171 - New API endpoints with comprehensive tests:
172 - `GET /api/boards` — list all boards across all categories
173 - `GET /api/boards/:id/topics` — list topics for a specific board
174 - `GET /api/categories/:id/boards` — list boards within a category
175 - Updated `POST /api/topics` to require `boardUri` (validates board exists)
176 - Indexer handles `space.atbb.forum.board` records from firehose
177 - Bruno collections updated with new board endpoints
178 - Files: `packages/lexicon/lexicons/space/atbb/forum/board.yaml`, `apps/appview/drizzle/0002_sturdy_maestro.sql`, `apps/appview/drizzle/0003_brief_mariko_yashida.sql`, `apps/appview/src/routes/boards.ts`, `apps/appview/src/routes/__tests__/boards.test.ts`
179
180#### Phase 2: Auth & Membership (Week 5–6)
181- [x] Implement AT Proto OAuth flow (user login via their PDS) — **Complete:** OAuth 2.1 implementation using `@atproto/oauth-client-node` library with PKCE flow, state validation, automatic token refresh, and DPoP. Supports any AT Protocol PDS (not limited to bsky.social). Routes in `apps/appview/src/routes/auth.ts` (ATB-14)
182- [x] On first login: create `membership` record on user's PDS — **Complete:** Fire-and-forget membership creation integrated into OAuth callback. Helper function `createMembershipForUser()` checks for duplicates, writes `space.atbb.membership` record to user's PDS. Login succeeds even if membership creation fails (graceful degradation). 9 tests (5 unit + 4 integration) verify architectural contract. Implementation in `apps/appview/src/lib/membership.ts` and `apps/appview/src/routes/auth.ts:163-188` (ATB-15, PR #27)
183- [x] Session management (JWT or similar, backed by DID verification) — **Complete:** Three-layer session architecture using `@atproto/oauth-client-node` library with OAuth session store (`oauth-stores.ts`), cookie-to-DID mapping (`cookie-session-store.ts`), and HTTP-only cookies. Sessions include DID, handle, PDS URL, access tokens with automatic refresh, expiry. Automatic cleanup every 5 minutes. Authentication middleware (`requireAuth`, `optionalAuth`) implemented in `apps/appview/src/middleware/auth.ts` (ATB-14)
184- [x] Forum DID authenticated agent for server-side PDS writes — **Complete:** `ForumAgent` service authenticates as Forum DID on startup with smart retry logic (network errors retry with exponential backoff, auth errors fail permanently). Integrated into `AppContext` with proactive session refresh every 30 minutes. Graceful degradation (server starts even if auth fails). Health endpoint (`GET /api/health`) exposes granular ForumAgent status. Implementation in `apps/appview/src/lib/forum-agent.ts`, health endpoint in `apps/appview/src/routes/health.ts` (ATB-18)
185- [x] Role assignment: admin can set roles via Forum DID records (ATB-17) — **Complete:** Full permission system implemented with 4 default roles, middleware enforcement, admin endpoints, and role seeding. Files: `apps/appview/src/middleware/permissions.ts`, `apps/appview/src/routes/admin.ts`, `apps/appview/src/lib/seed-roles.ts`, `packages/db/src/schema.ts:188-210` (2026-02-14)
186- [x] Middleware: permission checks on write endpoints — **Complete:** `requirePermission()` and `requireRole()` middleware integrated on all write endpoints (`POST /api/topics`, `POST /api/posts`). Future mod endpoints will use `canActOnUser()` for priority hierarchy enforcement.
187
188#### Phase 3: Moderation Basics (Week 6–7)
189- [x] **ATB-19: Moderation action write-path API endpoints** — **Complete:** 2026-02-16
190 - Implemented 6 moderation endpoints with comprehensive error handling and test coverage
191 - `POST /api/mod/ban` and `DELETE /api/mod/ban/:did` — ban/unban users (requires `banUsers` permission)
192 - `POST /api/mod/lock` and `DELETE /api/mod/lock/:topicId` — lock/unlock topics (requires `lockTopics` permission)
193 - `POST /api/mod/hide` and `DELETE /api/mod/hide/:postId` — hide/unhide posts (requires `moderatePosts` permission)
194 - All endpoints write `space.atbb.modAction` records to Forum DID's PDS via ForumAgent
195 - Idempotent design: returns 200 with `alreadyActive: true` for duplicate actions
196 - Error classification: 400 (validation), 404 (not found), 500 (server error), 503 (network/retry)
197 - Helper functions: `validateReason()` (1-3000 chars), `checkActiveAction()` (query most recent action)
198 - 421 tests total (added 78 new tests) — comprehensive coverage including auth, validation, business logic, infrastructure errors
199 - Files: `apps/appview/src/routes/mod.ts` (~700 lines), `apps/appview/src/routes/__tests__/mod.test.ts` (~3414 lines), `apps/appview/src/lib/errors.ts` (error classification helpers)
200 - Bruno API collection: `bruno/AppView API/Moderation/` (6 .bru files documenting all endpoints)
201- [ ] Admin UI: ban user, lock topic, hide post (ATB-24)
202- [x] **ATB-20: Enforce mod actions in read/write-path API responses** — **Complete:** 2026-02-16
203 - All API read endpoints filter soft-deleted posts (`deleted = false` in all queries)
204 - All API write endpoints (topic/post create) block banned users at request time
205 - `requireNotBanned()` middleware checks `mod_actions` before allowing writes
206 - Comprehensive test coverage for banned user scenarios across all write endpoints
207- [x] **ATB-21: Enforce mod actions in firehose indexer** — **Complete:** 2026-02-17
208 - New `BanEnforcer` class composed into `Indexer` (fail-closed: DB error → treat as banned)
209 - `handlePostCreate`: checks `isBanned(event.did)` before indexing; banned users' posts are silently dropped (never inserted)
210 - `handleModActionCreate`: after inserting a ban mod action, calls `applyBan(subjectDid)` to soft-delete all existing posts — retroactive enforcement
211 - `handleModActionDelete`: read-before-delete in a single transaction; if deleted action was a ban, calls `liftBan(subjectDid)` to restore posts
212 - Decision documented: skip (not soft-mark) for new posts; reuse `deleted` column (no new column); DB query with existing `mod_actions_subject_did_idx` index (no cache)
213 - Race condition handled: post-before-ban path covered by eventual consistency — post inserts normally, `applyBan` soft-deletes it when ban arrives
214 - 8 new integration tests in `indexer.test.ts`; 7 unit tests in `indexer-ban-enforcer.test.ts`; 491 total tests passing
215 - Files: `apps/appview/src/lib/ban-enforcer.ts` (new), `apps/appview/src/lib/indexer.ts` (3 handler overrides), `apps/appview/src/lib/__tests__/indexer-ban-enforcer.test.ts` (new), `apps/appview/src/lib/__tests__/indexer.test.ts` (additions)
216- [ ] Document the trust model: operators must trust their AppView instance, which is acceptable for self-hosted single-server deployments
217
218#### Phase 4: Web UI (Week 7–9)
219- [ ] Forum homepage: category list, recent topics
220- [ ] Category view: paginated topic list, sorted by last reply
221- [ ] Topic view: OP + flat replies, pagination
222- [ ] Compose: new topic form, reply form
223- [ ] Login/logout flow
224- [ ] Admin panel: manage categories, view members, mod actions
225- [ ] Basic responsive design
226
227#### Phase 5: Packaging & Deployment (Week 9–10)
228- [x] Dockerfiles for AppView and Web UI — **Complete:** Multi-stage Dockerfile with Node 22 Alpine, nginx reverse proxy, health checks (ATB-28)
229- [x] Docker Compose with Postgres, AppView, Web UI — **Complete:** `docker-compose.example.yml` with service orchestration (ATB-28)
230- [x] Config file: forum name, domain, admin DID, categories — **Complete:** `.env.example` with all required variables documented
231- [x] README: setup guide, architecture overview — **Complete:** README.md includes setup, architecture diagram, deployment instructions
232- [ ] Seed script for initial forum + categories — **Deferred:** Automated wizard tracked in Future Roadmap
233- [x] Basic health check / status endpoint — **Complete:** `GET /api/healthz` and `GET /api/healthz/ready` implemented (ATB-9)
234
235### Key Risks & Open Questions
236
2371. **~~Lexicon registration~~** ✅ RESOLVED — `atbb.space` domain is owned and controlled. Existing lexicon definitions in `atBB-Community/lexicon` repo may be reusable.
2382. **Firehose filtering:** The AT Proto firehose is *all* records from *all* users. Filtering for your Lexicon types at scale requires thought. For MVP (small user base), naive filtering is fine.
2393. **PDS write path:** Writing records to a user's PDS on their behalf requires proper OAuth scopes. Verify the current state of the AT Proto OAuth spec — it's been evolving.
2404. **Record deletion:** If a user deletes a post from their PDS, the firehose emits a tombstone. Your indexer needs to handle this (soft-delete from index).
2415. **Backfill:** If your AppView goes down and comes back, you need to backfill missed records. AT Proto supports repo sync for this, but it adds complexity.
242
243---
244
245## Future Roadmap (Post-MVP)
246
247### Privilege Delegation (`at.delegation` integration)
248This is the highest-impact architectural upgrade post-MVP. It replaces the "AppView holds Forum DID keys" shortcut with proper scoped delegation.
249
250- **Phase A — DGS Proxy:** Deploy a Delegation Gateway Server that sits in front of the Forum DID's PDS. The AppView routes admin/mod writes through the DGS instead of signing directly. Delegation records (`at.delegation.*`) define which DIDs can write to which collections.
251- **Phase B — Multi-admin:** Multiple admins can write forum metadata and mod actions from their own clients/contexts, not just through the AppView UI. The DGS validates delegation scopes and proxies authorized writes.
252- **Phase C — Federated moderation:** Moderators on different AppView instances (or using third-party tools) can perform mod actions on the Forum DID via delegation, enabling distributed moderation teams.
253- **Phase D — Extract and propose:** Package the delegation spec as a standalone AT Proto proposal with the forum as the reference use case. Contribute upstream.
254
255### Mobile Apps (iOS & Android)
256
257React Native + Expo cross-platform apps consuming the same `/api/*` endpoints as the web UI. Phased rollout: read-only browse → write/interact → push notifications → offline support & app store release. Full plan in [`docs/mobile-apps-plan.md`](mobile-apps-plan.md).
258
259### Other Future Work
260- **Setup wizard for first-time initialization** — Interactive web-based wizard for administrators to initialize a new forum instance (create forum record on PDS, configure categories, set admin roles). Currently requires manual PDS API calls.
261- **User self-deletion endpoint** — Allow users to delete their own posts with two modes: soft-delete (hide from forum index but keep on PDS) and hard-delete (actually remove from their PDS via deleteRecord). Complements moderation soft-delete which only hides content.
262- Nested/threaded replies
263- Full-text search (maybe Meilisearch)
264- User profiles & post history
265- Forum directory / discovery protocol
266- Cross-posting between forums
267- "Common forums" feature (mutual server display)
268- Rich text / markdown in posts
269- File/image attachments via blob storage
270- Custom themes & branding
271- Plugin/extension system
272- Email notifications
273- Push notifications (mobile + web)
274- RSS feeds per category/topic