Exosphere is a set of small, modular, self-hostable community tools built on the AT Protocol.
app.exosphere.site
1# Architecture
2
3## Project Structure
4
5Monorepo using Bun workspaces. Each module (feeds, feature-requests, etc.) is an independent package with its own backend routes and frontend components. Modules share common infrastructure but can be developed and deployed independently.
6
7```
8exosphere/
9├── packages/
10│ ├── core/ # Shared server-side infrastructure (auth, db, types, permissions)
11│ ├── client/ # Shared client-side utilities (API helpers, auth, router, theme, styles)
12│ ├── indexer/ # Jetstream consumer & shared module registry
13│ ├── mcp/ # MCP server framework (JSON-RPC, protocol handling)
14│ ├── feature-requests/ # Feature requests module
15│ ├── feeds/ # Feeds & discussions module
16│ └── app/ # Host app — assembles modules, server & client entry points
17├── bunfig.toml
18└── package.json
19```
20
21The `app` package is the host: it imports the desired modules, mounts their routes and UI, and produces the final deployable artifact. A Sphere operator picks which modules to enable.
22
23## Stack
24
25### Backend
26
27- **Runtime**: Bun
28- **Framework**: Hono.js
29- **Database**: SQLite (via `bun:sqlite`)
30- **Auth**: AT Protocol OAuth via `@atproto/*` packages
31
32### Frontend
33
34- **Framework**: Preact
35- **State management**: Preact Signals
36- **Styling**: vanilla-extract
37
38### Shared (core)
39
40- **Language**: TypeScript with strict types
41- **Validation**: Zod schemas at system boundaries
42
43## Module Design
44
45Each module:
46
47- Exposes a Hono sub-app for its API routes
48- Exposes Preact components for its UI
49- Manages its own SQLite tables (migrations scoped per module)
50- Exposes MCP tools for AI integration
51- Can depend on `core` and `client` but not on other modules
52
53### Anatomy of a Module
54
55A module is a self-contained package that owns its API, UI, database, indexer, and MCP tools. Not every module uses all of these — simpler modules may omit the indexer or MCP tools. The general skeleton looks like this:
56
57```
58packages/<module>/
59├── src/
60│ ├── index.ts # Backend entry — ExosphereModule definition
61│ ├── client.ts # Client entry — ClientModule with lazy-loaded routes
62│ ├── client.ssr.ts # SSR entry — same routes with eager imports
63│ ├── mcp.ts # MCP tool definitions (optional)
64│ ├── indexer.ts # Jetstream event handlers (optional)
65│ ├── api/ # Hono route handlers
66│ ├── db/ # Drizzle table definitions & operations
67│ ├── schemas/ # Zod schemas for validation
68│ └── ui/ # Preact pages, components, hooks, styles
69└── package.json
70```
71
72#### Package exports
73
74Each module declares several entry points in `package.json`, one per concern:
75
76| Export | File | Purpose | Imported by |
77| ---------------- | ------------------- | ----------------------------------------------------------------- | -------------------------- |
78| `"."` | `src/index.ts` | `ExosphereModule` — API routes, indexer, permissions | `app/src/server.ts` |
79| `"./client"` | `src/client.ts` | `ClientModule` — page routes with lazy imports for code splitting | `app/src/client.tsx` |
80| `"./client-ssr"` | `src/client.ssr.ts` | Same routes with eager imports (SSR cannot resolve lazy promises) | `app/src/entry-server.tsx` |
81| `"./mcp"` | `src/mcp.ts` | `McpTool[]` — MCP tool definitions for AI integration | `app/src/server.ts` |
82| `"./types"` | `src/types.ts` | Shared type definitions | Other module files |
83
84#### Backend entry (`src/index.ts`)
85
86The main export implements `ExosphereModule`:
87
88```typescript
89export const featureRequestsModule: ExosphereModule = {
90 name: "feature-requests",
91 api: featureRequestsApi, // Hono sub-app
92 indexer: featureRequestsIndexer, // Jetstream event handlers
93 permissions: {
94 create: { label: "Create feature request", defaultRole: "authenticated" },
95 vote: { label: "Vote on feature requests", defaultRole: "authenticated" },
96 // ...
97 },
98 permissionsCollection: "site.exosphere.featureRequest.permissions",
99};
100```
101
102The `app` package mounts `api` under `/api/<module-name>`, registers the `indexer` with the Jetstream consumer, and loads `permissions` into the admin panel.
103
104#### Client entry (`src/client.ts`)
105
106Declares routes with lazy-loaded page components:
107
108```typescript
109export const featureRequestsModule: ClientModule = {
110 name: "infuse",
111 routes: [
112 { path: "/infuse", component: FeatureRequestsListPage },
113 { path: "/infuse/:number", component: FeatureRequestPage },
114 ],
115};
116```
117
118#### MCP entry (`src/mcp.ts`)
119
120Exports an array of `McpTool` definitions. Each tool has a name, description, JSON Schema for inputs, and a handler that calls the module's API routes via `ApiFetch`:
121
122```typescript
123export const featureRequestMcpTools: McpTool[] = [
124 {
125 name: "list_feature_requests",
126 description: "List feature requests with optional filtering...",
127 inputSchema: {
128 type: "object",
129 properties: {
130 /* ... */
131 },
132 },
133 handler: async (args, apiFetch) => {
134 const res = await apiFetch(`/api/feature-requests?status=${args.status}`);
135 const data = await res.json();
136 return { content: [{ type: "text", text: JSON.stringify(data) }] };
137 },
138 },
139 // ...
140];
141```
142
143The `@exosphere/mcp` package provides the generic MCP framework (JSON-RPC routing, protocol handling). Module tools are passed into `createMcpRoutes()` and merged with core tools (like `get_sphere`).
144
145#### Indexer (`src/indexer.ts`)
146
147Implements `ModuleIndexer` — declares which AT Protocol collections to watch and handles create/update/delete events from the Jetstream consumer:
148
149```typescript
150export const featureRequestsIndexer: ModuleIndexer = {
151 collections: [
152 "site.exosphere.featureRequest.entry",
153 "site.exosphere.featureRequest.vote",
154 // ...
155 ],
156 handleCreateOrUpdate(event) {
157 /* index into SQLite */
158 },
159 handleDelete(event) {
160 /* remove from SQLite */
161 },
162};
163```
164
165#### Database (`src/db/`)
166
167- `schema.ts` — Drizzle table definitions (`featureRequests`, `featureRequestComments`, `featureRequestVotes`, etc.)
168- `operations.ts` — Insert/update/delete helpers used by both the API routes and the indexer
169
170Each module manages its own tables. The shared `core/db` package handles SQLite setup and provides the Drizzle instance.
171
172### Server / Client Separation
173
174Backend and frontend code must stay in separate entry points to avoid bundling server-only dependencies (e.g. `bun:sqlite`) into the browser bundle. `ExosphereModule` (backend) and `ClientModule` (frontend) are independent types — `ClientModule` does not extend `ExosphereModule` to prevent any transitive import of server code.
175
176### Client Module and Route Registration
177
178The `@exosphere/client` package (`packages/client`) provides shared client-side utilities (API helpers, auth state, router, hooks, theme, styles) and the `ClientModule` type:
179
180```typescript
181interface ModuleRoute {
182 path: string; // e.g. "/feeds" or "/feature-requests/:number"
183 component: ComponentType;
184}
185
186interface ClientModule {
187 name: string;
188 routes?: ModuleRoute[];
189}
190```
191
192Each module's `src/client.ts` exports a `ClientModule` that declares its routes and page components. The host app collects all client modules into an array and maps them to `<Route>` elements inside a preact-iso `<Router>` — adding a new module's pages requires only importing its `ClientModule` and appending it to the array.
193
194### Server-Side Rendering (SSR)
195
196The app server-renders pages so the browser receives ready-to-display HTML. SSR runs in both dev (via a Vite plugin) and production (Bun serves pre-built assets).
197
198#### How it works
199
2001. **Template** — the built `index.html` is loaded once at startup. In production, all CSS from the Vite manifest is injected as render-blocking `<link>` tags to prevent FOUC.
2012. **Data prefetch** — for each request, the server resolves auth (from the `sid` cookie), loads the current Sphere, and calls its own API routes internally (`app.request()`) to prefetch page data. Dependent prefetches (e.g. comments after a feature request) run sequentially.
2023. **Render** — `entry-server.tsx` sets Preact signals (`auth`, `sphereState`, `ssrPageData`) from the prefetched data, then calls `preact-iso/prerender` to produce HTML.
2034. **Hydration** — the HTML and serialized `__SSR_DATA__` are injected into the template. The client reads `__SSR_DATA__` to populate signals before hydration, so components skip their initial fetch via `useQuery`'s `initialData` option.
204
205#### Eager vs lazy imports
206
207Module routes use `lazy()` (from `preact-iso`) on the client for code splitting. During SSR, lazy components throw promises that the router can't resolve. Each module therefore exposes a `./client-ssr` entry with direct (eager) imports of the same page components. `entry-server.tsx` imports these eager modules; the client entry uses the default lazy ones.
208
209| Entry point | Module imports | Code splitting |
210| ---------------------- | --------------- | -------------- |
211| `src/client.tsx` | Lazy (`lazy()`) | Yes |
212| `src/entry-server.tsx` | Eager (direct) | N/A |
213
214#### Dev vs production
215
216- **Dev** — a Vite plugin (`vite-ssr-plugin.ts`) intercepts page requests before the SPA fallback. It fetches auth/sphere/page data from the running API server (`localhost:3001`) via HTTP, SSR-renders via `server.ssrLoadModule`, and inlines CSS collected from the module graph.
217- **Production** — the Hono catch-all route in `server.ts` handles SSR. The template and CSS links are prepared once at startup. Data prefetch uses `app.request()` to call API routes in-process (no HTTP round-trip).
218- Both environments share the same prefetch logic (`ssr-prefetch.ts`) — only the transport differs (HTTP in dev, in-process in prod).
219
220## AT Protocol Integration
221
222- Authentication via AT Protocol OAuth (handled in `core/auth`)
223- User data (posts, votes, etc.) is written to each user's PDS
224- The backend indexes relevant data from PDS for fast querying
225- Sphere configuration and membership are published on PDS for interoperability (see below)
226- The local SQLite database caches/indexes all PDS data for fast querying
227- Every record type written to a user's PDS has a formal Lexicon schema definition hosted at `../landing/lexicons`
228
229## Data Ownership
230
231- Logged-in users own their content via their PDS
232- The local SQLite database acts as a cache/index, not the source of truth for user content
233- Sphere configuration lives on the owner's PDS, membership is bilateral (both parties publish records)
234- A third party can reconstruct any public Sphere entirely from PDS data — no dependency on a specific Exosphere instance
235
236### PDS as source of truth
237
238For public Spheres, **no meaningful action should exist only in the local database**. Every user action (creating content, voting, commenting) and every admin action (status changes, moderation, role updates) must be represented as an AT Protocol record on someone's PDS. The local SQLite database indexes these records for fast querying, but it is always rebuildable from PDS data.
239
240This means:
241
242- **User actions** are published on the **user's PDS** (the user owns their data).
243- **Admin actions** that affect other users' content (e.g. removing a comment, changing a feature request's status) are published on the **admin's PDS** — not by modifying or deleting the original author's record. The admin has no authority over another user's PDS repo.
244- **Moderation** follows this pattern: when an admin removes content, they publish a `site.exosphere.moderation` record on their own PDS referencing the content's AT URI. The original content stays on the author's PDS. The local DB marks the content as hidden for fast filtering, but the moderation decision is reconstructable from protocol data.
245
246If a piece of state exists only in SQLite with no corresponding PDS record, it cannot survive a database rebuild and is invisible to third-party indexers. The exception is **private Spheres**, where content intentionally stays off-protocol (AT Protocol repos are public by design).
247
248## Deployment
249
250- Docker container for self-hosting
251- Single image containing both backend and frontend (backend serves static frontend assets)
252
253## Sphere Access Models
254
255A Sphere can be configured as private or public, each with different data storage and access patterns.
256
257### Sphere declaration on PDS
258
259When a Sphere is created, the owner publishes a `site.exosphere.sphere.profile` record on their PDS. This makes the Sphere discoverable by anyone crawling the AT Protocol network and serves as the canonical reference that other records (membership, content) point to via AT URI.
260
261### Private Spheres
262
263Private Spheres are invitation-only. All content is hidden from the public.
264
265- **Authentication**: Required. Users must authenticate with their AT Protocol DID.
266- **Data storage**: All content (posts, reactions, comments, polls, etc.) is stored in the Sphere's local SQLite database — not on users' PDS. AT Protocol repos are public by design, so private content must stay off-protocol.
267- **Sphere record**: The Sphere declaration is still published on the owner's PDS (with `visibility: "private"`), making it discoverable but not its content.
268- **Identity**: Users are still identified by their AT Protocol DID, which provides portable identity and verification without exposing content to the public network.
269
270```
271User (authenticated via AT Proto OAuth)
272 → Sphere API (checks membership)
273 → SQLite (read/write private data)
274```
275
276### Public Spheres
277
278Public Spheres content is readable by anyone. Write access depends on the Sphere's configuration.
279
280#### Authentication
281
282All interactions (posts, reactions, comments) require AT Protocol authentication. Users log in via AT Protocol OAuth and their content is written to their PDS as records using Exosphere's Lexicon schemas. This makes user data portable and user-owned.
283
284Public Spheres are publicly **readable** without authentication — anyone with the URL can view the content. But all **write** actions require a DID.
285
286#### Future: unauthenticated write access
287
288Unauthenticated write access (e.g. anonymous poll votes, guest comments) is deferred to a later phase. It would allow users without an AT Protocol account to interact with specific modules, potentially helping adoption by lowering the entry barrier.
289
290However, it introduces a hybrid data architecture: the AppView would need to merge data from two sources (user PDS records and local SQLite records) into a single unified view. This adds complexity to every query path and leaks into every module's implementation.
291
292When revisited, unauthenticated write should be:
293
294- Opt-in per module (not a core concern)
295- Limited to modules where it clearly adds value (polls, forms) rather than applied broadly
296- Designed with account linking in mind (what happens when an anonymous user later signs up)
297
298#### Write access modes
299
300- **Open**: Any authenticated user can participate (react, comment, post).
301- **Members only**: Public read, permissioned write. Anyone can view the content, but only invited members can interact (react, comment, post).
302
303Important: AT Protocol cannot prevent a user from writing records to their own PDS that reference a Sphere's content. Write restrictions are enforced at the AppView level — only interactions from approved members are indexed and displayed. Non-member interactions are silently ignored.
304
305## AppView and Data Consumption
306
307The AppView is the service that consumes, indexes, and serves AT Protocol data for a Sphere. It is the layer where access control and data aggregation happen.
308
309### Jetstream
310
311Jetstream is a lightweight alternative to the full AT Protocol firehose. Instead of decoding binary CBOR/CAR data, it provides a WebSocket stream of JSON events.
312
313- The AppView connects to a Jetstream instance and subscribes to events matching Exosphere's Lexicon record types.
314- Jetstream can be consumed from Bluesky's public instances, or self-hosted for full independence.
315- Self-hosting Jetstream also requires running a relay. For most deployments, using a public Jetstream instance is sufficient — the Sphere still self-hosts all indexing, storage, and access control.
316
317### Indexing pipeline
318
319The indexer lives in its own package (`packages/indexer`) and can run either **in-process** with the API server or as a **standalone service**. Both modes use the same `startJetstream()` function and share the same module registry (`packages/indexer/src/modules.ts`).
320
321#### Deployment modes
322
323| Mode | Command | Description |
324| ---------------------- | --------------------------------- | ----------------------------------------------------------------------------------------------- |
325| **Combined** (default) | `bun run start` | API server starts the indexer in-process. Simplest deployment — one process, one start command. |
326| **API only** | `DISABLE_INDEXER=1 bun run start` | API server without the Jetstream consumer. Use when the indexer runs separately. |
327| **Indexer only** | `bun run start:indexer` | Standalone Jetstream consumer. Useful for scaling or isolating the indexer from the API. |
328
329The shared module registry (`@exosphere/indexer/modules`) is the single source of truth for which modules are loaded. Both the API server and the standalone indexer import from it, so adding a new module requires updating only one file.
330
331#### Event flow
332
333For Public Spheres with authenticated users, the AppView indexes data from the AT Protocol network:
334
335```
336Jetstream (WebSocket, filtered by Exosphere Lexicons)
337 → Event received (e.g. a reaction record created on a user's PDS)
338 → Is this record referencing a known Sphere? → No → discard
339 → Is this Sphere open or is this user a member? → No → discard
340 → Index the record into SQLite
341 → Available via Sphere API
342```
343
344For Private Spheres and unauthenticated data, there is no Jetstream consumption. Data is written directly to the Sphere database through the API:
345
346```
347User → Sphere API (auth + membership check) → SQLite
348```
349
350### Data flow summary
351
352| Sphere type | Data written to | Data read from | Jetstream needed |
353| ----------- | --------------- | ---------------------- | ---------------- |
354| Private | Sphere SQLite | Sphere SQLite | No |
355| Public | User's PDS | AppView index (SQLite) | Yes |
356
357## Membership Model
358
359Membership uses a **bilateral model** inspired by AT Protocol follows. Both the Sphere admin and the member publish records on their respective PDS, creating a verifiable two-sided proof of membership. The local SQLite database indexes these records for fast access control checks.
360
361### Bilateral membership records
362
363Membership is represented by two complementary AT Protocol records:
364
3651. **`site.exosphere.sphere.memberApproval`** — published on the **owner/admin's PDS** when they invite or approve a member. Contains the Sphere AT URI, the member's DID, and their role.
3662. **`site.exosphere.sphere.member`** — published on the **member's PDS** when they accept the invitation. Contains the Sphere AT URI.
367
368Both records must exist for a membership to be considered fully established. This mirrors how AT Protocol handles social relationships (e.g. follows) and enables third-party indexers to reconstruct the full membership graph from PDS data alone.
369
370### Local index (SQLite)
371
372The `sphere_members` table caches membership state for fast access control:
373
374- **DID**: The user's AT Protocol decentralized identifier
375- **Role**: Owner, admin, member (extensible per Sphere needs)
376- **Status**: Active, invited, revoked
377- **Invited by**: DID of the user who sent the invitation
378- **Joined at**: Timestamp
379
380This table is the authoritative source for real-time access control decisions (API checks, indexer filtering). It is kept in sync with PDS records but allows the server to make fast membership lookups without querying the network.
381
382### Invitation flow
383
3841. A Sphere admin invites a user by their AT Protocol handle or DID.
3852. The handle is resolved to a DID (if needed).
3863. The admin publishes a `sphere.memberApproval` record on their PDS.
3874. An invitation record is created in the local membership table (status: invited).
3885. The invited user accepts via the Sphere UI.
3896. On acceptance, the user publishes a `sphere.member` record on their PDS.
3907. Local status is set to active and the user can participate according to the Sphere's write access mode.
391
392### Private Spheres and membership privacy
393
394For **private Spheres**, the Sphere record itself (`site.exosphere.sphere.profile`) has `visibility: "private"`. The bilateral membership records are still published on PDS (since AT Protocol repos are public), but the Sphere's content remains off-protocol. A third party could see that a user is a member of a private Sphere, but cannot access the Sphere's content. This is an acceptable trade-off — membership is public, content is private — similar to how a private GitHub repository's collaborator list can be partially visible.
395
396If full membership privacy is required, operators can skip PDS writes for membership and rely solely on the local SQLite table. This sacrifices interoperability for privacy.
397
398### Enforcement points
399
400| Layer | Role |
401| --------------------- | -------------------------------------------------------------------------------------------------------------- |
402| **AppView / Indexer** | Filters incoming Jetstream events. Only indexes interactions from active members (for members-only Spheres). |
403| **Sphere API** | Checks membership on write requests. Rejects unauthorized actions before they reach the database. |
404| **Client UI** | Hides write controls (comment box, reaction buttons) for non-members. UX convenience, not a security boundary. |
405
406Membership checks happen at both the API level (for direct writes) and the indexer level (for PDS records arriving via Jetstream). Both paths must enforce the same rules.