WIP! A BB-style forum, on the ATmosphere!
We're still working... we'll be back soon when we have something to show off!
node
typescript
hono
htmx
atproto
docs: add SQLite support implementation plan
+1612
+1612
docs/plans/2026-02-24-sqlite-support-plan.md
+1612
docs/plans/2026-02-24-sqlite-support-plan.md
···
1
+
# SQLite Support Implementation Plan
2
+
3
+
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
4
+
5
+
**Goal:** Add SQLite as a fully supported database backend alongside PostgreSQL, replacing the `permissions text[]` column with a normalized `role_permissions` join table that works on both dialects.
6
+
7
+
**Architecture:** `createDb(url)` in `packages/db/src/index.ts` inspects the URL prefix to select postgres.js or @libsql/client. Two Drizzle schema files mirror each other with dialect-appropriate column types. The Postgres schema is the TypeScript source of truth; the SQLite instance is cast at the boundary. The Postgres migration is split into two steps with a data migration script between them to avoid data loss.
8
+
9
+
**Tech Stack:** `@libsql/client`, `drizzle-orm/libsql`, `drizzle-orm/postgres-js`, Drizzle Kit 0.31
10
+
11
+
---
12
+
13
+
## Important notes before starting
14
+
15
+
- **`seed-roles.ts` needs no changes.** It creates AT Proto records with `permissions: string[]` — that's the lexicon format, not the DB schema. The indexer handles DB writes.
16
+
- **`rolePermissions` cascade deletes.** The `onDelete: "cascade"` FK means deleting a role automatically cleans up its `role_permissions` rows. Test cleanup code in `test-context.ts` does not need a separate `rolePermissions` delete.
17
+
- **Two-stage Postgres schema.** You MUST generate migration 0011 (add table) BEFORE removing the `permissions` column from schema.ts. Removing it first would generate a single migration that drops data.
18
+
- **Run commands inside `devenv shell`.** All `pnpm` commands must run inside the Nix devenv shell, or use the absolute path `.devenv/profile/bin/pnpm`.
19
+
- **Test environment.** `DATABASE_URL` must be set for all test runs. For SQLite: `DATABASE_URL=file::memory:`.
20
+
21
+
---
22
+
23
+
## Task 1: Add @libsql/client to packages/db
24
+
25
+
**Files:**
26
+
- Modify: `packages/db/package.json`
27
+
28
+
**Step 1: Add the dependency**
29
+
30
+
In `packages/db/package.json`, add `@libsql/client` to `dependencies`:
31
+
32
+
```json
33
+
"dependencies": {
34
+
"@libsql/client": "^0.14.0",
35
+
"drizzle-orm": "^0.45.1",
36
+
"postgres": "^3.4.8"
37
+
}
38
+
```
39
+
40
+
**Step 2: Install**
41
+
42
+
```sh
43
+
pnpm install
44
+
```
45
+
46
+
Expected: no errors, `@libsql/client` appears in `pnpm-lock.yaml`.
47
+
48
+
**Step 3: Commit**
49
+
50
+
```sh
51
+
git add packages/db/package.json pnpm-lock.yaml
52
+
git commit -m "feat(db): add @libsql/client dependency for SQLite support"
53
+
```
54
+
55
+
---
56
+
57
+
## Task 2: Create packages/db/src/schema.sqlite.ts
58
+
59
+
**Files:**
60
+
- Create: `packages/db/src/schema.sqlite.ts`
61
+
62
+
**Step 1: Write the full SQLite schema**
63
+
64
+
The SQLite schema is identical to the Postgres schema in table names and column names but uses `sqlite-core` helpers. Key differences: `integer({ mode: "bigint" }).primaryKey({ autoIncrement: true })` replaces `bigserial`, `integer({ mode: "timestamp" })` replaces `timestamp({ withTimezone: true })`, and `integer({ mode: "boolean" })` replaces `boolean`. There is no `permissions` column — `role_permissions` table is included from the start.
65
+
66
+
Create `packages/db/src/schema.sqlite.ts`:
67
+
68
+
```typescript
69
+
import {
70
+
sqliteTable,
71
+
text,
72
+
integer,
73
+
uniqueIndex,
74
+
index,
75
+
primaryKey,
76
+
} from "drizzle-orm/sqlite-core";
77
+
78
+
// ── forums ──────────────────────────────────────────────
79
+
export const forums = sqliteTable(
80
+
"forums",
81
+
{
82
+
id: integer("id", { mode: "bigint" }).primaryKey({ autoIncrement: true }),
83
+
did: text("did").notNull(),
84
+
rkey: text("rkey").notNull(),
85
+
cid: text("cid").notNull(),
86
+
name: text("name").notNull(),
87
+
description: text("description"),
88
+
indexedAt: integer("indexed_at", { mode: "timestamp" }).notNull(),
89
+
},
90
+
(table) => [uniqueIndex("forums_did_rkey_idx").on(table.did, table.rkey)]
91
+
);
92
+
93
+
// ── categories ──────────────────────────────────────────
94
+
export const categories = sqliteTable(
95
+
"categories",
96
+
{
97
+
id: integer("id", { mode: "bigint" }).primaryKey({ autoIncrement: true }),
98
+
did: text("did").notNull(),
99
+
rkey: text("rkey").notNull(),
100
+
cid: text("cid").notNull(),
101
+
name: text("name").notNull(),
102
+
description: text("description"),
103
+
slug: text("slug"),
104
+
sortOrder: integer("sort_order"),
105
+
forumId: integer("forum_id", { mode: "bigint" }).references(() => forums.id),
106
+
createdAt: integer("created_at", { mode: "timestamp" }).notNull(),
107
+
indexedAt: integer("indexed_at", { mode: "timestamp" }).notNull(),
108
+
},
109
+
(table) => [
110
+
uniqueIndex("categories_did_rkey_idx").on(table.did, table.rkey),
111
+
]
112
+
);
113
+
114
+
// ── boards ──────────────────────────────────────────────
115
+
export const boards = sqliteTable(
116
+
"boards",
117
+
{
118
+
id: integer("id", { mode: "bigint" }).primaryKey({ autoIncrement: true }),
119
+
did: text("did").notNull(),
120
+
rkey: text("rkey").notNull(),
121
+
cid: text("cid").notNull(),
122
+
name: text("name").notNull(),
123
+
description: text("description"),
124
+
slug: text("slug"),
125
+
sortOrder: integer("sort_order"),
126
+
categoryId: integer("category_id", { mode: "bigint" }).references(() => categories.id),
127
+
categoryUri: text("category_uri").notNull(),
128
+
createdAt: integer("created_at", { mode: "timestamp" }).notNull(),
129
+
indexedAt: integer("indexed_at", { mode: "timestamp" }).notNull(),
130
+
},
131
+
(table) => [
132
+
uniqueIndex("boards_did_rkey_idx").on(table.did, table.rkey),
133
+
index("boards_category_id_idx").on(table.categoryId),
134
+
]
135
+
);
136
+
137
+
// ── users ───────────────────────────────────────────────
138
+
export const users = sqliteTable("users", {
139
+
did: text("did").primaryKey(),
140
+
handle: text("handle"),
141
+
indexedAt: integer("indexed_at", { mode: "timestamp" }).notNull(),
142
+
});
143
+
144
+
// ── memberships ─────────────────────────────────────────
145
+
export const memberships = sqliteTable(
146
+
"memberships",
147
+
{
148
+
id: integer("id", { mode: "bigint" }).primaryKey({ autoIncrement: true }),
149
+
did: text("did").notNull().references(() => users.did),
150
+
rkey: text("rkey").notNull(),
151
+
cid: text("cid").notNull(),
152
+
forumId: integer("forum_id", { mode: "bigint" }).references(() => forums.id),
153
+
forumUri: text("forum_uri").notNull(),
154
+
role: text("role"),
155
+
roleUri: text("role_uri"),
156
+
joinedAt: integer("joined_at", { mode: "timestamp" }),
157
+
createdAt: integer("created_at", { mode: "timestamp" }).notNull(),
158
+
indexedAt: integer("indexed_at", { mode: "timestamp" }).notNull(),
159
+
},
160
+
(table) => [
161
+
uniqueIndex("memberships_did_rkey_idx").on(table.did, table.rkey),
162
+
index("memberships_did_idx").on(table.did),
163
+
]
164
+
);
165
+
166
+
// ── posts ───────────────────────────────────────────────
167
+
export const posts = sqliteTable(
168
+
"posts",
169
+
{
170
+
id: integer("id", { mode: "bigint" }).primaryKey({ autoIncrement: true }),
171
+
did: text("did").notNull().references(() => users.did),
172
+
rkey: text("rkey").notNull(),
173
+
cid: text("cid").notNull(),
174
+
title: text("title"),
175
+
text: text("text").notNull(),
176
+
forumUri: text("forum_uri"),
177
+
boardUri: text("board_uri"),
178
+
boardId: integer("board_id", { mode: "bigint" }).references(() => boards.id),
179
+
rootPostId: integer("root_post_id", { mode: "bigint" }).references((): any => posts.id),
180
+
parentPostId: integer("parent_post_id", { mode: "bigint" }).references((): any => posts.id),
181
+
rootUri: text("root_uri"),
182
+
parentUri: text("parent_uri"),
183
+
createdAt: integer("created_at", { mode: "timestamp" }).notNull(),
184
+
indexedAt: integer("indexed_at", { mode: "timestamp" }).notNull(),
185
+
bannedByMod: integer("banned_by_mod", { mode: "boolean" }).notNull().default(false),
186
+
deletedByUser: integer("deleted_by_user", { mode: "boolean" }).notNull().default(false),
187
+
},
188
+
(table) => [
189
+
uniqueIndex("posts_did_rkey_idx").on(table.did, table.rkey),
190
+
index("posts_forum_uri_idx").on(table.forumUri),
191
+
index("posts_board_id_idx").on(table.boardId),
192
+
index("posts_board_uri_idx").on(table.boardUri),
193
+
index("posts_root_post_id_idx").on(table.rootPostId),
194
+
]
195
+
);
196
+
197
+
// ── mod_actions ─────────────────────────────────────────
198
+
export const modActions = sqliteTable(
199
+
"mod_actions",
200
+
{
201
+
id: integer("id", { mode: "bigint" }).primaryKey({ autoIncrement: true }),
202
+
did: text("did").notNull(),
203
+
rkey: text("rkey").notNull(),
204
+
cid: text("cid").notNull(),
205
+
action: text("action").notNull(),
206
+
subjectDid: text("subject_did"),
207
+
subjectPostUri: text("subject_post_uri"),
208
+
forumId: integer("forum_id", { mode: "bigint" }).references(() => forums.id),
209
+
reason: text("reason"),
210
+
createdBy: text("created_by").notNull(),
211
+
expiresAt: integer("expires_at", { mode: "timestamp" }),
212
+
createdAt: integer("created_at", { mode: "timestamp" }).notNull(),
213
+
indexedAt: integer("indexed_at", { mode: "timestamp" }).notNull(),
214
+
},
215
+
(table) => [
216
+
uniqueIndex("mod_actions_did_rkey_idx").on(table.did, table.rkey),
217
+
index("mod_actions_subject_did_idx").on(table.subjectDid),
218
+
index("mod_actions_subject_post_uri_idx").on(table.subjectPostUri),
219
+
]
220
+
);
221
+
222
+
// ── firehose_cursor ─────────────────────────────────────
223
+
export const firehoseCursor = sqliteTable("firehose_cursor", {
224
+
service: text("service").primaryKey().default("jetstream"),
225
+
cursor: integer("cursor", { mode: "bigint" }).notNull(),
226
+
updatedAt: integer("updated_at", { mode: "timestamp" }).notNull(),
227
+
});
228
+
229
+
// ── roles ───────────────────────────────────────────────
230
+
// No `permissions` column — permissions are stored in role_permissions.
231
+
export const roles = sqliteTable(
232
+
"roles",
233
+
{
234
+
id: integer("id", { mode: "bigint" }).primaryKey({ autoIncrement: true }),
235
+
did: text("did").notNull(),
236
+
rkey: text("rkey").notNull(),
237
+
cid: text("cid").notNull(),
238
+
name: text("name").notNull(),
239
+
description: text("description"),
240
+
priority: integer("priority").notNull(),
241
+
createdAt: integer("created_at", { mode: "timestamp" }).notNull(),
242
+
indexedAt: integer("indexed_at", { mode: "timestamp" }).notNull(),
243
+
},
244
+
(table) => [
245
+
uniqueIndex("roles_did_rkey_idx").on(table.did, table.rkey),
246
+
index("roles_did_idx").on(table.did),
247
+
index("roles_did_name_idx").on(table.did, table.name),
248
+
]
249
+
);
250
+
251
+
// ── role_permissions ─────────────────────────────────────
252
+
// Normalized join table replacing the permissions text[] column.
253
+
// Cascade delete ensures permissions are cleaned up when a role is deleted.
254
+
export const rolePermissions = sqliteTable(
255
+
"role_permissions",
256
+
{
257
+
roleId: integer("role_id", { mode: "bigint" })
258
+
.notNull()
259
+
.references(() => roles.id, { onDelete: "cascade" }),
260
+
permission: text("permission").notNull(),
261
+
},
262
+
(t) => [primaryKey({ columns: [t.roleId, t.permission] })]
263
+
);
264
+
265
+
// ── backfill_progress ───────────────────────────────────
266
+
export const backfillProgress = sqliteTable("backfill_progress", {
267
+
id: integer("id", { mode: "bigint" }).primaryKey({ autoIncrement: true }),
268
+
status: text("status").notNull(),
269
+
backfillType: text("backfill_type").notNull(),
270
+
lastProcessedDid: text("last_processed_did"),
271
+
didsTotal: integer("dids_total").notNull().default(0),
272
+
didsProcessed: integer("dids_processed").notNull().default(0),
273
+
recordsIndexed: integer("records_indexed").notNull().default(0),
274
+
startedAt: integer("started_at", { mode: "timestamp" }).notNull(),
275
+
completedAt: integer("completed_at", { mode: "timestamp" }),
276
+
errorMessage: text("error_message"),
277
+
});
278
+
279
+
// ── backfill_errors ─────────────────────────────────────
280
+
export const backfillErrors = sqliteTable(
281
+
"backfill_errors",
282
+
{
283
+
id: integer("id", { mode: "bigint" }).primaryKey({ autoIncrement: true }),
284
+
backfillId: integer("backfill_id", { mode: "bigint" })
285
+
.notNull()
286
+
.references(() => backfillProgress.id),
287
+
did: text("did").notNull(),
288
+
collection: text("collection").notNull(),
289
+
errorMessage: text("error_message").notNull(),
290
+
createdAt: integer("created_at", { mode: "timestamp" }).notNull(),
291
+
},
292
+
(table) => [index("backfill_errors_backfill_id_idx").on(table.backfillId)]
293
+
);
294
+
```
295
+
296
+
**Step 2: Verify it compiles**
297
+
298
+
```sh
299
+
pnpm --filter @atbb/db build
300
+
```
301
+
302
+
Expected: `dist/` updated with no TypeScript errors.
303
+
304
+
**Step 3: Commit**
305
+
306
+
```sh
307
+
git add packages/db/src/schema.sqlite.ts
308
+
git commit -m "feat(db): add SQLite schema file"
309
+
```
310
+
311
+
---
312
+
313
+
## Task 3: Stage 1 — Add rolePermissions to Postgres schema (keep permissions column)
314
+
315
+
> **Critical:** Do NOT remove the `permissions` column yet. That happens in Task 8 after the data migration runs. Removing it here would collapse both migrations into one, losing existing data.
316
+
317
+
**Files:**
318
+
- Modify: `packages/db/src/schema.ts`
319
+
320
+
**Step 1: Add `primaryKey` to imports and add `rolePermissions` table**
321
+
322
+
In `packages/db/src/schema.ts`:
323
+
324
+
1. Add `primaryKey` to the import from `drizzle-orm/pg-core` (line 1):
325
+
326
+
```typescript
327
+
import {
328
+
pgTable,
329
+
bigserial,
330
+
text,
331
+
timestamp,
332
+
integer,
333
+
boolean,
334
+
bigint,
335
+
uniqueIndex,
336
+
index,
337
+
primaryKey,
338
+
} from "drizzle-orm/pg-core";
339
+
```
340
+
341
+
2. After the `roles` table definition (after line 215), add the `rolePermissions` table:
342
+
343
+
```typescript
344
+
// ── role_permissions ─────────────────────────────────────
345
+
// Normalized join table replacing the permissions text[] column.
346
+
// Cascade delete ensures permissions are cleaned up when a role is deleted.
347
+
export const rolePermissions = pgTable(
348
+
"role_permissions",
349
+
{
350
+
roleId: bigint("role_id", { mode: "bigint" })
351
+
.notNull()
352
+
.references(() => roles.id, { onDelete: "cascade" }),
353
+
permission: text("permission").notNull(),
354
+
},
355
+
(t) => [primaryKey({ columns: [t.roleId, t.permission] })]
356
+
);
357
+
```
358
+
359
+
Do NOT touch the `permissions` column on `roles` yet.
360
+
361
+
**Step 2: Export `rolePermissions` from packages/db/src/index.ts**
362
+
363
+
`packages/db/src/index.ts` line 40 currently does `export * from "./schema.js"` — `rolePermissions` is automatically included. No change needed.
364
+
365
+
**Step 3: Build to verify**
366
+
367
+
```sh
368
+
pnpm --filter @atbb/db build
369
+
```
370
+
371
+
Expected: clean build.
372
+
373
+
**Step 4: Commit**
374
+
375
+
```sh
376
+
git add packages/db/src/schema.ts
377
+
git commit -m "feat(db): add role_permissions table to Postgres schema (permissions column still present)"
378
+
```
379
+
380
+
---
381
+
382
+
## Task 4: Update createDb factory for URL-based dialect detection
383
+
384
+
**Files:**
385
+
- Modify: `packages/db/src/index.ts`
386
+
387
+
**Step 1: Rewrite index.ts**
388
+
389
+
Replace the entire file with:
390
+
391
+
```typescript
392
+
import { drizzle as drizzlePg } from "drizzle-orm/postgres-js";
393
+
import { drizzle as drizzleSqlite } from "drizzle-orm/libsql";
394
+
import { createClient } from "@libsql/client";
395
+
import postgres from "postgres";
396
+
import * as pgSchema from "./schema.js";
397
+
import * as sqliteSchema from "./schema.sqlite.js";
398
+
399
+
/**
400
+
* Create a Drizzle database instance from a connection URL.
401
+
*
402
+
* URL prefix determines the driver:
403
+
* postgres:// or postgresql:// → postgres.js (PostgreSQL)
404
+
* file: → @libsql/client (SQLite file)
405
+
* file::memory: → @libsql/client (SQLite in-memory, tests)
406
+
* libsql:// → @libsql/client (Turso cloud)
407
+
*/
408
+
export function createDb(databaseUrl: string): Database {
409
+
if (databaseUrl.startsWith("postgres")) {
410
+
return drizzlePg(postgres(databaseUrl), { schema: pgSchema }) as Database;
411
+
}
412
+
return drizzleSqlite(
413
+
createClient({ url: databaseUrl }),
414
+
{ schema: sqliteSchema }
415
+
) as unknown as Database;
416
+
}
417
+
418
+
// Database type uses the Postgres schema as the TypeScript source of truth.
419
+
// Both dialects produce identical column names and compatible TypeScript types
420
+
// (bigint for IDs, Date for timestamps), so the cast is safe at the app layer.
421
+
export type Database = ReturnType<typeof drizzlePg<typeof pgSchema>>;
422
+
423
+
export type Transaction = Parameters<Parameters<Database["transaction"]>[0]>[0];
424
+
425
+
export type DbOrTransaction = Database | Transaction;
426
+
427
+
export * from "./schema.js";
428
+
```
429
+
430
+
**Step 2: Build and verify**
431
+
432
+
```sh
433
+
pnpm --filter @atbb/db build
434
+
```
435
+
436
+
Expected: clean build. The `as unknown as Database` cast is intentional — it bridges the Drizzle dialect types at the one boundary where we cross dialects.
437
+
438
+
**Step 3: Commit**
439
+
440
+
```sh
441
+
git add packages/db/src/index.ts
442
+
git commit -m "feat(db): URL-based driver detection in createDb (postgres vs SQLite)"
443
+
```
444
+
445
+
---
446
+
447
+
## Task 5: Set up Drizzle config files and update package.json scripts
448
+
449
+
**Files:**
450
+
- Create: `apps/appview/drizzle.postgres.config.ts`
451
+
- Create: `apps/appview/drizzle.sqlite.config.ts`
452
+
- Modify: `apps/appview/package.json`
453
+
- The old `apps/appview/drizzle.config.ts` will be deleted
454
+
455
+
**Step 1: Create `drizzle.postgres.config.ts`**
456
+
457
+
```typescript
458
+
import { defineConfig } from "drizzle-kit";
459
+
460
+
export default defineConfig({
461
+
schema: "../../packages/db/src/schema.ts",
462
+
out: "./drizzle",
463
+
dialect: "postgresql",
464
+
dbCredentials: {
465
+
url: process.env.DATABASE_URL!,
466
+
},
467
+
});
468
+
```
469
+
470
+
**Step 2: Create `drizzle.sqlite.config.ts`**
471
+
472
+
```typescript
473
+
import { defineConfig } from "drizzle-kit";
474
+
475
+
export default defineConfig({
476
+
schema: "../../packages/db/src/schema.sqlite.ts",
477
+
out: "./drizzle-sqlite",
478
+
dialect: "sqlite",
479
+
dbCredentials: {
480
+
url: process.env.DATABASE_URL!,
481
+
},
482
+
});
483
+
```
484
+
485
+
**Step 3: Update `apps/appview/package.json` scripts**
486
+
487
+
Replace the two `db:*` scripts with four:
488
+
489
+
```json
490
+
"db:generate": "drizzle-kit generate --config=drizzle.postgres.config.ts",
491
+
"db:migrate": "drizzle-kit migrate --config=drizzle.postgres.config.ts",
492
+
"db:generate:sqlite": "drizzle-kit generate --config=drizzle.sqlite.config.ts",
493
+
"db:migrate:sqlite": "drizzle-kit migrate --config=drizzle.sqlite.config.ts"
494
+
```
495
+
496
+
**Step 4: Delete the old config**
497
+
498
+
```sh
499
+
rm apps/appview/drizzle.config.ts
500
+
```
501
+
502
+
**Step 5: Verify both configs parse correctly**
503
+
504
+
```sh
505
+
pnpm --filter @atbb/appview db:generate -- --dry-run 2>&1 | head -5
506
+
```
507
+
508
+
Expected: drizzle-kit starts without config errors (may error on DB connection which is fine for dry-run).
509
+
510
+
**Step 6: Commit**
511
+
512
+
```sh
513
+
git add apps/appview/drizzle.postgres.config.ts apps/appview/drizzle.sqlite.config.ts apps/appview/package.json
514
+
git rm apps/appview/drizzle.config.ts
515
+
git commit -m "feat(appview): add dialect-specific Drizzle configs and update db scripts"
516
+
```
517
+
518
+
---
519
+
520
+
## Task 6: Generate and apply Postgres migration 0011 (add role_permissions table)
521
+
522
+
**Step 1: Generate the migration**
523
+
524
+
```sh
525
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview db:generate
526
+
```
527
+
528
+
Expected: a new file `apps/appview/drizzle/0011_*.sql` is created. Verify its contents — it should `CREATE TABLE role_permissions` and nothing else. If it also tries to `DROP COLUMN permissions`, STOP — the schema.ts has the permissions column removed prematurely. Fix by re-adding permissions to schema.ts before continuing.
529
+
530
+
**Step 2: Apply the migration**
531
+
532
+
```sh
533
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview db:migrate
534
+
```
535
+
536
+
Expected: `role_permissions` table now exists in the database.
537
+
538
+
**Step 3: Verify**
539
+
540
+
Connect to the database and confirm the table exists:
541
+
542
+
```sh
543
+
psql postgres://atbb:atbb@localhost:5432/atbb -c "\d role_permissions"
544
+
```
545
+
546
+
Expected: table with `role_id` (bigint), `permission` (text), primary key on both.
547
+
548
+
**Step 4: Commit**
549
+
550
+
```sh
551
+
git add apps/appview/drizzle/
552
+
git commit -m "feat(db): migration 0011 — add role_permissions table"
553
+
```
554
+
555
+
---
556
+
557
+
## Task 7: Write the data migration script
558
+
559
+
**Files:**
560
+
- Create: `apps/appview/scripts/migrate-permissions.ts`
561
+
562
+
**Step 1: Write the script**
563
+
564
+
```typescript
565
+
/**
566
+
* One-time data migration: copies permissions from roles.permissions[]
567
+
* into the role_permissions join table.
568
+
*
569
+
* Safe to re-run (ON CONFLICT DO NOTHING).
570
+
* Must be run AFTER migration 0011 (role_permissions table exists)
571
+
* and BEFORE migration 0012 (permissions column is dropped).
572
+
*/
573
+
import postgres from "postgres";
574
+
import { drizzle } from "drizzle-orm/postgres-js";
575
+
import * as schema from "../../packages/db/src/schema.js";
576
+
import { sql } from "drizzle-orm";
577
+
578
+
const databaseUrl = process.env.DATABASE_URL;
579
+
if (!databaseUrl) {
580
+
console.error("DATABASE_URL is required");
581
+
process.exit(1);
582
+
}
583
+
584
+
const client = postgres(databaseUrl);
585
+
const db = drizzle(client, { schema });
586
+
587
+
async function run() {
588
+
// Read roles that still have permissions in the array column.
589
+
// We use raw SQL here because the Drizzle schema will have the
590
+
// permissions column removed by the time this script ships.
591
+
const roles = await db.execute(
592
+
sql`SELECT id, permissions FROM roles WHERE array_length(permissions, 1) > 0`
593
+
);
594
+
595
+
if (roles.length === 0) {
596
+
console.log("No roles with permissions to migrate.");
597
+
await client.end();
598
+
return;
599
+
}
600
+
601
+
let totalPermissions = 0;
602
+
603
+
for (const role of roles) {
604
+
const roleId = role.id as bigint;
605
+
const permissions = role.permissions as string[];
606
+
607
+
if (!permissions || permissions.length === 0) continue;
608
+
609
+
// Insert each permission as a row, skip duplicates (idempotent)
610
+
await db.execute(
611
+
sql`INSERT INTO role_permissions (role_id, permission)
612
+
SELECT ${roleId}, unnest(${sql.raw(`ARRAY[${permissions.map(p => `'${p.replace(/'/g, "''")}'`).join(",")}]`)}::text[])
613
+
ON CONFLICT DO NOTHING`
614
+
);
615
+
616
+
totalPermissions += permissions.length;
617
+
console.log(` Role ${roleId}: migrated ${permissions.length} permissions`);
618
+
}
619
+
620
+
console.log(
621
+
`\nMigrated ${totalPermissions} permissions across ${roles.length} roles.`
622
+
);
623
+
console.log("Safe to proceed with migration 0012 (drop permissions column).");
624
+
625
+
await client.end();
626
+
}
627
+
628
+
run().catch((err) => {
629
+
console.error("Migration failed:", err);
630
+
process.exit(1);
631
+
});
632
+
```
633
+
634
+
**Step 2: Add a package.json script to run it**
635
+
636
+
In `apps/appview/package.json`, add:
637
+
638
+
```json
639
+
"migrate-permissions": "tsx --env-file=../../.env scripts/migrate-permissions.ts"
640
+
```
641
+
642
+
**Step 3: Run it against your development database**
643
+
644
+
```sh
645
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview migrate-permissions
646
+
```
647
+
648
+
Expected output: "Migrated N permissions across M roles." or "No roles with permissions to migrate." (both are valid depending on whether you had seeded roles).
649
+
650
+
**Step 4: Verify (optional)**
651
+
652
+
```sh
653
+
psql postgres://atbb:atbb@localhost:5432/atbb -c "SELECT r.name, rp.permission FROM roles r JOIN role_permissions rp ON r.id = rp.role_id ORDER BY r.name, rp.permission"
654
+
```
655
+
656
+
**Step 5: Commit**
657
+
658
+
```sh
659
+
git add apps/appview/scripts/migrate-permissions.ts apps/appview/package.json
660
+
git commit -m "feat(appview): add migrate-permissions data migration script"
661
+
```
662
+
663
+
---
664
+
665
+
## Task 8: Stage 2 — Remove permissions column, generate and apply migration 0012
666
+
667
+
**Files:**
668
+
- Modify: `packages/db/src/schema.ts`
669
+
670
+
**Step 1: Remove the permissions column from roles in schema.ts**
671
+
672
+
In `packages/db/src/schema.ts`, remove the `permissions` line (currently line 205) from the `roles` table:
673
+
674
+
```typescript
675
+
// REMOVE this line:
676
+
permissions: text("permissions").array().notNull().default(sql`'{}'::text[]`),
677
+
```
678
+
679
+
Also remove the `sql` import from `drizzle-orm` if it's no longer used anywhere else in the file. Check by searching for other `sql\`` usages in schema.ts — if none, remove:
680
+
681
+
```typescript
682
+
import { sql } from "drizzle-orm";
683
+
```
684
+
685
+
**Step 2: Build to verify the schema compiles**
686
+
687
+
```sh
688
+
pnpm --filter @atbb/db build
689
+
```
690
+
691
+
Expected: clean build.
692
+
693
+
**Step 3: Generate migration 0012**
694
+
695
+
```sh
696
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview db:generate
697
+
```
698
+
699
+
Expected: a new `apps/appview/drizzle/0012_*.sql` containing only `ALTER TABLE roles DROP COLUMN permissions`.
700
+
701
+
**Step 4: Apply migration 0012**
702
+
703
+
```sh
704
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview db:migrate
705
+
```
706
+
707
+
**Step 5: Verify**
708
+
709
+
```sh
710
+
psql postgres://atbb:atbb@localhost:5432/atbb -c "\d roles"
711
+
```
712
+
713
+
Expected: no `permissions` column in the output.
714
+
715
+
**Step 6: Commit**
716
+
717
+
```sh
718
+
git add packages/db/src/schema.ts apps/appview/drizzle/
719
+
git commit -m "feat(db): migration 0012 — drop permissions column from roles (data already in role_permissions)"
720
+
```
721
+
722
+
---
723
+
724
+
## Task 9: Generate SQLite migrations
725
+
726
+
SQLite deployments always start clean, so a single migration contains the full schema without the `permissions` column.
727
+
728
+
**Step 1: Generate SQLite migrations**
729
+
730
+
Use an in-memory SQLite URL (no actual database needed, drizzle-kit generates from schema):
731
+
732
+
```sh
733
+
DATABASE_URL=file::memory: pnpm --filter @atbb/appview db:generate:sqlite
734
+
```
735
+
736
+
Expected: `apps/appview/drizzle-sqlite/` directory created with `0000_*.sql` containing the full schema including `role_permissions`, no `permissions` column.
737
+
738
+
**Step 2: Inspect the generated migration**
739
+
740
+
```sh
741
+
cat apps/appview/drizzle-sqlite/0000_*.sql
742
+
```
743
+
744
+
Verify: all 11 tables present, `role_permissions` table present, no `permissions` column in `roles`.
745
+
746
+
**Step 3: Commit**
747
+
748
+
```sh
749
+
git add apps/appview/drizzle-sqlite/
750
+
git commit -m "feat(db): add SQLite migrations (single clean initial migration)"
751
+
```
752
+
753
+
---
754
+
755
+
## Task 10: Update checkPermission and getUserRole (TDD)
756
+
757
+
**Files:**
758
+
- Modify: `apps/appview/src/middleware/__tests__/permissions.test.ts`
759
+
- Modify: `apps/appview/src/middleware/permissions.ts`
760
+
761
+
The current `checkPermission` at lines 57 and 62 uses `role.permissions.includes(...)`. After this change, it queries `rolePermissions` directly. The `getUserRole` function at line 91 returns `permissions: string[]` — its return type and select must change.
762
+
763
+
**Step 1: Update the existing failing tests**
764
+
765
+
In `permissions.test.ts`, find all `db.insert(roles).values({...})` calls that include a `permissions` field. For each, you must:
766
+
1. Remove `permissions` from the roles insert
767
+
2. Add a separate `db.insert(rolePermissions).values(...)` for each permission
768
+
769
+
Find all occurrences:
770
+
771
+
```sh
772
+
grep -n "permissions:" apps/appview/src/middleware/__tests__/permissions.test.ts
773
+
```
774
+
775
+
For each role insertion like:
776
+
```typescript
777
+
// OLD: single insert with permissions array
778
+
await ctx.db.insert(roles).values({
779
+
did: ctx.config.forumDid,
780
+
rkey: "member",
781
+
cid: "bafy...",
782
+
name: "Member",
783
+
permissions: ["space.atbb.permission.createPosts"],
784
+
priority: 30,
785
+
createdAt: new Date(),
786
+
indexedAt: new Date(),
787
+
});
788
+
```
789
+
790
+
Replace with:
791
+
```typescript
792
+
// NEW: role insert + separate rolePermissions inserts
793
+
const [memberRole] = await ctx.db.insert(roles).values({
794
+
did: ctx.config.forumDid,
795
+
rkey: "member",
796
+
cid: "bafy...",
797
+
name: "Member",
798
+
priority: 30,
799
+
createdAt: new Date(),
800
+
indexedAt: new Date(),
801
+
}).returning({ id: roles.id });
802
+
803
+
await ctx.db.insert(rolePermissions).values([
804
+
{ roleId: memberRole.id, permission: "space.atbb.permission.createPosts" },
805
+
]);
806
+
```
807
+
808
+
For the wildcard Owner role test:
809
+
```typescript
810
+
await ctx.db.insert(rolePermissions).values([
811
+
{ roleId: ownerRole.id, permission: "*" },
812
+
]);
813
+
```
814
+
815
+
Also add `import { rolePermissions } from "@atbb/db";` to the test file imports.
816
+
817
+
**Step 2: Run the tests to confirm they fail (permissions column no longer exists)**
818
+
819
+
```sh
820
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview test -- --reporter=verbose middleware/__tests__/permissions
821
+
```
822
+
823
+
Expected: compilation errors or test failures because `roles.permissions` no longer exists in the schema.
824
+
825
+
**Step 3: Update permissions.ts**
826
+
827
+
In `apps/appview/src/middleware/permissions.ts`:
828
+
829
+
1. Add `rolePermissions` to the import (line 4):
830
+
```typescript
831
+
import { memberships, roles, rolePermissions } from "@atbb/db";
832
+
```
833
+
834
+
2. Add `or` to the drizzle-orm import (line 5):
835
+
```typescript
836
+
import { eq, and, or } from "drizzle-orm";
837
+
```
838
+
839
+
3. Replace steps 4 and 5 in `checkPermission` (lines 56-62) with a `rolePermissions` query:
840
+
841
+
```typescript
842
+
// 4. Check if user has the permission (wildcard or specific)
843
+
const [match] = await ctx.db
844
+
.select()
845
+
.from(rolePermissions)
846
+
.where(
847
+
and(
848
+
eq(rolePermissions.roleId, role.id),
849
+
or(
850
+
eq(rolePermissions.permission, permission),
851
+
eq(rolePermissions.permission, "*")
852
+
)
853
+
)
854
+
)
855
+
.limit(1);
856
+
857
+
return !!match; // fail-closed: undefined → false
858
+
```
859
+
860
+
4. Update `getUserRole` return type (line 91) — remove `permissions` from the return type:
861
+
862
+
```typescript
863
+
async function getUserRole(
864
+
ctx: AppContext,
865
+
did: string
866
+
): Promise<{ id: bigint; name: string; priority: number } | null> {
867
+
```
868
+
869
+
5. Update the `select` inside `getUserRole` (lines 109-114) — remove `permissions` from the select:
870
+
871
+
```typescript
872
+
const [role] = await ctx.db
873
+
.select({
874
+
id: roles.id,
875
+
name: roles.name,
876
+
priority: roles.priority,
877
+
})
878
+
.from(roles)
879
+
.where(
880
+
and(
881
+
eq(roles.did, ctx.config.forumDid),
882
+
eq(roles.rkey, roleRkey)
883
+
)
884
+
)
885
+
.limit(1);
886
+
```
887
+
888
+
**Step 4: Run the tests again**
889
+
890
+
```sh
891
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview test -- --reporter=verbose middleware/__tests__/permissions
892
+
```
893
+
894
+
Expected: all permissions middleware tests pass.
895
+
896
+
**Step 5: Commit**
897
+
898
+
```sh
899
+
git add apps/appview/src/middleware/permissions.ts apps/appview/src/middleware/__tests__/permissions.test.ts
900
+
git commit -m "feat(appview): update checkPermission and getUserRole to use role_permissions table"
901
+
```
902
+
903
+
---
904
+
905
+
## Task 11: Update indexer for role records (TDD)
906
+
907
+
**Files:**
908
+
- Modify: `apps/appview/src/lib/__tests__/indexer-roles.test.ts`
909
+
- Modify: `apps/appview/src/lib/indexer.ts`
910
+
911
+
The indexer receives role records from the AT Proto firehose. Currently it stores `permissions: record.permissions` in the `roles` table. After this change, it must:
912
+
1. Upsert the role row (no permissions field)
913
+
2. Delete existing `role_permissions` rows for that role (handles permission updates)
914
+
3. Insert new `role_permissions` rows
915
+
916
+
**Step 1: Update the indexer tests**
917
+
918
+
In `indexer-roles.test.ts`:
919
+
920
+
1. Find all assertions like `expect(role.permissions).toEqual([...])` — these will now need to query `rolePermissions`:
921
+
922
+
```typescript
923
+
// OLD
924
+
const [role] = await ctx.db.select().from(roles).where(...);
925
+
expect(role.permissions).toEqual(["space.atbb.permission.createPosts"]);
926
+
927
+
// NEW
928
+
const [role] = await ctx.db.select().from(roles).where(...);
929
+
const perms = await ctx.db
930
+
.select({ permission: rolePermissions.permission })
931
+
.from(rolePermissions)
932
+
.where(eq(rolePermissions.roleId, role.id));
933
+
expect(perms.map(p => p.permission)).toEqual(["space.atbb.permission.createPosts"]);
934
+
```
935
+
936
+
2. Add `import { rolePermissions } from "@atbb/db";` to the test file imports.
937
+
938
+
**Step 2: Run tests to confirm they fail**
939
+
940
+
```sh
941
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview test -- --reporter=verbose lib/__tests__/indexer-roles
942
+
```
943
+
944
+
Expected: test failures (role.permissions no longer exists as a column).
945
+
946
+
**Step 3: Find the role indexing code in indexer.ts**
947
+
948
+
Search for where the indexer inserts into the `roles` table:
949
+
950
+
```sh
951
+
grep -n "roles\|permissions" apps/appview/src/lib/indexer.ts | head -40
952
+
```
953
+
954
+
Find the block that inserts/upserts a role. It likely looks like:
955
+
956
+
```typescript
957
+
// OLD pattern in indexer
958
+
await ctx.db
959
+
.insert(roles)
960
+
.values({
961
+
did: ...,
962
+
rkey: ...,
963
+
cid: ...,
964
+
name: record.name,
965
+
description: record.description,
966
+
permissions: record.permissions, // ← remove this
967
+
priority: record.priority,
968
+
createdAt: new Date(record.createdAt),
969
+
indexedAt: new Date(),
970
+
})
971
+
.onConflictDoUpdate({ target: [roles.did, roles.rkey], set: { ... } });
972
+
```
973
+
974
+
**Step 4: Update the role indexing code**
975
+
976
+
Replace the role upsert block with a three-step operation:
977
+
978
+
```typescript
979
+
// 1. Upsert the role row (no permissions field)
980
+
const [upsertedRole] = await ctx.db
981
+
.insert(roles)
982
+
.values({
983
+
did: record_did, // use whatever variable holds the record author's DID
984
+
rkey: record_rkey, // use whatever variable holds the record key
985
+
cid: record_cid,
986
+
name: record.name,
987
+
description: record.description ?? null,
988
+
priority: record.priority,
989
+
createdAt: new Date(record.createdAt),
990
+
indexedAt: new Date(),
991
+
})
992
+
.onConflictDoUpdate({
993
+
target: [roles.did, roles.rkey],
994
+
set: {
995
+
name: record.name,
996
+
description: record.description ?? null,
997
+
priority: record.priority,
998
+
cid: record_cid,
999
+
indexedAt: new Date(),
1000
+
},
1001
+
})
1002
+
.returning({ id: roles.id });
1003
+
1004
+
// 2. Replace all permissions for this role (handles updates)
1005
+
await ctx.db
1006
+
.delete(rolePermissions)
1007
+
.where(eq(rolePermissions.roleId, upsertedRole.id));
1008
+
1009
+
// 3. Insert new permissions (skip if empty)
1010
+
if (record.permissions && record.permissions.length > 0) {
1011
+
await ctx.db.insert(rolePermissions).values(
1012
+
record.permissions.map((permission: string) => ({
1013
+
roleId: upsertedRole.id,
1014
+
permission,
1015
+
}))
1016
+
);
1017
+
}
1018
+
```
1019
+
1020
+
Also add `rolePermissions` to the indexer's imports from `@atbb/db`.
1021
+
1022
+
**Step 5: Run indexer tests**
1023
+
1024
+
```sh
1025
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview test -- --reporter=verbose lib/__tests__/indexer-roles
1026
+
```
1027
+
1028
+
Expected: all indexer role tests pass.
1029
+
1030
+
**Step 6: Commit**
1031
+
1032
+
```sh
1033
+
git add apps/appview/src/lib/indexer.ts apps/appview/src/lib/__tests__/indexer-roles.test.ts
1034
+
git commit -m "feat(appview): update indexer to store role permissions in role_permissions table"
1035
+
```
1036
+
1037
+
---
1038
+
1039
+
## Task 12: Update admin routes to return permissions via join (TDD)
1040
+
1041
+
**Files:**
1042
+
- Modify: `apps/appview/src/routes/__tests__/admin.test.ts`
1043
+
- Modify: `apps/appview/src/routes/admin.ts`
1044
+
1045
+
The admin routes return role objects with a `permissions` array in the response. Since `getUserRole` no longer returns permissions, any code reading `role.permissions` from that function needs to query `rolePermissions` directly.
1046
+
1047
+
**Step 1: Find all admin.ts usages of permissions**
1048
+
1049
+
```sh
1050
+
grep -n "permissions" apps/appview/src/routes/admin.ts
1051
+
```
1052
+
1053
+
Look for:
1054
+
- Responses that include `role.permissions` (the GET /api/admin/roles endpoint)
1055
+
- The GET /api/admin/members/me endpoint that returns the caller's permissions
1056
+
1057
+
**Step 2: Update the test — ensure role inserts include rolePermissions**
1058
+
1059
+
In `admin.test.ts`, find all `db.insert(roles).values({...})` calls with `permissions` arrays. Apply the same pattern as Task 10 — split into role insert + rolePermissions insert.
1060
+
1061
+
Run the failing tests first:
1062
+
1063
+
```sh
1064
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview test -- --reporter=verbose routes/__tests__/admin
1065
+
```
1066
+
1067
+
**Step 3: Add a helper to fetch a role's permissions**
1068
+
1069
+
In `admin.ts`, wherever a role's permissions are needed in the response, add a helper query:
1070
+
1071
+
```typescript
1072
+
// Fetch permissions for a role by ID
1073
+
async function getRolePermissions(
1074
+
ctx: AppContext,
1075
+
roleId: bigint
1076
+
): Promise<string[]> {
1077
+
const perms = await ctx.db
1078
+
.select({ permission: rolePermissions.permission })
1079
+
.from(rolePermissions)
1080
+
.where(eq(rolePermissions.roleId, roleId));
1081
+
return perms.map((p) => p.permission);
1082
+
}
1083
+
```
1084
+
1085
+
Use this helper when building role response objects:
1086
+
1087
+
```typescript
1088
+
// Instead of: permissions: role.permissions
1089
+
permissions: await getRolePermissions(ctx, role.id),
1090
+
```
1091
+
1092
+
For the GET /api/admin/members/me endpoint returning the caller's permissions:
1093
+
1094
+
```typescript
1095
+
const userRole = await getUserRole(ctx, user.did);
1096
+
const permissions = userRole
1097
+
? await getRolePermissions(ctx, userRole.id)
1098
+
: [];
1099
+
```
1100
+
1101
+
Add `rolePermissions` to admin.ts imports from `@atbb/db`.
1102
+
1103
+
**Step 4: Run admin tests**
1104
+
1105
+
```sh
1106
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview test -- --reporter=verbose routes/__tests__/admin
1107
+
```
1108
+
1109
+
Expected: all admin tests pass.
1110
+
1111
+
**Step 5: Run the full test suite to catch any remaining permissions references**
1112
+
1113
+
```sh
1114
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview test
1115
+
```
1116
+
1117
+
Expected: all tests pass. If any still fail with `role.permissions` errors, find them with:
1118
+
1119
+
```sh
1120
+
grep -rn "\.permissions" apps/appview/src/routes/ apps/appview/src/middleware/ apps/appview/src/lib/
1121
+
```
1122
+
1123
+
Fix any remaining cases using the same `rolePermissions` query pattern.
1124
+
1125
+
**Step 6: Commit**
1126
+
1127
+
```sh
1128
+
git add apps/appview/src/routes/admin.ts apps/appview/src/routes/__tests__/admin.test.ts
1129
+
git commit -m "feat(appview): update admin routes to fetch permissions from role_permissions table"
1130
+
```
1131
+
1132
+
---
1133
+
1134
+
## Task 13: Update test context for SQLite support
1135
+
1136
+
**Files:**
1137
+
- Modify: `apps/appview/src/lib/__tests__/test-context.ts`
1138
+
1139
+
The test context currently hardcodes postgres.js. It needs to detect the URL and use `createDb` for both dialects. For SQLite, it must run migrations programmatically before tests, and skip the manual `sql.end()` call (libsql clients close automatically).
1140
+
1141
+
**Step 1: Update test-context.ts**
1142
+
1143
+
Replace the file with this updated version (preserving all existing cleanup logic):
1144
+
1145
+
```typescript
1146
+
import { eq, or, like } from "drizzle-orm";
1147
+
import { createDb } from "@atbb/db";
1148
+
import { forums, posts, users, categories, memberships, boards, roles, modActions, backfillProgress, backfillErrors } from "@atbb/db";
1149
+
import * as schema from "@atbb/db";
1150
+
import { createLogger } from "@atbb/logger";
1151
+
import path from "path";
1152
+
import { fileURLToPath } from "url";
1153
+
import type { AppConfig } from "../config.js";
1154
+
import type { AppContext } from "../app-context.js";
1155
+
1156
+
const __dirname = fileURLToPath(new URL(".", import.meta.url));
1157
+
1158
+
export interface TestContext extends AppContext {
1159
+
cleanup: () => Promise<void>;
1160
+
cleanDatabase: () => Promise<void>;
1161
+
}
1162
+
1163
+
export interface TestContextOptions {
1164
+
emptyDb?: boolean;
1165
+
}
1166
+
1167
+
export async function createTestContext(
1168
+
options: TestContextOptions = {}
1169
+
): Promise<TestContext> {
1170
+
const config: AppConfig = {
1171
+
port: 3000,
1172
+
forumDid: "did:plc:test-forum",
1173
+
pdsUrl: "https://test.pds",
1174
+
databaseUrl: process.env.DATABASE_URL ?? "",
1175
+
jetstreamUrl: "wss://test.jetstream",
1176
+
logLevel: "warn",
1177
+
oauthPublicUrl: "http://localhost:3000",
1178
+
sessionSecret: "test-secret-at-least-32-characters-long",
1179
+
sessionTtlDays: 7,
1180
+
backfillRateLimit: 10,
1181
+
backfillConcurrency: 10,
1182
+
backfillCursorMaxAgeHours: 48,
1183
+
};
1184
+
1185
+
const db = createDb(config.databaseUrl);
1186
+
1187
+
const isSqlite = !config.databaseUrl.startsWith("postgres");
1188
+
1189
+
// For SQLite: run migrations programmatically before any tests
1190
+
if (isSqlite) {
1191
+
const { migrate } = await import("drizzle-orm/libsql/migrator");
1192
+
const migrationsFolder = path.resolve(__dirname, "../../../../../apps/appview/drizzle-sqlite");
1193
+
await migrate(db as any, { migrationsFolder });
1194
+
}
1195
+
1196
+
const stubFirehose = {
1197
+
start: () => Promise.resolve(),
1198
+
stop: () => Promise.resolve(),
1199
+
} as any;
1200
+
1201
+
const stubOAuthClient = {} as any;
1202
+
const stubOAuthStateStore = { destroy: () => {} } as any;
1203
+
const stubOAuthSessionStore = { destroy: () => {} } as any;
1204
+
const stubCookieSessionStore = { destroy: () => {} } as any;
1205
+
const stubForumAgent = null;
1206
+
1207
+
const cleanDatabase = async () => {
1208
+
if (isSqlite) {
1209
+
// SQLite in-memory: delete in FK order (role_permissions cascade from roles)
1210
+
await db.delete(posts).catch(() => {});
1211
+
await db.delete(memberships).catch(() => {});
1212
+
await db.delete(users).catch(() => {});
1213
+
await db.delete(boards).catch(() => {});
1214
+
await db.delete(categories).catch(() => {});
1215
+
await db.delete(roles).catch(() => {}); // cascades to role_permissions
1216
+
await db.delete(modActions).catch(() => {});
1217
+
await db.delete(backfillErrors).catch(() => {});
1218
+
await db.delete(backfillProgress).catch(() => {});
1219
+
await db.delete(forums).catch(() => {});
1220
+
return;
1221
+
}
1222
+
1223
+
// Postgres: delete by test DID patterns
1224
+
await db.delete(posts).where(eq(posts.did, config.forumDid)).catch(() => {});
1225
+
await db.delete(posts).where(like(posts.did, "did:plc:test-%")).catch(() => {});
1226
+
await db.delete(memberships).where(like(memberships.did, "did:plc:test-%")).catch(() => {});
1227
+
await db.delete(users).where(like(users.did, "did:plc:test-%")).catch(() => {});
1228
+
await db.delete(boards).where(eq(boards.did, config.forumDid)).catch(() => {});
1229
+
await db.delete(categories).where(eq(categories.did, config.forumDid)).catch(() => {});
1230
+
await db.delete(roles).where(eq(roles.did, config.forumDid)).catch(() => {}); // cascades to role_permissions
1231
+
await db.delete(modActions).where(eq(modActions.did, config.forumDid)).catch(() => {});
1232
+
await db.delete(backfillErrors).catch(() => {});
1233
+
await db.delete(backfillProgress).catch(() => {});
1234
+
await db.delete(forums).where(eq(forums.did, config.forumDid)).catch(() => {});
1235
+
};
1236
+
1237
+
await cleanDatabase();
1238
+
1239
+
if (!options.emptyDb) {
1240
+
await db.insert(forums).values({
1241
+
did: config.forumDid,
1242
+
rkey: "self",
1243
+
cid: "bafytest",
1244
+
name: "Test Forum",
1245
+
description: "A test forum",
1246
+
indexedAt: new Date(),
1247
+
});
1248
+
}
1249
+
1250
+
const logger = createLogger({
1251
+
service: "atbb-appview-test",
1252
+
level: "warn",
1253
+
});
1254
+
1255
+
return {
1256
+
db,
1257
+
config,
1258
+
logger,
1259
+
firehose: stubFirehose,
1260
+
oauthClient: stubOAuthClient,
1261
+
oauthStateStore: stubOAuthStateStore,
1262
+
oauthSessionStore: stubOAuthSessionStore,
1263
+
cookieSessionStore: stubCookieSessionStore,
1264
+
forumAgent: stubForumAgent,
1265
+
backfillManager: null,
1266
+
cleanDatabase,
1267
+
cleanup: async () => {
1268
+
const testDidPattern = or(
1269
+
eq(posts.did, "did:plc:test-user"),
1270
+
eq(posts.did, "did:plc:topicsuser"),
1271
+
like(posts.did, "did:plc:test-%"),
1272
+
like(posts.did, "did:plc:duptest-%"),
1273
+
like(posts.did, "did:plc:create-%"),
1274
+
like(posts.did, "did:plc:pds-fail-%")
1275
+
);
1276
+
await db.delete(posts).where(testDidPattern);
1277
+
1278
+
const testMembershipPattern = or(
1279
+
eq(memberships.did, "did:plc:test-user"),
1280
+
eq(memberships.did, "did:plc:topicsuser"),
1281
+
like(memberships.did, "did:plc:test-%"),
1282
+
like(memberships.did, "did:plc:duptest-%"),
1283
+
like(memberships.did, "did:plc:create-%"),
1284
+
like(memberships.did, "did:plc:pds-fail-%")
1285
+
);
1286
+
await db.delete(memberships).where(testMembershipPattern);
1287
+
1288
+
const testUserPattern = or(
1289
+
eq(users.did, "did:plc:test-user"),
1290
+
eq(users.did, "did:plc:topicsuser"),
1291
+
like(users.did, "did:plc:test-%"),
1292
+
like(users.did, "did:plc:duptest-%"),
1293
+
like(users.did, "did:plc:create-%"),
1294
+
like(users.did, "did:plc:pds-fail-%")
1295
+
);
1296
+
await db.delete(users).where(testUserPattern);
1297
+
1298
+
await db.delete(boards).where(eq(boards.did, config.forumDid));
1299
+
await db.delete(categories).where(eq(categories.did, config.forumDid));
1300
+
await db.delete(roles).where(eq(roles.did, config.forumDid)); // cascades to role_permissions
1301
+
await db.delete(modActions).where(eq(modActions.did, config.forumDid));
1302
+
await db.delete(backfillErrors).catch(() => {});
1303
+
await db.delete(backfillProgress).catch(() => {});
1304
+
await db.delete(forums).where(eq(forums.did, config.forumDid));
1305
+
1306
+
// Close connection — only needed for postgres.js
1307
+
if (!isSqlite) {
1308
+
// Access the underlying postgres client via the db instance internals
1309
+
// The createDb function owns the client lifecycle; for tests we accept
1310
+
// that postgres connections close when the process exits.
1311
+
// If connection leak warnings appear, add explicit client tracking to createDb.
1312
+
}
1313
+
},
1314
+
} as TestContext;
1315
+
}
1316
+
```
1317
+
1318
+
> **Note on postgres connection cleanup:** The old test context held a `sql` reference to call `sql.end()`. Since `createDb` now owns the client, tests rely on process exit to close Postgres connections. If this causes "too many connections" errors in CI, refactor `createDb` to return `{ db, close: () => Promise<void> }` and thread `close()` through to the test context cleanup.
1319
+
1320
+
**Step 2: Run the full test suite with Postgres**
1321
+
1322
+
```sh
1323
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm --filter @atbb/appview test
1324
+
```
1325
+
1326
+
Expected: all tests pass.
1327
+
1328
+
**Step 3: Run the full test suite with SQLite**
1329
+
1330
+
```sh
1331
+
DATABASE_URL=file::memory: pnpm --filter @atbb/appview test
1332
+
```
1333
+
1334
+
Expected: all tests pass. If any tests fail due to SQLite behavior differences (e.g., boolean handling, BigInt serialization), fix them individually — the errors will be self-descriptive.
1335
+
1336
+
**Step 4: Commit**
1337
+
1338
+
```sh
1339
+
git add apps/appview/src/lib/__tests__/test-context.ts
1340
+
git commit -m "feat(appview): update test context to support SQLite in-memory testing"
1341
+
```
1342
+
1343
+
---
1344
+
1345
+
## Task 14: Create docker-compose.sqlite.yml
1346
+
1347
+
**Files:**
1348
+
- Create: `docker-compose.sqlite.yml`
1349
+
1350
+
**Step 1: Write the compose file**
1351
+
1352
+
```yaml
1353
+
# SQLite deployment — no external database required.
1354
+
# Data is persisted via a named volume at /data/atbb.db inside the container.
1355
+
#
1356
+
# Usage:
1357
+
# docker compose -f docker-compose.sqlite.yml up
1358
+
#
1359
+
# The existing docker-compose.yml (PostgreSQL) is unchanged.
1360
+
services:
1361
+
appview:
1362
+
build: .
1363
+
environment:
1364
+
DATABASE_URL: file:/data/atbb.db
1365
+
NODE_ENV: production
1366
+
env_file:
1367
+
- .env
1368
+
volumes:
1369
+
- atbb_data:/data
1370
+
ports:
1371
+
- "80:80"
1372
+
restart: unless-stopped
1373
+
1374
+
volumes:
1375
+
atbb_data:
1376
+
driver: local
1377
+
```
1378
+
1379
+
**Step 2: Commit**
1380
+
1381
+
```sh
1382
+
git add docker-compose.sqlite.yml
1383
+
git commit -m "feat: add docker-compose.sqlite.yml for SQLite deployments"
1384
+
```
1385
+
1386
+
---
1387
+
1388
+
## Task 15: Update nix/module.nix
1389
+
1390
+
**Files:**
1391
+
- Modify: `nix/module.nix`
1392
+
1393
+
**Step 1: Read the current module.nix**
1394
+
1395
+
Read `nix/module.nix` in full to find the exact location of:
1396
+
- The `database.enable` option definition
1397
+
- The `database.name` option definition
1398
+
- The `atbb-migrate` service script
1399
+
- The `atbb-appview` environment block
1400
+
- The `services.postgresql` block
1401
+
1402
+
**Step 2: Add `database.type` and `database.path` options**
1403
+
1404
+
Find the `database` attribute set in the options block and add two new options alongside the existing ones:
1405
+
1406
+
```nix
1407
+
type = lib.mkOption {
1408
+
type = lib.types.enum [ "postgresql" "sqlite" ];
1409
+
default = "postgresql";
1410
+
description = "Database backend. Use 'sqlite' for embedded single-file storage without a separate PostgreSQL service.";
1411
+
};
1412
+
1413
+
path = lib.mkOption {
1414
+
type = lib.types.path;
1415
+
default = "/var/lib/atbb/atbb.db";
1416
+
description = "Path to the SQLite database file. Only used when database.type = \"sqlite\".";
1417
+
};
1418
+
```
1419
+
1420
+
**Step 3: Make `database.enable` default to true only for PostgreSQL**
1421
+
1422
+
Find the `database.enable` option and update its default:
1423
+
1424
+
```nix
1425
+
enable = lib.mkOption {
1426
+
type = lib.types.bool;
1427
+
default = cfg.database.type == "postgresql";
1428
+
description = "Enable local PostgreSQL 17 service. Ignored when database.type = \"sqlite\".";
1429
+
};
1430
+
```
1431
+
1432
+
**Step 4: Make the DATABASE_URL conditional in atbb-appview**
1433
+
1434
+
Find the `environment` block in the `atbb-appview` systemd service and update the `DATABASE_URL` line (currently hardcoded to postgres):
1435
+
1436
+
```nix
1437
+
DATABASE_URL = if cfg.database.type == "sqlite"
1438
+
then "file:${cfg.database.path}"
1439
+
else "postgres:///atbb?host=/run/postgresql";
1440
+
```
1441
+
1442
+
**Step 5: Add StateDirectory for SQLite file persistence**
1443
+
1444
+
In the `serviceConfig` block of `atbb-appview`, add:
1445
+
1446
+
```nix
1447
+
StateDirectory = lib.mkIf (cfg.database.type == "sqlite") "atbb";
1448
+
```
1449
+
1450
+
This creates `/var/lib/atbb/` and makes it writable by the service user.
1451
+
1452
+
**Step 6: Make the atbb-migrate script dialect-aware**
1453
+
1454
+
Find the `atbb-migrate` oneshot service and update its `script`:
1455
+
1456
+
```nix
1457
+
script = if cfg.database.type == "sqlite"
1458
+
then "${atbb}/bin/atbb db:migrate:sqlite"
1459
+
else "${atbb}/bin/atbb db:migrate";
1460
+
```
1461
+
1462
+
**Step 7: Make PostgreSQL conditional**
1463
+
1464
+
Find the `services.postgresql` block and wrap it:
1465
+
1466
+
```nix
1467
+
services.postgresql = lib.mkIf (cfg.database.type == "postgresql" && cfg.database.enable) {
1468
+
# ... existing content unchanged
1469
+
};
1470
+
```
1471
+
1472
+
**Step 8: Commit**
1473
+
1474
+
```sh
1475
+
git add nix/module.nix
1476
+
git commit -m "feat(nix): add database.type option to NixOS module (postgresql | sqlite)"
1477
+
```
1478
+
1479
+
---
1480
+
1481
+
## Task 16: Update nix/package.nix
1482
+
1483
+
**Files:**
1484
+
- Modify: `nix/package.nix`
1485
+
1486
+
**Step 1: Find the drizzle migrations copy in installPhase**
1487
+
1488
+
Read `nix/package.nix` and find the line that copies the drizzle directory (search for `drizzle`).
1489
+
1490
+
**Step 2: Add drizzle-sqlite alongside the existing copy**
1491
+
1492
+
Find the line (approximately):
1493
+
```nix
1494
+
cp -r apps/appview/drizzle $out/apps/appview/drizzle
1495
+
```
1496
+
1497
+
Add immediately after:
1498
+
```nix
1499
+
cp -r apps/appview/drizzle-sqlite $out/apps/appview/drizzle-sqlite
1500
+
```
1501
+
1502
+
**Step 3: Commit**
1503
+
1504
+
```sh
1505
+
git add nix/package.nix
1506
+
git commit -m "feat(nix): include SQLite migrations in Nix package output"
1507
+
```
1508
+
1509
+
---
1510
+
1511
+
## Task 17: Update devenv.nix for optional PostgreSQL
1512
+
1513
+
**Files:**
1514
+
- Modify: `devenv.nix`
1515
+
1516
+
**Step 1: Wrap the postgres service in mkDefault**
1517
+
1518
+
Read `devenv.nix` and find the `services.postgres` block. Change `enable = true` (or the implicit enablement) to use `lib.mkDefault`:
1519
+
1520
+
```nix
1521
+
services.postgres = {
1522
+
enable = lib.mkDefault true;
1523
+
# ... rest of existing config unchanged
1524
+
};
1525
+
```
1526
+
1527
+
**Step 2: Add a comment explaining the SQLite override**
1528
+
1529
+
Add a comment above the `services.postgres` block:
1530
+
1531
+
```nix
1532
+
# PostgreSQL is enabled by default for development.
1533
+
# To use SQLite instead, create devenv.local.nix with:
1534
+
# { ... }: { services.postgres.enable = false; }
1535
+
# Then set DATABASE_URL=file:./data/atbb.db in your .env file.
1536
+
```
1537
+
1538
+
**Step 3: Update .env.example to document the SQLite option**
1539
+
1540
+
In `.env.example`, add a comment to the DATABASE_URL line:
1541
+
1542
+
```sh
1543
+
# PostgreSQL (default, used with devenv):
1544
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb
1545
+
1546
+
# SQLite alternative (no devenv postgres needed):
1547
+
# DATABASE_URL=file:./data/atbb.db
1548
+
# DATABASE_URL=file::memory: # in-memory, for tests only
1549
+
```
1550
+
1551
+
**Step 4: Commit**
1552
+
1553
+
```sh
1554
+
git add devenv.nix .env.example
1555
+
git commit -m "feat(devenv): make postgres optional via mkDefault, document SQLite alternative"
1556
+
```
1557
+
1558
+
---
1559
+
1560
+
## Task 18: Final verification
1561
+
1562
+
**Step 1: Run the full test suite with Postgres**
1563
+
1564
+
```sh
1565
+
DATABASE_URL=postgres://atbb:atbb@localhost:5432/atbb pnpm test
1566
+
```
1567
+
1568
+
Expected: all tests pass.
1569
+
1570
+
**Step 2: Run the full test suite with SQLite in-memory**
1571
+
1572
+
```sh
1573
+
DATABASE_URL=file::memory: pnpm test
1574
+
```
1575
+
1576
+
Expected: all tests pass. This is the proof that the abstraction works end-to-end.
1577
+
1578
+
**Step 3: Verify TypeScript builds cleanly**
1579
+
1580
+
```sh
1581
+
pnpm build
1582
+
```
1583
+
1584
+
Expected: clean build with no TypeScript errors.
1585
+
1586
+
**Step 4: Smoke test SQLite file deployment**
1587
+
1588
+
```sh
1589
+
DATABASE_URL=file:./data/atbb.db pnpm --filter @atbb/appview db:migrate:sqlite
1590
+
DATABASE_URL=file:./data/atbb.db pnpm --filter @atbb/appview dev
1591
+
```
1592
+
1593
+
Expected: server starts, migrations run, `/api/healthz` returns 200.
1594
+
1595
+
**Step 5: Clean up the data directory**
1596
+
1597
+
```sh
1598
+
rm -rf ./data/
1599
+
```
1600
+
1601
+
**Step 6: Final commit**
1602
+
1603
+
```sh
1604
+
git add .
1605
+
git commit -m "feat: SQLite support complete — dual-dialect database with role_permissions join table"
1606
+
```
1607
+
1608
+
---
1609
+
1610
+
## Operator upgrade instructions (reference)
1611
+
1612
+
See `docs/plans/2026-02-24-sqlite-support-design.md` → "Operator Migration Instructions" for the three-step Postgres upgrade procedure and NixOS-specific instructions. The key safety rule: **run `migrate-permissions` between applying migration 0011 and migration 0012**.