+35 -75

CONTRIBUTING.md

reviewed

··· 1 1 ## Getting started 2 2 3 3 - **Prerequisites:** [Bun](https://bun.sh) and [Node.js](https://nodejs.org) **v22**. Bun runs the appview and unit tests; Node is needed for the local ATProto PDS (`@atproto/pds` requires the `better-sqlite3` native addon which is incompatible with Bun's runtime, and only builds against Node 22). 4 4 - 5 5 - Use a version manager to pin Node 22 for this project — [mise](https://mise.jdx.dev) is recommended: 6 6 - 7 7 - ```bash 8 8 - mise use node@22 # creates .mise.toml, adds node=22 9 9 - ``` 3 3 + Prerequisites: [Bun](https://bun.sh) and [Node.js](https://nodejs.org) v22. Bun runs the appview and tests. Node is needed for the local ATProto PDS (`@atproto/pds` requires `better-sqlite3` which only builds against Node 22). 10 4 11 11 - Or with nvm: `echo "22" > .nvmrc && nvm use`. 5 5 + Use [mise](https://mise.jdx.dev) to pin Node 22: `mise use node@22`. Or nvm: `nvm use 22`. 12 6 13 7 ```bash 14 8 bun install 15 15 - bun run dev # appview server on :3000 (watch mode) 9 9 + bun run dev # appview on :3000 (watch mode) 16 10 bun test # run tests 17 11 bun run lint # check with Biome 18 12 bun run typecheck # tsc --noEmit ··· 20 14 21 15 The database auto-seeds with sample data on first run. To reset, delete `lichen.db` and restart. 22 16 23 23 - ## Development commands 17 17 + ## Full local stack 18 18 + 19 19 + `bun run dev:full` starts a test PDS (Node), the appview, and the firehose subscriber. Two test accounts are created automatically. Log in via: 20 20 + - `http://localhost:3000/dev/login/alice.test` 21 21 + - `http://localhost:3000/dev/login/bob.test` 22 22 + 23 23 + ## All commands 24 24 25 25 ```bash 26 26 bun run dev # appview server (watch mode) 27 27 - bun run dev:full # full local stack: PDS (Node) + appview + firehose 28 28 - bun run dev:firehose # firehose subscriber (separate process) 27 27 + bun run dev:full # full local stack: PDS + appview + firehose 28 28 + bun run dev:firehose # firehose subscriber only 29 29 bun run dev:viz # watch-mode viz bundle 30 30 bun run dev:editor # watch-mode editor bundle 31 31 - bun test # run all tests (unit + integration) 31 31 + bun test # all tests 32 32 bun run test:unit # unit tests only 33 33 - bun run test:integration # integration tests only (requires Node for PDS) 34 34 - bun run lint # check linting/formatting 35 35 - bun run lint:fix # auto-fix lint/format issues 36 36 - bun run typecheck # type check (tsc --noEmit) 33 33 + bun run test:integration # integration tests (requires Node for PDS) 34 34 + bun run lint:fix # auto-fix lint/format 35 35 + bun run typecheck # type check 37 36 bun run build:css # build Tailwind CSS 38 38 - bun run build:viz # bundle viz renderers → public/viz/dist.js 39 39 - bun run build:editor # bundle editor → public/editor/dist.js 37 37 + bun run build:viz # bundle viz renderers 38 38 + bun run build:editor # bundle editor 40 39 ``` 41 40 42 42 - ### Full local stack (`dev:full`) 43 43 - 44 44 - `bun run dev:full` starts a complete local environment: 45 45 - 46 46 - 1. A test PDS running under Node (via `@atproto/dev-env`) 47 47 - 2. The appview server on port 3000 (Bun, watch mode) 48 48 - 3. The firehose subscriber (Bun, watch mode) 49 49 - 50 50 - Two test accounts are created automatically (alice.test, bob.test). Log in via: 51 51 - - `http://localhost:3000/dev/login/alice.test` 52 52 - - `http://localhost:3000/dev/login/bob.test` 53 53 - 54 54 - This sets a `did` cookie directly — no OAuth flow needed. 55 55 - 56 41 ## Project layout 57 42 58 43 ``` 59 44 src/ 60 45 server/ # Elysia routes, database (schema, queries/, seed) 61 61 - atproto/ # ATProto OAuth (client, routes, session, env) + PDS write functions 46 46 + atproto/ # OAuth, PDS write functions 62 47 firehose/ # standalone firehose subscriber 63 63 - lib/ # shared logic (at-uri, diff, markdown, slugs, viz, image, blob refs, ws-polyfill, i18n, urls, response) 64 64 - shared/ # shared type definitions (viz-types) 48 48 + lib/ # shared logic (diff, markdown, slugs, viz, image, blob, i18n, urls...) 49 49 + shared/ # shared type definitions 65 50 views/ # HTML templates 66 51 lexicons/ # ATProto lexicon schemas 67 67 - public/ # static assets (editor, viz renderers, CSS) 68 68 - scripts/ # dev tooling (start-pds.mjs, dev-full.ts) 52 52 + public/ # static assets (editor, viz, CSS) 53 53 + scripts/ # dev tooling 69 54 tests/ # mirrors src/ structure 70 70 - integration/ # end-to-end tests against a real PDS (Node subprocess) 71 55 ``` 72 56 73 57 ## Code style 74 58 75 59 - TypeScript, strict mode 76 76 - - Biome for formatting (tabs, double quotes) — run `bun run lint:fix` before committing 60 60 + - Biome for formatting (tabs, double quotes), run `bun run lint:fix` before committing 77 61 - Named exports, no default exports 78 62 - Functions over classes unless state is needed 79 79 - - Prefer `const` and immutable patterns 80 80 - - Error handling: throw typed error classes (`WikiNotFoundError`, `AccessDeniedError`, `PdsWriteError`), caught by a single Elysia `onError` hook — route handlers never catch errors themselves 81 81 - - Keep route handlers thin: orchestrators in `src/lib/orchestrators/` own the PDS→DB lifecycle per domain; routes parse input, call the orchestrator, and redirect 82 82 - - PDS is canonical: PDS write must succeed before DB write. If PDS fails, the request fails (no partial state) 83 83 - - No ORMs — raw SQL with prepared statements 84 84 - - Route param names must be consistent across all routes sharing a path segment 85 85 - - UI strings must use the i18n system (`t(locale).section.key`) — no hardcoded English strings in views 86 86 - - Tailwind class names must come from `src/views/theme.ts` (`THEME.*`) — no hardcoded class strings in views. Client-side hex colors come from `THEME_HEX` 63 63 + - Throw typed error classes (`WikiNotFoundError`, `AccessDeniedError`, `PdsWriteError`), caught by a single `onError` hook. Route handlers never catch errors themselves. 64 64 + - Route handlers stay thin: orchestrators in `src/lib/orchestrators/` own the PDS-then-DB lifecycle. Routes parse input, call the orchestrator, redirect. 65 65 + - PDS write must succeed before DB write. If PDS fails, the request fails. 66 66 + - No ORMs, raw SQL with prepared statements 67 67 + - UI strings go through `src/lib/i18n/`, no hardcoded English in views 68 68 + - Tailwind classes come from `src/views/theme.ts` (`THEME.*`), no hardcoded class strings in views 87 69 88 70 ## Database 89 71 90 90 - SQLite via `bun:sqlite`. Schema defined in `src/server/db/schema.ts`. 91 91 - 92 92 - Tables: `wikis`, `notes`, `revisions`, `snapshots`, `memberships`, `requests`, `current_note`, `backlinks`, `blobs`, `firehose_cursor`, `oauth_sessions`, `oauth_state` 93 93 - 94 94 - - Schema created via `CREATE TABLE IF NOT EXISTS` on startup 95 95 - - All queries use prepared statements (`src/server/db/queries/`) 96 96 - - Write operations wrap related changes in transactions 97 97 - - Auto-seeds with sample data if tables are empty (`src/server/db/seed.ts`) 72 72 + SQLite via `bun:sqlite`. Schema in `src/server/db/schema.ts`, queries in `src/server/db/queries/`. All queries use prepared statements, writes are wrapped in transactions. 98 73 99 74 ## ATProto 100 75 101 101 - **Lexicon namespace:** `wiki.lichen` (domain: `lichen.wiki`). Lexicon JSON files live in `lexicons/`. 76 76 + Lexicon namespace: `wiki.lichen` (domain: `lichen.wiki`). Schemas in `lexicons/`. 102 77 103 78 Record types: `wiki.lichen.wiki`, `wiki.lichen.note`, `wiki.lichen.noteRevision`, `wiki.lichen.memberRequest`, `wiki.lichen.membership`, `wiki.lichen.bookmark` 104 79 105 105 - In dev mode (no OAuth configured), PDS writes are skipped and a mock DID (`did:plc:mock123`) is used. All data lives in SQLite only. 80 80 + In dev mode (no OAuth configured), PDS writes are skipped and a mock DID is used. 106 81 107 82 ## Tests 108 83 109 109 - Tests mirror `src/` under `tests/`. Run with `bun test`. 84 84 + Tests mirror `src/` under `tests/`. All test files share one in-memory DB singleton (`tests/preload.ts` sets `DB_PATH=:memory:`), so tests that create data must clean up in `afterAll`. 110 85 111 111 - - Add tests for new code; update existing tests when signatures change 112 112 - - Test edge cases, not just the happy path 113 113 - - `tests/preload.ts` sets `DB_PATH=:memory:` — all test files share one in-memory DB singleton, so tests that create data must clean up in `afterAll` 114 114 - - Wikilink tests should pass a `wikiSlug` parameter to `renderMarkdown()` for correct path resolution 115 115 - 116 116 - ### Integration tests 117 117 - 118 118 - `tests/integration/` contains end-to-end tests that write real ATProto records to a PDS and verify they flow through the firehose into the appview DB. 119 119 - 120 120 - Architecture: 121 121 - - A test PDS runs as a **Node subprocess** (via `scripts/start-pds.mjs`) because `@atproto/pds` depends on the `better-sqlite3` native addon, which is incompatible with Bun 122 122 - - The Bun test process connects to the PDS via `@atproto/sync` Firehose (using a `ws.createWebSocketStream` polyfill in `src/lib/ws-polyfill.ts`) 123 123 - - Records are written via `AtpAgent`, ingested by the in-process firehose handler, and assertions check the `bun:sqlite` appview DB 124 124 - 125 125 - Run integration tests only: `bun run test:integration` 126 126 - 86 86 + Integration tests in `tests/integration/` write real ATProto records to a PDS subprocess and verify they flow through the firehose into the appview DB. Run them with `bun run test:integration`.

+14 -44

README.md

reviewed

··· 1 1 - # Lichen 1 1 + A collaborative wiki built on [ATProto](https://atproto.com). Knowledge grows together. 2 2 3 3 - A wiki built on [ATProto](https://atproto.com). Sign in with your Bluesky account, create public or private wikis, and write in plain markdown with `[[wikilinks]]`. 3 3 + <!-- TODO: screenshot or demo gif --> 4 4 5 5 - Your data lives on your own PDS: records persist independently of this app, and your Bluesky identity works everywhere on ATProto. 5 5 + ## What is Lichen? 6 6 7 7 - **Features:** wikilinks · backlinks · revision history · access control (public/private wikis, roles) · D3 visualizations in fenced code blocks · image upload · EN/FR UI 7 7 + Sign in with your Bluesky account, create wikis, and write in plain markdown with `[[wikilinks]]`. Your data lives on your own PDS: it persists independently of this app, and your Bluesky identity works everywhere on ATProto. 8 8 9 9 - **Lexicon namespace:** `wiki.lichen` — domain: `lichen.wiki` 9 9 + ## Features 10 10 11 11 - ## Quick start 12 12 - ```bash 13 13 - bun install 14 14 - bun run dev 15 15 - ``` 16 16 - 17 17 - Opens at `http://localhost:3000` with a seeded sample wiki on first run. 18 18 - 19 19 - ### Full local stack (with PDS) 20 20 - 21 21 - ```bash 22 22 - bun run dev:full 23 23 - ``` 24 24 - 25 25 - Starts a local ATProto PDS (via Node), the appview, and the firehose subscriber — all wired together with test accounts. Visit `http://localhost:3000/dev/login/alice.test` to log in. 11 11 + - Wikilinks and backlinks: link notes with `[[double brackets]]`, backlinks are tracked automatically 12 12 + - Revision history: every edit is a diff, full history is always available 13 13 + - Access control: public or private wikis, with admin/contributor/viewer roles 14 14 + - Real-time sync via the ATProto firehose 15 15 + - Data portability: your wikis and notes are ATProto records on your PDS, not locked in 16 16 + - Markdown editor with split-pane live preview, image upload via drag-drop/paste 17 17 + - Multilingual UI (English and French) 18 18 + - Self-hostable: deploy your own instance, fully independent 26 19 27 20 ## How it works 28 21 29 29 - Contributors write revision records directly to their own PDS via OAuth. 30 30 - An AppView server ingests those records from the firehose, materializes wiki state, and serves reads. Private wikis use a hosted app PDS; public wiki data is fully portable. 31 31 - 32 32 - ## Tech stack 33 33 - 34 34 - | Layer | Tech | 35 35 - |---|---| 36 36 - | Runtime | [Bun](https://bun.sh) | 37 37 - | Framework | [Elysia](https://elysiajs.com) | 38 38 - | Database | SQLite via `bun:sqlite` | 39 39 - | UI | Server-rendered HTML + [HTMX](https://htmx.org) + [CodeMirror 6](https://codemirror.net) | 40 40 - | Auth | ATProto OAuth | 41 41 - 42 42 - Full stack details in [CONTRIBUTING.md](CONTRIBUTING.md). 43 43 - 44 44 - ## Self-hosting 45 45 - 46 46 - Clone the repo, set two env vars, deploy: 47 47 - 48 48 - - `PUBLIC_URL` — your instance's public HTTPS URL 49 49 - - `OAUTH_PRIVATE_KEY_PATH` — ES256 PEM key for ATProto OAuth 50 50 - 51 51 - Requires Bun, Node.js, and SQLite. Wildcard DNS optional (path-based URLs work without it). 52 52 - Blob storage defaults to proxying from PDS; R2/S3 can be added as a cache layer. 22 22 + Contributors write directly to their own [Personal Data Server](https://atproto.com/guides/glossary#pds) via OAuth. The Lichen appview ingests those records from the network firehose, assembles wiki state, and serves reads. Your data is never locked in: it follows you across the ATProto ecosystem. 53 23 54 24 ## Contributing 55 25