A system for building static webapps
test: add e2e testing for hono
+936
-12
+4
-4
deno.lock
+4
-4
deno.lock
···
1235
1235
"dependencies": [
1236
1236
"jsr:@astral/astral@~0.5.5",
1237
1237
"jsr:@b-fuze/deno-dom@~0.1.56",
1238
-
"jsr:@civility/blobs@^1.0.0-beta.3",
1239
-
"jsr:@civility/hono@^1.0.0-beta.2",
1238
+
"jsr:@civility/blobs@^1.0.0-beta.4",
1239
+
"jsr:@civility/hono@^1.0.0-beta.3",
1240
1240
"jsr:@civility/store@^1.0.0-beta.5",
1241
-
"jsr:@civility/sync@^1.0.0-beta.5",
1241
+
"jsr:@civility/sync@^1.0.0-beta.6",
1242
1242
"jsr:@cliffy/ansi@1.0.0",
1243
1243
"jsr:@cliffy/command@1.0.0",
1244
1244
"jsr:@cliffy/prompt@1",
···
1281
1281
},
1282
1282
"packages/sync": {
1283
1283
"dependencies": [
1284
-
"jsr:@civility/blobs@^1.0.0-beta.3",
1284
+
"jsr:@civility/blobs@^1.0.0-beta.4",
1285
1285
"jsr:@civility/store@^1.0.0-beta.5",
1286
1286
"jsr:@paulmillr/qr@~0.5.5"
1287
1287
]
+1
-1
packages/blobs/deno.json
+1
-1
packages/blobs/deno.json
+140
packages/cli/SPEC.md
+140
packages/cli/SPEC.md
···
1
+
# @civility/cli — Internal Specification
2
+
3
+
## Overview
4
+
5
+
The CLI is a build tool and dev server for Civility web projects. It scaffolds and builds PWAs, blogs, documentation sites, and browser extensions.
6
+
7
+
## Architecture
8
+
9
+
```
10
+
civility CLI (main.ts)
11
+
│
12
+
├── commands/
13
+
│ ├── init.ts → Project scaffolding
14
+
│ ├── start.ts → Dev server with HMR
15
+
│ ├── build/ → Build commands
16
+
│ │ ├── pwa.ts → PWA bundling (esbuild)
17
+
│ │ ├── blog.ts → Static blog generator
18
+
│ │ ├── docs.ts → Static docs generator
19
+
│ │ └── extension.ts → Browser extension bundler
20
+
│ ├── icons.ts → Icon generation
21
+
│ └── api.ts → API testing commands
22
+
│
23
+
└── utils/
24
+
├── config.ts → Load/resolve civility.json
25
+
├── build.ts → Shared build utilities
26
+
├── ui.ts → Logging and theming
27
+
└── validation.ts → Config validation
28
+
```
29
+
30
+
## Commands
31
+
32
+
### `civ init`
33
+
34
+
Interactive project scaffolding. Prompts for:
35
+
- Project type (pwa/blog/docs/extension)
36
+
- Project name
37
+
- URL
38
+
39
+
Generates a `civility.json` config and starter project structure.
40
+
41
+
### `civ build`
42
+
43
+
Builds the project based on `civility.json`:
44
+
- PWA: Bundles with esbuild, generates service worker, manifest
45
+
- Blog/Docs: Converts Markdown to static HTML
46
+
- Extension: Bundles and packages for Chrome/Firefox
47
+
48
+
Options:
49
+
- `--watch` — Watch mode (rebuild on changes)
50
+
- `--outdir` — Override output directory
51
+
52
+
### `civ start`
53
+
54
+
Development server with live reload:
55
+
- Serves from `root` (default: `./www`)
56
+
- Watches for file changes
57
+
- Rebuilds and streams to browser
58
+
59
+
### `civ icons`
60
+
61
+
Generates PWA icon sizes from source image:
62
+
- Uses ImageMagick (`convert`)
63
+
- Outputs standard sizes (16, 32, 48, 72, 96, 128, 144, 152, 192, 512)
64
+
- Creates `icon.ico` for IE/favicons
65
+
66
+
### `civ static`
67
+
68
+
Simple static file server. Serves a directory without building.
69
+
70
+
### `civ api`
71
+
72
+
API testing utilities (see `commands/api.ts`).
73
+
74
+
## Configuration
75
+
76
+
`civility.json` schema:
77
+
78
+
```ts
79
+
interface CivilityConfig {
80
+
name: string
81
+
type: 'pwa' | 'blog' | 'docs' | 'extension'
82
+
url?: string
83
+
root?: string // Source directory (default: ./www)
84
+
outdir?: string // Output directory (default: ./dist)
85
+
static?: string // Static assets to copy
86
+
input?: string // Input for blog/docs (default: ./md)
87
+
output?: string // Output for blog/docs
88
+
template?: string // Template file path
89
+
icon?: {
90
+
source: string
91
+
output: string
92
+
}
93
+
platforms?: ('chrome' | 'firefox')[] // Extension targets
94
+
entryPoints?: string[] // PWA entry points
95
+
}
96
+
```
97
+
98
+
## Build Pipeline
99
+
100
+
### PWA Build
101
+
102
+
1. Resolve entry points from config
103
+
2. Bundle with esbuild:
104
+
- Compile TypeScript
105
+
- Bundle imports
106
+
- Generate source maps
107
+
3. Generate `service-worker.js`:
108
+
- Cache strategies: cache-first for assets, network-first for data
109
+
- Offline support
110
+
4. Generate `manifest.json`:
111
+
- Name, description, icons
112
+
- Start URL, display mode
113
+
- Theme colors
114
+
5. Copy static assets to dist
115
+
116
+
### Extension Build
117
+
118
+
1. Bundle background, content scripts, popup, options
119
+
2. Generate `manifest.json` (MV3 for Chrome, MV2 for Firefox)
120
+
3. Package with web-ext or chrome-cli
121
+
122
+
### Blog/Docs Build
123
+
124
+
1. Scan input directory for `.md` files
125
+
2. Parse frontmatter (title, date, etc.)
126
+
3. Convert Markdown to HTML using marked
127
+
4. Apply template (layout.html / template.html)
128
+
5. Output to static HTML
129
+
130
+
## Dependencies
131
+
132
+
- `@cliffy/command` — CLI framework
133
+
- `esbuild` — Bundler
134
+
- `@std/fs` — File utilities
135
+
- `marked` — Markdown parser (blog/docs)
136
+
- ImageMagick — Icon generation
137
+
138
+
## Version
139
+
140
+
Current version: `0.3.1`
+5
-5
packages/cli/deno.json
+5
-5
packages/cli/deno.json
···
1
1
{
2
2
"name": "@civility/cli",
3
-
"version": "1.0.0-beta.6",
3
+
"version": "1.0.0-beta.7",
4
4
"deploy": {
5
5
"org": "bpev",
6
6
"app": "civility"
···
18
18
"@cliffy/command": "jsr:@cliffy/command@1.0.0",
19
19
"@cliffy/prompt": "jsr:@cliffy/prompt@^1.0.0",
20
20
"@cliffy/table": "jsr:@cliffy/table@1.0.0",
21
-
"@civility/blobs": "jsr:@civility/blobs@^1.0.0-beta.3",
22
-
"@civility/blobs/idb": "jsr:@civility/blobs@^1.0.0-beta.3/idb",
23
-
"@civility/hono": "jsr:@civility/hono@^1.0.0-beta.2",
21
+
"@civility/blobs": "jsr:@civility/blobs@^1.0.0-beta.4",
22
+
"@civility/blobs/idb": "jsr:@civility/blobs@^1.0.0-beta.4/idb",
23
+
"@civility/hono": "jsr:@civility/hono@^1.0.0-beta.3",
24
24
"@civility/store": "jsr:@civility/store@^1.0.0-beta.5",
25
25
"@civility/store/deno-kv": "jsr:@civility/store@^1.0.0-beta.5/deno-kv",
26
-
"@civility/sync": "jsr:@civility/sync@^1.0.0-beta.5",
26
+
"@civility/sync": "jsr:@civility/sync@^1.0.0-beta.6",
27
27
"@hono/hono": "npm:hono@^4.12.9",
28
28
"@hono/swagger-ui": "npm:@hono/swagger-ui@^0.6.1",
29
29
"@hono/zod-openapi": "npm:@hono/zod-openapi@^1.2.4",
+389
packages/hono/__tests__/server.test.ts
+389
packages/hono/__tests__/server.test.ts
···
1
+
/**
2
+
* @module E2E tests for @civility/hono server
3
+
*
4
+
* Tests full client ↔ server round-trip: auth, app CRUD, sync push/pull.
5
+
*/
6
+
7
+
import { assertEquals, assertExists } from '@std/assert'
8
+
import { createServer } from '../mod.ts'
9
+
import type { OpenAPIHono } from '@hono/zod-openapi'
10
+
11
+
interface ApiResponse<T> {
12
+
status: 'success' | 'error'
13
+
message: string
14
+
data: T | null
15
+
}
16
+
17
+
async function request(
18
+
app: OpenAPIHono,
19
+
input: string | URL,
20
+
init?: RequestInit,
21
+
): Promise<Response> {
22
+
const url = input instanceof URL ? input.toString() : input
23
+
const req = new Request(url, init)
24
+
return await app.fetch(req)
25
+
}
26
+
27
+
async function json<T>(res: Response): Promise<ApiResponse<T>> {
28
+
return await res.json() as ApiResponse<T>
29
+
}
30
+
31
+
Deno.test('E2E: Server', async (t) => {
32
+
const path = await Deno.makeTempFile({ suffix: '.kv' })
33
+
const kv = await Deno.openKv(path)
34
+
const app = await createServer({ kv })
35
+
36
+
const baseUrl = 'http://localhost:8081'
37
+
38
+
await t.step('signup creates user and returns token', async () => {
39
+
const res = await request(app, `${baseUrl}/api/v1/auth/signup`, {
40
+
method: 'POST',
41
+
headers: { 'Content-Type': 'application/json' },
42
+
body: JSON.stringify({
43
+
email: 'test@example.com',
44
+
password: 'password123',
45
+
}),
46
+
})
47
+
48
+
assertEquals(res.status, 201)
49
+
const body = await json<{ token: string; user: { id: string } }>(res)
50
+
assertEquals(body.status, 'success')
51
+
assertExists(body.data?.token)
52
+
assertExists(body.data?.user.id)
53
+
54
+
const cookie = res.headers.get('set-cookie')
55
+
assertExists(cookie)
56
+
})
57
+
58
+
await t.step('login authenticates and returns token', async () => {
59
+
const res = await request(app, `${baseUrl}/api/v1/auth/login`, {
60
+
method: 'POST',
61
+
headers: { 'Content-Type': 'application/json' },
62
+
body: JSON.stringify({
63
+
email: 'test@example.com',
64
+
password: 'password123',
65
+
}),
66
+
})
67
+
68
+
assertEquals(res.status, 200)
69
+
const body = await json<{ token: string; user: { id: string } }>(res)
70
+
assertEquals(body.status, 'success')
71
+
assertExists(body.data?.token)
72
+
})
73
+
74
+
await t.step('create app returns created app', async () => {
75
+
const loginRes = await request(app, `${baseUrl}/api/v1/auth/login`, {
76
+
method: 'POST',
77
+
headers: { 'Content-Type': 'application/json' },
78
+
body: JSON.stringify({
79
+
email: 'test@example.com',
80
+
password: 'password123',
81
+
}),
82
+
})
83
+
84
+
const loginBody = await json<{ token: string }>(loginRes)
85
+
const token = loginBody.data?.token!
86
+
87
+
const appRes = await request(app, `${baseUrl}/api/v1/sync/apps`, {
88
+
method: 'POST',
89
+
headers: {
90
+
'Content-Type': 'application/json',
91
+
'Authorization': `Bearer ${token}`,
92
+
},
93
+
body: JSON.stringify({
94
+
name: 'My Test App',
95
+
}),
96
+
})
97
+
98
+
assertEquals(appRes.status, 201)
99
+
const appBody = await json<{ app: { id: string; name: string } }>(appRes)
100
+
assertEquals(appBody.status, 'success')
101
+
assertExists(appBody.data?.app.id)
102
+
assertEquals(appBody.data?.app.name, 'My Test App')
103
+
})
104
+
105
+
await t.step('list apps returns user apps', async () => {
106
+
const loginRes = await request(app, `${baseUrl}/api/v1/auth/login`, {
107
+
method: 'POST',
108
+
headers: { 'Content-Type': 'application/json' },
109
+
body: JSON.stringify({
110
+
email: 'test@example.com',
111
+
password: 'password123',
112
+
}),
113
+
})
114
+
115
+
const loginBody = await json<{ token: string }>(loginRes)
116
+
const token = loginBody.data?.token!
117
+
118
+
const listRes = await request(app, `${baseUrl}/api/v1/sync/apps`, {
119
+
headers: { 'Authorization': `Bearer ${token}` },
120
+
})
121
+
122
+
assertEquals(listRes.status, 200)
123
+
const listBody = await json<{ apps: Array<{ id: string; name: string }> }>(
124
+
listRes,
125
+
)
126
+
assertEquals(listBody.status, 'success')
127
+
assertEquals(listBody.data?.apps.length, 1)
128
+
})
129
+
130
+
await t.step('get app returns app details', async () => {
131
+
const loginRes = await request(app, `${baseUrl}/api/v1/auth/login`, {
132
+
method: 'POST',
133
+
headers: { 'Content-Type': 'application/json' },
134
+
body: JSON.stringify({
135
+
email: 'test@example.com',
136
+
password: 'password123',
137
+
}),
138
+
})
139
+
140
+
const loginBody = await json<{ token: string }>(loginRes)
141
+
const token = loginBody.data?.token!
142
+
143
+
const listRes = await request(app, `${baseUrl}/api/v1/sync/apps`, {
144
+
headers: { 'Authorization': `Bearer ${token}` },
145
+
})
146
+
147
+
const listBody = await json<{ apps: Array<{ id: string }> }>(listRes)
148
+
const appId = listBody.data?.apps[0].id!
149
+
150
+
const getRes = await request(app, `${baseUrl}/api/v1/sync/apps/${appId}`, {
151
+
headers: { 'Authorization': `Bearer ${token}` },
152
+
})
153
+
154
+
assertEquals(getRes.status, 200)
155
+
const getBody = await json<{ app: { id: string; name: string } }>(getRes)
156
+
assertEquals(getBody.status, 'success')
157
+
assertEquals(getBody.data?.app.id, appId)
158
+
assertEquals(getBody.data?.app.name, 'My Test App')
159
+
})
160
+
161
+
await t.step('push changes applies them to store', async () => {
162
+
const loginRes = await request(app, `${baseUrl}/api/v1/auth/login`, {
163
+
method: 'POST',
164
+
headers: { 'Content-Type': 'application/json' },
165
+
body: JSON.stringify({
166
+
email: 'test@example.com',
167
+
password: 'password123',
168
+
}),
169
+
})
170
+
171
+
const loginBody = await json<{ token: string }>(loginRes)
172
+
const token = loginBody.data?.token!
173
+
174
+
const listRes = await request(app, `${baseUrl}/api/v1/sync/apps`, {
175
+
headers: { 'Authorization': `Bearer ${token}` },
176
+
})
177
+
178
+
const listBody = await json<{ apps: Array<{ id: string }> }>(listRes)
179
+
const appId = listBody.data?.apps[0].id!
180
+
181
+
const pushRes = await request(
182
+
app,
183
+
`${baseUrl}/api/v1/sync/apps/${appId}/push`,
184
+
{
185
+
method: 'POST',
186
+
headers: {
187
+
'Content-Type': 'application/json',
188
+
'Authorization': `Bearer ${token}`,
189
+
},
190
+
body: JSON.stringify({
191
+
changes: [
192
+
{
193
+
id: 'change-1',
194
+
documentId: 'doc-1',
195
+
patch: [{ op: 'add', path: '/title', value: 'Hello' }],
196
+
inversePatch: [{ op: 'remove', path: '/title' }],
197
+
hlc: '2024-01-01T00:00:00.000Z@local',
198
+
origin: 'local',
199
+
createdAt: '2024-01-01T00:00:00.000Z',
200
+
synced: false,
201
+
},
202
+
],
203
+
clientHLC: '2024-01-01T00:00:00.000Z@local',
204
+
storeName: 'default',
205
+
}),
206
+
},
207
+
)
208
+
209
+
assertEquals(pushRes.status, 200)
210
+
const pushBody = await json<{ accepted: string[] }>(pushRes)
211
+
assertEquals(pushBody.status, 'success')
212
+
assertEquals(pushBody.data?.accepted.length, 1)
213
+
assertEquals(pushBody.data?.accepted[0], 'change-1')
214
+
})
215
+
216
+
await t.step('pull changes returns changes since HLC', async () => {
217
+
const loginRes = await request(app, `${baseUrl}/api/v1/auth/login`, {
218
+
method: 'POST',
219
+
headers: { 'Content-Type': 'application/json' },
220
+
body: JSON.stringify({
221
+
email: 'test@example.com',
222
+
password: 'password123',
223
+
}),
224
+
})
225
+
226
+
const loginBody = await json<{ token: string }>(loginRes)
227
+
const token = loginBody.data?.token!
228
+
229
+
const listRes = await request(app, `${baseUrl}/api/v1/sync/apps`, {
230
+
headers: { 'Authorization': `Bearer ${token}` },
231
+
})
232
+
233
+
const listBody = await json<{ apps: Array<{ id: string }> }>(listRes)
234
+
const appId = listBody.data?.apps[0].id!
235
+
236
+
const pullRes = await request(
237
+
app,
238
+
`${baseUrl}/api/v1/sync/apps/${appId}/pull`,
239
+
{
240
+
method: 'POST',
241
+
headers: {
242
+
'Content-Type': 'application/json',
243
+
'Authorization': `Bearer ${token}`,
244
+
},
245
+
body: JSON.stringify({
246
+
sinceHLC: '2023-01-01T00:00:00.000Z@local',
247
+
storeName: 'default',
248
+
}),
249
+
},
250
+
)
251
+
252
+
assertEquals(pullRes.status, 200)
253
+
const pullBody = await json<{ changes: Array<{ id: string }> }>(pullRes)
254
+
assertEquals(pullBody.status, 'success')
255
+
assertEquals(pullBody.data?.changes.length, 1)
256
+
assertEquals(pullBody.data?.changes[0].id, 'change-1')
257
+
})
258
+
259
+
await t.step('delete app removes it from database', async () => {
260
+
const loginRes = await request(app, `${baseUrl}/api/v1/auth/login`, {
261
+
method: 'POST',
262
+
headers: { 'Content-Type': 'application/json' },
263
+
body: JSON.stringify({
264
+
email: 'test@example.com',
265
+
password: 'password123',
266
+
}),
267
+
})
268
+
269
+
const loginBody = await json<{ token: string }>(loginRes)
270
+
const token = loginBody.data?.token!
271
+
272
+
const createRes = await request(app, `${baseUrl}/api/v1/sync/apps`, {
273
+
method: 'POST',
274
+
headers: {
275
+
'Content-Type': 'application/json',
276
+
'Authorization': `Bearer ${token}`,
277
+
},
278
+
body: JSON.stringify({ name: 'Delete Me' }),
279
+
})
280
+
281
+
const createBody = await json<{ app: { id: string } }>(createRes)
282
+
const appId = createBody.data?.app.id!
283
+
284
+
const deleteRes = await request(
285
+
app,
286
+
`${baseUrl}/api/v1/sync/apps/${appId}`,
287
+
{
288
+
method: 'DELETE',
289
+
headers: { 'Authorization': `Bearer ${token}` },
290
+
},
291
+
)
292
+
293
+
assertEquals(deleteRes.status, 200)
294
+
295
+
const listRes = await request(app, `${baseUrl}/api/v1/sync/apps`, {
296
+
headers: { 'Authorization': `Bearer ${token}` },
297
+
})
298
+
299
+
const listBody = await json<{ apps: unknown[] }>(listRes)
300
+
assertEquals(listBody.data?.apps.length, 1)
301
+
})
302
+
303
+
await t.step('unauthorized request returns 401', async () => {
304
+
const res = await request(app, `${baseUrl}/api/v1/sync/apps`, {
305
+
method: 'GET',
306
+
})
307
+
308
+
assertEquals(res.status, 401)
309
+
})
310
+
311
+
await t.step('create token returns new token', async () => {
312
+
const loginRes = await request(app, `${baseUrl}/api/v1/auth/login`, {
313
+
method: 'POST',
314
+
headers: { 'Content-Type': 'application/json' },
315
+
body: JSON.stringify({
316
+
email: 'test@example.com',
317
+
password: 'password123',
318
+
}),
319
+
})
320
+
321
+
const loginBody = await json<{ token: string }>(loginRes)
322
+
const token = loginBody.data?.token!
323
+
324
+
const listRes = await request(app, `${baseUrl}/api/v1/sync/apps`, {
325
+
headers: { 'Authorization': `Bearer ${token}` },
326
+
})
327
+
328
+
const listBody = await json<{ apps: Array<{ id: string }> }>(listRes)
329
+
const appId = listBody.data?.apps[0].id!
330
+
331
+
const tokenRes = await request(
332
+
app,
333
+
`${baseUrl}/api/v1/sync/apps/${appId}/tokens`,
334
+
{
335
+
method: 'POST',
336
+
headers: {
337
+
'Content-Type': 'application/json',
338
+
'Authorization': `Bearer ${token}`,
339
+
},
340
+
body: JSON.stringify({ name: 'My Token', permissions: 'read-write' }),
341
+
},
342
+
)
343
+
344
+
assertEquals(tokenRes.status, 201)
345
+
const tokenBody = await json<{ token: { id: string; token: string } }>(
346
+
tokenRes,
347
+
)
348
+
assertEquals(tokenBody.status, 'success')
349
+
assertExists(tokenBody.data?.token.id)
350
+
assertExists(tokenBody.data?.token.token)
351
+
})
352
+
353
+
await t.step('list tokens returns app tokens', async () => {
354
+
const loginRes = await request(app, `${baseUrl}/api/v1/auth/login`, {
355
+
method: 'POST',
356
+
headers: { 'Content-Type': 'application/json' },
357
+
body: JSON.stringify({
358
+
email: 'test@example.com',
359
+
password: 'password123',
360
+
}),
361
+
})
362
+
363
+
const loginBody = await json<{ token: string }>(loginRes)
364
+
const token = loginBody.data?.token!
365
+
366
+
const listRes = await request(app, `${baseUrl}/api/v1/sync/apps`, {
367
+
headers: { 'Authorization': `Bearer ${token}` },
368
+
})
369
+
370
+
const listBody = await json<{ apps: Array<{ id: string }> }>(listRes)
371
+
const appId = listBody.data?.apps[0].id!
372
+
373
+
const tokensRes = await request(
374
+
app,
375
+
`${baseUrl}/api/v1/sync/apps/${appId}/tokens`,
376
+
{
377
+
headers: { 'Authorization': `Bearer ${token}` },
378
+
},
379
+
)
380
+
381
+
assertEquals(tokensRes.status, 200)
382
+
const tokensBody = await json<{ tokens: Array<{ id: string }> }>(tokensRes)
383
+
assertEquals(tokensBody.status, 'success')
384
+
assertEquals(tokensBody.data?.tokens.length, 1)
385
+
})
386
+
387
+
await kv.close()
388
+
await Deno.remove(path)
389
+
})
+226
packages/sync/README.md
+226
packages/sync/README.md
···
1
+
# @civility/sync
2
+
3
+
Change-based synchronization for Civility stores. Syncs `@civility/store` data with a Civility Sync server, handling conflict resolution, blob syncing, and offline support.
4
+
5
+
## Install
6
+
7
+
```sh
8
+
deno add @civility/sync
9
+
```
10
+
11
+
## Quick Start
12
+
13
+
```ts
14
+
import { Collection } from '@civility/store'
15
+
import { Synced } from '@civility/sync'
16
+
17
+
const todos = new Collection({
18
+
name: 'todos',
19
+
schema: { version: '1', schema: TodosSchema },
20
+
backend,
21
+
})
22
+
23
+
const synced = new Synced({
24
+
stores: [todos],
25
+
appId: 'my-app',
26
+
})
27
+
28
+
synced.connect('https://sync.example.com')
29
+
await synced.login('user@example.com', 'password')
30
+
synced.startSync()
31
+
```
32
+
33
+
## Core Concepts
34
+
35
+
### Synced Orchestrator
36
+
37
+
The `Synced` class wraps one or more stores and manages the sync lifecycle:
38
+
39
+
```ts
40
+
const synced = new Synced({
41
+
stores: [todosStore, settingsStore],
42
+
appId: 'my-app',
43
+
blobStore, // optional, for binary data
44
+
syncInterval: 30000, // ms between sync cycles (default: 30s)
45
+
debounceDelay: 500, // ms to wait after local changes (default: 500ms)
46
+
timeout: 10000, // request timeout in ms (default: 10s)
47
+
})
48
+
```
49
+
50
+
### Connection & Authentication
51
+
52
+
```ts
53
+
// Connect to server
54
+
synced.connect('https://sync.example.com')
55
+
56
+
// Login or signup
57
+
await synced.login('user@example.com', 'password')
58
+
// or
59
+
await synced.signup('user@example.com', 'password')
60
+
61
+
// Or restore an existing session
62
+
synced.setToken('existing-token')
63
+
64
+
// Start automatic sync
65
+
synced.startSync()
66
+
```
67
+
68
+
### Sync Events
69
+
70
+
Listen to sync events:
71
+
72
+
```ts
73
+
synced.addEventListener('connected', () => console.log('Connected'))
74
+
synced.addEventListener('auth', () => console.log('Authenticated'))
75
+
synced.addEventListener('sync', (e) => {
76
+
const { pushed, pulled } = e.detail
77
+
console.log(`Synced: ${pushed} pushed, ${pulled} pulled`)
78
+
})
79
+
synced.addEventListener('error', (e) => console.error('Sync error:', e.error))
80
+
```
81
+
82
+
### Manual Sync
83
+
84
+
```ts
85
+
// Force push without pulling
86
+
await synced.forcePush()
87
+
88
+
// Force pull without pushing
89
+
await synced.forcePull()
90
+
91
+
// Full sync cycle
92
+
await synced.sync()
93
+
```
94
+
95
+
### Blob Sync
96
+
97
+
If your documents contain binary data (images, files), provide a `BlobStore`:
98
+
99
+
```ts
100
+
import { BlobStore } from '@civility/blobs'
101
+
102
+
const blobStore = new BlobStore({ kv, namespace: 'blobs' })
103
+
104
+
const synced = new Synced({
105
+
stores: [todos],
106
+
appId: 'my-app',
107
+
blobStore,
108
+
})
109
+
110
+
// Blobs referenced in documents are automatically uploaded/downloaded
111
+
```
112
+
113
+
Blob references in documents should use this structure:
114
+
115
+
```ts
116
+
interface BlobRef {
117
+
hash: string // sha256:...
118
+
mime: string // image/png
119
+
size: number // bytes
120
+
}
121
+
```
122
+
123
+
## SyncApi (HTTP Client)
124
+
125
+
For scripts or admin tools, use `SyncApi` directly:
126
+
127
+
```ts
128
+
import { SyncApi } from '@civility/sync'
129
+
130
+
const api = new SyncApi({ baseUrl: 'https://sync.example.com' })
131
+
132
+
// Auth
133
+
const auth = await api.login('user@example.com', 'password')
134
+
api.setToken(auth.token)
135
+
136
+
// App management
137
+
const apps = await api.listApps()
138
+
const app = await api.createApp('My App', 'Description')
139
+
await api.deleteApp(app.id)
140
+
141
+
// Push/pull changes
142
+
const pushResult = await api.pushChanges(appId, changes, clientHLC)
143
+
const pullResult = await api.pullChanges(appId, sinceHLC)
144
+
145
+
// Blobs
146
+
await api.uploadBlob(hash, blobData)
147
+
const blob = await api.downloadBlob(hash)
148
+
```
149
+
150
+
## API Reference
151
+
152
+
### Synced
153
+
154
+
| Method | Description |
155
+
|--------|-------------|
156
+
| `connect(url)` | Connect to sync server |
157
+
| `disconnect()` | Disconnect and stop sync |
158
+
| `login(email, password)` | Authenticate user |
159
+
| `signup(email, password)` | Create new account |
160
+
| `logout()` | Log out and stop sync |
161
+
| `setToken(token)` | Set auth token directly |
162
+
| `startSync()` | Begin periodic sync |
163
+
| `stopSync()` | Stop periodic sync |
164
+
| `sync()` | Run single sync cycle |
165
+
| `forcePush()` | Push without pulling |
166
+
| `forcePull()` | Pull without pushing |
167
+
| `dispose()` | Release all resources |
168
+
169
+
### Properties
170
+
171
+
| Property | Type | Description |
172
+
|----------|------|-------------|
173
+
| `connected` | `boolean` | Whether connected to server |
174
+
| `authenticated` | `boolean` | Whether authenticated |
175
+
| `syncing` | `boolean` | Whether sync in progress |
176
+
| `lastSync` | `string \| null` | ISO timestamp of last sync |
177
+
| `appId` | `string` | Current app ID |
178
+
| `api` | `SyncApi \| null` | Underlying API client |
179
+
180
+
### SyncApi
181
+
182
+
| Method | Description |
183
+
|--------|-------------|
184
+
| `login(email, password)` | Authenticate |
185
+
| `signup(email, password)` | Create account |
186
+
| `logout()` | Log out |
187
+
| `verifyToken()` | Check token validity |
188
+
| `getMe()` | Get current user profile |
189
+
| `listApps()` | List user's apps |
190
+
| `createApp(name, desc?)` | Create app |
191
+
| `getApp(id)` | Get app by ID |
192
+
| `deleteApp(id)` | Delete app |
193
+
| `listAppTokens(appId)` | List app tokens |
194
+
| `createAppToken(appId, name, perms?)` | Create token |
195
+
| `deleteAppToken(appId, tokenId)` | Delete token |
196
+
| `registerApp(appId, schema)` | Register schema |
197
+
| `pushChanges(appId, changes, hlc)` | Push changes |
198
+
| `pullChanges(appId, sinceHlc, store?)` | Pull changes |
199
+
| `uploadBlob(hash, data)` | Upload blob |
200
+
| `downloadBlob(hash)` | Download blob |
201
+
| `hasBlobRemote(hash)` | Check blob exists |
202
+
203
+
## Conflict Resolution
204
+
205
+
When the same document is modified on multiple devices, conflicts are resolved automatically using HLC timestamps. The default strategy (`auto`) compares timestamps and newer wins.
206
+
207
+
Configure conflict strategy via schema:
208
+
209
+
```ts
210
+
const collection = new Collection({
211
+
name: 'todos',
212
+
schema: {
213
+
version: '1',
214
+
schema: TodosSchema,
215
+
// Strategies: 'auto' | 'local-first' | 'remote-first' | 'prompt'
216
+
initialSyncStrategy: 'remote-first',
217
+
},
218
+
backend,
219
+
})
220
+
```
221
+
222
+
## Requirements
223
+
224
+
- A running Civility Sync server (see `@civility/hono`)
225
+
- `@civility/store` for data storage
226
+
- Optionally `@civility/blobs` for binary data
+169
packages/sync/SPEC.md
+169
packages/sync/SPEC.md
···
1
+
# @civility/sync — Internal Specification
2
+
3
+
## Overview
4
+
5
+
The sync package provides change-based synchronization for `@civility/store` collections. It handles bidirectional sync with a Civility Sync server, including conflict resolution, blob syncing, and offline support.
6
+
7
+
## Architecture
8
+
9
+
```
10
+
┌─────────────────┐ ┌─────────────────┐
11
+
│ App Code │ │ Sync Server │
12
+
│ │ │ (@civility/ │
13
+
│ Collection │ │ hono) │
14
+
│ (local store) │◄───►│ │
15
+
│ │ │ REST API │
16
+
└────────┬────────┘ └────────┬────────┘
17
+
│ │
18
+
▼ ▼
19
+
┌─────────────────┐ ┌─────────────────┐
20
+
│ Synced │ │ SyncApi │
21
+
│ (orchestrator) │────►│ (HTTP client) │
22
+
└─────────────────┘ └─────────────────┘
23
+
```
24
+
25
+
## Components
26
+
27
+
### Synced (`synced.ts`)
28
+
29
+
The main orchestrator class that wraps store instances and manages the sync lifecycle.
30
+
31
+
**Responsibilities:**
32
+
- Connect/disconnect from server
33
+
- Authenticate users
34
+
- Coordinate periodic push/pull sync
35
+
- Handle debouncing of local changes
36
+
- Emit sync events
37
+
- Manage blob sync (upload/download)
38
+
39
+
**State Machine:**
40
+
```
41
+
disconnected → connected → authenticated → syncing
42
+
▲ │ │ │
43
+
└──────────────┴──────────────┴─────────────┘
44
+
(logout/disconnect)
45
+
```
46
+
47
+
**Key Methods:**
48
+
- `connect(url)` — Create SyncApi client
49
+
- `login()`/`signup()`/`setToken()` — Authenticate
50
+
- `startSync()` — Begin periodic sync loop
51
+
- `sync()` — Single sync cycle (push then pull)
52
+
- `forcePush()` / `forcePull()` — One-way sync
53
+
- `disconnect()`/`dispose()` — Cleanup
54
+
55
+
**Events:**
56
+
- `connected` — Server connection established
57
+
- `disconnected` — Server disconnected
58
+
- `auth` — Authentication state changed
59
+
- `sync` — Sync cycle completed (detail: {pushed, pulled, serverHLC})
60
+
- `error` — Sync error occurred
61
+
62
+
### SyncApi (`api.ts`)
63
+
64
+
HTTP client for the Civility Sync server REST API.
65
+
66
+
**Endpoints:**
67
+
- `POST /login` — Authenticate
68
+
- `POST /signup` — Create account
69
+
- `POST /logout` — Log out
70
+
- `GET /verify` — Verify token
71
+
- `GET /me` — Get user profile
72
+
- `GET/POST/DELETE /apps` — App CRUD
73
+
- `GET/POST/DELETE /apps/:id/tokens` — Token management
74
+
- `POST /apps/:id/register` — Register schema
75
+
- `POST /apps/:id/push` — Push changes
76
+
- `POST /apps/:id/pull` — Pull changes
77
+
- `POST/HEAD/GET /blobs/:hash` — Blob operations
78
+
79
+
**Error Handling:**
80
+
- Throws `ApiError` on failure
81
+
- `ApiError.status` — HTTP status code
82
+
- `ApiError.response` — Parsed API response
83
+
84
+
### Types (`types.ts`)
85
+
86
+
Shared types used by both Synced and SyncApi.
87
+
88
+
**Key Types:**
89
+
- `AuthToken` — Auth response with user info
90
+
- `UserProfile` — User data
91
+
- `App` — App record (legacy compatibility)
92
+
- `AppRegistration` — App with serialized schema
93
+
- `PushRequest`/`PushResult` — Change push protocol
94
+
- `PullRequest`/`PullResult` — Change pull protocol
95
+
- `InitialSyncStrategy` — Conflict resolution enum
96
+
97
+
## Sync Protocol
98
+
99
+
### Change Format
100
+
101
+
Changes are `ChangeEntry` objects from `@civility/store`:
102
+
- `id` — Unique change ID
103
+
- `docId` — Document ID
104
+
- `hlc` — Hybrid Logical Clock timestamp
105
+
- `patch` — Array of operations
106
+
- `synced` — Whether synced to server
107
+
108
+
### Push
109
+
110
+
1. Get unsynced changes from each store
111
+
2. Upload referenced blobs if using BlobStore
112
+
3. POST to `/apps/:id/push` with `{changes, clientHLC}`
113
+
4. Server returns accepted IDs, remote changes, server HLC
114
+
5. Mark accepted changes as synced
115
+
6. Apply any remote changes returned
116
+
117
+
### Pull
118
+
119
+
1. POST to `/apps/:id/pull` with `{sinceHLC, storeName?}`
120
+
2. Server returns changes since last HLC
121
+
3. Download any referenced blobs
122
+
4. Apply changes to local store
123
+
124
+
### HLC (Hybrid Logical Clock)
125
+
126
+
The sync protocol uses HLC timestamps for conflict resolution:
127
+
- Combines logical timestamps with wall-clock time
128
+
- Ensures causality is preserved across devices
129
+
- Newer timestamp wins in conflicts (default)
130
+
131
+
## Blob Sync
132
+
133
+
Blobs are identified by SHA-256 hash. The sync flow:
134
+
135
+
1. **Upload:** Before pushing changes, extract blob refs, check if server has them, upload missing
136
+
2. **Download:** After pulling changes, extract blob refs, check local store, download missing
137
+
138
+
Blob references use this structure:
139
+
```ts
140
+
interface BlobRef {
141
+
hash: string // sha256:...
142
+
mime: string
143
+
size: number
144
+
}
145
+
```
146
+
147
+
## Testing
148
+
149
+
Tests are in `__tests__/`:
150
+
- `synced.test.ts` — Synced orchestrator tests
151
+
- `api.test.ts` — SyncApi client tests
152
+
153
+
Run tests:
154
+
```sh
155
+
deno test packages/sync
156
+
```
157
+
158
+
## Dependencies
159
+
160
+
- `@civility/store` — Data storage and change tracking
161
+
- `@civility/blobs` — Optional, for binary data sync
162
+
- `@std/fs` — File utilities (Deno)
163
+
164
+
## Future Improvements
165
+
166
+
- WebSocket-based real-time sync
167
+
- CRDT-based conflict resolution
168
+
- Selective sync (filter by store/collection)
169
+
- Background sync with Service Worker
+2
-2
packages/sync/deno.json
+2
-2
packages/sync/deno.json
···
1
1
{
2
2
"name": "@civility/sync",
3
-
"version": "1.0.0-beta.5",
3
+
"version": "1.0.0-beta.6",
4
4
"license": "MIT",
5
5
"exports": {
6
6
".": "./mod.ts",
7
7
"./qr": "./utils/qr.ts"
8
8
},
9
9
"imports": {
10
-
"@civility/blobs": "jsr:@civility/blobs@^1.0.0-beta.3",
10
+
"@civility/blobs": "jsr:@civility/blobs@^1.0.0-beta.4",
11
11
"@civility/store": "jsr:@civility/store@^1.0.0-beta.5",
12
12
"@paulmillr/qr": "jsr:@paulmillr/qr@^0.5.5"
13
13
}