A system for building static webapps
0
fork

Configure Feed

Select the types of activity you want to include in your feed.

feat(hono): add storage info

+259 -65
+1
deno.json
··· 45 45 }, 46 46 "tasks": { 47 47 "cli": "deno install packages/cli/main.ts -Afg --name=civ --unstable-kv --config ./packages/cli/deno.json", 48 + "serve": "deno run -A --unstable-kv packages/cli/main.ts api start --port 8081", 48 49 "test": "deno fmt && deno lint && deno test packages/ -A --unstable-kv --coverage --quiet" 49 50 }, 50 51 "imports": {
+13 -8
packages/cli/SPEC.md
··· 32 32 ### `civ init` 33 33 34 34 Interactive project scaffolding. Prompts for: 35 + 35 36 - Project type (pwa/blog/docs/extension) 36 37 - Project name 37 38 - URL ··· 41 42 ### `civ build` 42 43 43 44 Builds the project based on `civility.json`: 45 + 44 46 - PWA: Bundles with esbuild, generates service worker, manifest 45 47 - Blog/Docs: Converts Markdown to static HTML 46 48 - Extension: Bundles and packages for Chrome/Firefox 47 49 48 50 Options: 51 + 49 52 - `--watch` — Watch mode (rebuild on changes) 50 53 - `--outdir` — Override output directory 51 54 52 55 ### `civ start` 53 56 54 57 Development server with live reload: 58 + 55 59 - Serves from `root` (default: `./www`) 56 60 - Watches for file changes 57 61 - Rebuilds and streams to browser ··· 59 63 ### `civ icons` 60 64 61 65 Generates PWA icon sizes from source image: 66 + 62 67 - Uses ImageMagick (`convert`) 63 68 - Outputs standard sizes (16, 32, 48, 72, 96, 128, 144, 152, 192, 512) 64 69 - Creates `icon.ico` for IE/favicons ··· 80 85 name: string 81 86 type: 'pwa' | 'blog' | 'docs' | 'extension' 82 87 url?: string 83 - root?: string // Source directory (default: ./www) 84 - outdir?: string // Output directory (default: ./dist) 85 - static?: string // Static assets to copy 86 - input?: string // Input for blog/docs (default: ./md) 87 - output?: string // Output for blog/docs 88 - template?: string // Template file path 88 + root?: string // Source directory (default: ./www) 89 + outdir?: string // Output directory (default: ./dist) 90 + static?: string // Static assets to copy 91 + input?: string // Input for blog/docs (default: ./md) 92 + output?: string // Output for blog/docs 93 + template?: string // Template file path 89 94 icon?: { 90 95 source: string 91 96 output: string 92 97 } 93 - platforms?: ('chrome' | 'firefox')[] // Extension targets 94 - entryPoints?: string[] // PWA entry points 98 + platforms?: ('chrome' | 'firefox')[] // Extension targets 99 + entryPoints?: string[] // PWA entry points 95 100 } 96 101 ``` 97 102
+33 -1
packages/hono/auth/ui_router.ts
··· 229 229 return c.html(dashboardLayout( 230 230 user, 231 231 html` 232 - <div></div> 232 + <div id="storage-stats" class="card"> 233 + <h2>Storage Usage</h2> 234 + <p>Loading...</p> 235 + </div> 236 + <script> 237 + (async () => { 238 + try { 239 + const res = await fetch('/api/v1/sync/storage', { 240 + credentials: 'same-origin' 241 + }); 242 + const data = await res.json(); 243 + if (data.status === 'success' && data.data) { 244 + const s = data.data; 245 + const formatBytes = (b) => { 246 + if (b === 0) return '0 B'; 247 + const k = 1024; 248 + const sizes = ['B', 'KB', 'MB', 'GB']; 249 + const i = Math.floor(Math.log(b) / Math.log(k)); 250 + return (b / Math.pow(k, i)).toFixed(1) + ' ' + sizes[i]; 251 + }; 252 + const html = '<h2>Storage Usage</h2>' + 253 + '<p><strong>Apps:</strong> ' + s.appCount + '</p>' + 254 + '<p><strong>Changes:</strong> ' + s.changeCount + '</p>' + 255 + '<p><strong>Data Size:</strong> ' + formatBytes(s.totalDataSize) + '</p>' + 256 + '<p><strong>Blob Storage:</strong> ' + formatBytes(s.blobStorageSize) + '</p>' + 257 + '<p><strong>Total:</strong> ' + formatBytes(s.totalDataSize + s.blobStorageSize) + '</p>'; 258 + document.getElementById('storage-stats').innerHTML = html; 259 + } 260 + } catch (e) { 261 + document.getElementById('storage-stats').innerHTML = '<p>Storage stats unavailable</p>'; 262 + } 263 + })(); 264 + </script> 233 265 `, 234 266 )) 235 267 })
+9
packages/hono/main.ts
··· 1 1 #!/usr/bin/env -S deno run -A 2 2 import { createServer } from './mod.ts' 3 + // import { createBunnyStorage } from './storage/bunny.ts' 3 4 4 5 const kv = await Deno.openKv() 6 + 7 + // const objectStorage = createBunnyStorage({ 8 + // storageZoneName: 'my-zone', 9 + // storageZonePassword: Deno.env.get('BUNNY_STORAGE_PASSWORD')!, 10 + // kv, 11 + // }) 12 + 5 13 const app = await createServer({ 6 14 kv, 15 + // objectStorage, 7 16 path: '/api', 8 17 }) 9 18
+72 -7
packages/hono/sync/adapters/deno-kv.ts
··· 179 179 appCount: number 180 180 changeCount: number 181 181 totalDataSize: number 182 + blobStorageSize: number 182 183 }> { 183 184 let appCount = 0 184 185 let changeCount = 0 185 186 let totalDataSize = 0 187 + let blobStorageSize = 0 186 188 189 + const apps: string[] = [] 187 190 const appsIter = this.#kv.list({ prefix: ['users', userId, 'apps'] }) 188 - for await (const _entry of appsIter) { 191 + for await (const entry of appsIter) { 189 192 appCount++ 193 + const app = entry.value as { id: string } 194 + apps.push(app.id) 190 195 } 191 196 192 - const changesIter = this.#kv.list({ prefix: ['sync'] }) 197 + for (const appId of apps) { 198 + const changesIter = this.#kv.list({ 199 + prefix: ['sync', appId], 200 + }) 201 + for await (const entry of changesIter) { 202 + if (entry.key.length >= 4 && entry.key[2] === 'changes') { 203 + const value = entry.value 204 + if (typeof value === 'object' && value !== null && 'patch' in value) { 205 + changeCount++ 206 + const patch = (value as { patch: unknown }).patch 207 + if (typeof patch === 'string') { 208 + totalDataSize += patch.length 209 + } else if (Array.isArray(patch)) { 210 + totalDataSize += JSON.stringify(patch).length 211 + } 212 + } 213 + } 214 + } 215 + } 216 + 217 + const blobMetaIter = this.#kv.list({ prefix: ['blob-meta'] }) 218 + for await (const entry of blobMetaIter) { 219 + const meta = entry.value as { size?: number } 220 + if (meta?.size) { 221 + blobStorageSize += meta.size 222 + } 223 + } 224 + 225 + return { appCount, changeCount, totalDataSize, blobStorageSize } 226 + } 227 + 228 + async getAppStorageStats(appId: string): Promise<{ 229 + changeCount: number 230 + dataSize: number 231 + blobCount: number 232 + blobSize: number 233 + }> { 234 + let changeCount = 0 235 + let dataSize = 0 236 + let blobSize = 0 237 + const blobKeys = new Set<string>() 238 + 239 + const changesIter = this.#kv.list({ 240 + prefix: ['sync', appId], 241 + }) 193 242 for await (const entry of changesIter) { 194 - if (entry.key.length >= 4 && entry.key[1] === 'sync') { 243 + if (entry.key.length >= 4 && entry.key[2] === 'changes') { 195 244 const value = entry.value 196 245 if (typeof value === 'object' && value !== null && 'patch' in value) { 197 246 changeCount++ 198 - const patch = value.patch 247 + const patch = (value as { patch: unknown }).patch 199 248 if (typeof patch === 'string') { 200 - totalDataSize += patch.length 249 + dataSize += patch.length 201 250 } else if (Array.isArray(patch)) { 202 - totalDataSize += JSON.stringify(patch).length 251 + dataSize += JSON.stringify(patch).length 203 252 } 204 253 } 205 254 } 206 255 } 207 256 208 - return { appCount, changeCount, totalDataSize } 257 + const blobMetaIter = this.#kv.list({ prefix: ['blob-meta'] }) 258 + for await (const entry of blobMetaIter) { 259 + const meta = entry.value as { key?: string; size?: number } 260 + if (meta?.key?.startsWith(appId) || meta?.key?.includes(`/${appId}/`)) { 261 + blobKeys.add(meta.key) 262 + if (meta?.size) { 263 + blobSize += meta.size 264 + } 265 + } 266 + } 267 + 268 + return { 269 + changeCount, 270 + dataSize, 271 + blobCount: blobKeys.size, 272 + blobSize, 273 + } 209 274 } 210 275 } 211 276
+28
packages/hono/sync/router.ts
··· 50 50 appCount: number 51 51 changeCount: number 52 52 totalDataSize: number 53 + blobStorageSize: number 54 + }> 55 + getAppStorageStats(appId: string): Promise<{ 56 + changeCount: number 57 + dataSize: number 58 + blobCount: number 59 + blobSize: number 53 60 }> 54 61 } 55 62 ··· 434 441 return c.redirect(`/dashboard/apps/${id}?success=token_deleted`) 435 442 } 436 443 return c.json(ok(null, 'Token deleted')) 444 + }) 445 + 446 + // GET /api/sync/storage — get user storage stats 447 + app.get('/sync/storage', async (c: Context) => { 448 + const ctx = c.get('authContext') 449 + const stats = await db.getStorageStats(ctx.user.id) 450 + return c.json(ok(stats)) 451 + }) 452 + 453 + // GET /api/sync/apps/:id/storage — get app storage stats 454 + app.get('/sync/apps/:id/storage', async (c: Context) => { 455 + const ctx = c.get('authContext') 456 + const id = c.req.param('id') 457 + if (!id) return c.json(err('App ID required'), 400) 458 + 459 + if (!await db.userOwnsApp(ctx.user.id, id)) { 460 + return c.json(err('App not found'), 404) 461 + } 462 + 463 + const stats = await db.getAppStorageStats(id) 464 + return c.json(ok(stats)) 437 465 }) 438 466 439 467 return app
+43 -1
packages/hono/sync/ui_router.ts
··· 129 129 ) 130 130 } 131 131 132 + function formatBytes(bytes: number): string { 133 + if (bytes === 0) return '0 B' 134 + const k = 1024 135 + const sizes = ['B', 'KB', 'MB', 'GB'] 136 + const i = Math.floor(Math.log(bytes) / Math.log(k)) 137 + return parseFloat((bytes / Math.pow(k, i)).toFixed(1)) + ' ' + sizes[i] 138 + } 139 + 132 140 function appDetailPage( 133 141 app: SyncApp, 134 142 tokens: SyncToken[], 143 + storageStats?: { 144 + changeCount: number 145 + dataSize: number 146 + blobCount: number 147 + blobSize: number 148 + }, 135 149 ) { 136 150 const tokensHtml = tokens.length > 0 137 151 ? html` ··· 167 181 <p><strong>Updated:</strong> ${new Date(app.updatedAt) 168 182 .toLocaleString()}</p> 169 183 </div> 184 + ${storageStats 185 + ? html` 186 + <div class="card"> 187 + <h2>Storage</h2> 188 + <p><strong>Changes:</strong> ${storageStats.changeCount}</p> 189 + <p><strong>Data Size:</strong> ${formatBytes( 190 + storageStats.dataSize, 191 + )}</p> 192 + <p><strong>Blobs:</strong> ${storageStats.blobCount} (${formatBytes( 193 + storageStats.blobSize, 194 + )})</p> 195 + <p><strong>Total:</strong> ${formatBytes( 196 + storageStats.dataSize + storageStats.blobSize, 197 + )}</p> 198 + </div> 199 + ` 200 + : ''} 170 201 <div class="card"> 171 202 <h2>API Tokens</h2> 172 203 ${tokensHtml} ··· 251 282 } 252 283 253 284 const tokens = await config.db.getTokens(id) 254 - return c.html(appDetailPage(app, tokens)) 285 + let storageStats: undefined | { 286 + changeCount: number 287 + dataSize: number 288 + blobCount: number 289 + blobSize: number 290 + } 291 + try { 292 + storageStats = await config.db.getAppStorageStats(id) 293 + } catch { 294 + // Storage stats not available 295 + } 296 + return c.html(appDetailPage(app, tokens, storageStats)) 255 297 }) 256 298 257 299 return app
+47 -47
packages/sync/README.md
··· 40 40 const synced = new Synced({ 41 41 stores: [todosStore, settingsStore], 42 42 appId: 'my-app', 43 - blobStore, // optional, for binary data 43 + blobStore, // optional, for binary data 44 44 syncInterval: 30000, // ms between sync cycles (default: 30s) 45 45 debounceDelay: 500, // ms to wait after local changes (default: 500ms) 46 - timeout: 10000, // request timeout in ms (default: 10s) 46 + timeout: 10000, // request timeout in ms (default: 10s) 47 47 }) 48 48 ``` 49 49 ··· 114 114 115 115 ```ts 116 116 interface BlobRef { 117 - hash: string // sha256:... 118 - mime: string // image/png 119 - size: number // bytes 117 + hash: string // sha256:... 118 + mime: string // image/png 119 + size: number // bytes 120 120 } 121 121 ``` 122 122 ··· 151 151 152 152 ### Synced 153 153 154 - | Method | Description | 155 - |--------|-------------| 156 - | `connect(url)` | Connect to sync server | 157 - | `disconnect()` | Disconnect and stop sync | 158 - | `login(email, password)` | Authenticate user | 159 - | `signup(email, password)` | Create new account | 160 - | `logout()` | Log out and stop sync | 161 - | `setToken(token)` | Set auth token directly | 162 - | `startSync()` | Begin periodic sync | 163 - | `stopSync()` | Stop periodic sync | 164 - | `sync()` | Run single sync cycle | 165 - | `forcePush()` | Push without pulling | 166 - | `forcePull()` | Pull without pushing | 167 - | `dispose()` | Release all resources | 154 + | Method | Description | 155 + | ------------------------- | ------------------------ | 156 + | `connect(url)` | Connect to sync server | 157 + | `disconnect()` | Disconnect and stop sync | 158 + | `login(email, password)` | Authenticate user | 159 + | `signup(email, password)` | Create new account | 160 + | `logout()` | Log out and stop sync | 161 + | `setToken(token)` | Set auth token directly | 162 + | `startSync()` | Begin periodic sync | 163 + | `stopSync()` | Stop periodic sync | 164 + | `sync()` | Run single sync cycle | 165 + | `forcePush()` | Push without pulling | 166 + | `forcePull()` | Pull without pushing | 167 + | `dispose()` | Release all resources | 168 168 169 169 ### Properties 170 170 171 - | Property | Type | Description | 172 - |----------|------|-------------| 173 - | `connected` | `boolean` | Whether connected to server | 174 - | `authenticated` | `boolean` | Whether authenticated | 175 - | `syncing` | `boolean` | Whether sync in progress | 176 - | `lastSync` | `string \| null` | ISO timestamp of last sync | 177 - | `appId` | `string` | Current app ID | 178 - | `api` | `SyncApi \| null` | Underlying API client | 171 + | Property | Type | Description | 172 + | --------------- | ----------------- | --------------------------- | 173 + | `connected` | `boolean` | Whether connected to server | 174 + | `authenticated` | `boolean` | Whether authenticated | 175 + | `syncing` | `boolean` | Whether sync in progress | 176 + | `lastSync` | `string \| null` | ISO timestamp of last sync | 177 + | `appId` | `string` | Current app ID | 178 + | `api` | `SyncApi \| null` | Underlying API client | 179 179 180 180 ### SyncApi 181 181 182 - | Method | Description | 183 - |--------|-------------| 184 - | `login(email, password)` | Authenticate | 185 - | `signup(email, password)` | Create account | 186 - | `logout()` | Log out | 187 - | `verifyToken()` | Check token validity | 188 - | `getMe()` | Get current user profile | 189 - | `listApps()` | List user's apps | 190 - | `createApp(name, desc?)` | Create app | 191 - | `getApp(id)` | Get app by ID | 192 - | `deleteApp(id)` | Delete app | 193 - | `listAppTokens(appId)` | List app tokens | 194 - | `createAppToken(appId, name, perms?)` | Create token | 195 - | `deleteAppToken(appId, tokenId)` | Delete token | 196 - | `registerApp(appId, schema)` | Register schema | 197 - | `pushChanges(appId, changes, hlc)` | Push changes | 198 - | `pullChanges(appId, sinceHlc, store?)` | Pull changes | 199 - | `uploadBlob(hash, data)` | Upload blob | 200 - | `downloadBlob(hash)` | Download blob | 201 - | `hasBlobRemote(hash)` | Check blob exists | 182 + | Method | Description | 183 + | -------------------------------------- | ------------------------ | 184 + | `login(email, password)` | Authenticate | 185 + | `signup(email, password)` | Create account | 186 + | `logout()` | Log out | 187 + | `verifyToken()` | Check token validity | 188 + | `getMe()` | Get current user profile | 189 + | `listApps()` | List user's apps | 190 + | `createApp(name, desc?)` | Create app | 191 + | `getApp(id)` | Get app by ID | 192 + | `deleteApp(id)` | Delete app | 193 + | `listAppTokens(appId)` | List app tokens | 194 + | `createAppToken(appId, name, perms?)` | Create token | 195 + | `deleteAppToken(appId, tokenId)` | Delete token | 196 + | `registerApp(appId, schema)` | Register schema | 197 + | `pushChanges(appId, changes, hlc)` | Push changes | 198 + | `pullChanges(appId, sinceHlc, store?)` | Pull changes | 199 + | `uploadBlob(hash, data)` | Upload blob | 200 + | `downloadBlob(hash)` | Download blob | 201 + | `hasBlobRemote(hash)` | Check blob exists | 202 202 203 203 ## Conflict Resolution 204 204
+13 -1
packages/sync/SPEC.md
··· 29 29 The main orchestrator class that wraps store instances and manages the sync lifecycle. 30 30 31 31 **Responsibilities:** 32 + 32 33 - Connect/disconnect from server 33 34 - Authenticate users 34 35 - Coordinate periodic push/pull sync ··· 37 38 - Manage blob sync (upload/download) 38 39 39 40 **State Machine:** 41 + 40 42 ``` 41 43 disconnected → connected → authenticated → syncing 42 44 ▲ │ │ │ ··· 45 47 ``` 46 48 47 49 **Key Methods:** 50 + 48 51 - `connect(url)` — Create SyncApi client 49 52 - `login()`/`signup()`/`setToken()` — Authenticate 50 53 - `startSync()` — Begin periodic sync loop ··· 53 56 - `disconnect()`/`dispose()` — Cleanup 54 57 55 58 **Events:** 59 + 56 60 - `connected` — Server connection established 57 61 - `disconnected` — Server disconnected 58 62 - `auth` — Authentication state changed ··· 64 68 HTTP client for the Civility Sync server REST API. 65 69 66 70 **Endpoints:** 71 + 67 72 - `POST /login` — Authenticate 68 73 - `POST /signup` — Create account 69 74 - `POST /logout` — Log out ··· 77 82 - `POST/HEAD/GET /blobs/:hash` — Blob operations 78 83 79 84 **Error Handling:** 85 + 80 86 - Throws `ApiError` on failure 81 87 - `ApiError.status` — HTTP status code 82 88 - `ApiError.response` — Parsed API response ··· 86 92 Shared types used by both Synced and SyncApi. 87 93 88 94 **Key Types:** 95 + 89 96 - `AuthToken` — Auth response with user info 90 97 - `UserProfile` — User data 91 98 - `App` — App record (legacy compatibility) ··· 99 106 ### Change Format 100 107 101 108 Changes are `ChangeEntry` objects from `@civility/store`: 109 + 102 110 - `id` — Unique change ID 103 111 - `docId` — Document ID 104 112 - `hlc` — Hybrid Logical Clock timestamp ··· 124 132 ### HLC (Hybrid Logical Clock) 125 133 126 134 The sync protocol uses HLC timestamps for conflict resolution: 135 + 127 136 - Combines logical timestamps with wall-clock time 128 137 - Ensures causality is preserved across devices 129 138 - Newer timestamp wins in conflicts (default) ··· 136 145 2. **Download:** After pulling changes, extract blob refs, check local store, download missing 137 146 138 147 Blob references use this structure: 148 + 139 149 ```ts 140 150 interface BlobRef { 141 - hash: string // sha256:... 151 + hash: string // sha256:... 142 152 mime: string 143 153 size: number 144 154 } ··· 147 157 ## Testing 148 158 149 159 Tests are in `__tests__/`: 160 + 150 161 - `synced.test.ts` — Synced orchestrator tests 151 162 - `api.test.ts` — SyncApi client tests 152 163 153 164 Run tests: 165 + 154 166 ```sh 155 167 deno test packages/sync 156 168 ```