Monorepo for Aesthetic.Computer aesthetic.computer
4
fork

Configure Feed

Select the types of activity you want to include in your feed.

at main 275 lines 8.5 kB view raw view rendered
1# Edge Personalized ISO Delivery 2 3## Problem 4 5Personalized AC OS images (with handle, auth token, Claude/GitHub tokens baked 6into `config.json`) are currently built on-demand by the oven in NYC. Users far 7from NYC (e.g. Novosibirsk) see slow downloads because every request does a full 8roundtrip to the origin. 9 10## Approach: Edge-Side Byte Patching 11 12The template ISO contains a fixed-size **identity block** at a known offset on 13the FAT32 EFI System Partition. The block is zero-padded placeholder data. 14 15A Cloudflare Worker fetches the template ISO from R2 (edge-cached globally), 16patches the identity block with the user's config, and streams the result. No 17origin roundtrip. First download is fast everywhere. 18 19### Identity Block 20 21The identity block lives on the uncompressed FAT32 partition inside the ISO — 22not inside the compressed initramfs. This means byte-level patching with zero 23decompression overhead. 24 25**Current (v1): 32KB — credentials + config** 26```json 27{ 28 "handle": "max", 29 "piece": "notepat", 30 "sub": "auth0|...", 31 "email": "max@example.com", 32 "token": "eyJ0eXAi...", 33 "claudeToken": "sk-ant-...", 34 "githubPat": "ghp_..." 35} 36``` 37~300 bytes of JSON, zero-padded to 32,768 bytes. Plenty of room to add fields. 38 39**Future (v2): 8MB — user identity pack** 40The block grows to include the user's creative data: 41``` 42[0x0000 - 0x7FFF] 32KB config.json (zero-padded) 43[0x8000 - 0x0FFF] 32KB manifest.json (file index) 44[0x10000 - ...] ~7.9MB user data: 45 - paintings (PNG thumbnails) 46 - audio samples (WAV/PCM) 47 - KidLisp pieces (.lisp source) 48 - notepat patterns 49 - custom themes 50``` 51This means a freshly flashed device boots with the user's creative identity 52already present — their paintings on the wall, their samples loaded, their 53pieces ready to run. 54 55The 8MB block on a ~128MB ISO is only 6% overhead. Could go larger if needed. 56 57### Marker & Offset 58 59The build script writes a known marker at the start of the identity block: 60 61``` 62AC_IDENTITY_BLOCK_V1\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 63``` 64 65The build also records the byte offset in a manifest file uploaded alongside 66the ISO: 67 68```json 69{ 70 "name": "oxide-tegu", 71 "hash": "6108a9738", 72 "timestamp": "2026-03-18T...", 73 "identityBlockOffset": 1048576, 74 "identityBlockSize": 32768 75} 76``` 77 78The Worker uses the offset from the manifest to seek directly — no scanning. 79The marker is a safety check to verify alignment before patching. 80 81## Architecture 82 83``` 84User (Siberia) 85 86 87Cloudflare Edge POP (nearest) 88 89 90oven-edge Worker 91 92 ├─ 1. Authenticate user (verify AC token via oven-origin) 93 ├─ 2. Fetch user config from oven-origin /api/user-config 94 ├─ 3. Fetch template ISO from R2 (edge-cached, ~128MB) 95 ├─ 4. Stream ISO, patching identity block on the fly 96 └─ 5. Done — no origin needed for the bulk data 97``` 98 99### Streaming Patch (no buffering the whole ISO) 100 101The Worker streams the template ISO from R2 and patches on-the-fly: 102 103```js 104async function streamPatchedISO(templateBody, config, manifest) { 105 const offset = manifest.identityBlockOffset; 106 const size = manifest.identityBlockSize; 107 const patch = makeIdentityBlock(config, size); // JSON + zero-pad 108 109 // TransformStream that patches bytes at the known offset 110 let bytesSeen = 0; 111 const { readable, writable } = new TransformStream({ 112 transform(chunk, controller) { 113 const chunkStart = bytesSeen; 114 const chunkEnd = bytesSeen + chunk.length; 115 bytesSeen = chunkEnd; 116 117 // Does this chunk overlap the identity block? 118 if (chunkEnd > offset && chunkStart < offset + size) { 119 const buf = new Uint8Array(chunk); 120 const patchStart = Math.max(0, offset - chunkStart); 121 const patchOffset = Math.max(0, chunkStart - offset); 122 const patchLen = Math.min( 123 size - patchOffset, 124 chunk.length - patchStart 125 ); 126 buf.set(patch.subarray(patchOffset, patchOffset + patchLen), patchStart); 127 controller.enqueue(buf); 128 } else { 129 controller.enqueue(chunk); 130 } 131 }, 132 }); 133 134 templateBody.pipeTo(writable); 135 return readable; 136} 137``` 138 139This uses ~zero memory overhead — chunks flow through, only the identity block 140region gets patched. The rest is untouched passthrough. 141 142## Implementation Steps 143 144### 1. Bump identity block to 32KB in build script 145 146In `ac-os` (line ~646), change the config.json padding from 4096 to 32768. 147Add the marker header. Record the offset. 148 149### 2. Enable Cloudflare R2 150 151- Enable R2 on the Cloudflare account (free 10GB, free egress) 152- Create bucket: `ac-os-images` 153- Bind to `oven-edge` Worker as `OS_IMAGES` 154 155### 3. Upload to R2 on publish 156 157Modify `ac-os upload`: 158 159```bash 160# After existing S3 upload: 161wrangler r2 object put ac-os-images/builds/${BUILD_NAME}/template.iso \ 162 --file /tmp/ac-native.iso 163wrangler r2 object put ac-os-images/builds/${BUILD_NAME}/manifest.json \ 164 --file /tmp/ac-manifest.json 165``` 166 167### 4. Add user-config API to oven 168 169New endpoint: `GET /api/user-config` (authenticated) 170 171Returns the user's full identity payload: 172```json 173{ 174 "handle": "max", 175 "piece": "notepat", 176 "sub": "auth0|...", 177 "email": "max@example.com", 178 "token": "<ac-auth-token>", 179 "claudeToken": "sk-ant-...", 180 "githubPat": "ghp_..." 181} 182``` 183 184The Worker calls this once per download — tiny request, fast. 185 186### 5. Edge-side patching in oven-edge Worker 187 188Add `/os-image` route to the Worker: 189 190```js 191async function handleOSImage(request, env) { 192 // 1. Auth — forward token to oven-origin 193 const token = request.headers.get("Authorization"); 194 const configRes = await fetch(ORIGIN + "/api/user-config", { 195 headers: { Authorization: token }, 196 }); 197 const config = await configRes.json(); 198 199 // 2. Get latest manifest 200 const manifestObj = await env.OS_IMAGES.get("latest-manifest.json"); 201 const manifest = await manifestObj.json(); 202 203 // 3. Get template ISO from R2 204 const iso = await env.OS_IMAGES.get( 205 `builds/${manifest.name}/template.iso` 206 ); 207 208 // 4. Stream with patch 209 const patched = await streamPatchedISO(iso.body, config, manifest); 210 211 return new Response(patched, { 212 headers: { 213 "Content-Type": "application/octet-stream", 214 "Content-Disposition": `attachment; filename="ac-${manifest.name}.iso"`, 215 "Content-Length": String(manifest.isoSize), 216 "X-Build": manifest.name, 217 "X-Edge-Pop": request.cf?.colo || "unknown", 218 }, 219 }); 220} 221``` 222 223### 6. Cache invalidation 224 225Only the template ISO is cached (in R2). Personalized ISOs are generated 226on-the-fly by patching — nothing to invalidate. When a new build drops: 227 228```bash 229# Upload new template + manifest, overwrite latest pointer 230wrangler r2 object put ac-os-images/latest-manifest.json \ 231 --file /tmp/ac-manifest.json 232``` 233 234Old builds auto-expire via R2 lifecycle rules (e.g. delete after 30 days). 235 236### 7. Native C code: read from identity block 237 238On boot, `ac-native.c` already reads `/mnt/config.json`. The FAT32 mount 239exposes the identity block as a regular file. No C changes needed for v1. 240 241For v2 (8MB identity pack), add a boot-time unpacker: 242```c 243// Read /mnt/identity.bin → extract paintings to /mnt/user/paintings/ 244// Extract samples to /mnt/user/samples/ 245// Extract pieces to /mnt/user/pieces/ 246``` 247 248## Cost Estimate 249 250- **R2 storage**: ~128MB per build (template only). 10 builds = 1.3GB. Free tier. 251- **R2 reads**: Template fetched once per download. Class B reads are free. 252- **R2 egress**: Free (!) 253- **Worker CPU**: Streaming patch uses minimal CPU. Well within free tier. 254- **No per-user storage**: Personalized ISOs are never stored, only streamed. 255 256Effectively free at any scale. 257 258## Rollout 259 2601. Bump identity block to 32KB, add marker, record offset in manifest 2612. Enable R2, create bucket, bind to Worker 2623. Modify `ac-os upload` to push template ISO + manifest to R2 2634. Add `/api/user-config` endpoint to oven 2645. Add streaming patch to `oven-edge` Worker 2656. Test with @jeffrey (NYC) and @sat (Novosibirsk) 2667. Add edge POP display to os.mjs download UI 267 268## Future: 8MB Identity Pack (v2) 269 270When ready to include user data in the image: 2711. Grow identity block to 8MB in build script 2722. Add user data export API to oven (paintings, samples, pieces) 2733. Worker fetches user data, packs into identity block format 2744. Native C unpacker extracts on first boot 2755. User boots a fresh device and their creative world is already there