Monorepo for Aesthetic.Computer
aesthetic.computer
1# Video Reframe Issue - Compositing Stack Investigation
2
3**Date:** 2025-10-19
4**Updated:** 2025-10-20 (Refocused on actual issue)
5**Problem:** Screen buffer freezes between reframes when `rec.present()` is enabled
6**Status:** � IN PROGRESS - Compositing stack issue in bios.mjs, NOT button repositioning
7
8---
9
10## 🎯 ACTUAL ISSUE (2025-10-20)
11
12### Corrected Problem Statement
13
14The issue is **NOT about button repositioning** - buttons work fine and reposition correctly.
15
16**The real problem:** During tape playback (`rec.present()` enabled), the screen buffer **temporarily freezes** when the window is resized. This causes:
17- Brief animation pause during dimension sync
18- Dimension mismatch between canvas and imageData
19- Multiple rapid reframe cycles before stabilizing
20
21**Root cause location:** The bios.mjs compositing stack, specifically around lines 11920-12520 where imageData is created and composited to canvas during underlayFrame (tape playback) mode.
22
23---
24
25## 🔍 Cleaned Up Logging (2025-10-20)
26
27### Logs Removed
28
29**video.mjs:**
30- ❌ Removed: `"✅ Buttons created at screen: X x Y"`
31- ❌ Removed: `"🎨 VIDEO PAINT: returning true"`
32- ❌ Removed: `"🎨 VIDEO PAINT: returning true (no buttons yet)"`
33
34**bios.mjs:**
35- ❌ Removed: `"🔄 REFRAME PATH (fallback)"`
36- ❌ Removed: `"📸 VIDEO: Created imageData in normal path"`
37- ❌ Removed: `"🚫 BLOCKED: imageData creation blocked"`
38- ❌ Removed: `"🔄 REFRAME PATH (main)"`
39- ❌ Removed: `"🎬 VIDEO: pixelsDidChange = ..."`
40
41**disk.mjs:**
42- ❌ Removed: `"🖌️ WIPE: Using screen dimensions X x Y"` (fired every frame)
43
44### Logs Kept/Updated
45
46**bios.mjs:**
47- ✅ `"⏸️ REFRAME: Dimension mismatch during tape playback. Canvas: X x Y | ImageData: X x Y"`
48 - Only logs during dimension mismatch (the freeze condition)
49 - Shows exactly what dimensions don't match
50
51**disk.mjs:**
52- ✅ `"📐 REFRAME: Worker dimensions updated X x Y → X x Y"`
53 - Only logs when dimensions actually change
54 - Shows the transition clearly
55
56### Expected Log Pattern
57
58**Normal reframe (working correctly):**
59```
60📐 REFRAME: Worker dimensions updated 300 x 197 → 300 x 215
61[smooth continuation - no freeze]
62```
63
64**Problematic reframe (freeze condition):**
65```
66⏸️ REFRAME: Dimension mismatch during tape playback. Canvas: 300 x 197 | ImageData: 300 x 211
67⏸️ REFRAME: Dimension mismatch during tape playback. Canvas: 300 x 197 | ImageData: 300 x 211
68[repeats multiple times - THIS IS THE FREEZE]
69📐 REFRAME: Worker dimensions updated 300 x 197 → 300 x 215
70[freeze ends - animation resumes]
71```
72
73---
74
75## 📊 Pipeline Flow During Reframe
76
77### What Should Happen
78
791. **User resizes window**
802. **Bios detects resize** → sends `"reframed"` message to worker
813. **Worker updates dimensions** → `screen.width/height` updated immediately
824. **Worker paints** → uses new dimensions in wipe/paint
835. **Worker sends pixels** → buffer matches new canvas size
846. **Bios receives pixels** → imageData dimensions match canvas
857. **Bios composites** → putImageData succeeds, overlays paint
868. ✅ **Animation continues smoothly**
87
88### What Actually Happens (Freeze Condition)
89
901. **User resizes window**
912. **Bios detects resize** → canvas resizes immediately
923. **Bios sends `"reframed"`** → but worker hasn't processed yet
934. **Worker still painting** → using OLD dimensions
945. **Worker sends pixels** → buffer has OLD dimensions
956. **Bios receives pixels** → imageData.width !== canvas.width ❌
967. **Bios BLOCKS rendering** → `⏸️ REFRAME: Dimension mismatch` logged
978. **Bios requests repaint** → setTimeout needs-paint
989. **[Steps 4-8 repeat]** → THIS IS THE FREEZE LOOP
9910. **Worker finally processes reframe** → `📐 REFRAME: Worker dimensions updated`
10011. **Worker paints with new dimensions** → buffer matches canvas
10112. **Bios receives matching pixels** → compositing succeeds
10213. ✅ **Animation resumes**
103
104### The Freeze Duration
105
106The freeze lasts for the number of frames it takes for:
107- The worker to receive the `"reframed"` message
108- The worker to process it and update dimensions
109- The worker to complete a paint cycle with new dimensions
110- The bios to receive the matching buffer
111
112Typically **10-30 frames** (shown in your logs as ~30 repetitions of the mismatch message).
113
114---
115
116## 🔬 Investigation Findings (2025-10-20)
117
118### Worker Message Queue Analysis
119
120**Architecture discovered:**
121```javascript
122// disk.mjs line 7207
123onmessage = makeFrame;
124
125async function makeFrame({ data: { type, content } }) {
126 // Sequential if-else chain, no queue
127 if (type === "init-from-bios") { /* ... */ return; }
128 if (type === "needs-paint") { noPaint = false; return; }
129 if (type === "reframed") {
130 // Lines 8234-8260: Updates dimensions immediately
131 screen.width = content.width;
132 screen.height = content.height;
133 screen.pixels = new Uint8ClampedArray(content.width * content.height * 4);
134 reframed = true;
135 return; // Early return - doesn't paint
136 }
137 if (type === "frame") { /* Main paint loop */ }
138 // ...
139}
140```
141
142**Key findings:**
143
1441. **No message priority system** - Messages processed in arrival order
1452. **Reframe returns early** - Updates dimensions but doesn't trigger paint
1463. **Next "frame" message** - Uses new dimensions for paint
1474. **The delay:** The time between:
148 - Bios sends `"reframed"` message
149 - Worker receives and processes it (updates dimensions)
150 - Bios sends `"needs-paint"`
151 - Worker receives and processes it
152 - Worker completes paint with new dimensions
153 - Worker sends buffer back to bios
154 - Bios receives matching buffer
155
156### Message Flow During Reframe
157
158**Timeline:**
159```
160T+0ms: User resizes window
161T+0ms: Bios canvas resizes (synchronous DOM)
162T+0ms: Bios sends "reframed" message
163T+0ms: Bios sends "needs-paint" message
164T+0ms: Worker painting frame N with OLD dimensions
165T+16ms: Worker completes frame N, sends OLD buffer
166T+16ms: Bios receives OLD buffer → MISMATCH #1
167T+16ms: Bios setTimeout → sends "needs-paint"
168T+16ms: Worker processes "reframed" → updates dimensions
169T+16ms: Worker processes "needs-paint" → noPaint = false
170T+32ms: Worker paints frame N+1 with NEW dimensions
171T+32ms: Worker sends NEW buffer
172T+32ms: Bios receives NEW buffer → MATCH! ✅
173```
174
175**Actual freeze duration: ~1-2 frames** (16-32ms)
176
177But because each mismatch triggers another paint request, and if the worker is busy, multiple frames can accumulate before sync, resulting in the 10-30 frame freeze seen in logs.
178
179### **ROOT CAUSE IDENTIFIED** (2025-10-20) 🎯
180
181The freeze was caused by a **buffer recycling bug** in disk.mjs:
182
183**The Bug:**
184```javascript
185// disk.mjs line 8298 (BEFORE FIX)
186if (type === "frame") {
187 let pixels;
188 if (content.pixels) {
189 pixels = new Uint8ClampedArray(content.pixels);
190 if (screen) screen.pixels = pixels; // ← BLINDLY OVERWRITES!
191 }
192}
193```
194
195**What happens:**
1961. User resizes → Bios sends `"reframed"` with new dimensions (300x145)
1972. Worker processes `"reframed"` → Creates NEW buffer: `screen.pixels = new Uint8ClampedArray(300 * 145 * 4)`
1983. Bios sends `"frame"` with OLD buffer (from previous size 300x154)
1994. Worker processes `"frame"` → **OVERWRITES** new buffer with old buffer!
2005. Worker paints → Sends buffer with 184,800 bytes but claims 300x145 (180,000 bytes expected)
2016. Bios receives → Dimension mismatch! → Freeze loop begins
202
203**The Fix:**
204```javascript
205// disk.mjs line 8298 (AFTER FIX)
206if (content.pixels) {
207 pixels = new Uint8ClampedArray(content.pixels);
208 const expectedLength = screen.width * screen.height * 4;
209 if (screen && pixels.length === expectedLength) {
210 screen.pixels = pixels; // OK - sizes match
211 } else if (screen && pixels.length !== expectedLength) {
212 // REJECT mismatched buffer - keep the reframed buffer
213 console.log('⚠️ FRAME: Ignoring mismatched buffer from bios');
214 }
215}
216```
217
218**Expected result:** Freeze should drop from 20-170 frames to **0-1 frames** (instantaneous resize).
219
220---
221
222### Diagnostic Logging Added
223
224**Bios (bios.mjs):**
225- `📤 REFRAME: Sending reframe message to worker. New dimensions: X x Y`
226- `⏸️ REFRAME: Dimension mismatch #N. Canvas: X x Y | ImageData: X x Y`
227 - Logs first mismatch, then every 10th to avoid spam
228- `✅ REFRAME: Dimension sync restored after N mismatched frames. Canvas: X x Y`
229
230**Worker (disk.mjs):**
231- `📐 REFRAME: Worker dimensions updated X x Y → X x Y`
232 - Only logs when dimensions actually change
233
234---
235
236## 🔬 Investigation: Compositing Stack Freeze
237
238### The Core Issue
239
240**Location:** `bios.mjs` lines ~12500-12520 - the dimension mismatch handler
241
242**Current behavior:**
243```javascript
244if (underlayFrame) {
245 // During tape playback, keep the canvas at correct size and wait for matching data
246 console.log('⏸️ REFRAME: Dimension mismatch during tape playback. Canvas: ...
247 skipImmediateOverlays = true; // Don't paint overlays
248 // Keep requesting paint so we get fresh data with correct dimensions
249 setTimeout(() => send({ type: "needs-paint" }), 0);
250}
251```
252
253This creates a **busy-wait loop** that causes the freeze:
2541. Canvas is resized → new dimensions (e.g., 300 x 215)
2552. ImageData arrives → old dimensions (e.g., 300 x 197)
2563. Mismatch detected → skip rendering, request repaint
2574. Worker sends another frame → still old dimensions
2585. Loop continues until worker processes reframe message
259
260### Why It Happens
261
262**Timing issue in the reframe message pipeline:**
263
2641. **Bios is too fast:**
265 - Canvas resizes instantly (synchronous DOM operation)
266 - `"reframed"` message sent to worker (async postMessage)
267
2682. **Worker is delayed:**
269 - Worker event loop processes messages between paint cycles
270 - Multiple paint frames may complete before reframe message is processed
271 - Each paint uses old screen dimensions
272
2733. **Result:**
274 - Bios has new canvas size (215)
275 - Worker keeps sending old buffer size (197)
276 - Dimension check fails → freeze loop
277
278### Potential Solutions
279
280#### Option 1: Immediate Worker Notification ⭐ BEST
281Process `"reframed"` messages with **highest priority** in worker:
282- Move reframe handling to top of message queue
283- Process before paint, before any other messages
284- Update dimensions synchronously before next paint cycle
285
286**UPDATE:** Investigation shows no message queue exists - messages are processed sequentially via `onmessage`. The issue is that worker sends one more frame with old dimensions before processing reframe message.
287
288#### Option 2: Render Mismatched Data with Scaling ⭐ PRACTICAL
289Instead of blocking, **scale** the imageData to fit canvas:
290- Use `ctx.drawImage()` with source and dest rects
291- Temporary visual distortion vs complete freeze
292- Less jarring user experience
293
294#### Option 3: Cache Last Valid Frame
295During dimension mismatch:
296- Keep rendering the last valid frame
297- Don't request repaint until dimensions match
298- Avoids busy-wait loop
299
300#### Option 4: Predictive Dimension Update
301Worker preemptively checks for pending reframes:
302- Before each paint, check if canvas dimensions changed
303- Update screen dimensions proactively
304- Reduces lag between reframe message and dimension update
305
306---
307
308## 📋 Completion Summary (2025-10-20)
309
310### ✅ Issue RESOLVED
311
312The video button reframe issue has been **successfully fixed**! The screen buffer no longer freezes during window resize when playing back tapes with `rec.present()`.
313
314**Final Results:**
315- ✅ Buffer freeze duration: **20-170 frames → 0-1 frames** (instantaneous)
316- ✅ Dimension mismatch: **Eliminated** - buffer sizes match immediately
317- ✅ Button repositioning: **Working correctly** - buttons stay in corner positions
318- ✅ Animation continuity: **Perfect** - no visible freeze or stutter
319- ✅ Rapid resizing: **Handles smoothly** without artifacts
320
321**Root Cause:**
322Worker's frame message handler (disk.mjs line 8298) was blindly overwriting the correctly-sized reframed buffer with an old buffer transferred from bios.
323
324**Solution:**
325Added buffer size validation before accepting transferred buffers. Worker now rejects mismatched buffers and keeps the correctly-sized reframed buffer.
326
327**Files Modified:**
328- `disk.mjs` (line 8300-8315): Added buffer size validation
329- `disk.mjs` (line 8306): Fixed TypeError with screen existence check
330
331**Cleanup Status:**
332- ✅ Feature-rich video.mjs confirmed as production version (1576 lines)
333- ✅ No oldvideo.mjs file exists (file search showed duplicates)
334- ✅ Button repositioning works correctly (reposition() called every frame)
335- ✅ All export features intact (POST, MP4, GIF, ZIP)
336
337---
338
339### ✅ UI Polish & Code Sharing (2025-10-20)
340
341**Additional improvements:**
342- ✅ **Removed black bar** under buttons in video.mjs - cleaner transparent overlay
343- ✅ **Fixed button tap behavior** - ZIP/GIF/MP4 buttons no longer pause video when tapped
344 - Issue: Play/pause toggle only checked `postBtn.down`, causing other buttons to trigger pause
345 - Fix: Now checks if ANY button is down before toggling play/pause (lines 1147-1163)
346- ✅ **Created shared tape-player library** (`disks/common/tape-player.mjs`)
347 - Extracted common progress bar rendering from replay.mjs
348 - Functions: `deriveProgressState()`, `renderLoadingProgressBar()`, `renderStreamingBadge()`, `formatMegabytes()`
349 - Ready for both video.mjs and replay.mjs to share code
350- ✅ **Global progress bar** already working via `rec.tapeProgress` system
351 - Renders VHS-style red progress bar underneath video during exports/playback
352 - No additional code needed - bios.mjs handles the rendering
353- ✅ **Refactored replay.mjs** to use shared library
354 - Removed ~210 lines of duplicate code
355 - Now imports and uses shared functions
356 - Consistent UI rendering with video.mjs
357
358**Files Created/Modified:**
359- `disks/common/tape-player.mjs`: New shared library for tape playback UI (260 lines)
360- `video.mjs` (lines 203-207): Removed black button background bar
361- `video.mjs` (lines 1147-1163): Fixed play/pause toggle to check all buttons
362- `replay.mjs`: Removed duplicate functions, added imports, updated to use shared library
363- `plans/tape-player-refactor.md`: Detailed refactoring summary document
364
365**Next Steps:**
366Both video.mjs and replay.mjs now share common UI code for consistent tape operation feedback. See `tape-player-refactor.md` for complete refactoring analysis and future opportunities.
367
368---
369
370## 📋 Action Items
371
372### Phase 1: Confirm the Freeze Pattern ✅ DONE
373- [x] Clean up verbose logging
374- [x] Add focused reframe tracking
375- [x] Test window resize during tape playback
376- [x] Observe freeze duration and frequency
377
378### Phase 2: Investigate Worker Message Priority ✅ COMPLETE
379- [x] Check worker message queue implementation in disk.mjs
380 - **Finding:** No message priority queue exists
381 - **Finding:** Messages processed sequentially via `onmessage = makeFrame`
382 - **Finding:** `"reframed"` is processed as early-return (lines 8234-8260)
383 - **Finding:** Updates dimensions immediately but paint still uses old buffer
384- [x] Add diagnostic timing logs
385 - Bios logs when sending reframe message
386 - Worker logs when processing reframe message
387 - Bios counts dimension mismatch iterations
388 - Bios logs when sync is restored
389- [x] Measure actual delay between reframe sent and reframe processed
390 - **Result:** Worker updates dimensions immediately!
391 - **Problem:** Worker continues sending OLD buffer size for 20-170 frames
392 - **Added:** More detailed worker send logs to trace buffer size
393- [ ] Investigate why worker sends old buffer despite updating dimensions
394
395### Phase 3: Implement Solution ✅ COMPLETE
396- [x] Identify root cause
397 - **ROOT CAUSE FOUND:** Worker's "frame" handler overwrites correctly-sized buffer
398 - Line 8298: `screen.pixels = pixels` blindly overwrites with OLD buffer from bios
399 - After reframe creates new buffer, next frame message destroys it
400- [x] Implement fix
401 - **SOLUTION:** Validate buffer size before accepting it
402 - Only use transferred buffer if `pixels.length === screen.width * screen.height * 4`
403 - Keep reframed buffer if bios sends mismatched size
404- [x] Test fix
405 - [x] Verify dimension mismatch count drops to 0-1 frames ✅ **CONFIRMED**
406 - [x] Verify no visual freeze during resize ✅ **CONFIRMED**
407 - [x] Test rapid resizing ✅ **WORKS PERFECTLY**
408 - [x] Test slow dragging ✅ **WORKS PERFECTLY**
409
410### Phase 4: Performance Optimization
411- [ ] Measure frame drops during reframe
412- [ ] Optimize dimension update pipeline
413- [ ] Consider batching rapid reframes
414- [ ] Add hysteresis to prevent resize thrashing
415
416---
417
418## 🧪 Testing Checklist
419
420**Test during tape playback (`rec.present()` active):**
421- [ ] Slow window resize (drag corner smoothly)
422- [ ] Fast window resize (rapid dragging)
423- [ ] Multiple rapid resizes in succession
424- [ ] Maximize/restore window
425- [ ] Fullscreen toggle
426- [ ] Different aspect ratios
427
428**Expected results after fix:**
429- ✅ No visible freeze during resize
430- ✅ Animation continues smoothly
431- ✅ No frame drops or stuttering
432- ✅ Buttons remain responsive
433- ✅ No visual artifacts or stretching
434
435---
436
437## Problem Statement
438
439Despite multiple optimization attempts, the export buttons in `video.mjs` **still do not move** when the window is resized. The current implementation:
440
4411. ✅ Creates buttons once (persistent instances)
4422. ✅ Returns `true` from paint() for continuous rendering
4433. ✅ Detects screen dimension changes via `screenChanged` flag
4444. ✅ Calls `reposition()` only when screen changes
4455. ❌ **BUTTONS STILL DON'T MOVE AFTER REFRAME**
446
447### Current Implementation (video.mjs)
448
449```javascript
450// Module-level variables
451let postBtn, mp4Btn, gifBtn, zipBtn;
452let lastScreenWidth = 0;
453let lastScreenHeight = 0;
454
455function paint({ screen, api, wipe, ink, ui, /* ... */ }) {
456 // Detect screen changes
457 const screenChanged = screen.width !== lastScreenWidth || screen.height !== lastScreenHeight;
458 if (screenChanged) {
459 lastScreenWidth = screen.width;
460 lastScreenHeight = screen.height;
461 }
462
463 // Transparent wipe during video playback
464 if (rec.presenting || rec.playing) {
465 wipe(0, 0, 0, 0); // DOM video overlay shows through
466 } else {
467 wipe(255);
468 }
469
470 // Draw buttons
471 if (exportAvailable) {
472 if (!postBtn) {
473 postBtn = new ui.TextButton("POST", { right: 6, bottom: 6, screen });
474 } else if (screenChanged) {
475 postBtn.reposition({ right: 6, bottom: 6, screen });
476 }
477 postBtn.paint(api);
478 // ... similar for mp4Btn, gifBtn, zipBtn
479 }
480
481 return true; // Always repaint
482}
483```
484
485### Observations
486
4871. **Screen change detection works** - `screenChanged` flag correctly identifies when dimensions change
4882. **`reposition()` is called** - Only when screen dimensions change (optimization working)
4893. **Buttons paint every frame** - Return `true` keeps paint loop running
4904. **Transparent wipe used** - `wipe(0,0,0,0)` for DOM video passthrough
4915. **Buttons never move visually** - They stay at their original screen corner positions
492
493---
494
495## Comparison: Working Examples from Other Pieces
496
497### 1. gameboy.mjs - Simple Recreate Pattern
498
499```javascript
500export function paint({ ink, wipe, screen, paste, sound, num, hud, ui }) {
501 // Create buttons on first paint or when screen size changes
502 if (!uiButtons.up || uiButtons.up.box.w !== 30) {
503 createGameBoyButtons({ screen, ui });
504 }
505 // ... paint logic
506}
507
508function createGameBoyButtons({ screen, ui }) {
509 const buttonSize = 24;
510 const dpadY = screen.height - buttonSize * 3; // Calculate from screen
511
512 // Recreate all buttons with new positions
513 uiButtons.up = new ui.Button(dpadX + buttonSize, dpadY, buttonSize, buttonSize);
514 uiButtons.down = new ui.Button(dpadX + buttonSize, dpadY + buttonSize * 2, buttonSize, buttonSize);
515 // ... etc
516}
517```
518
519**Key differences:**
520- ❌ No `reposition()` method used
521- ✅ **Recreates buttons** when screen changes
522- ✅ Simple condition: check if button exists or size changed
523- 🤔 Works reliably without complexity
524
525### 2. stample.mjs - Direct Box Manipulation on Reframe
526
527```javascript
528function act({ event: e, screen, ui, /* ... */ }) {
529 if (e.is("reframed")) {
530 genPats({ screen, ui }); // Regenerate button grid
531
532 // Direct box manipulation
533 micRecordButton.box.y = screen.height - 32;
534 patsButton.box.x = screen.width - patsButton.box.w;
535 }
536}
537
538function genPats({ screen, ui }) {
539 btns.length = 0; // Clear array
540 for (let i = 0; i < pats; i += 1) {
541 // Recreate all buttons
542 const button = new ui.Button(x, y, width, height);
543 btns.push(button);
544 }
545}
546```
547
548**Key differences:**
549- ✅ Listens for `"reframed"` event in `act()`
550- ✅ **Directly modifies `button.box` properties**
551- ✅ Recreates button arrays completely
552- 🎯 **No reposition() method needed**
553
554### 3. notepat.mjs - Geometry Rebuild Pattern
555
556```javascript
557function setupButtons({ ui, screen, geo }) {
558 // Recalculate layout metrics from screen
559 const layout = getButtonLayoutMetrics(screen, { /* ... */ });
560
561 buttonNotes.forEach((label, i) => {
562 const geometry = [x, y, buttonWidth, buttonHeight];
563
564 if (!buttons[label]) {
565 buttons[label] = new ui.Button(...geometry);
566 } else {
567 // Replace box with new geo.Box instance
568 buttons[label].box = new geo.Box(...geometry);
569 }
570 });
571}
572```
573
574**Key differences:**
575- ✅ **Replaces entire `button.box` with new `geo.Box`**
576- ✅ Recalculates complete layout from screen dimensions
577- ✅ Called on reframe/resize events
578- 🎯 **Direct box replacement, not reposition()**
579
580### 4. prutti.mjs - Hybrid Pattern
581
582```javascript
583function act({ event: e, screen, ui, /* ... */ }) {
584 if (e.is("reframed")) {
585 if (scrubButton) {
586 // Update existing button's box properties
587 scrubButton.box.x = buttonStartX;
588 scrubButton.box.y = 0;
589 scrubButton.box.w = buttonWidth;
590 scrubButton.box.h = barHeight;
591 } else {
592 // Create new button
593 scrubButton = new ui.Button(buttonStartX, 0, buttonWidth, barHeight);
594 }
595 }
596}
597```
598
599**Key differences:**
600- ✅ Responds to `"reframed"` event
601- ✅ **Directly mutates box properties** (x, y, w, h)
602- ✅ Recreates if missing
603- 🎯 **Manual property assignment**
604
605### 5. painting.mjs - Simple Creation, No Reposition
606
607```javascript
608function boot({ screen, ui, /* ... */ }) {
609 if (!showMode) {
610 printBtn = new ui.TextButton(`Print`, {
611 bottom: butBottom,
612 right: butSide,
613 screen,
614 });
615 }
616}
617
618function paint({ screen, /* ... */ }) {
619 if (printBtn) {
620 printBtn.paint(api);
621 }
622}
623```
624
625**Key differences:**
626- ✅ Creates once in boot
627- ❌ **No resize/reframe handling at all**
628- 🤔 Buttons may not work correctly after resize
629- ⚠️ Not a good pattern for reframeable pieces
630
631---
632
633## Pattern Analysis Summary
634
635| Pattern | Used By | Reposition Method | Direct Box Access | Recreate | Event |
636|---------|---------|-------------------|-------------------|----------|-------|
637| **Recreate on detect** | gameboy | ❌ | ❌ | ✅ | paint check |
638| **Direct box mutation** | stample, prutti | ❌ | ✅ | Sometimes | `"reframed"` |
639| **Box replacement** | notepat | ❌ | ✅ (replace) | ❌ | setup call |
640| **Reposition method** | video (current) | ✅ | ❌ | ❌ | screen change |
641| **No handling** | painting | ❌ | ❌ | ❌ | ❌ |
642
643### Key Finding: **NO OTHER PIECE USES `reposition()` METHOD**
644
645All working examples use one of:
6461. **Recreate buttons** (gameboy)
6472. **Direct `button.box.x/y/w/h` mutation** (stample, prutti)
6483. **Replace `button.box` with new `geo.Box`** (notepat)
649
650**NONE** use the `button.reposition()` method that video.mjs is trying to use!
651
652---
653
654## ROOT CAUSE IDENTIFIED: `reposition()` Implementation
655
656### Investigation Result: ✅ **`reposition()` EXISTS AND SHOULD WORK**
657
658From `lib/ui.mjs` lines 839-842:
659
660```javascript
661reposition(pos, txt) {
662 if (txt) this.txt = txt;
663 this.btn.box = Box.from(this.#computePosition(this.txt, pos));
664}
665```
666
667**How it works:**
6681. Takes `pos` object with `{right, bottom, screen}` or `{left, top, screen}`
6692. Calls internal `#computePosition()` method to calculate x, y, w, h
6703. **Replaces `this.btn.box` with new `Box`** instance
671
672**The `#computePosition()` method (lines 817-835):**
673```javascript
674#computePosition(txt, pos) {
675 const m = TYPEFACE_UI.metrics(txt);
676 const w = m.box.w + 8;
677 const h = m.box.h + 8;
678
679 let x = 0;
680 let y = 0;
681
682 if (pos.bottom !== undefined) {
683 y += pos.screen.height - pos.bottom - h;
684 } else {
685 y += pos.top || 0;
686 }
687
688 if (pos.right !== undefined) {
689 x += pos.screen.width - pos.right - w;
690 } else {
691 x += pos.left || 0;
692 }
693
694 return { x, y, w, h };
695}
696```
697
698### ⚠️ CRITICAL FINDING: `reposition()` SHOULD WORK!
699
700The implementation looks correct:
701- ✅ Handles corner anchoring (right/bottom)
702- ✅ Uses current screen dimensions
703- ✅ Replaces box entirely (not mutating)
704- ✅ Recalculates position from scratch
705
706### 🔍 Why Doesn't It Work Then?
707
708**Hypothesis:** The issue might be that `reposition()` is being called but:
709
7101. **Paint order issue?** - Are buttons painted before reposition happens?
7112. **Screen object stale?** - Is the `screen` object in paint() up-to-date?
7123. **Transparent wipe issue?** - Old button graphics not clearing?
7134. **Box reference issue?** - Internal `this.btn` reference not updating?
7145. **Event timing?** - Need to respond to `"reframed"` event instead of detecting in paint?
715
716### Why video.mjs Uses `reposition()` When Others Don't
717
718Looking at the patterns:
719- `TextButton` has `reposition()` because it needs to recalculate text metrics
720- Regular `Button` doesn't have `reposition()` - pieces just recreate or mutate box
721- video.mjs is the **ONLY** piece using `TextButton.reposition()`
722- This is either cutting-edge API usage or... it's broken in practice
723
7243. **Does the transparent wipe affect button rendering?**
725 - `wipe(0,0,0,0)` doesn't clear canvas
726 - Are old button graphics persisting?
727 - Do buttons need an opaque clear/redraw after reframe?
728
7294. **Is screen change detection working correctly?**
730 - Add debug logging to verify `screenChanged` is true after resize
731 - Verify `reposition()` is actually being called
732 - Check if `screen` object has updated dimensions
733
734---
735
736## Proposed Solutions (Priority Order)
737
738### Solution 1: Use Direct Box Mutation (Like stample.mjs) ⭐ RECOMMENDED
739
740Switch from `reposition()` to direct box property mutation in `act()`:
741
742```javascript
743function act({ event: e, screen, /* ... */ }) {
744 if (e.is("reframed")) {
745 // Direct box manipulation for corner positioning
746 if (postBtn) {
747 postBtn.box.x = screen.width - 6 - postBtn.box.w;
748 postBtn.box.y = screen.height - 6 - postBtn.box.h;
749 }
750 if (mp4Btn) {
751 mp4Btn.box.x = screen.width - 44 - mp4Btn.box.w;
752 mp4Btn.box.y = screen.height - 6 - mp4Btn.box.h;
753 }
754 // ... etc for gifBtn, zipBtn
755 }
756}
757```
758
759**Pros:**
760- ✅ Proven pattern from working pieces
761- ✅ Direct control over position
762- ✅ Responds to reframe event (standard AC pattern)
763- ✅ No dependency on potentially broken `reposition()` method
764
765**Cons:**
766- ❌ Need to know button dimensions (width/height)
767- ⚠️ More manual calculation
768
769### Solution 2: Replace Box with geo.Box (Like notepat.mjs)
770
771```javascript
772function paint({ screen, geo, ui, /* ... */ }) {
773 const screenChanged = /* ... */;
774
775 if (!postBtn) {
776 postBtn = new ui.TextButton("POST", { right: 6, bottom: 6, screen });
777 } else if (screenChanged) {
778 const x = screen.width - 6 - postBtn.box.w;
779 const y = screen.height - 6 - postBtn.box.h;
780 postBtn.box = new geo.Box(x, y, postBtn.box.w, postBtn.box.h);
781 }
782}
783```
784
785**Pros:**
786- ✅ Proven pattern from notepat
787- ✅ Clean box replacement
788- ✅ Stays in paint() (no act() needed)
789
790**Cons:**
791- ❌ Still need manual position calculation
792- ❌ Requires geo.Box import
793
794### Solution 3: Recreate Buttons (Like gameboy.mjs)
795
796```javascript
797function paint({ screen, ui, /* ... */ }) {
798 const screenChanged = /* ... */;
799
800 if (!postBtn || screenChanged) {
801 postBtn = new ui.TextButton("POST", { right: 6, bottom: 6, screen });
802 mp4Btn = new ui.TextButton("MP4", { right: 44, bottom: 6, screen });
803 gifBtn = new ui.TextButton("GIF", { right: 76, bottom: 6, screen });
804 zipBtn = new ui.TextButton("ZIP", { right: 108, bottom: 6, screen });
805 }
806}
807```
808
809**Pros:**
810- ✅ Simplest solution
811- ✅ Proven pattern
812- ✅ TextButton constructor handles positioning
813
814**Cons:**
815- ⚠️ Recreating objects (may lose internal state?)
816- ⚠️ Slightly less efficient
817- ❓ Will this work with TextButton's corner positioning API?
818
819### Solution 4: Create New Piece (newvideo.mjs) for Testing
820
821Copy `video.mjs` → `newvideo.mjs` and test different patterns without breaking existing video playback:
822
823```fish
824cp system/public/aesthetic.computer/disks/video.mjs system/public/aesthetic.computer/disks/newvideo.mjs
825```
826
827Then test each solution in isolation.
828
829---
830
831## Investigation Tasks
832
833- [✅] **Task 1:** Check `lib/ui.mjs` - Does `TextButton.reposition()` exist and work?
834 - **Result:** YES - It exists and implementation looks correct!
835 - See analysis above for details
836- [ ] **Task 2:** Add debug logging to verify:
837 - `screenChanged` is true after resize
838 - `reposition()` is being called
839 - Screen dimensions in paint vs actual window size
840- [ ] **Task 3:** Try Solution 1 (direct box mutation in act with reframed event)
841- [ ] **Task 4:** Try Solution 3 (recreate buttons on screen change)
842- [ ] **Task 5:** Create `newvideo.mjs` test piece to isolate changes
843- [ ] **Task 6:** Document which pattern works and why
844
845---
846
847## Debug Changes Added
848
849The following debug logging has been added to `video.mjs`:
850
8511. **Screen change detection** (line ~186):
852 ```javascript
853 if (screenChanged) {
854 console.log("🔄 SCREEN CHANGED:", {
855 old: { w: lastScreenWidth, h: lastScreenHeight },
856 new: { w: screen.width, h: screen.height }
857 });
858 }
859 ```
860
8612. **Button creation** (line ~217):
862 ```javascript
863 if (!postBtn) {
864 postBtn = new ui.TextButton("POST", { right: 6, bottom: 6, screen });
865 console.log("✅ POST button created at:", postBtn.btn.box);
866 }
867 ```
868
8693. **Button reposition** (line ~219):
870 ```javascript
871 else if (screenChanged) {
872 const oldBox = { ...postBtn.btn.box };
873 postBtn.reposition({ right: 6, bottom: 6, screen });
874 console.log("🔄 POST button repositioned:", {
875 old: oldBox,
876 new: postBtn.btn.box,
877 screen: { w: screen.width, h: screen.height }
878 });
879 }
880 ```
881
8824. **Reframed event** (line ~497):
883 ```javascript
884 if (e.is("reframed")) {
885 console.log("📐 REFRAMED event detected in act():", {
886 screen: { w: screen.width, h: screen.height },
887 postBtn: postBtn?.btn.box,
888 mp4Btn: mp4Btn?.btn.box
889 });
890 }
891 ```
892
893### What to Look For
894
895When you resize the window, check the console for:
896- ✅ Does "🔄 SCREEN CHANGED" appear?
897- ✅ Does "🔄 POST button repositioned" appear?
898- ✅ Does the `new` box position match the new screen dimensions?
899- ✅ Does "📐 REFRAMED event" appear in act()?
900- ❓ Is there a timing difference between reframed event and screen change detection?
901
902## Next Steps
903
9041. **Test with debug logging** - Resize window and observe console output
9052. **Analyze findings** - Determine if:
906 - Screen change is detected ✓
907 - Reposition is called ✓
908 - New box coordinates are correct ✓
909 - But buttons still don't move visually ✗
9103. **If reposition() is working but not visible**:
911 - Try adding opaque wipe after reframe
912 - Try listening to reframed event in paint()
913 - Try direct box mutation pattern from stample.mjs
9144. **If reposition() is NOT being called**:
915 - Screen change detection logic may be wrong
916 - Screen object may not be updating
9175. **Create `newvideo.mjs`** - Test alternative patterns without breaking existing functionality
918
919---
920
921## ⚠️ PARTIAL RESOLUTION (2025-10-20) - ISSUE STILL OPEN
922
923### Progress: Fixed Canvas Freeze, Button Repositioning Still Broken
924
925The button repositioning issue revealed a deeper architectural problem in the worker-bios rendering pipeline during tape playback with `rec.present()`. **The underlying freeze has been fixed, but buttons still do not reposition after window resize.**
926
927---
928
929## 🔍 LOG ANALYSIS (2025-10-20)
930
931### Current Implementation Issues Found
932
933**Problem 1: Misleading Debug Logs**
934The current video.mjs has both log statements firing every frame:
935```javascript
936if (exportAvailable) {
937 console.log('🎨 VIDEO PAINT: returning true'); // Inside if
938 // ... paint buttons
939}
940console.log('🎨 VIDEO PAINT: returning true (no buttons yet)'); // Outside if - ALWAYS FIRES!
941return true;
942```
943
944This makes debugging impossible because we see both messages even when buttons exist.
945
946**Problem 2: Buttons Reposition Every Frame**
947The current code repositions buttons on EVERY paint() call:
948```javascript
949postBtn.reposition({ right: 6, bottom: 6, screen });
950mp4Btn.reposition({ right: 44, bottom: 6, screen });
951gifBtn.reposition({ right: 76, bottom: 6, screen });
952zipBtn.reposition({ right: 108, bottom: 6, screen });
953```
954
955While this should work, it's inefficient and the logs show no evidence of:
956- Button creation (no "✅ Buttons created" message)
957- Button position changes
958- Screen dimension tracking
959
960**Problem 3: No Screen Change Detection**
961Unlike the documented plan, the current code has NO screen change detection:
962```javascript
963// MISSING:
964// let lastScreenWidth = 0;
965// let lastScreenHeight = 0;
966// const screenChanged = screen.width !== lastScreenWidth || screen.height !== lastScreenHeight;
967```
968
969### Log Evidence
970
971From recent console output:
972```
973🖌️ WIPE: Using screen dimensions 300 x 197
974⏭️ VIDEO: Dimension mismatch - waiting for worker. Canvas: 300 x 197 ImageData: 300 x 211
975🎨 VIDEO PAINT: returning true
976🎨 VIDEO PAINT: returning true (no buttons yet)
977[Repeats ~30 times during dimension sync]
978
979📐 WORKER: Updated screen dimensions from 300 x 197 to 300 x 215
980🔄 REFRAME PATH (fallback): Created fresh imageData with dimensions: 300 x 215
981
982🖌️ WIPE: Using screen dimensions 300 x 215
983📸 VIDEO: Created imageData in normal path
984🎨 VIDEO PAINT: returning true
985🎨 VIDEO PAINT: returning true (no buttons yet)
986[Continues indefinitely]
987```
988
989**Key observations:**
9901. ✅ Worker dimension updates work correctly
9912. ✅ Dimension mismatch fallback prevents canvas freeze
9923. ❌ NO button creation logs appear
9934. ❌ BOTH log messages fire every frame (logic error)
9945. ❓ Are buttons even being created? (`exportAvailable` might be false)
995
996### Hypothesis
997
998**Buttons may not be created at all** because:
9991. `rec.presenting` may be false during initial playback
10002. `rec.recorded` may not be set yet
10013. `exportAvailable` evaluates to false, so buttons never instantiate
1002
1003**OR** buttons ARE created but:
10041. `reposition()` method doesn't actually update button positions
10052. Button rendering happens at stale coordinates
10063. The UI system doesn't pick up box changes from reposition()
1007
1008### Required Diagnostic Changes
1009
1010To properly debug, video.mjs needs:
1011
1012```javascript
1013// Track screen changes
1014let lastScreenWidth = 0;
1015let lastScreenHeight = 0;
1016
1017function paint({ wipe, ink, screen, rec, ui, api, needsPaint }) {
1018 const screenChanged = screen.width !== lastScreenWidth || screen.height !== lastScreenHeight;
1019
1020 if (screenChanged) {
1021 console.log('📐 VIDEO: Screen changed from', lastScreenWidth, 'x', lastScreenHeight,
1022 'to', screen.width, 'x', screen.height);
1023 lastScreenWidth = screen.width;
1024 lastScreenHeight = screen.height;
1025 }
1026
1027 const presenting = rec?.presenting ?? false;
1028 const exportAvailable = presenting || (rec?.recorded ?? false);
1029
1030 console.log('📊 VIDEO: exportAvailable =', exportAvailable,
1031 'presenting =', presenting,
1032 'recorded =', rec?.recorded);
1033
1034 if (exportAvailable) {
1035 if (!postBtn) {
1036 postBtn = new ui.TextButton("POST", { right: 6, bottom: 6, screen });
1037 console.log('✅ POST button CREATED at:', postBtn.btn.box);
1038 } else if (screenChanged) {
1039 const oldBox = { ...postBtn.btn.box };
1040 postBtn.reposition({ right: 6, bottom: 6, screen });
1041 console.log('🔄 POST button REPOSITIONED from', oldBox, 'to', postBtn.btn.box);
1042 }
1043
1044 console.log('🖼️ Painting buttons at positions:', {
1045 post: postBtn.btn.box,
1046 mp4: mp4Btn?.btn.box,
1047 gif: gifBtn?.btn.box,
1048 zip: zipBtn?.btn.box
1049 });
1050
1051 postBtn.paint(api);
1052 mp4Btn?.paint(api);
1053 gifBtn?.paint(api);
1054 zipBtn?.paint(api);
1055
1056 return true; // WITH buttons
1057 }
1058
1059 return true; // WITHOUT buttons
1060}
1061```
1062
1063This will reveal:
10641. Whether buttons are ever created
10652. When screen dimensions change
10663. What rec.presenting and rec.recorded values are
10674. Whether reposition() actually changes box coordinates
1068
1069#### Initial Problem
1070After window resize during tape playback, the canvas buffer would **freeze** - paint() continued running in the worker, but no new frames were displayed to the user. Buttons wouldn't reposition because the entire canvas was frozen.
1071
1072#### Root Cause Discovery
1073The worker's `screen.width/height` was only being updated from `content.width/height`, which came from the PREVIOUS frame. This created a one-frame lag that became permanent during tape playback when `wipe()` used the old screen dimensions to create transparent buffers.
1074
1075**The freeze sequence:**
10761. Window resizes → canvas resizes in bios
10772. Worker still has old screen dimensions
10783. `wipe()` creates buffer with old dimensions
10794. Worker paints and sends buffer to bios
10805. Bios receives buffer with wrong dimensions → dimension mismatch
10816. Bios blocks rendering to prevent stretched/distorted canvas
10827. Worker never receives updated dimensions → permanent freeze
1083
1084#### Solution Implemented
1085
1086**Three key changes to the rendering pipeline:**
1087
10881. **Immediate dimension update in worker** (`disk.mjs` lines 8235-8260):
1089 ```javascript
1090 if (msg.type === "reframed") {
1091 const oldWidth = screen.width;
1092 const oldHeight = screen.height;
1093
1094 screen.width = content.width;
1095 screen.height = content.height;
1096 screen.pixels = new Uint8ClampedArray(content.width * content.height * 4);
1097
1098 console.log(`📐 WORKER: Updated screen dimensions from ${oldWidth} x ${oldHeight} to ${screen.width} x ${screen.height}`);
1099 }
1100 ```
1101 Worker now receives and applies new dimensions immediately via "reframed" message.
1102
11032. **Use current dimensions for paint API** (`disk.mjs` lines 9001-9007):
1104 ```javascript
1105 $api.screen = {
1106 width: screen.width, // Changed from content.width
1107 height: screen.height, // Changed from content.height
1108 // ...
1109 };
1110 ```
1111 The screen object passed to piece's `paint()` now uses worker's current dimensions, not lagged content dimensions.
1112
11133. **Message reordering** (`bios.mjs` lines 1053-1062):
1114 ```javascript
1115 send({
1116 type: "reframed",
1117 content: {
1118 width: screen.width,
1119 height: screen.height
1120 }
1121 });
1122 send({ type: "needs-paint" }); // Sent AFTER reframed
1123 ```
1124 Ensure worker updates dimensions before starting paint.
1125
11264. **Dimension mismatch handler for tape playback** (`bios.mjs` lines 12502-12524):
1127 ```javascript
1128 if (underlayFrame) {
1129 console.log("⏭️ VIDEO: Dimension mismatch - waiting for worker to catch up");
1130 skipImmediateOverlays = true;
1131 setTimeout(() => send({ type: "needs-paint" }), 0);
1132 return;
1133 }
1134 ```
1135 If dimensions don't match during tape playback, skip the frame but keep requesting paint until dimensions sync. This prevents stretching while maintaining animation.
1136
1137#### Result
1138- ✅ Window resize during tape playback works smoothly
1139- ✅ Worker receives correct dimensions immediately
1140- ✅ Canvas doesn't freeze or stretch
1141- ✅ Animation continues without interruption
1142- ✅ Buttons will now reposition correctly (with proper implementation)
1143
1144#### Logging Added for Debugging
1145
1146**Worker dimension updates:**
1147```
1148📐 WORKER: Updated screen dimensions from 300 x 162 to 300 x 192
1149```
1150
1151**Worker wipe operations:**
1152```
1153🖌️ WIPE: Using screen dimensions 300 x 192
1154```
1155
1156**Bios dimension mismatch handling:**
1157```
1158⏭️ VIDEO: Dimension mismatch - waiting for worker to catch up
1159```
1160
1161**Bios reframe fallback:**
1162```
1163🔄 REFRAME PATH (fallback): Created fresh imageData with dimensions: 300 x 192
1164```
1165
1166#### Test Results
1167Confirmed working - resize during tape playback shows:
11681. Worker updates dimensions: `📐 WORKER: Updated screen dimensions from 300 x 192 to 300 x 192`
11692. Worker uses new dimensions: `🖌️ WIPE: Using screen dimensions 300 x 192`
11703. Bios creates fresh imageData: `🔄 REFRAME PATH (fallback): Created fresh imageData`
11714. Animation continues smoothly without freezing or stretching
1172
1173✅ **Canvas freeze is FIXED**
1174❌ **Button repositioning is STILL BROKEN**
1175
1176### Remaining Task: Button Repositioning Implementation
1177
1178**Status: 🔴 CRITICAL - STILL BROKEN**
1179
1180The underlying freeze issue is now FIXED - the canvas correctly updates during resize and animation continues. However, **the export buttons (POST/MP4/GIF/ZIP) still do not move to their new corner positions after window resize.**
1181
1182**What's working:**
1183- ✅ Canvas resizes correctly
1184- ✅ Animation continues without freezing
1185- ✅ Worker receives updated dimensions
1186- ✅ Screen buffer updates properly
1187
1188**What's NOT working:**
1189- ❌ Buttons remain at their original pixel positions
1190- ❌ Buttons do not move to maintain corner alignment
1191- ❌ Button repositioning logic is ineffective
1192
1193**Next Steps:**
1194The button repositioning itself needs proper implementation using one of the proven patterns from other pieces (Solution 1: direct box mutation, or Solution 3: recreate buttons). Now that the canvas correctly updates during resize, buttons should be able to reposition using standard patterns.
1195
1196---
1197
1198## Questions for Further Investigation
1199
12001. Why was `reposition()` method chosen when no other piece uses it?
12012. Does `TextButton` handle corner positioning differently than `Button`?
12023. Is there documentation about proper button repositioning in AC framework?
12034. Should the transparent wipe be changed to opaque after reframe to clear old graphics?
12045. Does the DOM video overlay affect canvas button rendering after reframe?