Monorepo for Aesthetic.Computer
aesthetic.computer
1# Tape Viewer Implementation
2
3## Overview
4Created a new `tape.mjs` piece that loads and plays back tape recordings from their ZIP files, similar to how `painting.mjs` works for paintings.
5
6## Usage
7
8### Loading a Tape
9```
10tape %code
11```
12
13Where `code` is the short tape code (e.g., `abc`, `xyz`, etc.)
14
15Examples:
16- `tape %abc` - Load and view tape with code "abc"
17- `tape %abc:show` - Lightbox mode (no UI)
18- Visit `aesthetic.computer/#%abc` - Direct URL access
19
20### Visual Signifier
21The `%` symbol was chosen as a visual signifier for tapes because:
22- Resembles a film reel / movie symbol
23- Distinct from `#` (used for paintings)
24- Indicates video/motion content
25
26## Features Implemented
27
28### Core Functionality
29- ✅ Load tape metadata from MongoDB via `/api/get-tape?code=XXX`
30- ✅ Download and parse ZIP files from S3 (user or guest buckets)
31- ✅ Extract all frames, timing data, and metadata
32- ✅ Display frames with proper scaling and centering
33- ✅ Audio playback from `soundtrack.wav` (Web Audio API)
34- ✅ Frame-accurate timing using timing.json
35
36### Playback Controls
37- ✅ Play/Pause button
38- ✅ Keyboard controls:
39 - `Space` - Toggle play/pause
40 - `Arrow Left` - Previous frame (when paused)
41 - `Arrow Right` - Next frame (when paused)
42 - `Home` - Jump to first frame
43 - `End` - Jump to last frame
44- ✅ Frame counter display
45- ✅ Time display (current/total duration)
46- ✅ Playback indicator (▶/⏸)
47
48### UI Features
49- ✅ Download button - downloads original ZIP file
50- ✅ Loading state with animated ellipsis
51- ✅ Error handling with user-friendly messages
52- ✅ Show mode (`:show` suffix) for lightbox/embed view
53- ✅ Responsive frame scaling to fit screen
54
55### Audio Integration
56- ✅ Automatic detection of `soundtrack.wav` in ZIP
57- ✅ Web Audio API integration
58- ✅ Frame-synced audio playback
59- ✅ Audio offset calculation for mid-tape playback
60- ✅ Audio stops on pause/end
61
62## Technical Details
63
64### File Structure
65The tape viewer expects ZIPs with this structure:
66```
67tape-code.zip
68├── frame-00000.png
69├── frame-00001.png
70├── ...
71├── timing.json (frame durations)
72├── metadata.json (piece name, duration, etc.)
73└── soundtrack.wav (optional audio)
74```
75
76### API Endpoints Used
77- `GET /api/get-tape?code=XXX` - Fetch tape metadata from MongoDB
78 - Returns: `{ slug, code, when, bucket, user, mp4Status, nuked }`
79
80### ZIP Loading
81- Uses JSZip library (loaded dynamically from CDN)
82- Downloads from appropriate S3 bucket (user vs guest)
83- Extracts frames as ImageBitmaps for efficient rendering
84- Parses JSON files for timing and metadata
85
86### Timing System
87- Uses `timing.json` for frame-accurate playback
88- Each entry: `{ frame: number, duration: number (ms) }`
89- Advances frames based on actual duration
90- Fallback to 30fps if timing data missing
91
92### Audio Synchronization
93- Decodes WAV file using Web Audio API
94- Calculates playback offset based on current frame
95- Syncs audio start time with frame progression
96- Handles audio ending gracefully
97
98## Testing
99
100To test the tape viewer:
101
1021. **Record a tape** (if you haven't already):
103 ```
104 tape 3 notepat
105 ```
106 This will generate a tape code (e.g., `#abc`)
107
1082. **View the tape**:
109 ```
110 tape %abc
111 ```
112
1133. **Try different modes**:
114 ```
115 tape %abc:show (lightbox mode)
116 ```
117
1184. **Direct URL**:
119 Navigate to `https://aesthetic.computer/#%abc`
120
121## Code Location
122- **Piece file**: `/workspaces/aesthetic-computer/system/public/aesthetic.computer/disks/tape.mjs`
123- **Backend API**: `/workspaces/aesthetic-computer/system/netlify/functions/get-tape.mjs` (already exists)
124
125## Comparison to painting.mjs
126
127### Similarities
128- Short code system (`#code` for paintings, `%code` for tapes)
129- MongoDB lookup via API
130- S3 file download
131- Loading states and error handling
132- Show mode support
133- Download button
134
135### Differences
136- **File type**: ZIP (tapes) vs PNG (paintings)
137- **Playback**: Animated sequence vs static image
138- **Audio**: Includes soundtrack vs silent
139- **Controls**: Play/pause/scrub vs static view
140- **Timing**: Frame-based animation vs instant load
141
142## Future Enhancements
143
144### Planned Features
145- [ ] Scrubbing timeline/progress bar
146- [ ] Speed controls (0.5x, 1x, 2x)
147- [ ] Loop toggle
148- [ ] Frame-by-frame download (export individual frames)
149- [ ] Share button (copy link)
150- [ ] Fullscreen mode
151- [ ] Volume control
152- [ ] Keyboard shortcuts help overlay
153
154### Integration Ideas
155- [ ] Gallery view of user's tapes
156- [ ] Related tapes suggestions
157- [ ] Tape annotations/comments
158- [ ] MP4 fallback (when conversion service ready)
159- [ ] Social sharing previews
160
161## Notes
162
163### Browser Compatibility
164- Requires browsers with:
165 - `createImageBitmap` API (modern browsers)
166 - Web Audio API (for audio playback)
167 - Fetch API (for downloading)
168
169### Performance
170- All frames loaded into memory for smooth playback
171- May use significant RAM for long tapes (100+ frames)
172- Consider lazy loading for very long recordings
173
174### Audio Autoplay
175- Web Audio context may be suspended by browser autoplay policies
176- First interaction (play button click) resumes audio context
177- Audio starts synced with current frame position
178
179## Integration with Existing System
180
181The tape viewer integrates seamlessly with the existing ecosystem:
182
1831. **Recording**: `tape X piece` creates tape → generates short code
1842. **Upload**: ZIP uploaded to S3, record created in MongoDB
1853. **Viewing**: `tape %code` loads and plays back recording
1864. **Sharing**: Short codes make sharing easy (`%abc` vs full S3 URL)
187
188This completes the tape workflow from creation to playback! 🎬📼