···2233- **Owner:** AI Coding Agent (Codex CLI)
44- **Scope:** Help drive the Rust rewrite forward with tests-first changes, targeted features, and tight parity with TIC-80 behavior.
55-- **Last Updated:** 2025-08-26
55+- **Last Updated:** 2025-08-27
66+77+**Documentation Discipline — Agent Reminder (PROMINENT)**
88+- ALWAYS document code changes immediately after writing code:
99+ - Update this file’s Current Status and Worklog (with date + concise bullets).
1010+ - Update relevant docs (specs/architecture/testing) and cross-link from here.
1111+ - Update `docs/specs/implementation_status.md` to reflect new capabilities.
1212+ - If the plan changed, update the plan doc(s) and link them (Roadmap/Spec).
1313+- ALWAYS document plans as first-class docs:
1414+ - Add a plan/Implementation TODOs section under the relevant spec (e.g., audio FFT/VQT) or create a new spec.
1515+ - Reference new/updated plans from AGENTS.md and the docs index.
1616+- Keep hygiene visible: mention clippy/test status with each change.
617718**Context**
819- **Rewrite code location:** All Rust rewrite code lives under `tic80_rust/` (crate root). Tests live in `tic80_rust/tests/`. The windowed demo binary is `tic80_rust/src/main.rs`.
···3849- `print` implemented with default font (variable/fixed width, scale, newline advance) and returns width.
3950- Memory ops (`peek/poke` 1/2/4/8‑bit, `memcpy`, `memset`) implemented; VRAM updates reflect on screen.
4051- CLI loads bundled default cart or a provided `.lua` path.
5252+- Audio capture foundation: `cpal` input stream with mono downmix into a lock‑free ring buffer (8192 samples); CLI flags `--list-audio`, `--audio-device`, `--audio-vu`, `--audio-disable`; simple VU feedback prints peak dBFS once per second.
41534254**Near-Term Backlog**
5555+- FFT implementation (cpal + realfft) per `docs/specs/audio_fft_vqt.md` (see Implementation TODOs section); add headless tests and Lua `fft/ffts/fftr/fftrs`.
4356- Print edge cases: tests for scale>1 baseline/advance and multi‑line width parity.
4457- Small font: decide semantics and implement `smallfont=true` in `print` with tests.
4558- Lua error paths: add type/arity mismatch tests for core APIs (`pix/line/rect/print`).
···5164- Banks/persistence: `vbank`, `sync`, `pmem`; palette map/border color.
5265- Input/system: `btn/btnp`, `key/keyp`, `mouse`, `time`, `tstamp`, `trace`, `exit`, `reset` (fixed‑step repeat timing).
5366- Audio: `sfx`, `music` synth/mixer; capture ring for analysis.
5454-- Analysis: `fft/ffts/fftr/fftrs`, `vqt` variants; conformance carts + numeric tolerances.
6767+- Analysis (VQT): implement kernels + unwhitened/whitened paths per `docs/specs/audio_fft_vqt.md` (see Implementation TODOs) and expose `vqt*`/`vqt*w` APIs; conformance carts + numeric tolerances.
5568- Platform: WASM build path; window scaling and UX polish.
56695770**Open Questions**
···7891 - Renamed API parity to `docs/specs/lua_api_parity.md` and added specs stubs.
7992 - Added docs index at `docs/README.md`, architecture/testing pages, and ADRs.
8093 - Added detailed testing docs (`docs/testing/strategy.md`) and a test catalog (`docs/testing/test_catalog.md`).
9494+ - 2025-08-27:
9595+ - Implemented audio capture with `cpal`: device selection/listing, mono downmix, ring buffer.
9696+ - Added CLI: `--list-audio`, `--audio-device`, `--audio-vu`, `--audio-disable`.
9797+ - Integrated a 1s VU peak readout for manual verification.
9898+ - Kept clippy/tests green; documented the FFT/VQT plan and linked TODOs.
819982100**Docs Index**
83101- Start here: `docs/README.md`
+1-1
docs/README.md
···1010- `docs/specs/memory_map.md`: Canonical pointer to the root `MEMORY_MAP.md` and usage notes.
1111- `docs/specs/lua_api_parity.md`: API parity checklist for Lua (name, signature, side effects).
1212- `docs/specs/graphics.md`: Framebuffer, palette mapping, text/print semantics (stub to be expanded).
1313-- `docs/specs/audio_fft_vqt.md`: FFT/VQT behavior and parameters (points to `CLAUDE.md`).
1313+- `docs/specs/audio_fft_vqt.md`: FFT/VQT behavior and Rust implementation plan (cpal + realfft), with links to `CLAUDE.md`.
1414- `docs/specs/implementation_status.md`: What’s implemented vs pending, with notes on behavior.
15151616## Architecture
+3
docs/architecture/runtime.md
···1212- CPU-owned framebuffer as palette indices; `blit_to_rgba` maps to RGBA for window texture.
1313- Integer scaling to maintain crisp pixels.
14141515+Audio Capture
1616+- `cpal` input stream runs in a platform callback and writes mono f32 samples into a lock‑free ring buffer sized to 8192.
1717+- The analysis path (FFT/VQT) runs on the tick thread, reading from the ring’s tail to ensure determinism and keep heavy work off the audio thread.
+1-1
docs/roadmap/overview.md
···5252- Rendering backend: Start with software rasterizer in `tic-gfx` (deterministic, headless testable). Add a thin SDL2 or winit+pixels presentation layer later. Optional `wgpu` in a later phase.
5353- Audio IO: Use `cpal` for capture/output. Keep synth/tick deterministic in `tic-audio`. For capture, implement a lock-free ring buffer shared with `tic-fx`.
5454- Compression: Use `flate2`/`miniz_oxide` for zlib compatible packing/unpacking of carts.
5555-- FFT: Use `rustfft` for 2k (FFT) and 8k (VQT path) with exact binning and smoothing behavior; implement variable-Q kernel generation per CLAUDE.md.
5555+- FFT/VQT: Use `realfft` (R2C) for 2k/8k transforms with exact binning and smoothing behavior; implement variable‑Q kernel generation per CLAUDE.md. See `docs/specs/audio_fft_vqt.md` for the Rust plan.
5656- Testing: Frame-hash snapshots for VRAM, audio block-level comparisons, API-level golden tests. Conformance carts from `demos/`.
57575858## Phased Roadmap
+137-2
docs/specs/audio_fft_vqt.md
···11# Audio Analysis (FFT/VQT)
2233-Detailed behavior, parameters, and equations live in `CLAUDE.md` at the repo root.
33+This page captures the runtime behavior and the implementation plan for FFT/VQT in the Rust rewrite. Canonical derivations and design notes still live in `CLAUDE.md` at the repo root; this page focuses on API semantics, platform choices, and test strategy for parity with the C implementation.
44+55+## Goals
66+- API Parity: Implement `fft/ffts/fftr/fftrs`, `vqt/vqts/vqtr/vqtrs`, and `vqtw/vqtsw/vqtrw/vqtrsw` with identical signatures and observable behavior.
77+- Determinism: Stable outputs given fixed inputs (headless tests feed synthetic signals).
88+- Performance: Desktop targets (Windows/macOS/Linux) at real‑time rates (2k FFT ~21 fps; 8k VQT ~5.4 fps).
99+1010+## Technology Choices (Rust)
1111+- Audio I/O: `cpal` for capture (pure Rust), f32 stereo → mono downmix in callback.
1212+- FFT: `realfft` (R2C) for N=2048 and N=8192 transforms; built on `rustfft`.
1313+- Buffering: `ringbuf` (lock‑free SPSC) between audio callback and analysis thread.
1414+- Windows: `window-functions` or `apodize` for Hamming/Gaussian in VQT kernel generation.
1515+1616+## Sample Rate and Buffers
1717+- Sample Rate: 44100 Hz throughout (resample to 44.1k if device provides a different rate; future work).
1818+- Shared Audio Buffer: `AUDIO_BUFFER_SIZE = max(2*FFT_SIZE, VQT_FFT_SIZE) = 8192` mono f32 samples (aligned with C).
1919+- Capture Format: accept i16/u16/f32; convert to f32, average L/R to mono; no allocations in the callback.
2020+2121+## FFT (2k) — Behavior and Plan
2222+- Window: last 2048 mono samples (from the shared buffer).
2323+- Transform: R2C length 2048; magnitude per bin: `2.0 * hypot(re, im)`.
2424+- Bins: expose 0..1023 (drop Nyquist) to match C.
2525+- Buffers (mirroring C):
2626+ - Raw: `fftRawData[1024]`, `fftRawSmoothingData[1024]` (IIR smoothing factor 0.6).
2727+ - Display: `fftData[1024]`, `fftSmoothingData[1024]`, `fftNormalizedData[1024]`.
2828+ - Peak tracking: `fPeakMinValue=0.01`, `fPeakSmoothing=0.995`, `fPeakSmoothValue`, `fAmplification=1/peak`.
2929+- Update (per tick):
3030+ 1) Read last 2048 samples; zero‑pad until warm.
3131+ 2) R2C → magnitudes; update raw and smoothed raw.
3232+ 3) Update peak smoothing, recompute amplification, write normalized and smoothed normalized.
3333+- APIs:
3434+ - `fft(start,end=-1)`: normalized value at `start` or inclusive sum over `[start..end]` with C’s clamping rules.
3535+ - `ffts(start,end)`: smoothed normalized.
3636+ - `fftr/fftrs`: raw and raw‑smoothed (no normalization).
3737+3838+## VQT (8k) — Behavior and Plan
3939+- Window: last 8192 mono samples.
4040+- Kernels (once):
4141+ - Centers: 120 bins starting at 19.445 Hz (D#0/Eb0), semitone spacing `2^(1/12)`.
4242+ - Variable Q: replicate C’s schedule; window length `Q * fs / f`, clamped to 8192; minimum length guard.
4343+ - Windowing: Hamming or Gaussian (match current default); normalize by window length.
4444+ - Modulation: complex exponential across full FFT buffer, centered at N/2 (parity with C notes and indices).
4545+ - R2C kernel FFT; sparsify to `(indices[], real[], imag[])` above adaptive magnitude thresholds.
4646+- Runtime VQT (per tick):
4747+ 1) R2C of input frame (8192).
4848+ 2) For each kernel: sparse complex dot over half‑spectrum (0..4096), magnitude then `* 2.0`.
4949+ 3) Unwhitened path:
5050+ - Smooth `vqtSmoothingData` with `VQT_SMOOTHING_FACTOR=0.3`.
5151+ - Peak smoothing to `vqtPeakSmoothValue` (0.99 mix as in C pattern), normalize to `vqtNormalizedData` in [0,1].
5252+ 4) Whitened path (separate arrays):
5353+ - Whitening enabled by default: log(m+eps) → moving average over width=21 → subtract → exp → mix by `alpha=0.95`.
5454+ - Smooth to `vqtWhiteSmoothingData`; separate peak tracker `vqtWhitePeakSmoothValue`; normalize to `vqtWhiteNormalizedData`.
5555+- APIs (two parallel sets):
5656+ - Unwhitened: `vqt` (normalized), `vqts` (smoothed+normalized), `vqtr` (raw), `vqtrs` (raw smoothed).
5757+ - Whitened: `vqtw`, `vqtsw`, `vqtrw`, `vqtrsw` with identical semantics on whitened buffers.
5858+5959+## Parity Rules
6060+- Bins and scaling match C exactly: drop Nyquist for 2k FFT; use `2.0` magnitude factor.
6161+- Peak tracking and smoothing factors: FFT raw/display (0.6), peak (0.995); VQT smoothing (0.3), whitening width=21, alpha=0.95, eps=1e‑6.
6262+- Inclusive sums and clamping follow C’s logic for out‑of‑bounds and reversed ranges.
6363+- Whitening is strictly a separate path with its own buffers and peak tracker.
6464+6565+## Testing Strategy
6666+- Headless tests feed synthetic tones/noise into the analysis (bypass `cpal`).
6767+- FFT:
6868+ - Single‑tone peak lands at expected bin; normalized steady state ≈ 1.0.
6969+ - Raw vs smoothed differ per IIR; inclusive sum matches expected bin sums.
7070+- VQT:
7171+ - Kernel spot checks: indices non‑empty; ranges align with center frequencies.
7272+ - Single‑tone hits expected semitone bin; whitened vs unwhitened differ predictably on broadband inputs.
7373+ - Normalization clamps to [0,1]; whitened/unwhitened use independent peak trackers.
7474+7575+## Integration Plan (Milestones)
7676+1) FFT foundation: ring buffer, 2k R2C planner, raw/normalized/smoothed buffers, Lua `fft/ffts/fftr/fftrs`.
7777+2) VQT kernels: generation + storage; 8k R2C planner; unwhitened path with `vqt/vqts/vqtr/vqtrs`.
7878+3) Whitening path: `vqtw/vqtsw/vqtrw/vqtrsw` with separate smoothing/peaks; feature flag to toggle.
7979+4) Cross‑platform validation: confirm capture paths (WASAPI/CoreAudio/ALSA/Pulse/JACK) and resampling if needed.
8080+8181+## References
8282+- Canonical details: `CLAUDE.md`.
8383+- C sources for parity: `src/ext/fft.c`, `src/fftdata.h/.c`, `src/ext/vqt.c`, `src/vqtdata.h/.c`, `src/ext/vqt_kernel.c`.
8484+8585+8686+## Implementation TODOs
8787+8888+Status
8989+- Phase 1 implemented: `cpal` capture + mono downmix + ring buffer + CLI flags + VU feedback.
9090+- Next: Phase 2 (FFT 2k, realfft R2C, Lua APIs, tests).
9191+9292+Phase 1: Audio (cpal)
9393+- Device listing: Add `--list-audio` to print capture devices and default.
9494+- Device selection: Add `--audio-device "<name-substr>"` to pick device by substring; default to system default input.
9595+- Sample rate: Request 44100 Hz; if device differs, accept nearest and record actual rate; add `--audio-rate 44100` override (future resampling).
9696+- Channel format: Accept f32/i16/u16; convert to f32; downmix stereo to mono by average.
9797+- Buffering: Add SPSC ring buffer (capacity 8192 f32 mono samples). No allocations in callback.
9898+- Threading: Start `cpal` input stream on app init; push samples into ring buffer; graceful start/stop.
9999+- UX check: `--audio-vu` to print a simple VU meter/peak every second for manual verification.
100100+- Errors: Clear messages on device open failures; fallback to default device if selection fails; allow `--audio-disable`.
101101+102102+Phase 2: FFT (2k)
103103+- Planner: Initialize `realfft` R2C plan for N=2048; allocate scratch/output once.
104104+- State buffers:
105105+ - Raw: `fftRaw[1024]`, `fftRawSm[1024]` (IIR smoothing factor 0.6).
106106+ - Display: `fftData[1024]`, `fftSm[1024]`, `fftNorm[1024]`.
107107+ - Peaks: `fPeakMin=0.01`, `fPeakSmooth=0.995`, `fPeak`, `fAmpl=1/fPeak`.
108108+- Tick update:
109109+ - Read last 2048 mono samples (zero-fill until warm).
110110+ - R2C; magnitudes for bins 0..1023 as `2.0 * hypot(re, im)`.
111111+ - Update raw/smoothed raw; update `fPeak` and `fAmpl`; write normalized and smoothed normalized.
112112+- Lua APIs: `fft/ffts/fftr/fftrs` with C-identical clamping and inclusive-sum semantics.
113113+- Tests (headless): tone peak at expected bin; normalized steady-state ≈ 1; smoothed below raw; range sums; OOB clamping.
114114+- Telemetry: Optional `--debug-fft` dump.
411555-This page will summarize API-facing behavior (bins, windows, smoothing, normalization, whitening) and cross-link into the canonical derivations in `CLAUDE.md`.
116116+Phase 3: VQT (8k)
117117+- Planner: Initialize `realfft` R2C plan for N=8192; allocate scratch/output once.
118118+- Kernel generation (once):
119119+ - Centers: 120 semitone bins from 19.445 Hz (`2^(1/12)` spacing).
120120+ - Q schedule: Mirror C’s rules; window length `Q*fs/f`, clamp to 8192; min length guard.
121121+ - Windowing: Hamming (default) and Gaussian; normalize by window length.
122122+ - Modulation: Complex exponential over full 8192 buffer centered at N/2.
123123+ - R2C of time kernel; sparsify to `(indices, real, imag)` with adaptive thresholds.
124124+- VQT state buffers (unwhitened): `vqt[120]`, `vqtSm[120]` (0.3), `vqtNorm[120]`, `vqtPeak`.
125125+- Tick update (unwhitened): R2C input; sparse complex dot per bin over 0..4096; magnitude `* 2.0`; smooth; peak-smooth; normalize to [0,1].
126126+- Lua APIs: `vqt/vqts/vqtr/vqtrs`.
127127+- Tests: kernel sanity (non-empty indices, expected spans); 440 Hz tone peak; normalization bounds.
6128129129+Phase 4: Whitening (separate path)
130130+- Config: Feature flag `whitening=true` (default); `--no-whitening` toggle.
131131+- Whitening buffers: `vqtW[120]`, `vqtWSm[120]`, `vqtWNorm[120]`, `vqtWPeak`.
132132+- Whitening math: log(m+eps) → moving-average env (width=21) → subtract → exp → mix by alpha=0.95.
133133+- Tick update (whitened): smooth; peak-smooth; normalize to [0,1]; if disabled, mirror unwhitened into whitened buffers.
134134+- Lua APIs: `vqtw/vqtsw/vqtrw/vqtrsw`.
135135+- Tests: broadband input flatter in whitened vs unwhitened; toggle affects only whitened set.
136136+137137+Cross‑cutting
138138+- Determinism: All analysis in tick thread; audio callback only pushes to ring.
139139+- Performance: Reuse FFT plans/buffers; no alloc in hot paths; optional `--debug-fx` timing logs.
140140+- Error handling: Safe fail if no audio device; tests bypass `cpal`.
141141+- Docs: Keep this page as source of truth; update AGENTS.md Near-/Mid‑Term backlog as milestones are delivered.
+1
docs/specs/implementation_status.md
···2626Implemented (Runner/CLI)
2727- `.lua` loader: First CLI arg as a `.lua` path runs external script; fallback to bundled `assets/default.lua`.
2828- Window title: “rustic”.
2929+- Audio capture scaffolding: `cpal` input stream (44.1 kHz if supported), stereo→mono downmix, lock‑free ring buffer (8192 samples); CLI flags to list/select devices and optional VU meter output.
29303031Behavioral Notes
3132- Triangles: top-left inclusion; CCW orientation enforced internally; half-open bounding box prevents shared-edge double draws.