this repo has no description
Add CQT implementation plan to CLAUDE.md
- Design for 10 octaves (20 Hz - 20 kHz) with 12 bins per octave
- Use 4096-point FFT for better sub-bass resolution
- Plan for ESP32-based implementation with sparse kernels
- Include rapid 3-phase implementation plan for early testing
- Add Lua API design: cqt(), cqts(), cqto(), cqtos()
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
+285
+285
CLAUDE.md
+285
CLAUDE.md
···
1
+
# CLAUDE.md
2
+
3
+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
+
5
+
## Project Overview
6
+
7
+
TIC-80 is a free and open source fantasy computer for making, playing and sharing tiny games. It provides a complete development environment with built-in editors for code, sprites, maps, sound effects, and music.
8
+
9
+
## Build Commands
10
+
11
+
### macOS
12
+
```bash
13
+
# Recommended: Use the debug script
14
+
./debug-macos.sh # Standard build
15
+
./debug-macos.sh -f # Fresh build (clean first)
16
+
./debug-macos.sh -p # Build PRO version
17
+
./debug-macos.sh -d # Debug build
18
+
./debug-macos.sh -a # Build with Address Sanitizer
19
+
./debug-macos.sh -s # Static build
20
+
./debug-macos.sh -w # Build with warnings
21
+
22
+
# Manual build
23
+
cd build
24
+
cmake -DBUILD_WITH_ALL=On ..
25
+
make -j4
26
+
```
27
+
28
+
### Key CMake Options
29
+
- `BUILD_WITH_ALL`: Build with all language support (recommended)
30
+
- `BUILD_PRO`: Build the PRO version with additional features
31
+
- `BUILD_SDLGPU`: Enable SDL GPU support
32
+
- `BUILD_STATIC`: Build static runtime
33
+
- `BUILD_TOOLS`: Build additional tools (bin2txt, prj2cart)
34
+
35
+
## Architecture
36
+
37
+
### Core Components
38
+
39
+
1. **src/api/**: Language bindings for Lua, JavaScript, Python, Ruby, Janet, Wren, Fennel, Moonscript, Squirrel
40
+
- Each language has its own subdirectory with binding implementations
41
+
- All languages share the same core API functions
42
+
43
+
2. **src/core/**: Fantasy computer core
44
+
- `tic80.c`: Main TIC-80 runtime
45
+
- `draw.c`: Rendering functions
46
+
- `sound.c`: Audio synthesis
47
+
- `io.c`: Input/output handling
48
+
49
+
3. **src/studio/**: Development environment
50
+
- `studio.c`: Main studio orchestration
51
+
- `editors/`: Individual editors (code.c, sprite.c, map.c, music.c, sfx.c)
52
+
- `screens/`: UI screens (console.c, menu.c, surf.c, config.c)
53
+
54
+
4. **src/system/**: Platform-specific implementations
55
+
- `sdl/`: Primary desktop implementation
56
+
- `libretro/`: RetroArch core
57
+
- `n3ds/`: Nintendo 3DS port
58
+
- `baremetalpi/`: Raspberry Pi bare metal
59
+
60
+
### Key Concepts
61
+
62
+
- **Cartridge System**: Games are packaged as single .tic files containing code, graphics, maps, sounds, and music
63
+
- **Technical Constraints**: 240x136 display, 16 colors from 24-bit palette, 256 8x8 sprites, 240x136 map, 4 sound channels
64
+
- **Memory Map**: Fixed memory layout at 0x00000-0x17FFF including RAM, VRAM, and persistent storage
65
+
- **API Functions**: Core set of ~80 functions available across all languages (cls, spr, map, pix, clip, print, etc.)
66
+
67
+
### Development Workflow
68
+
69
+
1. **Adding Features**: Most features involve changes to both core (src/core/) and studio (src/studio/)
70
+
2. **Language Support**: New language bindings go in src/api/ and require CMake integration
71
+
3. **Platform Ports**: New platforms implement src/system/ interface
72
+
4. **Editor Improvements**: Each editor in src/studio/editors/ is relatively self-contained
73
+
74
+
### Running Tests
75
+
76
+
TIC-80 uses demo cartridges as integration tests:
77
+
```bash
78
+
# Build and run TIC-80
79
+
./build/bin/tic80
80
+
81
+
# In the TIC-80 console, load and run demos:
82
+
# load demos/tetris.lua
83
+
# run
84
+
```
85
+
86
+
Performance benchmarks are in demos/bunny/ for comparing language implementations.
87
+
88
+
## FFT Implementation
89
+
90
+
TIC-80 includes FFT support for audio visualization and livecoding:
91
+
92
+
### FFT Architecture
93
+
- **src/ext/fft.c**: Audio capture and FFT processing using KISS FFT
94
+
- **src/fftdata.h/c**: FFT data storage (1024 bins, smoothing, normalization)
95
+
- **Lua API**: `fft(freq)` and `ffts(freq)` functions for raw/smoothed data
96
+
97
+
### Key Details
98
+
- Uses miniaudio for audio capture (mic or loopback on Windows)
99
+
- 44100 Hz sample rate, 2048 sample buffer
100
+
- 1024 frequency bins (0-22050 Hz, ~21.5 Hz per bin)
101
+
- Auto-gain control with peak detection
102
+
- Smoothing factor: 0.6 for visual stability
103
+
- Processing in `tic_core_tick` before script execution
104
+
105
+
### Audio Flow
106
+
1. Audio capture via miniaudio callbacks
107
+
2. Stereo to mono conversion
108
+
3. KISS FFT real-to-complex transform
109
+
4. Magnitude calculation and normalization
110
+
5. Smoothing and peak tracking
111
+
6. Data available to scripts via API
112
+
113
+
## CQT Implementation Plan
114
+
115
+
### Overview
116
+
Add Constant-Q Transform (CQT) support alongside existing FFT for better musical frequency analysis with logarithmic frequency spacing and constant Q factor across octaves.
117
+
118
+
### Design Specifications
119
+
120
+
#### Parameters
121
+
- **Octaves**: 10 (20 Hz - 20 kHz full audio spectrum)
122
+
- **Bins per octave**: 12 (chromatic scale)
123
+
- **Total bins**: 120
124
+
- **Q factor**: ~17 (typical for 12 bins/octave)
125
+
- **Minimum frequency**: 20 Hz (below piano's lowest A)
126
+
- **Maximum frequency**: 20480 Hz (nearest note to 20 kHz)
127
+
128
+
#### API Design
129
+
```lua
130
+
-- Direct bin access (0-119)
131
+
value = cqt(bin) -- Get raw CQT magnitude for bin
132
+
value = cqts(bin) -- Get smoothed CQT magnitude for bin
133
+
134
+
-- Musical note access (octave 0-9, note 0-11)
135
+
value = cqto(octave, note) -- Get raw CQT for specific note
136
+
value = cqtos(octave, note) -- Get smoothed CQT for specific note
137
+
138
+
-- Note mapping: C=0, C#=1, D=2, D#=3, E=4, F=5, F#=6, G=7, G#=8, A=9, A#=10, B=11
139
+
```
140
+
141
+
### Implementation Architecture
142
+
143
+
#### File Structure
144
+
```
145
+
src/ext/
146
+
├── fft.c (existing FFT implementation)
147
+
├── cqt.c (new CQT implementation)
148
+
└── cqt_kernel.c (CQT kernel generation from ESP32 reference)
149
+
150
+
src/
151
+
├── fftdata.h/c (existing FFT data storage)
152
+
├── cqtdata.h/c (new CQT data storage)
153
+
154
+
src/api/
155
+
└── luaapi.c (add cqt(), cqts(), cqto(), cqtos() functions)
156
+
```
157
+
158
+
#### Data Structures (cqtdata.h)
159
+
```c
160
+
#define CQT_BINS 120
161
+
#define CQT_OCTAVES 10
162
+
#define CQT_BINS_PER_OCTAVE 12
163
+
164
+
typedef struct {
165
+
float cqtData[CQT_BINS]; // Raw CQT magnitudes
166
+
float cqtSmoothingData[CQT_BINS]; // Smoothed CQT data
167
+
float cqtNormalizedData[CQT_BINS]; // Normalized (0-1) data
168
+
bool cqtEnabled; // Enable flag (tied to fftEnabled initially)
169
+
170
+
// Sparse kernel storage from ESP32 approach
171
+
float* kernelReal[CQT_BINS]; // Real parts of kernels
172
+
float* kernelImag[CQT_BINS]; // Imaginary parts
173
+
int* kernelIndices[CQT_BINS]; // Non-zero indices
174
+
int kernelLengths[CQT_BINS]; // Number of non-zero elements
175
+
} CqtData;
176
+
```
177
+
178
+
### Updated Design (Electronic Music Focus)
179
+
180
+
#### FFT Size Change
181
+
- **Use 4096-point FFT** instead of 2048
182
+
- Bin spacing: 44100 / 4096 = 10.76 Hz (better for sub-bass)
183
+
- Critical for electronic music's 20-60 Hz range
184
+
- Extra ~0.5ms overhead acceptable
185
+
186
+
### Processing Pipeline
187
+
188
+
1. **Initialization** (startup):
189
+
- Pre-compute CQT kernels using Hamming window
190
+
- Apply sparsity threshold (0.01) to reduce memory
191
+
- Store only non-zero kernel values
192
+
193
+
2. **Runtime Processing** (per frame):
194
+
- Share audio buffer from existing capture
195
+
- Perform separate 4096-point FFT for CQT
196
+
- Apply sparse kernels to FFT output
197
+
- Calculate magnitudes and normalize
198
+
- Apply smoothing (factor 0.7 for stability)
199
+
200
+
3. **Memory Optimization**:
201
+
- Sparse kernels: ~30-40% of full matrix size
202
+
- Estimated memory: ~200KB for kernels + 2KB for data
203
+
- Acceptable for fantasy computer constraints
204
+
205
+
### Performance Considerations
206
+
207
+
#### Computational Cost
208
+
- One 4096-point FFT: ~1ms on modern CPU
209
+
- Sparse kernel application: ~0.5-1ms
210
+
- Total overhead: 1.5-2ms per frame (acceptable)
211
+
212
+
#### Optimization Strategies
213
+
1. Use SIMD where available (via compiler)
214
+
2. Process every other frame if needed
215
+
3. Cache kernel computations at startup
216
+
4. Use integer math for index calculations
217
+
218
+
### Integration Points
219
+
220
+
1. **Core Integration** (tic80.c):
221
+
- Process CQT in `tic_core_tick` after FFT
222
+
- Share `fftEnabled` flag initially
223
+
- Add separate `cqtEnabled` flag later
224
+
225
+
2. **Build System** (CMakeLists.txt):
226
+
- Add cqt.c and cqt_kernel.c to source list
227
+
- No new dependencies (uses existing KISS FFT)
228
+
229
+
3. **Testing Strategy**:
230
+
- Create demo showing CQT vs FFT comparison
231
+
- Verify musical note accuracy with test tones
232
+
- Benchmark on various platforms
233
+
234
+
### Rapid Implementation Plan (Test Early)
235
+
236
+
#### Phase 1: Minimal CQT (Test in 1-2 hours)
237
+
1. **Create cqtdata.h/c** with basic data arrays
238
+
2. **Copy/adapt ESP32 kernel generation** to cqt_kernel.c
239
+
3. **Create minimal cqt.c** that:
240
+
- Generates kernels on init
241
+
- Reuses existing audio buffer
242
+
- Performs 4096-point FFT
243
+
- Applies kernels to get CQT
244
+
4. **Add single test function** in luaapi.c: `cqt(bin)`
245
+
5. **Hook into tic_core_tick** after FFT processing
246
+
6. **Test with simple Lua visualization**
247
+
248
+
#### Phase 2: Full API (Test in 3-4 hours)
249
+
1. Add smoothing to CQT processing
250
+
2. Implement remaining API functions: `cqts()`, `cqto()`, `cqtos()`
251
+
3. Add normalization and auto-gain
252
+
4. Create demo comparing FFT vs CQT
253
+
254
+
#### Phase 3: Optimization (If needed)
255
+
1. Profile and identify bottlenecks
256
+
2. Implement sparse kernel optimizations
257
+
3. Consider frame-skipping for lower-end hardware
258
+
259
+
### Test Script Example
260
+
```lua
261
+
-- Quick CQT test visualization
262
+
function TIC()
263
+
cls(0)
264
+
-- Draw CQT bins as bars
265
+
for i=0,119 do
266
+
local val = cqt(i) * 100 -- scale up for visibility
267
+
rect(i*2, 136-val, 2, val, 12)
268
+
end
269
+
end
270
+
```
271
+
272
+
### Future Enhancements
273
+
1. Configurable octave range (6-10 octaves)
274
+
2. Variable bins per octave (24 for quarter-tones)
275
+
3. Separate enable flag for CQT
276
+
4. Phase information access
277
+
5. Inverse CQT for resynthesis
278
+
279
+
## Important Notes
280
+
281
+
- The project supports multiple scripting languages, all exposed through the same API
282
+
- Platform-specific code should be isolated in src/system/
283
+
- The studio uses immediate mode GUI principles
284
+
- Cartridge format is documented in wiki and src/studio/project.c
285
+
- PRO version enables additional features like extra memory banks