this repo has no description
more claude documentation
+151
-34
+151
-34
CLAUDE.md
+151
-34
CLAUDE.md
···
531
531
## Current CQT Implementation Status (December 2024)
532
532
533
533
### What's Working:
534
-
- **16K FFT implemented** - provides excellent low-frequency resolution
535
-
- **cqt(bin)** function working correctly - returns raw CQT magnitude for bin 0-119
534
+
- **FFT fully restored** - Returns 1024 bins with exact original behavior preserved
535
+
- **CQT with configurable FFT size** - Currently 8K (changeable in cqtdata.h)
536
+
- **Shared audio buffer** - Automatically sized to max(FFT needs, CQT needs)
537
+
- **cqt(bin)** function working - Returns raw CQT magnitude for bin 0-119
536
538
- **Frequency detection accurate** - 440Hz correctly maps to bin 54, etc.
537
-
- **Kernels properly generated** - sparse storage working, good frequency selectivity
538
-
- **Performance excellent** - Runtime on M1 Pro: 0.368ms total (0.255ms FFT + 0.113ms kernels)
539
+
- **Kernels properly generated** - Sparse storage, good frequency selectivity
540
+
- **Smoothing calculated** - But not exposed via API yet
541
+
542
+
### API Status:
543
+
| Function | Status | Description |
544
+
|----------|---------|-------------|
545
+
| `fft(bin)` | ✅ Complete | Raw FFT data (0-1023) |
546
+
| `ffts(bin)` | ✅ Complete | Smoothed FFT data |
547
+
| `cqt(bin)` | ✅ Complete | Raw CQT data (0-119) |
548
+
| `cqts(bin)` | ❌ TODO | Smoothed CQT data |
549
+
| `cqto(octave, note)` | ❌ TODO | Raw CQT by musical note |
550
+
| `cqtos(octave, note)` | ❌ TODO | Smoothed CQT by musical note |
551
+
552
+
### Performance with 8K FFT (Current Default):
553
+
- **Update rate**: ~5.4 fps
554
+
- **M1 Pro**: ~0.2ms total (1.2% of frame budget)
555
+
- **Low frequency quality**: Q≈3.7 at 20Hz (decent for electronic music)
556
+
- **Good balance** for livecoding applications
557
+
558
+
### Resolution Characteristics:
559
+
| FFT Size | 20Hz Quality | 40Hz Quality | Update Rate |
560
+
|----------|--------------|--------------|-------------|
561
+
| 4K | Q≈1.9 (poor) | Q≈3.7 (poor) | 10.8 fps |
562
+
| 8K (current) | Q≈3.7 (decent) | Q≈7.4 (good) | 5.4 fps |
563
+
| 16K | Q≈7.4 (good) | Q≈14.8 (excellent) | 2.7 fps |
564
+
565
+
### Implementation Architecture:
566
+
```
567
+
Audio Capture (44.1kHz)
568
+
↓
569
+
Shared Buffer (8192 samples currently)
570
+
├─→ FFT: Uses first 2048 samples → 1024 bins
571
+
└─→ CQT: Uses first 8192 samples → 120 bins
572
+
```
539
573
540
-
### Runtime Performance Expectations:
541
-
- **M1 Pro MacBook**: 0.368ms total CQT processing (~2.2% of frame budget)
542
-
- **ThinkPad i5-1130G7 (extrapolated)**:
543
-
- Performance mode: ~0.426ms (2.6% of frame budget)
544
-
- Power save mode: ~1.273ms (7.6% of frame budget)
545
-
- Runtime is ~3.8x slower than synthetic benchmarks due to real audio data, cache effects
574
+
### Key Implementation Details:
575
+
1. **Buffer management**:
576
+
- `AUDIO_BUFFER_SIZE` in fft.c automatically adjusts
577
+
- FFT always reads samples 0-2047
578
+
- CQT reads samples 0-(CQT_FFT_SIZE-1)
546
579
547
-
### Critical Implementation Details:
548
-
1. **FFT_SIZE temporarily changed** in fftdata.h from 1024 to 8192 to support CQT
549
-
- This breaks FFT resolution but enables CQT
550
-
- Must be reverted when separate audio buffer is implemented
551
-
552
-
2. **Key fixes that were essential**:
553
-
- Kernel phase calculation must use full FFT position: `2π * (f/fs) * (idx - N/2)`
554
-
- NO scaling factor after kernel FFT - this was critical!
555
-
- Normalization by windowLength before FFT
556
-
557
-
3. **Test scripts created**:
558
-
- `test_cqt_spectrum_v2.lua` - visual spectrum analyzer
559
-
- `test_cqt_a4.lua` - 440Hz tone generator
560
-
- `test_cqt_stable.lua` - controlled testing
580
+
2. **Smoothing implementation**:
581
+
- FFT: 0.6 factor (60% old, 40% new)
582
+
- CQT: 0.3 factor (30% old, 70% new) - calculated but not exposed
583
+
- Both use peak tracking with 0.99 decay
561
584
562
-
### Resolution Characteristics with 16K FFT:
563
-
- **20Hz**: Q≈7.4 (window truncated to 16384 samples, but much better than 6K's Q≈1.86)
564
-
- **30Hz**: Q≈11.2 (good resolution)
565
-
- **45Hz+**: Full Q≈17 (perfect - no truncation)
585
+
3. **Test scripts**:
586
+
- `demo_fft_cqt_hybrid.lua` - Combined FFT/CQT visualization
587
+
- `test_cqt_spectrum_v2.lua` - CQT spectrum analyzer
588
+
- `test_fft_restored.lua` - FFT verification
566
589
567
-
## Phase 3: Next Steps
568
-
1. ~~Implement 16K FFT based on benchmark results~~ (COMPLETE)
569
-
2. Add remaining API functions: `cqts()`, `cqto()`, `cqtos()`
570
-
3. Create separate audio buffer for CQT (restore FFT_SIZE to 1024)
571
-
4. Create FFT vs CQT comparison demo
572
-
5. Test and verify improved frequency resolution at 20Hz, 50Hz, 100Hz
590
+
## Next Steps
591
+
1. ✅ ~~Implement configurable FFT for CQT~~ (COMPLETE - using 8K default)
592
+
2. ✅ ~~Create separate audio buffer for CQT~~ (COMPLETE - shared buffer)
593
+
3. ✅ ~~Restore FFT_SIZE to 1024~~ (COMPLETE)
594
+
4. ❌ Add remaining API functions: `cqts()`, `cqto()`, `cqtos()`
595
+
5. ❌ Add CQT to other language bindings (currently Lua only)
596
+
6. ❌ Create comprehensive FFT vs CQT comparison demo
597
+
7. ❌ Add configuration options for CQT parameters
573
598
574
599
### Test Script Example
575
600
```lua
···
600
625
3. Separate enable flag for CQT
601
626
4. Phase information access
602
627
5. Inverse CQT for resynthesis
628
+
629
+
## FFT vs CQT: Understanding the Tradeoffs
630
+
631
+
### Fundamental Difference
632
+
- **FFT**: Linear frequency spacing (each bin = 21.5 Hz)
633
+
- **CQT**: Logarithmic frequency spacing (constant Q = ~17)
634
+
635
+
### Time-Frequency Resolution Tradeoff (Uncertainty Principle)
636
+
This is fundamental physics - to distinguish frequencies, you must observe for sufficient time:
637
+
- To tell 20 Hz from 21 Hz: Need 1 second of observation
638
+
- To tell 1000 Hz from 1001 Hz: Still need 1 second
639
+
- Shorter window = better time resolution, worse frequency resolution
640
+
- Longer window = better frequency resolution, worse time resolution
641
+
642
+
### FFT Characteristics
643
+
- **Window**: 2048 samples (46ms)
644
+
- **Update rate**: ~21 fps
645
+
- **Frequency resolution**: 21.5 Hz per bin (constant)
646
+
- **Best for**: Beat detection, rhythm visualization, transients
647
+
- **API**: `fft(bin)` raw, `ffts(bin)` smoothed (0.6 factor)
648
+
649
+
### CQT Characteristics
650
+
- **Window**: Variable per frequency (up to 16384 samples)
651
+
- **Update rate**: ~5.4 fps (8K FFT) or ~2.7 fps (16K FFT)
652
+
- **Frequency resolution**: Constant Q≈17 (logarithmic spacing)
653
+
- **Best for**: Note detection, chord analysis, harmonic content
654
+
- **API**: `cqt(bin)` raw, `cqts(bin)` smoothed (0.3 factor) - smoothed not yet implemented
655
+
656
+
### Resolution Comparison
657
+
658
+
| Method | 40 Hz Resolution | 440 Hz Resolution | 4400 Hz Resolution |
659
+
|--------|------------------|-------------------|-------------------|
660
+
| FFT | ±10.75 Hz (25%) | ±10.75 Hz (2.4%) | ±10.75 Hz (0.24%) |
661
+
| CQT | ±1.4 Hz (3.5%) | ±15 Hz (3.5%) | ±150 Hz (3.5%) |
662
+
663
+
### Configurable CQT FFT Sizes
664
+
665
+
| FFT Size | Update Rate | Low Freq Quality | Use Case |
666
+
|----------|-------------|------------------|----------|
667
+
| 4K | 10.8 fps | Poor (Q≈1.9 @ 20Hz) | Too lossy for music |
668
+
| 8K | 5.4 fps | Decent (Q≈3.7 @ 20Hz) | Good for livecoding |
669
+
| 16K | 2.7 fps | Good (Q≈7.4 @ 20Hz) | Best accuracy |
670
+
671
+
Current implementation uses 8K by default, configurable via `CQT_FFT_SIZE` in cqtdata.h.
672
+
673
+
### Smoothing Factors
674
+
- **FFT**: 0.6 (60% old, 40% new) - more stable
675
+
- **CQT**: 0.3 (30% old, 70% new) - more responsive
676
+
- Peak tracking uses 0.99 factor for slow decay
677
+
678
+
### Shared Audio Buffer Architecture
679
+
Both FFT and CQT share the same audio capture buffer:
680
+
- Buffer size: Maximum of (2048, CQT_FFT_SIZE) samples
681
+
- FFT reads samples 0-2047 (always the same)
682
+
- CQT reads samples 0-(CQT_FFT_SIZE-1)
683
+
- This preserves exact FFT behavior while allowing CQT flexibility
684
+
685
+
### Practical Usage for Electronic Music Visualization
686
+
687
+
**Use FFT for:**
688
+
- Kick drum detection (bins 2-4, ~40-80 Hz)
689
+
- Beat synchronization
690
+
- Energy meters by frequency band
691
+
- Reactive elements needing >10 fps update
692
+
693
+
**Use CQT for:**
694
+
- Bass note identification
695
+
- Chord/key detection
696
+
- Color mapping from musical content
697
+
- Melodic visualization
698
+
699
+
**Hybrid Approach (Recommended):**
700
+
```lua
701
+
-- Rhythm from FFT
702
+
local kick = fft(2) + fft(3) + fft(4)
703
+
704
+
-- Musical content from CQT
705
+
local bassNote = 0
706
+
for i=24,35 do -- 2nd octave
707
+
if cqt(i) > cqt(bassNote) then bassNote = i end
708
+
end
709
+
710
+
-- Combine for visuals
711
+
local pulse = kick * 2 -- Size from rhythm
712
+
local color = (bassNote % 12) + 1 -- Color from note
713
+
```
714
+
715
+
### Sample Rate and Frequency Ranges
716
+
- TIC-80 sample rate: 44100 Hz
717
+
- Nyquist frequency: 22050 Hz
718
+
- Both FFT and CQT analyze 0-22050 Hz
719
+
- CQT specifically tuned for 20 Hz - 20480 Hz (musical range)
603
720
604
721
## Important Notes
605
722