this repo has no description
CQT -> VQT
+724
-768
+52
-52
CLAUDE.md
+52
-52
CLAUDE.md
···
116
116
value = ffts(bin) -- Get smoothed FFT magnitude for bin (0-1023)
117
117
```
118
118
119
-
### CQT Implementation
119
+
### VQT Implementation
120
120
121
121
Constant-Q Transform provides logarithmic frequency spacing for better musical analysis.
122
122
123
123
#### Architecture
124
-
- **src/ext/cqt.c**: CQT processing with separate FFT
125
-
- **src/ext/cqt_kernel.c**: Kernel generation with sparse storage
126
-
- **src/cqtdata.h/c**: CQT data storage and configuration
124
+
- **src/ext/vqt.c**: VQT processing with separate FFT
125
+
- **src/ext/vqt_kernel.c**: Kernel generation with sparse storage
126
+
- **src/vqtdata.h/c**: VQT data storage and configuration
127
127
- **Processing**: In `tic_core_tick` after FFT processing
128
128
129
129
#### Specifications
···
132
132
- Update rate: ~5.4 fps with 8K FFT
133
133
- Variable-Q implementation optimized for 8K FFT constraint
134
134
- Smoothing factor: 0.3
135
-
- Spectral whitening: Enabled by default (toggle via `CQT_SPECTRAL_WHITENING_ENABLED`)
135
+
- Spectral whitening: Enabled by default (toggle via `VQT_SPECTRAL_WHITENING_ENABLED`)
136
136
137
137
#### Variable-Q Design (8K FFT Optimized)
138
138
| Frequency Range | Design Q | Effective Q | Resolution |
···
143
143
| 40-50 Hz | 14.5 | 7.4-9.3 | ~4.3 Hz |
144
144
| 50-65 Hz | 16.0 | 9.3-12.1 | ~4.1 Hz |
145
145
| 65-80 Hz | 17.0 | 12.1-17.0 | ~4.7 Hz |
146
-
| 80+ Hz | 17.0 | 17.0 | Standard CQT |
146
+
| 80+ Hz | 17.0 | 17.0 | Standard VQT |
147
147
148
148
#### API Functions
149
149
```lua
150
-
value = cqt(bin) -- Get raw CQT magnitude for bin (0-119)
150
+
value = vqt(bin) -- Get raw VQT magnitude for bin (0-119)
151
151
-- Note mapping: Bin = octave * 12 + note
152
152
-- Note: C=0, C#=1, D=2, D#=3, E=4, F=5, F#=6, G=7, G#=8, A=9, A#=10, B=11
153
153
```
154
154
155
-
### FFT vs CQT Comparison
155
+
### FFT vs VQT Comparison
156
156
157
-
| Aspect | FFT | CQT |
157
+
| Aspect | FFT | VQT |
158
158
|--------|-----|-----|
159
159
| Frequency spacing | Linear (21.5 Hz/bin) | Logarithmic (musical) |
160
160
| Update rate | ~21 fps | ~5.4 fps |
···
165
165
166
166
### Shared Audio Buffer Architecture
167
167
168
-
Both FFT and CQT share the same audio capture buffer:
169
-
- Buffer size: Maximum of (2048, CQT_FFT_SIZE) samples
168
+
Both FFT and VQT share the same audio capture buffer:
169
+
- Buffer size: Maximum of (2048, VQT_FFT_SIZE) samples
170
170
- FFT reads samples 0-2047
171
-
- CQT reads samples 0-(CQT_FFT_SIZE-1)
171
+
- VQT reads samples 0-(VQT_FFT_SIZE-1)
172
172
- Uses miniaudio for audio capture (mic or loopback on Windows)
173
173
174
174
### Practical Usage Guidelines
···
179
179
- Energy meters by frequency band
180
180
- Any rhythm visualization above 150 BPM
181
181
182
-
**Use CQT for:**
182
+
**Use VQT for:**
183
183
- Bass note identification
184
184
- Chord/key detection
185
185
- Color mapping from musical content
···
190
190
-- Rhythm from FFT (21 fps)
191
191
local kick = fft(2) + fft(3) + fft(4)
192
192
193
-
-- Musical content from CQT (5.4 fps)
193
+
-- Musical content from VQT (5.4 fps)
194
194
local bassNote = 0
195
195
for i=24,35 do -- 2nd octave
196
-
if cqt(i) > cqt(bassNote) then bassNote = i end
196
+
if vqt(i) > vqt(bassNote) then bassNote = i end
197
197
end
198
198
199
199
-- Combine for visuals
···
205
205
206
206
### Completed Features
207
207
- **FFT**: 1024 bins with exact original behavior preserved
208
-
- **CQT**: 120 bins with Variable-Q implementation
209
-
- **Spectral Whitening**: Per-bin normalization for CQT
210
-
- **Shared Audio Buffer**: Automatic sizing for both FFT and CQT
211
-
- **Lua API**: `fft()`, `ffts()`, `cqt()` functions implemented
208
+
- **VQT**: 120 bins with Variable-Q implementation
209
+
- **Spectral Whitening**: Per-bin normalization for VQT
210
+
- **Shared Audio Buffer**: Automatic sizing for both FFT and VQT
211
+
- **Lua API**: `fft()`, `ffts()`, `vqt()` functions implemented
212
212
213
213
### Configuration Options
214
-
- `CQT_FFT_SIZE`: Default 8192 (configurable in cqtdata.h)
215
-
- `CQT_SPECTRAL_WHITENING_ENABLED`: Toggle spectral whitening (0/1)
216
-
- `CQT_WHITENING_DECAY`: Running average decay factor (default 0.99)
217
-
- `CQT_SMOOTHING_FACTOR`: CQT smoothing factor (default 0.3)
214
+
- `VQT_FFT_SIZE`: Default 8192 (configurable in vqtdata.h)
215
+
- `VQT_SPECTRAL_WHITENING_ENABLED`: Toggle spectral whitening (0/1)
216
+
- `VQT_WHITENING_DECAY`: Running average decay factor (default 0.99)
217
+
- `VQT_SMOOTHING_FACTOR`: VQT smoothing factor (default 0.3)
218
218
219
219
### Test Scripts
220
-
- `demo_fft_cqt_hybrid.lua`: Combined FFT/CQT visualization
221
-
- `test_cqt_spectrum_v2.lua`: CQT spectrum analyzer
220
+
- `demo_fft_vqt_hybrid.lua`: Combined FFT/VQT visualization
221
+
- `test_vqt_spectrum_v2.lua`: VQT spectrum analyzer
222
222
- `test_fft_restored.lua`: FFT verification
223
-
- `test_cqt_variable_q.lua`: Variable-Q demonstration
224
-
- `test_cqt_whitening.lua`: Spectral whitening comparison
223
+
- `test_vqt_variable_q.lua`: Variable-Q demonstration
224
+
- `test_vqt_whitening.lua`: Spectral whitening comparison
225
225
226
226
## Future Enhancements
227
227
228
228
### Additional API Functions
229
-
- `cqts(bin)`: Smoothed CQT data
230
-
- `cqto(octave, note)`: Raw CQT by musical note
231
-
- `cqtos(octave, note)`: Smoothed CQT by musical note
229
+
- `vqts(bin)`: Smoothed VQT data
230
+
- `vqto(octave, note)`: Raw VQT by musical note
231
+
- `vqtos(octave, note)`: Smoothed VQT by musical note
232
232
233
233
### Signal Processing Enhancements
234
234
- **Harmonic-Percussive Separation**: Isolate tonal content from drums
···
237
237
- **Enhanced Variable-Q**: Support for 16K FFT for better bass resolution
238
238
239
239
### Platform Support
240
-
- Add CQT to other language bindings (currently Lua only)
240
+
- Add VQT to other language bindings (currently Lua only)
241
241
- GPU acceleration for kernel application
242
242
- Configurable FFT sizes at runtime
243
243
···
253
253
254
254
### Overview
255
255
256
-
Harmonic-Percussive Separation isolates tonal (harmonic) content from rhythmic (percussive) content in audio signals. This enhancement will apply HPS exclusively to CQT data, providing better separation of musical elements like sustained notes from drum hits.
256
+
Harmonic-Percussive Separation isolates tonal (harmonic) content from rhythmic (percussive) content in audio signals. This enhancement will apply HPS exclusively to VQT data, providing better separation of musical elements like sustained notes from drum hits.
257
257
258
258
### Algorithm Design
259
259
260
260
**Median Filtering Approach (Fitzgerald 2010):**
261
-
- Apply median filters on CQT magnitude spectrogram
261
+
- Apply median filters on VQT magnitude spectrogram
262
262
- Horizontal (time) median filter → captures harmonic content (sustained tones)
263
263
- Vertical (frequency) median filter → captures percussive content (transients)
264
264
- Generate masks by comparing filtered outputs
···
273
273
274
274
#### Data Flow
275
275
```
276
-
Audio Buffer → CQT Processing → CQT Magnitude → HPS Processing → Separated Components
276
+
Audio Buffer → VQT Processing → VQT Magnitude → HPS Processing → Separated Components
277
277
↓
278
-
Harmonic & Percussive CQT
278
+
Harmonic & Percussive VQT
279
279
```
280
280
281
281
#### Integration in tic_core_tick
···
283
283
if (fftEnabled) {
284
284
FFT_GetFFT(fftData); // Regular FFT processing (unchanged)
285
285
286
-
if (cqtEnabled) {
287
-
CQT_ProcessAudio();
286
+
if (vqtEnabled) {
287
+
VQT_ProcessAudio();
288
288
289
289
if (hpsEnabled) {
290
-
HPS_ProcessCQT(cqtData); // Apply HPS to CQT only
290
+
HPS_ProcessVQT(vqtData); // Apply HPS to VQT only
291
291
}
292
292
}
293
293
}
···
302
302
#define HPS_MEDIAN_SIZE_P 17 // Vertical filter size (1.4 octaves)
303
303
304
304
typedef struct {
305
-
// CQT magnitude history buffer
306
-
float cqtHistory[HPS_HISTORY_SIZE][120];
305
+
// VQT magnitude history buffer
306
+
float vqtHistory[HPS_HISTORY_SIZE][120];
307
307
int historyIndex;
308
308
309
-
// Separated CQT components
310
-
float harmonicCQT[120]; // Sustained tonal content
311
-
float percussiveCQT[120]; // Transient/rhythmic content
309
+
// Separated VQT components
310
+
float harmonicVQT[120]; // Sustained tonal content
311
+
float percussiveVQT[120]; // Transient/rhythmic content
312
312
313
313
// Smoothed outputs
314
314
float harmonicSmoothing[120];
···
329
329
### API Functions
330
330
331
331
```lua
332
-
-- CQT-based HPS functions (update rate ~5.4 fps)
333
-
hpsh(bin) -- Get harmonic component at CQT bin (0-119)
334
-
hpsp(bin) -- Get percussive component at CQT bin (0-119)
332
+
-- VQT-based HPS functions (update rate ~5.4 fps)
333
+
hpsh(bin) -- Get harmonic component at VQT bin (0-119)
334
+
hpsp(bin) -- Get percussive component at VQT bin (0-119)
335
335
hpshs(bin) -- Get smoothed harmonic component
336
336
hpsps(bin) -- Get smoothed percussive component
337
337
···
348
348
349
349
#### Median Filtering
350
350
- Use efficient sliding window median algorithm
351
-
- Handle CQT's logarithmic frequency spacing
351
+
- Handle VQT's logarithmic frequency spacing
352
352
- Circular buffer for time history
353
353
- Optimize for 120 bins and 32 frame history
354
354
355
355
#### Masking Strategy
356
-
1. Compute harmonic emphasis: `H = median_h(|CQT|²)`
357
-
2. Compute percussive emphasis: `P = median_p(|CQT|²)`
356
+
1. Compute harmonic emphasis: `H = median_h(|VQT|²)`
357
+
2. Compute percussive emphasis: `P = median_p(|VQT|²)`
358
358
3. Generate binary masks:
359
359
- Harmonic mask: `M_h = H >= P`
360
360
- Percussive mask: `M_p = P > H`
···
367
367
368
368
### Performance Considerations
369
369
370
-
- Process only when CQT is updated (~5.4 fps)
371
-
- Reuse existing CQT magnitude calculations
370
+
- Process only when VQT is updated (~5.4 fps)
371
+
- Reuse existing VQT magnitude calculations
372
372
- Optimize median filters for small kernel sizes
373
373
- Cache-friendly memory access patterns
374
374
- Optional SIMD for median operations
···
428
428
### Implementation Phases
429
429
430
430
**Phase 1: Core HPS Algorithm**
431
-
- Implement circular buffer for CQT history
431
+
- Implement circular buffer for VQT history
432
432
- Create median filtering functions
433
433
- Implement basic binary masking
434
434
- Add core API functions
-44
analyze_esp32_q.py
-44
analyze_esp32_q.py
···
1
-
#!/usr/bin/env python3
2
-
# Analyze effective Q factor in ESP32 approach
3
-
4
-
import math
5
-
6
-
fs = 44100 # Sample rate
7
-
N = 4096 # FFT size
8
-
fmin = 20 # Minimum frequency
9
-
10
-
print("ESP32 CQT Analysis")
11
-
print("==================")
12
-
print(f"FFT size: {N}, Sample rate: {fs} Hz, Min freq: {fmin} Hz")
13
-
print()
14
-
15
-
# Test frequencies
16
-
freqs = [20, 27.5, 50, 100, 200, 440, 1000, 2000, 5000, 10000, 20000]
17
-
18
-
print("Freq(Hz) | Window | Eff. Q | BW(Hz) | BW(semitones)")
19
-
print("---------|--------|--------|--------|---------------")
20
-
21
-
for f in freqs:
22
-
# ESP32 window calculation: N_window = N / (f/fmin)
23
-
factor = f / fmin
24
-
window_length = int(N / factor)
25
-
26
-
# Effective Q = window_length * f / fs
27
-
eff_q = window_length * f / fs
28
-
29
-
# Bandwidth = f / Q
30
-
bandwidth = f / eff_q if eff_q > 0 else float('inf')
31
-
32
-
# Bandwidth in semitones = 12 * log2(1 + bandwidth/f)
33
-
bw_semitones = 12 * math.log2(1 + bandwidth/f) if bandwidth < float('inf') else float('inf')
34
-
35
-
print(f"{f:7.1f} | {window_length:6d} | {eff_q:6.1f} | {bandwidth:6.1f} | {bw_semitones:13.1f}")
36
-
37
-
print()
38
-
print("Observations:")
39
-
print("1. Q varies dramatically with frequency (1.9 to 93.0)")
40
-
print("2. Low frequencies: Very low Q (wide bandwidth)")
41
-
print("3. High frequencies: Very high Q (narrow bandwidth)")
42
-
print()
43
-
print("For 12 bins/octave, ideal constant Q ≈ 17")
44
-
print("ESP32 only achieves this around 200-300 Hz")
+3
-3
cmake/core.cmake
+3
-3
cmake/core.cmake
···
16
16
set(TIC80CORE_DIR ${CMAKE_SOURCE_DIR}/src)
17
17
set(TIC80CORE_SRC
18
18
${TIC80CORE_DIR}/fftdata.c
19
-
${TIC80CORE_DIR}/cqtdata.c
19
+
${TIC80CORE_DIR}/vqtdata.c
20
20
${TIC80CORE_DIR}/core/core.c
21
21
${TIC80CORE_DIR}/core/draw.c
22
22
${TIC80CORE_DIR}/core/io.c
···
28
28
${TIC80CORE_DIR}/tilesheet.c
29
29
${TIC80CORE_DIR}/script.c
30
30
${TIC80CORE_DIR}/ext/fft.c
31
-
${TIC80CORE_DIR}/ext/cqt.c
32
-
${TIC80CORE_DIR}/ext/cqt_kernel.c
31
+
${TIC80CORE_DIR}/ext/vqt.c
32
+
${TIC80CORE_DIR}/ext/vqt_kernel.c
33
33
${TIC80CORE_DIR}/ext/kiss_fft.c
34
34
${TIC80CORE_DIR}/ext/kiss_fftr.c
35
35
)
+9
-9
src/api/luaapi.c
+9
-9
src/api/luaapi.c
···
21
21
// SOFTWARE.
22
22
23
23
#include "core/core.h"
24
-
#include "cqtdata.h"
24
+
#include "vqtdata.h"
25
25
26
26
#include <stdlib.h>
27
27
#include <lua.h>
···
1596
1596
return 0;
1597
1597
}
1598
1598
1599
-
static s32 lua_cqt(lua_State* lua)
1599
+
static s32 lua_vqt(lua_State* lua)
1600
1600
{
1601
1601
s32 top = lua_gettop(lua);
1602
1602
···
1605
1605
s32 bin = getLuaNumber(lua, 1);
1606
1606
1607
1607
// Validate bin range
1608
-
if (bin < 0 || bin >= CQT_BINS)
1608
+
if (bin < 0 || bin >= VQT_BINS)
1609
1609
{
1610
-
luaL_error(lua, "cqt bin out of range (0-%d)\n", CQT_BINS - 1);
1610
+
luaL_error(lua, "vqt bin out of range (0-%d)\n", VQT_BINS - 1);
1611
1611
return 0;
1612
1612
}
1613
1613
1614
-
// Return normalized CQT data
1615
-
lua_pushnumber(lua, cqtNormalizedData[bin]);
1614
+
// Return normalized VQT data
1615
+
lua_pushnumber(lua, vqtNormalizedData[bin]);
1616
1616
return 1;
1617
1617
}
1618
1618
1619
-
luaL_error(lua, "invalid params, cqt(bin)\n");
1619
+
luaL_error(lua, "invalid params, vqt(bin)\n");
1620
1620
return 0;
1621
1621
}
1622
1622
···
1674
1674
registerLuaFunction(core, lua_dofile, "dofile");
1675
1675
registerLuaFunction(core, lua_loadfile, "loadfile");
1676
1676
1677
-
// Register CQT function
1678
-
registerLuaFunction(core, lua_cqt, "cqt");
1677
+
// Register VQT function
1678
+
registerLuaFunction(core, lua_vqt, "vqt");
1679
1679
}
1680
1680
1681
1681
void luaapi_close(tic_mem* tic)
+8
-8
src/core/core.c
+8
-8
src/core/core.c
···
22
22
23
23
#include "fftdata.h"
24
24
#include "../ext/fft.h"
25
-
#include "cqtdata.h"
26
-
#include "../ext/cqt.h"
25
+
#include "vqtdata.h"
26
+
#include "../ext/vqt.h"
27
27
28
28
#include "api.h"
29
29
#include "core.h"
···
445
445
{
446
446
FFT_GetFFT(fftData);
447
447
448
-
// Process CQT using raw audio buffer
449
-
// For now, tie CQT to FFT enable flag
450
-
cqtEnabled = fftEnabled;
451
-
if (cqtEnabled)
448
+
// Process VQT using raw audio buffer
449
+
// For now, tie VQT to FFT enable flag
450
+
vqtEnabled = fftEnabled;
451
+
if (vqtEnabled)
452
452
{
453
-
// Process CQT from the shared audio buffer
454
-
CQT_ProcessAudio();
453
+
// Process VQT from the shared audio buffer
454
+
VQT_ProcessAudio();
455
455
}
456
456
}
457
457
if (!core->state.initialized)
-71
src/cqtdata.c
-71
src/cqtdata.c
···
1
-
#include "cqtdata.h"
2
-
#include <string.h>
3
-
#include <stdlib.h>
4
-
5
-
// Global CQT data arrays
6
-
float cqtData[CQT_BINS];
7
-
float cqtSmoothingData[CQT_BINS];
8
-
float cqtNormalizedData[CQT_BINS];
9
-
10
-
// Peak tracking for auto-gain
11
-
float cqtPeakValue = 1.0f;
12
-
float cqtPeakSmoothValue = 1.0f;
13
-
14
-
// Enable flag (tied to fftEnabled initially)
15
-
bool cqtEnabled = false;
16
-
17
-
// Array of kernels, one per CQT bin
18
-
CqtKernel cqtKernels[CQT_BINS];
19
-
20
-
// Spectral whitening data
21
-
float cqtBinAverages[CQT_BINS];
22
-
float cqtWhitenedData[CQT_BINS];
23
-
24
-
void CQT_Init(void)
25
-
{
26
-
// Zero all data arrays
27
-
memset(cqtData, 0, sizeof(cqtData));
28
-
memset(cqtSmoothingData, 0, sizeof(cqtSmoothingData));
29
-
memset(cqtNormalizedData, 0, sizeof(cqtNormalizedData));
30
-
31
-
// Initialize peak values
32
-
cqtPeakValue = 1.0f;
33
-
cqtPeakSmoothValue = 1.0f;
34
-
35
-
// Zero kernel pointers
36
-
memset(cqtKernels, 0, sizeof(cqtKernels));
37
-
38
-
// Initialize spectral whitening arrays
39
-
memset(cqtBinAverages, 0, sizeof(cqtBinAverages));
40
-
memset(cqtWhitenedData, 0, sizeof(cqtWhitenedData));
41
-
42
-
// Start with small initial averages to avoid divide-by-zero
43
-
for (int i = 0; i < CQT_BINS; i++)
44
-
{
45
-
cqtBinAverages[i] = CQT_WHITENING_FLOOR;
46
-
}
47
-
}
48
-
49
-
void CQT_Cleanup(void)
50
-
{
51
-
// Free all allocated kernel memory
52
-
for (int i = 0; i < CQT_BINS; i++)
53
-
{
54
-
if (cqtKernels[i].real)
55
-
{
56
-
free(cqtKernels[i].real);
57
-
cqtKernels[i].real = NULL;
58
-
}
59
-
if (cqtKernels[i].imag)
60
-
{
61
-
free(cqtKernels[i].imag);
62
-
cqtKernels[i].imag = NULL;
63
-
}
64
-
if (cqtKernels[i].indices)
65
-
{
66
-
free(cqtKernels[i].indices);
67
-
cqtKernels[i].indices = NULL;
68
-
}
69
-
cqtKernels[i].length = 0;
70
-
}
71
-
}
-66
src/cqtdata.h
-66
src/cqtdata.h
···
1
-
#pragma once
2
-
#include <stdbool.h>
3
-
4
-
#define CQT_BINS 120
5
-
#define CQT_OCTAVES 10
6
-
#define CQT_BINS_PER_OCTAVE 12
7
-
#define CQT_FFT_SIZE 8192 // 8K FFT - optimized variable-Q for responsive visualization, ~5.4 fps
8
-
9
-
// CQT frequency range
10
-
#define CQT_MIN_FREQ 20.0f // Sub-bass for electronic music
11
-
#define CQT_MAX_FREQ 20480.0f // Nearest note to 20kHz
12
-
13
-
// Smoothing parameters
14
-
#define CQT_SMOOTHING_FACTOR 0.3f // Reduced from 0.7f for more responsive display
15
-
#define CQT_SPARSITY_THRESHOLD 0.01f
16
-
17
-
// Variable-Q configuration (optimized for 8K FFT)
18
-
#define CQT_VARIABLE_Q_ENABLED 1
19
-
#define CQT_8K_OPTIMIZED 1 // Uses Q values that fit within 8K FFT
20
-
#define CQT_BASS_Q_MIN 7.4f // Minimum Q at 20 Hz (constrained by 8K)
21
-
#define CQT_BASS_Q_MAX 17.0f // Full Q achieved at 80+ Hz
22
-
#define CQT_TREBLE_Q_FACTOR 11.0f // Smoother for high frequencies
23
-
24
-
// Spectral whitening toggle (set to 0 to disable)
25
-
#define CQT_SPECTRAL_WHITENING_ENABLED 1
26
-
27
-
// Spectral whitening parameters
28
-
#define CQT_WHITENING_DECAY 0.99f // Running average decay (0.98-0.995 for 1-2 second adaptation)
29
-
#define CQT_WHITENING_FLOOR 0.001f // Minimum average to prevent divide-by-zero
30
-
31
-
// Raw CQT magnitude data
32
-
extern float cqtData[CQT_BINS];
33
-
34
-
// Smoothed CQT data for visual stability
35
-
extern float cqtSmoothingData[CQT_BINS];
36
-
37
-
// Normalized CQT data (0-1 range)
38
-
extern float cqtNormalizedData[CQT_BINS];
39
-
40
-
// Peak tracking for auto-gain
41
-
extern float cqtPeakValue;
42
-
extern float cqtPeakSmoothValue;
43
-
44
-
// Enable flag (tied to fftEnabled initially)
45
-
extern bool cqtEnabled;
46
-
47
-
// Spectral whitening data
48
-
extern float cqtBinAverages[CQT_BINS]; // Long-term running averages per bin
49
-
extern float cqtWhitenedData[CQT_BINS]; // Whitened CQT data
50
-
51
-
// Sparse kernel storage structures
52
-
typedef struct {
53
-
float* real; // Real parts of kernel (sparse)
54
-
float* imag; // Imaginary parts of kernel (sparse)
55
-
int* indices; // Non-zero FFT bin indices
56
-
int length; // Number of non-zero elements
57
-
} CqtKernel;
58
-
59
-
// Array of kernels, one per CQT bin
60
-
extern CqtKernel cqtKernels[CQT_BINS];
61
-
62
-
// Initialize CQT data structures
63
-
void CQT_Init(void);
64
-
65
-
// Free CQT kernel memory
66
-
void CQT_Cleanup(void);
-441
src/ext/cqt.c
-441
src/ext/cqt.c
···
1
-
#include "cqt.h"
2
-
#include "cqt_kernel.h"
3
-
#include "../cqtdata.h"
4
-
#include "../fftdata.h"
5
-
#include "fft.h"
6
-
#include "kiss_fftr.h"
7
-
#include <math.h>
8
-
#include <string.h>
9
-
#include <stdbool.h>
10
-
#include <stdio.h>
11
-
#include <time.h>
12
-
13
-
#define CQT_DEBUG
14
-
15
-
// FFT configuration for CQT
16
-
static kiss_fftr_cfg cqtFftCfg = NULL;
17
-
static float* cqtAudioBuffer = NULL;
18
-
static kiss_fft_cpx* cqtFftOutput = NULL;
19
-
20
-
// Benchmark different FFT sizes
21
-
static void CQT_BenchmarkFFT(void)
22
-
{
23
-
printf("\nCQT FFT Benchmark on this CPU:\n");
24
-
printf("================================\n");
25
-
26
-
int sizes[] = {4096, 6144, 8192, 12288, 16384, 24576, 32768};
27
-
int numSizes = 7;
28
-
29
-
for (int s = 0; s < numSizes; s++)
30
-
{
31
-
int fftSize = sizes[s];
32
-
33
-
// Allocate buffers
34
-
float* testInput = (float*)calloc(fftSize, sizeof(float));
35
-
kiss_fft_cpx* testOutput = (kiss_fft_cpx*)malloc((fftSize/2 + 1) * sizeof(kiss_fft_cpx));
36
-
kiss_fftr_cfg testCfg = kiss_fftr_alloc(fftSize, 0, NULL, NULL);
37
-
38
-
if (!testInput || !testOutput || !testCfg)
39
-
{
40
-
printf("Failed to allocate for size %d\n", fftSize);
41
-
continue;
42
-
}
43
-
44
-
// Fill with test signal
45
-
for (int i = 0; i < fftSize; i++)
46
-
{
47
-
testInput[i] = sin(2.0 * M_PI * 440.0 * i / 44100.0);
48
-
}
49
-
50
-
// Warm up
51
-
for (int i = 0; i < 10; i++)
52
-
{
53
-
kiss_fftr(testCfg, testInput, testOutput);
54
-
}
55
-
56
-
// Time 100 iterations
57
-
clock_t start = clock();
58
-
for (int i = 0; i < 100; i++)
59
-
{
60
-
kiss_fftr(testCfg, testInput, testOutput);
61
-
}
62
-
clock_t end = clock();
63
-
64
-
double avgTime = (double)(end - start) / CLOCKS_PER_SEC * 1000.0 / 100.0;
65
-
printf("%5d-point FFT: %.3f ms\n", fftSize, avgTime);
66
-
67
-
// Cleanup
68
-
free(testInput);
69
-
free(testOutput);
70
-
free(testCfg);
71
-
}
72
-
73
-
printf("================================\n\n");
74
-
}
75
-
76
-
// Initialize CQT processing
77
-
bool CQT_Open(void)
78
-
{
79
-
// Initialize data structures
80
-
CQT_Init();
81
-
82
-
// Allocate FFT buffers
83
-
cqtAudioBuffer = (float*)malloc(CQT_FFT_SIZE * sizeof(float));
84
-
cqtFftOutput = (kiss_fft_cpx*)malloc((CQT_FFT_SIZE/2 + 1) * sizeof(kiss_fft_cpx));
85
-
86
-
if (!cqtAudioBuffer || !cqtFftOutput)
87
-
{
88
-
CQT_Close();
89
-
return false;
90
-
}
91
-
92
-
// Create FFT configuration
93
-
cqtFftCfg = kiss_fftr_alloc(CQT_FFT_SIZE, 0, NULL, NULL);
94
-
if (!cqtFftCfg)
95
-
{
96
-
CQT_Close();
97
-
return false;
98
-
}
99
-
100
-
// Configure CQT kernels
101
-
CqtKernelConfig config = {
102
-
.fftSize = CQT_FFT_SIZE,
103
-
.numBins = CQT_BINS,
104
-
.minFreq = CQT_MIN_FREQ,
105
-
.maxFreq = CQT_MAX_FREQ,
106
-
.sampleRate = 44100.0f, // Standard TIC-80 sample rate
107
-
.windowType = CQT_WINDOW_HAMMING,
108
-
.sparsityThreshold = CQT_SPARSITY_THRESHOLD
109
-
};
110
-
111
-
// Generate kernels
112
-
if (!CQT_GenerateKernels(cqtKernels, &config))
113
-
{
114
-
CQT_Close();
115
-
return false;
116
-
}
117
-
118
-
// Run FFT benchmark on first initialization
119
-
static bool benchmarkRun = false;
120
-
if (!benchmarkRun)
121
-
{
122
-
CQT_BenchmarkFFT();
123
-
benchmarkRun = true;
124
-
}
125
-
126
-
// Debug: Print variable Q values across frequency spectrum
127
-
#ifdef CQT_DEBUG
128
-
float centerFreqs[CQT_BINS];
129
-
CQT_GenerateCenterFrequencies(centerFreqs, CQT_BINS, CQT_MIN_FREQ, CQT_MAX_FREQ);
130
-
131
-
printf("\nCQT Variable-Q Implementation (8K FFT Optimized):\n");
132
-
printf("================================================\n");
133
-
134
-
// Show Q values for key frequency ranges
135
-
float testFreqs[] = {20, 25, 30, 40, 50, 65, 80, 120, 160, 240, 320, 440, 640, 1000, 2000, 4000};
136
-
printf("Frequency | Design Q | Window Length | Effective Q | Resolution\n");
137
-
printf("----------|----------|---------------|-------------|------------\n");
138
-
139
-
for (int i = 0; i < 16; i++)
140
-
{
141
-
float freq = testFreqs[i];
142
-
// Find closest CQT bin
143
-
int closestBin = 0;
144
-
float minDiff = fabs(centerFreqs[0] - freq);
145
-
for (int j = 1; j < CQT_BINS; j++)
146
-
{
147
-
float diff = fabs(centerFreqs[j] - freq);
148
-
if (diff < minDiff)
149
-
{
150
-
minDiff = diff;
151
-
closestBin = j;
152
-
}
153
-
}
154
-
155
-
// Calculate Q values using 8K-optimized function
156
-
float designQ;
157
-
if (freq < 25.0f) designQ = 7.4f;
158
-
else if (freq < 30.0f) designQ = 9.2f;
159
-
else if (freq < 40.0f) designQ = 11.5f;
160
-
else if (freq < 50.0f) designQ = 14.5f;
161
-
else if (freq < 65.0f) designQ = 16.0f;
162
-
else if (freq < 80.0f) designQ = 17.0f;
163
-
else if (freq < 160.0f) designQ = 17.0f;
164
-
else if (freq < 320.0f) designQ = 15.0f;
165
-
else if (freq < 640.0f) designQ = 13.0f;
166
-
else designQ = 11.0f;
167
-
168
-
int windowLength = (int)(designQ * 44100.0f / freq);
169
-
if (windowLength > CQT_FFT_SIZE) windowLength = CQT_FFT_SIZE;
170
-
float effectiveQ = windowLength * freq / 44100.0f;
171
-
float bandwidth = freq / effectiveQ;
172
-
173
-
printf("%7.0f Hz | %7.1f | %13d | %11.1f | %7.1f Hz\n",
174
-
freq, designQ, windowLength, effectiveQ, bandwidth);
175
-
}
176
-
177
-
printf("\nFirst 10 CQT bins:\n");
178
-
for (int i = 0; i < 10 && i < CQT_BINS; i++)
179
-
{
180
-
int expectedBin = (int)(centerFreqs[i] * CQT_FFT_SIZE / 44100.0f);
181
-
printf(" Bin %d: %.2f Hz -> FFT bin %d\n", i, centerFreqs[i], expectedBin);
182
-
}
183
-
#endif
184
-
185
-
return true;
186
-
}
187
-
188
-
// Apply CQT kernels to FFT output
189
-
void CQT_ApplyKernels(const float* fftReal, const float* fftImag)
190
-
{
191
-
// Clear CQT output
192
-
memset(cqtData, 0, sizeof(cqtData));
193
-
194
-
// Apply each kernel to compute CQT bins
195
-
for (int bin = 0; bin < CQT_BINS; bin++)
196
-
{
197
-
CqtKernel* kernel = &cqtKernels[bin];
198
-
199
-
// Check if kernel is valid
200
-
if (!kernel->real || !kernel->imag || !kernel->indices || kernel->length == 0)
201
-
{
202
-
cqtData[bin] = 0.0f;
203
-
continue;
204
-
}
205
-
206
-
float real = 0.0f;
207
-
float imag = 0.0f;
208
-
209
-
// Sparse matrix multiplication
210
-
for (int k = 0; k < kernel->length; k++)
211
-
{
212
-
int idx = kernel->indices[k];
213
-
// Ensure index is within bounds
214
-
if (idx < 0 || idx >= CQT_FFT_SIZE/2 + 1)
215
-
continue;
216
-
217
-
// Complex multiplication: (a + bi) * (c + di) = (ac - bd) + (ad + bc)i
218
-
real += fftReal[idx] * kernel->real[k] - fftImag[idx] * kernel->imag[k];
219
-
imag += fftReal[idx] * kernel->imag[k] + fftImag[idx] * kernel->real[k];
220
-
}
221
-
222
-
// Calculate magnitude with gain boost
223
-
cqtData[bin] = sqrt(real * real + imag * imag) * 2.0f; // Match FFT gain factor
224
-
225
-
// Check for NaN or Inf
226
-
if (!isfinite(cqtData[bin]))
227
-
cqtData[bin] = 0.0f;
228
-
}
229
-
}
230
-
231
-
// Process CQT from audio data
232
-
void CQT_ProcessAudio(void)
233
-
{
234
-
if (!cqtFftCfg || !cqtEnabled) return;
235
-
236
-
// Check if kernels are initialized
237
-
bool kernelsValid = false;
238
-
for (int i = 0; i < CQT_BINS; i++)
239
-
{
240
-
if (cqtKernels[i].real && cqtKernels[i].length > 0)
241
-
{
242
-
kernelsValid = true;
243
-
break;
244
-
}
245
-
}
246
-
247
-
if (!kernelsValid)
248
-
{
249
-
// Kernels not initialized, set all output to zero
250
-
memset(cqtData, 0, sizeof(cqtData));
251
-
memset(cqtSmoothingData, 0, sizeof(cqtSmoothingData));
252
-
memset(cqtNormalizedData, 0, sizeof(cqtNormalizedData));
253
-
return;
254
-
}
255
-
256
-
// Copy audio data from the shared buffer
257
-
// sampleBuf is defined in fft.c as extern
258
-
extern float sampleBuf[];
259
-
memcpy(cqtAudioBuffer, sampleBuf, CQT_FFT_SIZE * sizeof(float));
260
-
261
-
// Check if we have any audio data
262
-
float audioSum = 0.0f;
263
-
for (int i = 0; i < CQT_FFT_SIZE; i++)
264
-
{
265
-
audioSum += fabs(cqtAudioBuffer[i]);
266
-
}
267
-
268
-
if (audioSum < 0.0001f)
269
-
{
270
-
// No audio data, set output to zero
271
-
memset(cqtData, 0, sizeof(cqtData));
272
-
memset(cqtSmoothingData, 0, sizeof(cqtSmoothingData));
273
-
memset(cqtNormalizedData, 0, sizeof(cqtNormalizedData));
274
-
return;
275
-
}
276
-
277
-
// Profiling variables
278
-
static double totalFftTime = 0.0;
279
-
static double totalKernelTime = 0.0;
280
-
static int profileCount = 0;
281
-
282
-
// Perform 16384-point FFT with timing
283
-
clock_t fftStart = clock();
284
-
kiss_fftr(cqtFftCfg, cqtAudioBuffer, cqtFftOutput);
285
-
clock_t fftEnd = clock();
286
-
287
-
// Extract real and imaginary components for kernel application
288
-
float fftReal[CQT_FFT_SIZE/2 + 1];
289
-
float fftImag[CQT_FFT_SIZE/2 + 1];
290
-
291
-
for (int i = 0; i <= CQT_FFT_SIZE/2; i++)
292
-
{
293
-
fftReal[i] = cqtFftOutput[i].r;
294
-
fftImag[i] = cqtFftOutput[i].i;
295
-
}
296
-
297
-
// Apply CQT kernels with timing
298
-
clock_t kernelStart = clock();
299
-
CQT_ApplyKernels(fftReal, fftImag);
300
-
clock_t kernelEnd = clock();
301
-
302
-
// Calculate times in milliseconds
303
-
double fftTime = (double)(fftEnd - fftStart) / CLOCKS_PER_SEC * 1000.0;
304
-
double kernelTime = (double)(kernelEnd - kernelStart) / CLOCKS_PER_SEC * 1000.0;
305
-
306
-
totalFftTime += fftTime;
307
-
totalKernelTime += kernelTime;
308
-
profileCount++;
309
-
310
-
// Print profiling info every 60 frames (~1 second)
311
-
if (profileCount % 60 == 0)
312
-
{
313
-
printf("CQT Performance (16K FFT):\n");
314
-
printf(" FFT avg: %.3fms\n", totalFftTime / profileCount);
315
-
printf(" Kernels avg: %.3fms\n", totalKernelTime / profileCount);
316
-
printf(" Total avg: %.3fms\n", (totalFftTime + totalKernelTime) / profileCount);
317
-
printf(" Samples: %d\n\n", profileCount);
318
-
}
319
-
320
-
#ifdef CQT_DEBUG
321
-
// Reduced debug output - CQT is working correctly now
322
-
static int debugCounter = 0;
323
-
if (++debugCounter % 300 == 0) // Print once every 5 seconds
324
-
{
325
-
// Find peak CQT bin
326
-
int peakBin = 0;
327
-
float peakVal = 0.0f;
328
-
for (int i = 0; i < CQT_BINS; i++)
329
-
{
330
-
if (cqtData[i] > peakVal)
331
-
{
332
-
peakVal = cqtData[i];
333
-
peakBin = i;
334
-
}
335
-
}
336
-
337
-
if (peakVal > 1.0f)
338
-
{
339
-
float peakFreq = CQT_MIN_FREQ * pow(2.0f, peakBin / 12.0f);
340
-
printf("CQT: Peak at bin %d (%.1f Hz) with magnitude %.1f\n",
341
-
peakBin, peakFreq, peakVal);
342
-
}
343
-
}
344
-
#endif
345
-
346
-
// Apply spectral whitening if enabled
347
-
#if CQT_SPECTRAL_WHITENING_ENABLED
348
-
for (int i = 0; i < CQT_BINS; i++)
349
-
{
350
-
// Update running average for this bin
351
-
cqtBinAverages[i] = cqtBinAverages[i] * CQT_WHITENING_DECAY +
352
-
cqtData[i] * (1.0f - CQT_WHITENING_DECAY);
353
-
354
-
// Ensure average doesn't go below floor
355
-
if (cqtBinAverages[i] < CQT_WHITENING_FLOOR)
356
-
cqtBinAverages[i] = CQT_WHITENING_FLOOR;
357
-
358
-
// Apply whitening by dividing by the average
359
-
cqtWhitenedData[i] = cqtData[i] / cqtBinAverages[i];
360
-
361
-
// Check for NaN or Inf
362
-
if (!isfinite(cqtWhitenedData[i]))
363
-
cqtWhitenedData[i] = 0.0f;
364
-
}
365
-
366
-
// Apply smoothing to whitened data
367
-
for (int i = 0; i < CQT_BINS; i++)
368
-
{
369
-
cqtSmoothingData[i] = cqtSmoothingData[i] * CQT_SMOOTHING_FACTOR +
370
-
cqtWhitenedData[i] * (1.0f - CQT_SMOOTHING_FACTOR);
371
-
}
372
-
#else
373
-
// Apply smoothing to raw data (spectral whitening disabled)
374
-
for (int i = 0; i < CQT_BINS; i++)
375
-
{
376
-
cqtSmoothingData[i] = cqtSmoothingData[i] * CQT_SMOOTHING_FACTOR +
377
-
cqtData[i] * (1.0f - CQT_SMOOTHING_FACTOR);
378
-
}
379
-
#endif
380
-
381
-
// Find peak for normalization
382
-
float currentPeak = 0.0f;
383
-
for (int i = 0; i < CQT_BINS; i++)
384
-
{
385
-
if (cqtSmoothingData[i] > currentPeak)
386
-
currentPeak = cqtSmoothingData[i];
387
-
}
388
-
389
-
// Initialize peak value if needed
390
-
if (cqtPeakSmoothValue <= 0.0f)
391
-
cqtPeakSmoothValue = 0.1f;
392
-
393
-
// Smooth peak value
394
-
if (currentPeak > cqtPeakSmoothValue)
395
-
cqtPeakSmoothValue = currentPeak;
396
-
else
397
-
cqtPeakSmoothValue = cqtPeakSmoothValue * 0.99f + currentPeak * 0.01f;
398
-
399
-
// Ensure peak value doesn't go too low
400
-
if (cqtPeakSmoothValue < 0.0001f)
401
-
cqtPeakSmoothValue = 0.0001f;
402
-
403
-
// Normalize data
404
-
float normalizer = 1.0f / cqtPeakSmoothValue;
405
-
for (int i = 0; i < CQT_BINS; i++)
406
-
{
407
-
cqtNormalizedData[i] = cqtSmoothingData[i] * normalizer;
408
-
if (cqtNormalizedData[i] > 1.0f)
409
-
cqtNormalizedData[i] = 1.0f;
410
-
411
-
// Final NaN check
412
-
if (!isfinite(cqtNormalizedData[i]))
413
-
cqtNormalizedData[i] = 0.0f;
414
-
}
415
-
}
416
-
417
-
418
-
// Close CQT processing and free resources
419
-
void CQT_Close(void)
420
-
{
421
-
if (cqtFftCfg)
422
-
{
423
-
free(cqtFftCfg);
424
-
cqtFftCfg = NULL;
425
-
}
426
-
427
-
if (cqtAudioBuffer)
428
-
{
429
-
free(cqtAudioBuffer);
430
-
cqtAudioBuffer = NULL;
431
-
}
432
-
433
-
if (cqtFftOutput)
434
-
{
435
-
free(cqtFftOutput);
436
-
cqtFftOutput = NULL;
437
-
}
438
-
439
-
// Clean up kernels
440
-
CQT_Cleanup();
441
-
}
-16
src/ext/cqt.h
-16
src/ext/cqt.h
···
1
-
#pragma once
2
-
3
-
#include <stdbool.h>
4
-
5
-
// Initialize CQT processing
6
-
// Returns true on success, false on failure
7
-
bool CQT_Open(void);
8
-
9
-
// Process CQT from audio buffer (uses shared audio capture buffer)
10
-
void CQT_ProcessAudio(void);
11
-
12
-
// Close CQT processing and free resources
13
-
void CQT_Close(void);
14
-
15
-
// Apply CQT kernels to FFT output
16
-
void CQT_ApplyKernels(const float* fftReal, const float* fftImag);
+17
-17
src/ext/cqt_kernel.c
src/ext/vqt_kernel.c
+17
-17
src/ext/cqt_kernel.c
src/ext/vqt_kernel.c
···
1
-
#include "cqt_kernel.h"
2
-
#include "../cqtdata.h"
1
+
#include "vqt_kernel.h"
2
+
#include "../vqtdata.h"
3
3
#include <math.h>
4
4
#include <stdlib.h>
5
5
#include <string.h>
···
10
10
#endif
11
11
12
12
// Generate logarithmically spaced center frequencies for musical notes
13
-
void CQT_GenerateCenterFrequencies(float* frequencies, int numBins, float minFreq, float maxFreq)
13
+
void VQT_GenerateCenterFrequencies(float* frequencies, int numBins, float minFreq, float maxFreq)
14
14
{
15
-
// For musical CQT with 12 bins per octave, we want equal temperament spacing
15
+
// For musical VQT with 12 bins per octave, we want equal temperament spacing
16
16
// Each semitone is a factor of 2^(1/12) ≈ 1.0594631
17
17
const float semitone = pow(2.0f, 1.0f / 12.0f);
18
18
···
35
35
}
36
36
37
37
// Calculate Q factor for constant-Q transform
38
-
float CQT_CalculateQ(int binsPerOctave)
38
+
float VQT_CalculateQ(int binsPerOctave)
39
39
{
40
40
// Q = 1 / (2^(1/binsPerOctave) - 1)
41
41
// For 12 bins/octave, Q ≈ 17.0
···
53
53
else if (centerFreq < 50.0f) return 14.5f; // 40-50 Hz: Near ideal
54
54
else if (centerFreq < 65.0f) return 16.0f; // 50-65 Hz: Almost full Q
55
55
else if (centerFreq < 80.0f) return 17.0f; // 65-80 Hz: Full standard Q
56
-
else if (centerFreq < 160.0f) return 17.0f; // 80-160 Hz: Standard CQT
56
+
else if (centerFreq < 160.0f) return 17.0f; // 80-160 Hz: Standard VQT
57
57
else if (centerFreq < 320.0f) return 15.0f; // 160-320 Hz: Slightly wider
58
58
else if (centerFreq < 640.0f) return 13.0f; // 320-640 Hz: Smoother
59
59
else return 11.0f; // 640+ Hz: Very smooth
···
95
95
}
96
96
}
97
97
98
-
// Generate a single CQT kernel
98
+
// Generate a single VQT kernel
99
99
static bool generateSingleKernel(
100
-
CqtKernel* kernel,
100
+
VqtKernel* kernel,
101
101
kiss_fftr_cfg fftCfg,
102
102
int fftSize,
103
103
float centerFreq,
104
104
float minFreq,
105
105
float sampleRate,
106
-
CqtWindowType windowType,
106
+
VqtWindowType windowType,
107
107
float sparsityThreshold)
108
108
{
109
109
// Use variable Q optimized for 8K FFT
···
124
124
windowLength = fftSize;
125
125
// Calculate effective Q after truncation
126
126
float effectiveQ = windowLength * centerFreq / sampleRate;
127
-
#ifdef CQT_DEBUG
127
+
#ifdef VQT_DEBUG
128
128
printf("Freq %.1f Hz: Q designed=%.1f, window=%d, Q effective=%.1f (truncated)\n",
129
129
centerFreq, Q, windowLength, effectiveQ);
130
130
#endif
···
158
158
159
159
switch (windowType)
160
160
{
161
-
case CQT_WINDOW_HAMMING:
161
+
case VQT_WINDOW_HAMMING:
162
162
generateHammingWindow(tempWindow, windowLength);
163
163
break;
164
-
case CQT_WINDOW_GAUSSIAN:
164
+
case VQT_WINDOW_GAUSSIAN:
165
165
generateGaussianWindow(tempWindow, windowLength);
166
166
break;
167
167
}
···
235
235
}
236
236
}
237
237
238
-
#ifdef CQT_DEBUG
238
+
#ifdef VQT_DEBUG
239
239
// Debug output for specific frequencies
240
240
if (fabs(centerFreq - 110.0f) < 1.0f || fabs(centerFreq - 440.0f) < 1.0f)
241
241
{
···
249
249
return true;
250
250
}
251
251
252
-
// Generate CQT kernels for all bins
253
-
bool CQT_GenerateKernels(CqtKernel* kernels, const CqtKernelConfig* config)
252
+
// Generate VQT kernels for all bins
253
+
bool VQT_GenerateKernels(VqtKernel* kernels, const VqtKernelConfig* config)
254
254
{
255
255
// Generate center frequencies
256
256
float* centerFreqs = (float*)malloc(config->numBins * sizeof(float));
257
257
if (!centerFreqs) return false;
258
258
259
-
CQT_GenerateCenterFrequencies(centerFreqs, config->numBins,
259
+
VQT_GenerateCenterFrequencies(centerFreqs, config->numBins,
260
260
config->minFreq, config->maxFreq);
261
261
262
262
// Create FFT configuration
···
267
267
return false;
268
268
}
269
269
270
-
// Generate kernel for each CQT bin
270
+
// Generate kernel for each VQT bin
271
271
bool success = true;
272
272
for (int i = 0; i < config->numBins; i++)
273
273
{
-31
src/ext/cqt_kernel.h
-31
src/ext/cqt_kernel.h
···
1
-
#pragma once
2
-
3
-
#include <stdbool.h>
4
-
#include "../cqtdata.h"
5
-
6
-
// Window types for CQT kernels
7
-
typedef enum {
8
-
CQT_WINDOW_HAMMING = 0,
9
-
CQT_WINDOW_GAUSSIAN
10
-
} CqtWindowType;
11
-
12
-
// CQT kernel configuration
13
-
typedef struct {
14
-
int fftSize; // FFT size (4096)
15
-
int numBins; // Number of CQT bins (120)
16
-
float minFreq; // Minimum frequency (20 Hz)
17
-
float maxFreq; // Maximum frequency (20480 Hz)
18
-
float sampleRate; // Sample rate (44100 Hz)
19
-
CqtWindowType windowType; // Window function type
20
-
float sparsityThreshold; // Threshold for sparse matrix (0.01)
21
-
} CqtKernelConfig;
22
-
23
-
// Generate CQT kernels for all bins
24
-
// Returns true on success, false on failure
25
-
bool CQT_GenerateKernels(CqtKernel* kernels, const CqtKernelConfig* config);
26
-
27
-
// Calculate Q factor for given parameters
28
-
float CQT_CalculateQ(int binsPerOctave);
29
-
30
-
// Generate center frequencies for CQT bins
31
-
void CQT_GenerateCenterFrequencies(float* frequencies, int numBins, float minFreq, float maxFreq);
+5
-5
src/ext/fft.c
+5
-5
src/ext/fft.c
···
19
19
kiss_fftr_cfg fftcfg;
20
20
ma_context context;
21
21
ma_device captureDevice;
22
-
// Include CQT header to get CQT_FFT_SIZE
23
-
#include "../cqtdata.h"
22
+
// Include VQT header to get VQT_FFT_SIZE
23
+
#include "../vqtdata.h"
24
24
25
-
// Shared audio buffer - must be large enough for both FFT and CQT
26
-
// FFT needs FFT_SIZE * 2 (2048) samples, CQT needs CQT_FFT_SIZE samples
27
-
#define AUDIO_BUFFER_SIZE (CQT_FFT_SIZE > (FFT_SIZE * 2) ? CQT_FFT_SIZE : (FFT_SIZE * 2))
25
+
// Shared audio buffer - must be large enough for both FFT and VQT
26
+
// FFT needs FFT_SIZE * 2 (2048) samples, VQT needs VQT_FFT_SIZE samples
27
+
#define AUDIO_BUFFER_SIZE (VQT_FFT_SIZE > (FFT_SIZE * 2) ? VQT_FFT_SIZE : (FFT_SIZE * 2))
28
28
float sampleBuf[AUDIO_BUFFER_SIZE];
29
29
30
30
void miniaudioLogCallback(void* userData, ma_uint32 level, const char* message)
+1
-1
src/ext/fft.h
+1
-1
src/ext/fft.h
+441
src/ext/vqt.c
+441
src/ext/vqt.c
···
1
+
#include "vqt.h"
2
+
#include "vqt_kernel.h"
3
+
#include "../vqtdata.h"
4
+
#include "../fftdata.h"
5
+
#include "fft.h"
6
+
#include "kiss_fftr.h"
7
+
#include <math.h>
8
+
#include <string.h>
9
+
#include <stdbool.h>
10
+
#include <stdio.h>
11
+
#include <time.h>
12
+
13
+
#define VQT_DEBUG
14
+
15
+
// FFT configuration for VQT
16
+
static kiss_fftr_cfg vqtFftCfg = NULL;
17
+
static float* vqtAudioBuffer = NULL;
18
+
static kiss_fft_cpx* vqtFftOutput = NULL;
19
+
20
+
// Benchmark different FFT sizes
21
+
static void VQT_BenchmarkFFT(void)
22
+
{
23
+
printf("\nVQT FFT Benchmark on this CPU:\n");
24
+
printf("================================\n");
25
+
26
+
int sizes[] = {4096, 6144, 8192, 12288, 16384, 24576, 32768};
27
+
int numSizes = 7;
28
+
29
+
for (int s = 0; s < numSizes; s++)
30
+
{
31
+
int fftSize = sizes[s];
32
+
33
+
// Allocate buffers
34
+
float* testInput = (float*)calloc(fftSize, sizeof(float));
35
+
kiss_fft_cpx* testOutput = (kiss_fft_cpx*)malloc((fftSize/2 + 1) * sizeof(kiss_fft_cpx));
36
+
kiss_fftr_cfg testCfg = kiss_fftr_alloc(fftSize, 0, NULL, NULL);
37
+
38
+
if (!testInput || !testOutput || !testCfg)
39
+
{
40
+
printf("Failed to allocate for size %d\n", fftSize);
41
+
continue;
42
+
}
43
+
44
+
// Fill with test signal
45
+
for (int i = 0; i < fftSize; i++)
46
+
{
47
+
testInput[i] = sin(2.0 * M_PI * 440.0 * i / 44100.0);
48
+
}
49
+
50
+
// Warm up
51
+
for (int i = 0; i < 10; i++)
52
+
{
53
+
kiss_fftr(testCfg, testInput, testOutput);
54
+
}
55
+
56
+
// Time 100 iterations
57
+
clock_t start = clock();
58
+
for (int i = 0; i < 100; i++)
59
+
{
60
+
kiss_fftr(testCfg, testInput, testOutput);
61
+
}
62
+
clock_t end = clock();
63
+
64
+
double avgTime = (double)(end - start) / CLOCKS_PER_SEC * 1000.0 / 100.0;
65
+
printf("%5d-point FFT: %.3f ms\n", fftSize, avgTime);
66
+
67
+
// Cleanup
68
+
free(testInput);
69
+
free(testOutput);
70
+
free(testCfg);
71
+
}
72
+
73
+
printf("================================\n\n");
74
+
}
75
+
76
+
// Initialize VQT processing
77
+
bool VQT_Open(void)
78
+
{
79
+
// Initialize data structures
80
+
VQT_Init();
81
+
82
+
// Allocate FFT buffers
83
+
vqtAudioBuffer = (float*)malloc(VQT_FFT_SIZE * sizeof(float));
84
+
vqtFftOutput = (kiss_fft_cpx*)malloc((VQT_FFT_SIZE/2 + 1) * sizeof(kiss_fft_cpx));
85
+
86
+
if (!vqtAudioBuffer || !vqtFftOutput)
87
+
{
88
+
VQT_Close();
89
+
return false;
90
+
}
91
+
92
+
// Create FFT configuration
93
+
vqtFftCfg = kiss_fftr_alloc(VQT_FFT_SIZE, 0, NULL, NULL);
94
+
if (!vqtFftCfg)
95
+
{
96
+
VQT_Close();
97
+
return false;
98
+
}
99
+
100
+
// Configure VQT kernels
101
+
VqtKernelConfig config = {
102
+
.fftSize = VQT_FFT_SIZE,
103
+
.numBins = VQT_BINS,
104
+
.minFreq = VQT_MIN_FREQ,
105
+
.maxFreq = VQT_MAX_FREQ,
106
+
.sampleRate = 44100.0f, // Standard TIC-80 sample rate
107
+
.windowType = VQT_WINDOW_HAMMING,
108
+
.sparsityThreshold = VQT_SPARSITY_THRESHOLD
109
+
};
110
+
111
+
// Generate kernels
112
+
if (!VQT_GenerateKernels(vqtKernels, &config))
113
+
{
114
+
VQT_Close();
115
+
return false;
116
+
}
117
+
118
+
// Run FFT benchmark on first initialization
119
+
static bool benchmarkRun = false;
120
+
if (!benchmarkRun)
121
+
{
122
+
VQT_BenchmarkFFT();
123
+
benchmarkRun = true;
124
+
}
125
+
126
+
// Debug: Print variable Q values across frequency spectrum
127
+
#ifdef VQT_DEBUG
128
+
float centerFreqs[VQT_BINS];
129
+
VQT_GenerateCenterFrequencies(centerFreqs, VQT_BINS, VQT_MIN_FREQ, VQT_MAX_FREQ);
130
+
131
+
printf("\nVQT Variable-Q Implementation (8K FFT Optimized):\n");
132
+
printf("================================================\n");
133
+
134
+
// Show Q values for key frequency ranges
135
+
float testFreqs[] = {20, 25, 30, 40, 50, 65, 80, 120, 160, 240, 320, 440, 640, 1000, 2000, 4000};
136
+
printf("Frequency | Design Q | Window Length | Effective Q | Resolution\n");
137
+
printf("----------|----------|---------------|-------------|------------\n");
138
+
139
+
for (int i = 0; i < 16; i++)
140
+
{
141
+
float freq = testFreqs[i];
142
+
// Find closest VQT bin
143
+
int closestBin = 0;
144
+
float minDiff = fabs(centerFreqs[0] - freq);
145
+
for (int j = 1; j < VQT_BINS; j++)
146
+
{
147
+
float diff = fabs(centerFreqs[j] - freq);
148
+
if (diff < minDiff)
149
+
{
150
+
minDiff = diff;
151
+
closestBin = j;
152
+
}
153
+
}
154
+
155
+
// Calculate Q values using 8K-optimized function
156
+
float designQ;
157
+
if (freq < 25.0f) designQ = 7.4f;
158
+
else if (freq < 30.0f) designQ = 9.2f;
159
+
else if (freq < 40.0f) designQ = 11.5f;
160
+
else if (freq < 50.0f) designQ = 14.5f;
161
+
else if (freq < 65.0f) designQ = 16.0f;
162
+
else if (freq < 80.0f) designQ = 17.0f;
163
+
else if (freq < 160.0f) designQ = 17.0f;
164
+
else if (freq < 320.0f) designQ = 15.0f;
165
+
else if (freq < 640.0f) designQ = 13.0f;
166
+
else designQ = 11.0f;
167
+
168
+
int windowLength = (int)(designQ * 44100.0f / freq);
169
+
if (windowLength > VQT_FFT_SIZE) windowLength = VQT_FFT_SIZE;
170
+
float effectiveQ = windowLength * freq / 44100.0f;
171
+
float bandwidth = freq / effectiveQ;
172
+
173
+
printf("%7.0f Hz | %7.1f | %13d | %11.1f | %7.1f Hz\n",
174
+
freq, designQ, windowLength, effectiveQ, bandwidth);
175
+
}
176
+
177
+
printf("\nFirst 10 VQT bins:\n");
178
+
for (int i = 0; i < 10 && i < VQT_BINS; i++)
179
+
{
180
+
int expectedBin = (int)(centerFreqs[i] * VQT_FFT_SIZE / 44100.0f);
181
+
printf(" Bin %d: %.2f Hz -> FFT bin %d\n", i, centerFreqs[i], expectedBin);
182
+
}
183
+
#endif
184
+
185
+
return true;
186
+
}
187
+
188
+
// Apply VQT kernels to FFT output
189
+
void VQT_ApplyKernels(const float* fftReal, const float* fftImag)
190
+
{
191
+
// Clear VQT output
192
+
memset(vqtData, 0, sizeof(vqtData));
193
+
194
+
// Apply each kernel to compute VQT bins
195
+
for (int bin = 0; bin < VQT_BINS; bin++)
196
+
{
197
+
VqtKernel* kernel = &vqtKernels[bin];
198
+
199
+
// Check if kernel is valid
200
+
if (!kernel->real || !kernel->imag || !kernel->indices || kernel->length == 0)
201
+
{
202
+
vqtData[bin] = 0.0f;
203
+
continue;
204
+
}
205
+
206
+
float real = 0.0f;
207
+
float imag = 0.0f;
208
+
209
+
// Sparse matrix multiplication
210
+
for (int k = 0; k < kernel->length; k++)
211
+
{
212
+
int idx = kernel->indices[k];
213
+
// Ensure index is within bounds
214
+
if (idx < 0 || idx >= VQT_FFT_SIZE/2 + 1)
215
+
continue;
216
+
217
+
// Complex multiplication: (a + bi) * (c + di) = (ac - bd) + (ad + bc)i
218
+
real += fftReal[idx] * kernel->real[k] - fftImag[idx] * kernel->imag[k];
219
+
imag += fftReal[idx] * kernel->imag[k] + fftImag[idx] * kernel->real[k];
220
+
}
221
+
222
+
// Calculate magnitude with gain boost
223
+
vqtData[bin] = sqrt(real * real + imag * imag) * 2.0f; // Match FFT gain factor
224
+
225
+
// Check for NaN or Inf
226
+
if (!isfinite(vqtData[bin]))
227
+
vqtData[bin] = 0.0f;
228
+
}
229
+
}
230
+
231
+
// Process VQT from audio data
232
+
void VQT_ProcessAudio(void)
233
+
{
234
+
if (!vqtFftCfg || !vqtEnabled) return;
235
+
236
+
// Check if kernels are initialized
237
+
bool kernelsValid = false;
238
+
for (int i = 0; i < VQT_BINS; i++)
239
+
{
240
+
if (vqtKernels[i].real && vqtKernels[i].length > 0)
241
+
{
242
+
kernelsValid = true;
243
+
break;
244
+
}
245
+
}
246
+
247
+
if (!kernelsValid)
248
+
{
249
+
// Kernels not initialized, set all output to zero
250
+
memset(vqtData, 0, sizeof(vqtData));
251
+
memset(vqtSmoothingData, 0, sizeof(vqtSmoothingData));
252
+
memset(vqtNormalizedData, 0, sizeof(vqtNormalizedData));
253
+
return;
254
+
}
255
+
256
+
// Copy audio data from the shared buffer
257
+
// sampleBuf is defined in fft.c as extern
258
+
extern float sampleBuf[];
259
+
memcpy(vqtAudioBuffer, sampleBuf, VQT_FFT_SIZE * sizeof(float));
260
+
261
+
// Check if we have any audio data
262
+
float audioSum = 0.0f;
263
+
for (int i = 0; i < VQT_FFT_SIZE; i++)
264
+
{
265
+
audioSum += fabs(vqtAudioBuffer[i]);
266
+
}
267
+
268
+
if (audioSum < 0.0001f)
269
+
{
270
+
// No audio data, set output to zero
271
+
memset(vqtData, 0, sizeof(vqtData));
272
+
memset(vqtSmoothingData, 0, sizeof(vqtSmoothingData));
273
+
memset(vqtNormalizedData, 0, sizeof(vqtNormalizedData));
274
+
return;
275
+
}
276
+
277
+
// Profiling variables
278
+
static double totalFftTime = 0.0;
279
+
static double totalKernelTime = 0.0;
280
+
static int profileCount = 0;
281
+
282
+
// Perform 16384-point FFT with timing
283
+
clock_t fftStart = clock();
284
+
kiss_fftr(vqtFftCfg, vqtAudioBuffer, vqtFftOutput);
285
+
clock_t fftEnd = clock();
286
+
287
+
// Extract real and imaginary components for kernel application
288
+
float fftReal[VQT_FFT_SIZE/2 + 1];
289
+
float fftImag[VQT_FFT_SIZE/2 + 1];
290
+
291
+
for (int i = 0; i <= VQT_FFT_SIZE/2; i++)
292
+
{
293
+
fftReal[i] = vqtFftOutput[i].r;
294
+
fftImag[i] = vqtFftOutput[i].i;
295
+
}
296
+
297
+
// Apply VQT kernels with timing
298
+
clock_t kernelStart = clock();
299
+
VQT_ApplyKernels(fftReal, fftImag);
300
+
clock_t kernelEnd = clock();
301
+
302
+
// Calculate times in milliseconds
303
+
double fftTime = (double)(fftEnd - fftStart) / CLOCKS_PER_SEC * 1000.0;
304
+
double kernelTime = (double)(kernelEnd - kernelStart) / CLOCKS_PER_SEC * 1000.0;
305
+
306
+
totalFftTime += fftTime;
307
+
totalKernelTime += kernelTime;
308
+
profileCount++;
309
+
310
+
// Print profiling info every 60 frames (~1 second)
311
+
if (profileCount % 60 == 0)
312
+
{
313
+
printf("VQT Performance (16K FFT):\n");
314
+
printf(" FFT avg: %.3fms\n", totalFftTime / profileCount);
315
+
printf(" Kernels avg: %.3fms\n", totalKernelTime / profileCount);
316
+
printf(" Total avg: %.3fms\n", (totalFftTime + totalKernelTime) / profileCount);
317
+
printf(" Samples: %d\n\n", profileCount);
318
+
}
319
+
320
+
#ifdef VQT_DEBUG
321
+
// Reduced debug output - VQT is working correctly now
322
+
static int debugCounter = 0;
323
+
if (++debugCounter % 300 == 0) // Print once every 5 seconds
324
+
{
325
+
// Find peak VQT bin
326
+
int peakBin = 0;
327
+
float peakVal = 0.0f;
328
+
for (int i = 0; i < VQT_BINS; i++)
329
+
{
330
+
if (vqtData[i] > peakVal)
331
+
{
332
+
peakVal = vqtData[i];
333
+
peakBin = i;
334
+
}
335
+
}
336
+
337
+
if (peakVal > 1.0f)
338
+
{
339
+
float peakFreq = VQT_MIN_FREQ * pow(2.0f, peakBin / 12.0f);
340
+
printf("VQT: Peak at bin %d (%.1f Hz) with magnitude %.1f\n",
341
+
peakBin, peakFreq, peakVal);
342
+
}
343
+
}
344
+
#endif
345
+
346
+
// Apply spectral whitening if enabled
347
+
#if VQT_SPECTRAL_WHITENING_ENABLED
348
+
for (int i = 0; i < VQT_BINS; i++)
349
+
{
350
+
// Update running average for this bin
351
+
vqtBinAverages[i] = vqtBinAverages[i] * VQT_WHITENING_DECAY +
352
+
vqtData[i] * (1.0f - VQT_WHITENING_DECAY);
353
+
354
+
// Ensure average doesn't go below floor
355
+
if (vqtBinAverages[i] < VQT_WHITENING_FLOOR)
356
+
vqtBinAverages[i] = VQT_WHITENING_FLOOR;
357
+
358
+
// Apply whitening by dividing by the average
359
+
vqtWhitenedData[i] = vqtData[i] / vqtBinAverages[i];
360
+
361
+
// Check for NaN or Inf
362
+
if (!isfinite(vqtWhitenedData[i]))
363
+
vqtWhitenedData[i] = 0.0f;
364
+
}
365
+
366
+
// Apply smoothing to whitened data
367
+
for (int i = 0; i < VQT_BINS; i++)
368
+
{
369
+
vqtSmoothingData[i] = vqtSmoothingData[i] * VQT_SMOOTHING_FACTOR +
370
+
vqtWhitenedData[i] * (1.0f - VQT_SMOOTHING_FACTOR);
371
+
}
372
+
#else
373
+
// Apply smoothing to raw data (spectral whitening disabled)
374
+
for (int i = 0; i < VQT_BINS; i++)
375
+
{
376
+
vqtSmoothingData[i] = vqtSmoothingData[i] * VQT_SMOOTHING_FACTOR +
377
+
vqtData[i] * (1.0f - VQT_SMOOTHING_FACTOR);
378
+
}
379
+
#endif
380
+
381
+
// Find peak for normalization
382
+
float currentPeak = 0.0f;
383
+
for (int i = 0; i < VQT_BINS; i++)
384
+
{
385
+
if (vqtSmoothingData[i] > currentPeak)
386
+
currentPeak = vqtSmoothingData[i];
387
+
}
388
+
389
+
// Initialize peak value if needed
390
+
if (vqtPeakSmoothValue <= 0.0f)
391
+
vqtPeakSmoothValue = 0.1f;
392
+
393
+
// Smooth peak value
394
+
if (currentPeak > vqtPeakSmoothValue)
395
+
vqtPeakSmoothValue = currentPeak;
396
+
else
397
+
vqtPeakSmoothValue = vqtPeakSmoothValue * 0.99f + currentPeak * 0.01f;
398
+
399
+
// Ensure peak value doesn't go too low
400
+
if (vqtPeakSmoothValue < 0.0001f)
401
+
vqtPeakSmoothValue = 0.0001f;
402
+
403
+
// Normalize data
404
+
float normalizer = 1.0f / vqtPeakSmoothValue;
405
+
for (int i = 0; i < VQT_BINS; i++)
406
+
{
407
+
vqtNormalizedData[i] = vqtSmoothingData[i] * normalizer;
408
+
if (vqtNormalizedData[i] > 1.0f)
409
+
vqtNormalizedData[i] = 1.0f;
410
+
411
+
// Final NaN check
412
+
if (!isfinite(vqtNormalizedData[i]))
413
+
vqtNormalizedData[i] = 0.0f;
414
+
}
415
+
}
416
+
417
+
418
+
// Close VQT processing and free resources
419
+
void VQT_Close(void)
420
+
{
421
+
if (vqtFftCfg)
422
+
{
423
+
free(vqtFftCfg);
424
+
vqtFftCfg = NULL;
425
+
}
426
+
427
+
if (vqtAudioBuffer)
428
+
{
429
+
free(vqtAudioBuffer);
430
+
vqtAudioBuffer = NULL;
431
+
}
432
+
433
+
if (vqtFftOutput)
434
+
{
435
+
free(vqtFftOutput);
436
+
vqtFftOutput = NULL;
437
+
}
438
+
439
+
// Clean up kernels
440
+
VQT_Cleanup();
441
+
}
+16
src/ext/vqt.h
+16
src/ext/vqt.h
···
1
+
#pragma once
2
+
3
+
#include <stdbool.h>
4
+
5
+
// Initialize VQT processing
6
+
// Returns true on success, false on failure
7
+
bool VQT_Open(void);
8
+
9
+
// Process VQT from audio buffer (uses shared audio capture buffer)
10
+
void VQT_ProcessAudio(void);
11
+
12
+
// Close VQT processing and free resources
13
+
void VQT_Close(void);
14
+
15
+
// Apply VQT kernels to FFT output
16
+
void VQT_ApplyKernels(const float* fftReal, const float* fftImag);
+31
src/ext/vqt_kernel.h
+31
src/ext/vqt_kernel.h
···
1
+
#pragma once
2
+
3
+
#include <stdbool.h>
4
+
#include "../vqtdata.h"
5
+
6
+
// Window types for VQT kernels
7
+
typedef enum {
8
+
VQT_WINDOW_HAMMING = 0,
9
+
VQT_WINDOW_GAUSSIAN
10
+
} VqtWindowType;
11
+
12
+
// VQT kernel configuration
13
+
typedef struct {
14
+
int fftSize; // FFT size (4096)
15
+
int numBins; // Number of VQT bins (120)
16
+
float minFreq; // Minimum frequency (20 Hz)
17
+
float maxFreq; // Maximum frequency (20480 Hz)
18
+
float sampleRate; // Sample rate (44100 Hz)
19
+
VqtWindowType windowType; // Window function type
20
+
float sparsityThreshold; // Threshold for sparse matrix (0.01)
21
+
} VqtKernelConfig;
22
+
23
+
// Generate VQT kernels for all bins
24
+
// Returns true on success, false on failure
25
+
bool VQT_GenerateKernels(VqtKernel* kernels, const VqtKernelConfig* config);
26
+
27
+
// Calculate Q factor for given parameters
28
+
float VQT_CalculateQ(int binsPerOctave);
29
+
30
+
// Generate center frequencies for VQT bins
31
+
void VQT_GenerateCenterFrequencies(float* frequencies, int numBins, float minFreq, float maxFreq);
+4
-4
src/system/sdl/main.c
+4
-4
src/system/sdl/main.c
···
24
24
#include "tools.h"
25
25
26
26
#include "ext/fft.h"
27
-
#include "ext/cqt.h"
27
+
#include "ext/vqt.h"
28
28
#include <stdlib.h>
29
29
#include <stdio.h>
30
30
#include <string.h>
···
324
324
if (studio_config(platform.studio)->fft)
325
325
{
326
326
FFT_Open(studio_config(platform.studio)->fftcaptureplaybackdevices, studio_config(platform.studio)->fftdevice);
327
-
// Initialize CQT when FFT is enabled
328
-
CQT_Open();
327
+
// Initialize VQT when FFT is enabled
328
+
VQT_Open();
329
329
}
330
330
331
331
platform.audio.device = SDL_OpenAudioDevice(NULL, 0, &want, &platform.audio.spec, 0);
···
2006
2006
if (studio_config(platform.studio)->fft)
2007
2007
{
2008
2008
FFT_Close();
2009
-
CQT_Close();
2009
+
VQT_Close();
2010
2010
}
2011
2011
}
2012
2012
+71
src/vqtdata.c
+71
src/vqtdata.c
···
1
+
#include "vqtdata.h"
2
+
#include <string.h>
3
+
#include <stdlib.h>
4
+
5
+
// Global VQT data arrays
6
+
float vqtData[VQT_BINS];
7
+
float vqtSmoothingData[VQT_BINS];
8
+
float vqtNormalizedData[VQT_BINS];
9
+
10
+
// Peak tracking for auto-gain
11
+
float vqtPeakValue = 1.0f;
12
+
float vqtPeakSmoothValue = 1.0f;
13
+
14
+
// Enable flag (tied to fftEnabled initially)
15
+
bool vqtEnabled = false;
16
+
17
+
// Array of kernels, one per VQT bin
18
+
VqtKernel vqtKernels[VQT_BINS];
19
+
20
+
// Spectral whitening data
21
+
float vqtBinAverages[VQT_BINS];
22
+
float vqtWhitenedData[VQT_BINS];
23
+
24
+
void VQT_Init(void)
25
+
{
26
+
// Zero all data arrays
27
+
memset(vqtData, 0, sizeof(vqtData));
28
+
memset(vqtSmoothingData, 0, sizeof(vqtSmoothingData));
29
+
memset(vqtNormalizedData, 0, sizeof(vqtNormalizedData));
30
+
31
+
// Initialize peak values
32
+
vqtPeakValue = 1.0f;
33
+
vqtPeakSmoothValue = 1.0f;
34
+
35
+
// Zero kernel pointers
36
+
memset(vqtKernels, 0, sizeof(vqtKernels));
37
+
38
+
// Initialize spectral whitening arrays
39
+
memset(vqtBinAverages, 0, sizeof(vqtBinAverages));
40
+
memset(vqtWhitenedData, 0, sizeof(vqtWhitenedData));
41
+
42
+
// Start with small initial averages to avoid divide-by-zero
43
+
for (int i = 0; i < VQT_BINS; i++)
44
+
{
45
+
vqtBinAverages[i] = VQT_WHITENING_FLOOR;
46
+
}
47
+
}
48
+
49
+
void VQT_Cleanup(void)
50
+
{
51
+
// Free all allocated kernel memory
52
+
for (int i = 0; i < VQT_BINS; i++)
53
+
{
54
+
if (vqtKernels[i].real)
55
+
{
56
+
free(vqtKernels[i].real);
57
+
vqtKernels[i].real = NULL;
58
+
}
59
+
if (vqtKernels[i].imag)
60
+
{
61
+
free(vqtKernels[i].imag);
62
+
vqtKernels[i].imag = NULL;
63
+
}
64
+
if (vqtKernels[i].indices)
65
+
{
66
+
free(vqtKernels[i].indices);
67
+
vqtKernels[i].indices = NULL;
68
+
}
69
+
vqtKernels[i].length = 0;
70
+
}
71
+
}
+66
src/vqtdata.h
+66
src/vqtdata.h
···
1
+
#pragma once
2
+
#include <stdbool.h>
3
+
4
+
#define VQT_BINS 120
5
+
#define VQT_OCTAVES 10
6
+
#define VQT_BINS_PER_OCTAVE 12
7
+
#define VQT_FFT_SIZE 8192 // 8K FFT - optimized variable-Q for responsive visualization, ~5.4 fps
8
+
9
+
// VQT frequency range
10
+
#define VQT_MIN_FREQ 20.0f // Sub-bass for electronic music
11
+
#define VQT_MAX_FREQ 20480.0f // Nearest note to 20kHz
12
+
13
+
// Smoothing parameters
14
+
#define VQT_SMOOTHING_FACTOR 0.3f // Reduced from 0.7f for more responsive display
15
+
#define VQT_SPARSITY_THRESHOLD 0.01f
16
+
17
+
// Variable-Q configuration (optimized for 8K FFT)
18
+
#define VQT_VARIABLE_Q_ENABLED 1
19
+
#define VQT_8K_OPTIMIZED 1 // Uses Q values that fit within 8K FFT
20
+
#define VQT_BASS_Q_MIN 7.4f // Minimum Q at 20 Hz (constrained by 8K)
21
+
#define VQT_BASS_Q_MAX 17.0f // Full Q achieved at 80+ Hz
22
+
#define VQT_TREBLE_Q_FACTOR 11.0f // Smoother for high frequencies
23
+
24
+
// Spectral whitening toggle (set to 0 to disable)
25
+
#define VQT_SPECTRAL_WHITENING_ENABLED 1
26
+
27
+
// Spectral whitening parameters
28
+
#define VQT_WHITENING_DECAY 0.99f // Running average decay (0.98-0.995 for 1-2 second adaptation)
29
+
#define VQT_WHITENING_FLOOR 0.001f // Minimum average to prevent divide-by-zero
30
+
31
+
// Raw VQT magnitude data
32
+
extern float vqtData[VQT_BINS];
33
+
34
+
// Smoothed VQT data for visual stability
35
+
extern float vqtSmoothingData[VQT_BINS];
36
+
37
+
// Normalized VQT data (0-1 range)
38
+
extern float vqtNormalizedData[VQT_BINS];
39
+
40
+
// Peak tracking for auto-gain
41
+
extern float vqtPeakValue;
42
+
extern float vqtPeakSmoothValue;
43
+
44
+
// Enable flag (tied to fftEnabled initially)
45
+
extern bool vqtEnabled;
46
+
47
+
// Spectral whitening data
48
+
extern float vqtBinAverages[VQT_BINS]; // Long-term running averages per bin
49
+
extern float vqtWhitenedData[VQT_BINS]; // Whitened VQT data
50
+
51
+
// Sparse kernel storage structures
52
+
typedef struct {
53
+
float* real; // Real parts of kernel (sparse)
54
+
float* imag; // Imaginary parts of kernel (sparse)
55
+
int* indices; // Non-zero FFT bin indices
56
+
int length; // Number of non-zero elements
57
+
} VqtKernel;
58
+
59
+
// Array of kernels, one per VQT bin
60
+
extern VqtKernel vqtKernels[VQT_BINS];
61
+
62
+
// Initialize VQT data structures
63
+
void VQT_Init(void);
64
+
65
+
// Free VQT kernel memory
66
+
void VQT_Cleanup(void);