alice.mosphere.at / TIC-80
this repo has no description
0
fork

Configure Feed

Select the types of activity you want to include in your feed.

GPT5-high pass 6: spectral whitening pt3 bindings etc.

alice 65911c9c 3a377d5c

+60 -14
+60 -14
CLAUDE.md
··· 129 129 - **Processing**: In `tic_core_tick` after FFT processing 130 130 131 131 #### Specifications 132 - - Frequency range: 20 Hz - 20480 Hz (10 octaves × 12 notes = 120 bins) 132 + - Frequency range: 19.445 Hz (D#0/Eb0) - 20480 Hz (10 octaves × 12 notes = 120 bins) 133 133 - FFT size: 8192 samples (configurable) 134 134 - Update rate: ~5.4 fps with 8K FFT 135 135 - Variable-Q implementation optimized for 8K FFT constraint 136 136 - Smoothing factor: 0.3 137 - - Spectral whitening: Enabled by default (toggle via `VQT_SPECTRAL_WHITENING_ENABLED`) 137 + - Spectral whitening: Provided via separate whitened VQT functions (see API) 138 138 139 139 #### Variable-Q Design (8K FFT Optimized) 140 140 | Frequency Range | Design Q | Effective Q | Resolution | ··· 152 152 value = vqt(bin) -- Get peak-normalized VQT magnitude for bin (0-119) 153 153 -- Note mapping: Bin = octave * 12 + note 154 154 -- Note: C=0, C#=1, D=2, D#=3, E=4, F=5, F#=6, G=7, G#=8, A=9, A#=10, B=11 155 + -- Smoothed: vqts(bin) 156 + 157 + -- Raw (non-normalized) VQT 158 + value = vqtr(bin) 159 + value = vqtrs(bin) -- smoothed raw 160 + 161 + -- Whitened VQT (spectral envelope flattened) 162 + value = vqtw(bin) -- peak-normalized whitened 163 + value = vqtsw(bin) -- smoothed peak-normalized whitened 164 + value = vqtrw(bin) -- raw whitened 165 + value = vqtrsw(bin) -- smoothed raw whitened 155 166 ``` 156 167 157 168 **Note:** VQT also returns peak-normalized values (auto-gain controlled), not raw magnitudes. 158 169 170 + ##### Function Index (FFT and VQT) 171 + - `fft(startFreq, endFreq=-1)`: Peak-normalized FFT magnitude over [start,end] Hz. 172 + - `ffts(startFreq, endFreq=-1)`: Smoothed peak-normalized FFT magnitude. 173 + - `fftr(startFreq, endFreq=-1)`: Raw FFT magnitude (no auto-gain). 174 + - `fftrs(startFreq, endFreq=-1)`: Raw smoothed FFT magnitude. 175 + - `vqt(bin)`: Peak-normalized VQT bin magnitude (0–119). 176 + - `vqts(bin)`: Smoothed peak-normalized VQT magnitude. 177 + - `vqtr(bin)`: Raw VQT magnitude (no auto-gain). 178 + - `vqtrs(bin)`: Raw smoothed VQT magnitude. 179 + - `vqtw(bin)`: Peak-normalized whitened VQT magnitude. 180 + - `vqtsw(bin)`: Smoothed peak-normalized whitened VQT magnitude. 181 + - `vqtrw(bin)`: Raw whitened VQT magnitude (no auto-gain). 182 + - `vqtrsw(bin)`: Raw smoothed whitened VQT magnitude. 183 + 184 + ##### Smoothing 185 + Smoothing reduces frame-to-frame jitter by applying an exponential moving average (EMA) over time to each frequency bin. It does not blur across frequency; instead it blends the current frame with previous frames to stabilize visuals and detections. The smoothing factor controls responsiveness versus stability (lower = more responsive, higher = steadier but laggier). Defaults: FFT uses 0.6; VQT uses `VQT_SMOOTHING_FACTOR` (0.3). “Smoothed” variants (`ffts`, `vqts`, `fftrs`, `vqtrs`, and their whitened counterparts) are the temporally smoothed forms of their respective raw/normalized signals. 186 + 159 187 ### FFT vs VQT Comparison 160 188 161 189 | Aspect | FFT | VQT | ··· 171 199 172 200 Both FFT and VQT share the same audio capture buffer: 173 201 - Buffer size: Maximum of (2048, VQT_FFT_SIZE) samples 174 - - FFT reads samples 0-2047 175 - - VQT reads samples 0-(VQT_FFT_SIZE-1) 202 + - FFT reads the latest 2048 samples 203 + - VQT reads the latest `VQT_FFT_SIZE` samples 176 204 - Uses miniaudio for audio capture (mic or loopback on Windows) 177 205 178 206 ### Practical Usage Guidelines ··· 210 238 ### Completed Features 211 239 - **FFT**: 1024 bins with exact original behavior preserved 212 240 - **VQT**: 120 bins with Variable-Q implementation 213 - - **Spectral Whitening**: Per-bin normalization for VQT (removed due to spreading issues) 241 + - **Spectral Whitening**: Whitened VQT outputs and APIs (`vqtw`, `vqtsw`, `vqtrw`, `vqtrsw`) 214 242 - **Shared Audio Buffer**: Automatic sizing for both FFT and VQT 215 - - **API Functions**: `fft()`, `ffts()`, `vqt()`, `vqts()` implemented for all supported languages 243 + - **API Functions**: `fft()`, `ffts()`, `fftr()`, `fftrs()`, `vqt()`, `vqts()`, `vqtr()`, `vqtrs()`, `vqtw()`, `vqtsw()`, `vqtrw()`, `vqtrsw()` implemented across languages 216 244 - **Peak Normalization**: Both FFT and VQT use auto-gain control 217 245 218 246 ### Configuration Options 219 247 - `VQT_FFT_SIZE`: Default 8192 (configurable in vqtdata.h) 220 - - `VQT_SPECTRAL_WHITENING_ENABLED`: Toggle spectral whitening (0/1) 221 - - `VQT_WHITENING_DECAY`: Running average decay factor (default 0.99) 222 248 - `VQT_SMOOTHING_FACTOR`: VQT smoothing factor (default 0.3) 223 249 250 + #### Whitening Configuration (Build-Time) 251 + These macros control the whitening stage during build. The runtime API exposes whitened data via separate functions (`vqtw*` and `vqtrw*`), so no runtime toggle is required. 252 + 253 + ```c 254 + #ifndef VQT_SPECTRAL_WHITENING_ENABLED 255 + #define VQT_SPECTRAL_WHITENING_ENABLED 1 256 + #endif 257 + 258 + #ifndef VQT_WHITENING_WIDTH_BINS 259 + #define VQT_WHITENING_WIDTH_BINS 21 // odd window width for envelope smoothing 260 + #endif 261 + 262 + #ifndef VQT_WHITENING_STRENGTH 263 + #define VQT_WHITENING_STRENGTH 0.95f // 0..1 mix toward whitened spectrum 264 + #endif 265 + 266 + #ifndef VQT_WHITENING_EPS 267 + #define VQT_WHITENING_EPS 1e-6f // floor to stabilize log domain 268 + #endif 269 + ``` 270 + 271 + - `VQT_SPECTRAL_WHITENING_ENABLED`: Compiles whitening path; defaults to 1. 272 + - `VQT_WHITENING_WIDTH_BINS`: Width of the local-mean envelope (log domain). Larger smooths more; must be odd. Default 21. 273 + - `VQT_WHITENING_STRENGTH`: Mix factor between raw and whitened magnitudes (0..1). Higher emphasizes notes more. Default 0.95. 274 + - `VQT_WHITENING_EPS`: Small positive floor added before `logf` to stabilize very small magnitudes. Default 1e-6. 275 + 224 276 ### Test Scripts 225 277 - `demo_fft_vqt_hybrid.lua`: Combined FFT/VQT visualization 226 278 - `test_vqt_spectrum_v2.lua`: VQT spectrum analyzer ··· 231 283 ## Future Enhancements 232 284 233 285 ### Additional API Functions 234 - - `vqts(bin)`: Smoothed VQT data (already implemented) 235 286 - `vqto(octave, note)`: VQT by musical note 236 287 - `vqtos(octave, note)`: Smoothed VQT by musical note 237 - - `fftr(bin)`: Raw (non-normalized) FFT magnitude 238 - - `fftrs(bin)`: Raw smoothed FFT magnitude 239 - - `vqtr(bin)`: Raw (non-normalized) VQT magnitude 240 - - `vqtrs(bin)`: Raw smoothed VQT magnitude 241 288 242 289 ### Signal Processing Enhancements 243 290 - **Adaptive Thresholding**: Dynamic noise floor removal 244 291 245 292 ### Platform Support 246 - - Add VQT to other language bindings (currently Lua only) 247 293 - GPU acceleration for kernel application 248 294 - Configurable FFT sizes at runtime 249 295