Monorepo for Aesthetic.Computer
aesthetic.computer
1# 🎸 Pedal: Audio Effect Plugin for Ableton Live
2
3## Overview
4
5Create an **audio effect** (filter-style) M4L device where audio passes IN from Ableton's signal chain, gets processed in an AC piece, and passes OUT back to Ableton.
6
7**Key difference from existing devices**: Current AC M4L devices (notepat, metronome, prompt) only output audio FROM the web page TO Ableton. This is an **effect** that processes incoming audio.
8
9## Technical Challenge
10
11### The jweb~ Limitation
12
13Per Cycling74 documentation:
14- **jweb~** has **signal outlets only** (audio OUTPUT from web page)
15- There are **no signal inlets** (no direct audio INPUT to web page)
16- The `signal` message mentioned in docs is for internal use, not audio input
17
18### Solution Architecture
19
20Since jweb~ cannot receive audio signals directly, we need to:
21
221. **Capture audio in Max** using `plugin~ 2` (stereo effect input)
232. **Analyze/sample the audio** and send data via messages to jweb~
243. **Process/visualize** in the AC piece (Web Audio)
254. **Output** via jweb~'s signal outlets → `plugout~`
26
27## Audio Data Flow Options
28
29### Option A: FFT Spectrum Data (Recommended for v1)
30```
31plugin~ 2 → pfft~ → snapshot~ → format → executejavascript window.acPedalFFT(data)
32 │
33 └→ [delay for latency compensation] → plugout~ 2
34```
35- Send FFT magnitude/phase arrays to web page
36- Web page visualizes and/or generates new audio based on spectrum
37- Original audio passes through with optional delay
38- **Best for**: Visualizers, spectrum-driven synths, reactive effects
39
40### Option B: Sample-by-Sample (High latency, limited use)
41```
42plugin~ 2 → snapshot~ @samps 128 → pack → executejavascript window.acPedalSamples(L, R)
43```
44- Send raw sample values to web page
45- Very high message overhead, significant latency
46- **Not recommended** for real-time effects
47
48### Option C: Peak/RMS Envelope (Simple, low latency)
49```
50plugin~ 2 → peakamp~ → snapshot~ → executejavascript window.acPedalEnvelope(peak)
51```
52- Send envelope followers (peak, RMS, etc.)
53- Web audio generates sounds based on amplitude
54- **Best for**: Envelope followers, ducking, gates
55
56### Option D: Hybrid (Audio thru + Web effects)
57```
58plugin~ 2 ────┬────────────────────────────────────→ *~ [dry] ─┐
59 │ │
60 └→ analysis → jweb~ → [web audio] →──────────→ +~ → plugout~
61 │
62 [web-generated audio only, wet]
63```
64- Original audio passes through (dry)
65- Web audio adds effects/layers (wet)
66- Mix control for dry/wet blend
67
68## Recommended Implementation: Option D (Hybrid)
69
70For the `pedal` piece, we'll use **Option D** because:
711. Audio passes through at native quality (no degradation)
722. Web Audio can add processing, visualization, or triggered sounds
733. Dry/wet mix gives flexibility
744. Lower latency than sample-by-sample approaches
75
76## M4L Device Structure
77
78### Max Patcher Architecture
79
80```
81┌─────────────────────────────────────────────────────────────────┐
82│ AC Pedal Effect │
83├─────────────────────────────────────────────────────────────────┤
84│ │
85│ plugin~ 2 │
86│ │ └────────────────────────────────────────┐ │
87│ │ │ │
88│ ▼ ▼ │
89│ ┌──────┐ ┌──────────────────────┐ ┌─────────┐ │
90│ │ FFT │──▶│ fft analysis js │──▶ │ *~ dry │ │
91│ │pfft~ │ │ send to jweb │ │ level │ │
92│ └──────┘ └──────────────────────┘ └────┬────┘ │
93│ │ │
94│ ┌────────────────────────────────────────────────┐│ │
95│ │ jweb~ ││ │
96│ │ ┌────────────────────────────────────────┐ ││ │
97│ │ │ aesthetic.computer/pedal?daw=1 │ ││ │
98│ │ │ [signal out L] [signal out R] [msgs] │ ││ │
99│ │ └───────┬────────────┬───────────────────┘ ││ │
100│ └──────────┼────────────┼───────────────────────┘│ │
101│ │ │ │ │
102│ ▼ ▼ ▼ │
103│ ┌─────────┐ ┌─────────┐ ┌──┴──┐ │
104│ │ *~ wet │ │ *~ wet │ │ +~ │ │
105│ │ level │ │ level │ └──┬──┘ │
106│ └────┬────┘ └────┬────┘ │ │
107│ └─────────┬──┘ │ │
108│ ▼ ▼ │
109│ ┌──┴──┐ ┌──┴──┐ │
110│ │ +~ ├───────────────────│ +~ │ │
111│ └──┬──┘ └──┬──┘ │
112│ │ │ │
113│ ▼ ▼ │
114│ plugout~ 2 │
115│ │
116│ ┌────────────────────────────────────────────────────────────┐ │
117│ │ Controls: │ │
118│ │ [live.dial dry/wet] [live.dial wet_vol] [live.dial drive] │ │
119│ └────────────────────────────────────────────────────────────┘ │
120└─────────────────────────────────────────────────────────────────┘
121```
122
123### Key M4L Objects
124
125| Object | Purpose |
126|--------|---------|
127| `plugin~ 2` | Receive stereo audio from Ableton |
128| `pfft~` | FFT analysis of input |
129| `peakamp~` | Amplitude envelope |
130| `jweb~` | Web view with audio output |
131| `*~` | Level/gain control |
132| `+~` | Mix signals |
133| `plugout~` | Send stereo audio back to Ableton |
134| `live.dial` | Automatable parameters |
135
136### JavaScript Functions (window.*)
137
138```javascript
139// Called from Max with FFT data (32-128 bins typically)
140window.acPedalFFT = function(magnitudes) {
141 // magnitudes: array of FFT bin magnitudes [0-1]
142 // Used for visualization and audio-reactive effects
143};
144
145// Called from Max with amplitude envelope
146window.acPedalEnvelope = function(peakL, peakR, rmsL, rmsR) {
147 // Used for envelope-following effects
148};
149
150// Called from Max with tempo/transport (inherited from base)
151window.acDawTempo = function(bpm) { ... };
152window.acDawTransport = function(playing) { ... };
153```
154
155## Web Audio Processing in pedal.mjs
156
157### Piece Structure
158
159```javascript
160// pedal.mjs - Audio Effect Pedal for Ableton Live
161
162let fftData = [];
163let envelope = { peakL: 0, peakR: 0, rmsL: 0, rmsR: 0 };
164let wetLevel = 0.5;
165let drive = 1.0;
166
167function boot({ sound, query }) {
168 // DAW mode detection
169 const dawMode = query?.daw === "1";
170
171 // Set up Web Audio processing chain
172 // (triggered by envelope/FFT data, outputs via jweb~)
173}
174
175function sim({ sound }) {
176 // Update envelope followers
177 // Trigger sounds based on input analysis
178}
179
180function paint({ wipe, ink, screen }) {
181 // Visualize FFT spectrum
182 // Show input level meters
183 // Display effect status
184}
185
186// Exported for M4L window.acPedal* functions
187export function setFFT(data) { fftData = data; }
188export function setEnvelope(pL, pR, rL, rR) {
189 envelope = { peakL: pL, peakR: pR, rmsL: rL, rmsR: rR };
190}
191```
192
193### Effect Ideas for pedal.mjs
194
1951. **Visualizer Only** - Display input spectrum, pass audio through
1962. **Envelope Follower** - Trigger synth notes based on input amplitude
1973. **Spectral Freeze** - Analyze and hold FFT, generate frozen drone
1984. **Vocoder-style** - Use input spectrum to modulate synth output
1995. **Transient Detector** - Trigger drum hits on input transients
200
201## Implementation Steps
202
203### Phase 1: Basic Effect Shell ✅ COMPLETE
2041. ✅ Create `pedal.mjs` piece with FFT visualization
2052. ✅ Add `pedal` device to `devices.json` with `"type": "effect"`
2063. ✅ Extend `build.py` with new `generate_effect_patcher()` function
2074. ✅ Build AMXD with audio input/output chain
208
209### Phase 2: Analysis Pipeline (NEXT)
2101. Add FFT analysis in Max (`pfft~` → `js` → `jweb~`)
2112. Add envelope followers (`peakamp~` → `snapshot~` → `jweb~`)
2123. Wire up `window.acPedal*` functions
213
214### Phase 3: Audio Generation
2151. Web Audio synth triggered by envelope
2162. Dry/wet mix controls
2173. Parameter automation
218
219---
220
221## How to Test
222
223### 1. Start the local dev server
224```bash
225npm run site # or npm run aesthetic
226```
227
228### 2. Build and install the pedal device
229```bash
230cd ac-m4l
231python3 build.py pedal --install
232```
233
234### 3. In Ableton Live
2351. Find "AC 🎸 pedal" in the **Audio Effects** section of the browser
2362. Drag it onto an **audio track** with audio playing
2373. Observe the FFT visualization responding to input audio
2384. The dry signal passes through; the wet signal (from jweb~) is mixed in
239
240### 4. Test controls
241- **Tap/Space**: Cycle effect modes (visualizer → envelope-synth → freeze → gate)
242- **Up/Down arrows**: Adjust trigger threshold (for envelope-synth mode)
243
244---
245
246## Files Created/Modified
247
248| File | Action | Description |
249|------|--------|-------------|
250| `system/public/aesthetic.computer/disks/pedal.mjs` | ✅ **CREATED** | AC piece for effect |
251| `ac-m4l/devices.json` | ✅ **MODIFIED** | Added pedal device config with `"type": "effect"` |
252| `ac-m4l/build.py` | ✅ **MODIFIED** | Added `generate_effect_patcher()` function |
253| `ac-m4l/AC 🎸 pedal (localhost:8888).amxd` | ✅ **BUILT** | Generated effect device |
254| `ac-m4l/ac-fft-analyzer.js` | 🔜 FUTURE | Max JS for FFT → jweb |
255| `plans/pedal.md` | ✅ **CREATED** | This planning document |
256
257## References
258
259- [jweb~ documentation](https://docs.cycling74.com/reference/jweb~)
260- [pfft~ documentation](https://docs.cycling74.com/reference/pfft~)
261- [M4L Audio Effect Guidelines](https://github.com/Ableton/maxdevtools/blob/main/m4l-production-guidelines)
262- Existing AC M4L integration: `ac-m4l/build.py`, `ac-m4l/ABLETON-INTEGRATION-PROGRESS.md`
263
264## Open Questions
265
2661. **Latency compensation**: How much latency does jweb~ add? Need to delay dry signal to match.
2672. **Sample rate matching**: Browser AudioContext vs. Ableton - handled by existing `acDawSamplerate`
2683. **FFT bin count**: 32? 64? 128? Balance between detail and message overhead
2694. **Message rate**: How often to send FFT data? Every vector? Every N ms?
270
271## Testing Checklist
272
273- [ ] Audio passes through when loaded (dry signal)
274- [ ] FFT visualization updates with input audio
275- [ ] Wet signal (from web audio) mixes correctly
276- [ ] Dry/wet control works
277- [ ] No clicks/pops on parameter changes
278- [ ] Works at 44.1kHz and 48kHz sample rates
279- [ ] Multiple instances work simultaneously