firmware/#

Rust firmware for the M5Stack Atom Echo–based nightstand sound machines. Built on esp-idf-svc (std) + the C ESP-IDF underneath. Targets the esp Xtensa Rust toolchain installed via espup.

For the what and why (architecture, MQTT contract, operating modes, signal chain), see ../reference/. This README is the how — current state, dev workflow, and the gotchas we've already paid for.

Status#

Milestone v0.3.x — OTA-capable online firmware. v0.3 adds end-to-end OTA: a two-slot partition layout, an HA update entity with installed/latest versions and a live progress bar, the make ota-publish workflow, and esp_ota_mark_app_valid_cancel_rollback boot validation so a broken firmware reverts to the previous slot. v0.2 (online mode: WiFi + MQTT + HA Discovery, round-tripped short-press) and v0.1 (offline button + audio + NVS + LED) are the foundation underneath. One binary works on every unit — identity is derived at runtime from the STA MAC.

Module layout#

firmware/src/
├── main.rs        — entry; spawns tasks; coordinator loop routes ButtonEvent → audio + outbound
├── events.rs      — ButtonEvent / AudioCommand / LedSignal / OutboundEvent / StateSnapshot
├── state.rs       — VOLUME_PRESETS, VolumeDirection, yo-yo cycle math, snap_to_preset_index
├── nvs.rs         — typed NVS wrapper for volume_index, volume_direction, was_playing
├── audio.rs       — I2S + xorshift white noise + Paul Kellet pink filter + audio task
├── button.rs      — 5-state button FSM + button task (owns G39 PinDriver)
├── led.rs         — SK6812 RMT driver + LED task; (net, audio) base + OTA/Error overrides
├── network.rs     — WiFi + MQTT state machine; gatekeeper for online/offline routing
├── discovery.rs   — HA Discovery JSON payloads (built via format!() — no serde_json)
├── ota.rs         — esp_https_ota wrapper with chunked progress + mark_app_valid
├── channels.rs    — FreeRTOS-queue-backed Sender/Receiver (std::sync::mpsc is broken on esp-idf)
└── secrets.rs     — toml-cfg config struct sourced from cfg.toml at compile time

The deliberate factoring: each task owns its peripherals exclusively; cross-task communication is via FreeRTOS-queue channels carrying typed events. The network task is the gatekeeper — it decides whether short presses toggle audio locally (offline) or wait for HA to publish back via MQTT (online), and it owns the OTA install path. Long-press cycles volume locally in both modes; double is a pure-MQTT gesture.

Build & flash#

Project uses direnv (.envrc at the project root). Open a terminal in any subdir of the project and the env loads automatically. There's also a Makefile at the repo root and in firmware/ that exports the same env so you can build without direnv (CI, fresh clones, etc.).

# from anywhere in the repo:
make firmware                   # cargo build --release
make firmware-flash             # headless: build + write bootloader + partition table + app
make firmware-flash-monitor     # interactive: build + flash + serial monitor (needs TTY)
make firmware-monitor           # serial monitor only
make firmware-check             # cargo check
make firmware-ota-publish       # build + save .bin to OTA dir + publish latest_version (see OTA below)
make firmware-clean

Or directly with cargo:

cd firmware
cargo build --release
espflash flash --port /dev/ttyUSB0 \
  --bootloader target/xtensa-esp32-espidf/release/bootloader.bin \
  --partition-table partitions.csv \
  --erase-parts otadata \
  target/xtensa-esp32-espidf/release/sound-machine

Incremental builds are ~5 s. First-time builds are ~20 min — they download and compile ESP-IDF (~500 MB) plus all the Rust deps.

cfg.toml — compile-time config#

Before the first build, copy cfg.toml.example to cfg.toml and fill in real values:

[sound-machine]
wifi_ssid = "your-wifi-ssid"
wifi_password = "your-wifi-password"
mqtt_url = "mqtt://your-broker.lan:1883"
ota_url_base = "http://firmware.example.lan/sound-machine"

cfg.toml is gitignored. The firmware checks for empty values at boot and refuses to proceed with an empty wifi_ssid/wifi_password/mqtt_url, so you can't accidentally flash a no-config binary. An empty ota_url_base only blocks OTA installs (with a warning); the rest of the firmware still runs.

Compile-time means wired-flash — changing any of these values requires a USB reflash of every device that needs the new value. WiFi/MQTT/OTA host changes are infrequent enough to be worth the inconvenience for the simpler model.

.envrc.private — host-only secrets#

The publish flow needs three host-side variables — OTA_LOCAL_DIR (where to copy the .bin), OTA_URL_BASE (matches cfg.toml), and MQTT_URL. Copy ../.envrc.private.example to ../.envrc.private and fill in real values; direnv loads it automatically. .envrc.private is gitignored.

Monitoring without re-flashing#

To see logs from an already-running device:

espflash monitor --port /dev/ttyUSB0

Ctrl+C exits the monitor; the device keeps running. Anything written via log::info!() etc. shows up here, prefixed with the crate name and a millisecond timestamp.

OTA workflow#

Once a device is on v0.3.x, the next version goes out over WiFi. Two parts: the publish, and the install.

Publish (your dev machine)#

# Cargo.toml: bump version
make firmware-ota-publish

That target:

1. Builds a release binary
2. espflash save-images it to $OTA_LOCAL_DIR/sound-machine-<version>.bin
3. mosquitto_pubs the new version retained to sound-machine/firmware/latest

Both nightstands' HA update cards light up within a couple seconds.

Install (HA UI)#

Click "Install" on the device's Firmware card. The device:

1. LED switches to a magenta pulse
2. Streams the binary into the inactive OTA slot, publishing update_percentage every 5 % (HA renders a progress bar)
3. Reboots into the new slot
4. Reconnects to WiFi + MQTT
5. Calls esp_ota_mark_app_valid_cancel_rollback — confirms the new firmware works and disables the bootloader's pending-rollback timer
6. Republishes installed_version matching latest_version; HA's card flips back to "Up to date"

If the new firmware fails to reach MQTT (panic, WiFi misconfig, etc.) the bootloader rolls back to the previous slot on the next reset, and the device comes up on the old version. HA notices installed_version reverted and shows "Update available" again.

Bootstrap (the one-time wired flash)#

Going from a pre-v0.3 partition layout (single-slot) to v0.3.x's two-slot layout requires a wired make firmware-flash once per device. The flash target writes the new bootloader, partition table, and otadata in addition to the app. After that, OTA is the path forever; if you ever need wired access (changing cfg.toml, debugging OTA breakage in the OTA path itself), USB still works.

Webserver expectations#

$OTA_LOCAL_DIR should be served as static HTTP at $OTA_URL_BASE. Plain HTTP is fine and intentional — the trust boundary is the LAN, same as MQTT. ESP-IDF will refuse plain HTTP for OTA unless CONFIG_ESP_HTTPS_OTA_ALLOW_HTTP=y is set in sdkconfig.defaults (it is).

Toolchain (one-time setup)#

Already done on Chris's vega; documented here for re-setup or a second machine.

# 1. Rust 1.88+ stable (espup needs it)
rustup update stable

# 2. espup CLI itself
cargo install espup

# 3. The Xtensa fork of Rust + LLVM + GCC for ESP32
espup install --targets esp32        # writes ~/export-esp.sh — see "Home dir cleanup" below

# 4. Flasher + linker shim
cargo install espflash
cargo install ldproxy

# 5. System packages (Ubuntu/Debian)
sudo apt install ninja-build         # ESP-IDF's CMake build needs it
sudo apt install libxml2             # see "libxml2 shim" gotcha below

Home dir cleanup#

espup install writes ~/export-esp.sh. We don't keep that — the two export lines it contains are inlined into the project's .envrc so the home dir stays clean. If you re-run espup install (e.g., for a toolchain update), copy the regenerated ~/export-esp.sh contents into .envrc and delete the home-dir file. Or pass --export-file ./.envrc-esp to espup install and source that explicitly.

Known gotchas (so we don't relearn these)#

These all bit us during initial setup. Some are environmental, some are esp-rs ecosystem quirks.

1. libxml2.so.2 missing on Ubuntu Questing (and any libxml2 ≥ 2.14 distro)#

ESP-IDF's bundled esp-clang is linked against the older libxml2.so.2 SONAME, but Ubuntu Questing ships libxml2 with SONAME .so.16 (upstream bumped it in 2.14). First cargo build fails with:

clang: error while loading shared libraries: libxml2.so.2: cannot open shared object file

Fix: project-local symlink in firmware/.lib/libxml2.so.2 → /usr/lib/x86_64-linux-gnu/libxml2.so.16, exposed via LD_LIBRARY_PATH in .envrc. The clang/libxml2 ABI surface is small and stable enough that the SONAME alias works. The .lib/ dir is gitignored.

2. Default Cargo features pulled in embassy and broke linking#

The vanilla esp-idf-template enables embassy in default features, which adds embassy-executor to the dep graph. Without an actual embassy executor wired up, the linker errors with undefined reference to __pender. We don't use embassy in v1, so Cargo.toml drops it from default = [...]. Re-add it (and properly wire an executor) if/when we want async.

3. I2sDriver::new_std_tx pin order is not what intuition expects#

Signature in esp-idf-hal 0.45.2:

new_std_tx(i2s, config, bclk, dout, mclk, ws)

That's dout before ws, not bclk → ws → dout as you'd guess from reading "Bclk, Ws, Data" in audio convention. Getting them swapped produces:

• "Static" instead of a tone (data on the LRCK line, clock on the data line)
• A persistent quiet squeal during silence (WS clock ticking on the data input)
• The NS4168 amp running warm (sustained switching, no real audio output)

The compiler accepts both orderings since the pin types are generic. Always check the signature against the version of esp-idf-hal you're on. Symptom: garbage output where you'd expect tone.

4. DMA replays stale data when starved#

ESP-IDF's I2S driver, once enabled, will keep clocking out whatever's in its DMA buffer. If you write data once and then stop, the buffer keeps getting replayed → an "idling" device produces a quiet repeating noise loop.

Fix in main.rs:

• Pre-fill DMA with zeros via i2s.preload_data(&silence) before tx_enable()
• In the main loop, write a silence chunk every iteration. i2s.write blocks for ~23 ms while DMA consumes it, which is also a perfectly fine button-poll cadence — no sleep needed.

This keeps the amp's input continuously fed, even when "doing nothing."

5. std::sync::mpsc and crossbeam-channel are broken on esp-idf-rs#

Both libraries assume the GNU/newlib pthread_mutex_t ABI (40 bytes, zero-initializer). ESP-IDF's pthread layer uses a 4-byte handle with 0xffffffff as the lazy-init sentinel. Sending across a channel under contention triggers pthread_mutex_lock on a misaligned struct → LoadProhibited exception. Symptom: <channel_type>::try_recv or ::send panics in the bowels of mpsc.

Fix: channels.rs wraps esp_idf_svc::hal::task::queue::Queue (the FreeRTOS native queue, ISR-safe, byte-copy semantics) with Sender/Receiver types that mimic the mpsc API. T: Copy is required (FreeRTOS queues memcpy values) — usually fine for typed events. Use these in place of std::sync::mpsc::channel everywhere.

6. ESP-IDF's CONFIG_PARTITION_TABLE_FILENAME resolves relative to the CMake source dir, not your project#

The ESP-IDF Kconfig docs imply the path is relative to your project. In an esp-idf-rs build, the CMake "project" is the embuild-generated directory under target/, so a relative path silently fails. The Makefile generates a small sdkconfig overlay at build time with the absolute path to partitions.csv, chained via ESP_IDF_SDKCONFIG_DEFAULTS. The committed sdkconfig.defaults only carries the boolean (CONFIG_PARTITION_TABLE_CUSTOM=y).

7. esp_https_ota rejects plain HTTP unless an opt-in flag is set#

Despite working fine at the protocol level, esp_https_ota validates the URL scheme and refuses plain HTTP unless CONFIG_ESP_HTTPS_OTA_ALLOW_HTTP=y is in sdkconfig. With the flag missing, esp_https_ota_begin returns immediately with an error and never opens a socket — webserver logs show no GET requests. The flag is set in sdkconfig.defaults.

8. The onboard NS4168 + tiny speaker is a prototype-only path#

We're using the I2S pins that drive the built-in amp (G19 BCLK, G22 DOUT, G33 LRCK) for hello-world. The plan (signal-chain.md) is to switch to an external MAX98357A on G21/G26/G32 once amps arrive. Don't run sustained white noise on the built-in speaker — short test bursts only. The thermal warning on this is real (we briefly demonstrated it via the swapped-pins bug above).

Layout#

firmware/
├── Cargo.toml              # deps + features
├── rust-toolchain.toml     # pins to the `esp` channel
├── .cargo/config.toml      # target = xtensa-esp32-espidf, ldproxy, runner
├── build.rs                # embuild bootstrap
├── Makefile                # build/flash/ota-publish entry points
├── sdkconfig.defaults      # ESP-IDF kconfig knobs (4MB flash, two-OTA layout, allow-HTTP-OTA)
├── partitions.csv          # custom two-slot OTA partition table
├── cfg.toml.example        # template for compile-time config
├── cfg.toml                # real values (gitignored)
├── src/
│   └── *.rs                # see "Module layout" above
├── .lib/                   # libxml2 shim (gitignored)
├── target/                 # cargo build output (gitignored)
│   └── gen/                # generated sdkconfig overlay for absolute partitions.csv path
└── .embuild/               # embuild-managed ESP-IDF clone (gitignored)

Hardware-pinout cheat sheet (Atom Echo, prototype config)#

Authoritative pinout: ../reference/atom-echo/pinmap.md.

Sources#

• ESP-IDF documentation (v5.3)
• esp-idf-svc on docs.rs
• esp-idf-hal on docs.rs
• esp-rs / Embedded Rust on Espressif (book)
• espup README — env setup options
• Project design docs — operating modes, MQTT contract, signal chain, pin map