What happens. data-disk.sh makes docker require the data volume's mounts, and the converge poller's unit Requires=docker.service. After a boot where the volume is not there, docker fails on its dependency, and the poller fails with it and is never started. Nothing retries an activation after that. activate.sh and the restore runbook assume the thirty-second poller is still running.
How to reproduce. infra/files/data-disk.sh:128-134 writes RequiresMountsFor= for every target into docker's drop-in. infra/files/user-data.sh:199-207 gives vibescrobble-converge.service both Requires=docker.service and After=docker.service. Restart=always restarts a process that exits, not a unit whose dependency failed. activate.sh:48-52 says that until the volume is attached "this fails, and converge tries again". docs/operations.md:165-187 boots the instance without a volume on purpose and relies on one scripts/deploy.sh converge over SSM. If that single run fails, for example because the attachment is still in progress, deploy.sh:213-215 tells the operator the poller retries and to watch journalctl -u vibescrobble-converge, a unit that is not running.
Why it matters. After a snapshot restore, or any boot that misses the volume, the board stays down until someone runs the deploy again by hand. After a successful restore the poller also stays dead until the next reboot, so later release parameter changes are not picked up on their own.
Fixed by https://tangled.org/permadeath.com/vibescrobble.com/pulls/338, now on main.