Nixie - bare metal provisioning engine for NixOS#

WARNING

Basic features are functional and you can try them out, but some parts are still missing and there are known issues.

Nixie enables zero-touch deployment of NixOS to multiple machines over the network, turning blank hard drives into fully configured systems in minutes.

Nixie is a simpler, stateless alternative to Foreman, MAAS, OpenStack Ironic, Tinkerbell, etc., focusing on NixOS deployments on bare metal (though it can also be used in virtualized environments).

Features#

Currently, only x86_64-linux is supported.

Simple, declarative JSON configuration
Build a custom NixOS installer from a flake
Built-in PXE server based on Pixiecore to serve netboot components from the custom installer
Install NixOS from a flake using nixos-anywhere
Remote power-on with Wake-on-LAN
Host status check with IP discovery
Stateless and ephemeral ¹
Fast, under 2 minutes to install NixOS from empty hard drives ²
Support IPv6 single-stack

Usage#

Example command to boot a custom NixOS installer and install the corresponding NixOS configuration from ./examples/flake.nix on multiple bare-metal machines based on the MAC addresses defined in ./examples/hosts.json.

# Running as root for privileged ports
sudo nixie \
    --installer ./examples#nixosConfigurations.installer \
    --flake ./examples \
    --hosts ./examples/hosts.json \
    --ssh-key ~/.ssh/id_ed25519

TODO add a demo video/asciinema.

Please see the full example in ./examples.

How it works#

TODO refine the diagram after implementation.

sequenceDiagram
    participant Nix
    participant Nixie
    participant Machines@{ "type" : "collections" }

    Nixie->>Nixie: Load hosts.json

    loop For each machine
    Nixie->>Machines: Try checking status
    Nixie->>Nixie: Skip if already installed
    end

    Nixie->>Nix: Build installer components<br/>(kernel, initrd, squashfs)
    Nixie->>Nixie: Start server components in goroutines<br/>(DHCP/TFTP/HTTP/API)

    loop For each machine
        Nixie->>Machines: Broadcast Wake-on-LAN magic packet

        activate Machines

        Note over Machines: Power on and start PXE boot

        Machines->>Nixie: UEFI firmware broadcast DHCP request
        Nixie->>Machines: DHCP provide IP (via Proxy DHCP) and next server info
        Machines->>Nixie: Request kernel
        Nixie->>Machines: TFTP send kernel
        Machines->>Nixie: Request initrd
        Nixie->>Machines: TFTP send initrd

        Note over Machines: Boot into NixOS installer
        Note over Machines: SystemD starts nixie-agent service
        Machines->>Nixie: nixie-agent phone home to request install with MAC address
        Nixie->>Nixie: Find flake based on MAC address and get client IP from API request
        Nixie->>Nix: Build NixOS configuration
        Nixie->>Machines: nixos-anywhere format disks via SSH based on disko configuration
        Nixie->>Machines: nixos-anywhere install system closure via SSH
        Nixie->>Machines: nixos-anywhere trigger reboot

        Note over Machines: Reboot after installation completed

        Nixie->>Machines: nixos-anywhere confirms machine rebooted
        deactivate Machines

        activate Machines

        Nixie->>Machines: Check host status
    end

    Note over Nixie: Return when all machines are installed

Acknowledgements#

This project is a rewrite of my previous ephemeral PXE server implementation in my khuedoan/homelab repository, which used a stateless PXE server in Docker with Ansible templating and targeted generic Linux distributions. It has been rebuilt from the ground up for NixOS, offering a more integrated and robust experience in this project. All credits from the previous implementation are also referenced here, as similar knowledge and concepts apply.

Ephemeral PXE server ¹ inspired by Minimal First Machine in the DC
NixOS netboot with pixiecore
The Pixiecore library
Custom agent for the installation process inspired by OpenStack ironic-python-agent and Tinkerbell Worker
Wireshark's WakeOnLAN wiki page
AMD's Magic Packet Technology white paper

No pre-configured PXE server is required to install other machines - you only need to run Nixie on your laptop or workstation. This solves the "first machine in the data center" problem: in bare-metal setups with persistent PXE servers, you would otherwise need to automate the installation of the PXE server itself. ↩︎ ↩︎
Using the provided examples with a warm cache on a 1 Gbps network. Factors that affect the speed include whether the configuration has been cached in the Nix store, the size of the NixOS configuration, the network speed, etc. ↩︎

Configure Feed