this repo has no description
1# Nixie - bare metal provisioning engine for NixOS
2
3<img width="196" height="196" alt="Nixie logo" src="https://github.com/user-attachments/assets/41cabf10-213b-4099-aa0c-d711cdab2ed6" />
4
5> [!WARNING]
6> Basic [features](#features) are functional and you can try them out, but some
7> parts are still missing and there are known issues.
8
9Nixie enables zero-touch deployment of NixOS to multiple machines over the
10network, turning blank hard drives into fully configured systems in minutes.
11
12Nixie is a simpler, stateless alternative to [Foreman](https://theforeman.org),
13[MAAS](https://canonical.com/maas), [OpenStack
14Ironic](https://docs.openstack.org/ironic),
15[Tinkerbell](https://tinkerbell.org), etc., focusing on NixOS deployments on
16bare metal (though it can also be used in virtualized environments).
17
18## Features
19
20Currently, only `x86_64-linux` is supported.
21
22- [x] Simple, declarative JSON configuration
23- [x] Build a custom NixOS installer from a flake
24- [x] Built-in PXE server based on [Pixiecore](https://github.com/danderson/netboot/tree/main/pixiecore) to serve netboot components from the custom installer
25- [x] Install NixOS from a flake using [nixos-anywhere](https://nix-community.github.io/nixos-anywhere)
26- [x] Remote power-on with [Wake-on-LAN](https://en.wikipedia.org/wiki/Wake-on-LAN)
27- [ ] Host status check with IP discovery
28- [x] Stateless and ephemeral [^1]
29- [x] Fast, under 2 minutes to install NixOS from empty hard drives [^2]
30- [ ] Support IPv6 single-stack
31
32[^1]: No pre-configured PXE server is required to install other machines - you
33 only need to run Nixie on your laptop or workstation. This solves the
34 "first machine in the data center" problem: in bare-metal setups with
35 persistent PXE servers, you would otherwise need to automate the installation
36 of the PXE server itself.
37[^2]: Using the provided examples with a warm cache on a 1 Gbps network.
38 Factors that affect the speed include whether the configuration has been
39 cached in the Nix store, the size of the NixOS configuration, the network
40 speed, etc.
41
42## Usage
43
44Example command to boot a custom NixOS installer and install the corresponding
45NixOS configuration from [`./examples/flake.nix`](./examples/flake.nix) on
46multiple bare-metal machines based on the MAC addresses defined in
47[`./examples/hosts.json`](./examples/hosts.json).
48
49```sh
50# Running as root for privileged ports
51sudo nixie \
52 --installer ./examples#nixosConfigurations.installer \
53 --flake ./examples \
54 --hosts ./examples/hosts.json \
55 --ssh-key ~/.ssh/id_ed25519
56```
57
58TODO add a demo video/asciinema.
59
60Please see the full example in [`./examples`](./examples).
61
62## How it works
63
64TODO refine the diagram after implementation.
65
66```mermaid
67sequenceDiagram
68 participant Nix
69 participant Nixie
70 participant Machines@{ "type" : "collections" }
71
72 Nixie->>Nixie: Load hosts.json
73
74 loop For each machine
75 Nixie->>Machines: Try checking status
76 Nixie->>Nixie: Skip if already installed
77 end
78
79 Nixie->>Nix: Build installer components<br/>(kernel, initrd, squashfs)
80 Nixie->>Nixie: Start server components in goroutines<br/>(DHCP/TFTP/HTTP/API)
81
82 loop For each machine
83 Nixie->>Machines: Broadcast Wake-on-LAN magic packet
84
85 activate Machines
86
87 Note over Machines: Power on and start PXE boot
88
89 Machines->>Nixie: UEFI firmware broadcast DHCP request
90 Nixie->>Machines: DHCP provide IP (via Proxy DHCP) and next server info
91 Machines->>Nixie: Request kernel
92 Nixie->>Machines: TFTP send kernel
93 Machines->>Nixie: Request initrd
94 Nixie->>Machines: TFTP send initrd
95
96 Note over Machines: Boot into NixOS installer
97 Note over Machines: SystemD starts nixie-agent service
98 Machines->>Nixie: nixie-agent phone home to request install with MAC address
99 Nixie->>Nixie: Find flake based on MAC address and get client IP from API request
100 Nixie->>Nix: Build NixOS configuration
101 Nixie->>Machines: nixos-anywhere format disks via SSH based on disko configuration
102 Nixie->>Machines: nixos-anywhere install system closure via SSH
103 Nixie->>Machines: nixos-anywhere trigger reboot
104
105 Note over Machines: Reboot after installation completed
106
107 Nixie->>Machines: nixos-anywhere confirms machine rebooted
108 deactivate Machines
109
110 activate Machines
111
112 Nixie->>Machines: Check host status
113 end
114
115 Note over Nixie: Return when all machines are installed
116```
117
118## Acknowledgements
119
120> This project is a rewrite of my previous ephemeral PXE server implementation
121> in my [khuedoan/homelab](https://github.com/khuedoan/homelab) repository,
122> which used a stateless PXE server in Docker with Ansible templating and
123> targeted generic Linux distributions. It has been rebuilt from the ground up
124> for NixOS, offering a more integrated and robust experience in this project.
125> All credits from the previous implementation are also referenced here, as
126> similar knowledge and concepts apply.
127
128- Ephemeral PXE server [^1] inspired by [Minimal First Machine in the DC](https://speakerdeck.com/amcguign/minimal-first-machine-in-the-dc)
129- [NixOS netboot with pixiecore](https://nixos.wiki/wiki/Netboot)
130- [The Pixiecore library](https://github.com/danderson/netboot/tree/main/pixiecore)
131- Custom agent for the installation process inspired by [OpenStack ironic-python-agent](https://opendev.org/openstack/ironic-python-agent) and [Tinkerbell Worker](https://tinkerbell.org/docs/services/tink-worker)
132- Wireshark's [WakeOnLAN wiki page](https://wiki.wireshark.org/WakeOnLAN)
133- AMD's [Magic Packet Technology](https://www.amd.com/content/dam/amd/en/documents/archived-tech-docs/white-papers/20213.pdf) white paper