feat(phase7.7): add user-facing docs and flake template for Darwin builder

+27 -3

PLAN.md

··· 19 19 | Phase 4 — Building | 🚧 Tooling ready | `scripts/build-trivial.sh` (new) | 20 20 | Phase 5 — Daemon | 🚧 Stubs done | `src/dirserv/` (new), `tests/dirserv/` (new) | 21 21 | Phase 6 — CI | 🚧 In progress | `.tangled/workflows/ci.yml`, `tests/darling-smoke.nix`, `tests/nix-in-darling.nix`, `tests/nix/compatibility-matrix.sh` (new) | 22 - | Phase 7 — Remote Builder | 🚧 Module & hook ready | `nix/darlingBuilderModule.nix` (new), `scripts/darling-build-hook` (new), `tests/darling-builder.nix` (new) | 22 + | Phase 7 — Remote Builder | 🚧 Module, hook & docs ready | `nix/darlingBuilderModule.nix` (new), `scripts/darling-build-hook` (new), `tests/darling-builder.nix` (new), `docs/darwin-builder.md` (new), `templates/darling-builder/` (new) | 23 23 | Phase 8 — Stretch | 📋 Planned | — | 24 24 25 25 ### Recently Completed 26 26 27 + - **Phase 7.7 — Documentation and flake template**: Created 28 + `docs/darwin-builder.md` — comprehensive user-facing guide covering 29 + NixOS module quick start, manual setup (sshd, SSH keys, builder 30 + registration), shared `/nix/store` configuration, verification 31 + procedures, custom build hook (no SSH) alternative, performance 32 + tuning (binary substitution, job parallelism, store sharing, storage), 33 + troubleshooting (connection refused, permission denied, unimplemented 34 + syscalls, database errors, sandbox issues, slow builds), security 35 + considerations, and architecture diagram. Created 36 + `templates/darling-builder/` — a `nix flake init` template that 37 + generates a ready-to-use NixOS configuration with the Darling builder 38 + module pre-configured. Wired into `flake.nix` as 39 + `templates.darling-builder`. Includes its own README with options 40 + reference, architecture diagram, and troubleshooting section. 27 41 - **Phase 7.5 — NixOS module for Darling builder**: Created 28 42 `nix/darlingBuilderModule.nix` — a full NixOS module that sets up a 29 43 Darling instance as a `nix.buildMachines` remote builder for ··· 187 201 | [plan/10-phase8-stretch.md](./plan/10-phase8-stretch.md) | `aarch64-darwin`, GUI testing, Hydra builder | 188 202 | [plan/11-architecture.md](./plan/11-architecture.md) | System diagram, key technical decisions, glossary | 189 203 | [plan/syscall-triage.md](./plan/syscall-triage.md) | Tracking table for unimplemented syscalls | 204 + | [docs/darwin-builder.md](./docs/darwin-builder.md) | **User guide** — setup, troubleshooting, performance tuning | 190 205 191 206 ## New File Map 192 207 ··· 195 210 ```text 196 211 darling-nix/ 197 212 ├── .tangled/workflows/ci.yml # tangled.org CI workflow (Phase 6) 198 - ├── flake.nix # Flake with package, devShell, NixOS module, builder (Phase 0, 7) 213 + ├── docs/ 214 + │ └── darwin-builder.md # NEW — User-facing setup guide, troubleshooting, perf tuning (Phase 7.7) 215 + ├── flake.nix # Flake with package, devShell, NixOS module, builder, templates (Phase 0, 7) 199 216 ├── nix/ 200 217 │ ├── package.nix # Darling Nix derivation (Phase 0) 201 218 │ ├── devShell.nix # Developer shell (Phase 0) ··· 235 252 │ ├── test_renameatx_np.c # renameatx_np tests (Phase 1) 236 253 │ ├── test_setattrlist_flags.c # setattrlist ATTR_CMN_FLAGS tests (Phase 1) 237 254 │ └── test_utimensat.c # utimensat/timestamp tests (Phase 1) 255 + ├── templates/ 256 + │ └── darling-builder/ # NEW — Flake template for Darwin builder setup (Phase 7.7) 257 + │ ├── flake.nix # Ready-to-use NixOS config with services.darling-builder 258 + │ └── README.md # Quick start, options reference, troubleshooting 238 259 └── plan/ 239 260 ├── README.md # Index + priority table 240 261 ├── 00-background.md # Motivation & current state ··· 373 394 ./tests/nix/compatibility-matrix.sh --tier 1 374 395 ./tests/nix/compatibility-matrix.sh --output results.json --compare previous.json 375 396 ``` 376 - - 7.7: Write user-facing docs and flake template 397 + - ~~7.7: Write user-facing docs and flake template~~ ✅ Done — see `docs/darwin-builder.md` and `templates/darling-builder/` 377 398 378 399 ### Completed Task Summary 379 400 380 401 | Task | Status | Description | 381 402 |------|--------|-------------| 403 + | 7.7 | ✅ | User-facing docs (`docs/darwin-builder.md`) and flake template (`templates/darling-builder/`) | 382 404 | 1.1 | ✅ | `setattrlist`/`fsetattrlist`/`getattrlist` — ATTR_CMN_FLAGS, CRTIME, CHGTIME | 383 405 | 1.2 | ✅ | `lchflags` return value — verified via 1.1 | 384 406 | 1.3 | ✅ | `renameatx_np` (syscall 488) — maps to Linux `renameat2` | ··· 424 446 | `nix build .#checks.x86_64-linux.darling-smoke -L` | NixOS VM smoke test (no network) | After building Darling | 425 447 | `nix build .#checks.x86_64-linux.nix-in-darling -L` | Full Nix-in-Darling integration test | End-to-end validation | 426 448 | `nix build .#checks.x86_64-linux.darling-builder -L` | Remote builder VM test (sshd, SSH auth, service) | After editing `nix/darlingBuilderModule.nix` | 449 + | `docs/darwin-builder.md` | User-facing setup guide, troubleshooting, perf tuning | Setting up a Darling builder for the first time | 450 + | `nix flake init -t .#darling-builder` | Generate a ready-to-use NixOS config with the builder | Bootstrapping a new Darling builder project | 427 451 428 452 See [plan/README.md](./plan/README.md) for the full priority table and effort 429 453 estimates.

+713

docs/darwin-builder.md

··· 1 + # Setting Up a Darling-Based Darwin Builder 2 + 3 + > Build `x86_64-darwin` Nix packages on Linux — no Apple hardware required. 4 + 5 + This guide walks you through setting up [Darling](https://www.darlinghq.org/) 6 + as a Nix remote builder so your Linux machine can build macOS (`x86_64-darwin`) 7 + derivations. This is analogous to how Wine enables running Windows binaries on 8 + Linux, but for macOS build toolchains. 9 + 10 + ## Table of Contents 11 + 12 + - [Overview](#overview) 13 + - [Prerequisites](#prerequisites) 14 + - [Quick Start (NixOS Module)](#quick-start-nixos-module) 15 + - [Manual Setup](#manual-setup) 16 + - [Shared /nix/store](#shared-nixstore) 17 + - [Verifying the Builder](#verifying-the-builder) 18 + - [Alternative: Custom Build Hook (No SSH)](#alternative-custom-build-hook-no-ssh) 19 + - [Performance Tuning](#performance-tuning) 20 + - [Troubleshooting](#troubleshooting) 21 + - [Security Considerations](#security-considerations) 22 + - [Architecture](#architecture) 23 + 24 + --- 25 + 26 + ## Overview 27 + 28 + Darling is a macOS compatibility layer for Linux that translates macOS system 29 + calls into Linux equivalents. By running Nix inside Darling, we get a builder 30 + that: 31 + 32 + - Reports `builtins.currentSystem == "x86_64-darwin"` 33 + - Executes Darwin derivations using macOS-compatible toolchains 34 + - Integrates with Nix's remote builder infrastructure over SSH 35 + - Can optionally share `/nix/store` with the host for zero-copy builds 36 + 37 + **Two approaches are available:** 38 + 39 + | Approach | Pros | Cons | 40 + |----------|------|------| 41 + | **NixOS Module** (SSH-based) | Declarative, integrates with `nix.buildMachines`, works with any Nix client | Requires sshd inside Darling | 42 + | **Custom Build Hook** (no SSH) | Simpler setup, lower overhead | Only works locally, less standard | 43 + 44 + --- 45 + 46 + ## Prerequisites 47 + 48 + - **Linux x86_64** host (NixOS recommended, but any Linux with Nix works) 49 + - **Nix** with flakes enabled (`experimental-features = nix-command flakes`) 50 + - **Kernel support**: unprivileged user namespaces and overlayfs 51 + ```bash 52 + # Check user namespace support 53 + sysctl kernel.unprivileged_userns_clone 54 + # Should be: kernel.unprivileged_userns_clone = 1 55 + 56 + # If not, enable it (NixOS handles this automatically): 57 + sudo sysctl kernel.unprivileged_userns_clone=1 58 + ``` 59 + 60 + --- 61 + 62 + ## Quick Start (NixOS Module) 63 + 64 + The fastest path on NixOS is the declarative module. Add this to your 65 + NixOS configuration: 66 + 67 + ```nix 68 + # /etc/nixos/flake.nix (or wherever your system flake is) 69 + { 70 + inputs = { 71 + nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; 72 + darling-nix = { 73 + url = "github:user/darling-nix"; # adjust to actual repo 74 + inputs.nixpkgs.follows = "nixpkgs"; 75 + }; 76 + }; 77 + 78 + outputs = { nixpkgs, darling-nix, ... }: { 79 + nixosConfigurations.myhost = nixpkgs.lib.nixosSystem { 80 + system = "x86_64-linux"; 81 + modules = [ 82 + ./configuration.nix 83 + 84 + # Base Darling support (programs.darling) 85 + darling-nix.nixosModules.nixos 86 + 87 + # Darling builder service (services.darling-builder) 88 + darling-nix.nixosModules.darling-builder 89 + 90 + { 91 + services.darling-builder = { 92 + enable = true; 93 + maxJobs = 4; 94 + shareStore = true; # share /nix/store between host and Darling 95 + }; 96 + } 97 + ]; 98 + }; 99 + }; 100 + } 101 + ``` 102 + 103 + Then rebuild and test: 104 + 105 + ```bash 106 + # Apply the configuration 107 + sudo nixos-rebuild switch 108 + 109 + # Test connectivity to the Darling builder 110 + darling-builder-test 111 + 112 + # Build a Darwin package from your Linux host! 113 + nix build nixpkgs#hello --system x86_64-darwin 114 + ``` 115 + 116 + ### Module Options Reference 117 + 118 + | Option | Type | Default | Description | 119 + |--------|------|---------|-------------| 120 + | `enable` | bool | `false` | Enable the Darling builder service | 121 + | `package` | package | `pkgs.darling` | Darling package to use | 122 + | `port` | int | `2222` | SSH port for the builder (inside Darling) | 123 + | `maxJobs` | int | `1` | Maximum concurrent build jobs | 124 + | `speedFactor` | int | `1` | Nix speed factor (lower = deprioritised vs native builders) | 125 + | `shareStore` | bool | `false` | Share `/nix/store` between host and Darling via `/Volumes/SystemRoot` | 126 + | `sshKeyPath` | path | `/etc/nix/darling-builder-key` | Path to the SSH private key for the Nix daemon | 127 + | `prefixPath` | path | `/var/lib/darling-builder` | Path to the Darling prefix directory | 128 + | `supportedFeatures` | list of str | `[]` | Nix supported features for this builder | 129 + | `mandatoryFeatures` | list of str | `[]` | Nix mandatory features for this builder | 130 + | `installNix` | bool | `true` | Automatically install Nix inside the Darling prefix | 131 + | `nixVersion` | str | `"2.24.10"` | Nix version to install inside Darling | 132 + 133 + --- 134 + 135 + ## Manual Setup 136 + 137 + If you're not on NixOS or prefer manual control, follow these steps. 138 + 139 + ### 1. Install Darling 140 + 141 + ```bash 142 + # Using the flake 143 + nix profile install github:user/darling-nix 144 + 145 + # Or build from source 146 + git clone https://github.com/user/darling-nix 147 + cd darling-nix 148 + nix build 149 + export PATH="$(pwd)/result/bin:$PATH" 150 + ``` 151 + 152 + ### 2. Initialise the Darling Prefix 153 + 154 + ```bash 155 + # Boot Darling (creates the prefix at ~/.darling by default) 156 + darling shell echo "Darling is working" 157 + 158 + # Verify macOS identity 159 + darling shell uname -s # → Darwin 160 + darling shell sw_vers # → macOS 11.7.4 (Big Sur) 161 + ``` 162 + 163 + ### 3. Install Nix Inside Darling 164 + 165 + Use the automated installer script: 166 + 167 + ```bash 168 + ./scripts/install-nix-in-darling.sh 169 + ``` 170 + 171 + Or install manually: 172 + 173 + ```bash 174 + # Download the Nix installer inside Darling 175 + darling shell bash -lc ' 176 + curl -L https://nixos.org/nix/install -o /tmp/install-nix 177 + chmod +x /tmp/install-nix 178 + /tmp/install-nix --no-daemon 179 + ' 180 + ``` 181 + 182 + Verify: 183 + 184 + ```bash 185 + ./scripts/verify-nix.sh 186 + 187 + # Or manually: 188 + darling shell bash -lc "nix --version" 189 + darling shell bash -lc "nix eval --raw --expr 'builtins.currentSystem'" 190 + # → x86_64-darwin 191 + ``` 192 + 193 + ### 4. Set Up sshd Inside Darling 194 + 195 + ```bash 196 + # Generate host keys 197 + darling shell ssh-keygen -A 198 + 199 + # Write sshd config 200 + darling shell tee /etc/ssh/sshd_config << 'EOF' 201 + Port 2222 202 + ListenAddress 127.0.0.1 203 + PermitRootLogin yes 204 + PubkeyAuthentication yes 205 + AuthorizedKeysFile .ssh/authorized_keys 206 + PasswordAuthentication no 207 + ChallengeResponseAuthentication no 208 + UsePAM no 209 + Subsystem sftp /usr/libexec/sftp-server 210 + EOF 211 + 212 + # Generate an SSH keypair for the Nix daemon 213 + sudo ssh-keygen -t ed25519 -N "" -f /etc/nix/darling-builder-key 214 + 215 + # Install the public key inside Darling 216 + darling shell mkdir -p /var/root/.ssh 217 + sudo cat /etc/nix/darling-builder-key.pub | darling shell tee /var/root/.ssh/authorized_keys 218 + darling shell chmod 700 /var/root/.ssh 219 + darling shell chmod 600 /var/root/.ssh/authorized_keys 220 + 221 + # Start sshd 222 + darling shell /usr/sbin/sshd -f /etc/ssh/sshd_config 223 + 224 + # Verify connectivity 225 + ssh -i /etc/nix/darling-builder-key -p 2222 -o StrictHostKeyChecking=no root@127.0.0.1 echo ok 226 + # → ok 227 + ``` 228 + 229 + ### 5. Register the Builder with Nix 230 + 231 + Add to `/etc/nix/machines` (or use `nix.buildMachines` on NixOS): 232 + 233 + ``` 234 + ssh://root@127.0.0.1 x86_64-darwin /etc/nix/darling-builder-key 4 1 - - - 235 + ``` 236 + 237 + The fields are: `store-uri system ssh-key max-jobs speed-factor supported-features mandatory-features public-host-key` 238 + 239 + Or in `nix.conf`: 240 + 241 + ```ini 242 + builders = ssh://root@127.0.0.1:2222 x86_64-darwin /etc/nix/darling-builder-key 4 1 - - - 243 + ``` 244 + 245 + Enable distributed builds: 246 + 247 + ```ini 248 + # In /etc/nix/nix.conf 249 + builders-use-substitutes = true 250 + ``` 251 + 252 + ### 6. Test a Build 253 + 254 + ```bash 255 + # Ping the builder 256 + nix store ping --store ssh://root@127.0.0.1:2222 257 + 258 + # Build a trivial derivation 259 + nix build --expr 'derivation { 260 + name = "hello-darwin"; 261 + builder = "/bin/bash"; 262 + args = ["-c" "echo hello from darwin > $out"]; 263 + system = "x86_64-darwin"; 264 + }' -L 265 + 266 + # Build a real package 267 + nix build nixpkgs#hello --system x86_64-darwin -L 268 + ``` 269 + 270 + --- 271 + 272 + ## Shared /nix/store 273 + 274 + By default, Nix copies store paths over SSH to the builder and back. Since 275 + Darling runs on the same machine, this is wasteful. You can share 276 + `/nix/store` directly. 277 + 278 + ### How It Works 279 + 280 + Darling mounts the host's root filesystem at `/Volumes/SystemRoot` inside the 281 + prefix. This means the host's `/nix/store` is accessible at 282 + `/Volumes/SystemRoot/nix/store` from within Darling. 283 + 284 + ### Setup 285 + 286 + ```bash 287 + # Inside Darling, symlink /nix to the host's /nix 288 + darling shell ln -sf /Volumes/SystemRoot/nix /nix 289 + ``` 290 + 291 + Or, if there's a conflict with Darling's overlay filesystem: 292 + 293 + ```bash 294 + # Bind mount the host's /nix into the prefix 295 + sudo mount --bind /nix ~/.darling/nix 296 + ``` 297 + 298 + ### Important Caveats 299 + 300 + 1. **Separate databases**: The Nix SQLite database 301 + (`/nix/var/nix/db/db.sqlite`) must NOT be shared between the host and 302 + Darling. Each Nix instance needs its own database. Configure the Darling 303 + Nix instance to use separate state: 304 + 305 + ```ini 306 + # In /etc/nix/nix.conf inside Darling: 307 + store = /nix 308 + state = /var/nix 309 + ``` 310 + 311 + 2. **Concurrent writes**: If both host and Darling write to `/nix/store` 312 + simultaneously, use content-addressed store paths (which are safe for 313 + concurrent writes) or ensure the Darling builder is the exclusive writer 314 + for `x86_64-darwin` paths. 315 + 316 + 3. **Permission mapping**: Darling's UID/GID namespace may differ from the 317 + host's. Ensure files written by Darling's `_nixbldN` users are readable 318 + by the host's Nix daemon. 319 + 320 + ### NixOS Module 321 + 322 + If you're using the NixOS module, just set: 323 + 324 + ```nix 325 + services.darling-builder.shareStore = true; 326 + ``` 327 + 328 + The module handles the symlink and state directory separation automatically. 329 + 330 + --- 331 + 332 + ## Verifying the Builder 333 + 334 + ### Automated Checks 335 + 336 + ```bash 337 + # NixOS module: use the built-in connectivity test 338 + darling-builder-test 339 + 340 + # Manual setup: use the build hook check 341 + ./scripts/darling-build-hook --check 342 + 343 + # Standalone Nix health-check 344 + ./scripts/verify-nix.sh 345 + ./scripts/verify-nix.sh --online # also test network/cache access 346 + 347 + # Run the compatibility matrix (after Nix builds work) 348 + ./tests/nix/compatibility-matrix.sh --tier 1 349 + ``` 350 + 351 + ### Progressive Build Tests 352 + 353 + The `build-trivial.sh` script tests derivation building at five increasing 354 + levels of complexity: 355 + 356 + ```bash 357 + # Run all five levels 358 + ./scripts/build-trivial.sh 359 + 360 + # Target a specific level with debug output 361 + ./scripts/build-trivial.sh --level 1 --debug 362 + 363 + # Levels: 364 + # 1. Echo to $out — minimal: sandbox-exec → bash → file creation 365 + # 2. Multi-line builder — mkdir, chmod, loops, multiple output files 366 + # 3. Input transformation — builtins.toFile, sort, wc 367 + # 4. Derivation dependency — one derivation consumes another's output 368 + # 5. Binary substitution — fetch pre-built package from cache.nixos.org 369 + ``` 370 + 371 + ### NixOS VM Tests 372 + 373 + Run the full test suite without needing a live Darling instance: 374 + 375 + ```bash 376 + # Smoke test (no network, fast) 377 + nix build .#checks.x86_64-linux.darling-smoke -L 378 + 379 + # Full Nix integration test (needs network) 380 + nix build .#checks.x86_64-linux.nix-in-darling -L 381 + 382 + # Remote builder test (sshd, SSH auth, service lifecycle) 383 + nix build .#checks.x86_64-linux.darling-builder -L 384 + 385 + # Directory Services stubs (pure shell, no Darling needed) 386 + nix build .#checks.x86_64-linux.dirserv-stubs -L 387 + 388 + # Run everything 389 + nix flake check 390 + ``` 391 + 392 + --- 393 + 394 + ## Alternative: Custom Build Hook (No SSH) 395 + 396 + If you don't want to run sshd inside Darling, you can use the custom build 397 + hook. This invokes `darling shell` directly instead of going through SSH. 398 + 399 + ### Setup 400 + 401 + ```bash 402 + # Verify the hook environment is ready 403 + ./scripts/darling-build-hook --check 404 + 405 + # Build a single derivation directly 406 + ./scripts/darling-build-hook --build /nix/store/...-foo.drv 407 + 408 + # Print the machine spec line 409 + ./scripts/darling-build-hook --machine-spec 410 + ``` 411 + 412 + ### Configure Nix to Use the Hook 413 + 414 + In `nix.conf`: 415 + 416 + ```ini 417 + builders = /path/to/darling-build-hook x86_64-darwin - 4 1 - - - 418 + ``` 419 + 420 + Or on NixOS: 421 + 422 + ```nix 423 + nix.settings.builders = [ 424 + "/path/to/darling-build-hook x86_64-darwin - 4 1 - - -" 425 + ]; 426 + ``` 427 + 428 + ### Environment Variables 429 + 430 + | Variable | Default | Description | 431 + |----------|---------|-------------| 432 + | `DARLING_BUILD_HOOK_DARLING` | `darling` | Path to the darling binary | 433 + | `DARLING_BUILD_HOOK_PREFIX` | auto | Darling prefix path | 434 + | `DARLING_BUILD_HOOK_NIX_PROFILE` | `/Users/root/.nix-profile/etc/profile.d/nix.sh` | Nix profile to source inside Darling | 435 + | `DARLING_BUILD_HOOK_MAX_JOBS` | `4` | Maximum concurrent jobs | 436 + | `DARLING_BUILD_HOOK_VERBOSE` | `0` | Verbosity level (0=quiet, 1=debug) | 437 + | `DPREFIX` | auto | Fallback for prefix path | 438 + 439 + --- 440 + 441 + ## Performance Tuning 442 + 443 + ### Binary Substitution 444 + 445 + The single most important performance optimization is to use binary 446 + substitution aggressively. Most `x86_64-darwin` packages in Nixpkgs are 447 + already built by Hydra and available from `cache.nixos.org`. 448 + 449 + ```ini 450 + # In /etc/nix/nix.conf inside Darling: 451 + substituters = https://cache.nixos.org 452 + trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= 453 + ``` 454 + 455 + On the host: 456 + 457 + ```ini 458 + # In /etc/nix/nix.conf on the host: 459 + builders-use-substitutes = true 460 + ``` 461 + 462 + This tells Nix to let the builder download substitutes directly from the 463 + cache, instead of building everything from source. 464 + 465 + ### Job Parallelism 466 + 467 + ```nix 468 + services.darling-builder = { 469 + maxJobs = 4; # concurrent derivations 470 + }; 471 + ``` 472 + 473 + Also limit per-build parallelism to avoid oversubscription: 474 + 475 + ```ini 476 + # In /etc/nix/nix.conf inside Darling: 477 + cores = 4 478 + ``` 479 + 480 + A good rule of thumb: `maxJobs × cores ≤ number of CPU cores`. 481 + 482 + ### Store Sharing 483 + 484 + If you're building many packages, enable store sharing to eliminate the 485 + SSH copy overhead: 486 + 487 + ```nix 488 + services.darling-builder.shareStore = true; 489 + ``` 490 + 491 + This makes build outputs instantly available on the host without any 492 + network transfer. 493 + 494 + ### Storage 495 + 496 + Put the Darling prefix and Nix store on fast storage (SSD/NVMe). Nix builds 497 + are I/O-heavy, and spinning disks will be a significant bottleneck. 498 + 499 + ### Speed Factor 500 + 501 + If you have native Darwin builders (e.g., a Mac mini), set the Darling 502 + builder's speed factor lower so Nix prefers the native machine: 503 + 504 + ```nix 505 + services.darling-builder.speedFactor = 1; 506 + # Native builder would be speedFactor = 10 or higher 507 + ``` 508 + 509 + --- 510 + 511 + ## Troubleshooting 512 + 513 + ### "Connection refused" when connecting to the builder 514 + 515 + **Cause**: sshd is not running inside the Darling prefix, or it's listening 516 + on a different port. 517 + 518 + ```bash 519 + # Check if sshd is listening 520 + ss -tlnp | grep 2222 521 + 522 + # Check if the Darling prefix is running 523 + darling shell echo ok 524 + 525 + # Restart sshd inside Darling 526 + darling shell /usr/sbin/sshd -f /etc/ssh/sshd_config 527 + 528 + # Check sshd logs 529 + darling shell cat /var/log/sshd.log 2>/dev/null 530 + ``` 531 + 532 + ### "Permission denied (publickey)" 533 + 534 + **Cause**: SSH key mismatch between the host and the Darling prefix. 535 + 536 + ```bash 537 + # Verify the key exists 538 + ls -la /etc/nix/darling-builder-key 539 + 540 + # Verify the public key is installed in Darling 541 + darling shell cat /var/root/.ssh/authorized_keys 542 + 543 + # Regenerate keys 544 + sudo ssh-keygen -t ed25519 -N "" -f /etc/nix/darling-builder-key 545 + sudo cat /etc/nix/darling-builder-key.pub | darling shell tee /var/root/.ssh/authorized_keys 546 + 547 + # Test manually 548 + ssh -vvv -i /etc/nix/darling-builder-key -p 2222 root@127.0.0.1 echo ok 549 + ``` 550 + 551 + ### "builder for '...' failed with exit code 1" 552 + 553 + **Cause**: The derivation build itself failed. Check the build log. 554 + 555 + ```bash 556 + # Get the full build log 557 + nix log /nix/store/...-failed.drv 558 + 559 + # Or build with verbose output 560 + nix build ... -L --keep-failed 561 + 562 + # Check for unimplemented syscalls 563 + darling shell bash -lc 'nix-build ...' 2>&1 | grep -i "unimplemented\|STUB" 564 + ``` 565 + 566 + ### "Unimplemented syscall" errors 567 + 568 + **Cause**: The derivation uses a macOS syscall that Darling doesn't implement yet. 569 + 570 + ```bash 571 + # Run the syscall triage tool to identify the issue 572 + ./scripts/triage-syscalls.sh --output /tmp/triage.md 573 + 574 + # Check the known triage table 575 + cat plan/syscall-triage.md 576 + ``` 577 + 578 + If you discover a new unimplemented syscall, please 579 + [file an issue](https://github.com/darlinghq/darling/issues) with: 580 + 581 + 1. The exact syscall name and number 582 + 2. The derivation that triggered it 583 + 3. The full error message 584 + 585 + ### "Store path not valid" / database errors 586 + 587 + **Cause**: When using shared `/nix/store`, the host and Darling Nix instances 588 + may have different database states. 589 + 590 + ```bash 591 + # Verify the store inside Darling 592 + darling shell bash -lc 'nix-store --verify --check-contents' 593 + 594 + # Re-register a missing path 595 + darling shell bash -lc 'nix-store --register-validity <<< "..."' 596 + 597 + # If all else fails, repair the store 598 + darling shell bash -lc 'nix-store --verify --repair' 599 + ``` 600 + 601 + ### "sandbox-exec: not found" or sandbox errors 602 + 603 + **Cause**: The sandbox-exec stub is not installed in the prefix. 604 + 605 + ```bash 606 + # Check if sandbox-exec is present 607 + darling shell test -x /usr/bin/sandbox-exec && echo ok || echo missing 608 + 609 + # Rebuild Darling with sandbox stubs 610 + nix build .#darling 611 + ``` 612 + 613 + ### Darling prefix won't start / crashes 614 + 615 + ```bash 616 + # Check if darlingserver is running 617 + pgrep darlingserver 618 + 619 + # Try shutting down and re-initialising 620 + darling shutdown 621 + darling shell echo ok 622 + 623 + # Check system requirements 624 + sysctl kernel.unprivileged_userns_clone # must be 1 625 + ``` 626 + 627 + ### Build is extremely slow 628 + 629 + See [Performance Tuning](#performance-tuning) above. The most common causes are: 630 + 631 + 1. **No binary substitution**: The builder is compiling everything from source. 632 + Make sure `substituters` and `builders-use-substitutes` are configured. 633 + 2. **No store sharing**: Large closures are being copied over SSH. Enable 634 + `shareStore = true`. 635 + 3. **Slow storage**: The Nix store is on a spinning disk. Move it to SSD. 636 + 4. **Oversubscription**: Too many concurrent jobs for available CPU cores. 637 + 638 + --- 639 + 640 + ## Security Considerations 641 + 642 + - **sshd inside Darling**: The SSH server only listens on `127.0.0.1` (loopback), 643 + so it's not accessible from the network. Only key-based authentication is 644 + allowed — no passwords. 645 + 646 + - **Root inside Darling**: The builder runs as root inside the Darling prefix, 647 + but this is a *virtual* root. Darling uses Linux user namespaces, so the 648 + "root" inside Darling maps to an unprivileged user on the host. 649 + 650 + - **Store integrity**: When sharing `/nix/store`, Nix's content-addressed paths 651 + provide integrity guarantees — a path's name includes a hash of its contents, 652 + so tampering is detectable. 653 + 654 + - **Network access**: Derivations built inside Darling can access the network 655 + unless the Nix sandbox restricts it. Fixed-output derivations (fetchers) 656 + need network access by design; regular builds should not. 657 + 658 + - **Upstream trust**: This is an experimental project. It should not be used as 659 + a trusted builder for production deployments without thorough security review. 660 + 661 + --- 662 + 663 + ## Architecture 664 + 665 + ``` 666 + ┌──────────────────────────────────────────────────────────┐ 667 + │ Linux Host (NixOS) │ 668 + │ │ 669 + │ ┌─────────────┐ SSH (localhost:2222) ┌──────────┐ │ 670 + │ │ Nix Daemon │ ──────────────────────── │ sshd │ │ 671 + │ │ │ or build-hook pipe │ (Darling)│ │ 672 + │ │ Offloads │ │ │ │ 673 + │ │ x86_64- │ │ ┌──────┐ │ │ 674 + │ │ darwin │ │ │ Nix │ │ │ 675 + │ │ builds │ │ │daemon│ │ │ 676 + │ └─────────────┘ │ └──────┘ │ │ 677 + │ │ │ │ │ 678 + │ │ ┌───────────────────┤ │ │ 679 + │ ▼ │ /Volumes/ │ │ │ 680 + │ ┌─────────────┐ │ SystemRoot/nix ──▶│ /nix │ │ 681 + │ │ /nix/store │◀──────┘ (shared store) │ (symlink)│ │ 682 + │ │ │ │ │ │ 683 + │ └─────────────┘ └──────────┘ │ 684 + │ Darling Prefix │ 685 + │ (~/.darling or │ 686 + │ /var/lib/darling-builder) │ 687 + └──────────────────────────────────────────────────────────┘ 688 + ``` 689 + 690 + **Key components:** 691 + 692 + | Component | Role | 693 + |-----------|------| 694 + | **darlingserver** | Userspace daemon that translates macOS syscalls to Linux | 695 + | **Darling prefix** | Virtual macOS filesystem tree (overlayfs-based) | 696 + | **sandbox-exec stub** | Passes through commands without sandboxing (Darling provides Linux-level isolation) | 697 + | **Directory Services stubs** | `dscl`, `dseditgroup`, `sysadminctl` — translate macOS user/group commands to `/etc/passwd` + `/etc/group` | 698 + | **diskutil stub** | Returns expected filesystem info for the Nix installer | 699 + | **darling-build-hook** | Alternative to SSH — invokes `darling shell` directly | 700 + | **darlingBuilderModule.nix** | NixOS module that wires everything together declaratively | 701 + 702 + For more technical details, see [plan/11-architecture.md](../plan/11-architecture.md). 703 + 704 + --- 705 + 706 + ## Further Reading 707 + 708 + - [Project plan](../plan/README.md) — full development plan with phases and tasks 709 + - [Known blockers](../plan/01-blockers.md) — detailed analysis of blocking issues 710 + - [Syscall triage](../plan/syscall-triage.md) — tracking table for unimplemented syscalls 711 + - [Darling documentation](https://docs.darlinghq.org/) — upstream Darling docs 712 + - [Nix remote builders](https://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html) — Nix manual on distributed builds 713 + - [Blog: Nix All The Way Down](https://ersei.net/en/blog/nix-all-the-way-down) — early exploration of Nix-in-Darling

+11

flake.nix

··· 29 29 30 30 packages.darling-sdk = pkgs: pkgs.darling.sdk; 31 31 32 + # ── Flake Templates ────────────────────────────────────────────── 33 + # 34 + # Initialise a new project with: 35 + # nix flake init -t github:nixie-dev/darling-nix#darling-builder 36 + # 37 + # See: docs/darwin-builder.md, plan/09-phase7-remote-builder.md (Task 7.7) 38 + templates.darling-builder = { 39 + path = ./templates/darling-builder; 40 + description = "NixOS configuration with a Darling-based x86_64-darwin remote builder"; 41 + }; 42 + 32 43 # ── NixOS Modules ──────────────────────────────────────────────── 33 44 # 34 45 # The base module (programs.darling) is autoloaded from

+27 -9

plan/09-phase7-remote-builder.md

··· 521 521 522 522 --- 523 523 524 - ### 7.7 — Documentation and Templates 524 + ### 7.7 — Documentation and Templates ✅ 525 525 526 526 Create user-facing documentation so others can set up their own Darling builders. 527 527 528 + **Status**: ✅ Complete — see `docs/darwin-builder.md` and `templates/darling-builder/`. 529 + 528 530 **Deliverables**: 529 531 530 - 1. **NixOS wiki page**: Step-by-step guide for setting up a Darling-based Darwin 531 - builder on NixOS. Cover both the NixOS module approach and the manual setup. 532 + 1. ✅ **User-facing setup guide** (`docs/darwin-builder.md`): Comprehensive 533 + documentation covering NixOS module quick start, manual setup (sshd, SSH 534 + keys, builder registration), shared `/nix/store` configuration, verification 535 + procedures (automated checks, progressive build tests, NixOS VM tests), 536 + custom build hook (no SSH) alternative, performance tuning (binary 537 + substitution, job parallelism, store sharing, storage, speed factor), 538 + troubleshooting (connection refused, permission denied, unimplemented 539 + syscalls, database errors, sandbox issues, slow builds), security 540 + considerations, and architecture diagram with component table. 532 541 533 - 2. **Flake template** (`templates/darling-builder`): 542 + 2. ✅ **Flake template** (`templates/darling-builder/`): 534 543 ```bash 535 - nix flake init -t github:user/darling-nix#darling-builder 544 + nix flake init -t github:nixie-dev/darling-nix#darling-builder 536 545 ``` 537 - Generates a minimal `flake.nix` + NixOS configuration that sets up the 538 - builder. 546 + Generates a `flake.nix` with a ready-to-use NixOS configuration that 547 + imports both the base Darling module and the builder module, with all 548 + options documented inline. Includes its own `README.md` with getting 549 + started steps, options reference table, architecture diagram, and 550 + troubleshooting section. Wired into `flake.nix` as 551 + `templates.darling-builder`. 539 552 540 - 3. **Troubleshooting guide**: Common issues and their solutions: 553 + 3. ✅ **Troubleshooting guide** (in `docs/darwin-builder.md`): Covers all 554 + planned scenarios: 541 555 - "Connection refused" → sshd not running or wrong port 542 556 - "Permission denied" → SSH key mismatch 543 557 - "Build failed with signal 11" → unimplemented syscall → file an issue 544 558 - "Store path not valid" → shared store database mismatch 545 559 - "builder for '...' failed with exit code 1" → check the build log 560 + - "sandbox-exec: not found" → sandbox stub missing 561 + - Darling prefix crashes → darlingserver / kernel requirements 562 + - Slow builds → substitution, store sharing, storage, oversubscription 546 563 547 - 4. **Performance tuning guide**: Tips for getting the best performance: 564 + 4. ✅ **Performance tuning guide** (in `docs/darwin-builder.md`): Covers: 548 565 - Use binary substitution aggressively (`substituters` in `nix.conf`) 549 566 - Set `max-jobs` based on available CPU cores 550 567 - Use `--cores N` to limit per-build parallelism 551 568 - Enable store sharing to avoid copy overhead 552 569 - Put the Nix store on fast storage (SSD/NVMe) 570 + - Speed factor configuration for multi-builder setups 553 571 554 572 --- 555 573

+2

plan/README.md

+151

templates/darling-builder/README.md

··· 1 + # Darling Builder Template 2 + 3 + This template sets up a NixOS configuration with a **Darling-based 4 + `x86_64-darwin` remote builder**, allowing your Linux machine to build macOS 5 + packages without Apple hardware. 6 + 7 + ## Getting Started 8 + 9 + ### 1. Initialise from the template 10 + 11 + ```bash 12 + mkdir my-darwin-builder && cd my-darwin-builder 13 + nix flake init -t github:nixie-dev/darling-nix#darling-builder 14 + ``` 15 + 16 + ### 2. Customise `flake.nix` 17 + 18 + Open `flake.nix` and: 19 + 20 + - Replace `"myhost"` with your machine's hostname. 21 + - Uncomment and add your existing NixOS modules (`hardware-configuration.nix`, 22 + `configuration.nix`, etc.). 23 + - Adjust `services.darling-builder` options to taste (see 24 + [Options](#module-options) below). 25 + 26 + ### 3. Apply the configuration 27 + 28 + ```bash 29 + sudo nixos-rebuild switch --flake .#myhost 30 + ``` 31 + 32 + This will: 33 + 34 + 1. Install Darling on the host. 35 + 2. Create and initialise a Darling prefix at `/var/lib/darling-builder`. 36 + 3. Install Nix inside the Darling prefix. 37 + 4. Start an SSH server inside the prefix on `127.0.0.1:2222`. 38 + 5. Register the Darling instance as a `nix.buildMachines` entry for 39 + `x86_64-darwin`. 40 + 41 + ### 4. Verify 42 + 43 + ```bash 44 + # Built-in connectivity check 45 + darling-builder-test 46 + 47 + # Build a Darwin package from your Linux host 48 + nix build nixpkgs#hello --system x86_64-darwin 49 + ``` 50 + 51 + ## Module Options 52 + 53 + | Option | Type | Default | Description | 54 + |--------|------|---------|-------------| 55 + | `enable` | bool | `false` | Enable the Darling builder service | 56 + | `package` | package | `pkgs.darling` | Darling package to use | 57 + | `port` | int | `2222` | SSH port inside the Darling prefix | 58 + | `maxJobs` | int | `1` | Maximum concurrent build jobs | 59 + | `speedFactor` | int | `1` | Nix builder speed factor (lower = deprioritised) | 60 + | `shareStore` | bool | `false` | Share `/nix/store` between host and Darling via `/Volumes/SystemRoot` | 61 + | `sshKeyPath` | path | `/etc/nix/darling-builder-key` | SSH private key for the Nix daemon | 62 + | `prefixPath` | path | `/var/lib/darling-builder` | Darling prefix directory | 63 + | `supportedFeatures` | list of str | `[]` | Nix supported features | 64 + | `mandatoryFeatures` | list of str | `[]` | Nix mandatory features | 65 + | `installNix` | bool | `true` | Auto-install Nix inside the prefix on first boot | 66 + | `nixVersion` | str | `"2.24.10"` | Nix version to install inside Darling | 67 + 68 + ## How It Works 69 + 70 + ``` 71 + ┌────────────────────────────────────────────────────────┐ 72 + │ Linux Host (NixOS) │ 73 + │ │ 74 + │ ┌───────────┐ SSH (localhost:2222) ┌────────────┐ │ 75 + │ │ Nix Daemon │ ──────────────────────▶ │ sshd │ │ 76 + │ │ │ │ (Darling) │ │ 77 + │ │ offloads │ │ ┌────────┐ │ │ 78 + │ │ x86_64- │ │ │ Nix │ │ │ 79 + │ │ darwin │ │ │ daemon │ │ │ 80 + │ └───────────┘ │ └────────┘ │ │ 81 + │ │ │ │ │ 82 + │ ▼ /Volumes/ │ │ │ 83 + │ ┌───────────┐ SystemRoot/nix ──────▶│ /nix │ │ 84 + │ │/nix/store │ (shared store) │ (symlink) │ │ 85 + │ └───────────┘ └────────────┘ │ 86 + │ Darling Prefix │ 87 + └────────────────────────────────────────────────────────┘ 88 + ``` 89 + 90 + Darling is a macOS compatibility layer that translates macOS system calls into 91 + Linux equivalents. By running Nix inside Darling, the builder reports 92 + `builtins.currentSystem == "x86_64-darwin"` and can execute Darwin derivations. 93 + 94 + When `shareStore` is enabled, the host's `/nix/store` is made available inside 95 + the Darling prefix via `/Volumes/SystemRoot`, eliminating the need to copy 96 + store paths over SSH. 97 + 98 + ## Troubleshooting 99 + 100 + ### "Connection refused" 101 + 102 + The sshd inside Darling isn't running or is on a different port. 103 + 104 + ```bash 105 + # Check if the service is running 106 + systemctl status darling-builder 107 + 108 + # Check if sshd is listening 109 + ss -tlnp | grep 2222 110 + 111 + # Restart the service 112 + sudo systemctl restart darling-builder 113 + ``` 114 + 115 + ### "Permission denied (publickey)" 116 + 117 + SSH key mismatch between the host and Darling prefix. 118 + 119 + ```bash 120 + # Verify the key exists 121 + ls -la /etc/nix/darling-builder-key 122 + 123 + # Verify the public key inside Darling 124 + darling --prefix /var/lib/darling-builder shell cat /var/root/.ssh/authorized_keys 125 + 126 + # Test SSH manually 127 + ssh -vvv -i /etc/nix/darling-builder-key -p 2222 root@127.0.0.1 echo ok 128 + ``` 129 + 130 + ### Build fails with "Unimplemented syscall" 131 + 132 + The derivation uses a macOS syscall that Darling doesn't support yet. This is 133 + expected for complex packages. Check the 134 + [syscall triage table](https://github.com/nixie-dev/darling-nix/blob/main/plan/syscall-triage.md) 135 + and consider filing an issue upstream. 136 + 137 + ### Build is very slow 138 + 139 + 1. **Enable binary substitution** — most `x86_64-darwin` packages are already 140 + on `cache.nixos.org`. Ensure `builders-use-substitutes = true` is set in 141 + the host's `nix.conf`. 142 + 2. **Enable store sharing** — set `shareStore = true` to avoid copying store 143 + paths over SSH. 144 + 3. **Use fast storage** — put the Darling prefix on SSD/NVMe. 145 + 146 + ## Further Reading 147 + 148 + - [Full setup guide](https://github.com/nixie-dev/darling-nix/blob/main/docs/darwin-builder.md) 149 + - [Project plan](https://github.com/nixie-dev/darling-nix/blob/main/plan/README.md) 150 + - [Darling documentation](https://docs.darlinghq.org/) 151 + - [Nix distributed builds](https://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html)

+87

templates/darling-builder/flake.nix

··· 1 + { 2 + description = "NixOS configuration with a Darling-based x86_64-darwin builder"; 3 + 4 + inputs = { 5 + nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; 6 + 7 + darling-nix = { 8 + url = "github:nixie-dev/darling-nix"; 9 + inputs.nixpkgs.follows = "nixpkgs"; 10 + }; 11 + }; 12 + 13 + outputs = 14 + { 15 + nixpkgs, 16 + darling-nix, 17 + ... 18 + }: 19 + { 20 + # ── NixOS system configuration ─────────────────────────────────── 21 + # 22 + # Replace "myhost" with your machine's hostname. 23 + # Adjust the module list and options to fit your setup. 24 + # 25 + # Build with: 26 + # sudo nixos-rebuild switch --flake .#myhost 27 + # 28 + nixosConfigurations.myhost = nixpkgs.lib.nixosSystem { 29 + system = "x86_64-linux"; 30 + modules = [ 31 + # Your existing hardware and system configuration: 32 + # ./hardware-configuration.nix 33 + # ./configuration.nix 34 + 35 + # Base Darling support — provides `programs.darling` 36 + darling-nix.nixosModules.nixos 37 + 38 + # Darling builder service — provides `services.darling-builder` 39 + darling-nix.nixosModules.darling-builder 40 + 41 + # ── Builder settings ──────────────────────────────────────── 42 + { 43 + services.darling-builder = { 44 + # Enable the builder service. After `nixos-rebuild switch`, 45 + # a systemd service will: 46 + # 1. Initialise a Darling prefix 47 + # 2. Install Nix inside the prefix 48 + # 3. Start sshd so the host Nix daemon can connect 49 + # 4. Register as a `nix.buildMachines` entry 50 + enable = true; 51 + 52 + # Maximum number of concurrent build jobs. 53 + # A safe default is half your CPU core count. 54 + maxJobs = 4; 55 + 56 + # Share /nix/store between the host and the Darling prefix 57 + # via /Volumes/SystemRoot. This avoids copying store paths 58 + # over SSH and makes build results instantly available. 59 + # Disable this if you encounter permission or database issues. 60 + shareStore = true; 61 + 62 + # SSH port for the builder (inside the Darling prefix). 63 + # Uses 2222 by default to avoid conflicting with the host's sshd. 64 + # port = 2222; 65 + 66 + # Nix speed factor — lower means Nix will prefer other builders 67 + # when available. Set higher if this is your only Darwin builder. 68 + # speedFactor = 1; 69 + 70 + # Path to the SSH private key used by the Nix daemon to connect 71 + # to the Darling builder. Generated automatically on first boot. 72 + # sshKeyPath = "/etc/nix/darling-builder-key"; 73 + 74 + # Path to the Darling prefix directory. 75 + # prefixPath = "/var/lib/darling-builder"; 76 + 77 + # Automatically install Nix inside the Darling prefix on first boot. 78 + # installNix = true; 79 + 80 + # Nix version to install inside Darling. 81 + # nixVersion = "2.24.10"; 82 + }; 83 + } 84 + ]; 85 + }; 86 + }; 87 + }

Configure Feed

Configure Feed