I’ve been running a NixOS based homelab for awhile now with 5 physical hosts and about 30 NixOS containers/VMs in an Incus cluster and I can’t imagine moving away from my central Nix repo and ability to rebuild/upgrade the entire fleet in one command and feel confident that things will work.
While I’m concerned about the disbanding, it would take quite a lot to make me look elsewhere, and there’s enough critical mass that I feel confident others will step up.
YMMV, I don’t run a home lab, I just have a NAS and run a personal website. I definitely don’t have a “fleet”. I have two servers running Debian and I can bring them back up from zero in <30 minutes.
“Rebuild everything in one command and have it work” is not something that I’m chasing after and I am skeptical that it would work anyway. Maybe there is something I’m missing, but when I update the software, the updates come with changes and it’s possible that things break. My goals are to keep reasonably up to date and to be able to fix things quickly if they do break.
Regarding Nix complaints:
Package maintenance is kind of a crapshoot. Maybe your package is in nixpkgs, maybe there is a flake for it, maybe you find something that is actively updated, maybe not. Maybe there is a package but half the features are turned off because the maintainer didn’t bother. Maybe there is a package but half the features are turned off on macOS for unknown reasons. What I want is just to know what level of distro-level maintenance the package has, including its transitive dependencies.
Docs are just kinda bad. Fragmented across different sites. Docs teaching you Nix forwards or backwards, or teaching old Nix, or teaching Nix with flakes, or teaching you Nix for end-users or developers or package maintainers, or Nix on Mac or NixOS. A surprising number of broken links. What I want is one site, with a little drop-down menu to select the version I am using. What I have is hours spent on the NixOS Discord trying to figure out how to do basic stuff.
Yeah. I fixed AI/GPU stack consisting of 6 existing packages to run CLI on Mac. Fixed all comments during several weeks. And then silence. No approve no nothing for few months. Closed PR and will never come back.
> “Rebuild everything in one command and have it work” is not something that I’m chasing after and I am skeptical that it would work anyway.
I'm not using "rebuild" to describe recreating an instance from scratch, but in the "nixos-rebuild" sense, i.e. I can deploy a fleet-wide configuration shared by all hosts, or apply package updates across the fleet, etc.
As for the skepticism, all I can offer is my experience which is that it does indeed work, and I've been running this way for awhile.
> Maybe there is something I’m missing, but when I update the software, the updates come with changes and it’s possible that things break.
The only time I've had breaking changes that required manual intervention was on a major release update. The issue amounted to "You're using abc configuration option but should be using xyz instead". This message was clearly logged, and fixing it was a matter of a few minutes of reading why a particular property name had changed.
Those updates happen twice/year, and in the last ~3 years I think two packages have forced me to make a change. Unless you're running unstable, you won't encounter this for ongoing package updates within a release.
On the topic of rebuilding from scratch, that's not a single command, but it's only a few. VMs/containers mount incus volumes (zfs-backed) for data storage, and a fresh rebuild is a matter of spawning a new instance from my homelab base image (I use OpenTofu to orchestrate this), running a nixos-rebuild targeting the new instance from my workstation, and things are back up and running. But I'm less focused on full rebuilds since I'm already doing Incus backups and can restore on any of the nodes in my Incus cluster.
> My goals are to keep reasonably up to date and to be able to fix things quickly if they do break.
My goals are similar. I can completely understand sticking with Debian for two hosts. I had experimented with NixOS for years but never really went all-in until I started expanding my homelab environment. At that point, NixOS just clicked and I'm far more productive with 30+ hosts than I used to be with 1-2.
> Package maintenance is kind of a crapshoot. Maybe your package is in nixpkgs, maybe there is a flake for it, maybe...
Which channel were you running on? And do you have any examples of specific packages? What you're describing just sounds foreign to me. I currently default to the latest stable release but if there's something that isn't in stable I'll override that specific package to use unstable. Much of the community just runs everything on unstable, but I prefer a slightly slower pace of updates.
Nixpkgs is the largest package repository across distros by volume, and as much as I love Debian, it's far more common to find things missing there.
> Maybe there is a package but half the features are turned off on macOS for unknown reasons
This sounds like you're branching into territory that can no longer be reasonably framed as NixOS vs. Debian (or other distro of choice).
> Docs are just kinda bad.
This is by far the project's greatest weakness. LLMs have been the saving grace. The frontier models are excellent at NixOS and I've switched to mostly having a conversation in my NixOS Claude project when I need info. This is in no way a defense of the docs, but for anyone motivated to use NixOS, there is at least a good option beyond the docs themselves.
I moved my homelab to Kubernetes. I just don’t see the use of single VPS’es. But I use the cluster as a training platform, so if I didn’t need Kubernetes in other parts of my life, I probably would have stuck with my NixOS VPS’es that all share config.
I use kubenix to manage the cluster, of course. ;-)
The two aren't mutually exclusive! I just finished porting my personal kubernetes cluster from running on Debian to running on NixOS. Now whenever I want to update the host OS on my kubernetes nodes, I just build a new bootable ISO from my Nix configs and upload it to my servers, swapping out the whole OS atomically. If anything goes wrong, I can swap back to my previous build. No more anxiety running `sudo apt-get upgrade` and hoping for the best.
But as the host OS I use Talos Linux, which itself is declarative and minimal. For small Kubernetes version upgrades, I upgrade the kubelet with an API command. When that isn't possible, I upload a new Talos image and swap them.
Since the nodes are cloud VMs, I will use a spare VM to add as a new node, and remove an old node. So I never run with fewer nodes. For my on-prem cluster, I will do the same with a spare hypervisor VM. It really helps ensure my nodes are ephemeral.
So NixOS, for now, has become a desktop/laptop operating system for me, and whenever I need an execution environment, e.g. a CI runner or a remote shell, I choose "something Nix-like" which is `nix` in a container without systemd.
I’ve been running a NixOS based homelab for awhile now with 5 physical hosts and about 30 NixOS containers/VMs in an Incus cluster and I can’t imagine moving away from my central Nix repo and ability to rebuild/upgrade the entire fleet in one command and feel confident that things will work.
While I’m concerned about the disbanding, it would take quite a lot to make me look elsewhere, and there’s enough critical mass that I feel confident others will step up.