At the time I was there we hadn't made it all the way to NixOS on the targets, so it was an Ubuntu "base" + Nix managing the app workspace as a kind of pseudo container, though able to bring more of its own system configuration with it (https://github.com/numtide/system-manager) than the more typical nuisance setup of container + separate outer config managed by ansible or a deb or whatever.

As a result of that, in "production" we still actually delivered full rootfs images just to be absolutely certain, and the delta upgrades were for smaller test fleets that could updated hourly just via simple push tooling.

If I was building it from scratch though, I'd probably just do NixOS + colmena, and do a push model forever. It's not worth the saved SSH connection to not have those logs and status messages coming back to the central coordinator immediately.