I use Nexenta, which uses the Solaris kernel, but with a Debian server-style userland, as a NAS. If ZFS were on Linux, I too would switch to it. Straight Solaris is like travelling back in time in comparison to the usual GNU userland, and while it's got nicely integrated ideas - like the services, logging and monitoring, etc., - the cost of being in a less mainstream branch is reasonably high.
At least with Nexenta, you have an apt-get which might work for some common packages. With straight Solaris, I have to go through all my shell scripts and only use POSIX options, etc., which turns out to be quite a pain.
We use Solaris 10. Which isn't quite the same thing, of course.
The Solaris kernel is excellent. The Solaris 10 userland, not so (except for the newer core stuff like SMF), and OpenSolaris is supposedly a Solaris kernel with a saner userland. But I'm yet to be convinced it is stable enough to run in production, and the new packaging system looks mildly troublesome.
And it's still difficult to get it to install without X.
Hard to figure out whether your service state modification (say, a service restart) worked too. It always returns 0, never has output, and you have to dig for a log file to figure out what happened.
And dependencies, configurable restarting based on failures in other services, delegation of administrative privileges to other users, configuration of managed services through arbitrary properties, integration into OS fault reporting, reliable management of server processes using Solaris Contracts.
I use it on a few workstations. It's a fine operating system, but I'm admittedly a bit of an enterprise UNIX head that's disenchanted with where Linux has gone. For the last decade or so, I've come to like some of the other free operating systems such as the BSD family and OpenSolaris.
I do, our webservers run it. ZFS with automatic snapshots has saved my ass quite a few times already.
Sending incremental snapshots (for backups) to offsite location (another OpenSolaris machine) is also easy and efficient.
But, if linux had ZFS i'd probably use that instead.
FreeBSD already has jails, so there's no point bringing in Solaris zones; but we're generally quick to bring in features from OpenSolaris. DTrace, for example, is available for the FreeBSD kernel now, and work is underway to make it available for FreeBSD userland too.
Could you clear something up for me? If ZFS is so highly coveted (which I've inferred from recent discussions), why doesn't every operating system under the sun support it?
In a nutshell, the issue is that the Linux kernel which is licensed under the GNU General Public License is incompatible with ZFS which is licensed under the Sun CDDL. While both the GPL and CDDL are open source licenses their terms are such that it is impossible to simultaneously satisfy both licenses. This means that a single derived work of the Linux kernel and ZFS cannot be legally distributed.
