Hacker Timesnew | past | comments | ask | show | jobs | submitlogin

I don't know what world you live in, but this is a problem for any software development.

You need to ensure that there is only one version of any library used globally throughout the code and that the set of versions is compatible with each other, and preferably you also want everything to be built against the same toolchain with the same flags.

That usually means onboarding third-party libraries into your own build system.



I'd say with semver becoming far better known, this is not a problem for "any" software development. The developer gets the choice to pick libraries that are stable, often also influencing language choice. Mistakes happen, Guava broke the Java ecosystem for about two years, but it's never something that is accepted as just a fact of software development, it is a mistake.

Wanting to hold Python+C ecosystem more accountable is fair I think, at least from my own experience around half a year ago, Anaconda doesn't work and you need a Dockerfile for any sort of reproducibility, which can have issues since GPU with docker isn't that easy. And this means developers from the vendors working with Anaconda, for example, on solving the issue rather than just hoping for contributors to do it. If AMD were to make easy, reproducible builds without root or VM a reality, that would be reason enough to try their hardware. If not, hopefully Nvidia does and then there really would be no way across the moat for me at least.


Semver is a joke and doesn't work. Languages like C and C++ can easily have problems if you link code built with different versions together (even if you aim for them to be compatible, or even if they are indeed the same source version but with subtly different flags), and there are no good solutions for this, except not doing it.

A docker container is not really any different from any other process; the main difference is that it runs in a chroot pretty much.


> problems if you link code built with different versions

But that has nothing to do with semver.

Semver gives you information about when when you can replace one version with another version. It doesn't promise that you can mix multiple versions together.


It gives you information about intent, not reality.

And you are mixing multiple versions if you are building against version x.y and linking against version x.(y+z).


Maybe I misunderstood "built with", because I thought you were talking about the compiler version there. I know semver is just intent, but the intent doesn't even touch mixing internal data from multiple versions.

If linking against a different version of the code breaks like that, that sounds like someone did semver wrong. If that happens a lot to you, then oh, I'm sorry about that happening.


Every versioning scheme necessarily describes intent, not reality.


This would be the work for Guix. Much better than docker, and exportable to a lot of formats. Or just build a vm from the CLI, an ad-hoc environment, a Docker export or a direct rootfs to deploy and run in any compatible machine.


It’s not a universal problem. A lot of modern languages allow multiple versions of a library to be pulled in to the same code base, through different dependency paths. (Eg nodejs, rust). It’s not a perfect answer by any means, but it’s nice not needing to worry about some package pulling in an inconvenient version of one of its dependencies.

Also, just to name it, it’s ridiculous that a specific graphics card manages to restrict the version of gproto that you’re using. You don’t have this problem with nvidia drivers, since cuda stuff is much less fiddly. AMD needs to pull a finger out and fix the bugs in their stack that make it so fragile like this.


In NixOS, I can install multiple versions of libraries

Or rather, I install no versions of libraries because NixOS will put them all in the store in different folders, and will compile the executable to use the correct path (or patch the elf when needed)

it has an issue with pip because it's allergic to just randomly executing things as part of package management, but pip in general is wtf


Ironically I'm having this problem in a Nix build context because of the broken approach Nix takes to packaging bazel—which itself is largely a consequence of the larger issue I'm grouching about here: unbundling tensorflow's locked dependencies is very hard to do when the underlying source is written to assume it's only targeting the exact version specified in the build rules. You can't just switch it to target the gproto in nixpkgs because then you get compilation failures.


That's trivial with Guix.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: