Actually, open source allows something that is not possible with commercial software. You don't have to read all of the code and understand all of the implementation details, but you have that option if you need to. I think having the option is a great benefit as we can see from this example. When they started debugging they could reference the express source and figure out what was going on. If they were using some sort of commercial framework they would have had to either refer to the docs or call the help desk.
Just because you have the option to read the implementation details of every library and service you are using doesn't mean that you have to. You only have to learn enough about it to decide whether you think it is a good addition to your stack, and to use it to do whatever you are trying to get done. But open source gives you the ability to figure out why you're using it wrong, or how it is broken when that time comes.
If you are saying that open source is not documented well enough because the developers fall back on "check the source," that is a different argument where commercial software may be better, but this is not true for any of the more common open source software I've used.
Making a decision of which software to depend on in your application is something that is always difficult whether you have access to the source code of all of the choices or not. It's a decision that you make with limited information. You only have extensive knowledge of the tools that you already have experience with, so using alternatives is always a risk, but understanding and managing that risk is part of the developer's job.
And with regard to keeping up with changes, you always can remain with previous versions for some time to avoid the slowdown associated with shifting to new APIs.
I agree with you that the greatest strength of open source is that you can just go read the code and (optionally) fix it. Netflix did this and it got them past their crisis.
I think you missed my point though, which was that there isn't any sort of fitness function being applied to open source. In the closed source environment that it 'price' (caveat walled gardens). By paying for the software customers "vote with their wallet" on the things they like/want/use. What is more if two folks are putting out the same capability they are incentivised to compete on a vague sense of value. That creates a 'culling' function which operates independently of the software creators (the only way to sell more is to be "better").
Open source doesn't have that forcing function, there isn't really even a reputation function involved where bad FOSS would give folks a bad reputation that a user could pick up on (yes we probably all know one or more committers who are jerks but that isn't quite the same thing). And there is no barrier to entry so we get multiple variations on the same conceptual solution (I've lost count of the number of CMS's or blogging packages there are out there for example).
Finally there is the various bits of entanglement, you wrote "And with regard to keeping up with changes, you always can remain with previous versions for some time to avoid the slowdown associated with shifting to new APIs." but that has not been my experience. As you're evolving your product interdependencies in the APIs and packages you are using force them to upgrade as well. There is no island of stability in the FOSS world (well maybe OpenBSD servers or something but I have not seen them in the 'stack') Backports only go on so long and suddenly you're no longer getting updates from your source as Ubuntu showed when they dropped updates for 12.x LTS. All those systems still shell shockable or heart bleedable, or something else. Because to upgrade that part, needs to upgrade the next part, and so on and so on until you've upgraded everything. And you pay that price again and again and again and again.
Much of this is just different than the other model which is not necessarily a bad model. But from a systemic point of view it is hard to partition the work and that leads to inefficiencies. A single change in an API implementation which results in dozens of engineering groups re-implementing stuff, versus a migration model that allows people to move over gradually. The promise of open source is "no capital expense" but I worry the down side will become "much higher operational expense". And when the opex cost of open source is > than the capex + opex cost of closed software, then closed software wins. Of the folks who have tried to prevent that equation from shifting in favor of closed source, I only see Redhat as a success story.
There totally is a fitness function for open-source. People who find that an open-source program creates more hassles than it solves don't use it. Folks who blog about their hassles induce other people not to use it.
There are a number of open-source projects - web.py, Mongo, Angular, Ember - that I am actively avoiding because IMHO their benefits to me don't outweigh risks or previous hassles experienced. Much better to use tried-and-true open source projects like MySQL, Postgres, vanilla JS, and Django that I have had good experiences using. There are many others like Polymer or Docker that I am intrigued by, would love to give a try, but am sitting on the sidelines until other people uncover the most obvious pitfalls.
Economically, open-source projects are just startups where the monetary cost of use is zero. That doesn't mean that the total cost of use is zero, and as you're evaluating potential products, you need to be careful to figure in time expenditure spent debugging as a cost. But you are incredibly naive if you believe that just because you are paying a million dollars for a piece of software, you will spend less time debugging and integrating it than a well-used piece of open-source software. Hell, I once was the college intern writing that million-dollar piece of software. All it takes to sell something is chutzpah.
People who find that an open-source program creates more hassles than it
solves don't use it. Folks who blog about their hassles induce other people
not to use it.
Would that it were so, but people still use mongo. You seem to be positing perfect knowledge, whereas crummy software seems to creep in places either because of marketing or because it's the default in some system.
Docker, btw, is awesome. You can use it to replace virtualenv (which only handles python libs) and system lib isolation in a much more scriptable, more robustly deployable fashion. So you can take python projects developed at different times that use different revs of packages (and those packages can require different libs) and make deploying them way better. Try it, it's just awesome.
The same problems with imperfect knowledge exist with closed-source software too, but they're worse because you can't examine the source yourself and the vendor often has an incentive not to let flaws come to light.
> People who find that an open-source program creates more hassles than it solves don't use it.
But bad open-source software can live on to upset the productive lives of others forever, as the distribution cost is essentially free. With bad closed-source software, if the producer goes out of business (without a buyer), typically it becomes unavailable.
This is both a feature and a bug. It is definitely caveat emptor; you should do your diligence before pulling in any piece of third-party software. However, there are many, many critical systems that incorporate unmaintained open-source libraries. libxml, for example, gets a bugfix release about every year, while TeX has been asymptotically approaching version pi since 1989. That's fine; those libraries are stable, they do what they do, and I'd much rather have them available than have them disappear like closed-source alternatives.
This mirrors my experience as well. Mature well known open source is usually excellent quality. Many of the more "modern" packages are often a lot more hassle.
Just because you have the option to read the implementation details of every library and service you are using doesn't mean that you have to. You only have to learn enough about it to decide whether you think it is a good addition to your stack, and to use it to do whatever you are trying to get done. But open source gives you the ability to figure out why you're using it wrong, or how it is broken when that time comes.
If you are saying that open source is not documented well enough because the developers fall back on "check the source," that is a different argument where commercial software may be better, but this is not true for any of the more common open source software I've used.
Making a decision of which software to depend on in your application is something that is always difficult whether you have access to the source code of all of the choices or not. It's a decision that you make with limited information. You only have extensive knowledge of the tools that you already have experience with, so using alternatives is always a risk, but understanding and managing that risk is part of the developer's job.
And with regard to keeping up with changes, you always can remain with previous versions for some time to avoid the slowdown associated with shifting to new APIs.