I enjoy diagnosing concurrency issues and thinking about what principles I could use to avoid similar issues in future.
Two principles that are relevant to this example are:
1. Reduce the number of places a lock is acquired.
2. Reduce the scope of locks.
(1) would suggest two potential improvements:
a. broadcastToStream shouldn't take the lock at "First, we check if we have a particular stream, if not, we return.". Leave the stream existence check to the go pool. This would have avoided the issue but could cause more contention on the GoPool in the case of unrecognised stream names, which might not be acceptable.
b. Don't acquire the lock in the go pool function within broadcastToStream. Rather, find the list of clients within broadcastToStream, and refer to that within the Go pool function. This also might not be acceptable, depending on hoe the lifetime of client and hub related: clients could receive messages after an unsubscribe.
(2) would suggest the fix you made.
That said, hindsight is 20:20 and I expect I'd make similar bug.
> Lisp guys have nothing to sell you but probably the software they create. And if they have a powerful tool, they are not that interested on you knowing their secret.
This level of claim is indistinguishable from fiction.
> I really like the Common Lisp world. *I would like it to be more popular, but at the same time, it is a differentiator for us*.
they reached to us so we add Kina on awesome-lisp-companies. But how many more think the same? And still, they evidently put little effort in making CL more "popular", because they have no interest. They found enough open-source libraries, they work on a hard problem, they form their developers and operators in-house, they develop their own Lisp-like language for the browser… and they silently use the power of SBCL.
> Although I like the spirit of this idea, I don't know that I could trust it in practice as it will be companies like Google, Amazon, and Microsoft that own the actual hardware.
This is the point that the author makes in the second part of their post. Quote:
"The dutifully critical part of me wants to shout: you shouldn’t trust these slabs! Their operators, G — and A — and M — and the rest, will surely betray you. The very signature of the corporate internet is the way it slips from your grasp. The leviathans swim off in pursuit new markets, and what do they leave you with? Deprecation notices."
Yup, it's kind of the analogue of: "Not your keys, not your coin"
So, something like: "not your chips, not your code"?
This also really reminds me of the absolute squandering of resources by "modern" systems - old x86 DOS systems with 1/16,000 the memory and processing power were literally more responsive for everyday use than the mountians of framework junk we run on today. There is a lot of headroom left to downsize and create very useful machines.
I misread this as “the operators G, A, and M” rather than “their”, and thought that this was going to be about some like, combinators of some kind allowing for expressing something about what computation (on what data) you want to outsource the running of run, in terms of 3 operators.
Like, some sort of verifiable computing kind of deal.
Because I hadn’t read the quotation carefully (as otherwise I would have caught the part about depreciation notices before reading the article).
I see this trend too: the systems powering the modern world are increasingly featureful, complicated, centralised into a few hands, and this will likely continue.
I'm a keen developer and user in this world, and recognize the vast users this world provides for.
At the same time, I appreciate a back-to-basics approach that emphasizes systems that can be understood and controlled by individuals and small communities:
* Hardware that has open specs: Pinephone, Pinebook, Raspberry Pi, ...
* Open source OSs: Linux desktop, Linux mobile, Lineage OS.
"... by individuals and small communities" almost invariably means "by experienced developers who want a playground."
Slab vs cloud is a non-issue. The real issue is technocracy vs humanity.
Currently we have no human computing of any kind. Non-experts have two choices: being monitored in as many different ways as is practical in order to be carpet-bombed with targeted ads and (increasingly) fake news. Or being forced into endless tinkering with opaque systems that sort-of work some of the time, maybe, and require expert knowledge for installation and configuration.
That's it. There is nothing else on the table. It's one or the other - and often both.
So when I read a phrase like "Google's largesse" I'm not sure what the point of the article is.
There is no largesse. And there's also no real choice for most users.
The independent dev community could change this, but it seems permanently attached to the wrong end of the telescope, looking at computing from the comfort of its tool- and toy-making treadmill.
"Why should an ordinary user care about this?" isn't asked nearly as often as it should be. And "Don't you understand the tech is fun to play with?" is not the right answer.
Many of the biggest innovations in computing happened because someone asked that question. For some reason the entire industry seems to have stopped asking it.
Except when there's an obvious possibility an answer can be monetised. And while that's certainly a reason, it's not necessarily the best reason.
I see your point, but I have some half-formed doubts. I apologize for a possibly incoherent reply.
> The independent dev community could change this, but it seems permanently attached to the wrong end of the telescope, looking at computing from the comfort of its tool- and toy-making treadmill.
In defense of the tool makers: the reason corporate IT can cater to regular users so well is because they can throw a lot of warm bodies at the problem. The tools we use in this industry are shit, but it doesn't matter when you can use hordes of developers as a protein substitute for better tooling.
I believe the road for "independent dev" software usable by masses starts with better tools, and better tools for making tools.
Additionally, I think "most of the biggest innovations in computing" actually happened because of toy-making treadmill. Even in the startup world, a common advice is to scratch your own itch - it often leads to something that's widely useful.
> "Why should an ordinary user care about this?" isn't asked nearly as often as it should be. And "Don't you understand the tech is fun to play with?" is not the right answer.
In defense of the "independent dev community": perhaps we care a little bit too much about ordinary users? The way I see it, most modern software is dumbed down, lowest-common-denominator toys, whose sole purpose is to sell well and/or sell their users out. Tech-savvy people, and even less savvy users who care about getting things done, are now considered a niche too small to care for. While there are some businesses still working on "power user" tools, the platforms themselves - operating systems - are being optimized for unsophisticated users, dumbed down and locked down.
Again, I believe most of the biggest innovations, the ones helping everyone, start with engineers scratching their own itch. To the extent it's becoming harder, all users lose out.
> Again, I believe most of the biggest innovations, the ones helping everyone, start with engineers scratching their own itch. To the extent it's becoming harder, all users lose out.
I really disagree. I think engineers scratching their own itch leading down to trickle-down tooling is what a lot of independent devs _like to believe_, but that it's motivated mostly by self-importance. I think the utter failure of the FOSS desktop is proof that devs are motivated to work on things they find fun and that these things do _not_ necessarily translate to things that general users want to use.
> Even in the startup world, a common advice is to scratch your own itch - it often leads to something that's widely useful.
I think that's bad advice. That's the _kind_ of advice that leads to things like the hundreds of now-dead clothes washing startups or valet parking startups. And while endless VC rounds blunt this, at least startups have some form of market pressure to have people use their software.
> In defense of the "independent dev community": perhaps we care a little bit too much about ordinary users?
Not really. Software devs are the last set of STEM-engineers that still insist on understanding _everything_ and holding entire systems in their heads. Civil Engineers don't start by considering the subatomic forces that hold their materials together; automotive engineers don't understand every aspect of the software and combustion reaction that goes into their designs. Most engineers accept abstraction as a cost for building useful things.
> The way I see it, most modern software is dumbed down, lowest-common-denominator toys, whose sole purpose is to sell well and/or sell their users out.
"Sell well" is just a euphemism for "software that others use". Money is just the easiest metric to calculate for software being bought and sold, but metrics like "downloads per month" are just as impactful.
Just like you didn't buy a new car and then spend days learning about how it works, most users of software don't want to either. That's not to say that there isn't a robust scene of modifying cars or building hobby cars, but that most people who drive cars for utility purposes don't care to pierce the abstraction veil of an automatic transmission, a brake pedal, and power steering. Most software users just want software that gets out of their way or enables to connect with others in novel ways. They don't care about how much energy their software uses (as long as it's affordable) or how "simple" it is or whether it uses Unix sockets or DBus or something.
> I really disagree. I think engineers scratching their own itch leading down to trickle-down tooling is what a lot of independent devs _like to believe_, but that it's motivated mostly by self-importance. I think the utter failure of the FOSS desktop is proof that devs are motivated to work on things they find fun and that these things do _not_ necessarily translate to things that general users want to use.
I'm not a desktop environment developer, but I can't imagine it's especially fun compared to other projects. The bigger projects (Gnome, KDE, etc) are definitely being run for instrumentalist reasons, and are supported to quite a degree by companies with a business interest in having them.
Thinking about this reminded me of a highly insightful comment by user Floegipoky[0]:
> By mimicking the Apple and Microsoft tactic of constructing vast monolithic environments and applications, you have all unwittingly been playing to their strengths, not yours. Such enormous proprietary companies can afford such brute-force strategies because they have vast financial and manpower resources to draw on.
> Projects like Gnome and Open Office become like our banking industries: vast, baroque, impossible to regulate effectively, and cripplingly expensive to maintain.
Doesn't exactly sound fun — and whose itch is being scratched by working on these?
> the Linux desktop world (and even the kernel world beneath it) has completely and utterly forgotten its roots. Unix Philosophy isn't merely a neat marketing phrase: it describes a very specific way to construct large, complex systems. Not by erecting vast imposing monoliths, ego-gratifying as that may be, but by assembling a rich ecosystem of small, simple, plug-n-play components that can be linked together in whatever arrangement best suits a given problem.
> "... by individuals and small communities" almost invariably means "by experienced developers who want a playground."
I acknowledge this.
> "Why should an ordinary user care about this?" isn't asked nearly as often as it should be. And "Don't you understand the tech is fun to play with?" is not the right answer.
I have spoken to friends and family and they are perfectly happy with their mainstream computing choices.
I acknowledge that my preference for back-to-basics is partly due to fun. I'm not trying to change minds. I enjoy this small paradise for hackers.
I enjoy diagnosing concurrency issues and thinking about what principles I could use to avoid similar issues in future.
Two principles that are relevant to this example are: 1. Reduce the number of places a lock is acquired. 2. Reduce the scope of locks.
(1) would suggest two potential improvements:
a. broadcastToStream shouldn't take the lock at "First, we check if we have a particular stream, if not, we return.". Leave the stream existence check to the go pool. This would have avoided the issue but could cause more contention on the GoPool in the case of unrecognised stream names, which might not be acceptable.
b. Don't acquire the lock in the go pool function within broadcastToStream. Rather, find the list of clients within broadcastToStream, and refer to that within the Go pool function. This also might not be acceptable, depending on hoe the lifetime of client and hub related: clients could receive messages after an unsubscribe.
(2) would suggest the fix you made.
That said, hindsight is 20:20 and I expect I'd make similar bug.