I kinda agree with the main point, but keep in mind those libraries with SIMD optimizations don't just appear out of nowhere... people write those. Also it's pretty common for someone to write software for an org that thas 10^5 or more identical cores running in a datacenter (or datacenters)... some specialized optimization can easily be cost-effective in those situations. Then there's crazy distributed systems stuff, where a small latency reduction in the right place can have significant impact for an entire cluster. And on and on....
Point being, while not everyone is in a position this stuff is relevant (and not everyone who sometimes finds this stuff relevant can say it's relevant often), it's more widely applicable than you're suggesting.
For sure there are obviously developers building those computation libraries like numpy, compilers, R, and so on. These people exist and are grinding out great code and abstractions for the rest of us to use, and many of them are regulars on HN. But these people are seldom the target of the "learn SIMD" content that appears on here regularly.
If someone is an average developer building a game or a corporate information or financial system or even a neural network implementation, if you are touching SIMD code directly you're probably approaching things in a less than optimal fashion and there are much better ways to utilize whatever features your hardware, or hardware in the future, may offer up.
This is not entirely accurate... think about it this way. Every time you issue a floating-point addition or multiplication, you're using an 8th or a 16th of your CPU's theoretical performance. Of course, it's a bit more complicated than that, but that's the general gist of it. Compilers won't generate SIMD code for you (autovectorisation) except for the simplest cases, and they certainly won't do the necessary transformations (AoS to SoA or AoSoA) necessary to efficiently use SIMD.
Now of course, many of these transformations can be wrapped in a higher-level API (think of sums over an array, reductions, string length, string encoding, etc.) but not all of them.
Of course, multithreading also exists to improve performance, but for many tasks, it's more worth it to run it on one core without the sync overhead, especially with data-parallel algorithms where you're doing the exact same thing on all of the data and you have a fairly large dataset. Or even better, you can combine the two, partition the data between multiple cores into a few small sets then use a SIMD "kernel" to process it. With extremely embarrassingly parallel problems, you can achieve 1000x speedups this way, not exaggerating. A typical speedup is much smaller but still, using your machine well can easily produce an order-of-magnitude difference in performance. If you read up on ISPC benchmarks, you'll find that for even any kind of existing, very branchy and not SIMD-friendly code, they regularly had a free 4x speedup without changing the behaviour or the result of the program.
Seriously, it's not that if you use SIMD-powered libraries then you're set for performance, if you have a holistic view of your system's performance, you can do really amazing things.
I'm not advocating against using vectorization/SIMD. Of course people should eek every bit of performance out.
The point is that the average developers job in achieving this is not knowing every SIMD primitive for every platform. It is almost always folly to ever touch a platform's SIMD instructions unless you're one of the aforementioned library/compiler developers.
Instead the ordinary developer is usually best served by using equipped libraries and understand how to structure code where it can enable vectorization. Whether it's Eigen, or even the C++ compiler (which, in my experience, is often excellent at vectorizing code), or the Accelerate library on Apple platforms. These things mean that your code can adapt and exploit every possible platform they're deployed to, versus Jimmy SIMD who carefully made his SSE 3.2 instructions and his code was then lost in time.
I agree. Most developers don't need to write SIMD code. Some do however and those need good documentation to write good SIMD code, even more so when it comes to porting.
Interesting that you mention Eigen. Long before I started my company VectorCamp, wrote the original Altivec/VSX, Arm and Z ports for Eigen and it took me a lot more to do a proper port back then -iirc I started that effort in 2008- than it would take me now, because now the tools are far far better. I started this company to provide SIMD optimizations for all architectures and this tool SIMD.info began because I wanted to help other developers find the information that I wish I had back then. It's that simple.
For me and my company, anything that needs to be performant critical to be written in a compiled language like C, C++, Rust -and now Zig- is worth optimizing. How much depends on your time, money and skills. Not everything needs SIMD of course. But it's definitely NOT only for library and compiler developers. You would be surprised how much SIMD code is even in application code out there. It all depends on the expectations and the required performance. Also, some of the libraries don't utilize all available instructions.
Point being, while not everyone is in a position this stuff is relevant (and not everyone who sometimes finds this stuff relevant can say it's relevant often), it's more widely applicable than you're suggesting.