Problem is the CUDA advantage is gigantic and it has been known for years in GPGPU processing, way before AI was a meme. AMD has lost countless developers over the year just on hello world style projects. Developers had a solid 6-7 years of living with OpenCL when the green rival had a very mature and nice CUDA sitting there. I’ve been out of that world for a while now, but it was truly painful and turned a lot of devs off programming AMD devices. Now there’s a big moat of entrenched developers that could take decades to displace. It’s like trying to displace C++ with Java 22 — possible, but it’s a slow, slow trudge and everyone still remembers Java 1.4
No, the amount of code written in CUDA for pytorch could easily be rewritten in CUDA for few million or tens of millions of investment. The problem is that it is damn near impossible to get good performance in AMD. For complicated CUDA programs like flash attention(few 100 lines of code), no amount of developers could write those few 100 lines for AMD to get the same performance.
Even worse: GPGPU is not only about LLM or even ML. It's also for computer vision, signal processing, pointcloud processing, e.g. Opencv has backend for CUDA, open3d, PCL the same. Even apple is kind of worse than AMD regarding ecosystem of libraries and open source high performance algorithms - when I tried to port some ICP pipeline to apple metal there was nothing there, most libraries and research code target only CUDA
While I agree with the sentiment towards CUDA, the example is a bit off, given that C++ basically lost all mindshare in distributed computing, to Java and others, and is hardly visible in CNCF projects landscape.
Displacing C++ in compiler development and HFT/HPC/GPGPU with Java 22, most likely not happening, everwhere else it has been loosing mindshare, the current cybersecurity laws versus WG21 attitude towards them, doesn't help.
