They do sell them - but through their struggling cloud business. Either way, Nvidia's margin is google's opportunity to lower costs.
> I can see Google's custom chips are 15x to 30x slower to train AI
TPUs are designed for inference not training - they're betting that they can serve models to the world at a lower cost structure than their competition. The compute required for inference to serve their billions of customers is far greater than training costs for models - even LLMs. They've been running model inference as a part of production traffic for years.
This breaks my brain, because I know Google trains it models on TPUs and they're seen as faster, and if they're better at inference, and can train, then why is Nvidia in a unique position? My understanding was always it's as simple as it required esoteric tooling
They do sell them - but through their struggling cloud business. Either way, Nvidia's margin is google's opportunity to lower costs.
> I can see Google's custom chips are 15x to 30x slower to train AI
TPUs are designed for inference not training - they're betting that they can serve models to the world at a lower cost structure than their competition. The compute required for inference to serve their billions of customers is far greater than training costs for models - even LLMs. They've been running model inference as a part of production traffic for years.