So Apple pivots from one thing that it doesn't have a natural advantage with to another.
Apple has gone from leading the industry to increasingly desperate and huge "me too" developments that they've failed at. AI, generative or otherwise, is going to be their next failure. To say AI moves fast is a serious understatement. I predict that which is white hot now (LLMs) will be a blind alley long left behind in a year or two.
How are Apple going to attract the talent they need to pull of something decent in AI? Their share price can't have the upside of hot startup or even a sleazy non-profit for that matter. I predict they buy something fairly pedestrian from some large player at some point.
Apple is a huge cash cow and with Cook at helm it will remain so but will slowly fade over time and resort even more to extractive behaviours on their customers and suppliers.
In a lot of Apple's hardware, the fastest way to inference is to ignore the built-in Neural hardware and use Metal Compute Shaders. Suffice to say that simply shipping dedicated silicon is not equivalent to accelerating everything well, or supporting it upstream.
SMOB (simply matter of bytes) as they say. Will they finally invest in AI software? My take from this announcement is that they will. Dedicated silicon with a dedicated hardware stack shouldn't be so hard for Apple. They can throw considerable resource at the problem.
It would be great to see, if they pull it off. I suspect that Apple's desire for control and curation (as well as limited liability) will prevent them from making an effective "one size fits all" model, though. Considering that local AI isn't a paid service or a system-seller, it makes more sense for Apple to double-down on software support rather than trying to Sherlock the world with their own model.
Time will tell what the future holds, but I sincerely believe it's too late for Apple to ride the AI bubble. The iPhone is their last stand; they need to deliver a competitive AI experience, ideally one that's open and supports older hardware models. Their work is cut out for them.
I think they'll be training-agnostic (will use whomever's hardware gets it done) but will optimize their silicon and hardware stack for running models locally. This is VERY EARLY days of use-facing AI (besides the Nvidia bubble, which is late-days)
I definitely agree that Apple will optimize their SOC, but IMO their best solution is to just copy Nvidia's homework. Dedicated inferencing hardware is power-hungry and rarely any better than the beefed-up GPU or even a RISC CPU core at a decent clock speed. Integrating AI at the GPU-level lets you get dual-purpose functionality out of the same silicon; it's what Nvidia has built towards since they started shipping CUDA.
> besides the Nvidia bubble, which is late-days
The Nvidia bubble might just be beginning, if we keep relying on phrases like "training agnostic" to pay our dividends.
I do a lot of open source LLM research/dev work on a Mac Studio. While it doesn't quite compete in terms of speed with a GPU for standard transformers models, I can run pretty huge models locally. When I'm working with llama.ccp, the speed and model size I can run is very impressive.
I can also run Stable Diffusion XL in reasonable time frames on an iPad Pro. The current gen Macbook Pro can perform almost as well as the Mac Studio with an M2 Ultra, only the M3 Max has about 1/2 the bus speed (though still wild that you can run good sized local LLMs on a laptop).
If local generative AI becomes a major part of computing in the future, Apple has a huge advantage over the other players out there. This was obvious the second I started working on my Mac Studio. I have spent plenty of time using a traditional GPU setup for LLM work, and yes it is faster, but the complexity of getting things running is way beyond the average user's ability. Not having to fight with cuda ever is amazing, and so far everything else has 'just worked' as is typical of Apple.
If Apple has a team of talented people working to get gen AI performance tuned specifically to their hardward, I suspect we'll see some very competitive offerings in this space.
I feel like there's going to be a lot of movement towards the CPU with AI compute, and Apple's processors show the possibilities.
GPUs happened to have a lot of throughput lying around so they got put to work, but already the importance of having lots of memory to hold huge models even just for inference is clear. I also think the future AI will have a lot more going in 'conventional' compute rather than just large arrays of simple tensor ops or the like.
CPUs will increasingly gain specialist hardware to accelerate AI workloads, beyond what we have now and less monolithic too, in that it'll probably have a variety of kinds of accelerators.
That will combine well with big main memory and storage that is ever closer to the CPU to enable very fast virtual memory. I wouldn't be surprised if we soon see CPUs with HBW storage as well as HBW memory.
> CPUs will increasingly gain specialist hardware to accelerate AI workloads
Maybe, but then you're describing a coprocessor instead of the CPU. The CPU portion of the M1 SOC should be simpler than Apple's Intel processors, considering they don't support the wide bevvy of AVX/SSE instructions in-hardware anymore. The goal of the ARM transition is to keep the CPU side as simple as possible to optimize for power.
Personally I think we're going to see more GPU-style accelerators in the future. People want high-throughput SIMD units, ideally with a good programming framework a-la CUDA to tie it together. It makes very little sense to design dedicated inferencing hardware when a perfectly usable and powerful GPU exists on most phones. It's practically redundant to try anything else.
Yeah, the consensus here seems to be that this is a good move because the car was a bad idea... but it doesn't follow that AI is a therefore a better idea? Apple's poor reputation in ML related areas not withstanding (I've heard its a lot of bad infra and silo'd development), I don't see how they will make money from this.
Siri is fine for sending a text or setting a timer, but I don't think most people are going to pay money for an "AI assistant" to plan vacations or make restaurant reservations, if for no other reason than most people aren't busy enough for it to be worth it?
> My guess is they'll try to turn VisionPro into some kind of generative gaming system.
I'd speculate they can't do this without 5-10x the compute, which means a wildly bigger battery. I can't think of how you'd mash VR gaming and generative AI together into a headset form factor in a way that's practical for gaming. We can _only just_ run image models in almost real time, and that still requires some heavy lifting and compromises on quality. Doing generative AI on-device for a gaming experience is a hugely tall ask. (And if you're doing it in the cloud, why not do it for the simpler, more popular hardware instead, like iPads?)
The problem with Siri (and with Alexa) is that Apple was not able to provide enough value for people to make it mainstream. Alexa at least worked for home appliances but using Siri for Macbook or iPhone? Why? Not to mention Apple was not able to monetize it.
When driving I use Siri to control Spotify and to call people. At home I use google home every day to know the weather, to set alarms and to play music. I'd use Siri for those functions but it responds way slower than google home.
Siri being terrible is thrown around thoughtlessly, you should try it. It's actually pretty good these days. It's shocking how many things it can respond with actual answers.
I use Siri all the time for really trivial things like settings alarms/timers. Any time I try to do something slightly more complicated, like maintaining a shopping list, it gets terrible pretty quickly.
> Apple has gone from leading the industry to increasingly desperate and huge "me too" developments that they've failed at.
Apples biggest advantage is having piles of cash to spend on R&D.
Developing new products that it doesn’t have much previous experience with is how it’s gotten this big to begin with. That’s what the iPod was, that’s what the iPhone was, that’s what Apple TV was, that’s what the Apple Watch was, that’s how it became such a good chip designer, etc…
The difference between now and 2001, is that it’s got so much cash today that it doesn’t need all, or even most, of its new R&D to succeed, so it can invest in more less-potentially-viable R&D, and it’s not such a big deal if it never gets to market. Which seems like the opposite of desperation to me…
I predict they are going to use whatever Google provides (Gemini) and asks Google to pay them money to make it the default. Siri powered by Gemini. Why develop a technology yourself when you have enough market clout to let others pay you to use their tech? And Google, desperate for market share and mind share, might just agree to that while paying Apple a tidy sum (probably not a whole lot).
Apple is one of richest companies in the world, already has chips most competent for AI in any widespread portable devices, and smaller teams like the one at Mistral can create something as competent as the 7b model without backing from the largest players, ie. if Apple wants it they could buy up a lot of talent create something wild.
Also Gemma is a complete joke, and gemini is still not on par in my view.
If they drop the ball it's organisational - like the way they have ignored any documentation for the Neural Engine and just started with MLX now, Ferret etc. But they are moving!
Apple has a massive war chest of cash and the equity they give people is liquid. I think you're really underestimating their ability to attract talent and pay people.
Have you read the headlines about how difficult and expensive it is to procure NVIDIA GPUs?
Apple has an enormous hardware advantage over Google, Microsoft, and even OpenAI in the space if the constraint is hardware. If software ends up being the "easy part" and hardware remains difficult, Apple is in a great spot between the volume of hardware they're already moving and their cash reserves.
> Apple has an enormous hardware advantage over Google, Microsoft, and even OpenAI in the space if the constraint is hardware.
The constraint isn't hardware. Google, Microsoft and OpenAI are buying datacenter-scale Nvidia hardware that Apple quite literally does not ship the equivalent of. There are no Macs that support DGX-sized workloads, and attempting to make one seems like a suicide mission a-la Xserve. Nobody is buying what Apple's selling, in the server market.
Apple and Nvidia both compete over new TSMC nodes, and with Nvidia reaching a trillion (!!!) dollar valuation you'd be foolish to assume they're not outbidding Apple as we speak. All things being equal, Nvidia is probably Apple's greatest threat in pure compute-per-watt: https://browser.geekbench.com/opencl-benchmarks
The biggest limitation and expense for most players looking to get in on the generative AI land-grab is hardware, which Apple already has completely solved. If anything, they’re already ahead of everyone else in the game. The “me too” thing is nothing new. They’re just following the classic Apple playbook and biding their time to wait for the market to solidify before unleashing the best user experience in it.
If they can make a voice assistant that runs on-device I'd pay good money for that. Now that a lot of LLMs are able to run (slowly) on raspberry pi's, they could get a great following by just not stealing data and not having their assistant get very noticeably worse over time.
Google assistant has completely dropped the ball from a user perspective, though I'm sure they're capturing great data and have reduced server costs 90%.
Apple has gone from leading the industry to increasingly desperate and huge "me too" developments that they've failed at. AI, generative or otherwise, is going to be their next failure. To say AI moves fast is a serious understatement. I predict that which is white hot now (LLMs) will be a blind alley long left behind in a year or two.
How are Apple going to attract the talent they need to pull of something decent in AI? Their share price can't have the upside of hot startup or even a sleazy non-profit for that matter. I predict they buy something fairly pedestrian from some large player at some point.
Apple is a huge cash cow and with Cook at helm it will remain so but will slowly fade over time and resort even more to extractive behaviours on their customers and suppliers.