On the other side of the AI-reasoning fence, the disdain seems to be mutual. “These ‘scientific’ papers from last summer — I would put this in big, big air quotes,” said Sébastien Bubeck, a member of OpenAI’s technical staff (and a prominent evangelist for the company’s reasoning models among scientists and mathematicians). He called earlier Apple results critiquing AI reasoning “wrong,” claiming that they were due to a training quirk in models that are now obsolete. “Modern models starting with GPT-5.5 do not suffer from this issue,” he said. “It would be interesting to revisit those results.” (Apple did not make its researchers available for interviews.)
Then, later:
The “think” part is what OpenAI, for one, is doubling down on. When I asked Bubeck if the splashy unit distance proof was produced with methods outside the LRM’s own chain of thought — perhaps with Lean verifying its results — he seemed to find the question almost nonsensical.
“It’s not like we’re making a mystery of it,” he said. “We have released the chain of thought. You can just go and look at it. The whole point is that the model is reasoning like a human would. And when humans reason, we don’t use Lean.” Technically, OpenAI released a “rewritten summary” of the model’s chain of thought produced by two human experts using Codex, another OpenAI model. Since 2024, the company has not publicly revealed “raw” chains of thought from its reasoning models, a policy also adopted by Google DeepMind and Anthropic.
That "training quirk" thing is obvious (yet unfalsifiable) BS, and who the hell is he to sneer about "science" when his company won't release the raw data for independent scientists to look at?
I have no idea what Bubeck meant, and I agree about OpenAI's hypocrisy, but the problems with that infamous (and non-peer-reviewed) Apple preprint were the nature of the tasks (insanely repetitive), the fact that simple coded solutions were not novel, and the automated assessment occurred without a human in the loop.
Most models in that study appear to have "failed" by offering a Python code solution to generate the repetitive assessment steps, rather than just mindlessly copying those steps out. This was discussed at length at the time.
None of this means that models "reason", whatever that means - but simply that the Apple study was not useful evidence either way.
I got the same impression as Bubeck that the ‘scientific’ papers were a bit more like blog post saying this LLM got stuff wrong so LLMs can't reason, but as he says they can now do that so it's not an inherent limit of the technology, just the 2025 versions weren't up to it.
