I am curious if LLMs are better at some kinds of problems than others. IIRC this and another big recent one were cases of the LLM producing a counterexample to a conjecture.
IMO, it's due to some problems being better documented, with more well-documented, previous research available. LLMs don't really create novel mathematics, they mostly "connect the dots". LLMs by design are not coming up with anything new, unless by statistical probability, aka "brute forcing".
I don't want to minimize LLMs capabilities, it's pretty cool they are doing this, and it's useful from a research point of view. But it's important to set expectations.
> LLMs don't really create novel mathematics, they mostly "connect the dots".
That is not what the mathematicians are saying. I don't have the knowledge to evaluate this myself, but a number of mathematicians - for example, in the SP - are saying it goes further than that - they really do introduce novel ideas. Of course everything is based on and inspired by some previous work, but that is true of all human mathematics as well.
LLMs that have been trained through reinforcement learning on mathematics are NOT simply token predictors. Only base models can be accurately described that way. They have learned how to do mathematics. They have learned to do coding. Its really amazing we're three years into instruct models and such a large part of Hacker News still does not understand the most basic facts about this field.
