Hacker Timesnew | past | comments | ask | show | jobs | submitlogin

Do you have evidence LLMs can indeed generalize outside their training data distribution?

https://twitter.com/abacaj/status/1721223737729581437/photo/...



No. I know only that they can generalize within it, and only to a limited degree, but don't have solid evidence of even that.


So what you're saying is there's tenuous-at-best, non-"solid" evidence that LLMs can reason even within their training data.

And yet I'm currently sitting at -1 for stating the blisteringly obvious. Lmao


Yes, that's basically what I'm saying. Just less bluntly. It's slightly more nuanced than "LLMs cannot reason" because lines of reasoning are often in their dataset and can sometimes be used by the model. It's just that the model can't be relied on to know the correct reasoning to use in a given situation.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: