I wish the authors of the article spent more time studying how LLM works before writing a doomsday scenario. Currently no matter how impressive LLMs are, they are just token auto complete machines. It cannot invent or think anything new which is not in its training dataset.
Do you even know what Cycle Double Conjecture is? Or what even a conjecture means in general? It’s a perfect use case for autocomplete with existing knowledge
One of the lead authors, Daniel Kokotajlo, worked at OpenAI for years before quitting. In 2021 he wrote a remarkably accurate forecast of how LLMs would develop over the following 5 years[0].
I think it should be obvious that he understands that LLMs are trained via next-token prediction.
