It is interesting to look back and evaluate the preferences / intuitions prominent researchers had in the field (most of whom started their careers experimenting with MNIST-scale of data, at best)

With access to unfathomable amounts of data, especially over the last couple of years, the game changed entirely and is not seeming to cool down anytime soon.

The field, certainly, values engineering a lot more than it used to, and it is exciting to see how major advances together with open-source contributions are going to take us

> It is interesting to look back and evaluate the preferences / intuitions prominent researchers had in the field (most of whom started their careers experimenting with MNIST-scale of data, at best)

Look back to how HN greeted the victory of the SuperVision team (Alex Krizhevsky, Ilya Sutskever, Geoffrey Hinton) in the 2012 ImageNet Large Scale Visual Recognition Challenge - https://hackertimes.com/item?id=4611830

We have been doing feature engineering, then we got into architecture engineering, and the future is dataset & prompt engineering. All models learn more or less the same, given the same training budget and dataset. But better data makes a better model.

Same for humans, the better the education and more advanced the science, the more we achieve with the same brains. The key ingredient is ideas, both for humans and AI.

Exactly. Listening to Marc's discussion about this reminds me of him wanting to jump into the space and see this calculation playing out in his head where cheap-money >> Adam's self-dealing. No surprise this deal happened right at the time when the high-interest environment was starting to kick in.

If this scales up, it can be thought of as "actionable Google search", and if taken to extreme, has the potential to make internet query-able for better or worse