> In the company’s words, Galactica “can summarize academic papers, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more.”
The first step to successfully publish prototypes is creating realistic expectations. That's being done all the time in papers and other ML projects. Instead Meta listed a set of features in a language model that can be summarized as "magic".
GPT-3 can already most of those things. If you haven’t spoken with GPT yourself, you might think it’s overhyped. But it really is quite amazing. I’m saying this as a 15-year veteran machine learning engineer who is not impressed by much in ML. The “hype” part of what’s happening (the disconnect between reality and the goal) is that these models do make silly mistakes. But they also can actually perform the operations on the list above, and not in some “works 1% of the time” sense. This kind of AI is at a turning point, becoming deeply impressive and significantly expanding what code can do.
I think it’s overly generous to say that GPT-3 can “do” any of those things. It can generate stochastic nonsense that can be shoehorned into the approximate shape of those things.
I wrote the software that uses GPT-3 to produce those. You can look at the papers to judge the accuracy for yourself.
2. Solve math problems:
What is the area of a circle with radius 2?
The area of a circle with radius 2 is 4π.
3. Generate wiki articles:
Write the first paragraph of a wiki article about tigers.
Tigers are the largest members of the cat family. They are apex predators, primarily preying on ungulates such as deer and bovids. They are territorial and generally solitary but social animals, often requiring large contiguous areas of habitat that support their prey requirements. This, coupled with the fact that they are indigenous to some of the more densely populated places on Earth, has caused significant conflicts with humans.
4. Write scientific code:
Write a matlab function to plot the largest three singular values of a matrix.
function plotLargestSingularValues(A)
s = svd(A);
plot(s(1:3));
I'm not trying the others because I think they're more specific to Galactica.
The models, including GPT-3, absolutely do make mistakes. We are in the first few years of these capabilities existing at this level, so much research is needed before the results are consistently good. In my experience, though, GPT-3 is extremely useful and reliable for many use cases. (Building and using tools like this is part of my career.)
The first step to successfully publish prototypes is creating realistic expectations. That's being done all the time in papers and other ML projects. Instead Meta listed a set of features in a language model that can be summarized as "magic".