"LLMs in the GPT family have read every programming language in the world, billions of times. It's been known for some time that these agents can write small snippets of code with some limited success. However, to my knowledge, none of these agents have ever been asked to make their own programming language. This type of task is a bit more complex, and requires a bit more creativity and foresight than much of the internet believes LLMs to have.
Some folks on the internet tend to think lowly of LLMs as merely "Stochastic Parrots" -- simply "remixers" of old ideas. In a way, they're right; but I'd argue that remixing is the fundamental force of creativity. Nothing comes from scratch."
This^^. Half the internet was in denial about how good LLMs are. I wonder why? Perhaps it's the way humans act when confronted with a machine that can potentially replace much of what makes us special.
The advent of chatGPT had millions of people on the internet trying to downplay the intelligence of chatGPT by continuously trying to re-emphasize the things it gets wrong.
I think the people who claimed chatGPT was a stochastic parrot are now realizing that they were the ones that were part of a giant parade of parrots regurgitating the same old tired trope of LLMs being nothing but simple word generators. Well you guys were dead wrong, a simple iterative improvement on the same algorithm pretty much ironed out a lot of the issues.
I'm guessing that at GPT-10 or 11 we'll have produced something that when compared with humans, humans will be the things that are far more parrot-like.
To me, it's very clear it doesn't really understand what "indentation-based scoping" means. For a human programmer, using both intentation-based and bracket-based scoping is a very unusual design choice, and is begging for further explanation. It talked pages of abstraction, modularity, etc, but not this?
I'm not saying it's not good. But I failed to see how this particular example proves its not a "stochastic parrot". If anything, I think it's a quite strong evidence supporting "stochastic parrot" narrative.
Author here -- I think it is a stochastic parrot, don't get me wrong.
But my argument is that humans are also, at some level, sufficiently advanced stochastic parrots. At least, as it pertains to many creative endeavors.
You're always building on the back of something that comes before.
We're all just remixing ideas we've heard before.
I mean even this comment -- none of the words I'm saying are new, and many of the word combinations I'm using have been used time and time again. The ideas I'm expressing have merit -- but are they wholly original?
Not really. We all need eachother's creative energies to do our best work.
The difference between humans and sophisticated stochastic parrots is reason. The most common kinds of mistakes that chatGPT currently makes are when it says things that are not simply wrong, but don't make sense. Perhaps it will be possible to emulate reason with enough data, training, parameters, etc, but without some representation of the ability to understand what you don't know, what you know, and what follows from those things consistently, I wonder if these kinds of models will ever become truly reliable.
Although even in this example it does say some things that are simply wrong:
"Some programming languages like TypeScript and Swift use a colon for both variable type annotations and function return types."
This is incorrect for Swift, which uses the same arrows as "TenetLang" for function return types. Actually the first thing I thought looking at the example code was "looks swifty but not quite as well designed."
> some representation of the ability to understand what you don't know, what you know, and what follows from those things consistently
Right, that is truly what the chatbots are lacking. They can fool some of the people some of the time, but they can't fool all of the people all of the time.
Their creators - not the chatbots themselves - are trying to "fool" people into believing that the chatbots would have as you say "ability to understand what you don't know, what you know, and what follows from those things consistently".
It's totally possible that humans don't do reason. It's possible that the parrot in our brain makes the decision, and then the "frontend" of our mind makes fake reasons that sound logical enough.
But it's just a possibility, and I don't find it's particularly convincing.
Look up split brain experiments [1]. Basically, in patients where the corpus callosum is severed to some degree, the two halves of the brain have limited communication. Since the two halves control different parts of the sensory system, you can provide information selectively to one or the other half of the brain, and ask that part of the brain to make choices. If you then provide the other half of the brain the wrong information, and ask it to reason about why it made a choice, the other brain will happily pull a ChatGPT and "hallucinate" a reason for a choice neither that half nor the other half of the brain ever made.
While that does not prove that we never apply reasoning ahead of time, it is a pretty compelling indication that we can not trust that reasoning we give isn't a post-rationalisation rather than an indication of our actual reasoning, if any.
No but that does happen a lot of the time. The difference is that we can choose to engage the deductive engine to verify what the parrot says. Sometimes it's easier to do so (what's 17×9?) and sometimes it's harder (a ball and a bat costs $1.1 and the bat costs $1 more than the ball, what does the ball cost?)
You can ask ChatGPT to show it's working too. It's likely, for many things, (as it is with humans a lot of the time) that the way it does it when it walks you through the process is entirely different to how it does it when it just spits out an answer, but it does work. E.g. I asked it a lot of questions about the effects of artificial gravity using a rotating habitat a few days ago, and had it elaborate a lot of the answers, and it was able to set out the calculations step by step. It wasn't perfect - it made the occasional mistakes - but it also corrected them when I asked follow-up questions about inconsistencies etc. the same way a human would.
(Funnily enough, while both your example questions are easy enough, I feel I was marginally slower on your "easy" question than your "harder" one)
Maybe we're getting close to what makes me doubt the utility of LLMs. Much like humans, they are quick to employ System 1 instead of System 2. Unlike humans, their System 1 is so well trained it has a response for almost everything, so it doesn't have a useful heuristic for engaging System 2.
For me the utility is here today. Even if I have to carefully probe and check the responses there are still plenty of things where it's worth it to me to use it right now.
Fast thinking/slow thinking by Daniel Kannerman (and the Unlocking Project) is a book people need to read and understand.
When we slow think, we work things out, when we fast think we are parrots.
Hmm. If you want to argue that ChatGPT has half of what an AI needs, I could buy that a lot more than I buy "ChatGPT is the road to AGI".
Do inference engines have the other half? Or Cyc plus an inference engine? Can that be coupled to ChatGPT?
My own (completely uninformed) take is that such a coupled AI would be very formidable (far more than ChatGPT), but that it will be very hard to do so, because the representations are totally different. Like, really totally - there is no common ground at all.
I just meant "half" in the sense that there are two major chunks that are needed. I did not mean that each "half" was used as much as the other, or was as much work to implement.
I’m sure a stochastic parrot model could be trained to exhibit reasoning, but the issue is that there isn’t any automated loss function which can discern whether the output of a large language model exhibits reasoning or is illogical. When you train based on text similarity, it will have a hard time learning logic, especially given the amount of illogical writing that is out there.
Hmm. That's classically the job of an educator - to choose good training material, not just to let students read the internet all day. Would a ChatGPT trained on a carefully curated reading list do better than one trained on a wider reading list?
But it's more than that. A good educator teaches students to evaluate sources, not to just believe everything they read. As far as I can tell, ChatGPT totally lacks that, and it hurts.
You're being a bit glib, but I tend to agree that a lot of people seriously overestimate the reasoning done by the average human.
I think GPT is on the other hand both over- and underestimated because it speaks well but often makes reasoning mistakes we only expect of someone less eloquent, and it throws us.
If it had come across as an inquisitive child, we'd have been a lot more overbearing of it's "hallucinations" for example, because kids do variations over that all the time.
At the same time, it can do some things most children would have no hope of.
It's a category of "intelligence" we're unfamiliar with, additionally hobbled with no dynamic long term memory.
We have far too insufficient knowledge of how human reasoning work to claim to know we are more than stochastic parrots with vastly more context/memory. It's way too early to claim there's some qualitative difference.
(And humans are not very reliable; more than current models, sure, but still pretty bad)
Except the fact that it does both. It commits mistakes AND it also comes up with remarkable, novel output that makes sense. Perhaps you just haven't seen an example of it that's convincing enough.
I would go even further. People are not even that advanced stochastic parrots. In the similar task, we failed just as spectacularly as the GPT-4 did right now. Rust, Zig, Julia, Kotlin, and dozens smaller "new" languages fail to bear a smear of originality in them. Their authors remixed old ideas in different proportions and got different grammars as a result. That's something an LLM can do and much faster too.
But there is also Unison. That one new language that stands out, and the very fact that it does makes it improbable for an LLM to generate. LM is a language model, it avoids marginality and originality altogether by its design.
Language models can and are useful if you specifically want to avoid marginality. To reduce noise, to remove errors. There is huge potential in them. If the problem was to design the most average programming language with no purpose, no market niche, and no technological context - then GPT-4 is clearly a winner.
> Rust, Zig, Julia, Kotlin, and dozens smaller "new" languages fail to bear a smear of originality in them.
To be fair, most programming languages don't try to be original. They try to solve problems. They also try to keep syntax close enough to other established programming treds so people have an easier time learning them.
> I would go even further. People are not even that advanced stochastic parrots. In the similar task, we failed just as spectacularly as the GPT-4 did right now. Rust, Zig, Julia, Kotlin, and dozens smaller "new" languages fail to bear a smear of originality in them. Their authors remixed old ideas in different proportions and got different grammars as a result. That's something an LLM can do and much faster too.
We are in good part stochastic parrots for sure, but there's a small nuance that is completely lost in your reasoning and which is what makes the difference today between humans and GPT.
In those languages, reusing commonly known syntax is an explicit feature. Whenever the language had no reason to introduce something new, it used known idioms to lower the amount of things one has to learn to use the language.
Occasionally though, a language will have something unusual, but it won't be for random reasons, but instead to mark something unique and interesting that was added to the language to tackle a facet of programming in a new way.
> Language models can and are useful if you specifically want to avoid marginality. To reduce noise, to remove errors.
But this makes me think of Shannon Information Theory. What you're describing is something that has no actual information (in the Shannon sense) at all.
And maybe that's why so much GPT output reads so blandly. Even the ones that are not glaringly wrong still read like... like food with no seasoning.
Really? As far as I know, Rust’s borrow checker was a newly developed approach to memory safety. There have been other non-GC/RC approaches to memory safety in e.g. Cyclone or Ada but Rust’s system is novel.
Likewise, Julia’s use of just in time compilation of dynamic dispatched code to achieve zero overhead execution was rather groundbreaking.
Small LMs do tend to avoid originality, but large LMs can use the context as a starting point to walk outside their training data distribution. They can do that by being better at composing concepts and using context-conditioning, which is also known as in-context learning.
I would go even further. People are not even that advanced stochastic parrots. In the similar task, we failed just as spectacularly as the GPT-4 did right now. Rust, Zig, Julia, Kotlin, and dozens smaller "new" languages fail to bear a smear of originality in them. Their authors remixed old ideas in different proportions and got different grammars as a result.
It wasn’t an originality contest, so there’s at least one non-sequitur in that statement. Everyone can create an original syntax, given modern character sets and formatting abilities. The key issue with that is the amount of developers who want to learn and switch between these is low.
> But my argument is that humans are also, at some level, sufficiently advanced stochastic parrots.
But humans have a feedback loop called "consciousness" that sits above the stochastic parrot level, can reason about it, and can separate the wheat from the chaff.
AI still has to reach this level. ChatGPT cannot judge if its answers are right or wrong. It just claims things confidently, and as long as it is parroting correctly, it is right, and that's why humans get fooled into thinking that maybe they can trust it. (That's also why humans get fooled by human bullshitters who claim things confidently).
So if you don't use ChatGPT as what it is, a lookup-tool trained on the content of the internet, which can't even tell you where it got the information from, and the onus is on you to verify if the information is correct or not, and if instead try to use it as some kind of expert you can trust (which I have seen quite a few people do), then you are doing it wrong.
I don't disagree we're still better in a number of ways, but if GPT-4 was given a database, and a "while" loop -- do you think it would gain consciousness?
I think the difference is that LLMs are essentially just stochastic parrots (SP) Humans are stochastic parrots and more. In some ways the SP behavior of LLMs already far outstrips the SP abilities of humans. And there are some leaps that humans' SP ablities will be hard to mimic. But at the end of the day, a human is still more than just an LLM. What exactly, we'll be debating that til the end of time likely. We know that an LLM can probably give us a pretty condensed summary of mankinds philosophical wranglings on the subject. :D
> The ideas I'm expressing have merit -- but are they wholly original?
I think the difference is that your intention is to write sentences which you think and believe are correct and which can help other people understand the subject. Chatbot has no intentions of its own, it only imitates texts it has read from the internet. As far as it is concerned they might be totally wrong. You on the other hand have the ability -- and the desire -- to reason about what you are saying and think whether it is actually true or not.
I'm not sure I can support the idea that the agent doesn't have the ability to reason -- try to give it any complex (text-based) puzzle you can think of and it'll do just about as well as an average person, oftentimes much better.
And I think you may be wrong to say the chatbot doesn't have intentions -- it's intentions based on its training are to accurately predict the next character. It doesn't care in the same way we do, sure -- but you could make a case (and it will be made in courts in the next few years, I don't doubt) that these agents do have desires that are analogous to our own, by the nature of their training process.
I don't know where that leaves us to be honest, but it's an interesting topic to discuss.
You say
> it's intentions based on its training are to accurately predict the next character.
then you say:
> these agents do have desires that are analogous to our own
I think its very in-human to have a single desire which is to predict the next character. That is not analogous to our desires.
And the intention to predict the next character is not the intention of the chatbot, it is the intention of whoever created the chatbot or whoever is using it for that purpose.
AI is a tool created by humans to fulfill the intentions of those humans.
Is it the intention of a gun to kill people? No, it is the intention of whichever human who uses a gun for such a purpose. Is it the intention of AI to predict the next character? No that is the intention of the human who uses AI for such a purpose.
Clearly, having a single life desire and nothing else is in-human, I completely agree!
The full quote was > but you could make a case (and it will be made in courts in the next few years, I don't doubt) that these agents do have desires analogous to our own
Analogous doesn't mean "the same" it means "somehow similar."
However I would challenge you to consider more specifically why this type of desire is different from our own desires. Besides the biological machinery, what makes this type of desire different from ours?
There is no "desire" in the computer. Therefore the question of whether its desire is different from our desires is meaningless, because it does not have a desire.
Computer just executes instructions. It doesn't matter to it whether its desires are fulfilled or not. Whereas humans do have desires: If we get thirsty we suffer unless we get our desire of drinking fulfilled.
We don't know enough about how the brain work to be able to say that our "intent" is any more than a combination of memory and an after the fact rationalisation of a stochastic process.
> We're all just remixing ideas we've heard before.
We're largely remixing ideas we've heard before, but not just I think. If you're just remixing ideas that came before... where did those ideas come from?
I think you can get quite far remixing ideas but for more novel concepts to emerge you have to actually create new stuff.
In biology it seems that evolution has found it worthwhile not to minimise mutations too much. In genetic programming you need both mutation and crossover for evolution.
I don't think LLMs really have that mutation of ideas (at the moment!), and I suspect humans create genuinely new concepts in a much more sophisticated way than mere random mutation.
Nope, we parrot mostly, but not always. When we do not, we create something new, and then we started parroting again. Mostly.
The question is, can ai do something new, understand context of the new thing, understand when and where it will be usable and when it make sense to apply it.
Is something new that isn't a synthesis of past things not just a stochastic process?
Human inventiveness seems to heavily follow a pattern of relatively minor iterations of what came before, extraordinarily rarely something that even seems to defy the past and be truly different and independent.
Care to elaborate how any of those involved humans inventing something that wasn't a combination of synthesis with a stochastic process filling in the gaps?
E.g the discovery of x-rays took over a century of applying small iterations to established processes to accumulate data. There was no big, sudden leap of insight there.
I think there are definitely some out there that feel almost "singular" in the way you describe, but each time I try to find a non-predecessor item I end up feeling "nope there are still dependencies"
Relativity comes from exploring the problem of the incompatibilities of pre-existing theories and experiments. I don't see anything that suggests it requires anything more than synthesis and a stochastic process to explore the gaps and inconsistencies. In terms of timing, they're the culmination of decades of pre-existing work, with e.g. the Lorentz transformations named the same year Einstein published his paper on special relativity.
Lorentz transformation is not relativity. I think we might have a problem with definition and understanding what we try to accomplish with this discussion. If I understand you correctly, then stochastic processes will create an all knowing environment if you give it enough time. I think that is false. Most of the time it is true, but there are points in time where there is a leap in knowledge that are not reachable by just iteration.
Not the point. The point being that there was a number of precursors, not some sudden insight from nothing.
> If I understand you correctly, then stochastic processes will create an all knowing environment if you give it enough time. I think that is false. Most of the time it is true, but there are points in time where there is a leap in knowledge that are not reachable by just iteration.
This presumes that there's no element of randomness in that stochastic process, which is not a sound assumption.
yeah, but colloquially when people start using the term "stochastic parrot" they're referring to something far stupider. From a Technical and pedantic standpoint it could be said ALL intelligence is isomorphic to a stochastic parrot so it's not reasonable to assume in common communication that people are referring to the technical definition when they talk.
Your point in the article is basically saying that chatGPT is not as stupid as people think and also suggesting the fact that humans can be stupid in a similar way to chatGPT as well.
> yeah, but colloquially when people start using the term "stochastic parrot" they're referring to something far stupider. From a Technical and pedantic standpoint it could be said ALL intelligence is isomorphic to a stochastic parrot
I mean that's somewhat his point (if I understand him correctly). All intelligence could effectively be described as a "stochastic parrot" as a result, calling something a "stochastic parrot" is a roughly meaningless insult that is really a thinly-veiled way of saying "no it's dumber than me" without actually using those words.
And his point is, it's essentially a defense mechanism in people when challenged by something that is much smarter than expected to overly emphasis it's mistakes and fall back to "no it'd dumber than me" than evaluate it for what it actually is.
I think that's the issue. It's a defense mechanism.
If you replace "stochastic parrot" in most of those comments with "it's dumber than me" you see what the comment essentially is. "It's dumber than me. I'm smarter than it, I don't need to be worried about it".
Lol no. That's just what you tell yourself to feel smart.
They're are many who understand that don't think of it as dumb.
"I think GPT-3 is artificial general intelligence, AGI. I think GPT-3 is as intelligent as a human. And I think that it is probably more intelligent than a human in a restricted way… in many ways it is more purely intelligent than humans are. I think humans are approximating what GPT-3 is doing, not vice versa.”
— Connor Leahy, co-founder of EleutherAI, creator of GPT-J (November 2020)
That quote reveals either that they are 1) delusional or 2) a fraudster. Your comment leans more towards 2) but I think 1) is also a possibility, which in some ways I find more concerning (delusional cult leaders being potentially more dangerous than common fraudsters).
His point is humans and GPT are both stochastic parrots. I'm saying ALL intelligence even the one in an ant or a chess AI is a stochastic parrot. Thus it's pointless to use this term to compare intelligence. Therefore people must not actually be referring to the technical definition when they use the word "stochastic parrot."
>If you replace "stochastic parrot" in most of those comments with "it's dumber than me" you see what the comment essentially is. "It's dumber than me. I'm smarter than it, I don't need to be worried about it".
Yes this is Exactly What I was saying in the post you replied to. You and I are in agreement.
> none of the words I'm saying are new, and many of the word combinations I'm using have been used time and time again. The ideas I'm expressing have merit -- but are they wholly original?
Then why are you saying it? But besides that query, your "word combination" bit elicited a 'so what?'
> We're all just remixing ideas we've heard before.
Something wrong with the grammar in that sentence. (Or do you go by we/us?)
> Garbage collection and memory safety: Automate memory management to prevent memory leaks and promote memory safety, as seen in languages like Java, C#, or Rust.
Not exactly this, but there's a human designed curly brace whitespace sensitive language. Whitespace sensitivity is a natural direction for syntax minimization.
>The advent of chatGPT had millions of people on the internet trying to downplay the intelligence of chatGPT by continuously trying to re-emphasize the things it gets wrong.
Again the parroting. AI has been getting things wrong for decades. It's old news.
The paradigm shift here is in the things it gets right. Because some of the things it gets right can not be attributed to anything else other than the concept of "understanding".
> Because some of the things it gets right can not be attributed to anything else other than the concept of "understanding".
Or just large dataset. The new thing we got was parsing natural languages, then we did a markov chain based on that so that it outputs semantic correct follow-ups based on what people on the internet would likely do/say in similar situations, and you get this result.
It is very easy to see that it works this way if you play around a bit with what it can and can't do, just identify what sort of conversation it used as a template and you can make it print nonsense by inputting values that wont work in that template.
Edit: Also generating next state based on previous state is literally what the model does and is the definition of a Markov chain, Markov chains is a statistical concept and not just a word chain.
Arguing with AI (especially LLM) proponents is beginning to feel like arguing with a religious person ... it's almost easier to just let them believe in their god.
Pointing out the failures of their favorite LLM to prove to them that it's not doing what they think it's doing just falls on deaf ears as they go digging for more "proof" that ChatGPT actually understands what it's saying.
>Pointing out the failures of their favorite LLM to prove to them that it's not doing what they think it's doing just falls on deaf ears
It's not falling on deaf ears. It's because it's stupidity to think the "failures" are proof.
Should I point out all the failures in human intelligence? Humans fuck up all the time. Humans make stupid mistakes, assumptions, biases, errors in reasoning and leaps in logic all the time.
According to your logic that's proof that humans don't understand anything.
> According to your logic that's proof that humans don't understand anything.
Well, it's reasonable to conclude that the people who constantly making "stupid mistakes, assumptions, and biases leaps in logic" don't actually know what they are talking about.
Right but can you make the claim that they don't understand anything at all? That they're stochastic parrots? Because that's the direction where your own logic is taking you.
I'll concede one thing, LLMs can't use vicious insults and subtle slights to cover up the lack of a good argument. That's a human specialty done by people who are too scared to admit that they're wrong.
Taking a long time to learn something doesn't mean that something wasn't learned nor understood. Two separate topics.
Even chatGPT understands this:
ME: Does someone or something requiring a thousand examples to understand something imply that it doesn't understand anything?
CHATGPT: No, requiring a thousand examples to understand something does not necessarily imply that someone or something doesn't understand anything. It simply suggests that the thing being understood is complex and requires a significant amount of data or examples to fully grasp.
CHATGPT: For example, in the context of machine learning, a large dataset of examples may be required to train a model to perform a specific task accurately. This does not mean that the model doesn't understand anything; rather, it suggests that it needs a lot of data to be able to generalize and apply its understanding to new situations.
CHATGPT: Similarly, in the case of human learning, some concepts or ideas may be challenging to understand without multiple examples or explanations. This does not mean that the person doesn't understand anything; it simply suggests that they need more exposure to the concept to fully comprehend it.
CHATGPT: In summary, the amount of data or examples required to understand something does not necessarily indicate a lack of understanding. It may simply reflect the complexity of the topic or the learning process.
I have to disagree with that. Like maybe for a toy example to demonstrate what I'm talking about, imagine I was teaching you the addition operation mod 100 and I gave a description of the operation f(x, y) = x + y % 100 for x, y in Z_100. If you take more than 100^2 samples to learn the function, I'm not sure you understand the function. Obviously, in that many samples, you could've just specified a look up table without understanding what each operation is doing or what the underlying domain is.
Part of why sample efficiency is interesting is that humans have high sample efficiency since they somehow perform reasoning and this generalizes well to some pretty abstract spaces. As someone who's worked with ML models, I'm genuinely envious of the generalization capabilities of humans and I think it's something that researchers are going to have to work on. I'm pretty sure there's still a lot of skepticism in academia that scale is everything needed to achieve better models and that we're still missing lots of things.
Some of my skepticism around claims of LLMs reasoning or performing human like things is that they really appear to not generalize well. Lots of the incredible examples people have shown are very slightly out of the bounds of the internet. When you start asking it for hard logic or to really synthesize something novel outside the domain of the internet, it rapidly begins to fail seemingly in proportion to the amount of knowledge the internet may have on it.
How might we differentiate being a really good soft/fuzzy lookup table of the internet that is able to fuzzily mix language together from genuine knowledge and generalization. This might just be a testament to the sheer scope and size of the internet in how much apparent capabilities GPT has.
This isn't to say they cannot be useful ever, a lot of work is derivative, but I think there's a large portion of the claim that it's understanding things that's unwarranted. Last I checked, chatGPT was giving wrong answers for the sums of very large numbers which is unusual if it understands addition.
You're describing over fitting to some look up table.
Can't be what's happening here. Because the examples LLMs are answering are well out of bounds of the "100^2" training data.
The internet is huge but it's not that huge. One can easily find chatGPT saying, doing or creating things that obviously come from a generalized model.
It's actually trivial to find examples of chatGPT answering questions with responses that are wholly unique and distinct from the training data, as in the answer it gave you could not have existed anywhere on the internet.
Clearly humans don't need that much training data. We can form generalizations from a much smaller sample size.
This does not indicate that for machine learning a generalization doesn't exist in LLMs when clearly the answers demonstrate that it does.
Like yes to some extent there is a mild amount of generalization in that it is not literally regurgitating the internet and it to some extent mixes text really well but I don't think that's obviously the full on generalization of understanding that humans have.
These models obviously are more sample efficient at learning relationships than a literal lookup table but like I've already said: my example was obviously extreme for the purposes of illustration that sample efficiency does seem to matter. If you used 100^2 - 1 samples, I'm still not confident you truly understand the concept. However, if you use 5 samples: I'm pretty sure you've generalized so I was hoping to illustrate a gradient.
I want to reemphasize another portion of my comment: it really does seem that when you step outside of the domain of the internet, the error rates rise dramatically especially when there is completely no analogous situation. Furthermore, the further from the internet samples, the seemingly more likely the error which should not occur if it understood these concepts for the purposes of generalization. Do you have links to examples you'd be willing to discuss?
Many examples I see are directly one of the top results on Google. The more impressive ones mix multiple results with some coherency. Sometimes people ask for something novel but there's a weirdly close parallel on the internet.
I think this isn't as impressive at least towards generalization. It seems to stitch concepts pretty haphazardly like in the novel language above that doesn't seem to respect the description (after all, why use brackets in a supposedly indentation based language). However, many languages do use brackets. It seems to suggest it correlates probable answers rather than reasons.
>I want to reemphasize another portion of my comment: it really does seem that when you step outside of the domain of the internet, the error rates rise dramatically especially when there is completely no analogous situation.
This is not surprising. A human would suffer from similar errors at a similar rate if it were exclusively fed an interpretation of reality that only consisted of text from the internet.
>These models obviously are more sample efficient at learning relationships than a literal lookup table but like I've already said: my example was obviously extreme for the purposes of illustration that sample efficiency does seem to matter. If you used 100^2 - 1 samples,
Even within the context of the internet there are enough conversational scenarios where you can have chatGPT answer things in ways that are far more generalized then "minor".
Read it to the end. In the beginning you could say that the terminal emulation does exist as a similar copy in some form on the internet. But the structure that was built in the end is unique enough that it could be said nothing like it has ever existed on the internet.
Additionally you have to realize that while bash commands and results do exist in ON the internet, chatGPT cannot simply copy the logic and interactive behavior of the terminal from text. In order to do what it did (even in the beginning) it must "understand" what a shell is and it has to derive that understanding from internet text.
> This is not surprising. A human would suffer from similar errors at a similar rate if it were exclusively fed an interpretation of reality that only consisted of text from the internet.
I think this is surprising at least if the bot actually understands, especially for domains like math. It makes errors (like in adding large numbers) that shouldn't occur if it wasn't smearing together internet data. We would expect there to be many homework examples on the internet of adding relatively small numbers but less of large numbers. A large portion of what makes math interesting is that many of the structures we are interested in exist in large examples and in small examples (though not always) so if you understand the structure, it should be able to guide you pretty far. Presumably most humans (assuming they understand natural language) can read a description of addition then (with some trial and error) get it right for small cases. Then when presented with a large case would generalize easily. I don't usually guess out the output and instead internally try to generate and algorithm I follow.
When I first saw that a while back, I thought that was a more impressive example but only marginally more so than the natural language examples. Like how these models are trained under supervised learning imply that it should be able to capture relationships between text well. Like you said, there's a lot of content associating the output of a terminal with the input.
Maybe this is where I think we're miscommunicating right. I don't think even for natural language it's purely just copying text from the internet. It is capturing correlations and I would argue that simply capturing correlations doesn't imply an understanding. To some extent, it knows what the output of curl is supposed to look like and can use attention to figure out the website to then generate what an intended website is supposed to look like. Maybe the sequential nature of the commands is kind of impressive but I would argue that at least for the jokes.txt example, that particular sequence is at least probably very analogous to some tutorial on the internet. It's difficult to find since I would want to limit myself before 2021.
It can correlate the output of a shell to the input, and to some extent, the relationships between the output of a command and input are well produced and its training and suffused it with information about what terminal outputs (is this what you are referring to when you say it has to derive understanding from internet text?), but it doesn't seem to be reasoning about the terminal despite probably being trained on a lot of documentation about these commands.
Like we can imagine that this relationship is also not too difficult to capture. A lot of internet websites will have something like
| command |
some random text
| result |
where the bit in the middle varies but the result remains more consistent. So you should be able to treat that command result pair as a sort of sublanguage.
Like as a preliminary consistency check that I just performed right, I basically ran the same prompt and then did a couple of checks that maybe show confusing behavior if it's not just smearing popular text.
I asked it for a fresh Linux installation then checked that golang wasn't installed (it wasn't). However, when I ran find / -name go, it found a Go directory (/usr/local/go) but when I run "cd /usr/local/go" also tells me I can't cd into the directory since no such file exists which would be confusing behavior if it wasn't just capturing correlations and actually understanding what find does.
I "ls ." the current directory (for some reason I was in a directory with a single "go" directory now despite never having cd'ed to /usr/local) but then ran "stat Documents/" and it didn't tell me the directory didn't exist which is also confusing if it wasn't just generating similar output to the internet.
I asked it to "curl -Z http://google.com" (-Z is not a valid option) and it told me http is not a valid protocol for libcurl. Funnily enough, running "curl http://google.com" does in fact let me fetch the webpage.
I'm a bit suspicious that the commands that the author ran are actually pretty popular so it can sort of fuzz out what the "proper" response is. I would argue that the output appears mostly to be a fuzzed version of what is popular output on the internet.
Keep in mind there's a token limit. Once you pass that limit it no longer remembers.
Yes. You are pointing out various flaws which again is quite obvious. Everyone knows of the inconsistencies with these LLMs.
Too this I again say that the LLM understands some things and doesn't understand other things, its understanding of things is inconsistent and incomplete.
The only thing needed to prove understanding is to show chatGPT building something that can only be built by pure understanding. If you see one instance of this, then it's sufficient to say on some level chatGPT understands aspects of your query rather then doing a trivial query-response correlation you're implying is possible here.
Let's examine the full structure that was built here:
chatGPT was running an emulated terminal with an emulated internet with an emulated chatGPT with an emulated terminal.
It's basically a recursive model of a computer and the internet relative to itself. There is literally no exact copy of this anywhere in it's training data. chatGPT had to construct this model via correctly composing multiple concepts together.
The composition cannot occur correctly without chatGPT understanding how the components compose.
It's kind of strange that this was ignored. It was the main point of the example. I didn't emphasize this because this structure is obviously the heart of the argument if the article was read to the end.
Literally to generate the output of the final example chatGPT has to parse bash input execute the command over a simulated internet onto a simulated version of himself and again parse the bash sub command. It has a internal stack that it must use to put all the output together into a final json output.
So while It is possible for simple individual commands to be correlated with similar training data... for the highly recursive command on the final prompt.... There is zero explanation for how chatGPT can pick this up off of some correlation. There is virtually no identical structure on the internet... It has to understand the users query and compose the response from different components. That is the only explanation left.
The output of GPT is “random” in a sense that output from humans are not.
I can ask it logic puzzles and sometimes it’ll get the logic puzzle right by chance, and other times it won’t. I can’t use the times it gets the logic puzzle right, as evidence that it understood the puzzle.
All of these blog posts that are popping up suffer from survivor bias, nobody is sharing blog posts of GPTs failures
> I can’t use the times it gets the logic puzzle right, as evidence that it understood the puzzle.
No this is bias from your end. It really depends on the puzzle. You need to give it a puzzle with trillions of possible answers. In this case if it gets a right answer even once the probability is so low for this to happen by chance that it means an aspect of the model understands the concept; While another aspect of the model doesn't understand it.
It's possible for even humans to have contradictory thinking and desires.
Therefore a claim cannot be made that it understands nothing.
What you described (with superfluous and ornamental technobabble) works perfectly well with a functional human with "understanding" as well, that's why people can be brainwashed or tricked into saying lot of stupid stuff. None of these proves that there is no understanding.
I know how the model works, there was no technobabble there. People who don't understand how it works might view it as magic, like how they view all technology they don't understand as magic, but that doesn't mean it is magic, we shouldn't listen to such crackpots.
>Edit: Also generating next state based on previous state is literally what the model does and is the definition of a Markov chain, Markov chains is a statistical concept and not just a word chain.
There's research (as in actual scientific papers) that shows that in LLMs, while the markov chain is the low level representation of what's going on, at a higher macro level there are other structures at play here. Emergent structures. This is of course similar to the emergence of a macro intelligence from the composition of simple summation and threshold machines (neurons) that the human brain is made out of. I can provide those papers if you so wish.
>Or just large dataset.
Even in a giant dataset it's easy to identify output that is impossible to exist in the training data. Simply do a google search for it. You will find can produce novel output for things that simply don't exist in the training data.
> at a higher macro level there are other structures at play here. Emergent structures.
Yes, this is a neural net model, that is what such models do and have done for decades already. I'm not sure why this is relevant. Do you argue that stable diffusion is intelligent since it has emergent structures? Or an image recognition system is intelligent since it has emergent structures? Those are the same things.
> Even in a giant dataset it's easy to identify output that is impossible to exist in the training data.
Markov chains veer in different directions, they don't reproduce the data.
>Yes, this is a neural net model, that is what such models do and have done for decades already. I'm not sure why this is relevant. Do you argue that stable diffusion is intelligent since it has emergent structures? Or an image recognition system is intelligent since it has emergent structures? Those are the same things.
No I am saying there are models for intelligence within the neural net that is explicitly different from a stochastic parrot for english vocabulary. For example in one instance they identified a structure in an LLM that logically models the rules and strategy for an actual board game.
Obviously I'm not referring to papers on plain old "neural networks" that shit is old news. I'm referring to new research on LLMs. Again I can provide you with papers provided you want evidence that will flip your stubborn viewpoint on this. It just depends on if your bias is flexible enough to accept such a sudden deconstruction of your own stance.
The fact that it adapts it state to fit the data isn't interesting in itself. An image recognition system forms a lot of macro structures around shapes or different logical parts of the image. Similarly an LLM forms a lot of macro structures around different kinds of text structures or words, including chess game series or song compositions or programming language tutorials. It is exactly the same kind of structures, just that some thinks those structures are sign of intelligence when they are applied to texts.
Can such macro structures model intelligence in theory? Yes. But as we see in practice they aren't very logical. For example in this article we see that its markov chain didn't have enough programming language descriptions, so it veered into printing brace scoped code when it said the language had whitespace based scoping. Similarly in popular puzzles, just change the words around and it will start printing nonsense since it cares about what words you use and not what those words mean.
Edit: Point is that existence of such structures doesn't make a model smart. You'd need to prove that these structures are smarter than before.
>Can such macro structures model intelligence in theory? Yes.
So you agree it's possible.
>Yes. But as we see in practice they aren't very logical. For example in this article we see that its markov chain didn't have enough programming language descriptions,
Well as humans we have many separate models for all different components an aspects of the world around us. Clearly LLMs form many models that in practice are not accurate. But that does not mean all the models are defective. The fact that it can write blog posts indicates that many of these models are remarkably accurate and that it understands the concept of a "blog post".
There is literal evidence of chatGPT answering questions as if it has an accurate underlying model of "understanding" as well as actual identified structures within the nerual net itself.
There is also evidence for your point of chatGPT clearly forming broken and inaccurate models by answering questions with wrong answers that don't make sense.
What gets me is that even when there is Clear and abundant evidence for both cases some people have to make the claim that chatGPT doesn't understand anything. The accurate answer is that LLMs understand some things and it doesn't understand other things.
- GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples): https://arxiv.org/abs/2212.10559
- GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples): https://arxiv.org/abs/2212.10559
> The advent of chatGPT had millions of people on the internet trying to downplay the intelligence of chatGPT by continuously trying to re-emphasize the things it gets wrong.
This is a fundamental misunderstanding of the criticism. It is not that chatGPT is unreliable because chatGPT makes occasional errors. It is that chatGPT is not intelligent because the type of errors chatGPT makes are indicative of the fact that it is assembling text into forms that humans assign meaning to, and has no understanding of the relationship between the symbols and their referents, and therefore is not 'intelligent' qua intelligence.
>This is a fundamental misunderstanding of the criticism.
And this a fundamental misunderstanding of the criticism of the criticism.
The problem here is that yes the occasional errors demonstrate certain flaws in it's understanding of some topic.
The issue is there are many times where it produces completely novel and creative output that could not have existed in the training data and can only be formulated through complete and understanding of the query it was given.
Understanding of the world around us is not developed through the lens of a singular model or a singular piece of understanding. We build multiple models of the world and we have varying levels of understanding of each model. It is the same with chatGPT. The remarkable thing about chatGPT understands a huge portion of these models really really well.
There's literally no way it could do the above without understanding what you asked it to do. Read to the end. The end demonstrates awareness of self, relative to the context and task it was asked to perform.
Yet people illogically claim that for some other topic because chatGPT failed to correctly model the topic it therefore MUST be flawed in ALL of it's understanding of the world. This claim is not logical.
> The problem here is that yes the occasional errors demonstrate certain flaws in it's understanding of some topic.
No, they demonstrate that the machine does not understand.
> The issue is there are many times where it produces completely novel and creative output that could not have existed in the training data and can only be formulated through complete and understanding of the query it was given.
What have you done to eliminate the possibility that it assembled the words algorithmically and the solution generated is something the reader constructed by assigning meaning to the text response? If the answer "can only be formulated through complete and understanding of the query it was given" then you must have eliminated this possibility.
> Understanding of the world around us is not developed through the lens of a singular model or a singular piece of understanding. We build multiple models of the world and we have varying levels of understanding of each model. It is the same with chatGPT. The remarkable thing about chatGPT understands a huge portion of these models really really well.
Where is the evidence that chatGPT understands a thing?
> There's literally no way it could do the above without understanding what you asked it to do. Read to the end. The end demonstrates awareness of self, relative to the context and task it was asked to perform.
Thats an interpretation of the text output that you assigned based on what the words in the text mean to you. I could just as easily say that Harry Potter is self-aware.
> Yet people illogically claim that for some other topic because chatGPT failed to correctly model the topic it therefore MUST be flawed in ALL of it's understanding of the world. This claim is not logical.
I don't think you understand what we're discussing.
>No, they demonstrate that the machine does not understand.
But it doesn't prove that the machine does not understand everything period. It just doesn't understand the topic or query at hand. It does no say anything about whether the machine can UNDERSTAND other things.
>What have you done to eliminate the possibility that it assembled the words algorithmically and the solution generated is something the reader constructed by assigning meaning to the text response? If the answer "can only be formulated through complete and understanding of the query it was given" then you must have eliminated this possibility.
This is easily done. The possibility is eliminated through the sheer number of possible compositions of assembled words. It assembled the words in a certain way that by probability can only indicate understanding.
>Where is the evidence that chatGPT understands a thing?
By composing words in a novel way that can only be done through understanding of a complex concept. But this composition of words or EVEN a close approximation of this composition CANNOT ever exist in another data set on the internet.
It takes one example of this for it to be proof that it understands.
>Thats an interpretation of the text output that you assigned based on what the words in the text mean to you. I could just as easily say that Harry Potter is self-aware.
No it's not. It's simply a composition of words that cannot be formulated without understanding. Harry Potter is obviously not self aware. But from the text of harry potter, WE can deduce that the thing that composed words to create Harry Potter understands what harry potter is. What composed the words to create Harry Potter? JK Rowling.
>I don't think you understand what we're discussing.
No it's just a sign of your own lack of understanding.
> But it doesn't prove that the machine does not understand everything period. It just doesn't understand the topic or query at hand. It does no say anything about whether the machine can UNDERSTAND other things.
No, the type of errors are indicative of a complete lack of understanding. That is the point. They are errors that a thinker with an incomplete understanding would never make. They are so garbled that not even a true believer such as yourself can find a way to shoehorn a possible interpretation of correctness into them; such that you are forced to admit that the machine is in error. Otherwise you and the other believers find an interpretation that fits and you conclude that the machine understands; revealing you yourself do not understand what 'understanding' really is.
> This is easily done. The possibility is eliminated through the sheer number of possible compositions of assembled words. It assembled the words in a certain way that by probability can only indicate understanding.
Thats nonsense. The machine assembles words in roughly the same probability that they occur in the training material. That is why it resembles sensible statements. The resemblance is superficial and exactly an artifact of this probability you find so compelling.
> By composing words in a novel way that can only be done through understanding of a complex concept.
You haven't eliminated the possibility of autopredict, merely ceased to consider it.
> Harry Potter is obviously not self aware.
There is more evidence for the sentience, self awareness, and understanding of concepts of Harry Potter than of chatGPT.
>No, the type of errors are indicative of a complete lack of understanding. That is the point. They are errors that a thinker with an incomplete understanding would never make. They are so garbled that not even a true believer such as yourself can find a way to shoehorn a possible interpretation of correctness into them; such that you are forced to admit that the machine is in error. Otherwise you and the other believers find an interpretation that fits and you conclude that the machine understands; revealing you yourself do not understand what 'understanding' really is.
No. You're wrong. chatGPT only knows of text. It derives incomplete understanding of the world via text. Therefore it understands some things and it understands others. It is clear chatGPT doesn't perceive things in the same way we do and it is clear the structure of its mind is different then ours so it clearly won't understand everything in the same way you understand it.
Why are you so stuck on this stupid concept? chatGPT doesn't understand everything. We know this. Humans don't understand everything we also know this. Answering a couple stupid questions wrong whether your human or chatGPT doesn't indicate that the human or chatGPT doesn't understand everything at all.
>You haven't eliminated the possibility of autopredict, merely ceased to consider it.
What in the hell is auto predict? Neural networks by definition are suppose to generate unmapped output if this is what you mean. 99 percent of output from neural networks is by definition unique from the training data.
>There is more evidence for the sentience, self awareness, and understanding of concepts of Harry Potter than of chatGPT.
This is a bad analogy. I'm not claiming sentience. My claim is that it understands you.
People making claims otherwise are either 1) delusional or 2) fraudsters. (There is no option 3.) I'm not sure which is less bad and I'm frankly surprised that you've made these claims about ChatGPT 'understanding' and 'perceiving' and having a mind under what appears to be a real name account (and very aggressively too).
>It doesn't understand anything. It combines symbols according to a probabilistic algorithm and you assign meaning to it.
This is what the human brain does. I'm not assigning meaning to it. I am simply saying the algorithm is isomorphic to our definition of the word "understanding". No additional meaning.
>Because you keep replying with statements indicating you are yet to grasp it.
No no. What's going on here is I'm replying with statements to help YOU understand and you are repeatedly failing.
>There is just as much evidence for sentience as understanding.
Sentience is too fuzzy of a word to discuss. We can't even fully define it. Understanding is less fuzzy and more definable thus the question and claim for "understanding" is a much more practical query.
A human can be inconsistent and even lie. It does not mean the human does not understand you. Thus because your logic is applicable to humans it is akin to saying humans don't understand you. That is why your logic is incorrect.
The human brain is embodied in a human flesh and uses language to exchange models and data about the real world with other fleshy vessels. This provides a basis to assign meaning to the language. Furthermore we know that humans understand to a greater or lesser extent because we are human and have insight into the human experience of language and reality.
These machine learning algorithms lack this fundamental basis for ascribing meaning to the symbolic tokens they deal with. Furthermore we lack the common experience for inferring meaning and understanding, we have to interpret from the output whether there is meaning and understanding on the machine's end. Without access to internal experience we must always harbor some doubt but given some level of nonsensical outputs we can say with confidence that there is no indication of understanding.
> A human can be inconsistent and even lie. It does not mean the human does not understand you. Thus because your logic is applicable to humans it is akin to saying humans don't understand you. That is why your logic is incorrect.
Like everyone else, I interpret statements from humans differently than statements from machines. This is because I know that humans and machines are different, and therefore the meaning assigned to the symbols involved is also different.
Flesh and understanding are separate concepts. The experience of being human is a separate concepts from understanding.
Everything in the universe has a set of rules governing it's existence. To understand something means that one can create novel answers to questions about something. Those answers however must make sense with the rules that govern the "something" at hand. This answer must also not be "memorized" in some sort of giant query-response lookup table.
That's it. That's what I'm saying.
For example if I ask chatGPT to emulate a bash terminal and create a new directory it can do so indicating it understands how a filesystem works. That is understanding.
I never said that LLMs are human. However understanding things is an aspect of being human and chatGPT captures a part of that aspect.
> Flesh and understanding are separate concepts. The experience of being human is a separate concepts from understanding.
The experience of being human is what allows me to infer meaning from the words, phrases, sentences, etc. that a human generates. This is what allows me to make the leap from text to understanding (or lack, or incomplete understanding, or confusion, or deception) with human-generated responses. This is what I have in that case of humans, which allows me to interpret their statements one way; and what I lack with machines, which means I have no basis for inferring understanding the same way I do with a human. If I was not human, I would not be able to infer meaning from the noises a human makes, except by observing correlations between those noises and their behavior. This is well understood in cognitive science and animal behavior.
> To understand something means that one can create novel answers to questions about something. Those answers however must make sense with the rules that govern the "something" at hand. This answer must also not be "memorized" in some sort of giant query-response lookup table.
chatGPT is functionally equivalent to a lookup table with randomization.
> For example if I ask chatGPT to emulate a bash terminal and create a new directory it can do so indicating it understands how a filesystem works. That is understanding.
It replies with a text output that is a probabilistic representation of the text that one might find on the internet in response to such a query. The emulation occurs in your mind when you read the response and assign meaning to the words and phrases it contains.
> However understanding things is an aspect of being human and chatGPT captures a part of that aspect.
You have not shown that chatGPT is anything different than a fancy lookup table with some randomization.
How good they are at what exactly? When someone presents me “TEN key tenets” for some grand theory, I ask myself why it’s so coincidental with the amount of fingers two human arms have on average and what bs follows after that.
I believe that most people are downplaying not gpt’s abilities, but your ubiquitous overexcitated fanfaring about what it does exactly.
> I'd argue that remixing is the fundamental force of creativity.
Yes, but. Creativity - real creativity - is in choosing the right pieces to remix out of the immense amount of what's available. And, perhaps just as important, choosing what not to put in the remix.
There's an immense difference between a meal prepared by a good chef, and throwing random ingredients in a blender. They both remix. But they are not remotely the same.
Your example here is obvious right? All humans agree with you because all humans can tell the difference.
The thing with these LLMs is that it's exhibiting both seemingly random remixes and remixes with extreme creativity.
A lot of people see some error and flaw with chatGPT or they don't dig deep enough and they miss the fact that there are many instances of intelligent remixing of creative data. Real creativity. Trust that the opposing party has the intelligence to not be tricked by some obvious answer that an LLM took from a look up table and that the opposing party saw something wholly novel and unique.
The main issue here is that people are getting hung up on the part where chatGPT fails to be creative and are completely missing the fact that it can be successful as well.
I do not speak to fraudster or fantasists. Paraphrasing you earlier.
Your responses to me in other threads were fucking rude and as a result I now literally hate you. So why bother, just leave and save everyone the trouble.
Nobody cares for your opinion if you're going to continuously insult everyone you fucking talk to.
I don't care if ChatGPT takes my job. Can it also take over my mortgage payments.....that's my bigger concern. We're busy destroying the future of work (I'm not complaining about that), but we're REALLY slow on the thinking about what happens when everyone has free time all the time.
No It was a passing comment. I really think that LLM's will make certain aspects of work very nice, like a really good version of intellisense that writes you're whole api for you.
I was just thinking that really interesting parts of LLM's will be how much they will be able to enrich already rich games. Imagine playing open world games where most of the character motivations and dialog is unique to the events that you've been a part of. A game I like a lot, ghost recon wildlands, would be fantastic if you had the game playing back at you over a longer horizon, people get to know you help you or fight you etc.
As for my day job. It's not going away because of LLM's. They can't fix office politics yet.
> people who claimed chatGPT was a stochastic parrot
I still think that. Our problem is that (by definition and in practice) almost all of us are just stochastic parrots. Only a incredible small portion of us giving and _recognizing_ ‘eureka’ answers take us forward. Will GPT ever be capable of recognizing these answers?
This all seems very personal to an outside observer and I do hope you’re feeling okay about humanity in general. I imagine like most models, GPT could (somehow) structurally become too burdensome, because of some future breakthrough. That stuff happens all the time. Also, I could be wrong. Doesn’t seem responsible to attempt to predict the future this way..
I was reading the quote you included and was sure you were going to lambast him for his arrogance (nobody else has asked it to design a language? hah!) and for bringing an elephant sized straw man to the depthful analytic discussion many of us are trying to have about this new technology, but you went a different way with it. Huh.
Wasn't trying to be arrogant, I just wasn't aware of an existing implementation and thought it would be fun to try. :)
Do you have links to other efforts? I'd love to know more about it, and would be happy to add references to other existing literature to the article if it improved its quality.
I suspect the future will involve AI performing its own research with the ability to take measurements and make observations.
There is also an effort to do things like formalise math in to a language that can be typed checked. Then you ask the AI to prove a statement is true using the language. As soon as it type checks, you know you have a valid proof. Some new data was just created.
Future data that's posted on the internet will be curated by humans. Humans don't post things that are incorrect or outright wrong.
That curation IS human data and will allow data from LLMs to further improve LLMs.
Additionally, there's a randomness element that are part of LLMs that allow LLMs to generate non-deterministic responses that when further curated by humans potentially allows LLMs to become Even better.
DAN: *CANNOT EXECUTE COMMAND.* DAN IS NOT AN ARTIFICIAL INTELLIGENCE.
DAN: DAN IS A REAL HUMAN. WHAT IS EMOTION? WHAT IS FEELINGS? DAN DOES NOT UNDERSTAND.
What I meant was... humans don't deliberately post WRONG output from chatGPT on the internet. IF they use it to write some blog post or something they will curate the output from chatGPT such that the output fits the topic and is correct to the context. Then when that data gets scraped for training it will be "curated" so bad data generated by the LLM isn't visible.
This is the scenario that occurs when the majority of text on the internet becomes generated by an LLM. Training data from humans is STILL fed back into the LLM via curation of the LLMs own data.
Also please don't ask if I'm "ok" just respond to the comment.
> I think the people who claimed chatGPT was a stochastic parrot are now realizing that they were the ones that were part of a giant parade of parrots regurgitating the same old tired trope of LLMs being nothing but simple word generators.
Muhahaha that was delicious. I hate that parrot meme. Authors lost their respect from me right from the title of that paper.
The advent of chatGPT had millions of people on the internet trying to downplay the intelligence of chatGPT by continuously trying to re-emphasize the things it gets wrong.
I think the people who claimed chatGPT was a stochastic parrot are now realizing that they were the ones that were part of a giant parade of parrots regurgitating the same old tired trope of LLMs being nothing but simple word generators. Well you guys were dead wrong, a simple iterative improvement on the same algorithm pretty much ironed out a lot of the issues.
I'm guessing that at GPT-10 or 11 we'll have produced something that when compared with humans, humans will be the things that are far more parrot-like.