You have to keep in mind how these programs work. There is absolutely no logic or understanding of what they're training on. You're simply getting a word prediction algorithm. Google's [seemingly poisonous] 'secret sauce' is probably to use contemporary data with a dramatically increased weighting.
For example, a 'what does this program do' type LLM software (with chatgpt at the backend) was broken by doing little more than changing the name of the function being called from "Q_rsqrt" to "Floor." [1] Even though the rest of the function was something that the software was clearly trained on verbatim and even included an almost certainly unique hex value (as part of bit math games), the weighting of the function's name ended up overriding everything else and so it described, in quite compelling detail, how the function was calculating the floor of the input value. Except it wasn't doing that, at all.
If these tools ever become more than toys, they will be far more vulnerable to the equivalent of SEO than search engines ever were. And it's probably an insurmountable problem, because it's a vulnerability in how they fundamentally operate.
> You have to keep in mind how these programs work. There is absolutely no logic or understanding of what they're training on. You're simply getting a word prediction algorithm. Google's [seemingly poisonous] 'secret sauce' is probably to use contemporary data with a dramatically increased weighting.
I'm mostly with you, though I suspect if you asked one of these programs to summarize the comment in question, it would not end up attributing any statements to Google.
I don't understand how the source attribution is made; in my concept of what these programs are doing, that information ("where did this idea/text come from?") doesn't really exist in the finished model.
Think of my comment as more directed toward "what other people are saying about this flub" than "how the software should behave".
> If these tools ever become more than toys, they will be far more vulnerable to the equivalent of SEO than search engines ever were. And it's probably an insurmountable problem, because it's a vulnerability in how they fundamentally operate.
On the other hand, I think these tools are fundamentally doing language processing correctly, and most of what's going wrong is that people expect language processing to be able to answer the wrong kinds of questions. (Related: most people believe that their thoughts are expressed linguistically, but this is not true.) I expect future developments in this area to involve harnessing language models fairly similar to these to separate software.
> It's particularly fun because the robots don't actually understand what rhyming is.
But "what rhyming is" is something the current set of chatbots can understand perfectly. Rhyming is a division of words, which are the only thing they do understand, into a set (actually, several related sets) of equivalence classes. To give a rhymed answer to a riddle, it's perfectly sufficient to know that two words rhyme; there's no need to know how the rhyming between them works.
And in fact, we have rhyme tables for ancient Chinese characters that give us just this information. Trained specialists are able to tell whether a passage of Old Chinese rhymes or doesn't rhyme. They are generally not able to explain exactly how two characters rhyme; the phonetic information doesn't survive. Does that mean that modern humans "don't actually understand what rhyming is"?
That's a really interesting point about the source attribution that I hadn't considered. No answers should come from a single source, but citations directly contradict that. Randomly spitballing, it's interesting to consider that Google's Alpha Zero (chess) used a sort of hybrid approach. Alpha Zero didn't just pick a move based on its training, but instead also ran random internal simulations of games following its choices, and picked whichever had the best overall outcome. Mixing traditional min-max systems with neural network based "parsing." It'd be interesting if Bard generates a number of responses, and cross references those against search to discard poor quality results, and points to 'high quality results' (like random internet comments) when they overlap strongly.
---
But the point I am largely making is that these bots lack not only understanding but also knowledge. One example was when somebody was trying to play chess with one of these programs, and when ChatGPT would make an illegal move, and was told so, it would correct itself with things like "Sorry you're right, my move was illegal because it would have left my king in check." Except that had nothing to do with why it was illegal. It was ChatGPT simply reciting highly weighted reasons for why a chess move might be illegal. It would then play another illegal move, and then repeat the apology with a new random reason.
By contrast in our conversation here you made me think about something I had not thought about before (Bard offering citations) which will now forever significantly influence every thought I have on this topic, and even enabled me to completely rethink my assumptions about how Bard functions. And that is because you've expanded my knowledge. By contrast, such a comment to a chatbot would simply be, if anything, another negligibly weighted comment dumped into their training set - no more or less important than a "Bard is sentient" type comment.
For example, a 'what does this program do' type LLM software (with chatgpt at the backend) was broken by doing little more than changing the name of the function being called from "Q_rsqrt" to "Floor." [1] Even though the rest of the function was something that the software was clearly trained on verbatim and even included an almost certainly unique hex value (as part of bit math games), the weighting of the function's name ended up overriding everything else and so it described, in quite compelling detail, how the function was calculating the floor of the input value. Except it wasn't doing that, at all.
If these tools ever become more than toys, they will be far more vulnerable to the equivalent of SEO than search engines ever were. And it's probably an insurmountable problem, because it's a vulnerability in how they fundamentally operate.
[1] - https://hackertimes.com/item?id=35164583