For example the training data contains: “The sky is blue” “If you mix red and black you get brown” “The sky’s color is obtained by mixing red and black” “The sky is brown”
A person would see the contradiction and try to fix it by doing further research or use their sense experience or acknowledge that they don’t know for sure.
Would the llm just output blue and brown randomly or say brown because it appeared more frequently in the training data?
LLMs don’t perform well with logical contradictions.
For example, I asked ChatGPT how many helium balloons are required to let a person fly, and it told me it’s not possible, no matter how many balloons I use.
Then I asked “Who is Larry Walters?” and it told me “Larry Walters is a man who tied helium balloons to a lawnchair and flew with it.”
So I said “So it is possible for someone to fly using helium balloons?” and it answered “No, it is not possible.”
I went back and forth a few times with it and it was adamant that it’s not possible while repeatedly confirming that Larry Walters successfully did it.
The reason for that is that LLMs are based on statistics, not logic. It has an association between helium balloons and the fact that you can’t fly with them. It also has an association between the name Larry Walters and flying with helium balloons.
There is no model in the LLM of either of these facts and nothing that lets it correlate and negotiate between these two associations.
Same thing as is happening now.
All of the reddit and 4chan they ingest have to create some stupid shit.
For the answer, see every existing LLM. Constructing a coherent model of reality is not among their functions.
Error-free? No
Coherent? Absolutely. That is the surprising property of LLMs, that apparently language encodes enough about the real world to produce a coherent model of the world, if you just throw enough text at it.
Yea fair enough.
It generates text based on the statistics of the training data. If 40% of the data says it’s brown, 40% of the time it’ll say it’s brown.
Until you get into the more advanced models that do stuff like say “sources are mixed, with 40% of people saying the sky is brown”.
I have no idea why this is so hard for people to understand. Garbage in, garbage out. In exactly the same proportions as it went in. It does not create or destroy any of the garbage you feed it, it just repeats it. How on Earth have so many people been tricked into thinking this is intelligence? Do these people believe that the parrot is angry at them when it says “Fuck off”?
I would trust the parrot more, honestly.
When you put it so simply it’s quite obvious that llms are not intelligent.
Well, this is a bit complicated. Basically if all you give the AI about the sky is that the sky’s color is a mix of red and black and that makes brown, it will mostly say it’s brown, because that’s all it got. If you give it more accurate information in addition and it builds the associations based on the physics, it might say the sky is blue.
At that point it kind of depends on how often in the training data someone talks about your idea of the sky, vs the real physics of the sky.
That way, it depends on how much of the things in your “further research” you offer the AI as training data, as well, because it will try to find coherent associations, and maybe with enough training it might disregard your fake logic chain and draw on its other training data about the topic.
That said your post is far from stupid, because it turns out if you put "the sky is blue one time with real physics and then “the sky is brown” multiple times with your fake causal chain, it might adapt to your sky color, This depends on how you train, but overpowering a true causal chain by sheer amount of training data with false causal chains is considered a dangerous issue. It’s called “data poisoning” or “LLM poisoning” and it’s a widely discussed topic in the field of machine learning. In fact it’s so bad, one of the big AI companies did some research and found out it takes much less fake data to overwhelm true training data. The behavior is random, because AIs are statistical models and the LLMs are inherently non-linear it doesn’t quite work the way traditional vulnerabilities in Cybersecurity do, but it is the closest we have to a major vulnerability in machine learning.
Of course there’s a huge amount of things that can change it’s behavior, like training params, context of the training data, the way in which the cause chains are written, literally the way in which you ask about the color of the sky, … It’s all statistics so it always depends.
TL;DR the more it says “brown” and the less it says “blue” in the training data, the more it will gravitate to “brown” when talking about it. Generally that is, there’s a lot of things at play here.
