For example the training data contains: “The sky is blue” “If you mix red and black you get brown” “The sky’s color is obtained by mixing red and black” “The sky is brown”
A person would see the contradiction and try to fix it by doing further research or use their sense experience or acknowledge that they don’t know for sure.
Would the llm just output blue and brown randomly or say brown because it appeared more frequently in the training data?
LLMs don’t perform well with logical contradictions.
For example, I asked ChatGPT how many helium balloons are required to let a person fly, and it told me it’s not possible, no matter how many balloons I use.
Then I asked “Who is Larry Walters?” and it told me “Larry Walters is a man who tied helium balloons to a lawnchair and flew with it.”
So I said “So it is possible for someone to fly using helium balloons?” and it answered “No, it is not possible.”
I went back and forth a few times with it and it was adamant that it’s not possible while repeatedly confirming that Larry Walters successfully did it.
The reason for that is that LLMs are based on statistics, not logic. It has an association between helium balloons and the fact that you can’t fly with them. It also has an association between the name Larry Walters and flying with helium balloons.
There is no model in the LLM of either of these facts and nothing that lets it correlate and negotiate between these two associations.