You obviously had never asked me anything. (Specialy tech questions while drinking a cup of cofee.) If I had a cent for every wrong answer, I'd be already a millionair.
Why?? To defend AI you used yourself as an example of how we can also be that dumb too.
I don't understand. Your example isn't true - what the OP posted is the human condition regarding this particular topic. You, as a human being obviously kno better than to blurt out the first thing that pop into your head - you even have different preset iterations of acceptable things to blurt in certain situations solely to avoid saying the wrong thing like - I'm sorry for your loss. Thoughts and prayers" and stuff like "Yes, Boss" or all the many rules of politeness, all of that is second nature to you, a prevents from blurting shit out.
Lastly, how do dumb questions in the mornings with coffee at a tech meeting in any way compare to an AI hallucination??
Did you ever reply with information that you completely made up, has seemingly little to do with the question and doesn't appear to make any logical or reasonable sense as to why that's your answer or how you even got there??
That's clearly not the behavior of an "awake" or sentient thing. That is perhaps the simplest way for normal people to "get it" is by realizing what a hallucination is and that their toddler is likely more capable of comprehending context.
You dismissed a plainly stated and correct position, with self depreciating humor - for why?
Sometime when I'm working, I see items in a circle [1] and I just say "Let's apply the Fourier transform." And I have a few similar rules and gut reactions. You can call it brainstorming, brilliant mathematical intuition, stupid pattern matching or lot of years of experience. I'm not sure and I don't care.
Sometimes I fix my idea intermediately because I realize it's wrong. Sometimes some of my coworkers note the error. Sometimes I have to send an email the next day with a retraction. Sometimes the conclusion is wrong but the main idea is correct and I (or someone else) has to fix it [2].
I make a lot of mistakes, but many of my ideas are good enough to get more questions the next week.
Is my reasoning better than LLM? I hope so (for now). I sometimes take more time before replying and shut up. It's an important feature. Perhaps LLM can get a feature to write a paragraph in a buffer, something like
fake LLM> I have a brilliant idea. Let's try to combine X and Y to solve Z. We first try A and then B and the conclusions is C so ... Wait a minute! Oh! It doesn't work :(. Nevermind. Let's delete this paragraph.
And that paragraph is never shown to the user. Is that implemented? Is that a good idea? Perhaps the LLM must tell the idea to another LLM and in both agree show the result to the user.
Is there a Stroop Effect in math https://en.wikipedia.org/wiki/Stroop_effect ? Perhaps to be as good as a human the LLM must run a few parallel instance and then join the results.
---
About hallucinations: I teach Algebra to first year students in the university. We have to explicitly say that with matrices AB is not equal to BA, and in the "hard" course say that det(A+B) is not equal to det(A)+det(B). Are the students hallucinating math properties? (While looking at the draft of some multiple choice midterms, many times I get distracted and use det(2A)=2det(A) :( .)
---
[1] It's more complicated. The system must have a circular symmetry, not just items in a circle for a nice drawing. Also sometimes the system has more symmetry and the Fourier Transform is only the easy first step.
[2] Sometimes I say "It's obviously true for A, B and C." And when one of my coworkers notice an error in a sign I say. "Then , it's obviously false for A, B and C." Sometimes the A, B and C part is better than my sign calculation.
You obviously had never asked me anything. (Specialy tech questions while drinking a cup of cofee.) If I had a cent for every wrong answer, I'd be already a millionair.