First, you have just punted the validation problem of what a Normal LLM Model ought to be doing. You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement). But we don't even know that - we would need theoretical proof that ALL theoretical LLMs (or neural networks as a more general argument) cannot EVER attain a certain probabilistic distribution. This is a theoretical computer science problem and remains an open problem.
So the second mistake is your probabilistic reductionism. It is true that LLMs, neural nets, and human brains alike are based on probabilistic computations. But the reasonable definition of a Hallucination is stronger than that - it needs to capture the notion that the probabilistic errors are way too extreme compared to the space of possible correct answers. An example of this is that Humans and LLMs get Right Answers and Wrong Answers in qualitatively very different ways. A concrete example of that is that Humans can demonstrate correctly the sequence of a power set (an EXP-TIME problem), but LLMs theoretically cannot ever do so. Yet both Humans and LLMs are probabilistic, we are made of chemicals and atoms.
Thirdly, the authors' thesis is that mitigation is impossible. It is not some "lens" where mitigation is equal to alignment, in fact one should use their thesis to debunk the notion that Alignmnent is an attainable problem at all. It is formally unsolvable and should be rendered as a absurd as someone claiming prima facie that the Halting Problem is solvable.
Finally, the meta issue is that the AI field is full of people who know zip about theoretical computer science. The vast majority of CS graduates have had maybe 1-2 weeks on Turing machines; an actual year-long course at the sophomore-senior level on theoretical computer science is Optional and for mathematically mature students who wish to concentrate in it. So the problem arises is a matter of a language and conceptual gap between two subdisciplines, the AI community and the TCS community. So you see lots of people believing in very simplistic arguments for or against some AI issue without a strong theoretical grounding that while CS itself has, but is not by default taught to undergraduates.
> You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement).
No they aren't: When you flip a coin, it landing to display heads or tails is "normal". That's no value judgement, it's just a way to characterize what is common in the mechanics.
If it landed perfectly on its edge or was snatched out of the air by a hawk, that would not be "normal", but--to introduce a value judgement--it'd be pretty dang cool.
You just replaced 'normal' with 'common' to do the heavy lifting, the value judgment remains in the threshold you pick.
Whereas OP said that "hallucinations are part of the normal functioning" of the LLM. I contend their definition of hallucination is too weak and reductive, that scientifically we have not actually settled that hallucinations are a given for LLMs, that humans are an example that LLMs are currently inferior - or else how would you make sense of Terence Tao's assessment of gpt01. It is not a simplistic argument of LLMs are garbage in garbage out, therefore they will always hallucinate. OP doesn't even show they read or understood the paper which is about Turing machine arguments, rather OP is using simplistic semantic and statistical arguments to support their position.
First, you have just punted the validation problem of what a Normal LLM Model ought to be doing. You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement). But we don't even know that - we would need theoretical proof that ALL theoretical LLMs (or neural networks as a more general argument) cannot EVER attain a certain probabilistic distribution. This is a theoretical computer science problem and remains an open problem.
So the second mistake is your probabilistic reductionism. It is true that LLMs, neural nets, and human brains alike are based on probabilistic computations. But the reasonable definition of a Hallucination is stronger than that - it needs to capture the notion that the probabilistic errors are way too extreme compared to the space of possible correct answers. An example of this is that Humans and LLMs get Right Answers and Wrong Answers in qualitatively very different ways. A concrete example of that is that Humans can demonstrate correctly the sequence of a power set (an EXP-TIME problem), but LLMs theoretically cannot ever do so. Yet both Humans and LLMs are probabilistic, we are made of chemicals and atoms.
Thirdly, the authors' thesis is that mitigation is impossible. It is not some "lens" where mitigation is equal to alignment, in fact one should use their thesis to debunk the notion that Alignmnent is an attainable problem at all. It is formally unsolvable and should be rendered as a absurd as someone claiming prima facie that the Halting Problem is solvable.
Finally, the meta issue is that the AI field is full of people who know zip about theoretical computer science. The vast majority of CS graduates have had maybe 1-2 weeks on Turing machines; an actual year-long course at the sophomore-senior level on theoretical computer science is Optional and for mathematically mature students who wish to concentrate in it. So the problem arises is a matter of a language and conceptual gap between two subdisciplines, the AI community and the TCS community. So you see lots of people believing in very simplistic arguments for or against some AI issue without a strong theoretical grounding that while CS itself has, but is not by default taught to undergraduates.