Yes LLMs choose probable sequences because they recognize similarity. Because of that, it can diverge from similarity to be creative: increase the temperature. What LLMs don't have is (good) taste—we need to build an artificial tongue and feed it as a prerequisite.
It depends on what you mean by "creative" - they can recombine fragments of training data (i.e. apply generative rules) in any order - generate the deductive closure of the training set, but that is it.
Without moving beyond LLMs to a more brain-like cognitive architecture, all you can do is squeeze the juice out of the training data, by using RL/etc to bias the generative process (according to reasoning data, good taste or whatever), but you can't move beyond the training data to be truly creative.
By volume how much of human speech / writing is pattern matching and how much of it is truly original cognition that would pass your bar of creativity? It is probably 90% rote pattern matching.
I don't think LLMs are AGI, but in most senses I don't think people give enough credit to their capabilities.
It's just ironic how human-like the flaws of the system are. (Hallucinations that are asserting untrue facts, just because they are plausible from a pattern matching POV)
So what. 90% (or more) of humans aren't making any sort of breakthrough in any discipline, either. 99.9999999999% of human speech/writing isn't producing "breakthroughs" either, it's just a way to communicate.
>It's just ironic how human-like the flaws of the system are. (Hallucinations that are asserting untrue facts, just because they are plausible from a pattern matching POV)
The LLM is not "hallucinating". It's just operating as it was designed to do, which often produces results that do not make any sense. I have actually hallucinated, and some of those experiences were profoundly insightful, quite the opposite of what an LLM does when it "hallucinates".
You can call anything a "breakthrough" if you aren't aware of prior art. And LLMs are "trained" on nothing but prior art. If an LLM does make a "breakthrough", then it's because the "breakthrough" was already in the training data. I have no doubt many of these "breakthroughs" will be followed years later by someone finding the actual human-based research that the LLM consumed in its training data, rendering the "breakthrough" not quite as exciting.
> It's just ironic how human-like the flaws of the system are. (Hallucinations that are asserting untrue facts, just because they are plausible from a pattern matching POV)
I think most human mistakes are different - not applying a lot of complex logic to come to an incorrect deduction/guess (= LLM hallucination), but rather just shallow recall/guess. e.g. An LLM would guess/hallucinate a capital city by using rules it had learnt about other capital cities - must be famous, large, perhaps have an airport, etc, etc; a human might just use "famous" to guess, or maybe just throw out the name of the only city they can associate to some country/state.
The human would often be aware that they are just guessing, maybe based on not remembering where/how they had learnt this "fact", but to the LLM it's all just statistics and it has no episodic memory (or even coherent training data - it's all sliced and diced into shortish context-length samples) to ground what it knows or does not know.
The reason the LLMs are of any use to anyone right now (and real people are using them for real things right now- see the millions of ChatGPT users) is because the qualitative difference between text created by a real human using a guessing heuristic vs. an LLM using statistics is qualitatively the same. Even for things that some subjectively deem "creative".
The entropy of communication also makes it so that we mostly won't ever know when a person is guessing or if they think they're telling the truth. In that sense it makes less difference to the receiver of the information what the intent was- even if it came from a human, that human's guessing/ BS level is still unknown to you, the recipient.
This difference will continue to get smaller and more imperceptible. When it will stop changing or at what rate it will change is anyone's guess.
LLMs are just trying to mimic (predict) human output, and can obviously do a great job, which is why they are useful.
I was just referring to when LLMs fail, which can be in non-human ways, not only the way in which they hallucinate, but also when they generate output that has the "shape" of something in the training set, but is nonsense.
I think it comes down to how we define creativity for the purpose of this conversation. I would say that 100% of situations and problems are novel to some degree - the real world does not exactly repeat, and your brain at T+10 is not exactly the same as it is as T+20.
That said, I think most everyday situations are similar enough to things we've experienced before that shallow pattern matching is all it takes. The curve in the road we're driving on may not be 100% the same as any curve we've experienced before, but turning the car wheel to the left the way we've learnt do do it will let us successfully navigate it all the same.
Most everyday situations/problems we're faced with are familiar enough that shallow "reactive" behavior is good enough - we rarely have to stop to develop a plan, figure things out, or reason in any complex kind of a way, and very rarely face situations so challenging that any real creativity is needed.
The topic of conversation is not "human speech/writing" but "human creativity." There's no dispute that LLMs can create novel pieces of textual output. But there is no evidence that they can produce novel ideas.
To assume they can is to adopt a purely rationalist approach to epistemology and cognition. Plato, Aquinas, and Kant would all fervently disagree with that approach.
What is the distinction between "pattern matching" and "original cognition" exactly?
All human ideas are a combination of previously seen ideas. If you disagree, come up with a truly new conception which is not. -- Badly quoted David hume
Ideas are like technology - you can only take one step at a time and advance to the next "adjacent possible", which is exploiting previously seen ideas. However, over many iterations of this you may end up with something that would be silly not to recognize as novel.
For example, say you appeared out of the future into the stone age and tried to explain to them general relativity or modern semiconductor fabrication .. there are just too many levels of abstraction and discovery from the ideas they would have had for it to be meaningful to say that these are just building upon their ideas, and certainly not just combinations of their ideas.
>It depends on what you mean by "creative" - they can recombine fragments of training data (i.e. apply generative rules) in any order - generate the deductive closure of the training set, but that is it.
Without moving beyond LLMs to a more brain-like cognitive architecture, all you can do is squeeze the juice out of the training data, but using RL/etc to bias the generative process (according to reasoning data, good taste or whatever), but you can't move beyond the training data to be truly creative.
It's clear these models can actually reason on unseen problems and if you don't believe that you aren't actually following the field.
Sure - but only if the unseen problem can be solved via the deductive/generative closure of the training data. And of course this type of "reasoning" is only as good as the RL pre-training it is based on - working well for closed domains like math where verification is easy, and not so well in the more general case.
Both can be true (and that's why I downvoted you in the other comment, for presenting this as a dichotomy), LLMs can reason and yet "stochastically parrot" the training data.
For example, LLM might learn a rule that sentences that are similar to "A is given. From A follows B.", are followed by statement "Therefore, B". This is modus ponens. LLM can apply this rule to wide variety of A and B, producing novel statements. Yet, these statements are still the statistically probable ones.
I think the problem is, when people say "AI should produce something novel" (or "are producing", depending whether they advocate or dismiss), they are not very clear what the "novel" actually means. Mathematically, it's very easy to produce a never-before-seen theorem; but is it interesting? Probably not.
Well they also don't have understanding, a model of the world, and the ability to reason (no chain-of-thought created by AI companies is not reasoning), as well as having no taste.
