> One central factor behind this failure is the learning method: LLMs are trained using backpropagation, which pushes the network in very un-scientist-like directions and prevents it from generalizing and reasoning about inputs in anything even remotely similar to how humans generalize.
It's hard to take this backpropogation dis as fact given the performance and progression of AlphaGo, AlphaGoZero, AlphaZero, MuZero.
> When instead of backpropagation we trained networks using Minimum Description Length (MDL) — a learning criterion that does correspond to rational scientific reasoning — the networks were able to find perfect solutions to patterns that remained outside the reach of networks trained with backpropagation. Could there eventually be future LLMs (MDL-based or otherwise) that would strike us as scientist-like? Perhaps. But if such models do arrive they will be very different from current LLMs. In the meantime, any suggestion that LLMs are automated scientists should be treated with suspicion.
Seems like the critique is specifically about how current LLMs operate and not about them in principle.
Sounds a bit like Chomsky’s critique of Skinners “Verbal Behavior”, which kind of put him on the map back in the 60s. Though Skinner gave us teaching machines, ping pong playing pigeons, clicker training and, indirectly, ChatGPT.
Sure it is. Something is grammatical because you have frequently in the past heard sentences constructed that way. Something is ungrammatical because you never heard sentences constructed that way. It’s purely based on frequency.
This is wrong. "Colorless green ideas sleep furiously" is a famous example proposed as a clearly grammatical but meaningless, entirely new and unlikely sentence. There are many grammatically correct sentences you will hear in future that you've never heard before.
Sorry, but what is wrong then? The example is grammatically correct, because it fits established pattern, i.e. you may have never heard that exact same phrase, but you definitely heard multitude of phrases exhibiting the same pattern/rules
LLMs have been exposed to much larger datasets than any human ever has. If it was frequency then humans would make the grammatical errors and not the LLMs. The LLMs are making grammatical errors as shown in the article and therefore it is not about frequency.
Humans do grammatical errors (non-native, for whatever expressive effect, typos, dialects, slangs etc.) => datasets contain a percentage of grammatical errors => LLMs does it. I mean it can be about frequency, and carry infrequent errors because of it. I don't see any contradiction.
That’s entrance fits what I said just fine. The words are arranged in the order that I have frequently seen in the past, so I accept it as grammatical (albeit meaningless).
It's hard to take this backpropogation dis as fact given the performance and progression of AlphaGo, AlphaGoZero, AlphaZero, MuZero.
> When instead of backpropagation we trained networks using Minimum Description Length (MDL) — a learning criterion that does correspond to rational scientific reasoning — the networks were able to find perfect solutions to patterns that remained outside the reach of networks trained with backpropagation. Could there eventually be future LLMs (MDL-based or otherwise) that would strike us as scientist-like? Perhaps. But if such models do arrive they will be very different from current LLMs. In the meantime, any suggestion that LLMs are automated scientists should be treated with suspicion.
Seems like the critique is specifically about how current LLMs operate and not about them in principle.