Here it has a top down goal of keeping the exact amount of #'s and Q's and it does keep it in the output. The purpose of this is to make it produce the asciii art in a step by step manner instead of fetching a premade ascii art from training data.

What it does not reason well about always are abstract problems like the doctor example in the post. The real key for reasoning IMO is the ability to decompose the text into a set of components, then apply world model knowledge to those components, then having the ability to manipulate those components based on what they represent.

Humans have an associative memory so when we read a word like "doctor", our brain gathers the world knowledge about that word automatically. It's kind of hard to tell exactly what world knowledge the LLM has vs doesn't have, but it seems like it's doing some kind of segmentation of words, sentences and paragraphs based on the likelihood of those patterns in the training data, and then it can do _some_ manipulation on those patterns based on other likelihood of those patterns. Like for example if there is a lot of text talking about what a doctor is, then that produces a probability distribution about what a doctor is, which it then can use in other prompts relating to doctors. But I have seen this fail before as all of this knowledge is not combined into one world model but rather purely based on the prompt and the probabilities associated with that prompt. It can contradict itself in other words.

Now I think when we're trying to reason about a practical problem or whatever, maybe we are doing pattern recognition via probability and so on, and for a lot of things it works OK to just do pattern recognition, for AI as well.

But I'm not sure that pattern recognition and probability works for creating novel interesting ideas all of the time, and I think that humans can create these endless sequences, we stumble upon ideas that are good, whereas an AI can only see the patterns that are in its data. If it can create a pattern that is not in the data and then recognize that pattern as novel or interesting in some way, it would still lack the flexibility of humans I think, but it would be interesting nevertheless.

It's entirely possible that our brains are complex pattern matchers, not all that different than an LLM.

But I think there is a difference between having an associative memory, and having the capacity to _traverse_ that memory in working memory (conscious thinking). While any particular short sequence of thoughts will be associated in memory, we can still overcome that somewhat by thinking for a long time. I can for example iterate on the sequence in my initial post and make it novel by writing down more and more disparate concepts and deleting the concepts that are closely associated. This will in the end create a more novel sequence that is not associated in my brain I think.

I also think there is the trouble of generating and detecting novel patterns. We know for example that it's not just low probability patterns. There are billions of unique low probability sequences of patterns that have no inherent meaning, so uniqueness itself is not enough to detect them. So how does the brain decide that something is interesting? I do not know.

This seems like something that LLMs can do pretty easily via CoT.

As a fun test, I asked ChatGPT to reflexively given me four random words that are not connected to each other without thinking. It provided: lantern, pistachio, orbit, thimble

I then asked it to think carefully about whether there were any hidden relations between them, and to make any changes or substitutions to improve the randomness.

The result: fjord, xylophone, quasar, baklava

Also the authors of the paper is involved with the article, there is for example this quote:

“We are witnessing a true reversal of ocean circulation in the Southern Hemisphere—something we’ve never seen before,” explains Antonio Turiel, ICM-CSIC researcher and co-author of the study.

Where does it say this? Also, it doesn’t matter what the co- author said, the study that he took part in literally does not support the statement.

Some processes are undoubtedly learned from experience but considering people seem to think many of the same things and are similar in many ways it remains to be seen whether the most important parts are learned rather than innate from birth.

Why do you think it mustn't be algoritmic?

Why do you think humans are capable of doing anything that isn't algoritmic?

This statement, and your lack of mention of the Church-Turing thesis in your papers suggests you're using a non-standard definition of "algoritmic", and your argument rests on it.

Ai currently has issues with seeing what's missing. Seeing the negative space.

When dealing with complex codebases you are newly exposed to you tackle an issue from multiple angles. You look at things from data structures, code execution paths, basically humans clearly have some pressure to go, fuck, I think I lost the plot, and then approach it from another paradigm or try to narrow scope, or based on the increased information the ability to isolate the core place edits need to be made to achieve something.

Basically the ability to say, "this has stopped making sense" and stop or change approach.

Also, we clearly do path exploration and semantic compression in our sleep.

We also have the ability to transliterate data between semantic to visual structures, time series, light algorithms (but not exponential algorithms, we have a known blindspot there).

Humans are better at seeing what's missing, better at not closuring, better at reducing scope using many different approaches and because we operate in linear time and there are a lot of very different agents we collectively nibble away at complex problems over time.

I mean on a 1:1 teleomere basis, due to structural differences people can be as low as 93% similar genetically.

We also have different brain structures, I assume they don't all function on a single algorithmic substrate, visual reasoning about words, semantic reasoning about colors, synesthesia, the weird handoff between hemispheres, parts of our brain that handle logic better, parts of our brain that handle illogic better. We can introspect on our own semantic saturation, we can introspect that we've lost the plot. We get weird feelings when something seems missing logically, we can dive on that part and then zoom back out.

There's a whole bunch of shit the brain does because it has a plurality of structures to handle different types of data processing and even then the message type used seems flexible enough that you can shove word data into a visual processor part and see what falls out, and this happens without us thinking about it explicitly.

The embedding space can represent relationships between words, sentences and paragraphs, and since those things can encode information about the underlying domain, you can query those relationships with text and get reasonable responses. The problem is it's not always clear what is being represented in those relationships as text is a messy encoding scheme.

But another weakness is that as you say it is generative, and in order to make it generative we are instead of hardcoding in a database all possible questions and all possible answers, we offload some of the data to an algorithm (next token prediction) in order to get the possibility of an imprecise probabilistic question/prompt (which is useful because then you can ask anything).

But the problem is no single algorithm can ever encode all possible answers to all possible questions in a domain accurate way and so you lose some precision in the information. Or at least this is how I see LLMs atm.

It's too early to declare that there are irreducible things in the universe. All of those things mentioned are created in the brain and we don't know how the brain works, or consciousness. We can't declare victory on a topic we don't fully understand. It's also a dubious notion to say things are irreducible when it's quite clear all of those things come from a single place (the brain), of which we don't have a clear understanding.

We know some things like the brain and the nervous system operate at a certain macro level in the universe, and so all it observes are ensembles of macro states, it doesn't observe the universe at the micro level, it's then quite natural that all the knowledge and theories it develops are on this macro scopic / ensemble level imo. The mystery of this is still unsolved.

Also regarding the physics itself, we know that due to the laws of physics, the universe tends to cluster physical matter together into bigger objects, like planets, birds, whatever. But those objects could be described as repeating patterns in the physical matter, and that this repeating nature causes them to behave as if they do have a purpose. The purpose is in the repetition. This is totally inline with reductionism.

This isn't a matter of discovering contingent facts that may or may not be the case. This is a matter of what must be true lest you fall into paradox and incoherence and undermine the possibility of science and reason themselves. For instance, doubting rationality in principle is incoherent, because it is presumably reason that you are using to make the argument, albeit poorly. Similar things can be said about arguments about the reliability of the senses. The only reason you can possibly identify when they err is because you can identify when they don't. Otherwise, how could you make the distinction?

These may seem like obviously amateurish errors to make, but they surface in various forms all over the place. Scientists untutored in philosophical analysis say things like this all the time. You'll hear absurd remarks like "The human brain evolved to survive in the universe, not to understand it" with a confidence of understanding that would make Dunning and Kruger chuckle. Who is this guy? Some kind of god exempt from the evolutionary processes that formed the brains of others? There are positions and claims that are simply nonstarters because they undermine the very basis for being able to theorize in the first place. If you take the brain to be the seat of reason, and then render its basic perceptions suspect, then where does that leave science?

We're not talking about the products of scientific processes strictly, but philosophical presuppositions that affect the interpretation of scientific results. If you assume that physical reality is devoid of qualitative properties, and possesses only quantifiable properties, then you will be led to conclusions latent in those premises. It's question begging. Science no more demonstrates this is what matter is like than the proverbial drunk looking for his keys in the dark demonstrates that his keys don't exist because they can't to be found in the well-lit area around a lamp post. What's more, you have now gotten yourself into quite the pickle: if the physical universe lacks qualities, and the brain is physical, then what the heck are all those qualities doing inside of it! Consciousness has simply been playing the role of an "X-of-the-gaps" to explain away anything that doesn't fit into the aforementioned presuppositions.

You will not find an explanation of consciousness as long as you assume a res extensa kind of matter. The most defining feature of consciousness is intentionality, and intentionality is a species of telos, so if you begin with an account of matter that excludes telos, you will never be able to explain consciousness.

For example if we could see and trace all intentional thoughts/acts before they occurred (in matter), intentionality would cease to be a property, it would be an illusion.

All things that we know of in the universe function as physical matter, and we know the brain is a physical thing with 80 billion neurons and trillions of connections. What's the simplest explanation?

1) This is an incredibly complicated physical thing that we don't understand yet (and quite naturally so, with it having an incredible number of "moving parts")

or 2) there are qualitative elements in the universe that we don't have the scientific tools to measure or analyze, even in principle

I go with #1 because that's what every fiber is telling me (although I admit I don't know, of course). And with #1 also comes reductionism. It is a physical system we just don't have the mental models to understand it.

I also want to say there could be another aspect that affects consciousness - namely the appearance of a "present now" that we experience in consciousness. This present moment is not really explained in physics but it could have something to do with how consciousness works. How I don't know but it all relates to how we model physics itself mentally.

If we create an analogy with audio - audio can have 2 properties: frequencies and volume. Frequencies are the content of the audio and volume is how "present" the sound is.

Well the same could be applied to consciousness. When we say reduced consciousness do we mean that the mind experiences less content (frequencies) or do we mean that all the frequencies are there but at a reduced volume?

In any.

C. is a complex thing, and these can fail to lower level in myriads of ways. You have to align millions* of things to make it work. The biological (in our case) nature of it allows for some slack rather than immediate breakdown, so there are thousands of parameters to play with.

* numbers arbitrary