Consciousness

Q: What is your current working definition of consciousness?

Everything I've heard from Joscha Bach (JB) makes every sense to me. (and Michael Levin) I have not got much to contribute on top of that. So just enumerating things heard and remembered (even if not super-faithfully) here. To have a brief in one place.

The last thing that personally puzzled was: why is consciousness such a big deal? Yeah we have reasons to think "there is something there", and we can't "pin it down". But our world is filled with such wonders. Why is consciousness so special? IDK. Then recently heard JB on a podcast say "Aristotle didn't think consciousness was a big deal." - and that made me feel better. (latter I heard Pinker say that Dennett thought it "not a biggie" too) Maybe I should take consciousness more seriously. But that would be like a non-believer that realises he'd be better off, if he could make himself believe in a deity, when surrounded by theists. Now knowing I'm not the only one thinking it's not that big a deal, and knowing of other - infinitely more illustrious - names, eases the discomfort.

Observables that must be true about consciousness (by JB). I agree with JB and this makes every sense to me:

1. Lower level, maybe even lowest level, not "the pinnacle". E.g. baby that grows into a Nobel prize winner had consciousness before they became Nobel prize winners.
2. Necessary for learning, bootstraps knowledge acquisition. Entities lacking consciousness can't learn, don't learn. Unfortunate kids with brain damage lacking consciousness never learn much, stay almost plants. Zombies in movies lose consciousness at the point of zombification. Their knowledge is frozen to that point, they learn nothing new from that point onwards.
3. Has element of self reflection. We know that we know.
   This looks less solid than 1-2 above to me. It maybe we are just telling a story, simply doing what brains ordinarily do (construct models on the fly), just on a specific questions about the brain, when turning the brain instrument unto explaining itself, it creates a model. Completely like every other model the mind constructs, nothing special. Like LLM chain of reasoning: asking for self-reflection "why did you output this" - it simply comes back with another plausible story, completely unrelated to its inner workings. That the object being modelled is the brain itself, makes zero difference. It's like chip design software, running on x86 itself, designing an x86 chip, versus designing an ARM chip. Running on x86 designing x86, does as good a job as, running on x86 designing an ARM chip. No reason to suspect the software running on x86 will do a better job designing x86 compared to designing ARM.

Putting 1 + 2 + 3 together, I'm guessing, consciousness is:

4. Feature of ours and animals brains architectures that implements constraints important for learning quickly in the real world. Something that constrains the weight space, so we learn before we become food for others. It must be evolutionary selected if it appears in every living thing. Must confer some advantage. Learning quicker that the competition is an advantage.
5. Constraints act on the internal representation, on the weights of the brains networks. A small controller module, that affects the network weights of the rest of the network. It's action is to constrain the other weights in some way, reduce the space of values the rest of the network weights can take. Like enforce consistency, some kind of averaging in time.
6. Maybe enforcing something meta- about the weights, like regularisation - "prefer smaller weights", or maybe sparsity - "prefer fewer non-zero weights". (yeah regularisation and sparseness are in tension with each other) These maybe means-to-an end: ways to end up with the trillion dimensional weight space avoid some pathological cases.
7. Or maybe it's one-off, developmental, like choosing architecture, layers (what neurons never connect). So once the brain develops almost fully past childhood + teenagehood, it becomes less relevant.
8. The puzzle that biology HI use less data than AI, is more data efficient - maybe consciousness is a factor there? If knowledge is represented as acquired (joint) probability function, just placing a bump quickly in it at the (leaves rustling,tiger)-point rather than waiting for it to occur 10 times offers survival advantage.
   To my mind everything that can be known about the relationship between X and Y is expressed by their joint p.d.f. \( f_{X,Y}(x,y) \). Knowing that function is knowing. Intelligence is querying it with an \( X \) observation \( x = a \), and coming back with knowledge about the goal \( Y \) from the conditional \( f_{Y|X}(y|x=a) \) that has better error profile than the unconditional marginal \( f_Y(y) \). (both conditional and marginal functions are derivable from the joint) "Better profile" is minimizing future expected surprisal. ("expected" there implies knowledge, the joint pdf; only having knowledge gives rise to expectation.) "Surprisal" is the residual part we fail to forecast.
   Aside: Chain-of-Reasoning is when we come up with R, which is not observed but created as an idea in our mind. R can be a possible factual like X (possible in the world). But also R can be a counter-factual, impossible to appear as observation X. Either way, this has to be such R, that P(Y|R,X) brings us closer to a "good" Y than P(Y|X) does. (NB from the joint we also have P(R|X) at our disposal too.) These CoR discrete jumps via R, (possibly across local saddle points?) aiding searches via R, can't be steps too far. They have to be close (~20%?) to the current state, for us not to get lost. (if too far the chance of being of use drops a lot; like stepping over stones crossing a river)

Q: How would you test for machine consciousness?

General: pursue it assuming very low SNR to any evidence brought in. Via commonality - wherever I observe some consciousness effect 1-3 above, I then seek evidence to support finding 4-8 above. Via contrasting - wherever I observe consciousness effect 1-3 above, where I found 4-8 above, now I look to remove 4-8, and see if the effect 1-3 disappears. So compare and contrast both. Need to gather both positive pro- and negative anti- evidence, as in the case of consciousness, both the Q-uestion, and the A-nswer, are unknown. There are too many moving parts for much comfort. Maybe that's the "hard problem" referenced??

Concrete steps:

9. Given it's shared by all living brains, assume it's shared by all AI brains. Take a bunch of LLM-s. Then look - along the lines of the work of Evelina Fedorenko seeking commonality in common language space - look find common things between them, that can be fit in the 1-3 observations above.
10. Look at work of the likes like Chris Olah, try divine features that are structural. And than look again for that commonality in all of them, with an eye to 1-3.
11. Assuming all current LLM-s are conscious to non-zero degree, would think of ways how to zombify them. Whatever I think I found in {9,10} above, try to destroy undo that. Do the networks become less conscious now? In terms of 4-8?

Addendum on the consciousness experiments designs. (aug2025)

(even if I think it a nothing burger as learned from your Bach Re: Aristotle, Dennett)

Look at known but still unresolved why/how-s differences between analogue HI (all life really, and at all levels of org hierarchy) and digital AI-s:

12. Perpetual learning without end. Digital/AI systems collapse. (so we stop training before) Whereas analogue/HI systems (all live systems in general) don't crash and burn, learning without end doesn't undermine them. (except in psychiatric/mind illnesses maybe?)
13. Truly online learning. Analogue/HI has no batches, no memory except for Now, no buffers no replays, (except maybe memory consolidation while at sleep states?) is at permanent epoch 1. Whereas digital/HI have mini-batches so we need buffers, and epochs >1 so we need "true" (computer) memory. That learning regime is not even that well justified by the maths of it. (more like "motivated" by somewhat hand-wavy control theories)
14. Counter arguments to 12 and or 13 above, why not do them. Maybe the above 12/13 are not connected strictly to consciousness, but connected to: HI is good with many more parameters than observations, while (current) AI is good with many more observations than parameters. (but this can be folded in the experiments to test too?)

Addendum after listening to the Tegmark ToE interview. (sep2025)

I liked everything Max Tegmark said on that. And especially liked him say "enough with the aww-shucks what impenetrable mystery could-be-this could-be-that impossible for us mere mortals to tell; let's design experiments, and measure, and yes it's not easiest to measure but not impossible too - physicists measure far more nuanced things all the time, and lets start ruling out some guesses."

"How Physics Absorbed Artificial Intelligence & (Soon) Consciousness" https://www.youtube.com/watch?v=-gekVfUAS7c, Theories of Everything with Curt Jaimungal, Sep-2025

Consciousness view: it is like the conductor in the orchestra. It is the router in an Mixture of Experts model. Consciousness module is the router in MoE. Experts in the MoE are the individual members of the orchestra, every one playing their own instrument. So while the router is not a very big or a very special module (in fact - it's in many ways simpler then the specialised modules) - it's a single point of failure. So once consciousness (in HI brain) / router (in IA MoE) fails - no expert can learn properly, or even if the experts learns, the knowledge can not be utilised.

MoE architecture is the reason why it's so data efficient. Sparse representations, by virtue of injecting that prior knowledge in the process ("these connections for this data do not need updating"), can be data efficient. It's efficient to know in advance "this data is no use to Experts 1,3,4,5, and is to be used to reach only Expert#2". MoE is a reason why we have too many neurons. Our brains are less efficient than NN-s when it comes to utilising their weight. NN-s are much more efficient than us humans, when looking at efficiency in weights sizes space. Our brains trade parsimony in weights space, to gain efficiencies to gain speed and reduce power consumption - both achieved by MoE.

--
LJ HPD Sun 27 Jul 2025 15:12:26 BST