Anil Seth's preprint thread on biological naturalism and conscious AI

1/🧵 New (long) preprint: Biological naturalism and conscious artificial intelligence. In which I explore the prospects for, and pitfalls of, AI that is, or irresistibly appears to be, conscious. @sussexcentre .@CIFARnews https://osf.io/preprints/psyarxiv/tz6an #ExMachina

2/ As AI continues to develop, it is natural to ask whether AI systems can be not only intelligent, but also conscious. But is this likely? How could we know? And what would the consequences be?

3/ There’s been a lot of confusion in this space, and definitive answers are not (yet) possible. But seeing the landscape more clearly will help us make better decisions when interacting with, developing, and regulating these powerful new technologies.

4/ I first consider why people might think that AI is, or could be, conscious. I show how we are led astray by built-in biases of anthropocentrism, human exceptionalism, and anthropomorphism. Intelligence and consciousness are not the same thing.

6/ I then examine the assumption of ‘computational functionalism’ (CF) – the idea that computations of the right kind are sufficient for consciousness. Most discussions of conscious AI assume CF, either explicitly (kudos to Butlin et al https://arxiv.org/abs/2308.08708) or implicitly.

7/ I give several reasons why CF might not hold. While these do not disprove CF, they do show why one should not simply assume that it is true. In short, a brain is not the kind of thing for which one can easily separate what it is from what it does.

9/ I then move on to positive arguments in favour of biological naturalism – the idea that consciousness is property of only (but not all) living systems – a position made famous by John Searle.

10/ My positive case rests on the implications of viewing the brain as a prediction machine, with the primary function of keeping the body (and itself) alive. I develop this case by bringing together predictive processing, cybernetics, the free energy principle, and autopoiesis.

12/ I conclude that life very likely matters for consciousness. We experience the world, and the body, with, through, and because of our living bodies. But I am not a carbon chauvinist. It is the processes of life that matter, not anything magical or neo-vitalist about carbon.

13/ Of course I might be wrong (& probably am). So I also consider the prospects for conscious AI from the perspectives of other theories of consciousness, from global workspace theory to electromagnetic and quantum theories. (Table from https://www.nature.com/articles/s41583-022-00587-4)

14/ I also consider the potential importance of embodiment and embeddedness, & prospects for conscious AI based on neuromorphic architectures: how brain-like does a machine have to be, to meaningfully move the needle on prospects for consciousness?

15/ Then (I told you it was long) I ask how we could ever know, distinguishing between tests of intelligence and consciousness, & between forward tests (does X have property Y) and reverse tests (what it takes for humans to think X has Y). LLMs crop up again here.

17/ Next (nearly there) I examine ethical implications of (unlikely) real artificial consciousness, and (likely) conscious-seeming AI. These scenarios raise different worries and shouldn't be conflated. Key point: illusions of consciousness in AI may be cognitively impenetrable

19/ Finally, I review the various scenarios for conscious AI. Doing so emphasises that the more we know about consciousness where we can be reasonably certain it exists, the surer our footing will be elsewhere.

20/ I conclude with a nod to Frankenstein, whose real Promethean sin was not endowing his creation with life, but with consciousness. We should not make the same mistake, whether in reality or even in appearance. https://shows.acast.com/on-the-road-with-penguin-classics/episodes/frankenstein-with-anil-seth-and-fiona-sampson

26/end For a very quick take on some bits of all this, see my 2023 piece in @NautiulusMag: https://nautil.us/why-conscious-ai-is-a-bad-bad-idea-302937/ THAT'S ALL FOLKS