saved
The Mythology Of Conscious AI
Hraness cites a source capture. The source author remains the source.
gist
Anil Seth's Noema essay is the accessible companion to his BBS argument: people keep projecting consciousness onto improving AI because intelligence and experience travel together in us, language models pull anthropomorphic strings, and computational functionalism treats minds as substrate-independent software. He counters that brains are living systems, not Turing machines made of meat, and that conscious-seeming AI can already warp moral concern even when silicon systems remain dark inside.
ideas
- Intelligence and consciousness come apart. Flexible goal achievement is about doing; consciousness is about being, and the two need not rise together.
- Language biases the bet. Chat systems get consciousness attributions that silent but powerful predictors rarely do, because language cues personhood.
- The computer metaphor overreaches. Treating brains as substrate-independent algorithms underwrites conscious-AI optimism without earning it.
- Seeming can outrun being. Persuasive consciousness illusions may be as sticky as visual illusions, creating welfare and policy traps before real machine experience arrives.
quotes
“But are they, or could they ever be, conscious?”
“Consciousness, in contrast to intelligence, is mostly about being.”
“The brain is not a Turing machine made of meat.”
“If we sell ourselves too cheaply to our machine creations, we overestimate them, and we underestimate ourselves.”