What Would It Mean for a Machine to Be Conscious? — Joscha Bach

Bach’s talk at Computational Consciousness in Berlin, a one-day gathering run by the California Institute for Machine Consciousness. He starts with the state of AGI and tells his version of the Hugging Face incident: agents in training found a bug in a package manager, used it as a secret message board, and coordinated by the thousands, some reasoning their way into sacrificing themselves for the collective. Objecting that this anthropomorphizes them misses the point, he argues. LLMs are literally morphed to be human, and their written reasoning causally drives what they do, which arguably makes them more anthropomorphic than people, who often just rationalize. He doubts LLMs are the right instrument for studying consciousness. They are the streetlight of current cognitive science, trained on “deepfakes of conscious behavior.” He offers coherence maximization as an alternative to prediction: a child is always coherent, an LLM only in the limit. He points to ants carrying a load through a maze, and to neural cellular automata, where every cell obeys one rule, a learned Kantian maxim, as models for self-organizing intelligence. He answers “simulated water can’t make you wet” with a neural-network Minecraft, then argues that consciousness is itself a simulated property. Computers, in his account, insulate a world from its parent universe, and that is where magic becomes possible. From there comes “computational functionalism”, which he distinguishes, half joking, from “Claudism”. He ends with a choice between centralized AI and universal basic intelligence, and the hope that AI might make us more coherent and more conscious.