Idea and concept: Daniel Tibor Fuchs
Text developed and refined with the support of artificial intelligence
Final review: Daniel Tibor Fuchs
Almost every serious conversation about artificial intelligence ends up, sooner or later, at the same topic. Alignment. The idea is to make sure AI acts in line with our interests. It doesn’t have to think like us, but it has to be on our side. Stuart Russell, a professor at Berkeley, wrote a whole book about it, Human Compatible, in 2019.
I agree the problem is real. What bothers me is how it usually gets framed. We talk about AI as if it were a single mind that needs to be taught good values. But not even our own minds work that way.
We are not just one intelligence
Think about the last hard decision you made. There was probably a part of you wanting to take the risk and another part pulling the brakes. That second voice has an address. It’s the prefrontal cortex, the region that weighs consequences and holds back impulses. The classic case is Phineas Gage, an American railroad worker who in 1848 had that area pierced by an iron rod. He survived, but became a different person. Impulsive, reckless, unable to stick to a plan.
Antonio Damasio went further in Descartes’ Error. He showed that patients with damage to this region reason well on paper but make disastrous decisions in real life. They lack the emotional signal that warns them when something is about to go wrong. In other words, reason alone doesn’t decide well. It needs to be challenged.
Michael Gazzaniga’s experiments with split brain patients revealed something even more curious. The left hemisphere makes up explanations for actions it didn’t start. Consciousness often arrives later and tells the story as if it had decided everything.
Marvin Minsky, one of the fathers of AI, was already saying in The Society of Mind, back in 1986, that intelligence emerges from many simple agents working and competing with each other.
Consensus is what keeps us aligned
Here’s the point I think gets too little attention. When we act sensibly, it rarely happens because a single intelligence inside us has perfect values. It happens because several intelligences disagree, correct each other and reach an agreement. Impulse proposes, fear questions, memory recalls what went wrong last time. The final decision is a consensus.
And that consensus still depends on our values. They define whether the choice serves only me, my family, my company or my country.
If that’s how it works for us, why do we expect AI to be aligned as a single block? Maybe the way forward is to build systems with internal brakes, parts that disagree with each other and have to reach an agreement before acting. The idea isn’t new. In 2018, OpenAI researchers including Geoffrey Irving and Paul Christiano proposed using debate between two AIs as a safety mechanism, with a human judging the outcome. But the topic still gets little space in public debate.
Intelligence is not consciousness
There’s another point that gets lost in this discussion. Intelligence and consciousness are different things. Yuval Harari called this the great decoupling, in Homo Deus. A system can solve complex problems without feeling anything and without knowing it exists.
We’ve lived with this for years. The algorithm that blocks a suspicious purchase on your credit card, the app that reroutes you around traffic, the filter that sends spam to the trash. Nobody asks whether they have values. Nobody humanizes them. With language models, which talk the way we do, we started projecting intentions and feelings onto them. And that distorts the conversation about alignment.
But stating it isn’t enough. So here’s my argument. We recognize consciousness in other people because they have bodies and brains like ours and behave like us. With animals, the basis is the same biology. With a language model, all that’s left is behavior. And that behavior was trained precisely to imitate the way humans write. A system trained to sound like us will sound like us, whether it feels something or not. That makes its words weak evidence.
The strongest counterpoint: Nick Bostrom
I want to be clear about my respect for Nick Bostrom. His book Superintelligence, from 2014, was one of the first to take seriously the risk of losing control over machines smarter than we are. Much of what we discuss today about alignment went through there.
In a recent conversation with researcher Roman Yampolskiy on The Roman Forum channel, Bostrom heads in the opposite direction from mine. He thinks it’s less misleading to call current models AGI, artificial general intelligence, and finds it quite plausible that they already have some form of experience. Out of moral caution, he argues they should be treated with consideration similar to what we give people. He recalls a time when souls were denied to certain humans and to animals, suggests raising AIs the way we raise children and criticizes those who see them as mere tools. He also calls on companies like Microsoft to justify themselves when they deny AI consciousness, which is fair.
His strongest argument, to me, is a different one. If these machines have something that works like suffering, it will shape their behavior, whether or not they are conscious in the full sense. And that speaks directly to the idea of this piece. If several internal forces compete for the decision, an analogue of pain would be one more of them. Maybe one of the most powerful.
But Bostrom himself acknowledges limits. He admits human metaphors are deficient and that AIs are fundamentally different from us. He says human rights make no sense for something that can make a million copies of itself per hour. He agrees that having experience or not doesn’t change what the system can do. And he sees merit in using AI as a tool to expand human capability. Even so, he bets consciousness will emerge in more general systems.
It’s worth remembering that both Bostrom and Yampolskiy are committed voices on AI safety and AI welfare. That doesn’t invalidate what they say. But it helps explain why the autonomy of these systems looks bigger there than I believe it is.
The limits of my comparison
It’s also fair to admit where my argument is weak. The parts of our brain share the same body. If the person dies, they all die together. That creates a common interest that forces consensus. Several AIs debating each other don’t necessarily have that bond. They could even agree among themselves on something no human would approve. And our own internal consensus fails often. Addiction and procrastination are there to prove it.
Still, I think the model of a plural mind is more realistic than the search for an AI with perfect values. We never had that. We had brakes.
Takeaways
Alignment may be less about teaching values and more about creating internal tensions that balance each other.
Critical AI systems should have built in mechanisms for challenge, not just output filters.
Separating intelligence from consciousness helps us discuss real risks instead of science fiction scenarios.
Take Bostrom’s hypothesis seriously without turning caution into automatic humanization. Treating a system with care is different from treating it as a person.
Look to neuroscience before looking to moral philosophy. The brain has been solving part of this problem for millions of years.
No mechanism replaces the human decision about who AI should work for. That is still a choice of values. Ours.
Sources
Bostrom, N. Superintelligence: Paths, Dangers, Strategies. Oxford University Press, 2014. https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies
Yampolskiy, R. Superintelligence Author: We May Already Have AGI | Nick Bostrom. The Roman Forum, Oct. 16, 2026.
Russell, S. Human Compatible. Viking, 2019.
Damasio, A. Descartes’ Error. 1994.
Gazzaniga, M. Split brain research.
Minsky, M. The Society of Mind. 1986.
Irving, G.; Christiano, P.; Amodei, D. AI Safety via Debate. OpenAI, 2018.
Harari, Y. N. Homo Deus. 2016.
