One of the most fascinating and potentially consequential debates taking place among the people developing artificial intelligence concerns a question that sounds deceptively simple: Do we actually understand what we are building?
The question comes from the fact that leading developers at play know how to design AI models, train them on enormous quantities of information, and improve their performance in solving problems that, until recently, required considerable human expertise. Most importantly, they are using AI to develop more sophisticated AI systems.
But there is an uncomfortable reality behind these extraordinary advances: we still cannot fully explain how the complex internal operations of these models produce many of their results.
In other words, in the AI world, high functionality is leaving knowledge behind: we know that AI works, but we know less and less about how.
Think of an automobile engine. A competent engineer can take it apart, identify every important component, explain precisely how each interacts with the others, and determine why the engine performs as it does.
But AI is different. Although human engineers design the architecture and training procedures, much of what a neural network actually learns emerges through training rather than through the deliberate programming of individual functions. Its internal organization develops through a process that resembles the growth of an organism more than conventional mechanical construction.
In a fascinating new essay titled The Beauty of Something Grown, published October 16, Dean W. Ball explores this problem and a potentially revolutionary development that could change the entire debate.
Ball is particularly well positioned to address the subject. He currently serves as Head of Strategic Futures at OpenAI and as a visiting fellow at the Heritage Foundation, having previously helped shape the Trump administration’s artificial intelligence policy. His analysis brings together technical understanding, political realism, and an unusual willingness to challenge the conventional assumptions surrounding AI.
The issue he examines is known as mechanistic interpretability: the scientific effort to discover what is actually happening inside a neural network when it produces an answer, makes a decision, or performs a complex task.
The distinction is more important than it might first appear. A system that consistently produces useful results is not necessarily a system whose internal behavior we understand. And this is one reason some prominent figures in the AI community have warned that the technology could eventually escape meaningful human control, particularly if increasingly capable AI systems begin playing a decisive role in designing and improving their successors.
Their concern is not merely that AI might become more intelligent than humans in particular areas. It is that we might find ourselves depending on systems whose capabilities are expanding faster than our ability to understand, predict, and govern their behavior.
Yet Ball introduces an intriguing possibility: what if our inability to understand AI has less to do with the nature of the technology than with the limitations of the conceptual tools we have been using to investigate it?
His essay examines a provocative new hypothesis advanced by Nick Cammarata, a pioneering researcher in AI interpretability who previously worked at OpenAI and is now associated with Goodfire.
In a paper titled Neural Networks Are Already Clean, Cammarata argues that the neural networks that make AI may be far more understandable than researchers have assumed. He believes that the tangle of artificial neurons possess an intelligible internal organization, but only if we learn to examine it “on its own terms” rather than demanding that it conform to our expectations.
Cammarata argues -with some interesting examples worth reading- that neural networks are not intrinsically inscrutable, because their internal organization may follow discoverable patterns that become comprehensible once we develop the right scientific methods.
Such a breakthrough would not instantly make AI safe or predictable; but Ball argues that the possibility is remarkable: a problem widely considered one of the greatest obstacles to AI safety might prove considerably more manageable than previously believed.
What makes Ball’s analysis especially interesting is that he moves beyond the technical argument to examine its political and philosophical consequences.
The debate over AI development is often portrayed as a struggle between two groups: the so-called accelerationists, who believe its progress should continue rapidly, and the doomers, who fear that sufficiently advanced AI could eventually cause catastrophic harm, including the loss of human control.
But Ball identifies a crucial division within the camp of those concerned about AI safety, one that could become much clearer if interpretability were substantially solved.
Some worry primarily about alignment: whether increasingly intelligent systems can be made to pursue objectives compatible with human intentions and values. For these people, the prospect of truly understanding and reliably governing advanced AI would represent a tremendous breakthrough. It could open the door to developing extraordinary technological capabilities without accepting unacceptable risks.
Others, however, fear something more fundamental. Their concern is not simply that superintelligent AI might become hostile, deceptive, or uncontrollable. They worry that the very existence of an “intelligence” that in practice operates in a manner that seems vastly superior to our own could ultimately disempower humanity, even if those systems behaved exactly as their designers intended.
This is the deeper disagreement, and Ball recognizes its significance. A major scientific advance could reassure one group while making the other profoundly more anxious.
For the first, better understanding means greater safety and greater opportunity. For the second, better understanding could simply remove the final obstacle to developing a technology whose existence might permanently alter humanity’s place in the world.
This is precisely where I believe the Catholic contribution to the debate becomes indispensable; since the Catholic intellectual refuses to reduce the question of technology to a simple conflict between optimism and fear.
The Church has never regarded the human capacity to understand and transform the material world as something intrinsically suspect. On the contrary, human intelligence, creativity, and the capacity to discover the laws governing creation are expressions of our dignity as beings made in the image and likeness of God.
The very existence of scientific inquiry rests on a conviction deeply compatible with the Christian understanding of creation: that the universe possesses an intelligible order, and that the human mind is capable, however imperfectly, of discovering it.
This does not mean that every scientific theory is correct or that every technological breakthrough is necessarily good. It means that there is no theological reason to assume that the internal complexity of artificial intelligence must remain forever beyond human comprehension.
Indeed, the possibility proposed by Cammarata is particularly interesting when viewed through this lens. Something that appears chaotic may reveal a remarkable intelligibility once we learn how to understand it.
But the Catholic tradition also introduces another essential distinction: the ability to understand something does not automatically justify everything we might choose to do with it.
That is to say that even if AI becomes fully interpretable, the moral questions surrounding its deployment will remain: Who will control these systems? Who will benefit from them? Will they strengthen families, communities, education, and meaningful human work, or undermine the institutions through which human beings flourish?
And, perhaps most importantly, will they remain genuinely at the service of the human person?
The Catholic tradition should make us equally skeptical of both so-called accelerationists and doomers.
Ball’s lesson in this regard is that if interpretability improves dramatically, it could reduce genuine technical risks and make extraordinary new applications possible. That would be good news. We should not be embarrassed to acknowledge it.
But we should also recognize that the better we become at building powerful AI systems, the more urgent the unresolved questions of governance, accountability, economic justice, and human freedom will become.
The greatest question, in short, will remain exactly where it has always belonged: not inside the machine, but within the human conscience responsible for deciding what to do with it.