A few weeks ago, I wrote here about the remarkable public appeal from more than a thousand employees of America’s leading AI companies asking governments to develop the capacity to “pace” frontier artificial intelligence.
I welcomed the concern but remained skeptical. The basic problem seemed obvious: asking OpenAI, Anthropic, Google DeepMind and their international competitors to slow down together sounded morally admirable but practically unrealistic. These companies are engaged in one of the fiercest technological races in modern history. Add China to the equation, and the danger becomes even clearer: badly designed regulation could end up restraining the most responsible Western companies while rewarding whichever competitor decides not to play by the rules.
Even worse, “slow down AI” can become a dangerous political slogan. Once governments acquire broad powers to decide which technologies may proceed, there is no guarantee those powers will remain limited to genuine catastrophic risks.
Now Tim Fist and Saif Khan of the Institute for Progress, joined by several other analysts, have offered what I think is a much more realistic way of approaching the problem.
Their essential insight could be described as guardrails for the guardrails.
They do not reject the possibility that AI development may eventually need to be deliberately paced. Nor do they dismiss the increasingly serious concerns coming from people inside frontier laboratories. Instead, they ask a much harder question: If we ever need to slow something down, how do we make sure the mechanisms designed to protect us do not themselves become destructive?
That is exactly the question that was missing from much of the original discussion.
Fist and Khan propose that any intervention should be narrowly focused on activities carrying the possibility of serious and irreversible harm. It should minimize interference with the spread of already useful AI. Ideally, it should actually accelerate beneficial uses. It should remain worthwhile even if some of today’s more frightening forecasts turn out to be exaggerated. It should not punish the companies or countries acting most responsibly. And, crucially, it should not create a regulatory apparatus easily captured by a handful of corporations or misused by whichever political party is running the government.
We have spent time and the proverbial “ink” talking about guardrails for AI. We should now be talking about guardrails for the people who build the guardrails.
From a Catholic perspective, this is simply realism about power. Political authority is necessary. Markets cannot regulate every danger on their own, especially when powerful incentives encourage companies to move faster than their competitors. But the government is not morally immaculate either. Institutions can be captured and “emergency” powers can become permanent powers.
Catholic social thought has never required us to choose between blind faith in the market and blind faith in the state. It asks instead whether power is being exercised prudently, proportionately and in service of the common good. Or in the words of a famous German theologian: “us much society as possible, as much government as necessary”
That is why the idea of “low-regret” AI policy is so attractive.
Rather than announcing a sweeping pause today, Fist and Khan propose preparing institutions for the possibility that certain thresholds may eventually be crossed. For that, they propose:
Build better transparency.
Improve the government’s technical competence.
Develop verification systems so that agreements can actually be enforced.
Strengthen cybersecurity and biosecurity.
Establish specific risk thresholds in advance.
If those thresholds are crossed, redirect some computing power and scientific talent away from the most dangerous forms of automated AI research and toward safer research and useful applications.
That is a much more intelligent version of pacing. In fact, slowing down one narrow area of AI development might allow the benefits of AI to spread faster elsewhere.
Compute and talent diverted from risky recursive self-improvement could instead be used for medicine, scientific discovery, cyberdefense, education and countless commercial applications that already exist.
This is not a choice between brakes and acceleration. A well-designed vehicle needs both.
But prudence does not mean proceeding slowly simply because slowness feels safer. Nor does it mean charging ahead because progress is exciting. Prudence asks what the circumstances actually require and then chooses the appropriate means to achieve a genuine good.
Sometimes prudence says accelerate, sometimes it says brake… and sometimes it says build better brakes precisely so that you can drive faster without killing anyone.
The AI debate desperately needs that kind of thinking.
And perhaps that is the emerging lesson of the AI debate: we need guardrails for artificial intelligence, but we also need guardrails for the guardrails. Because both technological power and regulatory power are human powers; and neither should ever be trusted without limits.