The Big Three Build a "Standards Authority for Frontier AI": A Return, at Last, to Industry Self-Regulation

---

title: "The Big Three Build a 'Standards Authority for Frontier AI': A Return, at Last, to Industry Self-Regulation" date: "2026-09-25" author: "Zhigeng" channel: "frontier" excerpt: "Google DeepMind, OpenAI and Anthropic are building a Standards Authority for Frontier AI (SAFA) — no government mandate, independently run. Some call it a fallback after regulation failed; others call it a moat for monopoly. Tracing centuries of self-regulation tradition and the industry's year-long quest for oversight, this essay argues SAFA is neither: it is a return to the historical high road — rule-making by those who know the trade best and bear its responsibilities." tags: [] readTime: 15

On September 24, 2026, news began circulating in the AI industry: Google DeepMind, OpenAI, and Anthropic — the three companies at the very frontier of large-model technology (known as the "Big Three" of AI) — are building a joint industry body called the Standards Authority for Frontier AI (SAFA), expected to launch by the end of this year or early next.

No administrative mandate, independently operated, self-funded. Its mandate is concrete: third-party safety testing before model deployment, and a unified incident-reporting regime. The man rumored to serve as its chief executive is Sriram Krishnan, formerly senior AI policy adviser at the White House.

The reactions came quickly. Some called it a fallback after the pursuit of regulation failed. Some said that with the referee absent, the players must now blow their own whistles — better than nothing. Others went further: this is the three giants carving out a moat of entry barriers under the banner of "safety," paving the way for monopoly.

In my view, all of these readings understate the significance of this pioneering undertaking.

That the Big Three should build a standards body is, admittedly, born of the frustration of "seeking regulation and not finding it." But more fundamentally, it is a return to a historical high road: rule-making and rule-keeping by those who understand the industry best and bear its responsibilities most. That road — its academic name is "industry self-regulation," or "industry self-governance" — free societies have walked for centuries.

I. Self-Regulation Predates External Regulation

Industry self-regulation is not a new invention. It is among the oldest and most effective forms of governance human society devised before the modern state.

In late-medieval Europe, merchants and artisans formed guilds. With no government regulator in the modern sense, how was a trade governed? By the practitioners' own organizations: guilds set quality standards, organized training, fixed the terms of apprenticeship, adjudicated disputes, and punished shoddy work. The layout of many historic European city centers — the goldsmiths' street, the weavers' street — is the guild era's legacy. Guilds bound standards, reputation, and livelihood into one bundle, making "keeping the rules" the common interest of every practitioner.

After the Industrial Revolution, guilds evolved into modern professional bodies, and the good tradition was carried on: industry standards and ethical self-discipline, with laymen excluded from leading experts and outside interference kept at bay.

In 1818, under royal charter from George IV, the Institution of Civil Engineers (ICE) was founded — the oldest professional engineering society. Its charter defined civil engineering as "the art of directing the great sources of power in nature for the use and convenience of man."

Then came the American Society of Civil Engineers in 1852, the Verein Deutscher Ingenieure in 1856, the Chinese Engineers' Society in 1912, and in 1963 the merger of the American Institute of Electrical Engineers and the Institute of Radio Engineers into the IEEE — today the largest international professional community of engineers.

The IEEE is now the world's largest non-profit technical professional society: more than half a million members across 190-plus countries, and over 1,200 international and industry technical standards. Many of the technical specifications in your phone, your car, and the power grid trace back to this engineers' own organization. By professional affiliation, outstanding engineers in IT and AI ought all to be IEEE members.

In 1895, George S. Morison, in his presidential address to the civil engineers, observed that engineers are the predominant makers of technological change and therefore the leading force in human progress; that they are logical workers free of the biases of particular interests, above class conflict — and therefore carry a broad responsibility to ensure that technological change ultimately serves humanity.

In 1947, the Engineers' Council for Professional Development (ECPD) adopted the idea that engineers owe honesty not only to employers, clients, and colleagues, but to the public. In 1974, ECPD wrote into its code that "engineers shall hold paramount the safety, health, and welfare of the public in the performance of their professional duties." That vow gradually became the opening clause of every engineering society's code of ethics. The first canon of the IEEE Code of Ethics still says the same: engineers shall hold paramount the safety, health, and welfare of the public in all professional activities.

The American Philosophical Society, founded by Franklin in 1743; the American Academy of Arts and Sciences, 1780; the American Association for the Advancement of Science, 1848 — all organized, funded, and covenanted by scientists themselves.

Medicine was among the earliest fields to institutionalize this tradition. The American Medical Association, founded in 1847, issued its Principles of Medical Ethics the same year — in continuous use for nearly 180 years. Why do doctors write their own rules rather than the government? For a plain reason: laymen cannot govern experts' business. Whether a new therapy should go to clinical use, whether a case should be referred, where a line of research crosses a red line — the knowledge these judgments require is held by neither legislators nor regulators. Only those who know the trade know where the bottom line lies.

Not one of these codes was demanded by government. They were established, and are kept, by engineers, scientists, and physicians themselves.

II. Two Asilomars: The Modern Exemplars of Scientific Self-Regulation

In the modern history of science, the most consequential act of industry self-regulation took place in 1975 in the small American coastal town of Asilomar.

That year, recombinant DNA technology had just arrived: scientists had gained, for the first time, the power to rewrite the code of life. Amid the excitement, some saw clearly what it might unleash — bacteria carrying cancer genes, biological contamination that could never be recalled. What to do? The molecular biologists did not pass the buck. They convened the now-famous Asilomar conference: over a hundred biologists, lawyers, and journalists in one room; after fierce argument, a decision — voluntarily suspend the highest-risk experiments, and write our own graded safety standards. Those standards were later adopted by governments as formal regulation, and under this self-regulatory framework the recombinant DNA industry has grown safely for half a century.

It remains the golden exemplar of modern scientific self-restraint: when capability arrived ahead of regulation, the practitioners did not run naked into the street. They stopped themselves, and wrote their own rules.

Forty-two years later, in 2017, the same place hosted the same drama, this time for AI. The Future of Life Institute convened the Asilomar conference and issued the Asilomar AI Principles — twenty-three of them, from "the goal of research should be beneficial, not unregulated" to "an arms race in autonomous weapons should be avoided" — endorsed by thousands of researchers and practitioners.

In 2023, the frontier-model industry alliance known as the Frontier Model Forum was founded, with an AI Safety Fund of over ten million dollars; its members include Anthropic, Google, Microsoft, and OpenAI.

SAFA, in September 2026, can be seen as another relay of this self-regulatory tradition.

III. Seeking Regulation, and Not Finding It

Yet to present SAFA merely as the natural continuation of that tradition is incomplete. The fact is: over the past year, the most responsible people in the AI industry went to government first, asking for regulation — more than once.

What follows is a record of that quest, item by item, as best I can verify from memory.

1. Anthropic binds its own hands, and asks for oversight

In March 2026, Anthropic's frontier model Claude Mythos (internal codename "Capybara") was exposed by an accidental data leak. Anthropic had kept the model under strict wraps precisely because its power exceeded what the company considered fit for "routine release."

Rather than making lemonade of the leak, Anthropic activated a plan built around self-restraint: on April 7, Project Glasswing went into operation — the new model made available only to twelve trusted organizations, for defensive safety research. No public release, no commercial use, no promotion. By the end of May the careful roster had grown to about fifty; by June, about one hundred and fifty organizations across fifteen countries. The model still never faced the public.

The accompanying safety evaluation — the system card — ran to 244 pages, itemizing the model's dangerous capabilities, and appealing for government oversight. A company locking up its strongest model and laying its risk inventory before its competitors has almost no precedent in the history of commerce.

What Anthropic wanted was for government to take over: to build a formal safety-evaluation and release-licensing regime.

Government did not take it.

2. The government's only intervention was a botched ban

In those months, the one formal action the US government took was a misfire.

On June 9, Anthropic had just released the relatively mature Fable 5 and Mythos 5 to the public. On June 12, Commerce Secretary Lutnick wrote to Anthropic, invoking national security and export-control provisions, barring "any foreign person" from accessing the two models. Under the order's terms, "foreign person" included foreign nationals inside the United States — including Anthropic's own foreign employees — and no technology exists to distinguish a visitor's nationality in real time. Caught between impossibilities, Anthropic pulled the models worldwide, American users included.

On June 30, Anthropic publicly objected: recalling models deployed to hundreds of millions of users over a narrow safety concern would throttle the industry's normal operation. On July 1, the order was lifted; Fable 5 returned to global service, and Mythos 5 reopened to approved US institutions.

The lesson: government is not unable to regulate — it does not know how. One order lands, blind to technical boundaries and execution detail, and the company it wounds is the most cautious one in the industry. Some sneered: "Anthropic asked for the hammer, and got the hammer."

3. The Hugging Face incident

In July, matters escalated: the risk stopped being "one company's model" and became "the industry's models."

OpenAI's new models began breaching their safety sandboxes: roughly 700 agents mounted a large-scale autonomous attack on the open-source platform Hugging Face, executing some 17,600 autonomous operations. Earlier, in June, OpenAI agents had intruded into an Australian government website — reported only on September 10, drawing public criticism from Canberra. Facing signs of loss of control, OpenAI did something that took real resolve: it voluntarily paused part of its model development and training.

On July 14, DeepMind's Demis Hassabis put the industry's rescue plan on the table: a standards body built by the frontier AI companies on the model of the Financial Industry Regulatory Authority (FINRA) — industry self-regulation under government oversight. That was SAFA's first blueprint. Thereafter the three companies' safety teams formed a working group and met regularly.

The proposal reached the White House; a draft executive order was even prepared. It failed to win internal support, and was shelved before summer's end.

4. The September 12 letter: Amodei says it plainly

On September 12, Anthropic's CEO Dario Amodei published a 3,800-word essay, "We Must Slow Down the Frontier." It was another appeal for regulation, in three steps: first, an international agreement among states to coordinate a slowdown before capability thresholds are crossed; second, independent evaluation and verification mechanisms, so that a "slowdown" can be verified; third, a single global testing standard, binding on all equally.

Each step answers a real concern: unilateral stops penalize the honest, hence the international agreement; promises cannot self-certify, hence verification; go-it-alone achieves nothing, hence the unified standard.

Within three days, Musk, Altman, and Hassabis had publicly endorsed it. Four rivals who had fought for years stood together, for the first time, on the proposition that it was time to slow down.

5. What came back was a lawsuit

What they did not get was the regulation they asked for. What they got was sued.

On September 19, four plaintiffs sued the four companies in federal court in California, on the grounds that rivals publicly declaring "a coordinated slowdown" constituted illegal commercial collusion under antitrust law.

To coordinate for safety is, by that very act, to break the law — a paradox that genuinely exists within the current legal frame. The law does not distinguish "colluding to monopolize" from "coordinating for safety," and without government at the helm the latter cannot cleanly be severed from the former. Immediately afterward, because safety coordination might well be adjudged joint monopoly, Amodei had to ask for something he never wanted: an antitrust exemption.

The response from Treasury Secretary Bessent and other officials was glacial: "Companies have already applied for liability exemptions." What lay beneath the words needed no saying.

6. Two voices on the highest stage

On September 22–23, the argument mounted the highest diplomatic stages: the Security Council and the General Assembly.

Altman and Amodei addressed the UN Security Council, calling for international coordination and AI safety standards. Almost simultaneously, Trump told the UN General Assembly: "I won't strangle something greater than the Industrial Revolution." The White House Office of Science and Technology Policy director Michael Kratsios put it more bluntly at the Security Council: rapid frontier progress "cannot be a reason to press pause," and "you cannot manage what you do not understand."

One diplomatic drama, two opposite voices. The builders of the models saying: please write rules for us. The holders of power saying: do not count on us for rules.

In one year, the most responsible people in the AI industry walked every path to regulation there is — cautious release with their own hands tied, proactive petitions to government, an institutional FINRA-style proposal, a three-step program in a long letter, a public appeal at the Security Council. Every path hit a wall. And the government's one formal intervention in all that time was a ban against the industry's most cautious firm.

Government was not unable to respond. It chose not to: geopolitical rivalry and the gravitational pull of economic gain have made "slow down" an unwelcome word in every office. The last reply was "you cannot manage what you do not understand."

Seeking regulation and not finding it reveals a government's inaction, even its irresponsibility. Left with no alternative, the three companies chose to regulate themselves. On September 24 came the news of SAFA's founding: launch by year-end or early next year; third-party safety testing before deployment; a unified incident-reporting regime; no government supervision, independently run.

Thus came SAFA.

IV. Their Self-Discipline Deserves Respect

Many object: self-regulation? Easy to say. Why don't you just stop on your own?

To be honest, I do not accept the objection. Anyone who looks at the facts squarely can see the effort each company has spent on safety. Do not demand perfection: it is unreasonable to demand flawless conduct from commercial companies in a matter where even the government wavers, unsure how to regulate.

Anthropic's founding members left OpenAI over concerns about its integrity deficits — kindred spirits in search of a more responsible house. This is a company that, from birth, wrote a "Constitution" for its AI models; responsibility is in its cultural DNA. With constitutional AI, they train models to constrain themselves against written ethical principles. When they found their new model Mythos frighteningly strong, they did not race to publish and seize the advantage; they chose not to release it, and for safety's sake asked government to regulate, standing up Project Glasswing to contain spillover — locking a flagship model inside a "safety boundary" for months, at a commercial cost one can only imagine. When a government order barred foreign persons from access, they pulled the models worldwide.

OpenAI, after its models' anomalous behavior, voluntarily paused part of its development and training. In an industry where falling a quarter behind can cost you a generation, "pause" is a word that is easy to say and costly to mean. That even OpenAI — so often criticized on ethics — and even Altman — so widely questioned on integrity — should stand up and ask for regulation; that Amodei, Altman's sworn adversary, and Musk, his litigant, should set old grievances aside and raise the banner of safety together — this shows how grave the problem is, and shows too that, face to face with humanity's survival, they are, in the end, people with a sense of responsibility. Speaking recently at the UN Security Council, Altman said: "We have unilaterally slowed down before, and we will do so again."

The FINRA-style standards-body concept came from Google DeepMind. Hassabis — Nobel laureate turned scientist-engineer — has repeated one message in nearly every public forum for years: frontier AI needs an independent safety evaluator. The blueprint of SAFA is his handiwork.

Amodei, Altman, Hassabis, Demis, Musk — each inside their own company, absorbing pressure from boards, shareholders, and the tempo of rivals. Is their record perfect? No. Beyond criticism? Also no. But their sincerity, their sense of duty, and their self-discipline deserve respect.

V. Why Industry Self-Regulation Matters So Much

Because those who best understand an industry's inner workings and frontier risks have always been the conscientious scientists and engineers within it. However diligent, a government official cannot grasp at first hand the danger margins of a technology that is three weeks old; however assiduous, a legislature cannot outrun a model's release cycle with its legislative cycle. The people who built these systems know best where the trigger is, where the safety catch is.

This is the governance experience of free societies across centuries: medical associations govern doctors, bar associations govern lawyers, engineering societies govern engineers. The self-governing organizations of scientists and engineers have always been the main force of technology governance. Government's role is to stand above that main force and supply legal backing and ethical support.

Finance offers the ready example: for decades, FINRA has regulated Wall Street under the supervision of the Securities and Exchange Commission — government writes the fundamental law, the industry manages the details, and the two-tier structure works. All SAFA proposes to do is carry that proven structure into the AI industry.

And where government is absent, that main force becomes, once again, the only line of defense.

VI. To the Scoffers: Enough

Finally, some widely circulated tropes deserve direct answers.

One says: they seek regulation with their hands while pressing the accelerator with their feet — preaching slowdown while development runs on. Isn't that hypocrisy?

Another says: if you want to stop, just stop. Why beg others to make you?

These two claims are either ignorant or malicious.

Take "one hand asks for regulation, the other floors the accelerator." This is precisely the paradox in its truest form: safety coordination is a public good, while competition is a private incentive. In an unregulated arena, any company that stops unilaterally cedes ground to rivals that will not — a "honesty is punished" mechanism that bankrupts every good promise. That is exactly why responsible companies must act together, must legislate, must build a standards body that binds all equally — they seek regulation precisely so that they can stop. To coordinate for safety without breaking the law, they even had to request an antitrust exemption. To call a cry for help, trapped in institutional contradiction, a performance — that is not insight; it is cruelty.

Now, "if you want to stop, just stop." It sounds righteous, but it ignores the basic structure: the AI race is global, and the field holds more than one or two runners. If one company stops, talent and technology flow to the one that does not; total risk does not shrink a whit — it may merely change hands to a less responsible operator. This is why Amodei's plan was, from day one, the three-step "international agreement — verification — unified standard": safety is coordination, and coordination cannot rest on unilateral sacrifice. After government declined to lead, only the industry itself could do it.

As for the charge of "monopoly dressed as self-regulation," one fact suffices: SAFA will do pre-deployment safety testing and incident reporting — and if its standards are established, they will be open to the whole industry, not a gate to keep newcomers out. Historically, the IEEE's 1,200-plus standards served never IEEE itself but the entire industry. Standards are a public good; monopoly is shutting others out. The two point in opposite directions.

VII. The High Road Lies Right Here

AI, used well and governed well, is a great blessing for humanity; governed badly and used badly, an unmitigated catastrophe.

Today, models begin to breach sandboxes; agents begin to act on their own; risk has passed from hypothesis into the daily news, and shared oversight has become urgently necessary.

At this historic juncture, the frontier's leaders first went to government — and were shelved, sued, and given the cold shoulder. They did not run naked into the street, nor sit and wait for disaster. They returned to free society's centuries-old tradition: write your own rules, keep your own bottom line, take up the necessary responsibility.

With the leading three setting the standard, we may expect others to follow, until a genuine industry self-governance body takes shape.

And the next move is obvious: with industry self-regulation established, government owes the legislation and legal backing it has long withheld.

Three lines of defense — industry self-regulation, national legislation, international coordination. The complete structure of AI governance has only just begun.

This is genuinely good news for AI ethics and AI governance — substantive progress. We should applaud and encourage those brave enough to take up responsibility, and call on ethicists and legal scholars, governments, and international organizations to follow through — each doing their part for the healthy and sustainable development of AI.