Unsafe AI or Consolidated Power: AI’s False Dichotomy

15 Jun, 2026

Parth Asawa and Joseph E. Gonzalez

We’re excited about AI’s potential to improve society but we’re concerned that the AI community is being polarized into supporting one of two bad outcomes: concentrating power in a few frontier labs, or allowing powerful open models to proliferate without adequate safety oversight.

The relationship between AI safety and concentrated power is complex. People who share the same goals—sustained innovation, preserving democratic accountability, and avoiding or reducing catastrophic risk—are coming to very different conclusions on the path forward.

We think this framing of the debate is wrong. We should be asking how to take safety seriously while enabling decentralized innovation that’s in the broader public interest.

The Two Camps

To provide a simplified overview of the arguments:

1. There’s a camp that believes that frontier models are unsafe due to their potential to be disruptive to society when used by bad actors. As a result, they believe access to these models must be tightly regulated and the ability to create them should not be proliferated more broadly. At the same time, because there is the potential for more powerful versions of this technology to be used in even more adversarial ways by authoritarian regimes or military adversaries, this camp often believes we’re in a race to achieve a lead on these capabilities.
2. The opposing view is that the safety concerns are "fearmongering", sincere or not, a mechanism towards restricting competition and consolidating economic and soft power amongst the labs. This camp is deeply concerned that a consolidation of power around such a transformational technology is in itself, the greatest existential risk. From this perspective, open weight models serve as a counterweight to this consolidation of power by virtue of competition.

Both concerns are absolutely legitimate, but it feels like the polarization of these opposing and simultaneously valid concerns has obscured any imagination of a third approach that strives to address both of these concerns. An ideal democratic ecosystem probably shouldn’t look like open models without any safety regulations, nor a world where power consolidates to a few labs who command wide influence over how AI is deployed, regulated, and used.

Safety

There’s a valid case for safety being made by various labs. For example, recently announced models have demonstrably improved the capabilities of actors to discover vulnerabilities in software systems that could disrupt society [1, 2]. The capabilities of models to advance the potential for bad actors to accelerate development of biological weapons also appears to be increasing with each generation of models [3, 4]. If continued scaling produces broader and more reliable capabilities, then today’s misuse concerns will only become more serious.

The core problem here lies around incentives. The concrete safety argument is often pushed by frontier labs as they’re the ones with the most visibility into what’s possible with frontier models. While they may not be acting in bad faith, these arguments could also be used as a lever through which they enact regulation that limits competition. These conflicts of interest can never be fully separated, which is why it’s naive to believe that frontier labs should be taken at face value when trying to recommend policies around safety and regulation.

If you truly want the safety agenda to be well-received, you have to look for alternative solutions that separate out the conflicts of interest and don’t use it as a justification for permanent opacity. Existing third-party evaluations are a step in this direction, but we don’t think they’re enough. In the current model, private labs still decide who gets access, when they get access, what information is shared, what’s in scope, and how much of the evaluation becomes public. These audits may improve confidence in specific safety claims, but they don’t create a durable public-interest knowledge base outside labs that more broadly informs the discussion.

De-Consolidation of Power

Our second camp takes issue with the straightforward solution to safety: broad restriction of frontier models, which would cause model development to be restricted to only a few frontier labs and the power associated with that lies with a few, unelected individuals.

From a market perspective, this sort of regulation would codify anti-competitive measures such that a few companies become single points of failure for many downstream applications. These companies would also wield extraordinary power over how models are deployed, who can access them, and under what conditions—forms of access control they have already shown a willingness to exercise [5].

From a political perspective, democracy thrives on a decentralization of power and dissemination of knowledge. Open science serves as a foundation of democracy by spreading the knowledge citizens, researchers, and policymakers need to reason independently about powerful technologies.

It’s important to take this statement within reason: knowledge dissemination isn’t a blanket requirement that applies to every domain. Nuclear weapons are the standard historical example here: society would not have been better served by freely proliferating every detail necessary to build nuclear weapons. Simultaneously, even in a world of classified weapons programs, the underlying science was understood broadly enough by bodies who represented the public interest (independent scientists, regulatory bodies, etc.) that they could reason about the technology, warn about its consequences, and push back on government policy.

To be absolutely clear: AI is not a nuclear weapon, and the analogy shouldn’t be taken 1:1. AI is more general-purpose, more economically enabling, and more deeply entangled with civilian infrastructure. But in one aspect, the AI situation is already worse: increasingly few people outside private labs understand how the most capable systems are made, what capabilities they actually have, and how they’re aligned. If policymakers and the public are forced to choose between unrestricted open weights and trusting a small number of private labs, we think we’ve already failed.

Despite the clear need for open science, solely pushing for open weights without simultaneously dealing with the safety problem isn’t a real solution. There’s a lot of calling for open-weight models on social media, but signalling does not by itself create them. Achieving SOTA requires training frontier models with data, talent, infrastructure, and compute at a scale that is increasingly out of reach. Because the return on investment for truly open models is uncertain, there are going to be too few organized, well-funded efforts if we rely purely on private markets. Furthermore, if we could achieve the SOTA open models, given that there are currently no solutions to enforcing safeguards on open weight models once broadly released, and the government is already restricting the deployment of Mythos-class models with safeguards, it should be clear that this won’t be accepted from a regulatory point of view either. Weaker open models on their own would avoid the safety problem, though not resolve the underlying consolidation of power.

A Third Path

We’re left to answer the harder question: what mechanisms decentralize power via open science without compromising on safety?

I don’t think one blog post can prescribe the exact answer. The design should come from serious engagement among all the stakeholders who would need to make it work: frontier labs, academics, policymakers, funders, civil society, and companies building on top of AI. My goal here is to sketch the shape of solutions and make the case that these conversations need to start now.

Claim 1: Regulation will fail if expertise remains concentrated inside labs

• Government involvement makes sense when there are questions around safety, anti-competitive behavior, and national security. If labs truly believe that there needs to be regulation of AI, then there should also be a belief that this regulation is designed by well-informed policymakers. However, policymakers today are largely uninformed and the people who have the closest knowledge of the technology are almost all in the frontier labs.
• This presents a problem from a governance perspective, as rather than be advised by neutral, third party experts/scientists, the frontier labs have put themselves in a position where most meaningful opinions can only come from them. Regulating yourself seems pretty obviously bad, there’s no way you can expect effective regulation in a democracy when technology is only understood by a few people.
• Just last week, we saw how a regulator took actions to introduce regulations that hurt economic growth and have debatable impacts on safety [2]. These disagreements could be attributed to miscommunication, but that’s all the more need for neutral, deeply technical translators in high-stakes decision making; people who truly understand frontier training and evaluation but are not beholden to corporate interests.

Claim 2: Open science for AI needs reimagined institutions

• Historically, academia has served as a counterweight to these problems by pursuing open science that seeks to disseminate knowledge. In its current state however, academia is under-resourced and incapable of effectively training the next generation of researchers nor pushing the frontiers of capabilities or safety research. This has led to a shift, where many students and professors decide well-resourced industry labs are the “only place” to do the most interesting research and shape a better world. People like us who stayed behind are left to wonder if we’re trying to do this through a broken system that people don’t realize needs to be fixed?
• Without intervention, we will lack the institutions needed for frontier knowledge to spread beyond private labs, for open science to accelerate innovation, and for neutral technical experts to advise policymakers. There is a need for reimagined nonprofit, third party institutions today that work at the frontier of AI and represent the public interest. These institutions should become channels through which frontier knowledge can be responsibly shared. They should train researchers and policymakers who can advance AI capabilities, study safety, disseminate knowledge responsibly, and design informed regulation.
• These institutions should go beyond the organizations that exist today which tend to be more evaluation focused and reactionary to frontier labs, rather than having the resources and expertise to engage in the frontiers of research and increase innovation by disseminating this knowledge safely.
• The rapid development of these institutions is in the interest of essentially all parties.
  • Those pushing for regulation and safety should want more informed, neutral opinions shaping policy, rather than a world where policymakers are forced to rely almost entirely on the companies they are trying to regulate.
  • Those who want to maintain the capitalist growth of frontier labs and pace of rapid advances in AI should want credible public-interest institutions that make continued progress politically and socially sustainable, and ensure that research advances aren’t bottlenecked by only a few private companies.
  • Those pushing for open science and competition should want institutions that can responsibly expand access to knowledge, evaluations, and capabilities without pretending that unrestricted release of every model is always socially optimal.
  • And those building platforms on top of AI should want an ecosystem that is not structurally dependent on the decisions, outages, policy choices, or access restrictions of a few frontier labs.
• Reading this may give some people déjà vu, because arguably this is what frontier labs had initially set out to do. The problem was that because of the scale of the financial requirements, the financial structure of the organizations had to change to include for profit entities that could support the research they needed to do, creating the conflicts of interest we see today. The key differentiator here, is that if these institutions are backstopped by the government, industry partners, and philanthropists, in perpetuity, we could avoid a similar failure mode to what we see today. While it may have been unclear to do this in the past because the economic impacts of AI were also unclear and it was more of a research bet, the situation has changed enough today for these new models of resourcing to make sense.

Claim 3: The missing institutions should sit between secrecy and unrestricted release.

• Open science doesn’t have to mean always releasing model weights, or even the key secrets that differentiate one lab from the next.
  • Take patents as an example from the past: the patent system was designed around a social contract in which inventors received temporary commercial protection in exchange for disseminating technical knowledge to the public. My point isn’t that AI labs should “patent their models”, but that society has built mechanisms before that lie between total secrecy and total disclosure, and we should be similarly creative here.
  • Medicine offers another imperfect but useful analogy: private companies develop vaccines and drugs, but validation and approval don’t depend solely on the company. Regulators, advisory bodies, outside researchers, and public-health institutions all play roles.
• Access to the resources for capabilities research for neutral researchers leads to better public understanding of the technology, subsequent regulation, and increased innovation. Better collaborations for evaluation or alignment science might involve secure ways to share white-box access to models with trusted researchers. Even if forms of external access exist right now, they aren’t fully impartial if frontier labs decide which researchers get access, what they can study, and when access can be revoked; there should be real governance standards. Intentionally designed institutions like these that represent the public interest would be a path to deconsolidation of power and accelerated innovation, all while taking safety seriously.
• Financial, compute, and knowledge sharing efforts with frontier labs, industry partners, and philanthropists are all logical ways to support this effort. Underpinning these institutions would be the need to have the resources to recruit competitive talent. These conversations to get alignment from all stakeholders can’t happen quick enough.

Whether you agree or disagree, if you have thoughts on the issues raised here, reach out. I feel a sense of urgency to fix these systemic problems so we can work towards a better end state for society, and we hope this blog highlights the narrow window in which we live. There’s also a geopolitical aspect to these issues that needs another post to address. The greatest risk I see right now is accepting that the current structure of the debate is inevitable.

Thanks to Andy Konwinski, K. Tighe, Matei Zaharia, Andrew Qin, Chaitanya Asawa and others for their helpful discussions and feedback on drafts of this post.

Citation

Parth Asawa and Joseph E. Gonzalez. Unsafe AI or Consolidated Power: AI’s False Dichotomy. June 2026. https://pgasawa.bearblog.dev/unsafe-ai-or-consolidated-power-ais-false-dichotomy/

Or

@article{2026aisfalsedichotomy,
  title   = "Unsafe AI or Consolidated Power: AI’s False Dichotomy",
  author  = "Parth Asawa and Joseph E. Gonzalez",
  year    = "2026",
  month   = "June",
  url     = "https://pgasawa.bearblog.dev/unsafe-ai-or-consolidated-power-ais-false-dichotomy/"
}

References

[1] Anthropic. “Project Glasswing: An initial update”. https://www.anthropic.com/research/glasswing-initial-update (2026).

[2] Anthropic. “Statement on the US government directive to suspend access to Fable 5 and Mythos 5”. https://www.anthropic.com/news/fable-mythos-access (2026).

[3] OpenAI. “Building an early warning system for LLM-aided biological threat creation” https://openai.com/index/building-an-early-warning-system-for-llm-aided-biological-threat-creation/ (2024).

[4] Anthropic. “Biorisk” https://red.anthropic.com/2025/biorisk/ (2025).

[5] Sam Schechner. ”Anthropic’s New Fable AI Model Is Met With User Backlash Over Restrictions”. https://www.wsj.com/tech/ai/anthropic-fable-restrictions-ai-developers-cd9bf57c (2026).