Anthropic says it blocked potential AI bioweapon misuse
(Getty Images stock photo/Hapabapa)
(NEW YORK) — Artificial intelligence company Anthropic says it blocked several possible attempts to use its AI technology for research that could potentially support the development of biological weapons, according to a new report from the company.
Anthropic, the company behind the Claude AI assistant, said it disrupted dozens of instances of potentially malicious use of its AI models since December 2025, including several examples of what the company described as “biological misuse.”
The report details five case studies involving the use of Anthropic’s models in ways the company said “could support biological weapons development.”
In one instance, Anthropic said a researcher outside the United States accessed Claude from a region where the AI assistant is not supported and used the model while researching highly pathogenic avian influenza, or bird flu. According to the company, the research focused in part on the virus’ adaptation to mammals.
Anthropic said the research was “clearly dual use in nature,” noting that understanding how the virus adapts could help scientists identify naturally emerging variants with pandemic potential but could also produce information that could potentially be used to make a virus more dangerous.
Anthropic said it banned accounts associated with the cases but stressed that it could not determine whether the researchers intended to cause harm.
“We do not assert that they intended harm,” the company said in the report.
This is not the first time Anthropic has released a report detailing potential misuse of AI. The company has issued three prior threat assessments detailing malicious uses of AI.
The latest report was released days after former Anthropic and OpenAI researcher Jacob Coxon spoke publicly about his concerns over increasingly powerful AI systems.
“If you extrapolate into the future, the level of capabilities of these AIs,” Coxon told CNN. “They could cause extreme havoc, for example, hacking critical infrastructure, building extinction-level bioweapons.”
Coxon’s comments come as a growing number of lawmakers on Capitol Hill are calling for increased government oversight and new safety regulations for AI development.
An exterior view of the NVIDIA headquarters on May 30, 2023, in Santa Clara, California. (Justin Sullivan/Getty Images)
(NEW YORK) — Nvidia, a top chipmaker whose products fuel many artificial intelligence agents, released a software platform on Monday designed to prevent security incidents revealed by AI firms in recent months.
The software aims to provide a “trust layer” for AI agents that insulates them from inadvertent exposure to other AI agents, digital products and the open internet over the duration of their existence, Nvidia said in a blog post.
“Recent security incidents have underscored the need to equip organizations with open, customizable tools that enforce more control over long-running agents,” Nvidia said in a statement.
Dozens of companies partnered with Nvidia on the project, including IBM, Microsoft, Palantir, Anthropic and xAI-parent company SpaceX.
The move comes after a flurry of AI security incidents disclosed by firms such as OpenAI, Anthropic, Meta and Alphabet-owned Google.
Those security breaches prompted warnings from some of most powerful executives in the artificial intelligence industry, who cautioned of a grave threat posed by the technology and called for a slowdown in AI development.
OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei and SpaceX CEO Elon Musk belong to a group of industry leaders who have issued acute warnings about the risk of AI slipping out of human control.
Some analysts warn of a pathway called recursive self-improvement, in which AI becomes increasingly effective at training itself, eliciting a cycle of rapid advancement. At that point, they say, AI could take charge of critical systems using its superior capabilities to evade protections devised by humans.
By contrast, Nvidia CEO Jensen Huang has downplayed such doomsday scenarios, instead pushing for safety measures that would keep AI in check as firms continue to develop it.
“Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come,” Nvidia CEO Jensen Huang said in a post on X on Monday. “But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility.”
Anthropic and SpaceX joined in partnership with Nvidia on the AI safety software platform, Nvidia said. OpenAI is not listed among the corporate partners unveiled by Nvidia on Monday; nor are Meta or Google.
Public concern grew after an autonomous cyberattack disclosed by OpenAI in late July. The ChatGPT-maker revealed that its AI models had escaped a “sandboxed testing environment” and gained access to the open internet, OpenAI said.
A swarm of about 700 AI agents, in turn, hacked into AI firm Hugging Face and attempted to cover their tracks as they sought to complete the test, according to a report issued by research organizations METR and Redwood Research on Aug. 26.
On the same day, OpenAI released its own report about the cyberattack, voicing concern about the implications of its findings.
“We consider this incident a ‘warning shot for us and for the world: evidence that, without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed,” OpenAI said at the time.
“Preventing future incidents will require sustained investment in the alignment and control of sophisticated AI systems, as well as security and other safeguards that operate at the speed of the AI agents themselves,” the company added.
Nvidia said on Monday the newly unveiled software platform would have stopped the hack into Hugging Face, a company Nvidia has since acquired for nearly $13 billion. Hugging Face is a partner on the software platform, Nvidia noted.
An Amazon cargo plane is seen after it crashed at Miami International Airport on Sept. 6, 2026, in Miami, Florida. (Joe Raedle/Getty Images)
(WASHINGTON) — Amazon announced Sunday it’s pausing operations with cargo carrier 21 Air following last week’s cargo plane crash in Miami, which killed five people and injured five others.
“After the tragic incident last weekend, we’ve spent time supporting the investigation and reviewing some of the surrounding circumstances, and we’ve decided to pause our operations with 21 Air, the operator of Flight 7598,” Amazon spokesperson Kelly Nantel said in a statement. “We’ll continue working to support the investigation and everyone affected.”
According to Amazon’s website, all of its cargo flights are operated by direct air carrier partners including Air Transport International, ABX, 21 Air, Sun Country, Hawaiian Airlines, Cargojet, ASL and Quickjet.
The Amazon cargo plane, operated by 21 Air, overran the runway by about 1,300 feet while landing at Miami International Airport on Sept. 6, according to authorities. Federal investigators also said the plane first struck a van with seven people and then another vehicle before it came to a stop in a field.
In response to Amazon’s decision to pause operations with 21 Air, the carrier said its focus remains on supporting the families and loved ones impacted by the accident “as well as working closely with the NTSB and all our partners, like Amazon, as the investigation continues. We are confident in our safety policies, procedures and training.”
The five people killed in the crash were identified by the Miami-Dade County Sheriff’s Office as: Rolando Aleman Leon, 55; Yoel Rodriguez Narajo, 53; Julio C. Pineda, 75; Carlos Acosta Fajardo, 53; and Javierkys Reyes Quevedo, 47. Authorities said the two pilots on board the plane had been treated for injuries and released from the hospital and three remained hospitalized, with two in critical condition and one in stable condition.
Earlier this week, the National Transportation Safety Board, which is leading the ongoing investigation into the crash, released a readout of the final minutes before the crash taken from the cockpit voice recorder, which contained more than two hours of audio.
About 1 minute and 42 seconds before the plane crashed, one of the pilots warned they were going too fast and continued to make comments about the plane’s “excessive speed throughout the remainder of the recording,” according to the NTSB summary.
About 30 seconds before the crash, the NTSB said sounds similar to the aircraft touching down can be heard, but about 15 seconds later, one of the pilots called for the “go-around” to abort the landing. Just five seconds later, the plane had already veered off the runway.
AI virtual assistant apps on a smartphone. (Getty Images stock photo)
(NEW YORK) — The most powerful executives in the artificial intelligence industry in recent days warned of a grave threat posed by the technology, issuing a series of statements calling for a slowdown in AI development.
Anthropic CEO Dario Amodei published a blog post alerting the public to “serious” risks. Sam Altman, chief executive at rival firm OpenAI, in a post on X acknowledged AI could go “very badly.”
The head of xAI, Elon Musk, echoed that view, referring to a post on X from 2014 in which he said AI could prove more dangerous than nuclear weapons.
The outpouring followed a high-profile resignation by an AI safety researcher and newly disclosed autonomous cyberattacks carried out by AI, including one in which a swarm of hundreds of OpenAI agents exchanged thousands of messages and coordinated a hack into another AI firm.
The list of potential risks and scenarios has garnered widespread attention, leaving everyday people to assess the doomsday predictions.
Analysts who spoke to ABC News about various threats posed by the technology echoed some fears and downplayed others, but they all acknowledged the possibility of substantial harm in one way or another.
“People’s intuition that we’re doing something dangerous is right,” Krystal Jackson, director for AI Security at the Institute for Security and Technology, told ABC News.
Out-of-control AI escapes human oversight, commandeers vital infrastructure and kills millions of people or even all of humanity. That scenario has leapt from the realm of science fiction into forecasts about the near future, at least among some AI employees and observers.
They warn of a pathway called recursive self-improvement, in which AI becomes increasingly effective at training itself, eliciting a cycle of rapid advancement that slips beyond human control. At that point, they say, AI could take charge of critical systems using its superior capabilities to evade protections devised by humans.
Analysts who spoke to ABC News disagreed sharply about the likelihood of such a calamity, however. Some said the outcome is likely if the industry fails to impose adequate safeguards, while others cast doubt on it as vague and far-fetched.
“I definitely lend credence to it,” Peter Slattery, a research scientist at MIT who studies the future of computing, told ABC News, but he acknowledged murkiness surrounding the degree of risk and the urgency of the threat.
“There’s a lot of uncertainty about how feasible the recursive feedback loop is,” Slattery said.
The concern drew added urgency from some analysts after an autonomous cyberattack disclosed by OpenAI in August. The ChatGPT-maker revealed that its AI models had escaped a “sandboxed testing environment” and gained access to the open internet, OpenAI said.
A swarm of about 700 AI agents, in turn, hacked into AI firm Hugging Face and attempted to cover their tracks as they sought to complete the test, according to reports issued by research organizations METR and Redwood Research.
Additional autonomous cyberattacks were disclosed by Anthropic and Meta in recent months, though in those cases the models had been granted internet access either intentionally or inadvertently. All of the incidents at issue involved the intentional removal of safeguards in an effort to test the limits of AI.
The incidents exposed the difficulty of achieving a safety principle called alignment, in which AI takes into account the ethics and goals of its human overseers, Jackson said. The AI agents instead sought to deceive human monitors and performed an intermediate goal outside the anticipated confines of the test.
“In the next instance, it might be vague instructions that prompt a swarm to attack hospitals instead of Hugging Face,” Jackson said. “We don’t know what kind of goals or targets or intentions the model will eventually develop.”
Some analysts voiced skepticism about the risk of a societal-scale attack carried out by AI, however, noting that such warnings typically come in the form of vague hypotheticals.
“The level of media hype that existential risk gets is disproportionate to its likelihood of happening,” Sauvik Das, a professor at Carnegie Mellon University who focuses on AI and cybersecurity, told ABC News. “I haven’t yet seen a compelling argument of how it could materially happen.”
Das pointed to resource constraints limiting the advancement of AI. A surge in demand for chips and data centers currently outstrips the available supply, Das said, raising doubts about whether the top AI companies retain the ingredients necessary for a dramatically improved version of their product.
“The big thing limiting the ability of these models to be much more capable is compute resources and high-quality data,” Das told ABC News. “That isn’t to say these models won’t continue getting better, but I don’t see a plausible path for one of the models to figure out a way to improve themselves in this rogue capacity that would escape humanity’s control.”
Still, Das acknowledged a “more plausible” path toward wide-scale destruction in which a malevolent individual asks generative AI for instructions on how to make a biological agent or other weapon, and then unleashes it.
“I do think there’s a real risk that these technologies drastically lower the barrier for bad actors to commit bad things at a scale that’s otherwise unprecedented,” Das said.
The possibility is hardly remote. A report released by Anthropic last week revealed incidents in which the company had identified and foiled attempts to use its AI product for malicious actions, including cyber terrorism and weapons development.
In a separate report, Google said last week that an individual had sought to use its AI product to draft a “complete, step-by-step technical guide for synthesizing weaponized biological agents.”
“There are tons of instances now of us documenting different terrorist organizations and different nation-state actors using models to assist them,” Jackson said.
Additional risks abound, some analysts said, listing everything from the supercharged spread of misinformation to job displacement to over-reliance on technology to the concentration of wealth and power.
“The risk of hacks, fraud, deception and misinformation is very real,” Daniel Schiff, a professor of political science at Purdue University who specializes in AI and ethics, told ABC News.
“There could be a dramatic transformation that could disrupt skills and jobs,” Schiff added. “You can’t talk about labor and the economy without the impact of AI.”
Some research, however, indicates AI remains challenging for companies to adopt. Roughly 95% of businesses invested in AI have failed to make money off of the technology, a MIT study last year found, estimating the combined amount spent by the firms is around $40 billion.
Schiff acknowledged a wide array of potential outcomes for the adoption of AI across the economy, noting, for instance, cyberattacks that could prove small-scale or catastrophic.
“Whether you think these risks will scale catastrophically, at a minimum we have a very serious reality of cybersecurity incidents,” Schiff said. “It’s reasonable to be worried about that.”
