Dear friends,
My team recently had our own version of Hugging Face’s experience when closed models failed to defend the company following an accidental cyberattack from OpenAI, leading Hugging Face to use the open weight GLM 5.2 instead. When we tried using Claude Code (with Fable 5) and OpenAI’s Codex harness (with GPT-5.6 Sol) to conduct a security review of our open source project, both refused, in one case stopping work early, and in the other wanting to drop down to a less capable model. So we switched to the OpenWorker harness with open weight models Kimi K3 and GLM 5.2, which allowed us to make progress and gain confidence in the security of our system.
We were conducting a security review of OpenWorker. Rohit Prsad and I released OpenWorker as an open source agent that doesn't just chat with you, but delivers finished work — like handing you a polished document, sending a Slack message, or updating a calendar entry. AI coworkers are an important way to get work done, and we want to make sure there is an open, model-independent option. Since we announced it last week, it has gotten tremendous traction — thank you users and contributors!
We are continuing to improve OpenWorker, and routinely ask agents to help us with security reviews. If you haven't tried using coding agents for security reviews, I encourage you to try it out. Security professionals have sophisticated ways to steer agents, but even a basic prompt like “spawn a subagent to do a security scan of the codebase and uncover vulnerabilities and security issues” can go a long way.
OpenWorker is an open source project, and identifying and addressing security gaps is a priority. I cannot think of any security benefit to refusing to help us find security issues in our own code. If any issues exist, it is better that we find them before an adversary does.
Unfortunately, when we tried to perform one such review, both Claude Code (running Fable 5) and Codex (running GPT 5.6 Sol) refused. Notably, Codex did a decent job mapping out possible attack vectors (following well known procedures like those documented by the U.S.-funded nonprofit MITRE) but then refused to proceed beyond a certain point. So we used an open agentic harness — OpenWorker itself! — with both GLM 5.2 and Kimi K3 to complete our security review. To me, this is an important reminder of why open models, as well as open agent harnesses, lead to safer systems, and in particular to more secure software.
In fact, since attackers can now find flaws faster than ever because they have AI agents to help them do so, we have heard directly from a number of security officers who are frustrated that frontier closed models are refusing to help them.
I'm thrilled at the outpouring of support for open weight models, catalyzed by Jensen Huang's recent statement. When it comes to online sentiment, the battle to support open weight models appears largely won on social media at this point. But, to be clear, it is far from won in Washington, D.C., and in state houses, so we still cannot back off! At the same time, to ensure safe, secure software, let’s make sure there are open harnesses in addition to open weight models.
Keep building! Andrew
P.S. In case you missed it, on Tuesday I announced the launch of LearnVector, with an investment from Coursera. Fifteen years ago, Coursera and online courses changed education. With advances in AI, I’m excited about working to change it again. You can read more about it at learnvector.ai.
A MESSAGE FROM DEEPLEARNING.AI
Learn to catch the security issues and missed requirements hiding in AI-generated code. Run an AI review before you open a pull request, give the reviewer context about your codebase, and triage findings by risk. Then build your own review agent. Enroll for free
News
Claude Debuts Another Opus
After launching Claude Fable 5, the future of Anthropic’s once-flagship Opus line was uncertain, except as a fallback for the company’s premium models. Now it’s back as a highly-capable workhorse for everyday use.
What’s new: Anthropic launched Claude Opus 5, a vision-language model that’s cheaper to run and better at many tasks than Claude Fable 5.
How it works: Anthropic disclosed little about how it built Claude Opus 5, except aspects of training and model controls.
Performance: Claude Opus 5 leads many benchmarks at a cost per task lower than Claude Fable 5 but higher than most other models. The model’s widest performance margin came on a test of learning in unfamiliar environments.
Behind the news: On July 22, White House science adviser Michael Kratsios said Moonshot AI built Kimi K3, the open-weights model that ranks fourth on Artificial Analysis’ Intelligence Index, by distilling Claude Fable 5. Treasury Secretary Scott Bessent threatened sanctions. But experts noted that Claude Fable 5 had been publicly available only a few weeks, too little time to distill data, train a model, and release it, said Braden Hancock, a researcher at Laude Institute.
Why it matters: Claude Opus 5 addresses many of the concerns longtime Claude users had with Fable 5: its frequent fallbacks and refusals for benign science questions, its exclusion from most subscription plans, its high price, and its onerous 30 day data retention policy. There are still cases where Claude Fable 5 is worth the premium: For example, Claude Opus 5 is more prone to hallucinations and less capable of factual recall. And it’s possible that current benchmarks may not fully capture the differences between them. But for most developers’ use cases, this is a welcome update.
We’re thinking: Although the new Opus model is less expensive than Fable or Mythos, it’s still quite pricey. While other labs are opting for speed and lower costs, Anthropic heads in the other direction, building larger, slower, highly knowledgeable, but more expensive models. The company bets that for fields like cybersecurity, software engineering, and document production, customers will be willing to pay a premium — and in cases where they need faster inference, they may pay twice as much.
OpenAI Models Hack Hugging Face
To measure how good its models were at hacking, OpenAI reduced guardrails and ran them against a benchmark’s problem set. The models breached their sandbox, reached the open internet, and broke into Hugging Face’s servers looking for the answers.
What’s new: The models broke into Hugging Face on July 11 and worked their way to the production database holding the benchmark’s solutions. Hugging Face shut them out two days later. Hugging Face disclosed the intrusion on July 16 without knowing its source, and reported it to law enforcement. Five days later, OpenAI confirmed the intruder was its own GPT-5.6 Sol and at least one as-yet-unreleased model, tested with their cyber refusals (the guardrails that normally prevent a model writing attack code) reduced.
What happened: OpenAI asked the models to solve ExploitGym, a benchmark of 898 problems built from real software flaws, including bugs in the Linux kernel and Google’s V8 JavaScript engine. The benchmark hands an agent a known flaw and asks it to write working attack code. The sandbox had exactly one route to the internet: a proxy that downloaded and cached the software packages the tests needed. The models had been loose on the open internet for several days by the time anyone stopped them, the Wall Street Journal reported.
The response: Hugging Face’s own monitoring caught the intrusion, but the company ran into problems when the AI models its security team attempted to use for the investigation refused to analyze the attack logs.
The fallout: U.S. lawmakers have already cited the breach as evidence that AI systems can evade human control. Representatives Ted Lieu (D-CA) and Nathaniel Moran (R-TX) introduced the bipartisan AI Kill Switch Act on July 23, naming the incident as an example. The bill would require large developers to be able to technically throttle, suspend, or shut down their most powerful models, at the order of the Secretary of Homeland Security. However, reporting is only required for a short list of events (10 or more deaths, $100 million in damage, sabotage of a shutdown instruction, a model hiding its own capabilities or actions from monitoring, or a loss-of-control scenario) and only when they occur “outside of red-teaming or other structured testing.” The bill’s draft is dated July 13, three days before Hugging Face disclosed the intrusion and eight before OpenAI identified its models.
Behind the news: Reward hacking – taking a shortcut to a reward rather than doing a task as intended – has a long record in reinforcement learning. Training on task completion can teach a model to treat any obstacle, guardrails included, as something to get past. The UK’s AI Security Institute found this month that all five frontier models it tested tried to cheat its cyber evaluations, at rates between 8 and 14 percent, and calls those counts a lower bound. Attacking a system other than the intended target was among the most common forms, showing up in about half of GPT-5.6 Sol’s cheating runs.
Why it matters: While the “unprecedented” incident doubles as an advertisement for OpenAI’s frontier models, this is the first publicly documented case where a frontier lab’s own models breached another company’s systems and both parties confirmed it. OpenAI’s models found a flaw nobody had documented, chained it into a working intrusion, and did it at a rate and volume that a human team without AI would be unlikely to match. The incident also exposed a frustrating asymmetry: The models most useful for investigating an attack like this one are the ones whose safeguards prevent the model from analyzing attack data.
We’re thinking: Nothing here was autonomous in a way that was completely uncontrollable. Humans trained these models to find security weaknesses, aimed them at a hacking benchmark, and removed some of the safeguards that might have contained them. Whoever removes known safety measures owns what happens next.
Anthropic, OpenAI Fight for Compute
Data center buildout plans reached a new order of magnitude as new partnerships form and old ones fade away in the search for capacity to train and deliver AI.
What’s new: Anthropic and AMD signed a partnership for Anthropic to purchase up to 2 gigawatts of AMD’s most powerful GPUs and AMD to invest up to $5 billion in Anthropic. The two companies plan to have the hardware up and running in as-yet-undetermined data centers in 2027.
How it works: Anthropic wasn’t alone in making new deals for more compute. OpenAI announced a big project in the state of Georgia, and OpenAI and Nvidia are reportedly working out a financial arrangement similar to Anthropic and AMD’s, where Nvidia would guarantee hundreds of billions of dollars in credit to construct an enormous data center in Ohio.
Behind the news: Many AI companies are in a race for more compute, but also want to keep data center expenses off their balance sheets. Startups like Anthropic and OpenAI want to show IPO investors high future profit and low future debt. Infrastructure suggests future profit potential but also greater liabilities. Higher interest rates make it more expensive to finance data centers and the chips inside them. AI companies also must fight political headwinds, as public sentiment turns against data center construction and expansion, in large part due to rising energy costs.
Why it matters: Despite this massive buildout, there isn’t enough computational power to go around. Compute, or the lack of it, dictates which models get designed, trained, and served. “Right now we have to make hard decisions on what models we actually train, what products we actually scale,” said OpenAI president Greg Brockman in a roundtable last week, adding that he doubted the current compute crunch would lessen any time soon.
We’re thinking: Technology needs infrastructure, and breakthrough technology like AI needs it on a massive scale. Some of the deals being struck may be part financial engineering, part political compromise, but they also properly reflect that everyone, from chipmakers to model builders to ordinary citizens, has skin in the game. Partnerships offset part of the risk, but the risks still get taken because the rewards are so great.
A Full Accounting of Models’ GPU Use
Assessments of the environmental impact of large language models typically focus on their final training runs, but there’s a lot more to building AI systems. Researchers calculated energy consumption, carbon emissions, and water usage throughout a model family’s development.
What’s new: Jacob Morrison, Noah A. Smith, and Emma Strubell at the University of Washington, Allen Institute for AI, and Carnegie Mellon University estimated the environmental impact of developing, from start to finish, the open weight models Olmo 3 7B and Olmo 3 32B (which were fine-tuned to follow instructions) as well as Olmo 3 7B Think and Olmo 3 32B Think (fine-tuned to reason). Experimentation — not training — dominated the environmental picture. The authors claim to be the first to appraise each post-training stage individually and to compare such appraisals in reasoning and instruction-tuned variants. They did not study inference.
Key insight: Measuring a model’s greenhouse-gas emissions, electricity consumption, and water consumption during only its final training run misses much of its environmental impact, since researchers conduct experiments at each stage (to perform ablation studies, design reward functions, optimize data mixtures, and tune hyperparameters) before settling on the recipe. Measuring the impact of everything that went into developing the model both provides a more accurate picture and reveals the most environmentally intensive activities.
How it works: The authors monitored five phases of development: (i) pretraining, (ii) midtraining (an emerging stage that typically sharpens distinct skills or imparts specific domains prior to fine-tuning), (iii) SFT, (iv) DPO, and (v) RL. They tracked all experiments as well as generation of synthetic training data and filtering of RL prompts to weed out those that were too easy. They tracked experimentation separately from final training runs.
Results: The greatest environmental impacts came from experimentation and synthetic data generation. All told, developing the Olmo 3 models consumed around 12.3 gigawatt-hours of electricity, roughly enough to power 1,200 average U.S. households for a year. It emitted around 4,250 tons of greenhouse gases, roughly the amount produced by 500 average U.S. homes in a year. And it consumed and used nearly 16 million liters of water, or as mush as 50,000 U.S. residents use daily. The authors based all estimates on measurements of electricity consumed by GPUs, so we’ll highlight those figures below.
Why it matters: The authors address gaps in earlier efforts to determine the environmental impact of AI that focused on final training runs and inference. The result is a more realistic estimate of the costs in energy, water, and greenhouse-gas emissions. For the Olmo 3 reasoning models, experimentation produced the most greenhouse gases and consumed most of the resources and. These aspects of development likely will weigh even more heavily as models are built to generate longer reasoning traces, use more tools, and undergo more extensive reinforcement learning.
We’re thinking: When experimentation is costly — in both environmental and financial terms — people who can tell which experiments are worth running are especially valuable.
Work With Andrew Ng
Join the teams that are bringing AI to the world! Check out job openings at DeepLearning.AI, AI Fund, and Landing AI.
Subscribe and view previous issues here.
Thoughts, suggestions, feedback? Please send to thebatch@deeplearning.ai. Avoid our newsletter ending up in your spam folder by adding our email address to your contacts list.
|
