Dear friends,
A key AI engineering skill is using coding agents. Your skill at steering them both to write code and to carry out non-code tasks, such as analyzing data or managing system operations, allows you to get a lot more done.
The rapid pace of evolution for coding agents means this skill, too, is evolving rapidly — faster than other top-level AI engineering skills. Proprietary agents (like Claude Code, Codex, and Cursor) and open agents (like OpenCode and Pi) progress in strides via both harness and model improvements. So keeping up with how to use coding agents requires a continuous process of experimentation, building, and learning.
In interviewing dozens of top AI Engineers and reflecting on our own team’s use of coding agents, we found a consistent high-level workflow for building software with them. The key steps are:
This high-level workflow is similar to the one typically used to build software before coding agents. Now, we focus much less on code and instead focus on deciding what to build, designing the architecture, writing the spec, and verifying outputs.
The duration of each step can vary significantly between projects, and steps can be omitted. For example, the spec for a greenfield (meaning built-from-scratch) prototype might be loosely described in a quickly written prompt, whereas the spec for a brownfield (pre-existing) project with many users might require much more effort to write and verify. Further, the workflow is highly iterative, and skilled developers know when feedback from a later step should lead them back to an earlier one. For example, if verification fails, they know how to steer the agent to rebuild and fix errors; or if monitoring surfaces issues, how to have agents update the system and redeploy.
To use coding agents effectively in this workflow, the key skills are:
Directing the workflow. You know how to navigate each step of the workflow above. This involves deciding how much human and how much agent effort to spend on each and when to go back to an earlier step to iterate. It requires deeply understanding the tradeoffs of speed, cost, technical risk, and human effort, so you can decide how much to research and plan up front, when to retain human ownership over critical work, how to choose the architecture, how much detail to write into a set of planning artifacts (like a spec), and how to decompose the work into verifiable steps.
Enabling agent autonomy. When applying a coding agent to the steps in the workflow, you choose the autonomy level: Do you watch it and go back-and-forth interactively or delegate a larger chunk of work to it? And when do you set a clear goal and have it loop until it succeeds? Additionally, you have to manage the context carefully for the agent. As the build proceeds through different phases, you will calibrate when to make sure key learnings, user feedback, and assumptions — including assumptions that changed partway through the build — are captured for the agent to use downstream. Additionally, you will decide when to set up many agents to run in parallel on a decomposition of the task — either by having a human or a higher-level agent orchestrate these other agents — and how to manage human attention across concurrent agent sessions. You also know how to run agents safely, setting permissions and gating actions appropriately to let development proceed quickly while limiting the risk of leaks, data loss, or other damage.
Reviewing the work. The output of a coding agent is uncertain. We don’t know in advance what good ideas it might come up with and what bugs it will implement. Reviewing and verifying the output is a key step to ensure you are getting the result you want and to redirect the agent if not. You will design testing and validation that is matched to the task, applying both behavioral and functional verification as needed. You might also test user flows, perhaps having an agent provide screenshots as evidence of success or failure. For qualitative/behavioral evaluation, eval sets, perhaps with LLM-as-a-judge, can be used.
You also need to decide how much of these tests should be automated. Some workflows will have all testing and validation fully automated so the agent can check its work and know when it has succeeded in completing a task. You have to evaluate the tests to ensure they correspond to your aims, and you will evolve them if not. Additionally, you use agentic code review and run AI-enabled security and architecture audits. When AI review isn’t sufficient, you judiciously insert human reviews of the code behavior (and, infrequently, of code as well) while exploring how to automate this review further. Finally, you verify deployment and can operationalize monitoring and incident management with agents.
Customizing the agent and its environment. Your ability to update both the agent and the environment it works in allows your agents to efficiently get the context they need, access tools, and build correctly and efficiently. You know how to integrate agent skills, plugins, and MCP servers. Occasionally you will prune them when they are no longer necessary (such as when a new model obviates an old skill). You can use hooks to automate repeatable parts of the development process, like triggering automated code reviews or CI/CD pipelines. You can also maintain the environment the agent works in: updating the standing context (such as AGENTS.md or CLAUDE.md) with information on the codebase, key architectural assumptions, code style, and data access patterns. You know how to preserve state across multiple sessions and across parallel agents, and accumulate agent learnings over time, perhaps by running post-run retrospectives to capture what did and did not work. You also know how to set up consistent conventions and structure to make your codebase navigable to the agent, and how to occasionally clear out agent-generated debt. When you work in a team, you consider how to coordinate context across different developers’ agents.
Coding agent foundations. Finally, to make good decisions throughout, you have a good understanding of how coding agents work: how they carry out codebase search/retrieval, how they manage their context windows, how different operations (like adding tool calls, MCP servers, etc.) affect context, how agents and subagents interact, and how the agent is built by wrapping a harness around an LLM. This makes the agent less of a black box and helps you to recognize failure modes, such as overengineering a simple solution, losing rigor because the agent lacks an explicit verification process, stopping short of the goal, or agent actions that risk destruction of files or production data. It also helps you reason about the agent’s state and steer it by giving it the right prescription or context. And when monitoring a run, this understanding allows you to better spot when the agent goes off-track and requires your intervention.
I find that social media often gives oversimplified descriptions of how to use coding agents. For example, it is sometimes useful to get agents to run autonomously for hours and burn millions or tens of millions of tokens. But currently the practical utility of very long-horizon tasks — especially relative to their cost — has been amplified beyond reality. Instead, most effective coding agent use is a complex, highly iterative process, and being able to intervene with high-skill judgement gives much better results.
Your skill at using coding agents will make you an effective builder. This positions you to also steer the overall build. I will say more about this in next week’s letter.
Keep building! Andrew
A MESSAGE FROM DEEPLEARNING.AI
Most advice about coding agents stops at "give it better context." In Spec-Driven Development, built with JetBrains, Paul Everitt teaches the method: project constitutions, a feature spec for each change, and a plan-implement-verify loop. Join in for free
News
Comparing OpenAI and Anthropic’s Data Retention Policies
Anyone who sends private data to an AI model or service deserves a clear answer to three questions: (i) Where do they store it? (ii) How long do they keep it? and (iii) What, if anything, will they do with it (such as train on it, show to humans, etc.) This week, business customers for two leading companies got two responses: one leaked, then confirmed, and another previewed. Technical details are scarce for both.
What’s new at Anthropic: Since June, businesses using Claude Fable 5 have been required to let Anthropic keep their conversations for 30 days. Now Anthropic will soften that rule, with a new program called Enterprise Frontier Safeguards (EFS) that will require companies with zero data retention (ZDR) policies to keep the data on their own servers or those of specific cloud providers instead. Until EFS is available sometime this fall, eligible enterprise customers will be able to use Fable 5 and Fable 5.1 without Anthropic retaining any data.
Meanwhile at OpenAI: One day before Bloomberg broke the Anthropic news, OpenAI published a post promising that for business customers, its most capable models come with ZDR. This means OpenAI never logs a business’s prompts and replies. It’s offered to approved businesses that plug the models into their own software. The post also previewed Private Safety Processing (PSP), a system meant to spot misuse spread across many requests without OpenAI reading the prompts. How it works: Both companies describe the same method: software watches requests over time and flags patterns. No human review is required. Here is what neither explains: “Our employees can’t see it” is not the same as “our systems can’t see it.” To scan the data, software the company wrote has to unlock and read it, wherever it is stored. Sholto Douglas, a member of Anthropic’s technical staff, described the plan on X as monitoring “done via automated systems we provide to you.” Here’s where each company stands.
Behind the news: Anthropic’s June rule forced an exception to the ZDR contracts some business customers already had. These customers had to switch retention on to use the new models at all. Consequently, many businesses declined to use Fable 5. As we reported in June, the ARC Prize Foundation also declined to run its verified tests of Fable 5 rather than expose its private test questions. Anthropic has since acknowledged the cost; in an August risk report it wrote that the rule would “be unpopular with customers who have come to expect zero retention” and could hurt its business if rivals didn’t do the same. OpenAI has not; its post reaffirmed zero retention.
Why it matters: Businesses are deciding now whether to trust these models with sensitive data based only on press releases. For regulated industries like law firms, hospitals, and banks, “we won’t train on your data” and “we don’t have your data” are different promises. Businesses also lack clarity on when Anthropic may change its definition of when someone is acting "unsafely" and may have their prompts — which often contain extremely sensitive data — read by Anthropic employees. And any data that can be found on a server can be compromised by an attacker or demanded by a court. Last year, in The New York Times’ copyright suit, a court ordered OpenAI to preserve chat logs it would normally have deleted, even ones users had erased. OpenAI said ZDR customers were unaffected because it never had their data. Whether logs held on a customer’s servers could be reached in a suit against the AI company is a question neither company has addressed. Neither has published an independent audit of its design.
We’re thinking: In 2024, we described four levels of cloud AI privacy, and the strongest, where the provider cannot access your data at all, is the level that matters most for sensitive work. This is especially true given loopholes that give frontier labs discretion to apply whatever standard they wish to define what is “safe.” We contrast this with the simple privacy policies of many hyperscalers, where rules are much clearer: We expect them not to look at our data unless a warrant, court order, or other more predictable legal process requires them to. Both companies now say they can preserve user privacy while still catching misuse across many requests. Neither has shown how. Until both of them publish their designs, that’s a roadmap, not a guarantee.
Ox Alpha Revealed as GLM-5.3-Flash
For over a week, the name and maker of the most-used model on OpenRouter remained unknown. Last week, it was publicly announced to be a new GLM series model — and in a surprise to many, the company says it served the model’s free, high-volume preview exclusively with Chinese-made chips. Now anyone can download its weights.
What’s new: Z.ai released GLM-5.3-Flash, a vision-language model it had previewed under the name “Ox Alpha.” It’s the company’s first vision model since April’s GLM-5V-Turbo, and the first model in the GLM-5 family whose vision capability was built from the start rather than added to a language model afterward.
How it works: Z.ai trained GLM-5.3-Flash on text, images, and video from the start rather than melding vision and text transformers afterward. Unlike the larger GLM-5.3, Z.ai pretrained this model from scratch and redesigned the attention layers to handle long inputs more efficiently.
Performance: Independent evaluations place GLM-5.3-Flash just below the top open weights models, while costing roughly an eighth per task of the open weights models just above it. Leading proprietary models cost between 10 to 35 times per task. GLM-5.3-Flash leads all other open weights models tested on one evaluation of real-world work and completes long-running coding tasks nearly as well as the larger GLM-5.3, which costs 16 times more per task.
Behind the news: For a week before the launch, Z.ai introduced its preview of GLM-5.3-Flash anonymously, available free and exclusively on the coding harness OpenCode and on the model marketplace OpenRouter for a week before launch. This way, it was able to collect feedback from developers unaware whose model they were testing. The company says Ox Alpha became the most popular model on those services that week. Users speculated that the mystery model belonged to the GLM family within days based on its tokenizer outputs. On August 26, Z.ai confirmed the model and released its weights under a standard MIT license. Two days later, the company released weights for its flagship GLM-5.3 under a license similar to MIT but added a clause requiring any business whose revenue surpasses $10 billion to pass a security review by Z.ai before using the weights commercially.
Why it matters: While GLM-5.3 still outpaces GLM-5.3 Flash (and virtually all open models) on text benchmarks, Z.ai’s cheap model is also the more advanced and versatile one, for now. GLM-5.3 is merely a highly capable fine-tune, while Flash received a new base, architecture, and vision capability. The company says its next flagship model will inherit this multimodal, hybrid attention architecture, while also training on more data and showing greater capabilities. Months ago, GLM-5V-Turbo outpaced Claude Opus 4.6 on vision-language tasks; the next GLM series model may similarly challenge top proprietary multimodal models.
We’re thinking: Ox Alpha’s anonymous preview created buzz and mystery but also allowed users to judge it on its merits (and deficits). Perhaps the biggest mystery revealed was its reliance on chips from China-based manufacturers. This shows that with the right memory optimization methods, companies can serve a cost-effective, high-performing model at scale on economically-priced hardware — albeit a somewhat smaller and slower model than we've come to expect from the cutting edge.
Learn More About AI With Data Points!
AI is moving faster than ever. Data Points helps you make sense of it just as fast. Data Points arrives in your inbox twice a week with six brief news stories. This week, we covered OpenAI cutting off Cursor’s access to its models and Anthropic making Claude Fable and Mythos less verbose. Subscribe today!
Custom Models for Law, News, and Finance
Thomson Reuters launched a proprietary large language model family called Thomson. It’s built on a Qwen model using a custom data engineering and re-training pipeline to provide more accurate and complete responses in fields such as law and finance than general-purpose LLMs.
What’s New: Thomson Reuters built Thomson as a domain-specific model that specializes in law, business, tax, finance, news, and related knowledge work fields. Thomson will be deployed first inside one of Thomson Reuters’ existing products, CoCounsel Legal, a tool for research, analysis, and drafting documents. It will be extended to other Thomson Reuters products at unspecified dates.
How it works: Thomson Reuters built Thomson-1.0-Large on top of Qwen3.5-397B-A17B, an instruction-tuned open weight model. It employed what the research authors call Continual Learning, a mix of full-weight mid-training and fine-tuning, on a selection of the company’s large data corpus. In a press release, the company disclosed that it invested $40 million in total training costs over three months.
Behind the news: Thomson Reuters has long used AI in its products for legal and financial professionals and government agencies, but the company’s latest AI-powered product is its first LLM. The company is under pressure from both new AI-native companies like Harvey and Legora and general-purpose LLMs and agents from OpenAI, Anthropic, Google, and others that offer alternatives to many of its products.
Why it matters: The Thomson paper authors argue that their approach to domain-specific LLMs represents a special case of sovereign AI, but at the corporate rather than the national level. Instead of yielding data to a larger AI company and being subject to its infrastructure and data retention policies, Thomson Reuters licenses its models to companies that run them on their own hardware, ensuring that sensitive data never leaves the premises. Also, its models can be further trained on a company’s private data using Thomson’s midtraining methods, without the high costs of pretraining or the relatively limited effectiveness of an inexpensive fine-tuning method like LoRA.
We’re thinking: Model customization remains an exciting prospect for a wide range of AI applications, and the growing availability of open weight models creates an important foundation for teams to experiment with them. Still, companies like Thomson Reuters will have to prove to customers that their products are worth the price given the relatively small improvement in performance.
A Simpler Method to Monitor Models
LLM safety monitors that act during generation often analyze a series of safety scores to catch incorrect or harmful outputs. Researchers found that analyzing a single safety score achieved nearly the same performance.
What’s new: Mona Schirmer, Metod Jazbec and colleagues at University of Amsterdam, University of Wisconsin-Madison and Johns Hopkins University introduced a simple monitoring system called CRC Monitor — CRC stands for conformal risk control. It evaluates an output’s safety by comparing a single safety score against a threshold that is carefully calibrated for the task at hand.
Key insight: Systems such as e-valuator score LLM output after every reasoning step, conversational turn, or tool call and analyze the history of scores to decide whether to stop. A simpler approach is to stop as soon as the latest step’s score falls below a threshold. The challenge is choosing this threshold reliably. A false-alarm rate measured on a limited validation set may be lower than on new data, simply by chance. Therefore, one needs to choose the threshold more conservatively.
How it works: The authors calibrated CRC Monitor’s threshold for two tasks in which a model should be stopped when its generation goes wrong: incorrect mathematical reasoning and harmful subject matter. For each dataset, they used a separate model, called a verifier, to score the output text at every step. They found thresholds that satisfied two criteria: (i) catch as many unsafe outputs as possible and (ii) keep the number of false alarms below a user-defined limit.
Results: CRC Monitor flagged unsafe responses faster and maintained the user-requested false-alarm rate while matching the performance of e-valuator in most experiments.
Why it matters: Stopping LLMs from generating undesirable output can be done more simply and less expensively. An effective monitor that catches such output as it’s being generated requires only a safety score for the latest generation, a calibrated threshold, and a predetermined tolerance for errors. This approach detects problems earlier than a competing method, reducing the user’s exposure to mistaken or objectionable output (and some inference cost). The authors suggest that their approach lends itself to a tiered monitoring architecture that generates a cheap signal continuously, stops according to a calibrated rule, and shifts to more-expensive verifiers only when the cheap signal indicates a poor output.
We’re thinking: Since a simple safety-detection method can work as well as a complicated one, improving the verifier behind it may lead to greater improvement in performance than designing more complex ways to weigh the results.
Work With Andrew Ng
Join the teams that are bringing AI to the world! Check out job openings at DeepLearning.AI, AI Fund, Landing AI, and LearnVector.
Subscribe and view previous issues here.
Thoughts, suggestions, feedback? Please send to thebatch@deeplearning.ai. Avoid our newsletter ending up in your spam folder by adding our email address to your contacts list.
|
