Dear friends,
How have software engineering fundamentals changed with agentic coding? Even when you use a coding agent to write all your code, understanding software fundamentals is important for steering your agent to make the tradeoffs you want — or to even know what tradeoffs exist to be made. Additionally, when you’re building an AI application, the AI core is often expressed through a broader software application, which a skilled engineer will shape.
A novice who vibe codes without understanding software fundamentals can create simple applications, but this often leads to the coding agent making bad tradeoffs in latency, availability, consistency, reliability, maintainability, simplicity, and/or cost. In most cases, the developer didn’t know such tradeoffs even existed and therefore did not steer the agent to make the right decisions for their application context.
This letter describes what our study of AI Engineering Skills shows are the most important things to know in software engineering. It requires being skilled at:
Building full-stack applications. Agentic coding enables many developers who previously played more specialized roles (like front-end developer or mobile developer) to play a broader, full-stack role. A coding agent can help with parts of the development process that you might be less familiar with. However, understanding how the full stack actually works is important. Skilled developers understand the key components and concepts of front-end and back-end systems, including UI components, caching, page rendering, API choice and design, authentication, state and session management, asynchronous processing, data persistence, testing, security, and accessibility.
Managing data. Data deserves special attention because it is a foundation that software is built on top of, that is relatively hard to change (even if agents help with migrations). When you know how to manage data, you can think through access patterns and use them to decide what to store and for how long. You can identify the right data models and select the appropriate storage types (such as relational tables, documents, key-value, or graphs) and infrastructure, which in turn affects speed, scalability, availability, reliability, and cost. You understand transactions, concurrency, and how to ensure your data is clean, consistent, and fresh. When needed, you can ensure proper privacy, governance, and compliance. You know how to manage the data lifecycle.
As an application evolves, you also know how to evolve the data architecture with it. Deciding how to manage data requires significant human-provided context. Your AI systems will get their own input context from your data source, so if data architecture is chosen poorly, the AI doesn’t know what it doesn’t know. This is why it takes skilled intervention from someone with the relevant context and skilled at AI engineering — you! — to set it right. How to build data infrastructure for agents — rather than only traditional software or humans — is also a rapidly evolving area, and you should continue to adjust your best practices as the field evolves.
Designing system architectures. When you understand the major components of the full stack of software and data, you are then better positioned to decide how to put the pieces together. Good system design requires understanding what the software is intended to do (how many users? how important is latency? how important is cost? etc.) so you can make choices about the application platform, the boundary between the frontend and backend, system decomposition, application state placement, and architectural granularity (monolith vs. microservices). You will also choose the stack (programming languages, runtimes, component/frontend/backend frameworks, data technologies) — sometimes by running experiments to evaluate options before settling on one.
Further, the right architecture is a moving target, depending on the phase of the project. The simple architecture you choose to build a quick prototype may not be the right architecture to build the first production system, and that too may change as the application scales. Making these decisions requires deep technical knowledge of both software components and the application context so you can design — and evolve — the architecture to make better tradeoffs.
Making systems secure and reliable. To build reliable systems, you should know how to develop testing strategies to verify the correctness of your system: What mix of unit tests and integration tests, what frameworks to use, and what level of coverage. You also know how to design around possible failures — how to handle failures (like an API hitting a rate limit), build in graceful degradation, and minimize the blast radius of failures. Additionally, rather than first writing software and then later figuring out how to secure it, the “shift left” movement is moving security work earlier in the lifecycle (to the left on a traditional project timeline). Just as all developers are moving toward becoming full stack developers, many developers are now also partly security engineers. You can now use AI tools to scan your code for vulnerabilities, check dependencies for supply chain injections, and examine your cloud configuration for attack surfaces. But doing this well still requires some knowledge of security.
Scaling and operating in production. To serve real users, you will have to know how to deploy your software to production.You will benefit from knowing how to execute the software development lifecycle (SDLC) which, in addition to building and testing, includes configuring the deployment environment, deciding on release strategy, applying deployment automation (CI/CD), and understanding infrastructure as a service (IaaS).
Operating in production requires putting in place observability tools, setting alerts, and managing incidents. Lastly, to scale your application, you should understand the real load and know how to scale servers, load-balance, and adapt your data infrastructure (via sharding, indexing, replication) or make architecture changes to allow your system to adapt to scale. Finally, understanding coding best practices like version control, code reviews, dependency maintenance, and how to manage technical debt helps you keep evolving your system over time.
Coding agents have changed how we build software, including software that does not contain any AI components. Some parts of coding knowledge — like memorizing coding syntax — are becoming obsolete. But developers who deeply understand how software works vastly outperform those who vibe code without understanding.
Understanding software fundamentals (in addition to AI) also helps you figure out what software can and cannot do. This makes them important context for how you use coding agents and shape the build. I will discuss these in the next two letters.
Keep building! Andrew
A MESSAGE FROM DEEPLEARNING.AI
Build coding agents that learn from experience. In “Building Adaptive AI Agents,” you’ll turn an agent’s own traces into reusable skills and build a code knowledge graph that finds what keyword search misses, so your agent gets better every run instead of repeating yesterday’s mistakes. Enroll for free
News
GLM-5.3 Makes Cybersecurity Gains
Z.ai’s latest flagship model effectively ties open-weights leader Kimi K3 on Artificial Analysis’ index of intelligence benchmarks. The company revealed that the model’s increased skill at finding and exploiting software vulnerabilities warranted safety testing before releasing its weights.
What’s new: Z.ai boosted GLM-5.3’s performance at coding and agentic work solely by fine-tuning its predecessor GLM-5.2, rather than by training a new model from scratch or modifying its architecture.
How it works: The company scaled GLM-5.2’s fine-tuning recipe, applying it to a larger and more varied set of environments (simulated workspaces where the model attempts assigned tasks). The recipe includes single-rollout asynchronous optimization, a reinforcement learning method that trains on attempts one at a time instead of waiting for an entire batch. The training method also splits long records of an agent’s attempts into compacted segments so the model learns from long-running tasks rather than only short ones.
Performance: Independent testing ranks GLM-5.3 on par with the top open-weights model and a few points behind leading proprietary models, with large gains in agentic work. Z.ai’s own tests show the biggest jumps in agentic coding and cybersecurity.
Z.ai’s stealth release: This week, Z.ai confirmed that Ox Alpha, a multimodal model in stealth mode that quickly gained popularity among users of OpenRouter and other platforms, is in fact GLM-5.3 Flash. The company released weights for the 320 billion parameter model under an MIT license.
Behind the news: GLM-5.3 arrived in the middle of debates about whether open weights models with advanced cybersecurity skills are too dangerous to release and lent both sides credibility.
Why it matters: Z.ai set out to build a stronger agentic coder but also got a model highly capable of discovering security exploits. The company deliberately added data and environments that rewarded the model when it found cybersecurity flaws. As intended, that skill climbed as training scaled, outperforming every other model on CyberBench. But the model’s gains at building exploits outstripped its designers’ goals of discovering them. The company didn’t intend for GLM-5.3 to more than double its predecessor’s score on ExploitBench; the model grew more capable simply by pursuing available rewards for exploiting vulnerabilities.
We’re thinking: Each Z.ai release is more capable and generates more buzz than the last. All users benefit from AI labs seeking to outdo each other, especially when they release the weights for everyone to study, modify, and run on their own hardware. Keep the new models coming!
Inside AI’s Need for Speed
Developers who already track models’ cost and accuracy have good new reasons to pay closer attention to a third essential factor: speed.
What’s new: OpenAI and Cerebras previewed Ultrafast, a new API service tier that runs GPT-5.6 Sol on Cerebras hardware instead of OpenAI’s usual infrastructure.
Two other releases landed the same week with speed as their headline pitch. Google shipped Gemini 3.7 Flash, and Nvidia released Nemotron 3.5 Lightning. All three are aimed at developers building applications where response time determines how well a product works.
What is speed, exactly?: Two measurements often show up under the word “speed” in AI, and different models and hardware handle them differently. Latency, sometimes called time-to-first-token, is the wait before a model’s response arrives; throughput is how many tokens per second it produces once it starts. Ultrafast targets throughput: Cerebras’ hardware keeps GPT-5.6 Sol’s weights in 44 gigabytes of on-chip SRAM, avoiding the trips to external memory that create bottlenecks on GPU-based inference.
Behind the news: Ultrafast isn’t OpenAI’s only investment in speed. This week, the company released its first test results for Jalapeño, its new inference chip produced in partnership with Broadcom. These tests show promising latency and throughput measurements when running both OpenAI’s and open weights models. Note that chipmakers also have to balance speed against the total number of users and power used by their hardware: Each affects cost and reliability of inference.
Why it matters: Inference at these speeds changes what a model can be used for rather than just how well it performs a fixed task.
We’re thinking: Faster AI models have already enabled agents that code, retrieve data, and respond to conversations at the speed of speech. We encourage developers not only to think about which of their applications require more throughput and less latency, but new applications that are only possible with the kinds of speed new models and hardware can deliver.
Learn More About AI With Data Points!
AI is moving faster than ever. Data Points helps you make sense of it just as fast. Data Points arrives in your inbox twice a week with six brief news stories. This week, we covered what’s known about the mysterious Ox Alpha preview model and the latest reveal that it’s a multimodal GLM model from Z.ai. Subscribe today!
DeepSeek-V4-Pro Gets Refreshed
DeepSeek’s flagship model graduated from preview with improved performance plus the harness the model was benchmarked in. The harness is free and open-source, but the company’s model prices increased.
What’s new: DeepSeek released DeepSeek-V4-Pro-0813, the official version of the larger of its two fourth-generation models.
The company also released an open-source agent harness in developer preview and increased its API prices for all models.
How it works: DeepSeek-V4-Pro-0813 keeps the parameter count and architecture of the April preview, including DSpark, the speculative decoding module DeepSeek earlier released for the preview. DeepSeek says the update sharpens the model’s agentic skills but did not specify what it changed.
Performance: Independent evaluators found that DeepSeek-V4-Pro-0813 made large gains over its April preview, but held only a narrow lead over DeepSeek’s smaller Flash model. DeepSeek-V4-Pro-0813 debuted at third place in overall intelligence among open-weights models, but is roughly 10 points behind the best proprietary models on Artificial Analysis’ Intelligence Index. Its clearest improvement is in coding capability. The model’s cost per task remains a fraction of most proprietary leaders’, but after price cuts, OpenAI’s nearly equally capable GPT-5.6 Luna now costs less per task.
Behind the news: DeepSeek recently released an experimental version of DeepSeekV4-Flash with vision capabilities. The company has not announced plans for a similar vision-language version of DeepSeekV4-Pro.
Why it matters: DeepSeek published the harness it used to benchmark its models, which few companies do. Agentic performance is a joint function of the model and its scaffolding, so a published harness is the difference between numbers that a developer can only read and those they can reproduce. The harness is also MIT-licensed and model-agnostic.
We’re thinking: The agentic harness market is getting crowded! Although it’s an open-source release, some users may be skeptical of putting DeepSeek Harness into production before it receives a thorough evaluation from security experts, plus benchmarks when running other models besides DeepSeek’s. Let’s hope we see solid third-party testing — followed by developer experiments and modifications — soon.
LLMs Take Out the Agents’ Trash
AI agents typically compact the contents of their context windows by summarizing or deleting the oldest material. Researchers devised a method that manages an agent’s memory more selectively.
What’s new: Xubin Hao and colleagues at the social-shopping site Xiaohongshu (also known as RedNote) built self-governing context (Self-GC). Like familiar agentic designs, Self-GC builds up a context of inputs and outputs that includes user requests and tool calls as well as results like URLs and file paths. Unlike typical compacting algorithms, a large language model decides which parts of the context to keep, trim, or throw away before sending it to the associated LLM.
Key insight: Rules based on types or shelf lives of inputs and outputs aren’t sufficient to enable an agent to decide which contextual information it should retain. An older tool output may hold the only copy of a URL that must be reopened later, while a recent output may have become outdated. An LLM can make such judgments in a more flexible way, because it can read the information available as the context accrues and decide which details are likely to be useful later.
How it works: When input tokens fill more than 30 percent of the primary LLM’s context window, Self-GC sends the history to a planner model (by default, Qwen3.6-Plus) and asks what to do with each user request or tool call, plus their results.
Results: The authors tested Self-GC on an agent that browses the web, runs shell commands, and edits documents for Xiaohongshu users. They compared it to methods that follow fixed rules such as deleting the oldest messages or deleting the tool outputs. Self-GC removed less history, but it was far less likely to lose useful information. To measure this, the authors replayed completed conversations between real-world users and its agent. For each conversation, they stopped partway through, ran Self-GC on the history up to that point, and used GPT-5.5 to judge whether the shortened history still held every detail the real-world conversation went on to use.
Yes, but: The authors’ evaluation tested Self-GC’s ability to retain important details, but not its ability to produce good output. The output of a system that uses a shortened context may contain the same details and output as that of of a full-context system, but it may not be as useful.
Why it matters: Reducing context to save cost carries the risk that a model may forget a detail that’s required later. Knowing what to keep, trim, and remove enables long-running agents to be affordable without becoming forgetful.
We’re thinking: John McCarthy, who coined the term artificial intelligence, invented “garbage collection” for computers around 1959 so programmers would not have to manage computer memory by hand. The need to properly manage agent memory is another way in which LLMs can be treated like an operating system — and another way software engineering fundamentals are essential to building good AI applications!
Work With Andrew Ng
Join the teams that are bringing AI to the world! Check out job openings at DeepLearning.AI, AI Fund, Landing AI, and LearnVector.
Subscribe and view previous issues here.
Thoughts, suggestions, feedback? Please send to thebatch@deeplearning.ai. Avoid our newsletter ending up in your spam folder by adding our email address to your contacts list.
|
