Dear friends,
I previously wrote about our AI Engineering Skills Map, with the highest level skills being (i) Building and deploying AI applications, (ii) Software engineering fundamentals, (iii) Using coding agents, and (iv) Shaping the build. In this letter, I will flesh out the first of them.
Being skilled at building and deploying AI applications means knowing:
This map of skills was formed by analyzing a large number of job postings, structured expert interviews, and survey responses.
The key difference between AI applications and non-AI software is that the former’s output is less predictable. You don’t know in advance what an LLM will output, or what predictions a supervised learning algorithm will make. Because of this uncertainty, building AI systems is a much more iterative process than building traditional software — it is harder to plan the process in advance. Skilled AI engineers repeatedly build a piece of software, examine it, and decide what to try next, taking a sequence of steps that are highly influenced by the intermediate results. Being able to skillfully decide what to do next allows you to create reliable software systems based on unreliable AI components. This requires knowing:
LLM foundations. Understanding how large language models tokenize input and generate output allows you to understand when to count on them and when they may fail. It also allows you to understand when to use a multimodal model, how to make tradeoffs on what to include in the context window, and reason about cache hits, knowledge cutoff, reasoning effort level, sampling parameters, and when to use special features such as tool calling. Understanding these foundations helps you choose the right model or mix of models and apply specialized techniques when needed, such as fine-tuning or self-hosting models.
Grounding models with data. LLMs require good input context to produce useful outputs. RAG using vector search was an early attempt to give LLMs relevant context, but the set of techniques for grounding models with data has grown significantly. For example, you will have to decide what to include in a prompt vs. what to let an LLM retrieve on demand using tools, and which representation fits the data and search queries: a vector index, a knowledge graph, or a semantic layer over structured data (such as customer records). You’ll also turn documents (text, PDFs, HTML, images) into LLM-ready inputs and engineer pipelines to keep data clean and fresh. When you understand the menu of techniques available to get data, you are better able to give your LLM relevant context.
Building agentic systems. Agentic systems range from workflows that execute a predefined sequence of LLM calls to ones based on an agent harness that lets an LLM repeatedly decide its own next step. You’ll have to choose the architecture — what steps to chain, what to parallelize, when to use code and when to use an LLM — and engineer the workflow or harness, with fallbacks. When designing the agent loop, you will also decide what tools the model can call (including MCP, CLI and sandbox execution environments), what memory architecture to use, how to manage context over long sessions, and when a task needs multi-agent orchestration instead of a single-agent architecture. You’ll also want to turn promising prototypes into reliable, safe and secure agents for production; this requires understanding guardrails, adversarial inputs, and identifying and working around key risks (such as data exfiltration), and governance.
Agentic workflows are evolving rapidly, and you will also benefit from understanding any cutting-edge techniques relevant to your application area, such as voice agents, computer-use agents, or generative UI.
Evaluation-driven development. In my experience, the most important trait that distinguishes someone great at building AI systems is whether you can drive a disciplined evals/error analysis loop to drive development. This allows you to repeatedly focus your effort on directions that are more likely to be fruitful. I’ve found this to be a tricky skill to master, because the right approach varies significantly by project and even according to the stage of the project.
Building good evals is a deep technical skill. You might look at a system’s traces and outputs, carry out exploratory data analysis, and combine that with product and business insight to decide what to measure. You should also understand the menu of options for evals, such as when to use deterministic (code-based) evaluations, when to use an LLM-as-a-judge, and when to have a human in the loop, and how to evaluate your evals so as to keep evolving them. These evaluations then feed into an iterative process that drives further development, and makes progress systematic rather than random.
Operating in production. Operating AI software is different from traditional software because of its unpredictability, cost, and latency. First, you should know how to build observability mechanisms to understand the system’s performance on real usage. You’ll track performance, detect drift, and respond quickly to model failures and security incidents such as adversarial prompt injections. Putting in place regression testing and CI/CD requires more statistical evaluations than traditional software, and the testing effort should be calibrated relative to the risk of a mistake. Additionally, it’s important to know how to select the right mix of techniques — such as model choice optimization, distillation and fine-tuning, and agentic workflow simplifications — to optimize for cost and latency, especially if your application reaches many users.
Machine learning foundations. Modern LLMs are built using machine learning techniques including supervised learning and reinforcement learning. Every engineer I know that’s good at building with LLMs also understands machine learning and deep learning at some depth. Additionally, many applications still require knowing how to use machine learning -- either a model someone else trained or one you train yourself. This requires knowing the popular machine learning and deep learning models and tradeoffs in accuracy, training speed, inference speed, and so on, and understanding how to engineer the data needed to train and evaluate these models. The machine learning concepts of bias/variance, error analysis, and engineering your data — all of which are core mental frameworks for navigating how to work with systems with uncertain output — also remain key to making a wide range of decisions in AI system development.
There is a lot to learn to become good at building and deploying AI systems. This is a field with significant technical depth. But every bit you learn will help you become better at AI Engineering and build more exciting applications. A strong complement to these skills is software engineering. I will write more about this in the next letter.
Keep building! Andrew
A MESSAGE FROM DEEPLEARNING.AI
We’re continuing to update the AI Engineering Skills map, and we’d love your input. Tell us more about what real AI engineers need to know to do their jobs. Complete the five-minute survey here
News
Grok’s Cursor Alliance Pays Off
Once a lab that produced mid-tier models, SpaceXAI has steadily improved. It just built one of the most capable models in the world while keeping prices relatively low.
What’s new: SpaceXAI introduced Grok 4.6, a vision-language model developed with Cursor and aimed at long-running agentic work. It’s available to developers now via the API, in Grok Build and Cursor, and is due in the consumer Grok apps later.
How it works: Grok 4.6 is the latest model in SpaceXAI’s 1.5-trillion-parameter model family, building on Grok 4.5. SpaceXAI credits gains in performance to longer training on curated data, followed by fine-tuning on data generated by Grok 4.5 and reinforcement learning on agentic tasks. The training data included anonymized coding-agent data from Cursor, which included use of non-Grok models.
Performance: Grok 4.6 improved its performance on both self-reported and independently measured benchmarks, rising to near the top of the leaderboards. On many benchmarks, Grok 4.6 rivals Claude Opus 5 and GPT-5.6 Sol and does so at a lower cost per task.
Behind the news: Grok 4.6 is the second model to come out of a partnership that led to an acquisition. In April, Cursor agreed to train its models on SpaceX’s Colossus supercomputer, a deal that gave SpaceX an option to buy the company. Cursor’s coding-agent data and SpaceXAI’s computation yielded results almost immediately: Grok 4.5, jointly trained with Cursor and introduced in July, lifted Grok 4.3 from 38 points on Artificial Analysis’ Intelligence Index to 56 points. SpaceX exercised its option in June, and the roughly $60 billion all-stock acquisition closed on August 14, days after Grok 4.6 launched. Three days later, Cursor introduced Origin, a code hosting service comparable to GitHub designed to handle the higher volume of code that agents generate.
Why it matters: Model makers used to tout benchmark scores at launch. Increasingly, they also publicize cost and steps per task. Grok 4.6's clearest advantage over its near competitors is completing long-running work with fewer turns. At the same price per token and task, an agent that finishes in half the turns costs around half as much, which affects what applications are feasible to build with that model.
We’re thinking: The Cursor team brought data and technical expertise to SpaceXAI, and deserves credit for supporting Grok's rapid rise in model capability. Let’s hope Grok’s continued progress and aggressive pricing makes other top labs follow suit to keep per-token prices in check.
How Claude's Watermarks Work
Anthropic introduced invisible, machine-readable signals that text and images were generated by Claude.
What’s new: The marks will be deployed worldwide in all Claude models launched after August 2, 2026. The company said the watermarks are necessary in order to comply with the European Union’s AI Act.
How it works: Anthropic will generate a digital watermark – a hidden, coded signature that is used to identify AI-generated content – for generated text and a metadata credential for edited images. The policy applies to all forthcoming models and will be phased in for existing models. On August 14, the company added more details about its watermarking methods, their limitations, and implications for users.
Behind the news: The announcement received widespread backlash, with many critics saying that watermarking is ineffective, harmful, or a privacy violation for AI users. Proponents argued that distinguishing between AI-generated and human-written text could actually help AI engineers.
Why it matters: Anthropic isn’t the only company that will have to build a way for generated text to be detectable; it’s just the first mover. Article 50 of the EU AI Act demands machine-readable watermarks for generated content, including text, image, audio, and video. However, Anthropic chose to apply this globally, rather than only within the EU. Some large AI developers, including OpenAI, Google, Meta, and Microsoft, have also signed the EU’s Code of Practice on Transparency of AI-generated Content, a voluntary framework for demonstrating compliance with the AI Act’s requirements for marking and labelling AI-generated content. These and other companies may use different techniques, and it still remains to be seen if they will implement watermarking only in the EU, or globally.
We’re thinking: We’ve long taken the view that harmful use cases of AI should be regulated rather than the technology itself. Anthropic’s watermarking and detection methods are built into the technology, subtly altering generated text. Addressing plagiarism and identifying synthetic content for model builders may be laudable goals, but we anticipate that universal watermarking will prove to be too blunt an instrument, opening a Pandora’s box of issues around privacy, quality of output, and harmful false positives.
Qwen3.8-Max Lands With A Bang
Open models are getting larger and more capable. A few weeks ago, Moonshot AI announced Kimi K3, the largest and best-performing open weights model yet. Last week, Alibaba answered by releasing weights for a giant of its own.
What’s new: Alibaba first unveiled Qwen3.8-Max, a 2.4 trillion-parameter vision-language model trained to carry out long-running coding and knowledge work tasks, on August 2. The company released the weights of both Qwen3.8-Max and the smaller Qwen3.8-27B within a week. While smaller Qwen releases have been open weights, this is the first Max-tier model with downloadable weights. However, the open weights version of Qwen3.8-Max is limited to text input and output and doesn’t support the full million-token context window.
How it works: Alibaba has not yet published a technical report or model card for Qwen3.8-Max. The company says the model is built from Qwen3.5, its earlier vision-language model. Alibaba’s release notes for the model highlight reinforcement learning for agentic work and training that teaches the model to visually verify its own output using the model’s vision capabilities.
Performance: In independent evaluations at launch, Qwen3.8-Max trailed only a handful of top proprietary models and Moonshot AI’s Kimi K3 in overall intelligence. It likewise placed near the top in front-end coding, vision, and agentic work, leaping over Qwen3.7-Max on virtually all benchmarks. But these gains came at a price. The model generated more tokens than its rivals, which drove up its cost per benchmark task completed despite low per-token prices.
Behind the news: Qwen 3.8-Max’s downloadable weights mark a strategic shift from Alibaba’s recent practice of keeping its most capable models to itself. Since the Max line began in January 2025, the company has offered those models, most recently Qwen3.7-Max, through its API and Qwen Chat app while releasing weights only for smaller models like Qwen3.6-27B. In Alibaba’s absence, other labs in China, including Moonshot AI and Z.ai, traded the open weights lead among themselves. Days after Kimi made its K3 model available, Alibaba previewed Qwen3.8-Max at the World AI Conference in Shanghai, before it made the model available via API on August 2. Z.ai followed with GLM-5.3 on August 14, a model that netted a 60 average on the Artificial Analysis Intelligence Index.
Why it matters: As always, open weights gives developers more options, from self-hosting to using Alibaba’s official tools or API. Admittedly, few teams can serve a 2.4 trillion-parameter model themselves, so the payoff will arrive via price-reducing competition among third-party hosting providers and distilled descendants like Qwen3.8-27B. Downloadable weights also let developers shop for better privacy terms, since any provider can serve them, including those that retain no customer data.
We’re thinking: It’s disappointing that the open weights version of Qwen3.8-Max doesn’t include the full capabilities of the Alibaba API version, but Qwen3.8-27B on a laptop is a fun consolation prize — even if it does seem to overthink things sometimes.
Agents Come to Speech Recognition
Most speech-to-text systems transcribe speech in a single pass, which makes them unable to correct errors in their outputs. Researchers built a system that allows for interactive corrections.
What’s new: Zixuan Jiang, Yanqiao Zhu, Peng Wang, and colleagues at Shanghai Jiao Tong University, Zhejiang University, Fudan University, and Xiaoice, a Microsoft spinoff, devised a workflow that pairs an automatic speech recognition (ASR) engine with an LLM that attempts to detect and correct transcription errors at every conversational turn. They call it Agentic ASR.
Key insight: Automatic speech recognition systems often manage mistakes by pairing a speech-to-text model with an LLM that rewrites its output. But correcting that system simply appends the user’s request for a correction to the existing transcription. Asking an LLM to rewrite a transcript from scratch risks introducing new errors, especially where the words spoken are idiosyncratic (for instance, the name Megan instead of Morgan). The LLM can more successfully revise the transcription if it divides the correction process into three steps: find the error, understand what the user said, and apply a fix. This approach enables the LLM to act more like an editor than a rewriter.
How it works: Agentic ASR treats transcription as a multi-turn process of refinement, in which users can dictate and then confirm or correct the transcription in further turns.
Results: The authors evaluated the workflow on multilingual speech-to-text benchmarks. Such benchmarks don’t include examples of corrections to output transcripts, so the authors used an LLM and text-to-speech engine to simulate a user dictating and then correcting the output text.
Why it matters: Many systems that accept voice input need to understand the user’s intent, not every word spoken. Agentic ASR does this very well, and when every word counts, it makes speech-to-text more robust in a natural way. This capability gains importance as UIs that depend on voice input become more widely used.
We’re thinking: The workflow of decomposing corrections into identifying where to edit, understanding what to change, and applying the change looks like a potential blueprint for interactive tasks such as editing documents, reviewing code, and iterating on a design.
Work With Andrew Ng
Join the teams that are bringing AI to the world! Check out job openings at DeepLearning.AI, AI Fund, Landing AI, and LearnVector. Subscribe and view previous issues here.
Thoughts, suggestions, feedback? Please send to thebatch@deeplearning.ai. Avoid our newsletter ending up in your spam folder by adding our email address to your contacts list.
|
-2.png?width=1200&upscale=true&name=The%20Batch%20top%20banners%20(2)-2.png)
%202.png?width=1200&upscale=true&name=Building%20and%20Deploying%20(Border)%202.png)
