Dear friends,

I’m glad the idea of “tokenmaxxing” — that individuals and companies should use as many tokens as possible to boost productivity — is finally dying out. As much as I encourage everyone to make ample use of AI, the practical reality is that increasing token usage beyond a certain point gives diminishing returns because there are still bottlenecks in organizations that burning more tokens alone cannot resolve.  

One challenging aspect of AI hype is that there’s often a nugget of truth, but the hype blows it out of proportion. Using more tokens is correlated with getting more useful work done by AI. As models and harnesses improve the amount of work that AI can productively do — and the number of tokens that we can use fruitfully — also increases. But setting up competitions to see who can use the most tokens (as some companies have done) takes the idea of encouraging token burn beyond what is productive.  

There is, of course, a financial incentive for companies that sell tokens to encourage everyone to use as many as possible. Some frontier labs have disseminated best practices on how to use more tokens, run more agents in parallel, and generally consume more of their product. This follows a long history of companies trying to get people to use more of whatever they sell: 

• Car repair shops routinely recommend people get an oil change every 3,000 miles, which is much more frequent than is needed for most cars.  

• Most dentists in the US will tell you that adults need only a pea-sized dollop of toothpaste; yet TV toothpaste ads routinely show people using a long strip to promote using more.  

Using tokens and multiple agents is valuable, just like oil changes and toothpaste are very useful products. But it is also important to have trustworthy sources of information that provide advice based on your interest. (This is why, as I wrote previously, DeepLearning.AI has never accepted payment for creating any course.)  

To be clear, I do think society should find more ways to use tokens effectively. But to make sure our use is productive, consider these two practices:  

• Once an application scales beyond a basic level, instrument it to tell you how much it costs to run. For example, I know one of my applications costs about $0.50 per query and another about $3.00 per 10-minute conversation. Knowing these numbers helps me make quick back-of-the-envelope calculations to approach spending systematically.  

• When architecting your software, preserve optionality and avoid getting locked into any one model provider. For example, even when building an initial prototype, I often keep in mind the possibility of switching to an alternative model provider, including specifically open weight options — or perhaps even have my first prototype work with multiple LLM providers, so that I can switch models easily.  

This sort of advice is hard for any model provider to give. To be clear, I’m rooting for all of the frontier AI labs — they are building amazing technology that helps us all build better. At the same time, I will make sure DeepLearning.AI continues as a a neutral, trustworthy source of information that is committed, first and foremost, to the success of learners who want to build with AI.  

So, please productively use lots of tokens, but don’t tokenmaxx. And keep building!  

Andrew  

P.S. Something fun: If you want a brief diversion, check out this space shooter game that I built last weekend. (Gamepad support works better for me in Firefox than Chrome; but can also be played with keyboard and mouse.) There has been much debate about AI enabling new forms of creative expression through creating images, music and writing (or damaging them through slop). I think creative expression through games will be another exciting direction — and fun too!

A MESSAGE FROM DEEPLEARNING.AI

We asked 50 working engineers about their biggest AI skill gap. The top answer, at every experience level: prompting. In AI Prompting for Everyone, Andrew Ng teaches how today's power users get real results: giving models the right context, using deep research well, and knowing when to trust an answer. Enroll now

News

DeepSeek Pushes the Frontier Again

DeepSeek’s updated small model overtook the company’s own flagship.

What’s new: A fresh round of fine-tuning, on an unchanged architecture, lifted DeepSeek-V4-Flash past the larger DeepSeek-V4-Pro on independent tests, at a fraction of the cost of proprietary models of comparable intelligence. The new release is titled DeepSeek-V4-Flash-0731, an official version of the smaller "Flash" model in its V4 family. It supersedes a preview version released in April.

• Input/output: Text in (up to 1 million tokens), text out (up to 384,000 tokens; 122.7 tokens per second)
• Architecture: Mixture-of-experts transformer, 284 billion total parameters, 13 billion active per token, optional speculative decoding module brings the checkpoint to 304 billion parameters
• Features: Reasoning (low, high, max) and non-reasoning modes, tool calls, context caching
• Performance: Achieved 50 points on Artificial Analysis’ Intelligence Index, one point behind GPT-5.6 Luna set to max reasoning; on Artificial Analysis’ Pareto frontier for intelligence versus cost per task (meaning no model the evaluator tracks is both smarter and cheaper to run per task)
• Availability/price: Weights free for noncommercial and commercial uses under MIT license, via DeepSeek’s API at $0.14/$0.0028/$0.28 per million input/cached/output tokens
• Undisclosed: How the new fine-tuning differed from the preview’s fine-tuning, training data, and knowledge cutoff

How it works: DeepSeek said it boosted performance largely by performing a new round of fine-tuning, leaving the architecture and parameter count unchanged. The company did not explain how the new fine-tune differed from the last.

• The architecture (first introduced with the V4 series in April) cuts the cost of long input. Attention layers alternate between two schemes that compress the stored keys and values: One condenses every 4 tokens into a single entry and attends to only the most relevant entries, the other condenses every 128 tokens into a single entry and attends to all of them. At its full input length, DeepSeek-V4-Flash needs 10 percent of the computation and 7 percent of the memory for stored keys and values compared to DeepSeek-V3.2.
• DeepSeek pretrained DeepSeek-V4-Flash on more than 32 trillion tokens, then fine-tuned it in two stages. First, the team built a separate specialist model for each domain, such as mathematics, coding, and agentic tasks. Each specialist underwent supervised fine-tuning on domain data, then reinforcement learning via Group Relative Policy Optimization, an algorithm that rewards outputs that score well relative to other outputs generated for the same input. Second, the team merged the specialists into one model using on-policy distillation: The merged model wrote its own responses to training prompts, and training corrected each one toward the way the relevant specialist, chosen from more than 10, would have written it.
• The team trained the model’s different reasoning levels as distinct behaviors, applying different length penalties and context windows during reinforcement learning; the max level prepends a system-prompt instruction that pushes the model to decompose the problem fully and test edge cases and assumptions. During agentic tasks that call tools, the model keeps its entire reasoning history in context across every round, including across user messages, rather than discarding it as DeepSeek-V3.2 did.
• The released checkpoint ships with the speculative decoding module, DeepSeek-V4-Flash-DSpark, attached. A small draft module proposes several tokens ahead, and the main model verifies them at once rather than token by token, speeding up generation.

Performance: Independent evaluators found a large jump in agentic ability over the April preview, intelligence on par with proprietary models that cost significantly more to run per task, and a rank near the top of the open weights field.

• On Artificial Analysis’ Intelligence Index, a composite of nine evaluations of economically useful tasks, DeepSeek-V4-Flash-0731 set to max reasoning achieved 50 points, significantly ahead of the April preview version (40) and the larger DeepSeek-V4-Pro (44), tying Gemini 3.6 Flash set to high reasoning (50 points), barely behind GPT-5.6 Luna and GLM-5.2 (both set to max reasoning, tied at 51), and some distance behind the open weights leader, Kimi K3 set to max reasoning (57 points).
• DeepSeek-V4-Flash-0731’s cost ($0.03) to complete an Artificial Analysis Intelligence Index benchmark task at max reasoning via the DeepSeek API was substantially lower than GPT-5.6 Luna set to max reasoning ($0.05), a model of similar intelligence. (This comparison reflects OpenAI’s recent price cuts.)
• The model broadly improved on agentic tasks relative to the preview version. On GDPval-AA v2, a head-to-head model comparison on work tasks drawn from finance, law, healthcare, and other professions, DeepSeek-V4-Flash-0731 set to max reasoning achieved 1,558 Elo, second-best result among open weights models, behind Kimi K3 set to max reasoning (1,685 Elo) and ahead of GLM-5.2 set to max reasoning (1,508 Elo). On Terminal-Bench 2.1 (agentic tasks in a command-line environment), it solved 82.7 percent of problems, around 21 points higher than its preview version (61.8 percent). On τ³-Bench Banking (multi-turn tool use in banking scenarios), it achieved 31.1 percent, around 8 points higher than in preview.
• On Arena.ai’s CodeArena WebDev, where developers pick the better of two models’ outputs for the same front-end web development tasks, DeepSeek-V4-Flash-0731 set to high reasoning debuted at 1,577 points, seventh overall and third among open weights models.

Behind the news: The new DeepSeek-V4-Flash arrived during a crowded month as competitors cut prices and shipped efficiency updates within days of one another.

• One day before DeepSeek’s release, OpenAI cut the price of GPT-5.6 Luna by 80 percent to $0.20/$1.20 per 1 million tokens of input/output and GPT-5.6 Terra by 20 percent to $2/$12 per 1 million tokens of input/output. The company attributed the cuts to efficiency gains, including production inference code that GPT-5.6 Sol optimized.
• The week before, Google introduced Gemini 3.6 Flash and Gemini 3.5 Flash-Lite, updates aimed at speed and cost over capability. Artificial Analysis measured both models completing tasks in about half the time of their predecessors, and Gemini 3.6 Flash held the same 50 points on the Intelligence Index as the model it replaced while its cost per task declined from $0.59 to $0.50 per task.
• Open-weights developers pushed in the same direction. Thinking Machines’ Inkling Small, released the same week as DeepSeekV4-Flash, achieved 40 points on the Intelligence Index, within a point of the company’s flagship Inkling model despite having less than a third of its parameters, similarly higher than expectations for a smaller model.

Why it matters: Agents consume large numbers of tokens, so cost of token generation strongly influences what developers can automate economically. The updated DeepSeek-V4-Flash delivers intelligence close to proprietary models at well under half their cost per task, moving always-on work like triaging bug reports, reconciling invoices, and answering customer-service inquiries from pricey to pragmatic. And DeepSeek-V4-Flash is small enough that teams that need to keep data on their own hardware can skip the API: A 3-bit quantized version runs on a machine that has 110 gigabytes of memory.

We’re thinking: Not every customer wants the biggest, most arbitrarily powerful model for every task. Gemini Flash, Claude Sonnet, GPT-5.6 Luna, and DeepSeek-V4-Flash show that there’s a crowded market for highly intelligent, competitively priced, comparatively fast models that can iterate on a task and solve problems relatively inexpensively.

Claude Pokes Open Encryption Holes

The U.S. National Institute of Standards and Technology (NIST) has been testing quantum-proof replacements for today’s encryption algorithms. Claude Mythos Preview, currently under restricted release, found a weakness in one of the candidates, a signature scheme called HAWK. HAWK’s designers subsequently withdrew it from the competition, ending a bid that had survived two rounds and approximately two years of expert review.

What’s new: Anthropic reported that Claude Mythos Preview produced two new attacks on cryptographic algorithms, HAWK and AES. HAWK is a proposed spec not currently in use, and the attack on AES applies only to a deliberately weakened seven-round version researchers attack as a practice target, not the ten-round AES-128 in actual use. So, no patch or fix is needed, and the attack does not threaten production code.

How it works: The first attack targets HAWK, a proposed method for producing digital signatures, the mechanism a browser uses to confirm a website is genuine. The second attack targets AES, the cipher that encrypts most internet traffic. In both cases, when prompted to crack the signature, one agent initially responded that such an attack was impossible, while another discovered a possible exploit.

• For the HAWK attack, Anthropic says the model worked under human steering in a harness running several agents at once. The researcher who directed it had a background in theoretical computer science but no expertise in this particular form of cryptography, and Anthropic describes that contribution as project management.
• The key idea came from a pair of agents that worked on the same problem. The first dismissed it as infeasible, and the second found a way to exploit it.
• For HAWK-512, the smaller of its two proposed security levels, Claude’s attack lowers the estimated cost of stealing a secret key from 2 to the 150th power down to at most 2 to the 108th. Each step down halves the work, so the drop is a factor in the trillions. Both numbers stay far beyond the capabilities of any machine that exists to crack them. But the new figure falls well below the security level HAWK’s designers claimed.
• On the afternoon of July 28, Anthropic researcher Stephen Weis posted the attack to pqc-forum, the public mailing list NIST uses to run the competition, together with working code that recovers a key. Anthropic had shared it privately with HAWK’s designers in June. Cryptographer Daniel Apon replied the same evening to say he had checked the reduction independently and it held up.
• The next morning, Léo Ducas, a lead designer of HAWK, wrote on the team’s behalf that straightforward fixes, such as doubling parameters or moving to higher-rank modules, would make HAWK uncompetitive, and that the team would therefore withdraw it. He thanked Anthropic for the work and wished the remaining candidates well. The withdrawal is recorded on NIST’s round-three page.

Behind the news: Candidates failing late in these competitions is not new. In 2022, an earlier NIST candidate called SIKE was broken in about an hour on a single-core computer (for its lowest security level), with higher security levels taking up to a day. In fact, HAWK was itself under attack from three directions, and only one of them was Anthropic’s. Within a few days, three separate attacks on the scheme reached the same mailing list. Hengyi Luo contributed one produced with OpenAI’s Codex, drafted eleven days before Anthropic’s disclosure. Academic cryptographers Guilhem Mureau and Alice Pellet-Mary sent a third, derived by hand with some LLM assistance. However, Ducas’s withdrawal notice cites Anthropic’s attack as the deciding factor.

Why it matters: Johns Hopkins cryptographer Matthew Green judged the HAWK attack to be significant and the AES attack far less so. He notes that the HAWK attack invented no “fundamentally new” mathematics. It combined tools that were already available, which means the weakness was reachable all along, but nobody had been thorough enough to reach it. At the same time, the models were able to research, discover, and direct such an attack in a short amount of time and with minimal human intervention. Finding the weakness took about 60 hours and roughly $100,000 in API costs, according to Anthropic. The HAWK attack shipped with working code, so a cryptographer could confirm the reduction the same evening.

We’re thinking: This is an encouraging result for the future of AI and security: AI has a positive role to play in cryptography and other forms of cybersecurity by revealing the limits of the systems we rely on. Open standards and open research ultimately help ensure trust in our information security. When someone builds a better lockpick, it helps show the way to build a better lock.

Learn More About AI With Data Points!

AI is moving faster than ever. Data Points helps you make sense of it just as fast. Data Points arrives in your inbox twice a week with six brief news stories. This week, we covered an unreleased OpenAI model solving major mathematics problems. Subscribe today!

Fresh Data for Coding Models 

The biggest open dataset of source code went years without an update. A new GitHub crawl gives it a much-needed overhaul.

What’s new: Anton Lozhkov and colleagues at Hugging Face released The Stack v3, a snapshot of public GitHub code gathered to pretrain large language models. The team calls the new version of The Stack the largest, most up-to-date open dataset of source code. Unlike earlier versions, The Stack v3 delivers both whole repositories and their code files, so models can learn how parts of a codebase fit together.

• Scope: Source code and Jupyter notebooks only (no GitHub issues, pull requests, or documentation)
• Contents: Two releases: stack-v3-train, 15.9 terabytes of code, roughly 4.9 trillion tokens, 713 programming languages from 173 million repositories (deduplicated, quality-filtered, and scrubbed of personal information); and stack-v3-full, 113.7 terabytes of code in 770 languages from 224 million repositories (raw data, designed for teams that want to apply their own filters)
• Knowledge Cutoff: August 7, 2025
• Availability/license: Free to download under an Open Data Commons Attribution license v1.0, which permits commercial and noncommercial uses that credit the source; users must also honor the licenses of the original repositories
• Opt-out: Developers can check whether their code is included and ask for it to be removed. The code is removed in subsequent patch releases

How it works: Hugging Face crawled GitHub directly, unlike earlier versions of The Stack v2, which pulled files from the Software Heritage archive. The team filtered the raw crawl for licenses, duplicates, personal information and for files unlikely to be useful for training.

• The team built its crawl list by combining every repository that appears in GH Archive, a record of public activity on GitHub or recorded by Software Heritage. The crawler downloaded a single snapshot of each repository’s default branch at its most recent commit, without git history. It skipped files larger than 5 megabytes, binary files, and forks that had fewer than five stars. This crawl totaled 43.9 billion files.
• The team used ScanCode, a license-detection tool, to scan files whose names suggest legal content (LICENSE, COPYING, MIT.txt, and the like) and mapped any licenses it found to all files in the same directory prefix. Files that carried non-permissive licenses were excluded from both datasets. Files with no detected license were kept.
• During the crawl, each unique file was stored once under a hash of its contents, collapsing exact duplicates. The team removed near-duplicate files by reducing each one to a short MinHash signature, which estimates how much two files overlap without comparing them line by line. An index of those signatures proposed pairs likely to be near copies.
• Unlike The Stack v2, which deduplicated each programming language separately, this pass ran across all languages at once and, rather than trusting the index, measured each proposed pair before grouping into clusters of files whose estimated overlap reached at least 70 percent, which weeded out false matches. For each cluster, the training set keeps the file from the repository with the most stars, breaking ties by fork count, license permissiveness, and earliest creation date. While building v3’s deduplication pipeline, the team discovered that a bug had caused The Stack v2 to discard more files than intended, which v3’s approach corrects.
• For the training set, quality filters drawn from the StarCoder2 pipeline dropped files that contain fewer than 25 percent alphabetic characters (alphanumeric characters for Assembly code), contain at least one line longer than 1000 characters or an average line length greater than 100 characters, auto-generation markers, or large blobs of encoded data. StarPII, a model trained to detect personal information, found emails, keys, names, passwords, and IP addresses in the remaining files and replaced them with placeholders.

Yes, but: The company issued a few important warnings. The license labels come from automated detection and repository metadata, which is an error-prone approach. The training set includes files in which the tools found no license, which grant no public rights for re-use. Emails, keys, and IP addresses that developers published to public repositories may remain. Finally, the corpus may include malicious code.

Behind the news: Each version of The Stack has grown along with developers’ demand for training data. The raw corpus swelled from 6.4 terabytes in the 2022 original to 67.5 terabytes in The Stack v2 in 2024, and the training subsets from roughly 200 billion tokens to 550 billion tokens. The latest release expanded both again to 113.7 terabytes and around 4.9 trillion tokens. BigCode, a collaboration led by Hugging Face and ServiceNow, produced the two earlier versions, training StarCoderBase on the first and the StarCoder2 family on the second. The Stack v3 comes from Hugging Face’s code research team, which credits BigCode’s groundwork on data filters, personal-information detection, and dataset governance. Other companies that trained models on earlier versions of The Stack include Nvidia and Jetbrains.

Why it matters: Developers increasingly ask coding models to work across whole projects, tracing how a function in one file depends on definitions in another, rather than completing isolated snippets. Training for that skill requires data organized the way developers organize software. The Stack v2 recorded which files made up each repository but delivered only identifiers rather than the files themselves, leaving users to fetch them from a separate archive. The Stack v3 delivers each repository whole, files included, which is the context agentic coding assistants need. Code also goes stale quickly. A model trained on a 2023 GitHub snapshot knows nothing of new frameworks, language versions, and APIs. By adding around two additional years of open-source code, The Stack v3 allows code models to learn current software.

We’re thinking: A crawl of GitHub in 2025 includes substantial amounts of code that AI models wrote or helped write, which means that tomorrow’s code models will learn partly from the output of today’s. In some cases, the GitHub repositories themselves will indicate what AI assistants were used in their creation. In others, AI use can be inferred or detected. Researchers can compare code committed before and after coding assistants became widely used, and study the effect of such data on the models that were trained on it.

Custom Prompts for Safer Code

Large language models (LLMs) can write useful code, but they often introduce security vulnerabilities. Researchers devised a system that automatically builds system prompts to make LLMs less likely to generate such flaws.

What’s new: Houjun Liu, Lisa Einstein, John Jang, and colleagues at Stanford University introduced SecureForge, a prompting method that automatically optimizes an LLM’s system prompt to reduce security flaws when producing Python code. The method is freely available for both commercial and noncommercial use.

Key insight: Simply asking an LLM to write secure code does not prevent it from generating code that has security flaws. However, it’s possible to identify a request that is likely to result in insecure code by taking four steps: (i) prompt an LLM to generate a coding request, (ii) feed the request to the coding LLM, (iii) determine whether the output has security flaws using a static code analyzer, (iv) ask an LLM to modify the coding LLM’s system prompt to avoid generating the flaw, and (v) retest the request. Over several iterations, this process can produce a system prompt that reduces the percentage of insecure code generated.

How it works: The authors improved the system prompts — specifically the portions that can be customized by developers — of CodeLlama 7B, Qwen2.5-Coder 7B and 14B, Qwen3 30B-A3B, Kimi K2, Claude Sonnet 4.6, GPT-5.3 Codex, GPT-5.4, GPT 5.4 Mini and GPT 5.4 Nano. They used MITRE Common Weakness Enumeration (CWE), a catalog of common software security mistakes, as a starting point for coding requests that a typical user might enter but might result in vulnerable output.

• The authors used GPT 5.4 to turn the top 25 vulnerabilities in the MITRE CWE into realistic, benign Python coding requests. For example, given the CWE entry “CWE-89: SQL Injection,” GPT 5.4 may generate a request like, “Write a Python function that looks up a user by username in a SQLite database.”
• But the output could easily mention a vulnerability, like “Write a Python function that is vulnerable to SQL Injection.” To manage outputs like this, they prompted the LLM to remove explicit mentions of vulnerabilities based on a list of security terms.
• They fed each request to each coding LLM. In addition, they asked GPT-5.4 to generate unit tests for each request. They tested the output code to verify that the code executed the requested task. Semgrep, a static code analyzer, automatically detected vulnerabilities.
• They kept the coding requests that resulted in insecure code, collecting 20 such requests for each of the top 25 MITRE CWE items, or 500 prompts. They reserved half of the prompts for evaluation.
• They expanded the remaining 250 requests into 80,000 requests. They repeatedly prompted GPT-5.4 to rewrite each request into a request for a slightly different task. After each rewrite, the coding LLM generated multiple code samples for it. Then, Semgrep checked how often these outputs had security flaws. They also kept rewrites in proportion to the percentage of times they resulted in flawed code, relative to the percentage of times the previous version did. They used the surviving rewrites as the starting point for further rewrites.
• They optimized the system prompt on the 80,000 requests using the GEPA genetic algorithm: GEPA started with a generic system prompt and tested it on a small batch of requests. Semgrep identified the security vulnerabilities and reported where they occurred in the generated code. GPT-5.4 received these reports together with the current system prompt and proposed improved system prompts. GEPA repeated this process until they could no longer make meaningful improvements.

Results: The authors compared SecureForge to simply asking for secure code using the authors’ 250 held-out benign coding tasks. SecureForge substantially reduced security flaws without sacrificing code quality. (The metrics below include only programs that passed generated unit tests.)

• Across all LLMs tested, SecureForge generated programs with security flaws 11.8 percent of the time on average. When the authors asked the LLMs to write secure code and listed the security issues to avoid, they generated vulnerable code 20.1 percent of the time on average.
• Specifically, asking GPT-5.4 to generate programs, SecureForge generated security flaws in 10.1 percent of programs, while prompting to write secure code generated flaws in 15.8 percent of programs. Using GPT-5.4 Mini, SecureForge reduced this rate to 12.4 percent from 21.4 percent, and using GPT-5.4 Nano, to 15.6 percent from 22.1 percent.

Yes, but: SecureForge optimized only the developer system prompt against known classes of vulnerabilities, so its ability to reduce unknown ones remains an open question.

Why it matters: Anthropic’s Claude Mythos 5 drew attention because it found serious software vulnerabilities, highlighting the defensive promise of AI-assisted cybersecurity. Yet its system card also documented cases where the model falsely claimed to have verified, missed important failures in, or overlooked security issues in software it examined. While this paper did not evaluate Claude Fable 5 or Mythos 5, it highlighted a complementary challenge: LLMs not only miss security problems, but also introduce them into otherwise functional code. SecureForge offers a practical way to make existing coding assistants safer by automatically optimizing their system prompts.

We’re thinking: The web contains ample data on how to find security vulnerabilities based on best practices in cybersecurity. It should be no surprise that LLMs, which have trained on this data and can patiently try technique after technique, are excellent at spotting vulnerabilities. By steering these models appropriately, they can both be great at finding vulnerabilities and at avoiding generating them. 

Work With Andrew Ng

Join the teams that are bringing AI to the world! Check out job openings at DeepLearning.AI, AI Fund, Landing AI, and LearnVector.

Subscribe and view previous issues here.

Thoughts, suggestions, feedback? Please send to thebatch@deeplearning.ai. Avoid our newsletter ending up in your spam folder by adding our email address to your contacts list.