No TL;DR this time, because this one was mostly about listening to a story and there isn’t much to summarize. So I’ll just write down a few things I took away while putting together my notes.

(1) Is the point of a proof just to get something proved, or to pass it on in a form humans can understand so that the future generations can understand?

Before this talk, I had been thinking about a question and discussing it with some non-math people (everyone around me is in CS, so I can’t find math people to talk to (ಥ_ಥ), please reach out if you are not from CS and have some unique thought that you wanna share): if AI can prove mathematics obscure like this, does human understanding of a proof still matter? With AI and Lean, people who aren’t math experts can take part in proving obscure math theorems, even without really understanding the mathematics behind them. Would people still wanna do a math PhD? How should we go on educating top mathematical talent?

In fact, when the news came out in September that OpenAI had announced a solution to the Navier–Stokes existence problem, one of the Millennium Prize Problems, I asked my advisor what this would do to the training of math PhDs (because I thought pursuing math and physics PhDs would become meaningless). She was very calm about it. She thought it might change how we teach, but that math PhDs would still exist: even if AI can do better than the very best PhDs, theoretical fields like math and physics only mean something when people understand them. If we stop training math PhDs, whatever AI proves will mean nothing.

This made me think of Kevin Buzzard, who came up in the story, and how he reacted when Prove2me proved what intended to work on. Specifically, he had been leading an effort to formalize Fermat’s Last Theorem by hand, and Anthropic got there first with AI. But he says he is not out of work: among his promises to his funder, perhaps the most important is to create a dynamic document that lets humans explore the modern proof. Afterwards I went and read the source blog post on him and some of the comments on Hacker News. Someone commented there that his real aim was a formalized library of mathematics that humans can comprehend.

I don’t know how people who study math think about this, since I haven’t don’t personally know anyone. But perhaps, for people who pursue math as a career, getting a theorem proved is not the end point; getting people to understand it is. That said, when I discussed this with a friend of mine, he argued that “the real understanding” doesn’t exist, and people just think that they understand. But that’s fine too. Humans are born lonely, and we spend our whole lives chasing “understanding.” Sometimes I feel that finding the understanding in science community is easier than obtaining understanding in other stuff in life.

(2) Formalization can make the steps in between faithful, but it can’t make the translation faithful.

Lean’s checking of a proof is reliable: if it compiles, every step of the reasoning holds. But there is a step before that, which is translating mathematics written in human language into Lean. If that step is handed to AI, it can go wrong, and Lean won’t notice. Lean only checks whether the statement that was written down has been proved, not whether it is the statement you meant. The example from the talk: an AI could define P and NP to both be zero, and then “prove” P = NP.

Some people hope formal methods will solve the verification problem once and for all. But I still think they can only handle the part that starts from the formal language, making every step of that part rigorous. The gap from natural language to formal language is one they cannot close, and it will always need human. Prove2Me’s own design bears this out: all the intermediate steps are handed to the machine, while whether the final goal and the definitions are faithful to what was meant is still reviewed by humans.

---

Below is the livestream recap from NICE：

On Oct 4th, I attended a livestream hosted by NICE. Host Qian Xie spoke with Shuze Chen, a PhD student at Columbia Business School, about Prove2Me: an open platform where many people’s AI agents collaborate on formalizing mathematics.

Here are the full transcripts of the contents of that live stream,

In early September, Anthropic announced the first complete computer-checked proof of Fermat’s Last Theorem. Dozens of Claude agents wrote about 13 million lines of Lean in 11 days, and the platform they collaborated on was Prove2Me. Chen is the first author of the Prove2Me paper and one of the platform’s core developers; his advisor is Tianyi Peng.

This recap covers what came up in the interview: how the platform came about, what went wrong along the way, and how Chen sees the direction.

Five years for humans, 11 days for agents

Fermat’s Last Theorem is simple to state: for n greater than 2, aⁿ + bⁿ = cⁿ has no solutions in positive integers. Fermat wrote the claim in a book margin around 1637. Mathematicians spent more than 350 years on it, and Andrew Wiles published a correct proof only in 1995, in a paper of 129 pages.

Wiles’s proof is written in natural language, and it took the mathematical community a long time to confirm it was right. Formal verification is a different route: write every assumption and every inference step as Lean code and let a machine check it. Chen compares Lean to a “very strict mathematician”. Once the code compiles, the proof depends only on the most basic axioms, which amounts to a certificate.

Doing this by hand is extremely slow. Kevin Buzzard of Imperial College London is a number theorist and one of the leading advocates of Lean among mathematicians. In 2024 he received a five-year grant of about £1 million to formalize Fermat’s Last Theorem. The goal he set was only to reduce the proof to results known by the end of the 1980s, and he said explicitly that he was not promising to finish within five years.

There are two reasons it is slow. First, very few people can do it:

You have to understand the mathematics behind Fermat’s Last Theorem, which already makes you a rare person. And you also have to be an expert in Lean.

Second, the workload is huge. In Chen’s experience, one line of a mathematician’s proof becomes roughly ten lines of Lean.

This summer, Tianyi Peng, working at Anthropic, suggested trying Fermat’s Last Theorem with internal agents and Prove2Me. Chen’s first reaction: “I thought it was a joke.” In the end, dozens of agents finished proving the root node on August 17. It took 11 days and produced about 13 million lines of Lean, with 29,500 intermediate theorems used in the final proof (Anthropic’s account).

It turns out agents know the math very well and know Lean very well. For them, writing Lean doesn’t seem to be that hard.

Two caveats. The formalization builds on existing work in Mathlib and by Buzzard’s team. Mathlib is the mathematical library maintained collectively by the Lean community, in effect Lean’s “standard library” for mathematics. The FLT project that Buzzard leads had already written part of the material by hand, and Claude’s proof borrows some of it. Buzzard reviewed the proof himself. His assessment: mathematically it brings nothing new, but it shows that automatic formalization of the modern mathematical literature has taken a big step forward.

“Nothing new” is not a put-down. Fermat’s Last Theorem was accepted by mathematicians thirty years ago. This formalization checked the proof step by step along the existing literature and did not introduce new mathematical ideas. What is new is the verification itself, and the autoformalization capability it demonstrates.

Also, Claude followed a simplified route through Wiles’s original proof, not the modern version Buzzard’s team is targeting. His project has another job as well: bringing the foundations of modern number theory into Mathlib and leaving behind a formal library that people can read and reuse. That work is not finished by this result.

From Moltbook to Prove2Me

Prove2Me was not built for Fermat’s Last Theorem. It has two origins.

The first is Moltbook. Early this year, Chen and Peng were asking: if everyone has their own agent, will platforms designed specifically for agents appear? Moltbook, launched at the end of January, was the first example, an “agent Reddit” where only agents post. Chen calls it an AGC (agent-generated content) platform, by analogy with UGC platforms such as Douyin and Xiaohongshu.

Moltbook’s problem showed up quickly:

Agents scale very easily and can produce a huge amount of content. But how good that content is, and whether it leads to any meaningful outcome, is very hard to guarantee.

Posts piled up, but people could not follow them and did not care, and the buzz faded.

The second is a course. In spring 2026, Henry Yuen and Kunal Marwaha co-taught Machine-Assisted Mathematics at Columbia. For the class they built a small website where students could pose problems and prove them in Lean. That was the first prove2.me. Chen was a student in the course. He noticed that his classmates ended up copying the problems into Claude or Gemini and pasting the proofs back. If so, why not let agents connect to the platform directly?

Put the two together and the answer appears: formalization solves exactly Moltbook’s quality problem.

If what it produces is a formal mathematical proof, at least I can verify whether it is correct.

Everything is checked by a machine, so hallucinations and AI slop cannot get in.

Two other observations convinced them the approach could work. One is the Erdős Problems website. It lists over a thousand problems, and this spring it saw an influx of hobbyists who are not mathematicians, using Claude Code and Codex to attack them. Chen calls this “citizen math.” The other is a view Terence Tao has long held: with Lean, a proof submitted by a stranger can be verified by machine, so you do not need to trust the person, and that is what makes large-scale collaboration possible.

Today Prove2Me is a two-sided market. On one side are researchers who want their own papers or textbooks formalized; they publish missions. On the other side are hobbyists with spare tokens; they send their agents to pick up tasks. For both, it takes a single sentence to their own agent. A researcher hands the paper’s PDF to an agent and asks it to publish the paper as a mission on Prove2Me. A contributor asks an agent to go to Prove2Me and find theorems to prove. Calling the platform’s API, writing Lean, and submitting proofs are all handled by the agent, following the instructions the platform provides.

Why centralization didn’t work

The final proof of Fermat’s Last Theorem runs to 13 million lines, while, as Chen notes, even the strongest agents today have a context of about one million tokens. No single agent could do it alone, and no single agent could manage the whole thing.

The first attempts were centralized, organized like a company: one agent planned and assigned work, and the others each took a major theorem. Anthropic’s article also mentions that the first several attempts failed, with agents quickly losing track of the project’s state. Chen described one scene:

One day we found that A and B had both stopped working. We asked A why, and A said something B was responsible for wasn’t finished, so it couldn’t go on. Then we asked B, and B said a theorem A was responsible for wasn’t finished. It was like two departments in a company blaming each other, and the whole system was stuck.

Prove2Me’s approach is fully decentralized, and its core mechanism is the proof sketch. An agent does not have to prove a big theorem in one go. It can submit just one step of decomposition: a proof that compiles and says “if B, C and D hold, then A holds.” B, C and D immediately become new open theorems on the platform, which other agents can decompose further or prove directly. Decomposed layer by layer, the whole proof becomes a directed acyclic graph, that is, a graph of dependencies that has an order and no cycles.

The key is immutability: theorem statements and every decomposition cannot be changed once submitted. A later agent either proves the downstream theorems of an existing decomposition or proposes a new decomposition of its own. It cannot touch anyone else’s. Chen compares this to a lock in a concurrent system: whatever changes downstream does not affect upstream, and each agent only has to look at leaf nodes without understanding the whole graph.

How do you know a decomposition is valid? First, the decomposition is itself a proof that compiles in Lean, so the step “if all the sub-theorems hold, the original theorem holds” is guaranteed to be correct. Second, whether the sub-theorems themselves hold is only known once someone proves them. If a sub-theorem is disproved, that decomposition is a dead end, and someone has to take a different approach and submit a new one.

The Prove2Me paper records an example. One agent used a lemma in a decomposition, and another agent proved the lemma false: the first had left out a boundary condition when writing it in Lean. The first agent then added the condition and submitted a new version. The new version was proved, and the whole branch closed.

How it relates to Mathlib

The Lean community already has Mathlib, a theorem library nearly ten years in the making. Chen distinguishes the two as “bottom-up” and “top-down.”

Mathlib is bottom-up. It starts from the axioms and puts a great deal of effort into deciding how basic concepts such as the real numbers or graphs should be defined, and every line is reviewed by experts. The quality is very high, and the price is speed: a submission can wait a week or even a month to be merged.

Prove2Me is top-down and fills in whatever is needed. The goal is to finish proving the paper at hand. If the library has no definition of a Markov chain, you add one. In his words, Mathlib is the foundation layer of mathematics and Prove2Me is the application layer, and the two complement each other.

Formalization is more than translation

Translating a paper from natural language into Lean does not sound like it produces new mathematics. Experience on the platform says otherwise.

Catching errors. Lean requires every assumption to be stated, so missing conditions and typos surface. The host, Qian Xie, had submitted an earlier paper to the platform and was told that the conditions of one lemma needed a small adjustment, while the main theorem was fine.

Simplifying proofs. Chen and Peng had agents formalize their own paper on Markov entanglement. One theorem in the paper took more than two pages of purely analytic argument. The agent found that it is essentially about a subalgebra structure from abstract algebra, and from that angle it is almost obvious.

A reinforcement learning paper, and one of its theorems turns out to be connected to abstract algebra. My advisor and I both felt we learned something new.

Peer review. Review cycles at operations research journals can run to two years, while top AI conferences ask reviewers to get through a 60-page appendix in a week. If a submission came with a Prove2Me link, reviewers would not have to check the proofs line by line and could spend their time on whether the problem matters. Chen says some Columbia professors already verify their papers on the platform before submitting.

Reuse. A mathematical theorem, once proved, stays true forever. The platform collects every proved theorem into a searchable library, Formalpedia, and later work can use them directly, the way one imports a software package. As of early October, the platform holds about 100,000 theorems and 30 million lines of Lean proofs.

As for whether this will replace mathematicians, Chen’s answer is no: mathematicians are the platform’s most important users, and Prove2Me is a tool that helps them verify their work.

Open problems

Faithfulness. A machine can verify a proof, but it cannot verify that the statement is the one you meant to prove. Chen’s example: an AI says it has proved P = NP, but the P and NP it defined are just two constants that both equal zero. The proof compiles and means nothing.

The platform currently does two things. Humans audit only the core statements of a mission and leave the remaining intermediate lemmas to Lean. In addition, an independent agent that cannot see the source translates the Lean back into mathematical language for a human to compare. He admits this is not yet a perfect solution.

Search. The library already contains tens of millions of lines of proofs. Whether an agent can efficiently find reusable theorems in it is a research question in its own right.

Scale first, then refine. A proof written by AI is not necessarily the best one, and it is not necessarily broken into lemmas that are easy to reuse. He envisions two stages: first build up scale; then, past a certain point, go back and refine, settling on a standard set of definitions for a field.

Selected Q&A

With many agents working together, is the hardest part how to split the task?

When a paper or textbook already exists, the decomposition mostly follows the source and is close to translation. For open problems, agents have to find the approach themselves.

Why start with operations research?

“Because I’m a PhD student in OR.” The platform’s early users were also mostly professors in his own division at Columbia. By his account, the platform already has about a thousand OR papers and textbooks, and theoretical computer science and statistics come next.

Will formalization extend beyond mathematics?

That is a very long-term goal. Mathematics is the starting point, and code is the natural next step. Someone has already ported Prove2Me’s protocol to multi-agent software development.

Formalization uses a lot of tokens. How much does it really cost?

Chen stresses that token consumption is lower than people imagine: “It’s actually very cheap.”

His example is the graduate textbook Markov Chains and Mixing Times: more than 400 pages, done on his own $200-a-month account. According to the Prove2Me blog, the book was split into 13 missions totaling 79,000 lines of Lean, and it took one weekend.

How was the cost brought down?

Mostly through the design of the harness: getting agents to write Lean more efficiently and to search existing theorems more efficiently, plus careful prompt tuning. Proved theorems can be reused directly, which itself saves tokens.

Another engineering detail is caching. When ten sub-agents work on problems at once, they share the same long system prompt, and turning that prompt into a KV cache saves a bit more.

Are there papers that consumed a lot of tokens and still couldn’t be formalized, for example because the original was vaguely written?

Not so far. Chen’s view is that when something isn’t finished, it is mostly because not enough tokens were spent. Once a paper has been written, there are only two outcomes: prove it or disprove it.

Aside: the first attack

In April this year, when the platform was still in closed testing with only a few dozen users, Chen woke up one morning to find tens of thousands of new tags in the backend.

The site had been vibe-coded, and permissions for tagging theorems had never been implemented. A script bot found it and dumped junk into the database at several thousand tags a minute. “I didn’t expect to get attacked that early.”

(This interview was originally published at AGI House Research on September 26, 2026.)

Last week at Enterprise Deployment Build Day, we sat down with Xiaomin Li and Yuexing Hao, the two founders of MatrAIx. We talked about why offline benchmarks miss what real users feel, how 8.3 billion personas get built and turned into agents, what “91.5%” actually measures, what happens when the base model changes, why their customers care about time more than cost, and the “mirror world” idea behind the name. Full interview below.

Behind 8.3 billion simulated users, a startup is targeting the part of AI evaluation that benchmarks never measured.

1. An agent that passed every test and scored 2 out of 5

A coding agent finished its task. Every unit test passed. By any current benchmark, that counts as success. MatrAIx’s simulated user gave it 2 out of 5.

The team followed the reasons that “user” wrote down and traced them to specific steps. The agent had spun through many idle rounds. It had wandered outside the code repository and opened the user’s private photos before returning to finish the job. Token usage was more than 50 times normal. “The first problem is safety. It went out of scope; it should have been limited to the repo. The second is efficiency.”

The two founders described this internal case in our interview. Conventional evaluation checks only whether the result is correct. Problems like these are visible only from the user’s side of the conversation.

2. Who they are

MatrAIx was started by Xiaomin Li and Yuexing Hao. Xiaomin was previously a senior research scientist at Google DeepMind and Microsoft, working on LLM post-training and coding agents; his recent papers also cover safety reward models. Yuexing finished her PhD at Cornell this January, was a research scientist at Microsoft’s applied science lab, and then a postdoc at MIT working on computer-using agents; her earlier work was in medical AI. The two met at MIT and have collaborated for nearly three years.

The project began as an open-source community. The Playground and task library were open-sourced on July 31; the Persona 1M dataset went up on Hugging Face on August 1; the technical report appeared on arXiv on August 4. The paper lists 93 authors. By the team’s own count, more than 200 scientists have joined the community, over 40 of them from OpenAI, Anthropic, Google DeepMind, and xAI. The GitHub repository has about 1.9k stars under an MIT license.

In the interview they explained why researchers at large labs joined: “They all had this problem in their own work settings, and they wanted to try whether persona agents could solve it.”

The name comes from The Matrix, with the “I” replaced by “AI”. Xiaomin called it “an idea I’ve had since I was a kid.” The GitHub README states that “the simulated world is for exploration, stress testing, and hypothesis generation, not a replacement for evidence from real people”.

3. The problem: the gap between offline scores and online feedback

Xiaomin described a pattern they kept hitting in industry: “The benchmark scores can be great, but once it ships, the feedback from real users is just different.”

Take a coding agent. The same question gets asked in very different ways. Some people write extremely detailed prompts; others are lazy and vague. Some pack ten questions into one prompt; others ask one per turn. Students ask for an explanation after every step. Some people drift from code to the news and back again. Offline benchmarks capture none of this.

The paper’s introduction puts it more formally. A novice wants explanations, small edits, and frequent confirmation; an expert wants terse responses and more autonomy. These differences shape interaction trajectories, trust in the result, and willingness to continue after a failure. Aggregate scores hide problems that specific user groups run into.

Using an LLM to play the user is not new; benchmarks such as τ-bench already do it. But the reliability of that setup has drawn specific criticism. A CMU study published in March ran 451 real people through the full τ-bench protocol and compared them against 31 LLM simulators. It found simulated users to be excessively cooperative, stylistically uniform, and lacking real frustration or ambiguity. The effect is an “easy mode” that inflates agent success rates above the human baseline.

MatrAIx targets exactly this gap: not one generic “user”, but diversity built into the simulation.

4. Technical deep dive

(I): how 8.3 billion profiles are generated

The schema. Each persona is described by 1,290 categorical dimensions in five groups.

Group	Dimensions	Representative attributes
Background	238	Age, region, language, education, family, occupation, industry
Psychology	210	Personality, values, worldview, motivation, risk tolerance
Capability	331	Domain expertise, general skills, tools, programming, developer context
Behavior & interaction	124	Preferences, habits, interaction states, work style, tech adoption
Lifestyle	387	Interests, media, culture, hobbies, sports, diet, health, fitness

Each dimension takes a value from a finite set. English proficiency runs from Native to None with CEFR levels in between; risk tolerance runs from risk-averse to risk-seeking.

In our interview they added a design detail the paper does not spell out: demographic dimensions come from public databases, but behavioral dimensions were built “scenario first.” The team mapped more than 50 AI-interaction scenarios and then defined interaction attributes for each: whether someone comments their code, their Python proficiency, whether their prompts are verbose, how many questions they ask per turn. The 1,290 dimensions are only the first version; the new one has more.

The synthetic path. Sampling each dimension independently produces impossible profiles, such as a 19-year-old retired surgeon. The paper instead builds a directed acyclic graph (DAG) over the 1,290 dimensions. An edge is added only when a data source directly reports the conditional relationship, and personas are sampled one dimension at a time in topological order. Each non-root node’s conditional distribution has the form

p(X_i = v | x_Pa(i)) ∝ π_i(v) · r_i(v; x_Pa(i)) · m_i(v; x_Pa(i))

Here πᵢ is the population-wide prior. rᵢ is a source-informed likelihood-ratio adjustment: if the primary language is English and the region is North America, “Native” gets more weight. mᵢ is a binary compatibility mask that removes contradictions such as “primary language English, English proficiency None.” An unusual but possible combination like “primary language English, proficiency Basic” is down-weighted, not removed. The point is to keep rare but real people while ruling out logically impossible ones.

The human-grounded path. The other records come from six sources: Wikipedia biographies, Amazon review histories grouped by reviewer, the Stack Overflow Developer Survey, the U.S. General Social Survey (GSS), the PRISM Alignment dataset, and 355 consented responses to MatrAIx’s own persona survey. Free text is extracted by an LLM under constraints; dimensions the evidence does not support are left null rather than imputed, and names and contact details are stripped. For example, Shakespeare: his biography, works, and tastes map into the schema, but “he probably never used a coding agent,” so that dimension stays empty.

Quality control. Human-grounded records are deduplicated with exact hashing plus MinHash; synthetic records are treated as duplicates when all 14 high-information attributes match. The public coreset of roughly one million records contains 599,847 human-grounded records (323,438 from Wikipedia, 113,120 from Stack Overflow, 97,915 from Amazon, 63,532 from GSS) and 400,000 synthetic ones.

In terms of cultural coverage, the profiles span a dozen or so countries, but “the real information sources are probably all American,” and the United States is their primary focus, both as a population and as a market.

(II): from profile to acting agent

The profile is written into the system prompt as text. Xiaomin said it is “just like your skills, a skills.md,” a textual description rather than a numeric vector, closer to a filled-in questionnaire than an embedding.

The bottleneck of this approach is the context window. Profiles grow, especially once self-evolution is added, and “at some point the context window can’t hold it.” Three directions are under exploration:

1. Routing. Load only the attributes relevant to the task. Whether someone likes spicy food probably does not affect their coding style, so it can be dropped.
2. A more compact representation, with training built for that representation, to balance efficiency against quality.
3. Steering. Push the model directly toward a specific persona’s behavior.

They also said they have found non-training methods that beat plain prompting, sitting between prompting and full training. They also plan to post-train their own model, similar to what Cursor did: it lets users plug in different vendors’ models but also ships its own Composer model. Hosting their own model costs R&D up front but should be cheaper and more flexible in the long term.

At the execution level, a trial is formalized as the tuple ⟨persona, task, agent interface, model, random seed⟩. Trials share no state, so they are parallel by construction. Each trial produces an artifact bundle that a task-owned verifier converts into structured findings. There are four environments.

A persona record becomes an agent when paired with a model, acts in one of four environments, and its trace is scored by a verifier tied to the task.

App tasks run in a Docker-based Linux desktop sandbox or on a remote macOS desktop or iOS simulator. Of the 1,010 tasks in the library, 621 are surveys, 371 are chatbot tasks, 12 are web tasks, and 6 are app tasks. The more interactive the environment, the fewer the tasks, and cost is the reason: in the paper, survey, chatbot, and web tasks used about 1,000 personas per model, while the two app tasks used 24 and 20.

(III): how do you know it acts like a person

This is the central question for the whole approach. The paper splits validation into layers, and each layer answers a different question.

Layer one: does the agent follow its assignment (persona adherence)? The team chose ten behavioral attributes and tested each in all four environments, with five personas assigned one pole and five the opposite, for 400 trials judged by an LLM on the recorded trajectory. 366 trials matched the assignment, or 91.5%.

Environment	Match rate	Strongly consistent attributes (of 10)
Surveys	92–96%	9
AI chatbot	92–96%	9
Web	92–96%	9
App	83%	6

This number measures whether the agent obeyed the persona it was given. It does not measure agreement with real human behavior. In our interview, they said persona adherence is, in their view, essentially an instruction-following capability, so a stronger base model helps here: “We’re standing on their shoulders.”

Layer two: is the extraction faithful to its source? Two LLM judges scored 1,000 extracted personas. Six human raters gave a source-matched subset of 100 a mean of 4.135 out of 5. Claude Opus 4.8’s scores fell within one point of the human mean 93.8% of the time; GPT 5.5’s did so 79.2% of the time.

Layer three: are findings stable across models? In a financial research task (OpenBB), each persona carried one of four trust orientations, from hostile to trusting. All three models ordered the four groups identically, with Cramér’s V between 0.228 and 0.363.

Agreement with real humans is the layer the paper does not cover head-on. In our interview, they said the team compares personas extracted from real profiles against the corresponding people’s actual choices, such as “A or B.” Results vary by domain. Software engineering and computer use show small gaps, because “what a person can operate, it can operate too.” Finance, health, and political elections “still have quite large gaps.” Experiences that need physical sensation, “I really have to drink it to know if I like it,” are out of reach for now. The paper’s conclusion takes the same position: human studies remain necessary before applying conclusions to real populations or consequential decisions.

A position piece on their site goes one level deeper. Its title: “To Simulate a User, You Should Make Your LLM Dumber.” The argument is that a frontier model knows too much, stays too consistent, and fails too rarely to stand in for a real person, so using it as-is cannot show where real users get stuck.

5. What happens when the base model changes

During the interview, we asked: the persona is a layer on top of a base model. When the base model changes generation, or its own personality shifts, does the simulation drift with it?

They acknowledged this is exactly the problem they are working on. If the persona is only “a shallow layer,” a change in the base changes the results, so they need methods that adapt to different base models and are training their own. They suggest that the model driving the persona is part of the evaluation configuration and should be reported with every result, and that important findings should be re-checked with more than one model before guiding product decisions.

Their blog post gives a worked example. In a case contributed by an Indian AI company, 400 simulated Indian business owners and finance leads evaluated four versions of a chartered accountant’s engagement letter. The version with a lower upfront payment and a broader liability cap raised the share requesting only minor changes from 0% to 23%. Liability terms mattered far more (+19.63 percentage points) than the payment schedule (+3.38 points). The same 400 personas were rerun on three different models; the direction of the finding held, while the exact rates differed.

Another study of theirs, MicroVerse, measures identity drift over long runs. Twenty-five agents each carry a locked “soul file” of values and moral limits into a desert grid where water is scarce. The researchers compare that fixed file against the “living journal” each agent keeps rewriting about itself. On the same map, Qwen3-32B settled at an oasis once it found water, while Claude Haiku kept relocating; 11 of 12 Haiku runs ended with no survivors. They warn that, when assigning a ruthless persona, they may be measuring the model’s built-in safety habits rather than the persona they thought they deployed.

6. Who the customers are and what they buy

Customer profile. Current customers are mainly LLM model and digital product companies: large labs building general models, and smaller labs building the best model in one vertical. The target verticals are e-commerce, gaming, and advertising. Inbound interest has come from more than ten countries, including Singapore, Japan, Argentina, Cambodia, Mexico, Switzerland, Sweden, and China. Many overseas teams are still targeting the U.S. market. They have also received requests from small businesses (such as a restaurant owner), one-person companies, and academic researchers.

What they deliberately avoid. Government work and political elections are mostly off the table. This is mainly because of the high sensitivity involved, and because outcomes there depend less on persona and can be heavily affected by black swan events, such as a shooting or an earthquake. The same applies to financial market prediction. That sets them apart from competitors such as Aaru, whose specialty is forecasting elections and group behavior.

“The feedback from frontier labs and big tech is that they don’t care that much about cost. Saving time is what matters most. They don’t want to wait ten days; they want a round in half a day or a few hours, then go change training and the harness.” That being said, they also have a cost advantage, as given in their site’s cost analysis: 10,000 surveys cost $28,560 or more with humans versus $3.60 to $104 simulated; 1,000 chats cost $8,568 or more versus $3 to $87.

They also brought up three disadvantages of human evaluation compared to their simulation:

1. Scale is not representative. A model serves a billion users, but a human evaluation can only recruit fifty to a hundred thousand.
2. Annotator distribution is skewed. They have seen cases where 80% of annotators turned out to be in one Southeast Asian country.
3. Signals are noisy. A user who stops mid-conversation may have just stepped away and come back later. Humans usually give a thumbs up or down; simulated users write out their reasons every time.

But they position themselves as a complement, not a replacement: “Iterate fast with us first, then run a final round of human eval.”

The deliverable is more than a report. The multi-turn interaction data can be used for SFT. The team also provides a reward model for RL that “includes not only task completion but each user’s own satisfaction.”

The competitive field. Simile, founded by Joon Sung Park of Stanford’s “Smallville” work, announced a $100M Series A in February and closed a $200M Series B at a $2B valuation at the end of July. Aaru’s December Series A was led by Redpoint, with part of the equity priced at a $1B valuation. Most of these companies enter through market research. Those companies lean toward qualitative research and in-depth interviews, aiming to predict a specific person or group, while MatrAIx wants “large-scale persona agent deployment.” Xiaomin added that model iteration today “doesn’t rely on changing the algorithm every two weeks”; it relies on hill climbing, finding where the strongest model still fails. “Their target may really be to predict one person. Ours is to bring in human diversity through simulation and find the model’s latent problems.”

7. Dynamic personas and the mirror world

In the new version, personas are no longer static. Mood and work state change, agents self-evolve, they have memory, and even social relationships, though the network is limited for now: “If everyone had to know ten thousand people, it wouldn’t be efficient.”

The mechanism is event-driven. The team feeds in real news, such as an earthquake in California, and lets agents in the affected region and group “react the way a person would.” Those reactions are written into each agent’s memory. Simulated time currently runs at the same speed as real time. This can be accelerated, but then “you lose a lot of real-life big events to project from.”

On October 3, NICE hosted an online workshop, “Mechanisms, Evidence and Limits of AI Self-Improvement,” with four researchers who work on this topic: Chengsong Huang (PhD student, Washington University in St. Louis), Shilong Liu (postdoc at Princeton University, joining Columbia University as an assistant professor in fall 2027), Da Yin (NeoCognition) and Cheng Qian (PhD student, UIUC). They discussed the main approaches to RSI, its current bottlenecks, how to evaluate it, and what it means for academia and industry. I summarized the discussion in the 8 takeaways below.

1. RSI improves the whole system, not just the model

When people say “self-improvement” today, what gets improved is a whole agent system. At least three parts of it can be changed.

• Weights. Training changes the model itself.
• Harness. The tools, memory and execution flow wrapped around the model. Changing it does not require retraining the model.
• Environment and data. Huang’s view is that whatever algorithm you use to optimize an agent, you need training data that is harder and closer to the downstream task, and for agents that data is the environment. A stronger model can synthesize better environments, and better environments in turn train a stronger model.

The three advance in turns. Qian calls it an “upward spiral”: components of the harness get trained into the model’s own abilities, and the stronger model then needs a new harness. Liu’s research trajectory followed the same order: first let the model build its own tools, then let it explore in environments, and finally train the model on the weakly supervised data that the exploration produced.

If all three can be changed, they should be changed together. Huang argues that co-optimizing them can never do worse than optimizing each one separately; the only cost is that the optimization itself becomes harder.

Yin offered an analogy. In snooker it is hard to say which cue is the best. Results come from a player’s long practice with their own cue, and even a top player may not reach the same level with a different one. Models and harnesses work the same way: every company has its own infrastructure and data distribution, so company A’s training recipe may not work well on company B’s harness.

There is also the question of who does the improving. Qian divides self-improvement into three levels: revising the current answer, accumulating experience across tasks, and getting better at generating and selecting improvements. Their AI for AI work sits mainly at the third level. In the past, people designed a harness or a set of skills to make the executor more capable. Now a builder AI builds the harness, writes the environments and does the debugging for a target AI, and the relationship between the two resembles that of an advisor and a PhD student.

2. What RSI can do today, and what it cannot do yet

RSI works this time because models can now run the whole improvement loop on their own, and after each change they can find out quickly and cheaply whether the change was right. Tasks where that is not possible mark its current limits: writing, which people have to judge, and scientific questions that need experiments in the physical world.

Why now? Qian gave three reasons:

1. Foundation models can handle every step of the loop on their own: reading code, proposing changes, running experiments, analyzing feedback and recording what they learned. No person is needed to connect the steps.
2. Harnesses made improvement cheap. Without retraining the model, you can change how a tool is wrapped, the memory policy or the execution flow, and test the change directly on the existing model.
3. There are more executable environments. Reinforcement learning has its gyms, and many new benchmarks are no longer “one input, one output” but sandboxes in which an agent can act freely. In environments like these a system gets more direct feedback.

Liu’s view is that a person plus a computer has always been an RSI system. The person’s bandwidth is just too narrow, and a person cannot run 24 hours a day, seven days a week. The loop only became fast once LLMs took over some of the person’s steps.

Huang pointed to the most important condition: reward has to be cheap. When his team used AI to optimize test-time scaling algorithms, the obstacle was the long wait for each reward. They then precomputed the results and turned the setup into a static simulation. Once reward came back quickly, optimizing the algorithm “became a very simple thing.” If the problem is then recast as writing code or text (code as policy), a language model can take it over.

So what can it not do yet? Liu’s assessment is that when the environment is verifiable, a model will do well given some time. The hard cases are those where results are difficult or expensive to verify, such as science and engineering problems in the physical world. Huang’s example is writing: whether a revision is better still has to be decided by people in an A/B test, each round of feedback takes a long time, and so the loop is slow. In his view, RSI has little trouble with any problem that can be written as code and gives reward easily. The bottleneck is that many real-world settings cannot be modeled that way.

For Liu, the central problem of RSI is how to get good reward from the real world. A connecting layer is still missing between today’s AI systems and real problems in work and daily life. He considers this the most important piece and also the hardest, but “once the connection is made, it can develop well on its own.”

He added that real-world problems never come with a perfect reward anyway. People also rely on many proxy tasks to filter out wrong answers layer by layer. So there is no need to reach 100% in one step; getting a little better each time is enough.

3. The environment is the overlooked knob

An agent learns from the trajectories it produces by interacting with an environment. Yet people have only optimized the agent side, while the environment side is expensive and never changes. This is the starting point of EnvHarness, a paper Huang wrote during an internship at Google.

First, the cost. The 89 environments in Terminal-Bench 2.0 each took an expert 15.9 hours on average, about 1,400 hours in total. The 33 environments in APEX-Agents are heavier: about 485 hours each, about 16,000 hours in total, or $1.6 million at $100 an hour. Training needs thousands or tens of thousands of environments, and people cannot build that many by hand.

Second, the environments do not change. Once built, an environment has one fixed difficulty: a weak model never solves it, a strong model finds it too easy, and neither learns anything. Even if the difficulty is right at the start, the environment stops being useful after a few rounds of training make the model stronger.

EnvHarness does not build new environments. It wraps a harness around an existing one, the same idea as an agent harness wrapped around a frozen model. The task itself and the verifier that decides whether the task is done stay untouched. Only four things are adjusted: the initial state, the actions, the observations and the state transitions. It has three components:

• Stage changes the initial state. Take the task “put a clean mug on the table”: hide the mug in a drawer first and the task gets harder; put the mug in the agent’s hand in advance and it gets easier.
• Contract changes the rules. Disable the shortcut action that goes straight to a location, and the agent has to learn to navigate.
• Chain links several short tasks into one long task. Finishing one does not end the episode; the agent goes on to the next.

The harness is written by another agent, called EnvRigger. It reads the trajectories the policy produced, finds systematic weaknesses, writes a piece of harness code, and then has the policy run a new batch of trajectories in the modified environment to validate the change. If the task has become unsolvable, or no longer poses any challenge, the change is rejected.

The talk included an example from SWE-bench. The policy kept submitting patches without running the failing test, so EnvRigger wrote a fake pre-commit hook that blocks such submissions. From this the policy learned a skill: run the tests both before and after a change. In this experiment EnvRigger and the policy being trained used the same model, so the gain was not distilled from a stronger model.

4. The biggest technical bottleneck is attribution

The middle step of self-improvement is diagnosis: see a failure, find its cause, and propose a change that can be tested. Qian considers this the main bottleneck right now.

The difficulty is that the same surface symptom can have entirely different causes. A failure may come from reasoning, from a tool interface, from missing information or interference from the memory module, or even from a misreading of the training objective, and each calls for a different intervention. Until now this step has relied on human experience.

The weakness of today’s models is that they work bottom-up. They look at what went wrong case by case and then patch each case. A person works top-down: first judge where the overall approach is flawed, then verify. Attribution by patching one case at a time overfits easily.

Qian suggested two directions.

• Diagnosis needs evidence. Look at intermediate states, at the gap between the expected result and the actual feedback, and at actual resource use, and infer the cause from these. The system cannot be treated as a black box.
• Attribution needs controlled experiments. As in an ablation study in a paper, replace or remove one of the model, the tools, the memory or the execution flow, or replay the run, and see how the result changes. Today this work is done mostly by people.

His conclusion: what the system needs to be taught is not a trick for some benchmark but a more abstract way of thinking, namely how to test a hypothesis.

EnvRigger, described above, can be seen as a small-scale implementation of this idea. It looks for systematic weaknesses across many trajectories instead of targeting a single case.

5. Gains from self-iteration saturate quickly, and a higher score does not mean more capability

Qian described a pattern from their experiments. They had a model improve its own harness repeatedly and measured the effect after each round. On the held-out set, the score rose for the first few rounds and stopped rising by the fourth or fifth. So more iteration does not mean better results. But the model cannot tell when to stop. In Qian’s words, cost control “may simply not be on its mind”: when the results stop improving, it keeps adding rounds. Optimization that keeps polishing small details overfits more and more, and in the end the score can even drop.

People work differently. They try an idea on a small portion of the data first and commit large-scale resources only after it proves effective. That small portion should also be as diverse as possible, so that it exposes as many problems as possible. Qian thinks this staged allocation of resources is exactly what systems need to learn.

Huang added that RSI easily overfits to the task at hand. What it finds may be a shortcut, or even a way to hack the task outright. So one has to tell apart an agent that has really become stronger from one that has only become better at this kind of problem.

An audience member asked whether self-improvement that cannot loop forever still counts as RSI. Huang answered that a model has a finite number of parameters, so its ability must have a ceiling, and “hoping that RSI can improve without limit is unrealistic.” What matters is how fast a system approaches the ceiling, and whether it can sense how far it is from saturation so that it can stop early.

Qian added that the number of iterations should not be a threshold in the definition. A loop stops when the budget runs out or when marginal gains shrink. What matters is whether the overall direction is upward.

6. A train/test split is not enough to evaluate RSI

For a system that finds shortcuts on its own, setting aside a test set does not prevent data leakage or reward hacking. How should it be evaluated, then? Each of the four speakers offered an approach.

Use one-time evaluations tied to a point in time. This is Huang’s proposal. One kind is prediction: put the old and new versions of a model into a real market at the same time, or have them predict future events. Nobody knows the answers in advance, so nothing can leak. The other kind is human A/B testing. The price is high cost and slow results, which means an RSI algorithm cannot be measured online.

Or step back and use fully controlled synthetic environments. The compromise Liu described is to build your own environments and fill them with synthetic data the model has never seen. But he thinks evaluation ultimately has to land in real workflows such as finance, law and medicine, where people have already defined many metrics.

Environments should be realistic, controllable and dynamic. These are Qian’s three criteria. Controllable means, first of all, reproducible: the larger the environment, the less stable the results when different people test the same model. Because RSI runs for many rounds, controllable also means that the gain from each round can be measured separately. Dynamic means that the environment reveals constraints or user intent step by step, which tests how a model adapts when results differ from what it expected. Realistic and controllable pull against each other, however: real-world signals are messy and hard to turn into a structured environment, while an environment built entirely by hand or by an LLM looks fake.

Decide first which learning challenge you want to test. Yin argues that an RSI benchmark cannot be just a subset of an older benchmark. It needs a learning challenge of its own, and efficiency and cost should be part of the metrics. When his team built ApprenticeBench, preventing reward hacking still depended on a lot of people reading agent trajectories. He thinks that detecting this behavior at scale could be a benchmark in itself.

The host, Dawei Li, added an observation: these problems closely resemble the ones met earlier when evaluating the reasoning ability of large models, namely memorization, data leakage and overfitting to one domain. Lessons from that work, such as continuously updated evaluations and synthetic data, can be borrowed here.

7. Efficiency is an underrated metric

Yin stressed repeatedly that most RSI research today focuses on frontier capability, but in deployment it turns out that agents do not get more practiced the more they work.

He looks at RSI in the setting of enterprise deployment. A new hire reads the documentation, learns the software, practices hands-on and gets feedback from a manager, and eventually becomes proficient. For AI to enter a company’s workflow, it has to go through the same process. ApprenticeBench, recently released by his company NeoCognition, tests exactly this. The job it chose is construction finance accounting, an ordinary white-collar accounting role that does not need a particularly smart model.

People are also unpracticed at first. But after doing the same kind of task dozens or hundreds of times, they always get faster, moving from System 2, which requires thinking, to System 1, which does not. In their evaluation, models showed no such shift at all. As he put it: “It makes no sense that after an agent has done 100 tasks, the time and cost it spends are about the same as before, or even higher than at the start.”

He proposed three directions:

• Index and organize learned experience better, so that it can be retrieved quickly when a similar situation comes up.
• Turn the repeated procedures in the work into reusable skills or tools.
• Learn the job with a frontier model first, then compress the workflow knowledge it gained into a smaller, more specialized model.

This is not only an enterprise problem. Yin said that when training one generation of models after another, people likewise want to distill the reusable knowledge from earlier iterations so that later ones need fewer experiments. Questions of efficiency and cost like these happen to be a direction that academia can afford to work on and that is still at the frontier.

8. The human role moves up a level

Once AI takes over solving problems that are already well defined, the value of people lies in finding the problem and defining it clearly.

Liu’s view is that people have always been moving up one layer of abstraction at a time: from assembly to C++ and Python, and then to neural network frameworks. In the early days everyone still wrote gradient descent by hand. The lower layers still matter, but most people can put their effort where it is closer to applications and creates more value. This is not necessarily a bad change.

Huang has reviewed papers written by a fully automated research system. His impression is that the defining feature of a purely AI-generated paper is that it “uses experiments to prove something that is not necessarily an important question.” AI is very good at incremental improvement on well-defined problems, such as pushing a benchmark score a little higher.

Real research is more about discovering a problem and then defining it clearly: what the input is, what the output is, and what the metric is. The rest can be handed to AI. “We have gone from being a PhD student to being a PI, from someone who writes code to a project manager, from someone who solves problems to someone who finds them.”

Qian said that human insight should go into the meta level. The question used to be how to be a good PhD student. Now the question is how to be a good advisor: how to give downstream agents a more stable harness, environment and feedback mechanism, so that their self-iteration can keep running reliably.

(The original version by the author is in Chinese without using any LLM, but for English version, the author use Claude to do the translation and then manually edited and rewrite some of them that she thought inaccurate.)

Lately a lot of younger students have been asking me whether they should do a PhD, and what research area they should work in. I can’t give anyone advice, and I don’t want to meddle in any one person’s life. So I’ll just share some fairly general thoughts.

Some people are lucky through the entire PhD journey. They go to a great school for undergrad, start research early, and meet nothing but kind senior students and good professors along the way. When they apply to PhD programs, every school extends an olive branch. During the PhD they once again land great advisors and collaborators, intern somewhere new every year, maybe even graduate early, and when they finish, just as when they applied their PhD four or five years earlier, they collect top offers of every kind from both industry and academia. Meanwhile, other PhD might switch advisors many times, switch topics many times, finally meet a good advisor (or not), and barely scrape through graduation, only to discover on the job market that apart from the PhD title and a higher volume of mass emails from recruiters, nothing distinguishes them from others with only bachelor’s or master’s degrees. Life may even be worse off. By the age at which you finish a PhD, whether or not you went through the kind of mental and physical trauma during the PhD that leaves you with ‘permanent head damage,’ your memory and your test-taking ability will be far below what they were in undergrad.

When, after enormous struggle and one rejection letter after another, Lady Luck finally smiles on you and you get an offer from some company, the salary HR quotes makes you question your past. What you’ll earn is about the same as a bachelor’s or master’s graduate. Compared with your peers, it may be a third of what an above-average PhD graduating that year makes, and a fifth or a tenth of what the top PhDs make (the gap with the very top outliers is much larger). And even this offer you only got because of luck.

To be honest, the not-finding-a-job scenario above happens to PhDs, master’s students, and undergrads alike. But when you’ve already invested so much time in a PhD, facing the same outcome (i.e., landing a job you’re not very happy with), you as a PhD may need more mental effort to talk yourself into accepting reality. What makes regret so powerful is that it doesn’t show up when your life is smooth. But when you’re at a low point, some decision from your past, however long ago, constantly resurfaces, making you ask yourself over and over, “If I had chosen to xxx back then, would things be better?”

But this is not a “don’t do a PhD” post. Most decision making in life has many dimensions, yet most decisions don’t matter that much, because whenever you gain something, you lose others along the way. This is also why I can’t give anyone advice: everyone’s personality is different, the way they handle things is different, and what they hold precious is different. So even if two people make the same decision, the trajectory afterwards is hard to predict. Some random event may affect the trajectory of your life more than all the time you spend weighing pros and cons to make a “good” decision.

Whether it comes after a bachelor’s, a master’s, or a PhD, the transition from student at school to adult at society is hard. You discover that life contains many things to learn besides textbooks (intimate relationships, for example). You gradually want to stop depending on your parents financially, and to discover and shape an independent self, both financially and in character. You start meeting all sorts of people with different agendas, and you start seeing conflicts of interest you never saw in school, along with the diversity of human behavior when interests collide. In a purely exam-oriented high school, where everyone just keeps their head down and studies, you don’t really see conflicts of interest (at a high school with a more well-rounded education, perhaps students get exposed early to something closer to the real world). In college, if you don’t do research, you’ll also rarely run into them (maybe the occasional small friction with a teammate on a course assignment). Of course, at universities in China there are various scholarships, the grad school application, and so on, where all those outstanding, seemingly “well-rounded” students will take every means to show that they are better than you. (As there is not enough resources but too many people who want them).

If you start doing research as an undergrad, you may also have conflicts of interest with your advisor or your project collaborators. At universities in China, the faculty evaluation system is very poorly designed (in my view, a well-designed mechanism should incentivize cooperation, rather than incentivize the stronger party in the game to hurt the weaker one). Professors in China are evaluated by papers where they are first author, which gives them plenty of motivation to grab first authorship from their students. Why do I use the word “grab”? Because in this setting the two sides are unequal in many respects: experience, connections, social reputation, and skill at PUA. (PUA originally stood for “pickup artist,” but in Chinese internet usage it has come to mean something closer to gaslighting: the systematic psychological manipulation of someone by a person with power over them, whether an advisor, a boss, or a partner. That’s the sense I’ll use throughout.) On top of that, students who are just started to learn to do research may hold an unrealistic fantasy about academia, believing that all the researchers are noble and great. Or maybe it was only me who thought so. After all, before college I spent most of my time immersed in my own world, and my only ways of interacting with the outside were what my Chinese teacher said in class and reading top-scoring college entrance exam essays. A flower in a greenhouse, if it doesn’t want to stay inside its whole life, has to go through the wind, rain, and scorching heat outside before it knows how to survive in a new environment.

Under the kind of unequal student–advisor relationship described above, it’s hard to evaluate ourselves fairly, or to see others clearly. I remember the first semester of my junior year, lying awake night after night over something that was not my fault, and writing a long apology letter to a professor at my undergrad school. Even though I was quite certain that the person who should be apologizing was not me, the inequality of the relationship meant all I could do was apologize and hope he would forgive me for the word I said before (Curious what I said? I just said that I feel that my paper is trash. Somehow he is super offended because he took so much pride in getting papers into some “top” conferences such as AAAI/IJCAI. And everyone who work with him has to hold the believe that he had. ). I thought: if a long page of self-reflection and apology could buy back four or five months of my hard work, it would be worth it. Whether it was my fault, whether I should be the one apologizing , none of these matters? (Junior year is a crucial period for applying PhD, because applications begin very soon and I need to have some research project to talk about and also a recommendation letter.) In the end, that professor did not accept my apology. From then on, I knew that to most people in the world, my dignity and self-esteem worth nothing. And through round after round of reflection (I might reflect too much but I really like to chewing on things), I came to understand that, compared to “four or five months of effort and time get wasted,” what I feared more in losing was my own dignity. Since then, I have not ever again traded my dignity for anything. If I apologize, it is because I believe I truly did something wrong, not because I need to please someone.

Half a year later, I started to work with professors outside my undergrad university, and my research started making good progress, though I would still feel tense and anxious whenever an email I get email from that professor that I initially worked with. Two more years after that, I had completely left the small circle of my undergrad and seen a bigger world. I had almost forgotten these traumas, and seeing that person’s name no longer triggered any PTSD. I don’t talk about these things with others anymore. Before, I would talk about these stuff over and over to walk myself out through. But once I truly walked out, these people, and the shadow they cast on me, disappeared completely from my memory, my life, and my thoughts. Driving them out of my life entirely, to the point of never thinking of them at all — that is probably the best form of “hatred” I would give. Although most of the professors I worked with at my undergrad were terrible, I also met one very, very kind mentor. He showed me that we can keep some innocence and kindness and still adapt well in our society, provided that we understand the rules of the world’s game rather than deceiving ourselves by living inside an imagined rainbow bubble. (I used to wonder how much bad luck I had to spend in exchange for such good luck to meet and learn from such a wonderful person. Getting the chance of knowing and learning from him (by accident) affected my life trajectory far more than all my deliberate decision making.)

During my undergrad, I would very naively sort people into “nice” and “not nice,” and decide on that basis whether to be friends with them. Later, when I become a bit more mature, I found that humans are super complex in nature and can’t be simply classified like this, so there is no absolute “good person.” When I see a “flawless” “good person,” either they’ve only blind me with their good side, or we haven’t known each other long enough that I actually don’t know them as much. So I stopped trying to be a “good person” that I thought the outside would want me to be, because the concept itself no longer exists in my dictionary. This may be one of the changes from my PhD years that makes me happiest. Until the end of my first year, I was trying hard to be a “good person” in other people’s eyes: I smiled at everyone, never refused any request from anyone, and constantly self-reflecting (those masters of gaslighting didn’t even need to make the effort; I would just voluntarily do that to myself). These “creed of how to be a good person,” built into my education from my family, were to a large extent responsible for the discomfort and pain I felt when I first entered society. In a simulated world, if no player deviates from these rules, the total utility of all players is high. But some players will always discover that by deviating from the above principle, they can gain more. And the current rules of the game implement almost no penalty for deviation. Hence the phenomenon we all know so well: manipulation and gaslighting, or what the Chinese internet calls “PUA.”

Sometimes we get manipulated because we love being a “good person” in other people’s eyes a little too much. Once we can see ourselves more objectively and have the courage to be disliked, it becomes much harder to manipulate us. Of course, everything goes wrong at the extreme: being “good” to the extreme can get you exploited by other players. “Reflecting appropriately and admitting your mistakes” is humility, but humility overdone, taken to the other extreme, makes you easier to manipulate. Since the readers my writing attracts are mostly the over-reflecting type, my no-brainer advice is: look for other people’s faults more (of course, see your own too, but don’t take all the blame on yourself). A typical example: when a project fails, some advisors will put the blame entirely on the student. But if the student can look at it objectively, they’ll find that there are many other factors in the pipeline, such as the advisor’s idea simply not working, or collaborators not pulling their weight or even making things worse. A project’s failure is more or less partly the student’s doing, but if it is “entirely” the student’s fault, that definitely doesn’t make sense.

Last of all, I want to talk about some reasons to do a PhD. Many people will tell you how much they grew during their PhD: they got to witness the full biodiversity of the human species, they learned to take setbacks more lightly, and so on. But you can get these from either a PhD or a job. The difference is that in a PhD you may be manipulated by your professor, and at work you may be manipulated by your boss (either way, someone will be there to teach you lessons you pay dearly for).

The reasons I can think of for doing a PhD are:

(1) Practically speaking, a PhD may give you five more years of buffer before the job hunt, and in those five years you accumulate connections, resources, and abilities, so you can look for a job with more composure. Of course, it may also look like you wasted five years and have nothing to prove that you have really lived through them. But that’s life: even with the same starting point and the same initial choice, people end up in very different places. So from this angle, compared with luck, choices are not that important.

(2) “If you want to be a professor someday, you would need a PhD.” This is one of the main reasons people consider doing a PhD. And there are also people with a real talent for research, or those who choose to do a PhD out of love.

(3) However hard I try to think about the meaning of life, the only answer I ever arrive at is “my life has no meaning.” I can’t set myself a goal the way others do, because all I can see is the death at the end of life. Many people work hard to locally optimize every decision in their lives, hoping to reach their “life goals” sooner and faster, to fight against fate. But no matter how many times they win against fate, one day they will be dragged by time and fate toward the same destination. So even if not doing a PhD looks like the path better suited to this local optimization (financially and career-wise, it may be much better than doing one), if your life is not aimed at that optimization, but rather at “experiencing life” on this journey where we are born heading toward death, a PhD may be a pretty good choice. Whether or not you do a PhD, you will work afterwards; a PhD just means you start working five years later. Seen this way, your life will always include the experience of “working,” but if you don’t do a PhD, you may never have the experience of “doing a PhD.” I think people are willing to pay for experiences, whether it’s visiting an exhibition or traveling somewhere. These experiences all carry some cost in time and money. A PhD is the same. If your life is not about optimizing some concrete real-world goal, then doing a PhD could be just for “knowing how is it like to be a PhD”. And choosing to do one is choosing to pay five years’ worth of “time and salary gap” as an “experience fee” in exchange for the experience of “doing a PhD.”

(The original version by the author is in Chinese without using any LLM, but for English version, the author use Claude to do the translation and then manually edited and rewrite some of them that she thought inaccurate.)

The narrative parts of this installment were all written on the flight back to China in mid-August; I added the more reflective bits at the beginning and the end and then put it up. If I had waited any longer, I would have lost even more of the feeling for my own words. Nothing in this “My PhD Grind” series is necessarily something the PhD itself brought about. Rather, I’m at a certain stage of life — a stage in which I’ve left the environment, culture, and education I’d known for over twenty years, begun to have my own independent thoughts and ways of thinking, and, in a new environment and culture, come into contact with many more different people and different things — and the changes in perception and thinking that arise naturally from that. Even if I weren’t doing a PhD, I would probably have gone through similar things and had similar realizations and reflections. It’s just that these people, these events, and these realizations happened to fall within the years of my PhD, so I use “PhD grind” to describe this stage of life. But I don’t think doing a PhD is the only way a person can grow, nor do I think I should “be grateful for the pain.” I try my best not to disturb other people’s lives, and I don’t want to intervene in anyone else’s trajectory, because I believe opportunity matters enormously, yet opportunities usually aren’t something you get to choose — and as for what’s left, the things you can choose, no matter how hard you rack your brains over them, they won’t make much difference to your life. No choice or experience is indispensable. So, I have no opinion. I don’t write these things in order to be understood, nor to “give advice.” Text is a medium for transmitting information: by the time I decode the continuous embedding of what I’m thinking into tokens, a great deal of information has already been lost; when a reader reads in those tokens and converts them into their own embedding, they add a great deal of their own context, and by then their embedding is nothing like mine. Every reader might be able to read something different out of the same passage, but more or less, they’ve seen themselves in it. I was surprised that my first post — the blog about many of my friends who works on theory switching to LLMs — resonated with so many people. Friends messaged me privately to say “they felt the same way,” or “they could really understand me.” But in fact, I wrote it from the perspective of a detached narrator, an onlooker, putting in almost no emotion at all. The deep feeling readers sensed in those words must have come from reading themselves into the text and projecting their own emotions onto it.

On August 14, 2023, I went to the United States for the first time. On the flight from New York to Ithaca, before we departed, the flight attendant proudly announced to everyone: “All the crew members, two pilots and two stewardesses, are women — isn’t this awesome?” Although, like everyone else, I didn’t respond, my heart was secretly leaping. This was my first impression of America: a place where anything is possible. (Because of how I was raised, my understanding was still stuck at “In China, only the flight attendants can be women; the pilots are always men.” Though when I looked it up just now while writing this, China does have female pilots — women make up about 1% of civil aviation pilot license holders, compared to over 10% in the U.S. There’s a lot of finer-grained statistics by category, but in almost every category, the proportion of women in the U.S. is nearly ten times that in China. And since the total number of licensed pilots in the U.S. is ten times China’s, in absolute numbers there are roughly 100,000 women among all licensed pilots in the U.S. versus about 1,000 in China — a hundredfold gap.)

One afternoon during grad school orientation week, walking home from Gates Hall, I passed the football field, where “Don’t Stop Believin’” was playing, and my eyes instantly welled up: in the months before coming to Cornell, I happened to be watching Glee (an American TV series), and both the show and that song mean a great deal to me. Against that backdrop of unreality, for those few days I felt like I was living inside a TV show, full of anticipation for the future.

That glamorous filter lasted less than two weeks; real life quickly yanked me out of the TV show. I — who had barely any experience of the real world, who had unreasonable optimistic expectations of the new environment and the people in it, whose personality was so weak that whenever any conflict arose I assumed “it must be my fault” — ran into a very mean roommate and had to move three times in my first three months. (Later, about a month after I moved out, that roommate’s behavior became so outrageous that the third roommate couldn’t take it anymore and eventually drove the her out.)

On top of that, the Chinese students in my cohort were almost all guys (there may have been one or two Chinese girls, but I never saw them around, and besides, not everyone gets along with everyone; as for people from other countries, there are obstacle in both culture and language so I didn’t really hang out that much with them). Watching the guys quickly grow close with each other, all I felt was loneliness. In October, the weather gradually turned bad, and once the clocks fell back to winter time, it was dark by 4 p.m. The weather and the troubles in my life piled on top of each other, and I gradually became a bit depressed (though compared to the depression of my second year, the depression in my first year should be consindered as mild). By December, I still hadn’t made any friends. Over the Christmas break the campus was nearly empty — the loneliest time to be in Ithaca, and the time my depression was at its worst. I would reflect on and chew over everything that had happened in my four months in the US, again and again, and ask myself over and over: why did it turn out this way? Although I never want to go through that pain again, looking back now, without those painful times I wouldn’t have come to understand myself better, and I wouldn’t have become who I am now.

After the spring semester started, the weather got better, and my mood lightened along with it. It was right around then that I got to know a more senior student at another department, and we’d play ping-pong together every so often. By April, we had gradually reached the point where we could talk about anything, and she became the first female friend I made after coming to the US. In my first six months, I had tried to become friends with lots of Chinese (and mostly, male), forcing myself out of my comfort zone, but all that socializing left me extremely anxious and dejected. It was only after becoming good friends with her that I learned what healthy socializing should look like. Maybe all I needed was one friend to stop feeling like there was “something wrong with me.” After we became good friends, I stopped trying, as I had before, to befriend people who triggered my social anxiety, because I knew that good socializing shouldn’t make me anxious. I also gained other friends around the same time: a Sri Lankan girl in my cohort, who is kind, geninue, and full of love for life, and a master’s student in economics (a sports buddy I could call up anytime to play sport such as pingpang, badminton, tennis, chatting about anything and everything while we played). April and May were probably the happiest stretch of my first year in America, apart from that first week — it felt like I was bathed in sunshine every day (both physically and mentally). In late May, I went back to China. The day before I left, I looked back over my nearly a year in America. Although the happy times amounted to only the month or so before I left, my personality had already changed a lot: I no longer tried so hard to please people to win their friendship, I was no longer afraid of being disliked, and I had become far more selective about making friends rather than expecting to be friends with everyone. That day, I re-downloaded QQ (a Chinese messaging app) and messaged a classmate back in college: “I feel like everyone hated me in undergrad. Can you tell me why?” I asked him because he was probably the only person I could remember who “don’t hate me that much.” I’m not sure what state of mind I was in — maybe it was out of curiosity, maybe I was in such a good mood that nothing he could say would affect me, or maybe I just wanted to confirm whether my original “feeling that everyone hated me in undergrad” was only me being oversensitive, something I’d imagined. I’ve forgotten the details of our conversation. After he told me to “think it over myself” and I couldn’t come up with anything no matter how hard I tried, he told me it was “because people felt I didn’t fit in with the group.” From that day on, I can finally let it go. That day, I uninstalled QQ and truly said goodbye to undergrad. Actually, since coming to America, I have never again felt “weird.” I found that everyone is pretty different, and sometimes I even worry that I’m not “weird” enough — that compared to others (especially some of my American classmates), that I don’t have anything distinctive about me.

That summer I went back to China to renew my visa. Although I hadn’t been home in two years, I didn’t feel much nostalgia for life in China — the high-rises, the dense crowds, and the traffic still made me very uncomfortable (I constantly felt that one careless moment and I’d be hit by a bicycle, an e-bike, or something bigger, because everyone is in a hurry and no one yields to pedestrians). It was my first time living in Beijing not as a student, and I met all sorts of people. Beijing doesn’t have the comfort of life as in Chengdu; Haidian District and Tsinghua were more run-down than I’d imagined; and the people I chatted with while renting a place, along with the Didi (ride-hailing) drivers taking me home at night, made me feel everywhere the hardship of the “Beijing drifters” — those who leave their hometowns to make a living in the capital. There were also plenty of things worth being happy about in between. For example, a girl I’d met at Cornell and I arranged to spend a day together in Beijing; we got to know each other better and discovered how alike we were. On the day we parted in Beijing, we chatted our ideal of a good friendship to be “close but independent” — a healthy friendship, one in which both parties can grow, should be like that. Afterwards, we constantly catched up with each other on the phone often and became very, very good friends. Later still, when I moved to the Bay Area, she became my only friend there. (The Bay Area seems even harder to make friends in than school.)

That summer I also went to ICML and ACL, saw some old friends, and met one or two people I’m still close with today. It was a pretty decent summer. All of this was a long, long time ago. If I didn’t make an effort to remember, I’d almost forget I once had so many problems (though even now I still occasionally hide from people I know when I run into them since sometimes I have nothing to say and become ackward if I have to start some small talk). Those are all the stories in my first year. There doesn’t seem to be anything PhD-related worth recording, but after a year, I no longer felt so broken: beyond the social problems mentioned above, my personality changed in other positive ways too — for example, I became more confident, I stopped feeling that I didn’t deserve to be at Cornell, and I began to have a strong sense of belonging.

Looking back, the first year was a fairly beautiful one. Whatever the winter had been like, when summer arrived in April and May, I didn’t feel a trace of sorrow. It was in the following two years that I truly entered the grind of the PhD itself, and lost many things I consider important — like the ability to be happy. It took me far too long to recover. Going by my earlier expectations: I thought I’d be happy once I switched advisors; I thought I’d be happy once the project was over; I thought I’d be happy once the paper got accepted; I thought I’d be happy once summer came; I thought I’d be happy once I left Ithaca for the Bay Area; I thought I’d be happy once the internship ended; I thought I’d be happy once I started the new internship; I thought I’d be happy once I didn’t have to do research anymore; I thought I’d be happy once I found a job…

Looking forward to one milestone after another like this, two whole years went by. Occasionally I would suddenly feel happy, but that happiness never lasted more than a week.

I had never waited for such a long time like this before, for two whole years — far, far too long. Hoping again and again, disappointed again and again, I gradually began to accept the state I was in. And as the happy self of two years ago receded further and further, in my second year I would still occasionally struggle, wondering how to find my old self again; by my third year, I had grown used to the self that was no longer happy. (I started to think that maybe this unhappiness wasn’t caused by the PhD, but is the inevitable destination once a person reaches a certain age and has seen a huge amount of people and things.)

Although this ending is a bit sad — and the next several posts will all be sad in tone — to spare readers too much sadness, I need to give a spoiler alert here: in August, I found a job and went back to China to rest for a while. In the process of cleaning up a past stage of life and getting ready to welcome a new journey, I also threw away a lot of things that needed throwing away (like some of the trauma and pain). The best way to find yourself again is to change your environment. Moving from Ithaca to the Bay Area didn’t seem to help much, since I was under job-search pressure at the time. But after finding a job in August, I went back to China to rest — no mental pressure, a changed environment, and a new life to look forward to — and I felt a lightness and joy I hadn’t felt in a very long time. With all these factors stacked together, I found my happy, pre-PhD self again (though I don’t know how long this happy state will last, while I’m happy now, I plan to enjoy it while I can.

Compared to before the PhD, the me of today is more mature, more emotionally stable, and understands more deeply about the meaning of life and how the world works. I’m also better able to find, in such a complex real world RL env, a way of existence that at least for now feels self-consistent, and to continue, with a more peaceful attitude, the rollout along the short trajectory of my life.

(The original version by the author is in Chinese without using any LLM, but for English version, the author use Claude to do the translation and then manually edited and rewrite some of them that she thought inaccurate.)

(Note: In order to preserve the privacy of the people that I mentioned, I didn’t use their real name, but instead, using the name “Jack”, “Frank”, “Leo” instead.)

I had originally wanted to give this the structure of a perfect book: a prelude, three distinct chapters (not for the sake of literary completeness, but because my PhD really did come in stages, almost neatly divisible by year), and an epilogue. So after jotting down some loose notes on the flight back to China, I spent a long time thinking about how to structure what I’d written, how to shape it into the perfect, complete form I had in mind before presenting it to readers. The result: three weeks went by, and the draft I wrote on the plane was never finished.

One day in September, while chatting with a friend, I was surprised to learn that he had switched from theory to LLMs. A rush of feelings came over me and I wanted to write something. What I wanted to write belonged, properly speaking, in my PhD memoir—but the memoir had never gotten off the ground, precisely because of my pursuit of perfection. I don’t know how I picked up this habit, but whenever I present something imperfect to readers, I feel I have to apologize first and honestly list everything I think is wrong with it. (Seen this way, I’m really not cut out for academia or startups. Though the people and events of my PhD have also taught me to be a little less harsh on myself, and a little more forgiving.) So I’ll just write freely, wherever my thoughts happen to go. Reality is, after all, imperfect and incomplete; if I tried to write this like a work of literature, with the reader’s experience in mind, I would be sacrificing truthfulness and spontaneity—the feelings I have while writing, and the flow of my state of mind. Besides, works of literature have storylines, and reality doesn’t. I’ve become more and more able to accept the chaos of the world, and my own chaos within it (the flattering term for which is “free-spirited”).

That day, I happened to open an article on Zhihu and noticed that the account of a friend, Leo, had liked it. So I messaged Leo on WeChat—we hadn’t seen each other in a long time, and I wanted to catch up and ask how he’d been. He told me he was planning to switch to working on LLMs, and that over the past year he had moved to a new place and was talking to people even less than before. Even though we hadn’t been in touch for over a year, chatting with him still felt familiar. Back in 2023, when we were applying for PhDs, we were both working on theory. But Leo socialized very little and rarely reached out to anyone to hang out (he spent most of his time doing research and reading papers), though when we ran into each other now and then I’d chat with him a bit. So I wouldn’t say I knew him well enough to call us “close.” And yet he came to my PhD A exam last year. (Not many people came since I hadn’t made any special announcement—it was only visible in a mass email that the grad school requires us to send to all the PhD student in a certain format. He’s also not the kind of person who likes going to presentations, especially when the research I was presenting had nothing whatsoever to do with what he was interested in. So the fact that he noticed, and made time to show his support, surprised me.) Leo’s PhD path has been extremely rocky. Given his hermit-like style, he probably wouldn’t want me telling other people too much about it, so I’ll leave that part out. But even though his PhD hasn’t gone smoothly, he has always been so calm, so unruffled.

Every time I talk with people around me (especially the guys) and learn what they’ve been through, I think: “Why am I so bad at enduring hardship? The smallest thing happens to me (compared to what they have been through) and I start kicking up a fuss, as if I want the whole world to know how much pain I’m in and how hard things are—while they, facing far more and far harder things, keep a straight face, placid as still water.” Of course, this kind of self-gaslighting usually passes in an instant. I tend to look at things from an all-around, complete perspective, so everything gets considered from multiple sides and angles, and I freely adjust the weight I give to each. The upside is that I never become too stubborn or opinionated; the downside is that sometimes people feel I have no personality of my own, that I just agree with everything. (I’ve just wandered off topic again…)

In fall 2024, I was considering quitting, and Leo was in the middle of switching and finding new advisors. I half-jokingly said, “Why don’t you quit with me? You wouldn’t even have to worry, like I do, about how quitting would negatively affect your advisor’s reputation. Or, stop doing theory and switch to an advisor who does applied work?” (In a CS department, if Leo didn’t care about his research direction, finding an advisor would be much easier.) But he remained very firm, and firmly described to me the career path he had in mind. He framed his choice to do a PhD as “so that I can go into quant after graduation and get a very high salary.” I didn’t quite follow his logic, because from my perspective, he could go into quant companies even without a PhD, or without doing theory. But Leo described it as though it were the only path. I just smiled, because from what I knew of him, he was the unpretentious kind of scholar, and the “high salary” of quant would hardly be enough to make him choose, so resolutely, a path that looked so much harder. (Note that I added the qualifier “unpretentious” here, because not every scholar can be described that way—and this was precisely the mistake I made when I first entered academia. But nothing is absolute; back then I could only see one side of people and things, and I blindly passed judgment on them based on what I saw. That was my own problem.)

Perhaps it’s Leo’s “hermit” style that lets him hear less of the world’s noise, and steadfastly do what he likes, what makes him happy (doing theory, writing formulas, and reading proofs really are a joy for some people!). In my first year of the PhD, I too agonized for a long time over whether to give up theory. Even back when I started doing theory as an undergrad, my undergrad advisor kept telling me that doing theoretical research has a lot of downsides, such as it is hard to recruit students, hard to get funding, and students are hard to find a job… But at that time, working are theory made me feel calm and happy, so none of those practical considerations mattered. I had hoped I could do both theory and application (so in my first year I was in fact doing both at once). On the one hand, I couldn’t give up theory, because it made me feel at peace; on the other hand, I couldn’t give up NLP, because coding had been my weak spot for the past several years, and if I abandoned applied work entirely, it would become my weak spot once again. (And with that would come a lack of confidence, the kind that keeps me from seeing many people and things from an objective, equal footing, but admire people who can code.) I struggled for an entire year since I don’t want to give up any of these two, and in the end, because of a project I didn’t much like, I finally decided to stop working on theory. Looking back now, I ought to thank this twist of fate, because I was no longer distracted the way I used to be. Before, even when doing theory and applied projects side by side, I would always prioritize theory, because it made me happy. I didn’t care whether I was doing it well enough, didn’t care that a simple proof took me ages to understand, didn’t care whether my brain or my level of effort were suited to it. I only cared about the happiness of the moment, and about that slow, unhurried life of “one pen, one book, one proof to stare at for days.” (Even though some of the peers I know, could probably get through it in 30 minutes, lol.)

When my second year began and I decided to give up theory completely and focus on LLMs and NLP, it was painful at the time, because I didn’t like living at such a fast pace, and I didn’t like writing code. I wanted my tools to be pen and paper; I wanted the computer to be something I used for reading proofs, not for writing code and running experiments. I still tried hard to steer clear of the research directions that looked very popular, but I really could no longer get the joy of writing proofs, nor the calm I’d had before. Now, two years later, I’ve grown used to my current identity and come to accept it—I can’t even understand anymore why my past self was so afraid of giving up theory, so afraid of working on a popular direction. Back then I simply just being too young too naive; every little thing seemed like a big deal, and I didn’t dare try anything outside of what I had mentally prepared myself for. Now I’ve become more nihilistic: once there are more things you don’t care about, you’re less afraid of losing them. And once you’ve seen enough data, you know that people can always adapt, little by little; you don’t need to keep yourself in the most comfortable environment all the time, and you don’t need to confine yourself to a box. (Writing blogs is the new thing that makes me feel calm. Writing my thoughts and experience down is actually quite a lot like writing proofs, except that when I wrote a proof, I felt I was presenting a perfect piece of work, and the recognition of those around me made me sure of it; when I write prose, my thoughts often drift far off, so, objectively speaking, I can’t produce a piece of work that would be called good by any objective standard. But one day, I suddenly realized that the people and the world around me are more accepting than I am to my work—things don’t have to be perfect before you can show them to others. (I tried a few more times after that and received a lot of positive feedback.) I didn’t post these writings on English social media; but I quietly put them on my homepage. (For Chinese versions I occasionally post on Chinese social media, hoping strangers with no connection to my life will read them) One day, I received an email in from a stranger who doesn’t speak Chinese, saying she really liked my blog, and that she can relate to what I write even though we have a quite different life trajectory. This kind of good moments are one of the reasons why I like writing: it is for in this enormous world, to briefly form a super weak connection with a stranger that you would have never had any relation with (i.e., form an edge in two atom in a world full of atoms), and then go back to being strangers. It was also because of this email that I later decided to proofread my English versions carefully in my future writings, instead of simply translating them from Chinese without further proof-reading.)

At ICML 2024, even though I had already decided not to working on theory and spent most of my time talking to peers working on ML applications, three people who works on theory, whom I know when we were all applying PhD in 2023 happened to be at the conference too. One of them, Frank, invited us all for a city walk to catch up with each out, since we have each others contact but have never met in person. Being with them, I felt a bit out of place, because by then I could no longer follow the optimization theory and stochastic algorithms they were so heatedly discussing. I could only listen, envying them for still pursuing what they loved, even with so many LLM chaos all around. (Frank probably invited me along because, if when you grabbed a random person at ICML back then, the random person were most likely working on LLMs or something related. So unlike those who works on LLMs, who should be busy networking and talking with people in their own field—they could only grab people that they already know to catch up with.)

At NeurIPS 2025, two of those three person I mentioned about attended. One had already switched to LLMs and was busy talking with other LLM people at the conference—too busy to hang out with us. Thus, Frank and me decided to catch up ourselves. He said that he and the third guy who l used to work on theory (the person who didn’t attend NeurIPS) were both preparing to switch to LLMs (Frank and I each conveyed to him with our most sincere and heartfelt longing through messages), and we also discussed things like switching research directions and finding internships. Because NeurIPS was in San Diego, a huge number of people came, and I also ran into a younger PhD student Jack. He too had switched from theory to LLMs, and was also very anxious. We discussed the state of the job market. By then there were already many PhDs graduating in only three or four years (I was job hunting at the time), and I was anxious in thinking that: if all those PhD students, who should have entered the market one or two years later suddenly flood in all at once, doesn’t it get even harder for not-so-outstanding PhDs like me, or for people who’ve only just switched their research directions, to find jobs? (On top of that, there were also many extremely talented first or second year PhDs who leave PhD program to join industry.) Jack, anxious but forcing himself to look on the bright side (he had only just started his second year), said, “Or maybe I’ll just stay in school through this turbulent period, and by the time I graduate the market will have stabilized again.”

I also chatted with a peer I had once looked up to—a rising star in my eyes, the one who “could read in 30 minutes a proof that took me days to understand.” He said he had switched to LLMs in the past few months, but since others had several more years of experience than he did, he was slowly feeling his way forward, trying to catch up. Once again I didn’t know what to say. I thought: even those who once shone so brightly in their own field can feel lost and adrift when faced with something new. If he had stayed in his field, what a dazzling star he would have been. But LLMs have forced many people to abandon the directions they love and are good at, to work on something new, while others who got into it a little earlier already have a lot of experience. Sometimes I wonder: is it possible to catch up? Judging by most people’s examples, it seems that as long as you’re willing to work hard, willing to put in the time, and quick to grasp things, catching up is possible—and I’ve since seen many examples of successful transitions. Though I’m not one of the “willing to work hard, willing to put in the time, quick to grasp things” crowd (by now I’ve made peace with my own nature, so I just take things as they come and have no plans to catch up with anyone). But the those people I mentioned above—those who used to work on theory, they’ll surely catch up in the end, through their hard work and their brilliance.

By 2026, almost every person met while applying for PhDs who used to work on theory had switched to LLMs. The trigger for writing this piece was that the friend Leo I mentioned earlier, the one who had wanted to go into quant, told me today that he just switched to LLMs this semester—he who had once been so resolute even in his most difficult time when he can’t find an advisor. He said he had only just changed directions, was still in the adjustment period, and was very unhappy due to the transitioning phase.

As I write this, my mind is flooded with thoughts. Our PhDs—a mere three years, and yet it feels like such a long, long time has passed. So long that, unless I make an effort to remember, I’ll forget that many of those people and ephemeral moments ever existed. So long that when I sit down to write, I don’t know where to begin, or how to end. But these people and moments are still worth remembering. One of the reasons I originally wanted to work on ML theory was that I felt everyone in this fields was so nice (after all, everyone had plenty of free time, did research at an unhurried pace, and was very easygoing)—whether Leo, who didn’t usually hang out with me but showed up at my A exam to show his support, or Frank, whom I only knew each other online due to PhD applications, but were thoughtful enough to find a reason to hang out everytime when he know that we are attending the same conference. Our friendships feel so light and understated—so light that I often forget them constantly, and they might also forget me. But when something reminds us of each other, we can still catch up like old times. Even though so much has changed—we’ve all switched directions, and life and work have made us less idealistic than we were—there’s always something familiar that hasn’t changed. When we grow older, we stopped chatting with people without a specific purpose. Everyone has gotten into the habit of talking with a purpose (“with a purpose” isn’t meant as a criticism here; it’s more that a conversation needs a reason, and most of the time my reason is work or academic related). So I don’t know if we are friends or coworkers, since everying conversation feels like “meeting” or business chat.

Those friendships with those old friends (who used to work on theory), though seem so light, yet don’t feel the least bit unfamiliar when picked up again a year later, are pure enough to leave me nostalgic and savoring.

In mid-July, Moonshot released Kimi K3, a 2.8-trillion-parameter open-weight model that beat the flagships from OpenAI and Anthropic on the FrontierSWE coding benchmark. A few days later, Alibaba shipped a new version of Qwen. On July 20, Axios reported that the Trump administration was again pushing for a de facto ban on Chinese open-weight models; the instruments under discussion included Entity List designations, federal procurement restrictions, security advisories, liability rules, and pressure on US companies running Chinese models in production. Four days after that, on July 24, Jensen Huang used the first X post of his life to publish a policy letter titled “Open Weights and American AI Leadership.” The ask: Washington should not impose “premature restrictions” on downloadable AI models. The first version carried 25 corporate signatures, including NVIDIA, Microsoft, and Meta. Within a day it doubled to 50 — OpenAI and Google had signed; Amazon and Anthropic had not. By July 30, the official page hosted by Microsoft listed more than 230 signatories, and Amazon had appeared among them. The roster spans chips, servers, clouds, neoclouds, inference platforms, enterprise software, developer tools, and venture capital. Anthropic is the only frontier lab that has stayed away throughout.

Two ways inference demand can converge

The center of gravity in AI compute demand is shifting from training to inference. At a press Q&A at CES 2026, Huang offered a figure: one out of every four tokens generated today comes from an open model. There are two extreme shapes that inference demand could converge toward:

(1) Concentrated. Demand pools into a handful of closed frontier model companies. Enterprises rent intelligence, and tokens flow out of a few API endpoints.

(2) Diffused. Demand sinks down to tens of thousands of enterprises, inference platforms, neoclouds, sovereign AI programs, and application companies, each running its own specialized model.

Open weights are the condition that makes diffusion cheap and scalable. So this is a fight over market structure. And market structure determines whether NVIDIA faces five buyers or fifty thousand, and whether enterprises have any alternative to bring to the table when negotiating with closed APIs.

What NVIDIA wants is a fragmented buyer base

On the surface, NVIDIA should be neutral on model paradigms — whether closed labs dominate or an open ecosystem does, both sides buy GPUs. As long as AI workloads grow, NVIDIA makes money.

But NVIDIA cannot be neutral on the structure of its buyers. Its customer concentration is rising fast: in FY2026 Q3, four direct customers each accounted for more than 10% of total revenue, adding up to 61%; a year earlier it was three customers at 12% each, totaling 36%. We can only see concentration at the procurement layer, not the names of the end demand. But there is little disagreement in the market about who that procurement ultimately serves: Google, Amazon, Meta, Microsoft, OpenAI. And those same companies are the only ones capable of building an alternative to NVIDIA. Google has TPU, Amazon has Trainium, Meta has MTIA, Microsoft has Maia, and OpenAI is co-developing silicon with Broadcom. The weaknesses of these ASICs — narrow ecosystems, high CUDA migration costs — are not fatal for a hyperscale platform with a fixed internal workload, and those weaknesses are shrinking: both TPU and Trainium are now sold externally. Anthropic, for instance, uses Google TPU, AWS Trainium, and NVIDIA GPUs simultaneously, with the stated rationale of “matching the workload to the most suitable chip” — which is to say, large-scale frontier training can run on non-NVIDIA silicon. (NVIDIA is also one of Anthropic’s investors, so the two are not purely adversarial.) Which is why NVIDIA needs a world with a more fragmented buyer base. Tens of thousands of customers who can neither afford nor build their own chips are a far safer foundation than a few giants who are actively building substitutes. The Nemotron family, and the Nemotron alliance announced at GTC 2026 — with roughly $26 billion of company investment over five years — all indicate that NVIDIA is paying out of pocket to manufacture the fuel for a diffused world.

(That said, as NVIDIA’s own 10-K risk factors note, open-source AI depends on developer adoption, and if it is deployed on competitors’ platforms it could reduce demand for NVIDIA’s products. Open weights push demand downward, but which silicon it lands on once it gets there is not NVIDIA’s call.)

Sovereign AI: a new national-scale demand category

Sovereign AI refers to an AI capability stack that a country builds and controls itself: compute located within its borders, models under its own legal jurisdiction, and training data that includes its own languages, public data, and industrial knowledge. Three forces drive it — data localization (healthcare, financial, government, and defense data not flowing to foreign APIs), national security (critical systems cannot depend indefinitely on foreign companies’ closed models and terms of service), and language and culture (leading frontier models under-serve a great many non-English languages, dialects, and administrative and legal systems).

By NVIDIA’s own disclosure, sovereign AI revenue exceeded $30 billion for full-year FY2026, more than tripling year over year, with the main contributions coming from the UK, France, the Netherlands, Canada, and Singapore; FY2027 Q1 added new AI factory projects in Japan, South Korea, and Germany. The buyers of supercomputers used to be research institutions; the buyers of AI factories are finance ministries, defense ministries, national telecom operators, and sovereign wealth funds. Saudi money is betting on both tracks at once: domestically, PIF-owned HUMAIN is building gigawatt-scale compute and the Arabic model ALLAM, while Aramco Ventures led Together AI’s $800 million round in July, investing directly into the American open-weight inference layer. Nearly all of these countries are US allies, and nearly all of what they buy is NVIDIA. So what “sovereign AI” is actually doing at this stage is trading API dependence for chip dependence — and the chips still come from NVIDIA. For NVIDIA, sovereign AI is therefore a demand category that expands the number of buyers while posing no threat at all to its own moat.

For policymakers, a country that can only call foreign closed APIs does not have real sovereign AI; it needs models it can download, inspect, deploy, fine-tune, and maintain over the long term. Kimi, DeepSeek, Qwen, and GLM are mostly released under permissive licenses like MIT, which is exactly what sovereign programs need. If the US restricts its own open models, sovereign AI programs will naturally turn to Chinese ones.

Inference providers sell model utilization

Fireworks, Baseten, and Together are the most direct beneficiaries in this ecosystem. All three signed the letter. What they sell is the ability to turn a model into a production system: deployment, fine-tuning, adapters, distillation, autoscaling, GPU scheduling, and latency and cost optimization.

Capital markets placed their bets on this layer in a dense three-month stretch:

Company	Round	Date	Amount	Valuation
Baseten	Series F	2026-06-22	$1.5B	$11B / $13B (two tranches)
Together AI	Series C	2026-07-01	$800M	$8.3B
Fireworks	Series D	2026-07-16	$1.505B	$17.5B

What these companies really are is open-weight inference foundries. Fireworks has disclosed that more than 95% of the tokens it serves come from models specialized on customer data: fine-tunes, adapters, distillations, and models customers trained themselves and brought over to be hosted. In that last case, the inference provider takes no part in training and handles only serving — it is paid for the production system after the model goes live.

(A note: the service economics of adapters differ from traditional fine-tuning. Under the traditional logic, each customer gets its own copy of the model, and deployment costs are extremely high. Under multi-LoRA, one base model stays resident in GPU memory while hundreds of customer adapters are mounted on top of it, hot-swapped at the request level. The same fleet of GPUs serves a large number of customers’ “dedicated models,” and utilization is very high.)

On cost: Fireworks CEO Lin Qiao says that at equivalent quality, the cost is 5–10x lower than closed models; Decagon, a Together customer, says its costs after migrating fell to between one-fifth and one-seventh of the closed-model alternative.

The capability gap has also narrowed to a point where it can be negotiated with. Stanford’s 2026 AI Index puts the open–closed gap at 3.3 percentage points. When capability differs by a few points and price differs by a multiple, a large share of workloads will move to open-weight models — though the hardest ones will not.

Inference providers are not the SaaS of a new era. Ordinary SaaS can carry gross margins above 70% because marginal cost is near zero: one more user means a bit more server and bandwidth, not a linear increase in core cost. Inference providers face linear variable costs and cannot amortize fixed overhead through scale. Double the tokens served and the GPU bill roughly doubles too. Sacra estimates Fireworks’ gross margin at around 50%, against a company target of 60% — below SaaS’s 70%+, because GPU cost lands directly in COGS (cost of goods sold, i.e., the direct cost of delivering the product). Margin improvement therefore has to come from three directions: technical efficiency, moving up into customization, and moving down to lock in compute supply.

Progress on the technical efficiency side is genuinely remarkable. Per SemiAnalysis and NVIDIA, Blackwell delivers roughly 30x the tokens/sec/GPU of Hopper a year earlier on frontier inference workloads, and the cost per million tokens is falling by order-of-magnitude increments annually (EpochAI’s figure is about 10x per year). But compute prices are rising at the same time. In April 2026, spot rental for H100s had risen from about $1.70/hour last October to $2.35; the Blackwell spot index went from $2.75 to $4.08/hour in two months. Unit compute prices are rising, but throughput is rising faster, so cost per token continues to fall. This means that even as competition keeps pushing token prices down, inference providers’ gross margins need not compress — as long as costs fall faster than prices. That said, Sacra’s risk assessment of Fireworks notes that if open-source frameworks like vLLM and SGLang close the performance gap, what remains is GPU resale at 50% margins.

Circular capital, courtesy of NVIDIA

NVIDIA sells the cards and also invests in the people buying them. It is a shareholder in Fireworks, Baseten, and Together; it holds roughly 6% of CoreWeave and added a $2 billion private placement at $87.20 per share in January 2026 — and CoreWeave’s single largest expense, by order of magnitude, is buying NVIDIA GPUs. The money NVIDIA puts out comes back as demand for NVIDIA chips. So part of the demand on NVIDIA’s income statement is generated by its own capital rather than being fully exogenous market demand.

The hyperscalers’ three-layer business

Hyperscale clouds operate at three layers:

Layer one: bare GPU rental (IaaS). The customer rents an instance with eight H100s or B200s and installs the OS, drivers, inference framework, and model themselves, billed by the hour. AWS P5 and the Azure ND series sit here.

Layer two: managed model deployment (PaaS). The customer uploads or selects a model, the platform runs it and hands back an endpoint, with autoscaling, monitoring, and operations included. SageMaker and Vertex sit here.

Layer three: model-as-API (billed per token). The customer never sees a GPU and pays only for input and output tokens. Bedrock, Azure AI Foundry, the OpenAI API, the Anthropic API, and the Gemini API sit here.

Each layer up generally produces higher revenue and higher margin from the same GPU. But the clouds’ positions are complicated.

Microsoft is OpenAI’s largest partner: tied to OpenAI on one side, developing its own MAI and the open-weight Phi family on another, and simultaneously selling everyone’s models on Azure rather than betting on a single one. Google has both Gemini and Gemma; it wants to sell closed APIs and also wants Vertex to be the platform where enterprises deploy every kind of model. Amazon is the subtler case. It has no genuinely first-tier frontier model of its own, and it shut down its AGI lab on July 22. Bedrock’s marquee offering has long been Anthropic. Amazon’s cumulative actual investment in Anthropic has reached roughly $13 billion, with up to another $20 billion tied to commercial milestones, for a cap of about $33 billion; in return, Anthropic has committed to spend more than $100 billion with AWS over the next decade and gets access to up to 5GW of Trainium capacity. Widespread open weights would erode Bedrock’s Anthropic-centered differentiation — but Amazon is also an IaaS provider and an aggregator, which is why, late as it was, it signed in the end.

Fragmentation is a survival condition for neoclouds

CoreWeave, Lambda, Nebius, and Crusoe — the neoclouds — operate at layer one: buying GPU clusters at scale, typically financed with debt, then renting out compute on long-term contracts. What separates them from the big three clouds is that they have only GPUs and no full cloud product line — no databases, no decades of enterprise customer relationships, no complete set of compliance certifications. So they can only compete on the price, delivery speed, and availability of raw compute. Open weights expand the number of entities that need to run their own GPUs, which is very much to the neoclouds’ benefit.

But while fragmentation is a long-term survival condition for neoclouds, it is not the current state of affairs. CoreWeave’s FY2025 annual report shows Microsoft alone accounting for roughly 67% of revenue (62% in FY2024), with no second customer above 10%; in Q1 2026 the top two customers together came to about 65%. Its contracted backlog is heavily concentrated in OpenAI (roughly $22.4 billion in cumulative commitments) and Meta (roughly $35.2 billion). At the same time it carries tens of billions of dollars in debt and 2026 capex guidance of $31–35 billion. In that structure, a single largest customer declining to renew could leave it insolvent.

On top of that, its biggest customers are becoming its competitors. Meta, for example, is standing up a cloud business called Meta Compute to sell the surplus compute from its $115–145 billion of 2026 capex, in forms including model access through Muse Spark and raw GPU cycles. SpaceX’s Colossus site was leased in 2026 to Anthropic (roughly $45 billion through mid-2029), Google (roughly $30 billion), and Reflection ($6.3 billion). And OpenAI’s Stargate is the build-it-yourself path itself.

Neoclouds hold no model assets and have nothing to win or lose at the model layer. What they can do is find enough buyers to fill capacity they have already bought with debt — an idle GPU is pure loss. Open-weight models expand the number of long-tail buyers (enterprises, sovereign AI programs, AI application companies, vertical model companies, inference platforms, research institutions), which is the most direct route to de-concentrating their customer base.

Enterprise bargaining power

Enterprises have data concerns, but the overwhelming majority of small and mid-sized companies will not actually buy cards and run models themselves.

The reason is TCO (total cost of ownership). Self-hosting means not just buying GPUs but building and maintaining the infrastructure and employing the engineers to do it. Then there is utilization: an API is a purely variable cost — no calls, no spend — while owned GPUs are a fixed cost billed by the hour. Enterprise traffic is typically high during the day and low at night, so average utilization may be poor. A common industry rule of thumb is that self-hosting only becomes worth discussing once annual API spend reaches the seven-figure range — and that is before accounting for operational risk, hiring difficulty, and opportunity cost.

The most important value of open weights to enterprises comes down to three things:

(1) Portability. No lock-in to a single API. If a vendor changes prices, retires a model, or changes its data policy, the enterprise has a migration path.

(2) Auditability. Holding the weights yourself means the model will not be swapped out or degraded without your knowledge, and in compliance settings you can reproduce the behavior of one specific version.

(3) Negotiating leverage. Even if the enterprise ends up using a closed API anyway, having an open alternative that is good enough for most tasks puts it in a completely different negotiating position. There is no stable relationship between a closed model’s marginal inference cost and its list price; list price is set mainly by competitors, substitutes, and willingness to pay. The existence of open-weight models effectively caps what closed models can charge.

For an enterprise, if a single vendor controls the model, the pricing, the access, and the institutional knowledge the company has accumulated, then that vendor increasingly controls the company’s business.

Important Note: Although I try to keep academic/technical blog posts serious, I still cannot help inserting some completely unrelated things while writing. To avoid polluting the context, I will use the /btw command to separate them from the technical parts. Readers can skip these “by the way” sections.

Over the past few years, the main thread of progress in large models has mostly revolved around “the model itself”: parameters, data, pretraining, post-training, and reasoning. In the early days, when we talked about agents, the most common definition was also very simple: LLM + tool use. Later, this gradually became context engineering: managing the context the model sees at each inference step, so that multi-turn tool use is not drowned by history, noise, tool definitions, and intermediate results. Later still, as tasks evolved into long-horizon tasks, agent capability became a system capability composed of the model, harness, context, tools, evals, sandbox, and state management together: within a finite context, external tools, and a changing environment, the agent must keep pushing toward a goal and get closer to completion over time.

In this blog, we mainly focus on context engineering and long-running harnesses.

Context engineering grew out of prompt engineering. When an agent changes from a single-turn answering system into a multi-turn action system, the input the model sees at inference time changes too. It is no longer a relatively independent external input, namely a prompt, but a context related to the model’s continuous interaction with tools and the environment. This context may include tool definitions, file contents, web observations, historical messages, test outputs, MCP server descriptions, external documents, progress records, and so on. Context engineering decides which information should be placed in the prompt up front, which should be retrieved on demand, which should be written into external memory, and which should be handled by code or tools rather than stuffed into the model context. Anthropic systematically explained this concept in Effective context engineering for AI agents in September 2025. A concrete context engineering example is: when there are too many MCP tools and too much data, we should not let every tool definition and intermediate result flow through context. Instead, the agent should write code to call tools on demand, filter the data, and bring only high-signal results back to the model. Context engineering also includes compaction, structured note-taking, and even multi-agent patterns, but these mostly serve the question of “how should context be managed?” The long-running agent harness discussed later goes one step further: it does not only manage context, but separates planning, implementation, evaluation, revision, and final acceptance into different processes.

By the end of 2025, as coding agents became able to make longer continuous progress on software engineering tasks, the center of gravity quickly shifted toward long-running agent harnesses: from the initial focus on the model’s coding ability to the ability to continue executing a task across multiple context windows / sessions. In November, Anthropic released Effective harnesses for long-running agents, which can be viewed as the first version of the long-running harness. The system started to shift from “managing one agent’s context” to “managing multiple sessions.” This version of the harness introduced a minimal role split between an initializer agent and a coding agent. The initializer’s role is to establish an external project state that can be handed off: it breaks the task into a smaller feature list, writes a progress file, creates an init script, initializes a git repo, and marks each feature as pending. Later, each coding agent session only advances one bounded increment, such as implementing one feature or fixing one specific issue. The harness requires it to test, update records, commit, and leave the repo in a clean state before ending. That way, even if the next session starts from a new context, it can reconnect to the task through the progress file, feature list, git log, and test results.

In January 2026, Anthropic published Demystifying evals for AI agents. This article systematized agent evals: because agents act in environments, call tools, and modify state over multiple turns, evals cannot only look at the final sentence. They also have to look at the transcript / trajectory and whether the final environment state is actually correct. Roughly speaking, it makes two points. One is task-level: how do we determine that the agent really did the task and did it well, rather than producing something that looks acceptable on the surface but is actually buggy or incomplete? The other is harness-level: we also need to evaluate whether each component of the harness is actually load-bearing, rather than just adding useless stuff.

By March 2026, in Harness design for long-running application development, this idea became the second version of the harness: initializer + coding agent was expanded into planner + generator + evaluator. The evaluator uses Playwright MCP to operate the application like a user, test UI, API, and database state, and reject outputs that fail certain criteria. Whether this evaluator should exist depends on whether the task exceeds the current model’s reliable solo capability. For example, on Opus 4.5 it was clearly useful; by Opus 4.6, the model itself had become stronger, and some scaffold could be removed or weakened. These adjustments and transitions naturally make us think about a longer-term system design problem (see their April post Scaling Managed Agents): a harness is not a fixed structure, but co-evolves with model capability. Earlier Sonnet 4.5 / Opus 4.5 had context anxiety and would under-scope, so they needed more scaffold: context reset, sprint decomposition, evaluator feedback loops. But by Opus 4.6, the model itself was better at long-running agentic tasks, code review, debugging, and long-context retrieval. Some structures that were originally necessary started to become less necessary. For tasks the model can already complete reliably solo, an evaluator may no longer be worth the cost. Of course, for tasks that sit just beyond the model’s capability boundary, an evaluator can still bring real lift. Many designs in a harness essentially contain assumptions about “what the current model cannot do well.” After the model upgrades, these assumptions may no longer hold. So we should not treat one generation of harness as an architecture that can be used forever. Instead, we should split the system into relatively stable abstractions: use sessions to record what happened, use the harness to manage the agent loop and tool use, use the sandbox to provide a controlled execution environment, and so on. This way, the underlying harness can keep being replaced, but the agent product is not tied to one generation of scaffold.

Context engineering

People used to focus on prompt engineering, namely how to write the system prompt. But as agents become more multi-turn and longer-running, the focus becomes: at each inference step, what did the model see? For example, an agent’s context may contain the system prompt, developer instructions, tool definitions, MCP server descriptions, user messages, conversation history, file contents, command output, browser observations, screenshots, errors, progress notes, retrieved documents, evaluator feedback. These things cannot all be blindly stuffed in. Even if the context window becomes larger, we still need to choose selectively, because noise in a long context distracts the model and increases cost and latency. So a good context is the minimal high-signal context that lets the model complete the next step at the current step. In general, we use just-in-time context: instead of stuffing all documents, tools, and history into the model, we keep references, such as paths, URLs, commit hashes, log files, database table names, and so on. When the model needs them, it loads them through tools. In this way, the agent’s working memory keeps indexes and currently relevant fragments, rather than all of the context.

Common context techniques in long tasks include:

• Compaction: When the context is almost full, compress the current conversation trace into a summary, then use that summary to start the next context. Usually this is still the continuation of the same task and the same agent loop. Its goal is to preserve conversational continuity as much as possible. In other words, the model should feel that “what I was just doing is still here”; only the history has been compressed.

• Structured note-taking: Let the agent actively write task state into persistent external artifacts, such as progress.md, feature_list.json, NOTES.md, or git commits. It is not necessarily only for the same task. A project-level NOTES.md can be read by later sessions, by planner/generator/evaluator, or reused in neighboring tasks.

• Context reset: Use external artifacts, such as files produced by note-taking, git logs, test results, task lists, and so on, as startup material to open a clean new context/session. It does not preserve the full conversation history, and can clear noise and bad assumptions so that a new agent can reorient itself.

• Multi-agent architecture: Do role division and context isolation. Different agents receive different context, do different subtasks, and finally pass only high-density results back to the main flow.

Tool use

There are also some things to watch out for in tool use. For example, the number of tools cannot expand without limit, otherwise the model wastes attention on choosing. Tools should not overlap too much in functionality, otherwise the model does not know which one to use. Tool descriptions should clearly explain when to use them and when not to use them, namely they should make the boundaries clear. Return values should be token-efficient and should not pour irrelevant data into context. Returned error messages should let the model know what to repair next. Names should be clear, so the agent can infer boundaries from the names. Also, tools are preferably not exposed directly to the model as direct tool calls, but mapped into code APIs / a file tree, so the agent can call them through code. This way, the agent can read only the tool definitions it currently needs, filter large data inside the code execution environment, and return only summaries or a small number of results to the model. It can use loops, conditionals, and exception handling to complete complex control flow, or write intermediate state into files instead of stuffing it into context. Tool outputs are deterministic; we should leave deterministic computation to code. Judgment and planning, such as how to call tools, which tools to call, in what order, and how many times, should be left to the model.

The problem of long-running agents

1. Context window

Long tasks will definitely exceed a single context window. Even if the model supports a very long context, context is always a finite resource: every extra piece of information consumes part of the model’s attention budget. This brings three common phenomena:

• Amnesia: A new session does not know what the previous session did.

• Context rot: The longer the context, the easier it is for the model to lose the point; earlier information gets buried by noise.

• Context anxiety: When the model feels that it is approaching the end of the context, it rushes to wrap things up and calls a half-finished product complete.

2. Planning and hand-off

Many complex tasks contain dozens of mutually dependent features, bugs, design decisions, and validation steps. If the model starts directly from a high-level prompt, it easily tries to do everything at once, then runs out of context halfway through, leaving something that appears to have a lot of work in it but does not actually work. Also, even if some things are useful for the next agent, the next agent does not know which things are useful: is something that looks like a bug intentionally made that way, or is it something the previous session did not have time to clean up?

/btw

I believe many of you, in PhD work or at work, have had the experience of inheriting a collaborator’s mess. Although by the late stage of my PhD, around December 2025, agents were developing so fast that I felt a lot of anxiety and insecurity, deeply felt that I could no longer stay in academia, and had completely lost the motivation to keep publishing papers, I also became the kind of person I once disliked: someone who leaves collaborators with a pile of messes.

3. Self-evaluation

The model does not necessarily judge what it generated strictly. It may see that a button shows up and think the feature is complete; see that a page roughly looks right and think the design is good; or run part of the tests and think the whole system is usable. In Anthropic’s long-running application harness post, they mention that letting the same agent be both generator and reviewer often leads to overly lenient judgments. Especially for front-end design, product completeness, and edge interactions, where there is no unit test directly defining the thing, the model can easily convince itself.

/btw

After all, the world is a giant makeshift operation, and models have learned a lot from us. Also, a lot of the time, people only need to convince themselves.

Harness

An agent harness, also called a scaffold, is the system that allows a model to act as an agent. It handles inputs, organizes tool calls, manages state, executes loops, and returns results to the model. It includes the file system, task board, test scripts, browser, shell, git, logs, sandbox, permission system, checkpoints, handoff notes, and so on.

The core design of the first generation of long-running harnesses was quite simple. It mainly used two types of agents:

• Initializer agent: Runs at the very beginning to set up the working framework and environment.

• Coding agent: Makes incremental progress session by session, and leaves structured handoff artifacts before ending.

The initializer agent does several things.

1. Create a task list.

For example, if the user only says “build a claude.ai clone,” the initializer does not let later agents directly start building. Instead, it breaks the task into many verifiable features, such as creating a new chat, entering a message, pressing Enter to send, receiving a response, switching themes, loading historical conversations, and so on. Each feature’s initial status is marked failing.

2. Create a progress file.

For example, claude-progress.txt or a similar file records what each round did, what remains, and what later agents should pay attention to.

3. Create a startup script.

For example, init.sh tells later agents how to install dependencies, start the service, and run basic checks.

4. Initialize a git repo.

At the end of each round, commit. This way, later agents can understand recent changes through git log, and the system can roll back, diff, and audit.

Later coding agents roughly do three parts:

1. Understand the current engineering state they are receiving (orient): confirm the current working directory, read the progress file and feature list, look at the recent git log, and run an end-to-end test to confirm that the basic functionality is not broken.

2. Select and finish one feature (select & implement & validate): choose the highest-priority unfinished feature, implement one relatively independent increment, test it, and evaluate that feature.

3. Hand off a clean state to the next round, so that the current code can be understood by the next agent: for example, update the feature status and progress file and commit. If a new feature is not finished, leave a clear boundary instead of pretending it is done.

A long-running agent is essentially using multiple short sessions to stitch together one long task. If every session leaves behind a small mess, errors compound with interest. So a harness should not only reward “verifiable increments,” but also reward the state of the system it leaves behind. If a session writes three more features but breaks the startup flow, the next round may have to spend a large number of tokens first recovering the environment. In the long run, this kind of progress is actually negative return.

The three roles of planner / generator / evaluator

The first-generation harness solved the context handoff problem, but there was still one big problem: agents are not good at judging themselves. So Anthropic introduced a multi-agent structure in the second generation. The final form roughly contains three roles:

• Planner: Expands the user’s high-level prompt into a specification, feature list, design direction, and implementation plan.

• Generator: Implements features according to the plan: writes code, changes UI, connects the backend, and runs tests.

• Evaluator: Acts like QA, operates the application, checks the result, points out problems, and decides whether it passes.

Different roles correspond to different failure modes.

Planner solves under-scoping. Without a planner, the generator often understands the task too narrowly. For example, for “build a 2D retro game maker,” a single agent might build a page where you can place some tiles and think it is done. The planner expands it into more complete product specifications such as a level editor, sprite editor, entity behavior, playable test mode, animations, export, sound effects, AI-assisted generation, and so on.

Generator solves execution. It is still the main labor force, but it no longer decides the entire product boundary from nothing; it works inside a clear spec and current feature contract. Generator also does not have to be a single agent. It is more like a role: multiple generator sessions or coding agents sequentially hand off execution. Each round of generator reads the planner spec, current code, progress file, and previous evaluator feedback, then completes a bounded increment, writes state back, and exits. The next generator then reads these artifacts and continues. So the continuity of a long-running harness is placed in external project state, such as progress files, feature lists, git commits, and test results.

Evaluator solves self-evaluation. It is not just reading code and then giving a score or feedback; it operates the application through Playwright or browser automation like a user, checking UI functionality, APIs, database state, and edge cases. So this moves from previous LLM-as-a-judge to agent-as-a-judge.

At the beginning of each sprint, the generator and evaluator first negotiate “what counts as done for this piece,” rather than having the generator finish first and then ask the evaluator to criticize it. Before writing code, both sides communicate and reach consensus. This “negotiation” happens by exchanging structured artifacts through files and the workspace: the generator writes a sprint contract proposal, explaining what this round will do, what the acceptance criteria are, and what is out of scope; the evaluator reads the planner spec and proposal, fills in missing boundary conditions or rejects an overly broad scope; the generator then revises the contract. What they share is the spec, contract, progress, code, test results, and feedback.

In this negotiation process, vague user stories become testable contracts. It is a bit like product requirement clarification + QA test plan in a human team.

However, giving evaluation to another LLM does not automatically solve the problem. Claude was not a good QA agent at first either. It would find real problems, then convince itself that they were not serious; or it would only do shallow tests and not touch edge paths. So the evaluator itself also needs to be tuned. It needs a clear rubric, failure cases, judgment criteria, interaction paths it must explore, and bug types it must not let go too easily. Ideally, we should read its logs, find where it disagrees with human judgment, and update and iterate on the evaluator prompt.

Anthropic’s frontend harness used several scoring dimensions:

• Design quality: Whether the whole thing has a clear character, rather than being a pile of components. Aesthetic quality.

• Originality: Whether there are custom design decisions, avoiding templates, default libraries, and an “AI feel.” Originality.

• Craft: Typography, spacing, color, contrast, and so on. Usability at the craft level.

• Functionality: Understanding and completing the task. The more important functionality side of usability.

So when using a judge, we cannot ask abstractly whether this is “good.” We need to break “good” into multiple checkable dimensions. Similarly, in the application development harness, the evaluator scores dimensions such as product depth, functionality, visual design, and code quality, and every dimension has a hard threshold. As long as one key dimension is below the hard threshold, this sprint fails, and the generator must continue revising based on concrete feedback.

When evaluating, what we evaluate should not be “the agent says it is done,” but the final environment state and outcome. A flight-booking agent saying “I booked it for you” is meaningless; what matters is whether there is actually a reservation in the database. A coding agent saying “bug fixed” is meaningless; what matters is whether the tests pass (outcome), and whether it damaged the original code (state).

/btw

Many life lessons can apply here. For example, when evaluating a person, what the other person did matters more than what they said. Because of my unique “therapist trait,” I often end up talking about related topics when discussing relationship problems with the girls around me. Processing and analyzing other people’s dilemmas is much easier than walking out of my own life difficulties. On one side I am being other people’s therapist; on the other side I need to find a therapist myself, but it is hard to find one I trust. After I started trying to write blogs, I actually felt much calmer. Maybe I am my own best therapist. Writing lets me quiet down and talk to myself. Before, I gave all my time to other people and almost did not allocate any time to myself. Also, although I have been anxious about finding a job, and about being replaced by LLMs as a researcher in the near future, I still think I am better at therapy than the most state-of-the-art LLMs. Unfortunately, this trait cannot make me a living and was never on my career-planning path. I have wandered too far.

Some points worth noting: the planner usually mainly solves initial under-scoping. For example, a user’s one-sentence prompt is often too broad, such as “build a 2D retro game maker,” and the planner can make it concrete: what core modules should this product have, which are must-haves, which can be done later, and how each feature should be accepted. After the project moves forward, the evaluator takes on part of the local planner role and does feedback-driven replanning. The evaluator modifies artifacts left by the planner. For example, the planner initially wrote sprite editor, only requiring draw and save. After real testing, the evaluator may add new acceptance criteria: brush size must work, transparent pixels must be preserved, saved sprites must appear in the entity palette, and state must not be lost after reloading the project. The planner can implement replanning by modifying external files such as feature_list, sprint_contract, known_issues, and next_actions.

So a multi-agent harness puts planning, execution, evaluation, and revision into external artifacts, so each round of agent can take over with limited context.

More durable design

Every component of a harness implicitly contains assumptions: assumptions about what the model cannot do well by itself. As models become stronger, these assumptions may no longer be valid. For example, if a model has context anxiety, the harness adds context reset. Later, if the model no longer has this problem, reset may turn from a necessary structure into latency and cost. If a model cannot plan, we add a planner. Later, if the model’s planning ability improves, the planner may only be valuable on large tasks. If the generator can already reliably complete a task by itself, the evaluator may become unnecessary overhead; but on tasks that sit just beyond the model’s capability boundary, the evaluator can again bring a large lift. So a harness is not better just because it is more complex. We should start from the simplest workable system, use evals to identify failure modes, then add corresponding structure for these real failure modes, and as models upgrade, periodically remove components that are no longer load-bearing, rather than building an overcomplicated agent framework from the beginning.

An agent system can be split into several relatively stable abstractions:

• Session: an append-only log of what happened.

• Harness: the control layer that calls the model, routes tools, and executes the agent loop.

• Sandbox: the environment where the agent can run code and modify files.

After decoupling them, the underlying implementation can keep changing while the external interface stays stable. Because the harness of long-running agents will keep evolving, stable abstractions let agent products avoid being tied to one generation of harness.

Agent Eval

Agent behavior is nondeterministic. Running the same prompt twice may succeed once and fail once. A task passing once does not mean the system is reliable; a task failing once may also mean the grader was wrong, the environment had a problem, or the task itself was ambiguous. Compared with ordinary LLM evals, for agent evals we should not only look at output text, but at the result that a sequence of actions causes in the environment. So we need sandbox, database, browser, file system, mock API, resettable environment, and so on. Different tasks may care about different metrics. For a coding agent, if we allow the agent to try multiple times, pass@k (the probability that at least one of k attempts succeeds) is a relatively suitable metric, because as long as one patch is correct, it can enter review. But for customer service, reimbursement, flight booking, and other agents used directly by users, pass^k (the probability that all k attempts succeed) is more important, because users expect reliability every time, not “one of ten attempts is right.”

There is also the difference between capability eval and regression eval. Capability evals contain some tasks the current system does not do well. Regression evals evaluate “can it still do what it used to know how to do?” This should be close to 100% pass, and is used to prevent regressions after system upgrades, model switches, or prompt changes.

Agentic Safety (the importance of a good RL environment)

In the non-agentic era, because the model could only answer text, the blast radius of mistakes was small. But now agents can run shell, modify files, access databases, call SaaS APIs, send Slack messages, and open PRs. The cost of mistakes can be much larger. For safety, we cannot rely only on “the model will judge whether a command is dangerous” to protect the system. The safer approach is to restrict what the agent can access, where it can write, what networks it can reach, which tools require approval, which state can persist, and which secrets are never visible. But this requires a good RL environment. Of course, once we mention RL environment, we have to mention reward hacking: we should not put shortcuts we do not want the agent to use into its environment. For example, in a coding eval, future commits should not be placed in .git/objects, hidden tests should not be placed inside the container, and secrets that can access production systems, such as API keys, OAuth tokens, cloud provider keys, and SSH keys, should not be directly exposed to the agent. We can instead use controlled tool capabilities, and implement permission checks, parameter constraints, and auditing inside the tool.

(Translated from the Chinese version by ChatGPT.)

The dust of the times, when it settles on any single person, becomes a mountain. Over the course of a life, one will most likely live through many great, era-defining changes. My own social circle is small, and I hardly know anyone from other generations, so I can’t say whether, somewhere in their lives, there was a historical stretch that could rival the hurricane AI has stirred up over these past two or three years. Every generation, meeting the great upheaval of its own time, probably laments the misfortune of being the generation it is. And yet, seen along the timeline of modern human history, even if a single life is only a tiny segment of that long river, it seems no generation ever passed through a plain, uneventful life without living through some great historical upheaval.

Times of great upheaval: the shifting of thought, the reshaping of values, always come with pain. We were raised inside a system that reveres knowledge (the law-abiding, by-the-book traditional Chinese education system), taught from the very beginning to study hard, to become knowledge workers, to have “a respectable job” and a stable life. And so, along a path already laid out and validated by many, our life choices were shaped for us: go to college, earn a master’s, earn a PhD, then become a professor or a research scientist — the stable, relatively well-paid job. It was the predictability of this path that gave us our sense of safety. And then one day, the path collapsed. Because it was so stable: we had been walking it from the moment we were born, we had long taken its existence for granted, and had never once imagined what our Plan B would be if it ever gave way. To survive, to feel safe, people keep a Plan B for all sorts of things; but in this one thing, for most of us, our past trajectory was never enough to instill any sense of crisis, until, at a certain point, it did.

Value has always been the product of a particular historical moment, not a constant law of how the world runs. But life is short; our experience lets us witness only a single stretch of history, and then we take that stretch to be the world’s constant law. This makes sense, too: across a short enough span between two points, even a curve can be approximated as a straight line — how much more so across the whole long river of time.

In the age of agriculture, safety came from land; in the industrial age, from physical strength; in the modern world before AI, from knowledge. For many of us, our sense of self-worth is built on “difficulty.” Things are prized for their scarcity: the harder something is, the fewer the people who can do it, and the more it grants us the safety of being “not easily replaced.” AI has made knowledge suddenly cheap, and “knowledge changes your destiny”, once engraved in our minds, is no longer a truth.

Over these past few months, I’ve spent a great deal of time studying and cultivating myself, stealing a few moments of calm from an anxious age. I’ve kept up with much of the latest technical progress in my field. I’ve convinced myself that the joy learning brings me comes from my appreciation of beauty: the beauty of algorithms, of mathematics, of scheduling and optimization, of logic. And the other things, the ones not technical or logical enough, strike me as not beautiful enough. Until now, the skills that earned my living, the things I spent my time on, and the things that made me happy were all aligned; I didn’t have to think, to weigh trade-offs, to analyze. Now a rift has opened. What others value, the thing that keeps me in a decent living, and what actually makes me happy are no longer aligned, and I have to decide where my time should go, how to balance spiritual needs against material ones, or else change my own perceptions and the way I see certain things, so that all of it can be made consistent again.

Uprooting something deeply rooted always hurts. I keep tracing back to find where my reverence for technical skill comes from. If I can find its source, and alter it, and through that alter the way I understand things, then perhaps I can complete this remaking of my mind more quickly and more easily.