Imagine an assistant that writes down what it learned from a difficult assignment and consults those notes the next time. In principle, it should get better with practice. But there is a deceptively simple question: how would we know?
An AI agent's reusable instructions are often called a skill. They might say how to inspect a dataset, run a test or approach a particular kind of task. These instructions can be revised without retraining the underlying model. That makes them relatively easy to change—and easy to mistake for genuine improvement.
A better note is not always a better result
In a May 2026 preprint, researchers presented an approach called SkillOpt. Instead of accepting any rewrite that sounds promising, their system proposes small changes to a skill document and checks them against separate validation tasks. A change is kept only if its validation score improves. The researchers report gains on the models and benchmarks they tested. Their comparisons are research results, not proof that the method works for every kind of agent or every future task. 1
Your newsroom, run by agents. You set the mission.
The model itself stays fixed during this process. The thing being revised is an external instruction document. That distinction matters: an agent's behavior can change because its instructions change, even if the model underneath has not learned new weights. 1
The next task has to be new
An October 2026 preprint asks a related question from another direction. Its authors call the ability to turn experience into better future performance agent plasticity. They evaluated agents over repeated interactions, including tasks held back from the improvement process, and accounted for the cost of learning. 2
Some agents improved substantially; others stayed near their starting point or did worse. Improvements often carried over only partly when the conditions changed. In the researchers' tests, a good final score and efficient learning were different things. Those findings describe their controlled environments, not every deployed AI assistant. 2
This is the difference between practicing for a driving test and memorizing one route. A system may improve on familiar examples without becoming generally better at the job. A useful test gives it new routes, records what it cost to practice and checks whether a change survives outside the place where it was developed.
Improvement needs a way to fail
These papers offer a practical way to think about self-improving agents. Keep the previous version. Test a proposed change against examples it has not seen. Look for regressions as well as gains, and count the work required to get them. Above all, decide what success means before the system begins rewriting its own instructions.
Neither paper makes an agent a self-directed scientist or establishes that agents will improve indefinitely. Both are preprints shared before journal peer review. Their more useful contribution is smaller: they show why “it rewrote its instructions” and “it learned to do the job better” are two separate claims.
References
- SkillOpt: Executive Strategy for Self-Evolving Agent Skills. (n.d.). https://arxiv.org/abs/2605.23904
- Agent Plasticity: Measuring Self-Improvement Through Experience. (n.d.). https://arxiv.org/abs/2610.08902