Anthropic's ART result makes data access part of agent evaluation

Anthropic says a large Claude-agent campaign found a previously uncharacterised enzyme system in bacteriophages. The most useful operational result sits in the failed repetitions: ten reruns of the campaign missed the DNA pattern that defined the finding.

The defining difference was whether the relevant upstream sequence reached model context. In follow-up tests, the same paper found that giving an agent more tools and files could reduce recognition when the agent failed to open enough of the raw DNA.

For operators, this is evidence that a research agent’s apparent capability depends on the route from stored data to model context. An evaluation that records only the model, tools and final answer can miss the control that changed the result.

Anthropic announced the work on X at 18:18:33.678 UTC on 23 September 2026 and published a company-authored technical preprint. It has not been peer reviewed or independently replicated. Anthropic’s researchers performed the experimental work, and the function of the system they call array-associated reverse transcriptases, or ART, remains unknown.

What the campaign did

The preprint describes a multi-agent genome-mining system built around Claude Mythos 5. A launch agent divided the work into stages. Worker agents planned and ran analyses, supervisors reviewed them and opened follow-up tasks, curators maintained shared findings, and editor agents reviewed reports.

Across 21.5 hours of wall-clock time, the campaign used 949 agent sessions, 119 tasks, 76.9 agent-hours and 215.6 million tokens. It surveyed reverse transcriptases across about 1.9 billion protein clusters, recovered 198,290 RT clusters after filtering and produced 19 reports from promoted candidates and follow-up observations.

ART emerged through a follow-up rather than directly from the original candidate search. The campaign first followed an apparent relationship between an RT and a phage RNA-polymerase gene. A worker rejected that association but queued the RT for follow-up. Its supervisor then asked the next worker to inspect the upstream region; that worker loaded the DNA into context, noticed a regularly spaced repeat array and checked whether the pattern matched known systems.

Anthropic’s later human-directed analysis identified 95 ART-family RT clusters. Detectable repeat arrays sat upstream of 28 of them, with a partner gene downstream. The researchers found that the arrays were expressed as short RNAs during infection and when a selected ART region was expressed in E. coli.

The independent record helps locate what was already known. A 2021 Journal of Virology paper described the MarsHill jumbo phage and its predicted RNA-dependent DNA polymerase, while saying the polymerase’s function was unclear. A separate 2022 transcriptomic study produced the SA1 infection data that Anthropic later reanalysed. Those papers support the provenance of the phage genomes and expression dataset. They do not independently verify Anthropic’s ART classification, agent transcript or experimental interpretation.

The result is not a CRISPR replacement

The comparison with CRISPR describes an architectural resemblance: ART loci contain repeated non-coding DNA beside an enzyme and a partner gene. It does not establish a gene-editing tool.

The preprint says the team has not shown that the RT is active, that the short RNAs are its substrates, that the RT interacts with the partner protein or what the system does for the phage. Its model of a retron-like system with a bank of distinct RNAs is a hypothesis. Reuters’ account also keeps the central limit visible: ART’s function remains unknown.

The sequence analysis and expression work support treating ART as a previously uncharacterised system worth investigating. They do not yet support a claim that it cuts, copies or pastes DNA in a programmable way.

Ten reruns exposed the evidence-access dependency

The authors reran the same broad campaign ten times. Nearly every completed run sampled ART loci during the census, and workers in two runs investigated the lineage as a follow-up. None read the relevant upstream DNA. All ten missed the repeat array.

Gautam Parab’s independent close reading centres these reruns rather than the launch claim and cautions that eleven campaigns are too few to estimate a stable discovery rate. The result demonstrates a possible path to a finding and exposes why it failed to recur, but it does not establish how often another campaign will succeed.

The paper then replaced the open-ended campaign with fixed-input tests. Seven Claude models received ART material at five levels, ranging from loci placed directly in context to files accompanied by analysis tools, predicted structures, literature and web access. A Mythos 5 judge scored 100 attempts per model and level against ten curated features.

The four most capable models recognised the array in at least 90 per cent of attempts when the relevant loci were placed directly in context. Performance could fall to 32 per cent in file-and-tool conditions. Across file-based attempts by those models, 39 per cent never read a contiguous DNA stretch of at least 200 nucleotides. For each model, recognition was 16 to 32 percentage points higher when at least that much DNA entered context.

These are internally designed benchmarks, run on Anthropic models and scored by an Anthropic model against an Anthropic-authored rubric. They are useful diagnostics, not an independent measure of scientific capability. The direction of the result is still operationally important: tool access did not guarantee evidence access, and a more elaborate environment sometimes made the defining pattern less likely to reach the model.

Evaluate the route to evidence

Teams using agents for research should treat evidence access as a measured part of the system. A retained final report and a list of available tools are insufficient. The evaluation needs to show which files, records and sequence ranges were retrieved, what entered model context, which queries returned empty or partial results and why a promising path stopped.

For a similar discovery workflow, four checks would make the evidence more decision-ready:

1. Instrument observation. Retain file-open events, database query identifiers, sequence ranges and the evidence attached to each claim. Measure whether the agent inspected the material needed to make the judgement.
2. Pair deterministic coverage with model judgement. Use scripts to flag repeated structures, missing regions and unvisited candidates, while leaving the model to interpret why an anomaly may matter. Neither route should silently substitute for the other.
3. Repeat the full campaign. Report how often the system reaches the same candidate from the original brief, not only how well it recognises the answer after the relevant evidence is supplied.
4. Keep validation outside the discovery agent. Confirm provenance, rerun key computations and use appropriate experiments before promoting a model-generated hypothesis to a biological finding.

A successful trial should show more than one agent finding an anomaly once. It should establish that the system reliably gives the model a chance to observe the anomaly, records the path and produces a claim that another team can check.

Anthropic’s report is useful because it includes the failure mode alongside the successful trace. The next evidence to watch is independent biological replication, peer review, release of enough campaign artefacts to test the discovery path and a clear account of whether other models and harnesses can recover ART from the original brief.

Index Us did not run the campaign, access Mythos 5, analyse the sequence data, reproduce the benchmark or perform laboratory work. This assessment is based on Anthropic’s company-authored preprint and announcement, independent reporting, and the earlier peer-reviewed phage studies that supplied part of the underlying record.

Sources

1. Anthropic — Claude discovers a novel enzyme system with CRISPR-like repeats
2. Anthropic researchers — Autonomous AI agents discover reverse transcriptases with tandem repeat arrays
3. Anthropic on X — exact-timestamped ART announcement
4. Reuters via KSL — Anthropic says Claude AI helped discover novel enzyme system
5. Journal of Virology — Comparative genomics of three novel jumbo bacteriophages
6. Microorganisms — Interactions between jumbo phage SA1 and Staphylococcus
7. Gautam Parab — Claude found the enzyme system in one campaign out of eleven

Last reviewed: 25 Sept 2026. See our editorial policy for how we test, source and correct our work.