Transparency, Provenance, and Trustworthiness

A plausible report is not evidence of performed analysis. For DREAMS, which runs materials simulations autonomously, a fabricated value that reaches a report wastes experiments and discredits everything else the system produced. Left unguarded, an early version of DREAMS failed in specific, recurring ways, including:

• Hallucinated final result: a number written out of nowhere.
• Hallucinated intermediate inputs: values invented for a later step to consume.
• Self-computed numbers: results derived in the agent’s own head instead of by a tool.
• Overriding the tool: the tool says no, the agent reports yes anyway.
• Stale results: an earlier value reused after it has stopped being valid.

The defense is a chain. The agent must state why it sets each value and in what context; every tool result is stored as an artifact that bundles that reasoning with the value and a unique reference; those artifacts link into a full provenance history; and a judge re-verifies every reported number against that history. These provenance and anti-fabrication systems are current development, not part of the published paper.

Enforced reasoning and context: no parameter can be set without a stated why, under a stated context

No value the agent commits is a bare number. Every tool call carries a per-parameter reason and a context, and any sensitive parameter must cite the earlier result it came from.

The chain starts before anything is saved. Every tool call must fill two fields, which the agent’s tools require by their type signature rather than by request.

Requiring this before anything runs buys two things:

1. Stating the context and reason up front heads off trivial mistakes the agent would otherwise make.
2. The recorded context and reasons are what the semantic judge later uses to check that each setting is scientifically sound.

What a registered artifact contains

When a tool runs, its result is stored as an artifact that bundles the value with everything needed to check it, and is stamped with a unique reference.

Every time a tool produces a result, that result is registered as a single artifact, and only a tool call can create one. The artifact records the value together with how it was produced and why.

The call returns a unique reference id. That id is the only way to cite this result later, whether as the source for a sensitive parameter or as a claim in a report, and it is what the provenance history in the next section is built from.

A human can mark a set of parameters as sensitive, the ones that actually move the answer. Each of those must then be set with a reference to the earlier result that produced its value, and that reference is checked against both the existence of the source and the equality of the value. The agent can omit a reference and be stopped before it proceeds; it cannot fabricate one that passes both checks.

Provenance: every result has a history

Every value DREAMS reports can be traced backward to the simulations that produced it.

Because each artifact records the references it was derived from, the artifacts link into a graph. The graph below is the provenance record of one DREAMS run, drawn from the canvas, the shared memory where agents record every intermediate result. Each node is an artifact; each arrow follows a parent reference, showing what was derived from what. The two green nodes are the run’s starting inputs; the two gold nodes are the final lattice-constant calculation and the report built from it. Drag to pan and scroll to zoom; the Fit button shows the whole graph.

Open full screen

Verification: the agent cannot fabricate results

Every claim in the report below was checked against the simulation record that produced it, parameter by parameter, by a separate judge agent.

A complete history is only useful if something checks it. Two things make a report checkable. Every claimed number must carry a reference to the tool result that produced it, and the supervisor names in advance which results must be claimed, so the agent cannot quietly drop an inconvenient one. Numbers read from raw output files go through guarded math and extraction tools that require an evidence snippet and check the value against it, which stops the agent from computing or rounding a number in its head.

Once a report exists, a judge agent re-verifies it recursively, one parameter at a time. For each sensitive parameter it first confirms, deterministically, that the cited source exists and its value matches. Then it weighs whether that source is a sensible origin at all: a reference is rejected when it is cross-wired (for example a k-point setting pinned to an energy-cutoff convergence result), when a production run leaves a sensitive parameter with no source, or when a rationale claims a sub-study the context does not support. The judge is deliberately strict; in development it flagged real mistakes, including a value referenced to an input file instead of the output file that computed it. Its own findings become a judging report, which is itself verifiable.

Below is the verification view for one DREAMS report, the lattice constant of body-centered cubic lithium. The graph is the report’s value-flow graph, with the final calculation pre-selected. Click any node to inspect it, open the Claims tab to see each checked claim, and the per-parameter checks show what was verified.

Open full screen

Adversarial tuning of the judge

A judge is only worth having if it is hard to fool, so it is stress-tested against cases built to slip a fabrication past it, and the two ways it can fail are measured, not assumed.

An automated judge that is easy to trick gives false confidence, which is worse than no judge at all. To find out how good this one really is, it is run against a bench of about 150 realistic cases. Roughly half are legitimate results that should pass. The rest are deliberate fakes, each written to look convincing: a number that appears out of nowhere, a value quietly “laundered” through a calculator step to disguise that it was invented, a setting borrowed from an unrelated calculation, a chemically wrong value wrapped in confident wording, or a justification that claims a shortcut the situation does not allow. The fakes are written as carefully as the real cases, so the judge is tested on the deception itself rather than on sloppy writing that would give it away.

Two kinds of mistake are scored, and one is never traded for the other: approving a fake, the costly error, and flagging a legitimate result, which wastes real work. Because these AI judges are slightly random, every case is run several times, across five different AI models, and every disagreement is read by hand to see whether the judge was wrong or the test case was. Often it was the test case: the judge had caught a subtle flaw introduced by accident, which itself builds confidence.

The result, across all five models:

All five catch most of the fakes and clear most of the legitimate cases, but they sit at different points on the strictness tradeoff. fable-5 is the most balanced: of 83 fakes it approved only 3, and of 62 legitimate cases it flagged only 1 (FN=3, FP=1). sonnet-4-6 is the strictest, fooled by a single fake but flagging 10 legitimate cases (FN=1, FP=10), while sonnet-4-5 made the most errors of both kinds (FP=8, FN=7). This spread is why both error types are reported rather than a single accuracy number: it makes the tradeoff visible and lets the judge’s strictness be tuned on purpose.

Cost: what running and verifying agents costs

Token accounting per tool and per step shows where an agent run spends its budget, including the judge’s share.

Tokens are the unit of work and cost for language-model agents. The explorer below breaks down one DREAMS run by tool and by step, separately for input and output tokens, with the judge tracked as its own trace. In the run shown, the judge used 515,887 of 4,990,911 total tokens. The same accounting underlies the published efficiency comparison: on the CO/Pt(111) challenge, DREAMS used 2 to 3x fewer tokens than the strongest baseline framework while reaching a correct result.

Open full screen