Announcing onepot-Bench 0.5 — can AI interpret mass spectra?

Today we are releasing onepot-Bench 0.5, a benchmark for the step that comes right after reaction execution: looking at the analytical chemistry data and deciding what happened. It has two parts:

Both use raw acquisitions from our own reaction screens, which have never been published. The full results are on our benchmarks page.

LC–MS (liquid chromatography–mass spectrometry) is a Swiss Army knife of analytical chemistry: The method relies on separating components of a mixture with a chromatography column and constantly analyzing them using a mass-spectrometer (essentially a molecular scale that accurately weighs out individual molecules of the mixture). By combining the weight of each molecule together with its retention time (that is, how long it stayed on the column), we can identify the structure of the molecules making up a mixture.

Our fab runs a ton of reactions per day and collects their LC–MS data. Afterwards, somebody has to read the trace. Did the product form? Is that molecule at the right mass actually our target, or something that happens to weigh the same? What else is in there? A single chemist can only look at about 8–16 wells per day.

onepot-Bench 0 asked whether language models know enough chemistry to make synthesis decisions. Bench 0.5 asks whether they can decide on their own whether those decisions worked.

Setup

Every agent gets the acquisition file for one well, the reaction description (structures, formulas, reagents, equivalents) and a sandbox where it can write and run code.

LCMSFind (59 wells, 17 reaction classes). The question is yes or no: was the intended product formed? The question sounds easier than it is — mass collisions (the molecular scales have limited precision!), low intensity, and unusual adducts can affect the results. To score well on LCMSFind, models must combine a mixture of data analysis, synthetic chemistry intuition, and knowledge about mass spectrometry.

LCMSFind is split into a stratified 20-well subset, which is enriched for outcomes closer to the decision boundary, and an (easier) extension subset (39 wells) without near-threshold enrichment.

LCMSElucidate (20 wells) covers Buchwald–Hartwig aminations, Ugi couplings and N-alkylations. For each reaction type, the agent gets a list of things that could have happened, in plain language ("under- or over-alkylation", for example), and has to work out what each would look like in the spectrum, then find it or rule it out. This is a proxy for the question "tell me everything that happened in this well".

The reference answers come from a fixed set of detection rules, with disputed wells adjudicated by a chemist looking at the chromatograms and spectra. We are not publishing the prompts, the rule thresholds or the full outcome lists to prevent leaking the answers to upcoming model generations.

What we found

Confirming a product is within reach. With a more detailed prompt, the best models get 92% of LCMSFind wells right: Gemini 4 Argon and Grok 4.6 on the 20-well set, Claude Opus 5, Opus 5.5 and Grok 4.7 on the 39-well set. With the more open-ended prompt shown in the figure, the best score on the 20-well set is Opus 5’s 82%. Opus 5 is also the most consistent, at 0.87 averaged over both prompts and both well sets.

How you ask matters as much as which model you ask. We ran LCMSFind with two prompts. The first, more open-ended, gives a simple, sufficient rule to qualify peaks but it does not spell out specific reasons to disqualify a peak; this is closer to how you might give the task to an expert in analytical chemistry. The second prompt gives additional hints on how spectra should be interpreted, containing some reminders and reasons to disqualify peaks that a chemist less familiar with mass spectrometry might need to succeed at the task. On the 20-well set, average accuracy goes from 0.65 to 0.82 between them. Some models do not need additional hints in the prompt: Opus 5 and Gemini 4 Argon are strong either way. Others change dramatically. Without the extra guidance, GPT-5.6 Terra and Sol call nearly everything a hit; with it, Terra goes from 0.50 accuracy to 0.87. For scale: a single-threshold detector we calibrated on wells outside the benchmark beats 16 of the 22 models on that set under the first prompt, and only two under the second.

The most common mistake is the right mass on the wrong molecule. Under the more open-ended prompt, almost all errors are false positives, and two thirds of those are peaks at the product's mass whose isotope pattern doesn't match the product's formula. A seasoned chemist would notice the mismatch immediately. With the explicit prompt that error mostly goes away. Models instead exhibit a new error: they start rejecting real products that are clearly there.

Working out what else happened is harder. On LCMSElucidate the best model, Grok 4.6, reaches an F1 of 0.79, followed by GLM 5.3 Flash at 0.77 (on the 16 wells it answered within the 60-minute wall time) and Grok 4.7 at 0.76. Starting materials, whose structures are in the prompt, are found relatively reliably. On the other hand, byproducts and reagent-derived species, which must be constructed from the reaction context, are harder to find. Meanwhile, nearly half of all wrong answers are species reported as present without corresponding LC–MS evidence at all. Sometimes the mistake happens before the spectrum is even opened: one run got a phosphine ligand's formula wrong by 175 Da, then searched for both the ligand and its oxide at the wrong masses.

Reading the results

The benchmarks page has all 22 models on both evaluations: accuracy, sensitivity and specificity under both prompts, F1 by species kind, everything plotted against cost, and the calibrated detector drawn in for reference.

Outlook

onepot-Bench 0.5 is a step towards more autonomous chemistry lab agents. While onepot-Bench 0 was purely chemistry question-answer, onepot-Bench 0.5 required exploration and data analysis. Both chemistry knowledge and analytical capability are prerequisites to autonomously running experiments in the lab. Subsequent evaluations will probe models’ abilities (or lack thereof) to investigate research questions on their own.

If you want to run onepot-Bench on your model, or talk through the methods or results, write to us at research@onepot.ai .