onepot-Bench 0.5
Language-model agents reading LC–MS reaction screens: deciding whether the intended product formed, and what else occurred. Built on unpublished screening acquisitions from onepot's own lab, judged against chemist-adjudicated references.
Product confirmation. The agent receives a well's LC–MS acquisition file, the intended product, the starting materials and reagents, and decides whether the intended product formed. It works with code execution, extracting ion traces and weighing the evidence itself.
20 wells, half of them with the product, enriched for hard cases. Each model is scored with a more open-ended prompt and with one giving more guidance on judging detection. The dashed line is a simple detection rule, a level any model should clear.
| Model | Sensitivity | Specificity | Accuracy | ¢ / well |
|---|---|---|---|---|
| Gemini 4 Argon | 100 | 83 | 91.7 | 34.2 |
| Grok 4.6 | 93 | 90 | 91.7 | 33.6 |
| Claude Opus 5 | 87 | 93 | 90.0 | 53.8 |
| GLM 5.3 Flash | 93 | 87 | 90.0 | 2.2 |
| Grok 4.7 | 83 | 93 | 88.3 | 70.1 |
| Gemini 3.7 Flash | 93 | 80 | 86.7 | 14.2 |
| Claude Opus 5.5 | 80 | 93 | 86.7 | 15.1 |
| GPT-5.6 Terra | 87 | 87 | 86.7 | 12.2 |
| Claude Haiku 5.5 | 77 | 93 | 85.0 | 0.6 |
| DeepSeek V4.1 Flash | 73 | 97 | 85.0 | 11.8 |
| Claude Fable 5.1 | 77 | 90 | 83.3 | 38.6 |
| Gemini 3.8 Flash | 93 | 70 | 81.7 | 32.5 |
| Claude Sonnet 5.5 | 70 | 93 | 81.7 | 4.8 |
| Claude Sonnet 5 | 83 | 80 | 81.7 | 19.9 |
| GPT-5.6 Sol | 97 | 67 | 81.7 | 25.3 |
| GPT-5.6 Luna | 100 | 60 | 80.0 | 1.4 |
| Gemini 3.1 Pro | 63 | 90 | 76.7 | 39.1 |
| DeepSeek V4 Flash | 60 | 90 | 75.0 | 2.1 |
| GPT-6.1 Sol | 53 | 97 | 75.0 | 9.3 |
| GPT-6 Astra | 50 | 100 | 75.0 | 61.3 |
| Calibrated rule (baseline) | 75.0 | |||
| GPT-6 Luna | 57 | 80 | 68.3 | 0.8 |
| Claude Haiku 4.5 | 70 | 60 | 65.0 | 17.1 |
Values are percentages.