| Baseline | Model | Marks | Success | Reward | Evidence | Invalid | Steps | Errors |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| scripted | deterministic/scripted | 93.7 | 1.00 | 0.946 | 1.00 | 0.00 | 4.47 | 0 |
| frontier | openai/gpt-5.5 | 67.7 | 0.73 | 0.603 | 0.83 | 0.01 | 6.40 | 1 |
| react | openai/gpt-5-mini | 21.5 | 0.00 | 0.040 | 0.81 | 0.15 | 4.40 | 13 |
| prompting | openai/gpt-5-mini | 17.7 | 0.00 | 0.000 | 0.63 | 0.00 | 3.73 | 12 |
| open_source | ibm-granite/granite-4.1-8b | 16.6 | 0.00 | 0.010 | 0.57 | 0.00 | 8.20 | 0 |
| random | deterministic/random | 5.6 | 0.00 | 0.001 | 0.08 | 0.21 | 3.69 | 0 |