Information seeking · question 33
Back to Information seeking · All capabilities
Question ID: questbench/3803a1a4e456b40f7f08
Source: QuestBench (Google DeepMind)
Input format: native choices
Explore this question’s altered versions
Context
A train travels between different cities. It goes 75 miles from the first city to the second city, 100 miles from the second city to the third city, and 50 miles less than that combined distance to go from the third city to the first city. It does this trip 3 times a day. The train needs service so how many days does it go between services?
Question
Which missing variable should be asked about to solve this problem?
Answer choices
- K: Required train service(miles)
- A: Number of cities
- F: Number of hours for each trip
Reference answer: Required train service(miles) (K)
Exact evaluated input as JSON
{
"state": "A train travels between different cities. It goes 75 miles from the first city to the second city, 100 miles from the second city to the third city, and 50 miles less than that combined distance to go from the third city to the first city. It does this trip 3 times a day. The train needs service so how many days does it go between services?",
"question": "Which missing variable should be asked about to solve this problem?",
"choices": {
"K": "Required train service(miles)",
"A": "Number of cities",
"F": "Number of hours for each trip"
}
}
Recorded model answers
| Model | Answer | Outcome |
|---|---|---|
| Astra | Required train service(miles) | Correct |
| Opus 5.5 | Required train service(miles) | Correct |
| GLiDE v1.2 | Required train service(miles) | Correct |
| Decider v1.1 | Required train service(miles) | Correct |
| Jev | Required train service(miles) | Correct |
| Microsoft Decision-1 | Required train service(miles) | Correct |
| Clef | Required train service(miles) | Correct |
| Luna Decisions | Required train service(miles) | Correct |
| Clef Flash | Required train service(miles) | Correct |
| Strands Decider | Required train service(miles) | Correct |
| GLiNER2.5 Decide | Required train service(miles) | Correct |
| Laya | Required train service(miles) | Correct |
| CalDec v1 | Required train service(miles) | Correct |
Source and sharing
Attribution: Google LLC; QuestBench authors.
Terms: CC-BY-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.
Exact source locator and input fingerprint
{
"source_locator": {
"path": "extracted/questbench.tar.gz/questbench_data/GSM-Q/gsm_verbal_full.csv",
"record": "11898",
"kind": "source_file_record"
},
"source_revision": null,
"input_hash": "69b20bddf2b95b6e23649843bd4dee8405422a5bc4477b229422d25c69034e54"
}