RaindropA practical guide to choosing decision modelsGitHub ↗

Information seeking · question 33

Back to Information seeking · All capabilities

Question ID: questbench/3803a1a4e456b40f7f08
Source: QuestBench (Google DeepMind)
Input format: native choices

Explore this question’s altered versions

Context

A train travels between different cities. It goes 75 miles from the first city to the second city, 100 miles from the second city to the third city, and 50 miles less than that combined distance to go from the third city to the first city. It does this trip 3 times a day. The train needs service so how many days does it go between services?

Question

Which missing variable should be asked about to solve this problem?

Answer choices

  • K: Required train service(miles)
  • A: Number of cities
  • F: Number of hours for each trip

Reference answer: Required train service(miles) (K)

Exact evaluated input as JSON
{
  "state": "A train travels between different cities. It goes 75 miles from the first city to the second city, 100 miles from the second city to the third city, and 50 miles less than that combined distance to go from the third city to the first city. It does this trip 3 times a day. The train needs service so how many days does it go between services?",
  "question": "Which missing variable should be asked about to solve this problem?",
  "choices": {
    "K": "Required train service(miles)",
    "A": "Number of cities",
    "F": "Number of hours for each trip"
  }
}

Recorded model answers

Model Answer Outcome
Astra Required train service(miles) Correct
Opus 5.5 Required train service(miles) Correct
GLiDE v1.2 Required train service(miles) Correct
Decider v1.1 Required train service(miles) Correct
Jev Required train service(miles) Correct
Microsoft Decision-1 Required train service(miles) Correct
Clef Required train service(miles) Correct
Luna Decisions Required train service(miles) Correct
Clef Flash Required train service(miles) Correct
Strands Decider Required train service(miles) Correct
GLiNER2.5 Decide Required train service(miles) Correct
Laya Required train service(miles) Correct
CalDec v1 Required train service(miles) Correct

Source and sharing

Attribution: Google LLC; QuestBench authors.
Terms: CC-BY-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "extracted/questbench.tar.gz/questbench_data/GSM-Q/gsm_verbal_full.csv",
    "record": "11898",
    "kind": "source_file_record"
  },
  "source_revision": null,
  "input_hash": "69b20bddf2b95b6e23649843bd4dee8405422a5bc4477b229422d25c69034e54"
}

Type to search questions, variants and documentation.