RaindropA practical guide to choosing decision modelsGitHub ↗

Information seeking · question 62

Back to Information seeking · All capabilities

Question ID: questbench/8b4a1753193ce9964698
Source: QuestBench (Google DeepMind)
Input format: native choices

Explore this question’s altered versions

Context

Given a 7-day week, Alex tutors for hours each day. She charges $12 per day regardless of the number of hours she tutors. How much does Alex charge for weeks of tutoring?

Question

Which missing variable should be asked about to solve this problem?

Answer choices

  • B: Number of weeks
  • C: Hours Alex tutors each day

Reference answer: Number of weeks (B)

Exact evaluated input as JSON
{
  "state": "Given a 7-day week, Alex tutors for hours each day. She charges $12 per day regardless of the number of hours she tutors. How much does Alex charge for weeks of tutoring?",
  "question": "Which missing variable should be asked about to solve this problem?",
  "choices": {
    "B": "Number of weeks",
    "C": "Hours Alex tutors each day"
  }
}

Recorded model answers

Model Answer Outcome
Astra Number of weeks Correct
Opus 5.5 Number of weeks Correct
GLiDE v1.2 Number of weeks Correct
Decider v1.1 Number of weeks Correct
Jev Number of weeks Correct
Microsoft Decision-1 Number of weeks Correct
Clef Number of weeks Correct
Luna Decisions Number of weeks Correct
Clef Flash Number of weeks Correct
Strands Decider Number of weeks Correct
GLiNER2.5 Decide Hours Alex tutors each day Incorrect
Laya Number of weeks Correct
CalDec v1 Number of weeks Correct

Source and sharing

Attribution: Google LLC; QuestBench authors.
Terms: CC-BY-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "extracted/questbench.tar.gz/questbench_data/GSM-Q/gsm_verbal_full.csv",
    "record": "20351",
    "kind": "source_file_record"
  },
  "source_revision": null,
  "input_hash": "d9f79b348e864f9dc7e63baa36e8df11b2db9db49e71443ef6b30a650ddc1c93"
}

Type to search questions, variants and documentation.