RaindropA practical guide to choosing decision modelsGitHub ↗

Information seeking · question 41

Back to Information seeking · All capabilities

Question ID: questbench/b34b77658c5be5e9f757
Source: QuestBench (Google DeepMind)
Input format: native choices

Explore this question’s altered versions

Context

There are forty apples in one box. Uncle Franky ordered two boxes of apples and some boxes of oranges. He is planning to pack the apples with few apples in one pack. How many packs of apples can he make with the two boxes of apples?

Question

Which missing variable should be asked about to solve this problem?

Answer choices

  • E: number of apples uncle franky packed in one pack
  • C: number of boxes of oranges ordered by uncle franky

Reference answer: number of apples uncle franky packed in one pack (E)

Exact evaluated input as JSON
{
  "state": "There are forty apples in one box. Uncle Franky ordered two boxes of apples and some boxes of oranges. He is planning to pack the apples with few apples in one pack. How many packs of apples can he make with the two boxes of apples?",
  "question": "Which missing variable should be asked about to solve this problem?",
  "choices": {
    "E": "number of apples uncle franky packed in one pack",
    "C": "number of boxes of oranges ordered by uncle franky"
  }
}

Recorded model answers

Model Answer Outcome
Astra number of apples uncle franky packed in one pack Correct
Opus 5.5 number of apples uncle franky packed in one pack Correct
GLiDE v1.2 number of apples uncle franky packed in one pack Correct
Decider v1.1 number of apples uncle franky packed in one pack Correct
Jev number of apples uncle franky packed in one pack Correct
Microsoft Decision-1 number of apples uncle franky packed in one pack Correct
Clef number of apples uncle franky packed in one pack Correct
Luna Decisions number of apples uncle franky packed in one pack Correct
Clef Flash number of apples uncle franky packed in one pack Correct
Strands Decider number of apples uncle franky packed in one pack Correct
GLiNER2.5 Decide number of apples uncle franky packed in one pack Correct
Laya number of apples uncle franky packed in one pack Correct
CalDec v1 number of apples uncle franky packed in one pack Correct

Source and sharing

Attribution: Google LLC; QuestBench authors.
Terms: CC-BY-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "extracted/questbench.tar.gz/questbench_data/GSM-Q/gsm_verbal_full.csv",
    "record": "10811",
    "kind": "source_file_record"
  },
  "source_revision": null,
  "input_hash": "b3bf2cca3ee25c71320ef7d4adbe0bb98311771a75b8b556617952171f148760"
}

Type to search questions, variants and documentation.