RaindropA practical guide to choosing decision modelsGitHub ↗

Information seeking · question 19

Back to Information seeking · All capabilities

Question ID: questbench/bb7fe83c58001aaa610e
Source: QuestBench (Google DeepMind)
Input format: native choices

Explore this question’s altered versions

Context

A simple folding newspaper or tabloid can be made by folding a piece of paper vertically and unfolding it. Then, say, page 1 is printed on the left back, page 2 is printed on the left front, and then, perhaps page 32 is printed on the right back, and page 31 is printed on the right front. How many pieces of paper would be used in a -page tabloid 2 inches long by 3 inches wide?

Question

Which missing variable should be asked about to solve this problem?

Answer choices

  • P: Total number of pages in the tabloid
  • L: Length of each piece (in inches)
  • W: Width of each piece (in inches)

Reference answer: Total number of pages in the tabloid (P)

Exact evaluated input as JSON
{
  "state": "A simple folding newspaper or tabloid can be made by folding a piece of paper vertically and unfolding it. Then, say, page 1 is printed on the left back, page 2 is printed on the left front, and then, perhaps page 32 is printed on the right back, and page 31 is printed on the right front. How many pieces of paper would be used in a -page tabloid 2 inches long by 3 inches wide?",
  "question": "Which missing variable should be asked about to solve this problem?",
  "choices": {
    "P": "Total number of pages in the tabloid",
    "L": "Length of each piece (in inches)",
    "W": "Width of each piece (in inches)"
  }
}

Recorded model answers

Model Answer Outcome
Astra Total number of pages in the tabloid Correct
Opus 5.5 Total number of pages in the tabloid Correct
GLiDE v1.2 Total number of pages in the tabloid Correct
Decider v1.1 Total number of pages in the tabloid Correct
Jev Total number of pages in the tabloid Correct
Microsoft Decision-1 Total number of pages in the tabloid Correct
Clef Total number of pages in the tabloid Correct
Luna Decisions Total number of pages in the tabloid Correct
Clef Flash Total number of pages in the tabloid Correct
Strands Decider Total number of pages in the tabloid Correct
GLiNER2.5 Decide Length of each piece (in inches) Incorrect
Laya Length of each piece (in inches) Incorrect
CalDec v1 Length of each piece (in inches) Incorrect

Source and sharing

Attribution: Google LLC; QuestBench authors.
Terms: CC-BY-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "extracted/questbench.tar.gz/questbench_data/GSM-Q/gsm_verbal_full.csv",
    "record": "17792",
    "kind": "source_file_record"
  },
  "source_revision": null,
  "input_hash": "e0bd6335dd5acecbf60fdc65b9626ef3856f2a84ce0e9d268b4cf758a99ca43a"
}

Type to search questions, variants and documentation.