RaindropA practical guide to choosing decision modelsGitHub ↗

Information seeking · question 35

Back to Information seeking · All capabilities

Question ID: questbench/9bcd74a64d46e22b4182
Source: QuestBench (Google DeepMind)
Input format: native choices

Explore this question’s altered versions

Context

Aiden and 12 of his friends are going to see a film at the cinema, and meet up with 7 more friends there. They each save a seat and then buy enough drinks and snacks to fill the seats. Each seat has enough room to hold one person, two drinks, and three snacks. Additionally, each person decides to buy a souvenir keychain, but they later realize that the keychains are free today, so they don't have to spend any extra money on them. If drinks cost $2 each, how much money, in dollars, has the group spent overall on snacks and drinks?

Question

Which missing variable should be asked about to solve this problem?

Answer choices

  • F: Cost of one snack
  • L: Cost of each keychain

Reference answer: Cost of one snack (F)

Exact evaluated input as JSON
{
  "state": "Aiden and 12 of his friends are going to see a film at the cinema, and meet up with 7 more friends there. They each save a seat and then buy enough drinks and snacks to fill the seats. Each seat has enough room to hold one person, two drinks, and three snacks. Additionally, each person decides to buy a souvenir keychain, but they later realize that the keychains are free today, so they don't have to spend any extra money on them. If drinks cost $2 each, how much money, in dollars, has the group spent overall on snacks and drinks?",
  "question": "Which missing variable should be asked about to solve this problem?",
  "choices": {
    "F": "Cost of one snack",
    "L": "Cost of each keychain"
  }
}

Recorded model answers

Model Answer Outcome
Astra Cost of one snack Correct
Opus 5.5 Cost of one snack Correct
GLiDE v1.2 Cost of one snack Correct
Decider v1.1 Cost of one snack Correct
Jev Cost of one snack Correct
Microsoft Decision-1 Cost of one snack Correct
Clef Cost of one snack Correct
Luna Decisions Cost of one snack Correct
Clef Flash Cost of one snack Correct
Strands Decider Cost of one snack Correct
GLiNER2.5 Decide Cost of each keychain Incorrect
Laya Cost of one snack Correct
CalDec v1 Cost of one snack Correct

Source and sharing

Attribution: Google LLC; QuestBench authors.
Terms: CC-BY-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "extracted/questbench.tar.gz/questbench_data/GSM-Q/gsm_verbal_full.csv",
    "record": "9565",
    "kind": "source_file_record"
  },
  "source_revision": null,
  "input_hash": "52609fe30cd2ab8089a198bf0c6e9596141b5b03f17814ee6733a288c46e85a3"
}

Type to search questions, variants and documentation.