RaindropA practical guide to choosing decision modelsGitHub ↗

Information seeking · question 59

Back to Information seeking · All capabilities

Question ID: questbench/c50c11171baf0be610e9
Source: QuestBench (Google DeepMind)
Input format: native choices

Explore this question’s altered versions

Context

Vincent is planning a party and wants to buy flowers for decoration. He can buy flowers in packages of 3 or in packages of 2 for $1. He also notices that balloons are sold in packages of 5, but he decides not to buy any balloons. How much money does Vincent save by buying 18 flowers at the better price instead of the more expensive option?

Question

Which missing variable should be asked about to solve this problem?

Answer choices

  • B1: Cost of the first package option in dollars
  • G: Cost of the balloons are sold in packages of 5

Reference answer: Cost of the first package option in dollars (B1)

Exact evaluated input as JSON
{
  "state": "Vincent is planning a party and wants to buy flowers for decoration. He can buy flowers in packages of 3 or in packages of 2 for $1. He also notices that balloons are sold in packages of 5, but he decides not to buy any balloons. How much money does Vincent save by buying 18 flowers at the better price instead of the more expensive option?",
  "question": "Which missing variable should be asked about to solve this problem?",
  "choices": {
    "B1": "Cost of the first package option in dollars",
    "G": "Cost of the balloons are sold in packages of 5"
  }
}

Recorded model answers

Model Answer Outcome
Astra Cost of the first package option in dollars Correct
Opus 5.5 Cost of the first package option in dollars Correct
GLiDE v1.2 Cost of the first package option in dollars Correct
Decider v1.1 Cost of the first package option in dollars Correct
Jev Cost of the first package option in dollars Correct
Microsoft Decision-1 Cost of the first package option in dollars Correct
Clef Cost of the first package option in dollars Correct
Luna Decisions Cost of the first package option in dollars Correct
Clef Flash Cost of the first package option in dollars Correct
Strands Decider Cost of the first package option in dollars Correct
GLiNER2.5 Decide Cost of the balloons are sold in packages of 5 Incorrect
Laya Cost of the first package option in dollars Correct
CalDec v1 Cost of the first package option in dollars Correct

Source and sharing

Attribution: Google LLC; QuestBench authors.
Terms: CC-BY-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "extracted/questbench.tar.gz/questbench_data/GSM-Q/gsm_verbal_full.csv",
    "record": "6376",
    "kind": "source_file_record"
  },
  "source_revision": null,
  "input_hash": "1df2bd8cba2d96985c505879d4b84f0b91181fbffc3e5fd6f7b3c1939efbb980"
}

Type to search questions, variants and documentation.