RaindropA practical guide to choosing decision modelsGitHub ↗

Information seeking · question 49

Back to Information seeking · All capabilities

Question ID: questbench/442e591f009035b4865a
Source: QuestBench (Google DeepMind)
Input format: native choices

Explore this question’s altered versions

Context

At a salon, a basic manicure starts at $35 and a pedicure starts at $40. The salon offers a 20% discount if you get both. Karen decides to get both, and she also wants to add nail art on each of her fingers, which costs per nail. However, the salon has a special promotion where if you get nail art on all your fingers, you get a discount on nail art for your toes. But Karen decides not to get any nail art on her toes. How much would she pay in total?

Question

Which missing variable should be asked about to solve this problem?

Answer choices

  • G: Cost of nail art per nail
  • J: Discount on nail art for toes

Reference answer: Cost of nail art per nail (G)

Exact evaluated input as JSON
{
  "state": "At a salon, a basic manicure starts at $35 and a pedicure starts at $40. The salon offers a 20% discount if you get both. Karen decides to get both, and she also wants to add nail art on each of her fingers, which costs per nail. However, the salon has a special promotion where if you get nail art on all your fingers, you get a discount on nail art for your toes. But Karen decides not to get any nail art on her toes. How much would she pay in total?",
  "question": "Which missing variable should be asked about to solve this problem?",
  "choices": {
    "G": "Cost of nail art per nail",
    "J": "Discount on nail art for toes"
  }
}

Recorded model answers

Model Answer Outcome
Astra Cost of nail art per nail Correct
Opus 5.5 Cost of nail art per nail Correct
GLiDE v1.2 Cost of nail art per nail Correct
Decider v1.1 Cost of nail art per nail Correct
Jev Cost of nail art per nail Correct
Microsoft Decision-1 Cost of nail art per nail Correct
Clef Cost of nail art per nail Correct
Luna Decisions Cost of nail art per nail Correct
Clef Flash Cost of nail art per nail Correct
Strands Decider Cost of nail art per nail Correct
GLiNER2.5 Decide Cost of nail art per nail Correct
Laya Cost of nail art per nail Correct
CalDec v1 Cost of nail art per nail Correct

Source and sharing

Attribution: Google LLC; QuestBench authors.
Terms: CC-BY-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "extracted/questbench.tar.gz/questbench_data/GSM-Q/gsm_verbal_full.csv",
    "record": "8931",
    "kind": "source_file_record"
  },
  "source_revision": null,
  "input_hash": "c17569b9894fef1c9aee34a0a130a6bf3ee835bca8efb6e9764fb6dae83e61d7"
}

Type to search questions, variants and documentation.