Information seeking · question 49
Back to Information seeking · All capabilities
Question ID: questbench/442e591f009035b4865a
Source: QuestBench (Google DeepMind)
Input format: native choices
Explore this question’s altered versions
Context
At a salon, a basic manicure starts at $35 and a pedicure starts at $40. The salon offers a 20% discount if you get both. Karen decides to get both, and she also wants to add nail art on each of her fingers, which costs per nail. However, the salon has a special promotion where if you get nail art on all your fingers, you get a discount on nail art for your toes. But Karen decides not to get any nail art on her toes. How much would she pay in total?
Question
Which missing variable should be asked about to solve this problem?
Answer choices
- G: Cost of nail art per nail
- J: Discount on nail art for toes
Reference answer: Cost of nail art per nail (G)
Exact evaluated input as JSON
{
"state": "At a salon, a basic manicure starts at $35 and a pedicure starts at $40. The salon offers a 20% discount if you get both. Karen decides to get both, and she also wants to add nail art on each of her fingers, which costs per nail. However, the salon has a special promotion where if you get nail art on all your fingers, you get a discount on nail art for your toes. But Karen decides not to get any nail art on her toes. How much would she pay in total?",
"question": "Which missing variable should be asked about to solve this problem?",
"choices": {
"G": "Cost of nail art per nail",
"J": "Discount on nail art for toes"
}
}
Recorded model answers
| Model | Answer | Outcome |
|---|---|---|
| Astra | Cost of nail art per nail | Correct |
| Opus 5.5 | Cost of nail art per nail | Correct |
| GLiDE v1.2 | Cost of nail art per nail | Correct |
| Decider v1.1 | Cost of nail art per nail | Correct |
| Jev | Cost of nail art per nail | Correct |
| Microsoft Decision-1 | Cost of nail art per nail | Correct |
| Clef | Cost of nail art per nail | Correct |
| Luna Decisions | Cost of nail art per nail | Correct |
| Clef Flash | Cost of nail art per nail | Correct |
| Strands Decider | Cost of nail art per nail | Correct |
| GLiNER2.5 Decide | Cost of nail art per nail | Correct |
| Laya | Cost of nail art per nail | Correct |
| CalDec v1 | Cost of nail art per nail | Correct |
Source and sharing
Attribution: Google LLC; QuestBench authors.
Terms: CC-BY-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.
Exact source locator and input fingerprint
{
"source_locator": {
"path": "extracted/questbench.tar.gz/questbench_data/GSM-Q/gsm_verbal_full.csv",
"record": "8931",
"kind": "source_file_record"
},
"source_revision": null,
"input_hash": "c17569b9894fef1c9aee34a0a130a6bf3ee835bca8efb6e9764fb6dae83e61d7"
}