Information seeking · question 60
Back to Information seeking · All capabilities
Question ID: questbench/70f3e0ce25a345f3b019
Source: QuestBench (Google DeepMind)
Input format: native choices
Explore this question’s altered versions
Context
In a family, there are 2 brothers and 3 sisters. All sisters are the same age. One of the brothers is 12 years old, which is half the age of the older brother. The family also has a pet dog that is years old and a pet parrot that is years old. What is the total age of all these siblings, not including the ages of the pets?
Question
Which missing variable should be asked about to solve this problem?
Answer choices
- B: SECOND SISTER'S AGE
- F: PET DOG'S AGE
- G: PET PARROT'S AGE
Reference answer: SECOND SISTER'S AGE (B)
Exact evaluated input as JSON
{
"state": "In a family, there are 2 brothers and 3 sisters. All sisters are the same age. One of the brothers is 12 years old, which is half the age of the older brother. The family also has a pet dog that is years old and a pet parrot that is years old. What is the total age of all these siblings, not including the ages of the pets?",
"question": "Which missing variable should be asked about to solve this problem?",
"choices": {
"B": "SECOND SISTER'S AGE",
"F": "PET DOG'S AGE",
"G": "PET PARROT'S AGE"
}
}
Recorded model answers
| Model | Answer | Outcome |
|---|---|---|
| Astra | SECOND SISTER'S AGE | Correct |
| Opus 5.5 | SECOND SISTER'S AGE | Correct |
| GLiDE v1.2 | SECOND SISTER'S AGE | Correct |
| Decider v1.1 | SECOND SISTER'S AGE | Correct |
| Jev | SECOND SISTER'S AGE | Correct |
| Microsoft Decision-1 | SECOND SISTER'S AGE | Correct |
| Clef | SECOND SISTER'S AGE | Correct |
| Luna Decisions | SECOND SISTER'S AGE | Correct |
| Clef Flash | SECOND SISTER'S AGE | Correct |
| Strands Decider | SECOND SISTER'S AGE | Correct |
| GLiNER2.5 Decide | PET DOG'S AGE | Incorrect |
| Laya | PET DOG'S AGE | Incorrect |
| CalDec v1 | SECOND SISTER'S AGE | Correct |
Source and sharing
Attribution: Google LLC; QuestBench authors.
Terms: CC-BY-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.
Exact source locator and input fingerprint
{
"source_locator": {
"path": "extracted/questbench.tar.gz/questbench_data/GSM-Q/gsm_verbal_full.csv",
"record": "8369",
"kind": "source_file_record"
},
"source_revision": null,
"input_hash": "d4f2b5b8635d6d09577a245da1eb96a93ab865ecfa2f43ffedb4e0a936dad6d3"
}