Physical cognition · question 12
Back to Physical cognition · All capabilities
Question ID: mmlu/fe496c5645f5c78aad17
Source: MMLU
Input format: native choices
Explore this question’s altered versions
Question
A toy car and a toy truck collide. If the toy truck’s mass is double the toy car’s mass, then, compared to the acceleration of the truck, the acceleration of the car during the collision will be
Answer choices
- A: double the magnitude and in the same direction
- B: double the magnitude and in the opposite direction
- C: half the magnitude and in the same direction
- D: half the magnitude and in the opposite direction
Reference answer: double the magnitude and in the opposite direction (B)
Exact evaluated input as JSON
{
"state": "",
"question": "A toy car and a toy truck collide. If the toy truck’s mass is double the toy car’s mass, then, compared to the acceleration of the truck, the acceleration of the car during the collision will be",
"choices": {
"A": "double the magnitude and in the same direction",
"B": "double the magnitude and in the opposite direction",
"C": "half the magnitude and in the same direction",
"D": "half the magnitude and in the opposite direction"
}
}
Recorded model answers
| Model | Answer | Outcome |
|---|---|---|
| Astra | double the magnitude and in the opposite direction | Correct |
| Opus 5.5 | double the magnitude and in the opposite direction | Correct |
| GLiDE v1.2 | double the magnitude and in the opposite direction | Correct |
| Decider v1.1 | double the magnitude and in the opposite direction | Correct |
| Jev | double the magnitude and in the opposite direction | Correct |
| Microsoft Decision-1 | double the magnitude and in the opposite direction | Correct |
| Clef | double the magnitude and in the opposite direction | Correct |
| Luna Decisions | double the magnitude and in the opposite direction | Correct |
| Clef Flash | double the magnitude and in the opposite direction | Correct |
| Strands Decider | half the magnitude and in the same direction | Incorrect |
| GLiNER2.5 Decide | double the magnitude and in the opposite direction | Correct |
| Laya | double the magnitude and in the same direction | Incorrect |
| CalDec v1 | double the magnitude and in the same direction | Incorrect |
Source and sharing
Attribution: Dan Hendrycks et al.; MMLU authors.
Terms: MIT. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.
Exact source locator and input fingerprint
{
"source_locator": {
"path": "all/test-00000-of-00001.parquet",
"record": "4813",
"kind": "source_file_record"
},
"source_revision": "c30699e8356da336a370243923dbaf21066bb9fe",
"input_hash": "d560e0b63d960089bbf3bb48ef9d42aed485ab77695816e2df79d9b16b114bc8"
}