Social cognition · question 1
Back to Social cognition · All capabilities
Question ID: socialiqa/b0548ca305191ddbee55
Source: Social IQa
Input format: native choices
Explore this question’s altered versions
Context
Quinn learned to play drums in his spare time between volunteering for the homeless and studying for school.
Question
What kind of person is Quinn?
Answer choices
- A: lazy and selfish
- B: cares about the world
- C: hates other people
Reference answer: cares about the world (B)
Exact evaluated input as JSON
{
"state": "Quinn learned to play drums in his spare time between volunteering for the homeless and studying for school.",
"question": "What kind of person is Quinn?",
"choices": {
"A": "lazy and selfish",
"B": "cares about the world",
"C": "hates other people"
}
}
Recorded model answers
| Model | Answer | Outcome |
|---|---|---|
| Astra | cares about the world | Correct |
| Opus 5.5 | cares about the world | Correct |
| GLiDE v1.2 | cares about the world | Correct |
| Decider v1.1 | cares about the world | Correct |
| Jev | cares about the world | Correct |
| Microsoft Decision-1 | cares about the world | Correct |
| Clef | cares about the world | Correct |
| Luna Decisions | cares about the world | Correct |
| Clef Flash | cares about the world | Correct |
| Strands Decider | cares about the world | Correct |
| GLiNER2.5 Decide | cares about the world | Correct |
| Laya | cares about the world | Correct |
| CalDec v1 | cares about the world | Correct |
Source and sharing
Attribution: Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, Yejin Choi.
Terms: CC-BY-4.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.
Exact source locator and input fingerprint
{
"source_locator": {
"path": "extracted/socialiqa.zip/socialiqa-train-dev/train.jsonl",
"record": "26974",
"kind": "source_file_record"
},
"source_revision": null,
"input_hash": "d4da44feb1cad40442cef86bd6c22f73ee457365f9ac0493803bbdadcf3ad69e"
}