Social cognition · question 49
Back to Social cognition · All capabilities
Question ID: tombench/df9b91721df3610533d8
Source: ToMBench
Input format: native choices
Explore this question’s altered versions
Context
Lele and Xiaowei are in an outdoor sports equipment store, choosing climbing gear. They plan to participate in an upcoming climbing course, but they still lack a team member. While they are chatting, they see Qingqing in the park across the street through the glass window of the shop. Qingqing also notices them and waves at them. Lele suddenly smiles at Xiaowei, purses her lips, and points at Qingqing in the park. Xiaowei looks over with interest. Then, Lele gets up and leaves the store, and Xiaowei quickly follows and walks towards Qingqing.
Question
Qingqing notices Lele and Xiaowei walking towards her, what is her mood?
Answer choices
- A: Qingqing feels surprised because she does not expect to see them in the park.
- B: Qingqing feels curious, wanting to know what they are buying in the equipment store.
- C: Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her.
- D: Qingqing feels scared, thinking they will invite her to participate in dangerous sports.
Reference answer: Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. (C)
Exact evaluated input as JSON
{
"state": "Lele and Xiaowei are in an outdoor sports equipment store, choosing climbing gear. They plan to participate in an upcoming climbing course, but they still lack a team member. While they are chatting, they see Qingqing in the park across the street through the glass window of the shop. Qingqing also notices them and waves at them. Lele suddenly smiles at Xiaowei, purses her lips, and points at Qingqing in the park. Xiaowei looks over with interest. Then, Lele gets up and leaves the store, and Xiaowei quickly follows and walks towards Qingqing.",
"question": "Qingqing notices Lele and Xiaowei walking towards her, what is her mood?",
"choices": {
"A": "Qingqing feels surprised because she does not expect to see them in the park.",
"B": "Qingqing feels curious, wanting to know what they are buying in the equipment store.",
"C": "Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her.",
"D": "Qingqing feels scared, thinking they will invite her to participate in dangerous sports."
}
}
Recorded model answers
| Model | Answer | Outcome |
|---|---|---|
| Astra | Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. | Correct |
| Opus 5.5 | Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. | Correct |
| GLiDE v1.2 | Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. | Correct |
| Decider v1.1 | Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. | Correct |
| Jev | Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. | Correct |
| Microsoft Decision-1 | Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. | Correct |
| Clef | Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. | Correct |
| Luna Decisions | Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. | Correct |
| Clef Flash | Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. | Correct |
| Strands Decider | Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. | Correct |
| GLiNER2.5 Decide | Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. | Correct |
| Laya | Qingqing feels surprised because she does not expect to see them in the park. | Incorrect |
| CalDec v1 | Qingqing feels surprised because she does not expect to see them in the park. | Incorrect |
Source and sharing
Attribution: Zhuang Chen et al.; ToMBench authors.
Terms: MIT. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.
Exact source locator and input fingerprint
{
"source_locator": {
"path": "data/Ambiguous Story Task.jsonl",
"record": "141",
"kind": "source_file_record"
},
"source_revision": "4f491f0784ed31ef93a8837615fbc48f885cf78d",
"input_hash": "57b169da7edc25b9ce786dc73fdc80c0aabeb4ef4de31e0ef0846700c59de5d5"
}