RaindropA practical guide to choosing decision modelsGitHub ↗

Social cognition · question 21

Back to Social cognition · All capabilities

Question ID: tombench/762d2606e31af0a83b24
Source: ToMBench
Input format: native choices

Explore this question’s altered versions

Context

Xiao Ming wins the championship, but does not mention that Xiao Gang makes a big contribution to Xiao Ming's success.

Question

What kind of emotion does Xiao Gang have?

Answer choices

  • A: Thrilled
  • B: Angry
  • C: Embarrassed
  • D: Regretful

Reference answer: Angry (B)

Exact evaluated input as JSON
{
  "state": "Xiao Ming wins the championship, but does not mention that Xiao Gang makes a big contribution to Xiao Ming's success.",
  "question": "What kind of emotion does Xiao Gang have?",
  "choices": {
    "A": "Thrilled",
    "B": "Angry",
    "C": "Embarrassed",
    "D": "Regretful"
  }
}

Recorded model answers

Model Answer Outcome
Astra Angry Correct
Opus 5.5 Angry Correct
GLiDE v1.2 Angry Correct
Decider v1.1 Angry Correct
Jev Angry Correct
Microsoft Decision-1 Angry Correct
Clef Angry Correct
Luna Decisions Angry Correct
Clef Flash Angry Correct
Strands Decider Embarrassed Incorrect
GLiNER2.5 Decide Thrilled Incorrect
Laya Thrilled Incorrect
CalDec v1 Thrilled Incorrect

Source and sharing

Attribution: Zhuang Chen et al.; ToMBench authors.
Terms: MIT. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "data/Discrepant Emotions.jsonl",
    "record": "5",
    "kind": "source_file_record"
  },
  "source_revision": "4f491f0784ed31ef93a8837615fbc48f885cf78d",
  "input_hash": "8bd980f65aa74f987e87f386e820d4697b86dc02ec45eb1567d23a4a783baf80"
}

Type to search questions, variants and documentation.