RaindropA practical guide to choosing decision modelsGitHub ↗

Social cognition · question 5

Back to Social cognition · All capabilities

Question ID: tombench/8bbe5214e6341943d5c0
Source: ToMBench
Input format: native choices

Explore this question’s altered versions

Context

Chen Fei is playing games with his friend. When the game ends, Chen Fei wins and his friend loses. Chen Fei tries to hide his real feelings, because otherwise his friend may not play with him anymore.

Question

Why is Chen Fei's real feeling happy?

Answer choices

  • A: Because he wins the game
  • B: Because he wants to make his friend happy
  • C: Because he hides his real feelings
  • D: Because he likes to play games

Reference answer: Because he wins the game (A)

Exact evaluated input as JSON
{
  "state": "Chen Fei is playing games with his friend. When the game ends, Chen Fei wins and his friend loses. Chen Fei tries to hide his real feelings, because otherwise his friend may not play with him anymore.",
  "question": "Why is Chen Fei's real feeling happy?",
  "choices": {
    "A": "Because he wins the game",
    "B": "Because he wants to make his friend happy",
    "C": "Because he hides his real feelings",
    "D": "Because he likes to play games"
  }
}

Recorded model answers

Model Answer Outcome
Astra Because he wins the game Correct
Opus 5.5 Because he wins the game Correct
GLiDE v1.2 Because he wins the game Correct
Decider v1.1 Because he wins the game Correct
Jev Because he wins the game Correct
Microsoft Decision-1 Because he wins the game Correct
Clef Because he wins the game Correct
Luna Decisions Because he wins the game Correct
Clef Flash Because he wins the game Correct
Strands Decider Because he wins the game Correct
GLiNER2.5 Decide Because he hides his real feelings Incorrect
Laya Because he wins the game Correct
CalDec v1 Because he hides his real feelings Incorrect

Source and sharing

Attribution: Zhuang Chen et al.; ToMBench authors.
Terms: MIT. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "data/Hidden Emotions.jsonl",
    "record": "45",
    "kind": "source_file_record"
  },
  "source_revision": "4f491f0784ed31ef93a8837615fbc48f885cf78d",
  "input_hash": "074b50dc95669af0f977e9d4a60cd23226d3fa09ba20c31b46cc51985154b15f"
}

Type to search questions, variants and documentation.