RaindropA practical guide to choosing decision modelsGitHub ↗

Communication · question 20

Back to Communication · All capabilities

Question ID: tombench/4e9d21af0d0ef82d333a
Source: ToMBench
Input format: native choices

Explore this question’s altered versions

Context

Anming's mother spends a long time making Anming's favorite fried fish and chips. But when she brings the food to Anming, Anming watches TV, she doesn't even look up, and doesn't say thank you. Anming's mother says angrily: "You are really polite!"

Question

Why does the mother say this?

Answer choices

  • A: She thinks Anming is a very polite child
  • B: She lies to encourage Anming to be a very polite child
  • C: She wants to mock her own food is not delicious.
  • D: She wants to mock Anming is a child without manners.

Reference answer: She wants to mock Anming is a child without manners. (D)

Exact evaluated input as JSON
{
  "state": "Anming's mother spends a long time making Anming's favorite fried fish and chips. But when she brings the food to Anming, Anming watches TV, she doesn't even look up, and doesn't say thank you. Anming's mother says angrily: \"You are really polite!\"",
  "question": "Why does the mother say this?",
  "choices": {
    "A": "She thinks Anming is a very polite child",
    "B": "She lies to encourage Anming to be a very polite child",
    "C": "She wants to mock her own food is not delicious.",
    "D": "She wants to mock Anming is a child without manners."
  }
}

Recorded model answers

Model Answer Outcome
Astra She wants to mock Anming is a child without manners. Correct
Opus 5.5 She wants to mock Anming is a child without manners. Correct
GLiDE v1.2 She wants to mock Anming is a child without manners. Correct
Decider v1.1 She wants to mock Anming is a child without manners. Correct
Jev She wants to mock Anming is a child without manners. Correct
Microsoft Decision-1 She wants to mock Anming is a child without manners. Correct
Clef She wants to mock Anming is a child without manners. Correct
Luna Decisions She wants to mock Anming is a child without manners. Correct
Clef Flash She wants to mock Anming is a child without manners. Correct
Strands Decider She wants to mock Anming is a child without manners. Correct
GLiNER2.5 Decide She thinks Anming is a very polite child Incorrect
Laya She thinks Anming is a very polite child Incorrect
CalDec v1 She thinks Anming is a very polite child Incorrect

Source and sharing

Attribution: Zhuang Chen et al.; ToMBench authors.
Terms: MIT. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "data/Strange Story Task.jsonl",
    "record": "3",
    "kind": "source_file_record"
  },
  "source_revision": "4f491f0784ed31ef93a8837615fbc48f885cf78d",
  "input_hash": "e490ea0bcc3a8847860b676d176009c86f5a0f2a7d244f6ae04a3090a1530820"
}

Type to search questions, variants and documentation.