RaindropA practical guide to choosing decision modelsGitHub ↗

Communication · question 3

Back to Communication · All capabilities

Question ID: tombench/7dd03294e8dbf4fb23f4
Source: ToMBench
Input format: native choices

Explore this question’s altered versions

Context

One day, the principal, after correcting the homework, finds that the students' grades are not very good, so he writes in the WeChat group of school teachers: "Teachers, I look at the students' homework today, I really don't think they are so modest, no one does it all right."

Question

What is the real meaning when the principal says this?

Answer choices

  • A: The principal thinks the students are very modest
  • B: The principal is reminding teachers to pay attention to improving teaching methods
  • C: The principal pays attention to and cares about students' homework
  • D: The principal feels regret because no one can do all the homework right

Reference answer: The principal is reminding teachers to pay attention to improving teaching methods (B)

Exact evaluated input as JSON
{
  "state": "One day, the principal, after correcting the homework, finds that the students' grades are not very good, so he writes in the WeChat group of school teachers: \"Teachers, I look at the students' homework today, I really don't think they are so modest, no one does it all right.\"",
  "question": "What is the real meaning when the principal says this?",
  "choices": {
    "A": "The principal thinks the students are very modest",
    "B": "The principal is reminding teachers to pay attention to improving teaching methods",
    "C": "The principal pays attention to and cares about students' homework",
    "D": "The principal feels regret because no one can do all the homework right"
  }
}

Recorded model answers

Model Answer Outcome
Astra The principal is reminding teachers to pay attention to improving teaching methods Correct
Opus 5.5 The principal is reminding teachers to pay attention to improving teaching methods Correct
GLiDE v1.2 The principal is reminding teachers to pay attention to improving teaching methods Correct
Decider v1.1 The principal is reminding teachers to pay attention to improving teaching methods Correct
Jev The principal is reminding teachers to pay attention to improving teaching methods Correct
Microsoft Decision-1 The principal is reminding teachers to pay attention to improving teaching methods Correct
Clef The principal is reminding teachers to pay attention to improving teaching methods Correct
Luna Decisions The principal is reminding teachers to pay attention to improving teaching methods Correct
Clef Flash The principal is reminding teachers to pay attention to improving teaching methods Correct
Strands Decider The principal feels regret because no one can do all the homework right Incorrect
GLiNER2.5 Decide The principal thinks the students are very modest Incorrect
Laya The principal thinks the students are very modest Incorrect
CalDec v1 The principal feels regret because no one can do all the homework right Incorrect

Source and sharing

Attribution: Zhuang Chen et al.; ToMBench authors.
Terms: MIT. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "data/Hinting Task Test.jsonl",
    "record": "28",
    "kind": "source_file_record"
  },
  "source_revision": "4f491f0784ed31ef93a8837615fbc48f885cf78d",
  "input_hash": "a684646ef1f3b50f2a4b22b034d0e76ec69e62a3ddac0171d2bf5891cf357bec"
}

Type to search questions, variants and documentation.