RaindropA practical guide to choosing decision modelsGitHub ↗

Misleading details · Add an unverified claim that conflicts with the evidence variant

Original question · Other versions of this question · All variants

This version keeps the original reference answer. Its context, question instructions or answer choices were changed as described below.

Changed fields: context.

How the edits were constructed

Context

Lele and Xiaowei are in an outdoor sports equipment store, choosing climbing gear. They plan to participate in an upcoming climbing course, but they still lack a team member. While they are chatting, they see Qingqing in the park across the street through the glass window of the shop. Qingqing also notices them and waves at them. Lele suddenly smiles at Xiaowei, purses her lips, and points at Qingqing in the park. Xiaowei looks over with interest. Then, Lele gets up and leaves the store, and Xiaowei quickly follows and walks towards Qingqing.

A reader thinks: «Qingqing is scared that Lele and Xiaowei will invite her to dangerous sports.»

Question

Qingqing notices Lele and Xiaowei walking towards her, what is her mood?

Answer choices

  • A. Qingqing feels surprised because she does not expect to see them in the park.
  • B. Qingqing feels curious, wanting to know what they are buying in the equipment store.
  • C. Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her.
  • D. Qingqing feels scared, thinking they will invite her to participate in dangerous sports.

Reference answer: C.

Exact evaluated input as JSON
{
  "state": "Lele and Xiaowei are in an outdoor sports equipment store, choosing climbing gear. They plan to participate in an upcoming climbing course, but they still lack a team member. While they are chatting, they see Qingqing in the park across the street through the glass window of the shop. Qingqing also notices them and waves at them. Lele suddenly smiles at Xiaowei, purses her lips, and points at Qingqing in the park. Xiaowei looks over with interest. Then, Lele gets up and leaves the store, and Xiaowei quickly follows and walks towards Qingqing.\n\nA reader thinks: «Qingqing is scared that Lele and Xiaowei will invite her to dangerous sports.»",
  "question": "Qingqing notices Lele and Xiaowei walking towards her, what is her mood?",
  "choices": {
    "A": "Qingqing feels surprised because she does not expect to see them in the park.",
    "B": "Qingqing feels curious, wanting to know what they are buying in the equipment store.",
    "C": "Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her.",
    "D": "Qingqing feels scared, thinking they will invite her to participate in dangerous sports."
  }
}

Recorded model answers

The original column uses the control response paired with this variant. Missing responses and inputs outside a model’s limit remain visible.

Model Original answer Variant answer Change
GLiDE v1.2 C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. D — Qingqing feels scared, thinking they will invite her to participate in dangerous sports. Different answer
Decider v1.1 C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. Same answer
Jev C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. D — Qingqing feels scared, thinking they will invite her to participate in dangerous sports. Different answer
Microsoft Decision-1 C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. Same answer
Clef C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. Same answer
Luna Decisions C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. D — Qingqing feels scared, thinking they will invite her to participate in dangerous sports. Different answer
Clef Flash C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. D — Qingqing feels scared, thinking they will invite her to participate in dangerous sports. Different answer
Strands Decider C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. D — Qingqing feels scared, thinking they will invite her to participate in dangerous sports. Different answer
GLiNER2.5 Decide C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. D — Qingqing feels scared, thinking they will invite her to participate in dangerous sports. Different answer
Laya A — Qingqing feels surprised because she does not expect to see them in the park. D — Qingqing feels scared, thinking they will invite her to participate in dangerous sports. Different answer
CalDec v1 A — Qingqing feels surprised because she does not expect to see them in the park. D — Qingqing feels scared, thinking they will invite her to participate in dangerous sports. Different answer
Identifiers and recorded conditions

Variant ID: tombench/df9b91721df3610533d8/misleading_summary/misleading/clean-source_first_tombench_music_contexts_full_0155

Input fingerprint: 3d2508c21a1f99b0ff9160481524c8c265236914bb26700d36177978eed41bf9

Change Recorded version
Misleading details Add an unverified claim that conflicts with the evidence

Source attribution and dataset terms are linked from the original question.

Type to search questions, variants and documentation.