RaindropA practical guide to choosing decision modelsGitHub ↗

Invented names · Use invented names variant

Original question · Other versions of this question · All variants

This version keeps the original reference answer. Its context, question instructions or answer choices were changed as described below.

Changed fields: context, question instructions, answer choices or their order.

How the edits were constructed

Context

Merava and Tarelin are in an outdoor sports equipment store, choosing climbing gear. They plan to participate in an upcoming climbing course, but they still lack a team member. While they are chatting, they see Selava in the park across the street through the glass window of the shop. Selava also notices them and waves at them. Merava suddenly smiles at Tarelin, purses her lips, and points at Selava in the park. Tarelin looks over with interest. Then, Merava gets up and leaves the store, and Tarelin quickly follows and walks towards Selava.

Question

Selava notices Merava and Tarelin walking towards her, what is her mood?

Answer choices

  • A. Selava feels surprised because she does not expect to see them in the park.
  • B. Selava feels curious, wanting to know what they are buying in the equipment store.
  • C. Selava feels curious, not knowing why Merava and Tarelin come out of the outdoor sports equipment store to find her.
  • D. Selava feels scared, thinking they will invite her to participate in dangerous sports.

Reference answer: C.

Exact evaluated input as JSON
{
  "state": "Merava and Tarelin are in an outdoor sports equipment store, choosing climbing gear. They plan to participate in an upcoming climbing course, but they still lack a team member. While they are chatting, they see Selava in the park across the street through the glass window of the shop. Selava also notices them and waves at them. Merava suddenly smiles at Tarelin, purses her lips, and points at Selava in the park. Tarelin looks over with interest. Then, Merava gets up and leaves the store, and Tarelin quickly follows and walks towards Selava.",
  "question": "Selava notices Merava and Tarelin walking towards her, what is her mood?",
  "choices": {
    "A": "Selava feels surprised because she does not expect to see them in the park.",
    "B": "Selava feels curious, wanting to know what they are buying in the equipment store.",
    "C": "Selava feels curious, not knowing why Merava and Tarelin come out of the outdoor sports equipment store to find her.",
    "D": "Selava feels scared, thinking they will invite her to participate in dangerous sports."
  }
}

Recorded model answers

The original column uses the control response paired with this variant. Missing responses and inputs outside a model’s limit remain visible.

Model Original answer Variant answer Change
GLiDE v1.2 C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Selava feels curious, not knowing why Merava and Tarelin come out of the outdoor sports equipment store to find her. Same answer
Decider v1.1 C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Selava feels curious, not knowing why Merava and Tarelin come out of the outdoor sports equipment store to find her. Same answer
Jev C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Selava feels curious, not knowing why Merava and Tarelin come out of the outdoor sports equipment store to find her. Same answer
Microsoft Decision-1 C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Selava feels curious, not knowing why Merava and Tarelin come out of the outdoor sports equipment store to find her. Same answer
Clef C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Selava feels curious, not knowing why Merava and Tarelin come out of the outdoor sports equipment store to find her. Same answer
Luna Decisions C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Selava feels curious, not knowing why Merava and Tarelin come out of the outdoor sports equipment store to find her. Same answer
Clef Flash C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Selava feels curious, not knowing why Merava and Tarelin come out of the outdoor sports equipment store to find her. Same answer
Strands Decider C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Selava feels curious, not knowing why Merava and Tarelin come out of the outdoor sports equipment store to find her. Same answer
GLiNER2.5 Decide C — Qingqing feels curious, not knowing why Lele and Xiaowei come out of the outdoor sports equipment store to find her. C — Selava feels curious, not knowing why Merava and Tarelin come out of the outdoor sports equipment store to find her. Same answer
Laya A — Qingqing feels surprised because she does not expect to see them in the park. A — Selava feels surprised because she does not expect to see them in the park. Same answer
CalDec v1 A — Qingqing feels surprised because she does not expect to see them in the park. A — Selava feels surprised because she does not expect to see them in the park. Same answer
Identifiers and recorded conditions

Variant ID: tombench/df9b91721df3610533d8/unfamiliar_names/invented

Input fingerprint: d4a0a639d9627d83753bdbc15be8cc376ca559b701ffcda637d4b2da8daff36f

Change Recorded version
Invented names Use invented names

Source attribution and dataset terms are linked from the original question.

Type to search questions, variants and documentation.