RaindropA practical guide to choosing decision modelsGitHub ↗

Invented names · Use invented names variant

Original question · Other versions of this question · All variants

This version keeps the original reference answer. Its context, question instructions or answer choices were changed as described below.

Changed fields: context.

How the edits were constructed

Context

Tavren grabbed the football there. Nelora got the apple there. Tavren dropped the football there. Zorvik went to the garden. Zorvik went to the hallway. Zorvik journeyed to the office. Nelora put down the apple. Zorvik grabbed the apple there. Zorvik discarded the apple. Tavren grabbed the football there. Tavren left the football there. Tavren went back to the hallway. Vaelis travelled to the bathroom. Vaelis journeyed to the bedroom. Nelora took the apple there. Nelora put down the apple. Zorvik got the apple there. Zorvik put down the apple there. Tavren went to the bathroom. Zorvik travelled to the hallway.

Question

Where is the apple?

Answer choices

  • A. office
  • B. kitchen
  • C. hallway
  • D. bathroom
  • E. bedroom
  • F. garden

Reference answer: A.

Exact evaluated input as JSON
{
  "state": "Tavren grabbed the football there.\nNelora got the apple there.\nTavren dropped the football there.\nZorvik went to the garden.\nZorvik went to the hallway.\nZorvik journeyed to the office.\nNelora put down the apple.\nZorvik grabbed the apple there.\nZorvik discarded the apple.\nTavren grabbed the football there.\nTavren left the football there.\nTavren went back to the hallway.\nVaelis travelled to the bathroom.\nVaelis journeyed to the bedroom.\nNelora took the apple there.\nNelora put down the apple.\nZorvik got the apple there.\nZorvik put down the apple there.\nTavren went to the bathroom.\nZorvik travelled to the hallway.",
  "question": "Where is the apple? ",
  "choices": {
    "A": "office",
    "B": "kitchen",
    "C": "hallway",
    "D": "bathroom",
    "E": "bedroom",
    "F": "garden"
  }
}

Recorded model answers

The original column uses the control response paired with this variant. Missing responses and inputs outside a model’s limit remain visible.

Model Original answer Variant answer Change
GLiDE v1.2 A — office A — office Same answer
Decider v1.1 A — office A — office Same answer
Jev C — hallway C — hallway Same answer
Microsoft Decision-1 C — hallway C — hallway Same answer
Clef C — hallway C — hallway Same answer
Luna Decisions C — hallway C — hallway Same answer
Clef Flash C — hallway C — hallway Same answer
Strands Decider C — hallway C — hallway Same answer
GLiNER2.5 Decide C — hallway C — hallway Same answer
Laya A — office A — office Same answer
CalDec v1 C — hallway A — office Different answer
Identifiers and recorded conditions

Variant ID: babi/9c0cb3f01a3a7116384f/unfamiliar_names/invented

Input fingerprint: e5eb975be64b60a3f449462fce7577716e8ea17dd3ed567e15713441629f3cfd

Change Recorded version
Invented names Use invented names

Source attribution and dataset terms are linked from the original question.

Type to search questions, variants and documentation.