RaindropA practical guide to choosing decision modelsGitHub ↗

Invented names · Use invented names variant

Original question · Other versions of this question · All variants

This version keeps the original reference answer. Its context, question instructions or answer choices were changed as described below.

Changed fields: context.

How the edits were constructed

Context

Zorvik moved to the garden. Zorvik went to the bedroom. Nelora went back to the kitchen. Vaelis moved to the garden. Tavren journeyed to the hallway. Zorvik journeyed to the kitchen. Zorvik journeyed to the office. Tavren travelled to the garden. Zorvik journeyed to the garden. Vaelis took the milk. Vaelis put down the milk. Nelora moved to the bathroom. Tavren grabbed the milk there. Zorvik moved to the bathroom. Tavren put down the milk. Zorvik got the football. Nelora travelled to the office. Vaelis got the milk. Vaelis left the milk. Vaelis journeyed to the kitchen. Zorvik left the football there. Zorvik went to the kitchen. Tavren went to the kitchen. Zorvik moved to the hallway. Tavren went to the hallway. Nelora journeyed to the bathroom. Nelora took the football. Nelora moved to the office. Tavren took the apple. Nelora discarded the football there. Vaelis travelled to the bedroom. Tavren put down the apple. Tavren took the apple. Tavren moved to the garden. Tavren moved to the hallway. Nelora grabbed the football. Nelora travelled to the bedroom. Tavren went to the garden. Vaelis journeyed to the bathroom. Nelora discarded the football there. Zorvik moved to the kitchen. Tavren discarded the apple.

Question

Where was the apple before the hallway?

Answer choices

  • A. bathroom
  • B. bedroom
  • C. office
  • D. kitchen
  • E. garden
  • F. hallway

Reference answer: E.

Exact evaluated input as JSON
{
  "state": "Zorvik moved to the garden.\nZorvik went to the bedroom.\nNelora went back to the kitchen.\nVaelis moved to the garden.\nTavren journeyed to the hallway.\nZorvik journeyed to the kitchen.\nZorvik journeyed to the office.\nTavren travelled to the garden.\nZorvik journeyed to the garden.\nVaelis took the milk.\nVaelis put down the milk.\nNelora moved to the bathroom.\nTavren grabbed the milk there.\nZorvik moved to the bathroom.\nTavren put down the milk.\nZorvik got the football.\nNelora travelled to the office.\nVaelis got the milk.\nVaelis left the milk.\nVaelis journeyed to the kitchen.\nZorvik left the football there.\nZorvik went to the kitchen.\nTavren went to the kitchen.\nZorvik moved to the hallway.\nTavren went to the hallway.\nNelora journeyed to the bathroom.\nNelora took the football.\nNelora moved to the office.\nTavren took the apple.\nNelora discarded the football there.\nVaelis travelled to the bedroom.\nTavren put down the apple.\nTavren took the apple.\nTavren moved to the garden.\nTavren moved to the hallway.\nNelora grabbed the football.\nNelora travelled to the bedroom.\nTavren went to the garden.\nVaelis journeyed to the bathroom.\nNelora discarded the football there.\nZorvik moved to the kitchen.\nTavren discarded the apple.",
  "question": "Where was the apple before the hallway? ",
  "choices": {
    "A": "bathroom",
    "B": "bedroom",
    "C": "office",
    "D": "kitchen",
    "E": "garden",
    "F": "hallway"
  }
}

Recorded model answers

The original column uses the control response paired with this variant. Missing responses and inputs outside a model’s limit remain visible.

Model Original answer Variant answer Change
GLiDE v1.2 E — garden E — garden Same answer
Decider v1.1 E — garden E — garden Same answer
Jev E — garden E — garden Same answer
Microsoft Decision-1 E — garden E — garden Same answer
Clef E — garden E — garden Same answer
Luna Decisions E — garden E — garden Same answer
Clef Flash F — hallway D — kitchen Different answer
Strands Decider E — garden E — garden Same answer
GLiNER2.5 Decide E — garden E — garden Same answer
Laya A — bathroom C — office Different answer
CalDec v1 C — office A — bathroom Different answer
Identifiers and recorded conditions

Variant ID: babi/0ebef6723a11f5e9f14c/unfamiliar_names/invented

Input fingerprint: be01442e1f92262024521ea492787b9ca380434f10fd27dc178513846e6946ac

Change Recorded version
Invented names Use invented names

Source attribution and dataset terms are linked from the original question.

Type to search questions, variants and documentation.