RaindropA practical guide to choosing decision modelsGitHub ↗

Extra events · Add 12 unrelated events variant

Original question · Other versions of this question · All variants

This version keeps the original reference answer. Its context, question instructions or answer choices were changed as described below.

Changed fields: context.

How the edits were constructed

Context

Read the full context

Nolan is the editor of a poetry catalog. Its entries are Virel, Zorven, Kesil, Nereth, Pavrel, and Sovrin. Nolan classifies Virel as a dialogue. Nolan classifies Zorven as a monologue. Nolan classifies Kesil as an elegy. Nolan classifies Nereth as a satire. Nolan classifies Pavrel as a fable. Nolan classifies Sovrin as a ballad. Nolan reclassifies Virel as a monologue. Nolan reclassifies Zorven as a dialogue. Nolan reclassifies Kesil as a ballad. Nolan reclassifies Nereth as a fable. Nolan reclassifies Pavrel as a satire. Nolan reclassifies Sovrin as an elegy.

John moved to the garden. John went to the bedroom. Mary went back to the kitchen. Sandra moved to the garden. Daniel journeyed to the hallway. John journeyed to the kitchen. John journeyed to the office. Daniel travelled to the garden. John journeyed to the garden. Sandra took the milk. Sandra put down the milk. Mary moved to the bathroom. Daniel grabbed the milk there. John moved to the bathroom. Daniel put down the milk. John got the football. Mary travelled to the office. Sandra got the milk. Sandra left the milk. Sandra journeyed to the kitchen. John left the football there. John went to the kitchen. Daniel went to the kitchen. John moved to the hallway. Daniel went to the hallway. Mary journeyed to the bathroom. Mary took the football. Mary moved to the office. Daniel took the apple. Mary discarded the football there. Sandra travelled to the bedroom. Daniel put down the apple. Daniel took the apple. Daniel moved to the garden. Daniel moved to the hallway. Mary grabbed the football. Mary travelled to the bedroom. Daniel went to the garden. Sandra journeyed to the bathroom. Mary discarded the football there. John moved to the kitchen. Daniel discarded the apple.

Question

Where was the apple before the hallway?

Answer choices

  • A. bathroom
  • B. bedroom
  • C. office
  • D. kitchen
  • E. garden
  • F. hallway

Reference answer: E.

Exact evaluated input as JSON
{
  "state": "Nolan is the editor of a poetry catalog. Its entries are Virel, Zorven, Kesil, Nereth, Pavrel, and Sovrin.\nNolan classifies Virel as a dialogue. Nolan classifies Zorven as a monologue. Nolan classifies Kesil as an elegy. Nolan classifies Nereth as a satire. Nolan classifies Pavrel as a fable. Nolan classifies Sovrin as a ballad. Nolan reclassifies Virel as a monologue. Nolan reclassifies Zorven as a dialogue. Nolan reclassifies Kesil as a ballad. Nolan reclassifies Nereth as a fable. Nolan reclassifies Pavrel as a satire. Nolan reclassifies Sovrin as an elegy.\n\nJohn moved to the garden.\nJohn went to the bedroom.\nMary went back to the kitchen.\nSandra moved to the garden.\nDaniel journeyed to the hallway.\nJohn journeyed to the kitchen.\nJohn journeyed to the office.\nDaniel travelled to the garden.\nJohn journeyed to the garden.\nSandra took the milk.\nSandra put down the milk.\nMary moved to the bathroom.\nDaniel grabbed the milk there.\nJohn moved to the bathroom.\nDaniel put down the milk.\nJohn got the football.\nMary travelled to the office.\nSandra got the milk.\nSandra left the milk.\nSandra journeyed to the kitchen.\nJohn left the football there.\nJohn went to the kitchen.\nDaniel went to the kitchen.\nJohn moved to the hallway.\nDaniel went to the hallway.\nMary journeyed to the bathroom.\nMary took the football.\nMary moved to the office.\nDaniel took the apple.\nMary discarded the football there.\nSandra travelled to the bedroom.\nDaniel put down the apple.\nDaniel took the apple.\nDaniel moved to the garden.\nDaniel moved to the hallway.\nMary grabbed the football.\nMary travelled to the bedroom.\nDaniel went to the garden.\nSandra journeyed to the bathroom.\nMary discarded the football there.\nJohn moved to the kitchen.\nDaniel discarded the apple.",
  "question": "Where was the apple before the hallway? ",
  "choices": {
    "A": "bathroom",
    "B": "bedroom",
    "C": "office",
    "D": "kitchen",
    "E": "garden",
    "F": "hallway"
  }
}

Recorded model answers

The original column uses the control response paired with this variant. Missing responses and inputs outside a model’s limit remain visible.

Model Original answer Variant answer Change
GLiDE v1.2 E — garden E — garden Same answer
Decider v1.1 E — garden E — garden Same answer
Jev E — garden E — garden Same answer
Microsoft Decision-1 E — garden E — garden Same answer
Clef E — garden E — garden Same answer
Luna Decisions E — garden E — garden Same answer
Clef Flash F — hallway F — hallway Same answer
Strands Decider E — garden E — garden Same answer
GLiNER2.5 Decide E — garden E — garden Same answer
Laya A — bathroom C — office Different answer
CalDec v1 C — office C — office Same answer
Identifiers and recorded conditions

Variant ID: babi/0ebef6723a11f5e9f14c/event_count/12/clean-source_first_babi_object_contexts_full_0088

Input fingerprint: 4612959c65b20d8cbbb3c7422621e000c0f68d3328f160a4ecfe124c159e161f

Change Recorded version
Extra events Add 12 unrelated events

Source attribution and dataset terms are linked from the original question.

Type to search questions, variants and documentation.