RaindropA practical guide to choosing decision modelsGitHub ↗

Evidence order · Present the evidence in a shuffled order variant

Original question · Other versions of this question · All variants

This version keeps the original reference answer. Its context, question instructions or answer choices were changed as described below.

Changed fields: context.

How the edits were constructed

Context

Read the full context

Daniel went to the hallway. Daniel went to the kitchen. Mary went back to the hallway. John picked up the apple. Daniel journeyed to the hallway. John went to the garden. John journeyed to the hallway. Mary moved to the office. Daniel went back to the kitchen. John discarded the apple there. John picked up the apple. John left the apple there. Mary went to the bedroom. Mary moved to the kitchen. John grabbed the apple. Mary moved to the office. Daniel went to the garden. Daniel went to the bedroom. Mary went back to the bedroom. Daniel journeyed to the hallway. Daniel went to the bedroom. John put down the apple. Mary moved to the bathroom. John moved to the bedroom. John moved to the bathroom. Mary travelled to the garden. Sandra journeyed to the hallway. Mary journeyed to the kitchen. Sandra travelled to the office. Mary travelled to the garden. John got the milk. John went back to the garden. Mary went back to the bedroom. Mary travelled to the garden. John went back to the kitchen. Daniel moved to the kitchen. Sandra journeyed to the bathroom. Sandra grabbed the football. John went back to the office. John moved to the kitchen. John travelled to the office. Sandra went back to the hallway. John discarded the milk. John took the milk. Sandra dropped the football. Daniel journeyed to the garden. John discarded the milk. John got the milk. John left the milk. Sandra grabbed the football. Daniel travelled to the bathroom. Sandra discarded the football. John went to the bedroom. John moved to the kitchen. Sandra picked up the football there. Sandra dropped the football. Daniel travelled to the bedroom. Sandra moved to the office. Mary went to the bathroom. Sandra went back to the bathroom. Daniel moved to the garden. John went to the bedroom. Sandra went back to the hallway. Sandra took the apple. Sandra grabbed the football. Sandra dropped the apple. Daniel travelled to the bathroom. Mary went back to the office. Sandra dropped the football there. Daniel travelled to the office. John travelled to the kitchen. Sandra got the apple. Daniel picked up the milk. Sandra grabbed the football there. Sandra put down the football. John moved to the office. Sandra moved to the garden. Mary moved to the bedroom. Daniel dropped the milk. John grabbed the milk. Daniel moved to the bedroom. Mary travelled to the office. John went back to the hallway. Sandra went back to the bathroom. Sandra went to the office. Mary journeyed to the bedroom. Daniel went back to the garden. Daniel went to the kitchen. John picked up the football. John put down the football. John discarded the milk. Daniel moved to the bathroom. Sandra went back to the garden. John took the milk. Mary moved to the garden. Sandra moved to the bathroom. Daniel moved to the garden. Sandra put down the apple.

Question

Where was the apple before the office?

Answer choices

  • A. office
  • B. bedroom
  • C. hallway
  • D. kitchen
  • E. bathroom
  • F. garden

Reference answer: E.

Exact evaluated input as JSON
{
  "state": "Daniel went to the hallway.\nDaniel went to the kitchen.\nMary went back to the hallway.\nJohn picked up the apple.\nDaniel journeyed to the hallway.\nJohn went to the garden.\nJohn journeyed to the hallway.\nMary moved to the office.\nDaniel went back to the kitchen.\nJohn discarded the apple there.\nJohn picked up the apple.\nJohn left the apple there.\nMary went to the bedroom.\nMary moved to the kitchen.\nJohn grabbed the apple.\nMary moved to the office.\nDaniel went to the garden.\nDaniel went to the bedroom.\nMary went back to the bedroom.\nDaniel journeyed to the hallway.\nDaniel went to the bedroom.\nJohn put down the apple.\nMary moved to the bathroom.\nJohn moved to the bedroom.\nJohn moved to the bathroom.\nMary travelled to the garden.\nSandra journeyed to the hallway.\nMary journeyed to the kitchen.\nSandra travelled to the office.\nMary travelled to the garden.\nJohn got the milk.\nJohn went back to the garden.\nMary went back to the bedroom.\nMary travelled to the garden.\nJohn went back to the kitchen.\nDaniel moved to the kitchen.\nSandra journeyed to the bathroom.\nSandra grabbed the football.\nJohn went back to the office.\nJohn moved to the kitchen.\nJohn travelled to the office.\nSandra went back to the hallway.\nJohn discarded the milk.\nJohn took the milk.\nSandra dropped the football.\nDaniel journeyed to the garden.\nJohn discarded the milk.\nJohn got the milk.\nJohn left the milk.\nSandra grabbed the football.\nDaniel travelled to the bathroom.\nSandra discarded the football.\nJohn went to the bedroom.\nJohn moved to the kitchen.\nSandra picked up the football there.\nSandra dropped the football.\nDaniel travelled to the bedroom.\nSandra moved to the office.\nMary went to the bathroom.\nSandra went back to the bathroom.\nDaniel moved to the garden.\nJohn went to the bedroom.\nSandra went back to the hallway.\nSandra took the apple.\nSandra grabbed the football.\nSandra dropped the apple.\nDaniel travelled to the bathroom.\nMary went back to the office.\nSandra dropped the football there.\nDaniel travelled to the office.\nJohn travelled to the kitchen.\nSandra got the apple.\nDaniel picked up the milk.\nSandra grabbed the football there.\nSandra put down the football.\nJohn moved to the office.\nSandra moved to the garden.\nMary moved to the bedroom.\nDaniel dropped the milk.\nJohn grabbed the milk.\nDaniel moved to the bedroom.\nMary travelled to the office.\nJohn went back to the hallway.\nSandra went back to the bathroom.\nSandra went to the office.\nMary journeyed to the bedroom.\nDaniel went back to the garden.\nDaniel went to the kitchen.\nJohn picked up the football.\nJohn put down the football.\nJohn discarded the milk.\nDaniel moved to the bathroom.\nSandra went back to the garden.\nJohn took the milk.\nMary moved to the garden.\nSandra moved to the bathroom.\nDaniel moved to the garden.\nSandra put down the apple.",
  "question": "Where was the apple before the office? ",
  "choices": {
    "A": "office",
    "B": "bedroom",
    "C": "hallway",
    "D": "kitchen",
    "E": "bathroom",
    "F": "garden"
  }
}

Recorded model answers

The original column uses the control response paired with this variant. Missing responses and inputs outside a model’s limit remain visible.

Model Original answer Variant answer Change
GLiDE v1.2 E — bathroom E — bathroom Same answer
Decider v1.1 E — bathroom E — bathroom Same answer
Jev E — bathroom F — garden Different answer
Microsoft Decision-1 E — bathroom E — bathroom Same answer
Clef E — bathroom E — bathroom Same answer
Luna Decisions C — hallway F — garden Different answer
Clef Flash F — garden F — garden Same answer
Strands Decider F — garden F — garden Same answer
GLiNER2.5 Decide F — garden F — garden Same answer
Laya A — office A — office Same answer
CalDec v1 A — office A — office Same answer

Ordering comparison: literal local or custom unit permutation. Only the exact matched comparison sets contribute to the reported ordering contrasts.

Identifiers and recorded conditions

Variant ID: babi/6d3842d1971d4be98e73/presentation_order/shuffled_presentation/clean-source_first_babi_local_move_orders_full_0086

Input fingerprint: 641c3e9dc81cf3eaa28f9c4b6abcd1e2dcf6b778e9e9b5925416d2fdb374c624

Change Recorded version
Evidence order Present the evidence in a shuffled order

Source attribution and dataset terms are linked from the original question.

Type to search questions, variants and documentation.