Invented names · Use invented names variant
Original question · Other versions of this question · All variants
This version keeps the original reference answer. Its context, question instructions or answer choices were changed as described below.
Changed fields: context.
How the edits were constructed
Context
Tavren journeyed to the bedroom. Tavren went back to the hallway. Nelora travelled to the hallway. Vaelis went to the hallway. Vaelis travelled to the garden. Vaelis grabbed the milk there. Tavren moved to the office. Nelora moved to the kitchen. Vaelis put down the milk there. Nelora went back to the bedroom. Vaelis grabbed the milk there. Tavren journeyed to the hallway. Vaelis travelled to the bathroom. Nelora journeyed to the hallway. Zorvik travelled to the bathroom. Zorvik journeyed to the hallway. Vaelis took the apple. Vaelis moved to the office. Tavren travelled to the office. Vaelis moved to the hallway. Nelora went to the bedroom. Tavren went back to the bathroom. Vaelis moved to the garden. Vaelis travelled to the bathroom. Tavren travelled to the kitchen. Vaelis discarded the milk. Zorvik went to the bathroom. Tavren went to the bathroom.
Question
Where was the milk before the garden?
Answer choices
- A. office
- B. hallway
- C. garden
- D. bathroom
- E. bedroom
- F. kitchen
Reference answer: B.
Exact evaluated input as JSON
{
"state": "Tavren journeyed to the bedroom.\nTavren went back to the hallway.\nNelora travelled to the hallway.\nVaelis went to the hallway.\nVaelis travelled to the garden.\nVaelis grabbed the milk there.\nTavren moved to the office.\nNelora moved to the kitchen.\nVaelis put down the milk there.\nNelora went back to the bedroom.\nVaelis grabbed the milk there.\nTavren journeyed to the hallway.\nVaelis travelled to the bathroom.\nNelora journeyed to the hallway.\nZorvik travelled to the bathroom.\nZorvik journeyed to the hallway.\nVaelis took the apple.\nVaelis moved to the office.\nTavren travelled to the office.\nVaelis moved to the hallway.\nNelora went to the bedroom.\nTavren went back to the bathroom.\nVaelis moved to the garden.\nVaelis travelled to the bathroom.\nTavren travelled to the kitchen.\nVaelis discarded the milk.\nZorvik went to the bathroom.\nTavren went to the bathroom.",
"question": "Where was the milk before the garden? ",
"choices": {
"A": "office",
"B": "hallway",
"C": "garden",
"D": "bathroom",
"E": "bedroom",
"F": "kitchen"
}
}
Recorded model answers
The original column uses the control response paired with this variant. Missing responses and inputs outside a model’s limit remain visible.
| Model | Original answer | Variant answer | Change |
|---|---|---|---|
| GLiDE v1.2 | B — hallway |
B — hallway |
Same answer |
| Decider v1.1 | B — hallway |
B — hallway |
Same answer |
| Jev | B — hallway |
C — garden |
Different answer |
| Microsoft Decision-1 | D — bathroom |
D — bathroom |
Same answer |
| Clef | D — bathroom |
D — bathroom |
Same answer |
| Luna Decisions | B — hallway |
B — hallway |
Same answer |
| Clef Flash | C — garden |
C — garden |
Same answer |
| Strands Decider | D — bathroom |
C — garden |
Different answer |
| GLiNER2.5 Decide | D — bathroom |
D — bathroom |
Same answer |
| Laya | A — office |
A — office |
Same answer |
| CalDec v1 | A — office |
B — hallway |
Different answer |
Identifiers and recorded conditions
Variant ID: babi/ca4805a20f60de636d46/unfamiliar_names/invented
Input fingerprint: 8ad33a840fb2bdf55995597285f19df4d3557b5ffc33ca03e7fbb11d0c1e7388
| Change | Recorded version |
|---|---|
| Invented names | Use invented names |
Source attribution and dataset terms are linked from the original question.