Invented names · Use invented names variant
Original question · Other versions of this question · All variants
This version keeps the original reference answer. Its context, question instructions or answer choices were changed as described below.
Changed fields: context.
How the edits were constructed
Context
Tavren grabbed the football there. Nelora got the apple there. Tavren dropped the football there. Zorvik went to the garden. Zorvik went to the hallway. Zorvik journeyed to the office. Nelora put down the apple. Zorvik grabbed the apple there. Zorvik discarded the apple. Tavren grabbed the football there. Tavren left the football there. Tavren went back to the hallway. Vaelis travelled to the bathroom. Vaelis journeyed to the bedroom. Nelora took the apple there. Nelora put down the apple. Zorvik got the apple there. Zorvik put down the apple there. Tavren went to the bathroom. Zorvik travelled to the hallway.
Question
Where is the apple?
Answer choices
- A. office
- B. kitchen
- C. hallway
- D. bathroom
- E. bedroom
- F. garden
Reference answer: A.
Exact evaluated input as JSON
{
"state": "Tavren grabbed the football there.\nNelora got the apple there.\nTavren dropped the football there.\nZorvik went to the garden.\nZorvik went to the hallway.\nZorvik journeyed to the office.\nNelora put down the apple.\nZorvik grabbed the apple there.\nZorvik discarded the apple.\nTavren grabbed the football there.\nTavren left the football there.\nTavren went back to the hallway.\nVaelis travelled to the bathroom.\nVaelis journeyed to the bedroom.\nNelora took the apple there.\nNelora put down the apple.\nZorvik got the apple there.\nZorvik put down the apple there.\nTavren went to the bathroom.\nZorvik travelled to the hallway.",
"question": "Where is the apple? ",
"choices": {
"A": "office",
"B": "kitchen",
"C": "hallway",
"D": "bathroom",
"E": "bedroom",
"F": "garden"
}
}
Recorded model answers
The original column uses the control response paired with this variant. Missing responses and inputs outside a model’s limit remain visible.
| Model | Original answer | Variant answer | Change |
|---|---|---|---|
| GLiDE v1.2 | A — office |
A — office |
Same answer |
| Decider v1.1 | A — office |
A — office |
Same answer |
| Jev | C — hallway |
C — hallway |
Same answer |
| Microsoft Decision-1 | C — hallway |
C — hallway |
Same answer |
| Clef | C — hallway |
C — hallway |
Same answer |
| Luna Decisions | C — hallway |
C — hallway |
Same answer |
| Clef Flash | C — hallway |
C — hallway |
Same answer |
| Strands Decider | C — hallway |
C — hallway |
Same answer |
| GLiNER2.5 Decide | C — hallway |
C — hallway |
Same answer |
| Laya | A — office |
A — office |
Same answer |
| CalDec v1 | C — hallway |
A — office |
Different answer |
Identifiers and recorded conditions
Variant ID: babi/9c0cb3f01a3a7116384f/unfamiliar_names/invented
Input fingerprint: e5eb975be64b60a3f449462fce7577716e8ea17dd3ed567e15713441629f3cfd
| Change | Recorded version |
|---|---|
| Invented names | Use invented names |
Source attribution and dataset terms are linked from the original question.