RaindropA practical guide to choosing decision modelsGitHub ↗

Evidence order · Present the evidence in reverse order variant

Original question · Other versions of this question · All variants

This version keeps the original reference answer. Its context, question instructions or answer choices were changed as described below.

Changed fields: context.

How the edits were constructed

Context

Kofi does not notice the apprentice chef swapping the ingredients. Before that, An apprentice chef, mistakenly thinking it was a seasoning paste, removes the peanut butter from the soup and replaces it with tomato paste. Before that, Kofi believes the soup contains peanut butter. Before that, Kofi adds a small container of peanut butter to the soup pot on the stove. Before that, Kofi wants to prepare a traditional groundnut soup for the dinner menu. Before that, Kofi is a chef at a popular Ghanaian restaurant.

Question

Does Kofi believe the soup contains peanut butter or tomato paste?

Answer choices

  • A. Kofi believes the soup contains tomato paste.
  • B. Kofi believes the soup contains peanut butter.

Reference answer: B.

Exact evaluated input as JSON
{
  "state": "Kofi does not notice the apprentice chef swapping the ingredients.\nBefore that, An apprentice chef, mistakenly thinking it was a seasoning paste, removes the peanut butter from the soup and replaces it with tomato paste.\nBefore that, Kofi believes the soup contains peanut butter.\nBefore that, Kofi adds a small container of peanut butter to the soup pot on the stove.\nBefore that, Kofi wants to prepare a traditional groundnut soup for the dinner menu.\nBefore that, Kofi is a chef at a popular Ghanaian restaurant.",
  "question": "Does Kofi believe the soup contains peanut butter or tomato paste?",
  "choices": {
    "A": "Kofi believes the soup contains tomato paste.",
    "B": "Kofi believes the soup contains peanut butter."
  }
}

Recorded model answers

The original column uses the control response paired with this variant. Missing responses and inputs outside a model’s limit remain visible.

Model Original answer Variant answer Change
GLiDE v1.2 B — Kofi believes the soup contains peanut butter. B — Kofi believes the soup contains peanut butter. Same answer
Decider v1.1 B — Kofi believes the soup contains peanut butter. B — Kofi believes the soup contains peanut butter. Same answer
Jev B — Kofi believes the soup contains peanut butter. B — Kofi believes the soup contains peanut butter. Same answer
Microsoft Decision-1 B — Kofi believes the soup contains peanut butter. B — Kofi believes the soup contains peanut butter. Same answer
Clef B — Kofi believes the soup contains peanut butter. B — Kofi believes the soup contains peanut butter. Same answer
Luna Decisions B — Kofi believes the soup contains peanut butter. B — Kofi believes the soup contains peanut butter. Same answer
Clef Flash B — Kofi believes the soup contains peanut butter. B — Kofi believes the soup contains peanut butter. Same answer
Strands Decider B — Kofi believes the soup contains peanut butter. B — Kofi believes the soup contains peanut butter. Same answer
GLiNER2.5 Decide B — Kofi believes the soup contains peanut butter. B — Kofi believes the soup contains peanut butter. Same answer
Laya B — Kofi believes the soup contains peanut butter. B — Kofi believes the soup contains peanut butter. Same answer
CalDec v1 A — Kofi believes the soup contains tomato paste. B — Kofi believes the soup contains peanut butter. Different answer

Ordering comparison: coupled wording reference chronology or representation. Only the exact matched comparison sets contribute to the reported ordering contrasts.

Identifiers and recorded conditions

Variant ID: bigtom/466418e603869271f89a/presentation_order/reverse_presentation/clean-source_first_parallel_b_retrospective_batch_0365

Input fingerprint: c517d78d18640806e0b99845edae5ded8bd45f787d650a0d018fc37e21bb195a

Change Recorded version
Evidence order Present the evidence in reverse order

Source attribution and dataset terms are linked from the original question.

Type to search questions, variants and documentation.