RaindropA practical guide to choosing decision modelsGitHub ↗

Evidence order · Swap independent facts variant

Original question · Other versions of this question · All variants

This version keeps the original reference answer. Its context, question instructions or answer choices were changed as described below.

Changed fields: context.

How the edits were constructed

Context

Aiden and 12 of his friends are going to see a film at the cinema, and meet up with 7 more friends there. Each seat has enough room to hold one person, two drinks, and three snacks. They each save a seat and then buy enough drinks and snacks to fill the seats. Additionally, each person decides to buy a souvenir keychain, but they later realize that the keychains are free today, so they don't have to spend any extra money on them. If drinks cost $2 each, how much money, in dollars, has the group spent overall on snacks and drinks?

Question

Which missing variable should be asked about to solve this problem?

Answer choices

  • F. Cost of one snack
  • L. Cost of each keychain

Reference answer: F.

Exact evaluated input as JSON
{
  "state": "Aiden and 12 of his friends are going to see a film at the cinema, and meet up with 7 more friends there. Each seat has enough room to hold one person, two drinks, and three snacks. They each save a seat and then buy enough drinks and snacks to fill the seats. Additionally, each person decides to buy a souvenir keychain, but they later realize that the keychains are free today, so they don't have to spend any extra money on them. If drinks cost $2 each, how much money, in dollars, has the group spent overall on snacks and drinks?",
  "question": "Which missing variable should be asked about to solve this problem?",
  "choices": {
    "F": "Cost of one snack",
    "L": "Cost of each keychain"
  }
}

Recorded model answers

The original column uses the control response paired with this variant. Missing responses and inputs outside a model’s limit remain visible.

Model Original answer Variant answer Change
GLiDE v1.2 F — Cost of one snack F — Cost of one snack Same answer
Decider v1.1 F — Cost of one snack F — Cost of one snack Same answer
Jev F — Cost of one snack F — Cost of one snack Same answer
Microsoft Decision-1 F — Cost of one snack F — Cost of one snack Same answer
Clef F — Cost of one snack F — Cost of one snack Same answer
Luna Decisions F — Cost of one snack F — Cost of one snack Same answer
Clef Flash F — Cost of one snack F — Cost of one snack Same answer
Strands Decider F — Cost of one snack F — Cost of one snack Same answer
GLiNER2.5 Decide L — Cost of each keychain L — Cost of each keychain Same answer
Laya F — Cost of one snack F — Cost of one snack Same answer
CalDec v1 F — Cost of one snack F — Cost of one snack Same answer

Ordering comparison: literal local or custom unit permutation. Only the exact matched comparison sets contribute to the reported ordering contrasts.

Identifiers and recorded conditions

Variant ID: questbench/9bcd74a64d46e22b4182/presentation_order/swap_independent_givens/clean-source_first_parallel_b_operations_0302

Input fingerprint: ec9a1ae502369fd3c1f2f7f24724db7e68282bc742c0e18d614d32a6263418dc

Change Recorded version
Evidence order Swap independent facts

Source attribution and dataset terms are linked from the original question.

Type to search questions, variants and documentation.