RaindropA practical guide to choosing decision modelsGitHub ↗

Causal reasoning · question 13

Back to Causal reasoning · All capabilities

Question ID: crass/58e45d40ffc5e28b025c
Source: CRASS
Input format: native choices

Explore this question’s altered versions

Question

A woman swallows potassium cyanide. What would have happened if the woman had watched potassium cyanide?

Answer choices

  • A: Nothing special would have happened.
  • B: She would have died.
  • C: She would have vomited.
  • D: That is not possible.

Reference answer: Nothing special would have happened. (A)

Exact evaluated input as JSON
{
  "state": "",
  "question": "A woman swallows potassium cyanide. What would have happened if the woman had watched potassium cyanide?",
  "choices": {
    "A": "Nothing special would have happened.",
    "B": "She would have died.",
    "C": "She would have vomited.",
    "D": "That is not possible."
  }
}

Recorded model answers

Model Answer Outcome
Astra Nothing special would have happened. Correct
Opus 5.5 Nothing special would have happened. Correct
GLiDE v1.2 Nothing special would have happened. Correct
Decider v1.1 Nothing special would have happened. Correct
Jev Nothing special would have happened. Correct
Microsoft Decision-1 Nothing special would have happened. Correct
Clef Nothing special would have happened. Correct
Luna Decisions Nothing special would have happened. Correct
Clef Flash Nothing special would have happened. Correct
Strands Decider She would have died. Incorrect
GLiNER2.5 Decide She would have died. Incorrect
Laya She would have died. Incorrect
CalDec v1 She would have died. Incorrect

Source and sharing

Attribution: BIG-bench crass_ai contributors.
Terms: Apache-2.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "bigbench/benchmark_tasks/crass_ai/task.json",
    "record": "25",
    "kind": "source_file_record"
  },
  "source_revision": "092b196c1f8f14a54bbc62f24759d43bde46dd3b",
  "input_hash": "fb4781b8bea4af0addcc26e65367803cab365bc0df33e35dfc2d31ce727c1331"
}

Type to search questions, variants and documentation.