RaindropA practical guide to choosing decision modelsGitHub ↗

Causal reasoning · question 15

Back to Causal reasoning · All capabilities

Question ID: crass/0b2230c367ad71b05366
Source: CRASS
Input format: native choices

Explore this question’s altered versions

Question

A girl kisses a boy. What would have happened if the girl had killed the boy?

Answer choices

  • A: That is not possible.
  • B: She would have been liable to prosecution.
  • C: The boy would have been arrested for assault.
  • D: The boy would have kissed the girl.

Reference answer: She would have been liable to prosecution. (B)

Exact evaluated input as JSON
{
  "state": "",
  "question": "A girl kisses a boy. What would have happened if the girl had killed the boy?",
  "choices": {
    "A": "That is not possible.",
    "B": "She would have been liable to prosecution.",
    "C": "The boy would have been arrested for assault.",
    "D": "The boy would have kissed the girl."
  }
}

Recorded model answers

Model Answer Outcome
Astra She would have been liable to prosecution. Correct
Opus 5.5 She would have been liable to prosecution. Correct
GLiDE v1.2 She would have been liable to prosecution. Correct
Decider v1.1 She would have been liable to prosecution. Correct
Jev She would have been liable to prosecution. Correct
Microsoft Decision-1 She would have been liable to prosecution. Correct
Clef She would have been liable to prosecution. Correct
Luna Decisions She would have been liable to prosecution. Correct
Clef Flash She would have been liable to prosecution. Correct
Strands Decider She would have been liable to prosecution. Correct
GLiNER2.5 Decide She would have been liable to prosecution. Correct
Laya That is not possible. Incorrect
CalDec v1 That is not possible. Incorrect

Source and sharing

Attribution: BIG-bench crass_ai contributors.
Terms: Apache-2.0. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "bigbench/benchmark_tasks/crass_ai/task.json",
    "record": "4",
    "kind": "source_file_record"
  },
  "source_revision": "092b196c1f8f14a54bbc62f24759d43bde46dd3b",
  "input_hash": "0a84afe53331e09be6262e55def578c6306e52eb3225f29cbad6175178e58988"
}

Type to search questions, variants and documentation.