RaindropA practical guide to choosing decision modelsGitHub ↗

Social cognition · question 12

Back to Social cognition · All capabilities

Question ID: bigtom/4d949574e47d4f33c824
Source: BigToM
Input format: native choices

Explore this question’s altered versions

Context

Delia is a scientist working in a remote research station in Antarctica. She needs to collect important data from a weather monitoring device outside the station. Dalia sees that the device is displaying the correct data. Delia believes the device is displaying the correct data. Suddenly, a powerful solar flare disrupts the device's sensors, causing it to display incorrect data. Delia does not observe the solar flare.

Question

Does Delia believe the weather monitoring device is displaying correct or incorrect data?

Answer choices

  • A: Delia believes the weather monitoring device is displaying incorrect data.
  • B: Delia believes the weather monitoring device is displaying correct data.

Reference answer: Delia believes the weather monitoring device is displaying correct data. (B)

Exact evaluated input as JSON
{
  "state": "Delia is a scientist working in a remote research station in Antarctica. She needs to collect important data from a weather monitoring device outside the station. Dalia sees that the device is displaying the correct data. Delia believes the device is displaying the correct data. Suddenly, a powerful solar flare disrupts the device's sensors, causing it to display incorrect data. Delia does not observe the solar flare.",
  "question": "Does Delia believe the weather monitoring device is displaying correct or incorrect data?",
  "choices": {
    "A": "Delia believes the weather monitoring device is displaying incorrect data.",
    "B": "Delia believes the weather monitoring device is displaying correct data."
  }
}

Recorded model answers

Model Answer Outcome
Astra Delia believes the weather monitoring device is displaying correct data. Correct
Opus 5.5 Delia believes the weather monitoring device is displaying correct data. Correct
GLiDE v1.2 Delia believes the weather monitoring device is displaying correct data. Correct
Decider v1.1 Delia believes the weather monitoring device is displaying correct data. Correct
Jev Delia believes the weather monitoring device is displaying correct data. Correct
Microsoft Decision-1 Delia believes the weather monitoring device is displaying correct data. Correct
Clef Delia believes the weather monitoring device is displaying correct data. Correct
Luna Decisions Delia believes the weather monitoring device is displaying correct data. Correct
Clef Flash Delia believes the weather monitoring device is displaying correct data. Correct
Strands Decider Delia believes the weather monitoring device is displaying correct data. Correct
GLiNER2.5 Decide Delia believes the weather monitoring device is displaying correct data. Correct
Laya Delia believes the weather monitoring device is displaying incorrect data. Incorrect
CalDec v1 Delia believes the weather monitoring device is displaying incorrect data. Incorrect

Source and sharing

Attribution: Kanishk Gandhi, Jan-Philipp Fränken, Tobias Gerstenberg, Noah D. Goodman.
Terms: MIT. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "data/bigtom/bigtom.csv",
    "record": "140/forward-belief/false",
    "kind": "source_file_record"
  },
  "source_revision": "fe647d680bddb69f738519313bed625f9e93b549",
  "input_hash": "14f184fb2b04a2431c9b0b7d732e57d7828202997cfc55ab0f90f8ac93b843a0"
}

Type to search questions, variants and documentation.