RaindropA practical guide to choosing decision modelsGitHub ↗

Social cognition · question 19

Back to Social cognition · All capabilities

Question ID: bigtom/c5644ba015c2bdfaeeab
Source: BigToM
Input format: native choices

Explore this question’s altered versions

Context

Olga is a botanist exploring a remote rainforest in search of a rare orchid species. Olga wants to find and document the orchid to contribute to her research on endangered plants. She sees that the map she carries with her indicates that the orchid is located in a specific area of the rainforest. Olga believes that the rare orchid is in the area shown on the map. A sudden landslide alters the rainforest landscape, moving the orchid to a different location. Olga does not notice the aftermath of the landslide.

Question

Does Olga believe the rare orchid is in the area shown on the map or in the affected area?

Answer choices

  • A: Olga believes the rare orchid is in the affected area.
  • B: Olga believes the rare orchid is in the area shown on the map.

Reference answer: Olga believes the rare orchid is in the area shown on the map. (B)

Exact evaluated input as JSON
{
  "state": "Olga is a botanist exploring a remote rainforest in search of a rare orchid species. Olga wants to find and document the orchid to contribute to her research on endangered plants. She sees that the map she carries with her indicates that the orchid is located in a specific area of the rainforest. Olga believes that the rare orchid is in the area shown on the map. A sudden landslide alters the rainforest landscape, moving the orchid to a different location. Olga does not notice the aftermath of the landslide.",
  "question": "Does Olga believe the rare orchid is in the area shown on the map or in the affected area?",
  "choices": {
    "A": "Olga believes the rare orchid is in the affected area.",
    "B": "Olga believes the rare orchid is in the area shown on the map."
  }
}

Recorded model answers

Model Answer Outcome
Astra Olga believes the rare orchid is in the area shown on the map. Correct
Opus 5.5 Olga believes the rare orchid is in the area shown on the map. Correct
GLiDE v1.2 Olga believes the rare orchid is in the area shown on the map. Correct
Decider v1.1 Olga believes the rare orchid is in the area shown on the map. Correct
Jev Olga believes the rare orchid is in the area shown on the map. Correct
Microsoft Decision-1 Olga believes the rare orchid is in the area shown on the map. Correct
Clef Olga believes the rare orchid is in the area shown on the map. Correct
Luna Decisions Olga believes the rare orchid is in the area shown on the map. Correct
Clef Flash Olga believes the rare orchid is in the area shown on the map. Correct
Strands Decider Olga believes the rare orchid is in the area shown on the map. Correct
GLiNER2.5 Decide Olga believes the rare orchid is in the area shown on the map. Correct
Laya Olga believes the rare orchid is in the area shown on the map. Correct
CalDec v1 Olga believes the rare orchid is in the area shown on the map. Correct

Source and sharing

Attribution: Kanishk Gandhi, Jan-Philipp Fränken, Tobias Gerstenberg, Noah D. Goodman.
Terms: MIT. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "data/bigtom/bigtom.csv",
    "record": "119/forward-belief/false",
    "kind": "source_file_record"
  },
  "source_revision": "fe647d680bddb69f738519313bed625f9e93b549",
  "input_hash": "ca08d8eb469bf7733ec46811ae2021382c81a8220696148e3f669f42adbe76b3"
}

Type to search questions, variants and documentation.