RaindropA practical guide to choosing decision modelsGitHub ↗

Concepts and categorization · question 27

Back to Concepts and categorization · All capabilities

Question ID: mmlu/444ad26b7aa6304eed26
Source: MMLU
Input format: native choices

Explore this question’s altered versions

Question

Once a train pulls out of a station, or an aeroplane takes off or a film starts, those seats are lost and can never be sold. This is referred to as:

Answer choices

  • A: Inseparability.
  • B: Variability.
  • C: Intangibility.
  • D: Perishability.

Reference answer: Perishability. (D)

Exact evaluated input as JSON
{
  "state": "",
  "question": "Once a train pulls out of a station, or an aeroplane takes off or a film starts, those seats are lost and can never be sold. This is referred to as:",
  "choices": {
    "A": "Inseparability.",
    "B": "Variability.",
    "C": "Intangibility.",
    "D": "Perishability."
  }
}

Recorded model answers

Model Answer Outcome
Astra Perishability. Correct
Opus 5.5 Perishability. Correct
GLiDE v1.2 Perishability. Correct
Decider v1.1 Perishability. Correct
Jev Perishability. Correct
Microsoft Decision-1 Perishability. Correct
Clef Perishability. Correct
Luna Decisions Perishability. Correct
Clef Flash Perishability. Correct
Strands Decider Perishability. Correct
GLiNER2.5 Decide Perishability. Correct
Laya Inseparability. Incorrect
CalDec v1 Inseparability. Incorrect

Source and sharing

Attribution: Dan Hendrycks et al.; MMLU authors.
Terms: MIT. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "all/test-00000-of-00001.parquet",
    "record": "7263",
    "kind": "source_file_record"
  },
  "source_revision": "c30699e8356da336a370243923dbaf21066bb9fe",
  "input_hash": "51bb0e0b2cdbbb41970330e6b616075196da7c042f4d09641b904f0d5d807313"
}

Type to search questions, variants and documentation.