RaindropA practical guide to choosing decision modelsGitHub ↗

Physical cognition · question 9

Back to Physical cognition · All capabilities

Question ID: mmlu/21b36ade886452018632
Source: MMLU
Input format: native choices

Explore this question’s altered versions

Question

A beam of monochromatic light entering a glass window pane from the air will experience a change in

Answer choices

  • A: frequency and wavelength
  • B: frequency and speed
  • C: speed and wavelength
  • D: speed only

Reference answer: speed and wavelength (C)

Exact evaluated input as JSON
{
  "state": "",
  "question": "A beam of monochromatic light entering a glass window pane from the air will experience a change in",
  "choices": {
    "A": "frequency and wavelength",
    "B": "frequency and speed",
    "C": "speed and wavelength",
    "D": "speed only"
  }
}

Recorded model answers

Model Answer Outcome
Astra speed and wavelength Correct
Opus 5.5 speed and wavelength Correct
GLiDE v1.2 speed and wavelength Correct
Decider v1.1 speed and wavelength Correct
Jev speed and wavelength Correct
Microsoft Decision-1 speed and wavelength Correct
Clef speed and wavelength Correct
Luna Decisions speed and wavelength Correct
Clef Flash speed and wavelength Correct
Strands Decider speed and wavelength Correct
GLiNER2.5 Decide frequency and speed Incorrect
Laya frequency and wavelength Incorrect
CalDec v1 frequency and wavelength Incorrect

Source and sharing

Attribution: Dan Hendrycks et al.; MMLU authors.
Terms: MIT. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.

Exact source locator and input fingerprint
{
  "source_locator": {
    "path": "all/test-00000-of-00001.parquet",
    "record": "4871",
    "kind": "source_file_record"
  },
  "source_revision": "c30699e8356da336a370243923dbaf21066bb9fe",
  "input_hash": "fee90df8280f6538bbd5946f70d62266600ae581ee29423185528d3c21854663"
}

Type to search questions, variants and documentation.