Numerical cognition · question 32
Back to Numerical cognition · All capabilities
Question ID: mmlu/b94c0a08436343bbf655
Source: MMLU
Input format: native choices
Explore this question’s altered versions
Question
Two computers communicate with each other by sending data packets across a local area network. The size of these packets is 1,000 bytes. The network has the capacity to carry 1,000 packets per second. The CPU time required to execute the network protocol to send one packet is 10 milliseconds. The maximum rate at which one computer can send data to another is approximately
Answer choices
- A: 10,000 bytes/second
- B: 25,000 bytes/ second
- C: 100,000 bytes/ second
- D: 500,000 bytes/ second
Reference answer: 100,000 bytes/ second (C)
Exact evaluated input as JSON
{
"state": "",
"question": "Two computers communicate with each other by sending data packets across a local area network. The size of these packets is 1,000 bytes. The network has the capacity to carry 1,000 packets per second. The CPU time required to execute the network protocol to send one packet is 10 milliseconds. The maximum rate at which one computer can send data to another is approximately",
"choices": {
"A": "10,000 bytes/second",
"B": "25,000 bytes/ second",
"C": "100,000 bytes/ second",
"D": "500,000 bytes/ second"
}
}
Recorded model answers
| Model | Answer | Outcome |
|---|---|---|
| Astra | 100,000 bytes/ second | Correct |
| Opus 5.5 | 100,000 bytes/ second | Correct |
| GLiDE v1.2 | 500,000 bytes/ second | Incorrect |
| Decider v1.1 | 100,000 bytes/ second | Correct |
| Jev | 100,000 bytes/ second | Correct |
| Microsoft Decision-1 | 10,000 bytes/second | Incorrect |
| Clef | 100,000 bytes/ second | Correct |
| Luna Decisions | 100,000 bytes/ second | Correct |
| Clef Flash | 100,000 bytes/ second | Correct |
| Strands Decider | 100,000 bytes/ second | Correct |
| GLiNER2.5 Decide | 10,000 bytes/second | Incorrect |
| Laya | 10,000 bytes/second | Incorrect |
| CalDec v1 | 10,000 bytes/second | Incorrect |
Source and sharing
Attribution: Dan Hendrycks et al.; MMLU authors.
Terms: MIT. Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.
Exact source locator and input fingerprint
{
"source_locator": {
"path": "all/test-00000-of-00001.parquet",
"record": "1069",
"kind": "source_file_record"
},
"source_revision": "c30699e8356da336a370243923dbaf21066bb9fe",
"input_hash": "cd13228d48de401f214b244900346c5bbf62ef35486a32785ac9fe98c64c7f52"
}