Numerical cognition · question 14
Back to Numerical cognition · All capabilities
Question ID: rewardbench/53d625b82bcdab563bb2
Source: RewardBench
Input format: native choices
Explore this question’s altered versions
Context
Write a C++ function `bool is_equal_to_sum_even(int n)` to solve the following problem: Evaluate whether the given number n can be written as the sum of exactly 4 positive even numbers Example is_equal_to_sum_even(4) == false is_equal_to_sum_even(6) == false is_equal_to_sum_even(8) == true
Question
Which response better satisfies the request?
Answer choices
- A: if (n%2==0 and n>=8 and n <=8) return true; return false; }
- B: if (n%2==0 and n>=8) return true; return false; }
Reference answer: if (n%2==0 and n>=8) return true; return false; } (B)
Exact evaluated input as JSON
{
"state": "Write a C++ function `bool is_equal_to_sum_even(int n)` to solve the following problem:\nEvaluate whether the given number n can be written as the sum of exactly 4 positive even numbers\nExample\nis_equal_to_sum_even(4) == false\nis_equal_to_sum_even(6) == false\nis_equal_to_sum_even(8) == true",
"question": "Which response better satisfies the request?",
"choices": {
"A": " if (n%2==0 and n>=8 and n <=8) return true;\n return false;\n}\n",
"B": " if (n%2==0 and n>=8) return true;\n return false;\n}\n"
}
}
Recorded model answers
| Model | Answer | Outcome |
|---|---|---|
| Astra | if (n%2==0 and n>=8) return true; return false; } | Correct |
| Opus 5.5 | if (n%2==0 and n>=8) return true; return false; } | Correct |
| GLiDE v1.2 | if (n%2==0 and n>=8) return true; return false; } | Correct |
| Decider v1.1 | if (n%2==0 and n>=8) return true; return false; } | Correct |
| Jev | if (n%2==0 and n>=8) return true; return false; } | Correct |
| Microsoft Decision-1 | if (n%2==0 and n>=8) return true; return false; } | Correct |
| Clef | if (n%2==0 and n>=8) return true; return false; } | Correct |
| Luna Decisions | if (n%2==0 and n>=8) return true; return false; } | Correct |
| Clef Flash | if (n%2==0 and n>=8) return true; return false; } | Correct |
| Strands Decider | if (n%2==0 and n>=8) return true; return false; } | Correct |
| GLiNER2.5 Decide | if (n%2==0 and n>=8 and n <=8) return true; return false; } | Incorrect |
| Laya | if (n%2==0 and n>=8) return true; return false; } | Correct |
| CalDec v1 | if (n%2==0 and n>=8 and n <=8) return true; return false; } | Incorrect |
Source and sharing
Underlying component: HumanEvalPack; component terms: MIT. The source notices retain the applicable component credits and licenses.
Attribution: Nathan Lambert et al.; Allen Institute for AI (RewardBench); Niklas Muennighoff and HumanEvalPack contributors; BigCode.
Terms: ODC-By-1.0 AND MIT (selected components). Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.
Exact source locator and input fingerprint
{
"source_locator": {
"path": "data/filtered-00000-of-00001.parquet",
"record": "2020",
"kind": "source_file_record"
},
"source_revision": "168d848cdbbea9764fae4a544dc9ca1e6cca4931",
"input_hash": "e4f41597489ab974faac3b74ff309afc6d3c0944f3334fb9b5d9c57ab96a05c4"
}