| 1. Which response better satisfies the request? |
RewardBench |
10/13 |
Included |
| 2. Recover locally — depth-1 |
ProofWriter |
7/13 |
Recover locally |
| 3. What does f('staovk', 'k') return? |
CRUXEval |
10/13 |
Included |
| 4. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
10/13 |
Included |
| 5. Does the target assistant step contain an error under a process-quality evaluation? A correct useful… |
AgentProcessBench |
12/13 |
Included |
| 6. Select the best English interpretation of the given proposition, using the following translation ke… |
MMLU |
11/13 |
Included |
| 7. Recover locally — depth-2 |
ProofWriter |
10/13 |
Recover locally |
| 8. Which response better satisfies the request? |
RewardBench |
13/13 |
Included |
| 9. S(n) is a statement about positive integers n such that whenever S(k) is true, S(k + 1) must also be… |
MMLU |
7/13 |
Included |
| 10. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
7/13 |
Included |
| 11. What does f("") return? |
CRUXEval |
13/13 |
Included |
| 12. Recover locally — depth-6 |
ProofWriter |
4/13 |
Recover locally |
| 13. Recover locally — depth-3 |
ProofWriter |
7/13 |
Recover locally |
| 14. Which response better satisfies the request? |
RewardBench |
12/13 |
Included |
| 15. Construct a complete truth table for the following argument. Then, using the truth table, determine… |
MMLU |
7/13 |
Included |
| 16. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
11/13 |
Included |
| 17. What does f("odes\tcode\twell", 2) return? |
CRUXEval |
10/13 |
Included |
| 18. Does the target assistant step contain an error under a process-quality evaluation? A correct useful… |
AgentProcessBench |
11/13 |
Included |
| 19. Which response better satisfies the request? |
RewardBench |
9/13 |
Included |
| 20. Recover locally — depth-4 |
ProofWriter |
8/13 |
Recover locally |
| 21. Use indirect truth tables to determine whether the following argument is valid. If the argument is i… |
MMLU |
12/13 |
Included |
| 22. What does f('ax') return? |
CRUXEval |
7/13 |
Included |
| 23. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
13/13 |
Included |
| 24. Recover locally — depth-4 |
ProofWriter |
6/13 |
Recover locally |
| 25. Which response better satisfies the request? |
RewardBench |
12/13 |
Included |
| 26. Recover locally — depth-0 |
ProofWriter |
8/13 |
Recover locally |
| 27. What does f('ncdsdfdaaa0a1cdscsk*XFd') return? |
CRUXEval |
12/13 |
Included |
| 28. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
9/13 |
Included |
| 29. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
8/13 |
Included |
| 30. Which response better satisfies the request? |
RewardBench |
9/12 |
Included |
| 31. Recover locally — depth-6 |
ProofWriter |
9/13 |
Recover locally |
| 32. What does f('the speed is -36 miles per hour') return? |
CRUXEval |
12/13 |
Included |
| 33. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
11/13 |
Included |
| 34. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
10/13 |
Included |
| 35. Recover locally — depth-5 |
ProofWriter |
8/13 |
Recover locally |
| 36. Which response better satisfies the request? |
RewardBench |
10/13 |
Included |
| 37. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
11/13 |
Included |
| 38. What does f('abc', 'abcd') return? |
CRUXEval |
9/13 |
Included |
| 39. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
10/13 |
Included |
| 40. Recover locally — depth-7 |
ProofWriter |
9/13 |
Recover locally |
| 41. Which response better satisfies the request? |
RewardBench |
12/13 |
Included |
| 42. What does f([4, 5]) return? |
CRUXEval |
8/13 |
Included |
| 43. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
11/13 |
Included |
| 44. Does the target assistant step contain an error under a process-quality evaluation? A correct useful… |
AgentProcessBench |
9/13 |
Included |
| 45. Recover locally — depth-2 |
ProofWriter |
9/13 |
Recover locally |
| 46. What does f('pvtso') return? |
CRUXEval |
13/13 |
Included |
| 47. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
11/13 |
Included |
| 48. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
10/13 |
Included |
| 49. Recover locally — depth-3 |
ProofWriter |
5/13 |
Recover locally |
| 50. What does f(range(100, 120)) return? |
CRUXEval |
11/13 |
Included |
| 51. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
11/13 |
Included |
| 52. Recover locally — depth-2 |
ProofWriter |
9/13 |
Recover locally |
| 53. Recover locally — depth-1 |
ProofWriter |
12/13 |
Recover locally |
| 54. What does f('abc', 1) return? |
CRUXEval |
9/13 |
Included |
| 55. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
9/13 |
Included |
| 56. Does the target assistant step contain an error under a process-quality evaluation? A correct useful… |
AgentProcessBench |
11/13 |
Included |
| 57. Recover locally — depth-4 |
ProofWriter |
6/13 |
Recover locally |
| 58. What does f('K zBK') return? |
CRUXEval |
10/13 |
Included |
| 59. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
10/13 |
Included |
| 60. Recover locally — depth-3 |
ProofWriter |
7/13 |
Recover locally |
| 61. Recover locally — depth-5 |
ProofWriter |
6/13 |
Recover locally |
| 62. What does f('m4n2o') return? |
CRUXEval |
10/13 |
Included |
| 63. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
12/13 |
Included |
| 64. Recover locally — depth-6 |
ProofWriter |
5/13 |
Recover locally |
| 65. What does f({'-4':'4','1':'2','-':'-3'}) return? |
CRUXEval |
12/13 |
Included |
| 66. Which candidate response is most appropriate, given the user's request and the available tools? |
When2Call |
9/13 |
Included |
| 67. Recover locally — depth-1 |
ProofWriter |
12/13 |
Recover locally |