RaindropA practical guide to choosing decision modelsGitHub ↗

Numerical cognition

All capabilities · Methods

67 questions. ‘Correct / answered’ counts the recorded outcomes across all 13 models.

Question Source Correct / answered Question text
1. ((0 * -4 - -7 * 5) + (-4 * 0 + 6 + 0)) = BIG-Bench Hard selected tasks 9/13 Included
2. A number’s prime factors are 2, 5, 7, 13, and 31. Which of the following must be a factor of the num… MMLU 10/13 Included
3. what was the 5 year return on ventas common stock? FinQA 10/13 Included
4. Which response better satisfies the request? RewardBench 12/13 Included
5. Is smallpox survival less likely than smallpox death overall? CLadder 8/13 Included
6. Consider an input sequence consisting of 20 boxes containing distinct non-negative real numbers. Ali… HLE-MultiChoice 4/13 Included
7. Of the following, which gives the best upper bound for the value of f(N) where f is a solution to th… MMLU 10/13 Included
8. Which response better satisfies the request? RewardBench 12/13 Included
9. Is the chance of black hair larger when observing tanning salon treatment? CLadder 7/13 Included
10. ((-5 + 6 + -6 * -1) + (-5 + -5 + 9 + -2)) = BIG-Bench Hard selected tasks 5/13 Included
11. what was devon's average translation adjustments included in accumulated other comprehensive income … FinQA 13/13 Included
12. Say a mother bird has $n \geq 2$ offspring and a limited amount of food, $R$, to distribute among th… HLE-MultiChoice 4/12 Included
13. Which expression is equivalent to 5 x 9? MMLU 10/13 Included
14. Which response better satisfies the request? RewardBench 11/13 Included
15. Is glimx more likely than not glimx overall? CLadder 11/13 Included
16. ((-5 - 6 * 2 + 4) - (1 - 9 - 0 * 5)) = BIG-Bench Hard selected tasks 5/13 Included
17. average annual rent per square foot changed in 2005 from 2004 by what amount? FinQA 10/13 Included
18. The 24-point game (or sometimes called the "Make 24" game) is a simple but engaging math puzzle. Her… HLE-MultiChoice 5/13 Included
19. In a group of 10 scores, the largest score is increased by 40 points. What will happen to the mean? MMLU 10/13 Included
20. Is the chance of kwox larger when observing jyka? CLadder 10/13 Included
21. ((5 * -2 + 8 * 1) * (9 + -3 * -8 - -5)) = BIG-Bench Hard selected tasks 4/13 Included
22. what is the net change in the balance of deferred sales in 2011? FinQA 11/13 Included
23. The 24-point game (or sometimes called the "Make 24" game) is a simple but engaging math puzzle. Her… HLE-MultiChoice 6/13 Included
24. What is the product of the greatest even prime number and the least odd prime number? MMLU 11/13 Included
25. Is rukz less likely than not rukz overall? CLadder 11/13 Included
26. ((-2 * -8 * 9 + -5) + (7 - -1 - -4 - 1)) = BIG-Bench Hard selected tasks 5/13 Included
27. what would other income have increased to in 2009 absent the pre-tax gain from a land sale , in mill… FinQA 10/13 Included
28. The American flag is usually made with its width and length in the ratio of 10 to 19. Which dimensio… MMLU 12/13 Included
29. Is rukz less likely than not rukz overall? CLadder 11/13 Included
30. ((-5 - -7 - 4 - 8) - (-4 + 4 + 1 + 0)) = BIG-Bench Hard selected tasks 10/13 Included
31. what is the net change in total statutory capital from 2007 to 2008? FinQA 10/13 Included
32. Two computers communicate with each other by sending data packets across a local area network. The s… MMLU 8/13 Included
33. Is muvq more likely than not muvq overall? CLadder 13/13 Included
34. ((8 - -9 * 4 * 9) * (-4 * -3 * 7 * 9)) = BIG-Bench Hard selected tasks 9/13 Included
35. what is the total amount of stock options cancelled in millions during 2017 , 2016 and 2015? FinQA 9/13 Included
36. The quadratic $2x^2+5x+12=19-7x$ has two solutions. What is the positive difference between these so… MMLU 6/13 Included
37. Is xevu less likely than not xevu overall? CLadder 8/13 Included
38. ((9 + 7 - 4 - 7) - (-5 + 6 + -1 * 0)) = BIG-Bench Hard selected tasks 6/13 Included
39. as part of the company completed the acquisition of several software and service-based providers in … FinQA 11/13 Included
40. A theater collected $6 for each ticket sold to a movie. The theater sold 500 tickets to the movie. T… MMLU 12/13 Included
41. Is xevu more likely than not xevu overall? CLadder 11/13 Included
42. ((6 - 5 + 5 + -9) - (9 + 8 * -6 * -1)) = BIG-Bench Hard selected tasks 9/13 Included
43. what is the range , in thousands , for united states' revenue from 2010-2012? FinQA 8/13 Included
44. Which group of numbers is in order from least to greatest? MMLU 10/13 Included
45. Is lirg more likely than not lirg overall? CLadder 12/13 Included
46. ((-9 - -7 + 2 + 2) + (-7 + -9 - -1 - 0)) = BIG-Bench Hard selected tasks 9/13 Included
47. what is the mathematical mean for all three investments as of dec 31 , 2017? FinQA 10/13 Included
48. Is uvzi less likely than not uvzi overall? CLadder 9/13 Included
49. ((3 * 8 * -1 * -9) - (-2 + 2 * -8 * 8)) = BIG-Bench Hard selected tasks 9/13 Included
50. Is kwox more likely than not kwox overall? CLadder 11/13 Included
51. ((-8 * 0 + 2 * 2) * (-9 - 1 * -9 * 3)) = BIG-Bench Hard selected tasks 8/13 Included
52. Is uvzi less likely than not uvzi overall? CLadder 7/13 Included
53. ((7 - -9 + 0 - 0) + (-9 + 5 - -9 * -1)) = BIG-Bench Hard selected tasks 7/13 Included
54. Is uvzi more likely than not uvzi overall? CLadder 9/13 Included
55. ((-7 + 9 + -9 * -4) - (-7 * 6 - 7 * 3)) = BIG-Bench Hard selected tasks 6/13 Included
56. Is kwox less likely than not kwox overall? CLadder 12/13 Included
57. ((-1 - -9 * 8 - -9) + (-9 - 2 * -6 + -6)) = BIG-Bench Hard selected tasks 7/13 Included
58. ((4 * -4 + -6 - -2) - (9 + -7 + -8 * 6)) = BIG-Bench Hard selected tasks 8/13 Included
59. ((-6 + -2 * 3 * 0) + (3 + -4 - 7 - -4)) = BIG-Bench Hard selected tasks 6/13 Included
60. ((-6 - -4 + 9 + 0) + (1 + -4 - -9 * 6)) = BIG-Bench Hard selected tasks 6/13 Included
61. ((-5 + 1 - 2 - -3) * (-9 * 3 + 9 + -1)) = BIG-Bench Hard selected tasks 7/13 Included
62. ((3 + 3 - -4 - 2) + (-2 - -7 * -2 * 1)) = BIG-Bench Hard selected tasks 7/13 Included
63. ((-8 - -3 * 5 + 4) * (2 * 7 - -3 - -1)) = BIG-Bench Hard selected tasks 8/13 Included
64. ((-2 * -1 + 1 * -3) - (7 * -7 + -5 * 6)) = BIG-Bench Hard selected tasks 9/13 Included
65. ((1 + 1 * 3 - 2) * (-8 - -2 - 9 + -9)) = BIG-Bench Hard selected tasks 10/13 Included
66. ((-4 + -6 + -6 - -9) + (-2 * -5 * 5 * 7)) = BIG-Bench Hard selected tasks 6/13 Included
67. ((-8 - -6 * 7 * 0) * (0 + -5 * -8 - 0)) = BIG-Bench Hard selected tasks 9/13 Included

Type to search questions, variants and documentation.