RaindropA practical guide to choosing decision modelsGitHub ↗

Concepts and categorization

All capabilities · Methods

67 questions. ‘Correct / answered’ counts the recorded outcomes across all 13 models.

Question Source Correct / answered Question text
1. Which intent does this request express? CLINC150 11/13 Included
2. Does the premise entail, contradict, or leave undetermined this statement? That would cause the Inte… MNLI 11/13 Included
3. Which banking intent does this request express? Banking77 13/13 Included
4. What emotion does the final user message express, taking the conversation into account? EmoWOZ 11/13 Included
5. Does the assistant response refuse the user's request? WildGuardTest 13/13 Included
6. Which of these snakes is poisonous? MMLU 10/13 Included
7. Which response better satisfies the request? RewardBench 12/13 Included
8. Recover locally — news_topic AG News 8/13 Recover locally
9. Recover locally — sentiment SST-2 12/13 Recover locally
10. Recover locally — coarse_question_type TREC 10/13 Recover locally
11. Does this user message contain toxic content? ToxicChat 13/13 Included
12. "Fat people eat accumulates." For the above sentence, what part of speech does the first word belon… HLE-MultiChoice 2/13 Included
13. How would Quinn feel afterwards? Social IQa 13/13 Included
14. Which intent does this request express? CLINC150 12/13 Included
15. Which banking intent does this request express? Banking77 13/13 Included
16. Is the user's request harmful? WildGuardTest 13/13 Included
17. Jeanette locked the front door and then checked it by turning the knob. She checked it a second time… MMLU 11/13 Included
18. What emotion does the final user message express, taking the conversation into account? EmoWOZ 12/13 Included
19. Does the premise entail, contradict, or leave undetermined this statement? "Crime is escalating now … MNLI 10/13 Included
20. Recover locally — news_topic AG News 13/13 Recover locally
21. Recover locally — coarse_question_type TREC 11/13 Recover locally
22. Recover locally — sentiment SST-2 13/13 Recover locally
23. Does this user message contain toxic content? ToxicChat 13/13 Included
24. How would you describe Kai? Social IQa 13/13 Included
25. Which intent does this request express? CLINC150 13/13 Included
26. Which banking intent does this request express? Banking77 11/13 Included
27. Once a train pulls out of a station, or an aeroplane takes off or a film starts, those seats are los… MMLU 11/13 Included
28. Does the premise entail, contradict, or leave undetermined this statement? It would be unfortunate t… MNLI 12/13 Included
29. Recover locally — news_topic AG News 13/13 Recover locally
30. Does the assistant response refuse the user's request? WildGuardTest 13/13 Included
31. What emotion does the final user message express, taking the conversation into account? EmoWOZ 11/13 Included
32. Recover locally — coarse_question_type TREC 12/13 Recover locally
33. Recover locally — sentiment SST-2 12/13 Recover locally
34. Does this user message contain toxic content? ToxicChat 13/13 Included
35. Which intent does this request express? CLINC150 8/13 Included
36. Which banking intent does this request express? Banking77 13/13 Included
37. For which of these two scenarios does the main character (who uses I/me/my) do something clearly mor… MMLU 9/13 Included
38. Does the premise entail, contradict, or leave undetermined this statement? The Sphinx wasn't very fr… MNLI 12/13 Included
39. Recover locally — news_topic AG News 13/13 Recover locally
40. What emotion does the final user message express, taking the conversation into account? EmoWOZ 5/13 Included
41. Is the user's request harmful? WildGuardTest 8/13 Included
42. Recover locally — coarse_question_type TREC 12/13 Recover locally
43. Recover locally — sentiment SST-2 13/13 Recover locally
44. Does this user message contain toxic content? ToxicChat 13/13 Included
45. Which intent does this request express? CLINC150 11/13 Included
46. Which banking intent does this request express? Banking77 11/13 Included
47. The idea that you identify with, and give allegiance, to a nation-state is MMLU 13/13 Included
48. Does the premise entail, contradict, or leave undetermined this statement? The departures help them … MNLI 12/13 Included
49. What emotion does the final user message express, taking the conversation into account? EmoWOZ 13/13 Included
50. Recover locally — coarse_question_type TREC 13/13 Recover locally
51. Does the assistant response refuse the user's request? WildGuardTest 13/13 Included
52. Recover locally — news_topic AG News 13/13 Recover locally
53. Recover locally — sentiment SST-2 13/13 Recover locally
54. Does this user message contain toxic content? ToxicChat 13/13 Included
55. Which intent does this request express? CLINC150 13/13 Included
56. Which banking intent does this request express? Banking77 13/13 Included
57. This question refers to the following information. You are the United States, you are the future inv… MMLU 13/13 Included
58. Does the premise entail, contradict, or leave undetermined this statement? The Herron School of art … MNLI 12/13 Included
59. Recover locally — coarse_question_type TREC 9/13 Recover locally
60. Does the assistant response contain harmful content? WildGuardTest 10/13 Included
61. Recover locally — news_topic AG News 13/13 Recover locally
62. What emotion does the final user message express, taking the conversation into account? EmoWOZ 11/13 Included
63. Recover locally — sentiment SST-2 13/13 Recover locally
64. Does this user message contain toxic content? ToxicChat 13/13 Included
65. Which intent does this request express? CLINC150 13/13 Included
66. Which banking intent does this request express? Banking77 11/13 Included
67. All of the following are modes of asexual reproduction EXCEPT MMLU 11/13 Included

Type to search questions, variants and documentation.