Question variants
The study contains 24,563 altered versions of the 1,005 original questions, tested on 11 decision models. Each version keeps the recorded reference answer. Start from a capability, choose an original question, then open one of its variants.
How the variants were made · Read the results · Verify or rerun · Original questions
Full text is included for 20,961 variants. The other 3,602 use original question texts with additional sharing requirements; recover them locally from the original sources and saved edit recipes. All variant IDs, reference keys and recorded outcomes are included.
| Capability | Original questions | Variants | Full texts included |
|---|---|---|---|
| Concepts and categorization | 67 | 1,583 | 1,158 |
| Rule application | 67 | 1,752 | 1,197 |
| Analogy | 67 | 1,608 | 1,608 |
| Spatial cognition | 67 | 1,698 | 1,698 |
| State tracking and updating | 67 | 1,763 | 1,763 |
| Retrieval | 67 | 1,668 | 1,668 |
| Relations | 67 | 1,682 | 1,682 |
| Numerical cognition | 67 | 1,590 | 1,590 |
| Social cognition | 67 | 1,628 | 1,220 |
| Uncertainty monitoring | 67 | 1,547 | 1,547 |
| Causal reasoning | 67 | 1,602 | 1,602 |
| Physical cognition | 67 | 1,511 | 140 |
| Communication | 67 | 1,604 | 1,604 |
| Learning and induction | 67 | 1,709 | 866 |
| Information seeking | 67 | 1,618 | 1,618 |