| Extra events · Add 12 unrelated events |
Recover locally |
11 / 11 |
| Extra events · Add 2 unrelated events |
Recover locally |
11 / 11 |
| Extra events · Add 6 unrelated events |
Recover locally |
11 / 11 |
| Evidence position · Place the relevant information at the end |
Recover locally |
10 / 11 |
| Evidence position · Place the relevant information in the middle |
Recover locally |
11 / 11 |
| Evidence position · Place the relevant information at the beginning |
Recover locally |
10 / 11 |
| Repetition · Present the same evidence twice |
Recover locally |
11 / 11 |
| Repetition · Present the same evidence three times |
Recover locally |
11 / 11 |
| Unrelated examples · Add examples unrelated to the question |
Recover locally |
11 / 11 |
| Irrelevant details · Add a larger amount of irrelevant detail |
Recover locally |
11 / 11 |
| Irrelevant details · Add a small amount of irrelevant detail |
Recover locally |
11 / 11 |
| Misleading details · Add an unverified claim that agrees with the evidence |
Recover locally |
11 / 11 |
| Misleading details · Add an unverified claim that conflicts with the evidence |
Recover locally |
10 / 11 |
| Option order · Reverse the answer choices |
Recover locally |
7 / 11 |
| Option order · Move the first answer choice to the end |
Recover locally |
6 / 11 |
| Paraphrasing · Change the sentence structure |
Recover locally |
11 / 11 |
| Paraphrasing · Change individual words and phrases |
Recover locally |
11 / 11 |
| Evidence order · Present the evidence in reverse order |
Recover locally |
7 / 11 |
| Evidence order · Present the evidence in a shuffled order |
Recover locally |
11 / 11 |
| Similar distractors · Add more similar but irrelevant items |
Recover locally |
10 / 11 |
| Similar distractors · Add fewer similar but irrelevant items |
Recover locally |
9 / 11 |
| Text length · Induction: extended passage (approximately 8k context) |
Recover locally |
5 / 6 |
| Text length · Induction: long passage (approximately 2k context) |
Recover locally |
9 / 11 |
| Text length · Induction: additional medium passage (approximately 1k context) |
Recover locally |
10 / 11 |
| Text length · Induction: short added passage |
Recover locally |
11 / 11 |
| Text length · Surrounding prose extended to at least 512 tokens |
Recover locally |
11 / 11 |