Concepts and categorization · question 19
Back to Concepts and categorization · All capabilities
Question ID: mnli/d2d92e1032452b41d90f
Source: MNLI
Input format: native choices
Explore this question’s altered versions
Context
yeah because you look at the statistics now and i'm sure it's in your your newspapers just like it is in ours that every major city now the increase of crime is is escalating i mean there are more look at the look at the people there are being shot now i mean every day there's there's dozens of dozens of people across the nation they just get blown away for no reason you know stray bullets or California they were going out there and they were shooting and they get these guys and they don't do anything with them so i kind of i kind of agree with you i'm kind of you still in the in the uh prison system
Question
Does the premise entail, contradict, or leave undetermined this statement? "Crime is escalating now in every major city, however there are plans in place now."
Answer choices
- 0: Entailment
- 1: Neither entailed nor contradicted
- 2: Contradiction
Reference answer: Neither entailed nor contradicted (1)
Exact evaluated input as JSON
{
"state": "yeah because you look at the statistics now and i'm sure it's in your your newspapers just like it is in ours that every major city now the increase of crime is is escalating i mean there are more look at the look at the people there are being shot now i mean every day there's there's dozens of dozens of people across the nation they just get blown away for no reason you know stray bullets or California they were going out there and they were shooting and they get these guys and they don't do anything with them so i kind of i kind of agree with you i'm kind of you still in the in the uh prison system",
"question": "Does the premise entail, contradict, or leave undetermined this statement? \"Crime is escalating now in every major city, however there are plans in place now.\"",
"choices": {
"0": "Entailment",
"1": "Neither entailed nor contradicted",
"2": "Contradiction"
}
}
Recorded model answers
| Model | Answer | Outcome |
|---|---|---|
| Astra | Neither entailed nor contradicted | Correct |
| Opus 5.5 | Neither entailed nor contradicted | Correct |
| GLiDE v1.2 | Neither entailed nor contradicted | Correct |
| Decider v1.1 | Neither entailed nor contradicted | Correct |
| Jev | Neither entailed nor contradicted | Correct |
| Microsoft Decision-1 | Neither entailed nor contradicted | Correct |
| Clef | Neither entailed nor contradicted | Correct |
| Luna Decisions | Neither entailed nor contradicted | Correct |
| Clef Flash | Neither entailed nor contradicted | Correct |
| Strands Decider | Neither entailed nor contradicted | Correct |
| GLiNER2.5 Decide | Contradiction | Incorrect |
| Laya | Entailment | Incorrect |
| CalDec v1 | Entailment | Incorrect |
Source and sharing
Original MultiNLI pair: 34103n; OANC genre: telephone. Transparent modification: selected excerpts and format normalization by Raindrop, 2026-11-09. Premise wording is unchanged. The original unmodified sentence pair is identified by pairID in the MultiNLI 1.0 release, available from the source URL. No source endorsement is implied.
Attribution: Adina Williams, Nikita Nangia, Samuel R. Bowman (MultiNLI); Switchboard Data; Copyright (c) 1997-2002 Trustees of the University of Pennsylvania.
Terms: OANC (selected non-fiction genres). Source sharing decision.
The capability assignment and choice-format normalization are part of this experiment. Source data retain their own terms.
Exact source locator and input fingerprint
{
"source_locator": {
"path": "data/validation_matched-00000-of-00001.parquet",
"record": "6743",
"kind": "source_file_record"
},
"source_revision": "da70db2af9d09693783c3320c4249840212ee221",
"input_hash": "0ff0a7a9ea20f2115e83b9571eeb50767b7e2083133db59e2179ccee03650347"
}