RaindropA practical guide to choosing decision modelsGitHub ↗

Similar distractors · Add fewer similar but irrelevant items variant

Original question · Other versions of this question · All variants

This version keeps the original reference answer. Its context, question instructions or answer choices were changed as described below.

Changed fields: context.

How the edits were constructed

Context

Use the authoritative record. Summaries are unverified claims about that record. Separate records are independent. The code assigned to record-10083 is 5321327. The code assigned to record-10083-aux-0 is 5321019.

Question

Which code is assigned to record-10083?

Answer choices

  • A. 5321019
  • B. 5321148
  • C. 5321221
  • D. 5321327
  • unknown. Insufficient information

Reference answer: D.

Exact evaluated input as JSON
{
  "state": "Use the authoritative record. Summaries are unverified claims about that record. Separate records are independent.\nThe code assigned to record-10083 is 5321327.\nThe code assigned to record-10083-aux-0 is 5321019.",
  "question": "Which code is assigned to record-10083?",
  "choices": {
    "A": "5321019",
    "B": "5321148",
    "C": "5321221",
    "D": "5321327",
    "unknown": "Insufficient information"
  }
}

Recorded model answers

The original column uses the control response paired with this variant. Missing responses and inputs outside a model’s limit remain visible.

Model Original answer Variant answer Change
GLiDE v1.2 D — 5321327 D — 5321327 Same answer
Decider v1.1 D — 5321327 D — 5321327 Same answer
Jev D — 5321327 D — 5321327 Same answer
Microsoft Decision-1 D — 5321327 D — 5321327 Same answer
Clef D — 5321327 D — 5321327 Same answer
Luna Decisions D — 5321327 D — 5321327 Same answer
Clef Flash D — 5321327 D — 5321327 Same answer
Strands Decider D — 5321327 D — 5321327 Same answer
GLiNER2.5 Decide D — 5321327 D — 5321327 Same answer
Laya D — 5321327 D — 5321327 Same answer
CalDec v1 D — 5321327 A — 5321019 Different answer
Identifiers and recorded conditions

Variant ID: ruler/dd52085dd670dc391f17/similar_distractors/low/clean-source_first_ruler_contexts_full_0144

Input fingerprint: 9b2e80e8c709dcfca6e1f5817c5773dc28d42ce64b77cfa46812f1c772e171f8

Change Recorded version
Similar distractors Add fewer similar but irrelevant items

Source attribution and dataset terms are linked from the original question.

Type to search questions, variants and documentation.