RaindropA practical guide to choosing decision modelsGitHub ↗

Variant inputs and comparison sets

Browse readable pages · Methods · Recover inputs

File Contents
manifest.jsonl One row per distinct variant, with original ID, capability, reference key, ordered choice keys, supported models, changed fields, sharing status, fingerprint and reading page.
conditions.jsonl Logical factor/level memberships and their representative evaluated input IDs. Several memberships can refer to the same input.
comparisons.json Exact comparison families, required levels, before/after contrasts and source construction strata.
inputs/<capability>.jsonl Full evaluated inputs for the 20,961 shareable variants, split by capability.
recovery-recipes.jsonl Authored edits and original-text span references for reconstructing the remaining 3,602 variants from locally obtained source questions.

JSONL means one JSON object per line. Choice order matters to the input fingerprint. gold_choice is the reference answer key, not a model output. Full inputs use gold for the same key.

common_input_fit marks inputs fitting all eleven recorded models. supported_models records the input-fit eligibility of each individual variant. This eligibility is determined from the complete inputs, not from whether predictions were correct.

ordering_roles distinguishes full-evidence permutations from local or coupled edits using the construction audit. The main ordering comparison uses the explicit original/reversed/shuffled families, not every input carrying an ordering label.

Type to search questions, variants and documentation.