RaindropA practical guide to choosing decision modelsGitHub ↗

Recorded variant answers

Read results · Explore questions · Comparison methods

Each of the eleven model files contains one row per variant: 24,563 rows per model. These include successful answers, errors, refusals, unresolved requests and inputs outside the model's supported limit. An unsupported input is not an attempted prediction.

Model files are named by their command aliases: glide.jsonl, decider11.jsonl, jev.jsonl, microsoft-decision-1.jsonl, clef.jsonl, openai-decisions.jsonl, clef-flash.jsonl, strands.jsonl, gliner.jsonl, laya.jsonl and caldec.jsonl.

Field Meaning
case_id, model, input_hash Variant ID, model alias and fingerprint of its complete evaluated input.
status ok, refusal, error, unknown, unknown_prior_attempt or not_supported. Unresolved outcomes are not successful answers.
prediction The chosen answer key; null when no usable answer is available.
original_result_id Exact linked original response in original-controls.jsonl, or null for an unsupported input.
source_result_id Opaque identity of the saved response supplying this logical outcome.
source_case_id, source_input_hash, input_alias Proven native-input equivalence can let more than one logical condition use the same response. These retain the source relationship.
recovered_error True for a Clef answer recovered after an initial execution error. The analysis uses the recovered answer.

The same original can have more than one control response. Join through original_result_id, rather than selecting an arbitrary row by original ID and model. Reference answer keys and links to the full-text pages are in the manifest.

Type to search questions, variants and documentation.