RaindropA practical guide to choosing decision modelsGitHub ↗

Variant comparison summaries

comparisons.csv contains model-level and capability-level results for each explicit before/after comparison. See the field guide for denominators, accuracy, answer changes and unavailable responses.

Regenerate it offline into a separate local file:

python3 variants.py summarize

The output is .local/variant-summaries/comparisons.csv. python3 variants.py verify checks it against the published table. Input selection comes from the comparison families; unrelated authored levels are not averaged into one perturbation score.

Type to search questions, variants and documentation.