Variant comparison summaries
comparisons.csv contains model-level and capability-level results for each explicit before/after comparison. See the field guide for denominators, accuracy, answer changes and unavailable responses.
Regenerate it offline into a separate local file:
python3 variants.py summarize
The output is .local/variant-summaries/comparisons.csv. python3 variants.py verify checks it against the published table. Input selection comes from the comparison families; unrelated authored levels are not averaged into one perturbation score.