fix metric_class names in avg_at_n / avg_at_n_math / maj_at_n test cases

[jayzuccarelli]

Fixes #1304.

Summary

tests/unit/metrics/test_cases/avg_at_k.json, avg_at_k_math.json and maj_at_k.json set metric_class to avg_at_k / avg_at_k_math / maj_at_k, but those metrics are registered as avg_at_n / avg_at_n_math / maj_at_n (metrics.py:154, :161, :434).

An unresolvable name falls into the not available in METRIC_CLASSES branch of run_test_case, which returns "success": True. test_automated_metrics_pytest.py filters skips out of failed_tests and only prints them, so all 10 cases have been reporting as passing without ever calling compute():

Skipped 4 tests in 'Maj At K Test Suite':
  - Maj at K - Majority Correct: Metric 'maj_at_k' not available in METRIC_CLASSES

Not just a rename

Fixing metric_class alone gives Exception: You did not set the value of n. Two more things were wrong in each fixture:

• metric_params used {"k": N}, but the constructors are AvgAtN(n=...) / MajAtN(n=...). Metric.__call__ setattrs whatever key it is given, so {"k": 3} set a dead .k and left .n as None.
• expected_output keyed on avg@k:k=N, but the key is f"{metric_name}:{params}", so it needs avg@n:n=N.

Files are renamed to match their metric, consistent with the rest of test_cases/.

Guard against a repeat

An unresolvable metric_class now raises instead of silently skipping, unless it is listed in SKIPPED_METRICS. That silent branch is what let this sit unnoticed.

maj_at_n is skipped for now

With the names fixed, avg_at_n and avg_at_n_math pass (6 cases). maj_at_n fails:

IndexError: list index out of range
src/lighteval/tasks/requests.py:222, in Doc.get_golds

Its first case has gold_index: [1] against a single-element choices, which is the bug #1274 fixes. Rather than land red CI, maj_at_n is added to SKIPPED_METRICS with a pointer to that PR; it can be unskipped once #1274 merges. Happy to drop the skip and rebase on #1274 instead if you would rather they land together.

Testing

tests/unit/metrics/ is green: 46 passed. The one remaining failure, extractiveness.json, fails identically on unmodified main (missing optional summ_eval dependency) and is unrelated. ruff check and ruff format --check are clean.

Note: .gitattributes marks tests/unit/metrics/test_cases/*.json as -diff, so GitHub renders these three as binary. The changes are the k to n substitutions described above on metric_class, metric_params and the expected_output keys, plus the human-readable name / description strings.