Skip to content

The information if a validator outputs score.txt, score_multiplier.txt, or neither should be available statically #581

Description

@gkreitz

A validator can output score.txt, score_multiplier.txt, or nothing. It must pick between score.txt and score_multiplier.txt across invocations per https://www.kattis.com/problem-package-format/spec/2025-09.html#reporting-additional-feedback :

The validator may create a score.txt, a score_multiplier.txt, or neither, but it must not create both. This applies over all invocations. That is, if a validator ever creates a score.txt, then it must never create a score_multiplier.txt, and vice versa.

There are two problems here. Firstly, to figure out if a validator produces score.txt or score_multiplier.txt, we must run it. Secondly, if I read the spec correctly, a validator can always decide to output neither file (which will be interpreted as "full score"). This feels like a fragile design if there are any buggy paths in a validator causing it to for some reason fail to write a score. (Also, as a remark, this second problem also means we can't even run the validator once and see what it produces, in the general sense, figuring out if a validator produces score.txt or score_multiplier.txt is likely equivalent to solving the halting problem. :))

For legacy problems, we require an output validator for a scoring problem to always output a score. I think the same should hold for the new version - a problem author should specify somewhere which of the 3 modes a validator uses to emit a score (score.txt, score_multiplier.txt, or neither). I would strongly prefer this information to be available without having to compile and run the validator.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions