Smoothen transition to 2025-09 by backporting additive features to legacy

[gkreitz]

Many good features have been added in the new problem format version. Several of these are pure additions to legacy which would be safe to backport to legacy (languages even existed in legacy, but was removed). Thus, I think we should backport additive features from 2025-09 to legacy when problemtools and Kattis supports them to smoothen the transition between the format versions and allow adoption of good new features without needing to tackle the breaking changes in 2025-09.

Preferably also renaming legacy to 1.0, and denoting that as the current version, and 2025-09 as an RC (rather than "latest").

[mzuenni]

Just one concern: This could increase incompatibilities for legacy and and hurt the adaption of 2025-09 in the long run.

For example BAPCtools has a dedicated branch for legacy that is considered feature complete and only gets bug fixes.
If new features are added to legacy this means that the BAPCtools legacy version would not support them and the same might be true for others. IMO all development should be targeted towards 2025-09 support. legacy should just exist for historic reasons...

The workflow for the past GCPCs and NWERCs was already to prepare problems in the 2025-09 format which worked. However, support for 2025-09 at judges is still limited and it should be the main priority?

[Tagl]

I do understand the concern, but this is always the case with big transitions (see Python 2 to 3)
Our issue here is we are jumping from version 1 to version 2 basically with a lot of things that are very reasonable to have as intermediate steps.
With hindsight I would have preferred to start with non-breaking changes into legacy and essentially go this route, but just in the past.
Most legacy problems will be automatically updateable to the newer format.
This will only happen for this first transition to versions, and in the future legacy will be historic as I understand it.

I think the solution to preventing people from using the legacy in the future is to be aggressive regarding deprecation when the time comes.

[mzuenni]

I think the solution to preventing people from using the legacy in the future is to be aggressive regarding deprecation when the time comes.

I think BAPCtools was already very aggressive and we hope others follow. (Support was completely dropped from main)

Even when your 2025-09 support is not complete yet, you can force people to use it as long as your implementation is as powerfull/expressive as legacy was? I.e. as soon as you don't loose anything by switching... just switch?

Most legacy problems will be automatically updateable to the newer format.

Yes and I think that is the way to go. (again as example: BAPCtools run on a legacy problem will just tell you to call bt upgrade)

Our issue here is we are jumping from version 1 to version 2 basically with a lot of things that are very reasonable to have as intermediate steps.

Yes, but I see no issue with only having them in the new version?

[Tagl]

Yes, but I see no issue with only having them in the new version?

Kattis has a lot of things implemented which can be added to legacy.
@gkreitz would prefer to do smaller incremental updates and have it publicly documented somewhere that they are in fact supported, which I think is better than silently supporting them.
Then once the time comes, the switch to the newer version which has breaking changes comes.

[mzuenni]

@gkreitz would prefer to do smaller incremental updates and have it publicly documented somewhere that they are in fact supported, which I think is better than silently supporting them.

yeah I understand that and I think its a good idea to implement this incrementally and document it properly, but that is independent of the backporting question, right?
I.e. you can still implement 2025-09 support incrementally and document that without adding/enabling this for legacy problems?

[Tagl]

Well the idea is that this is a good place to document that.
It would be possible to branch off the legacy to some legacy++ version, but if legacy is just meant to die afterward then it seems harmless to just build on it without the branching.

But yes, it would be possible to document it some other way.

[gkreitz]

yeah I understand that and I think its a good idea to implement this incrementally and document it properly, but that is independent of the backporting question, right?
I.e. you can still implement 2025-09 support incrementally and document that without adding/enabling this for legacy problems?

I don't think we can implement 2025-09 support incrementally. If we do it that way, problems wanting to use the new features would need to mark as 2025-09. But if we haven't implemented all breaking changes of 2025-09, those packages aren't actually 2025-09 packages, they're in some weird twilight zone where they're legacy for the parts we haven't implemented (which currently includes scoring).

Note that we end up supporting packages for a long time. Once we've installed it, if several years later, someone finds a crash bug in the output validator, we tend to fix it. That gets very painful if there are packages where the version information is a lie.

As @Tagl says, we could of course create a version called legacy++/1.0/1.1/2026-07, copy-paste legacy over there, and then add features. And either take on the complexity of making problemtools support legacy + intermediary version + 2025-06, or simply drop support for legacy from problemtools (I'd be inclined to do the latter). In my mind, that does more harm than good compared to just backporting to legacy.

[mzuenni]

I don't think we can implement 2025-09 support incrementally. If we do it that way, problems wanting to use the new features would need to mark as 2025-09. But if we haven't implemented all breaking changes of 2025-09, those packages aren't actually 2025-09 packages, they're in some weird twilight zone where they're legacy for the parts we haven't implemented (which currently includes scoring).

This is not what I suggested. As far as I see, there are two separate things that need to be done:

1. add new stuff (each new feature can be added more or less independently)
2. do breaking changes (all of those must happen at once)

You suggest doing (1) and then (2), which is possible but I would suggest doing (2) and then (1).
As soon as (2) is done, people can use the new format regardless of the status of (1). At this point the new format would always be at least as good as the legacy format, right?

My reasoning here is as follows:

• (2) is always a big chunk of work, and this chunk can't be done incrementally for the reasons you mentioned yourself.
• postponing (2) will not make it easier... it will just postpone when people can start using the new format
• postponing (2) could even harm adoption of the new format... if you already got the new features, why bother switching to 2025-09? And then why invest the time to do the breaking changes...
• doing (2) will help others to do the same changes (e.g. I would not start implementing scoring for BAPCtools until there is another implementation, problems etc. from people who know what they are doing... I am not familiar with scoring problems at all)
• (adding new features retrospectively is not a good manner? IMO this would need some kind of version "legacy++". Just as an example from the past: at some point the UUID was backported 523cbd4 and as far as I know Kattis required this UUID for uploads... Hoewever at the same time Domjudge uploads broke if there was a UUID. That s was quite annoying and should not have happened.)

simply drop support for legacy from problemtools

This sounds wrong... BAPCtools can do such things since its purely for new problem development... for the reference implementation (not sure if problemtools intends to be that) this is not a good manner though? (You could of course also have a problemtools-legacy)