Refactor: structured INPUT parameter availability (Phase 1) by Stardust0831 · Pull Request #7783 · deepmodeling/abacus-develop

Stardust0831 · 2026-08-05T14:30:13Z

Reminder

I have read AGENTS.md and docs/developers_guide/agent_governance.md.
I have linked an issue or explained why this PR does not need one.
I have added adequate unit tests and/or case tests, or explained why not.
I have listed the exact verification commands run and their results.
I have described user-visible behavior changes, including INPUT parameter changes.
I have explained core-module impact for ESolver, HSolver, ElecState, Hamilt, Operator, Psi, or other source/ changes.
I have requested any needed governance exception below.

Linked Issue

Ref #7719 — normalize INPUT availability into a machine-readable form and align it with parameter validation.

Unit Tests and/or Case Tests for my changes

Commands run:
- New C++ parser unit test source/source_io/test_serial/availability_test.cpp, wired into the test build as MODULE_IO_availability_serial (run via ctest -R MODULE_IO); covers leaf / in-set / contains / and-or nesting with parentheses / round-trip / empty / invalid-input rejection (incl. prose like "Only used for plane wave basis.").
- Remote CPU build + abacus --generate-parameters-yaml — COMPLETED; regenerated docs/parameters.yaml and input-main.md (generated output byte-consistent with the checked-in docs).
Result summary: docs/parameters.yaml has 526 params / 216 non-empty availability, all concrete boolean Expression (0 Label / 0 Unstructured).
Checks not run, with reason: full runtime/MPI test suite not run — this change is metadata-only (availability strings + help export); no numerical or code-path behavior change.

What's changed?

Make the Input_Item availability field a single-source, machine-evaluable boolean condition so the INPUT docs, validation and tooling can consume and evaluate it instead of re-parsing free text or bare tags.

C++ data model: new dependency-free module source/source_io/module_parameter/availability.{h,cpp} (wired into CMake) with AvailabilityCondition / AvailabilityExpr and parse_availability() (operators ==/!=/>/>=/</<=/in/contains, combinators and/or/,, (...) grouping). Input_Item keeps the canonical availability string plus availability_expr (parsed tree) and a set_availability() helper keeping both in sync (single source of truth); the raw availability string is now private, so the only way to set it is through the grammar-validating set_availability().
Migration: all 216 non-empty availability registrations rewritten to canonical boolean syntax. Former bare "label"-style tags (OFDFT, Numerical atomic orbital basis, ...) are now concrete conditions over existing parameters (esolver_type==ofdft, basis_type==lcao, calculation==gen_bessel, dft_plus_u==1, ...). Vector parameters use contains (e.g. td_ttype contains 2) to preserve containment semantics.
Export: --generate-parameters-yaml now emits the canonical availability string (empty string = always available) and nothing else, keeping the YAML lean; the structured parse tree stays in C++ for a future validation/error layer.
Docs: regenerated docs/parameters.yaml and input-main.md.

Manual fidelity: rewrites preserve the manual's original meaning. Where the manual states a gate the code does not enforce via an INPUT boolean, it is called out in Governance Notes.

Governance Notes

INPUT/docs changes: docs/parameters.yaml and docs/advanced/input_files/input-main.md regenerated from the new binary (per AGENTS.md). Full old→new mapping was reviewed locally.
Core module impact: source/source_io/module_parameter/input_item.h gains an availability_expr member and the new availability module; metadata-only, no runtime/numerical behavior change.
Exceptions requested:
- Header dependency review: input_item.h includes availability.h because Input_Item holds AvailabilityExpr by value (a forward declaration is not possible); availability.h includes <string>/<vector> for its members (std::string, std::vector). Required includes, not reducible to forward declarations.
Manual-vs-code discrepancies documented (not acting on them in this PR):
- mixing_dmr: manual >= 0.0 (always true) kept as mixing_restart>=0; esolver checks > 0 — likely loose manual wording.
- mixing_tau: "Only relevant for meta-GGA" is a note decided at runtime (XC_Functional::get_ked_flag), not an INPUT gate; availability empty, note kept in description.
- fixed_ibrav: encoded as relax_method in [cg 2] and latname != none (both manual clauses; the latter also enforced by check_value).

The Input_Item availability field mixes prose and ad-hoc conditions as free text, which cannot be consumed programmatically for tree-structured docs, validation or agent tooling. Add a small dependency-free parser (tools/03_code_analysis/availability_parser.py) that normalises the historical spellings (==, =, 'is set to', 'contains') onto a canonical form and classifies each value as an Expression, a bare Label, or Unstructured prose waiting for review. - availability_parser.py: parse_availability() -> Availability - test_availability_parser.py: unit tests - generate_input_main.py: add --check-availability to report the Expression/Label/Unstructured distribution without changing the generated markdown. On the current docs/parameters.yaml this classifies 217 non-empty values as 82 Expression / 84 Label / 51 Unstructured.

Make the Input_Item availability field a concrete, machine-readable boolean condition (single source of truth) so the INPUT docs, validation and tooling can consume the actual condition instead of free text or bare tags. - Add AvailabilityKind/AvailabilityCondition/AvailabilityExpr and parse_availability() in a new module (availability.{h,cpp}, wired into CMake). - Input_Item now carries availability_kind + availability_expr and a set_availability() helper that keeps the canonical string and the structured form in sync (single source of truth). - Rewrite all 217 non-empty availability registrations to canonical boolean syntax; the exported YAML classifies them as 216 Expression / 0 Unstructured. - Export the structured fields from --generate-parameters-yaml (input_help.cpp). - Consume the structured fields in generate_input_main.py; extend availability_parser.py to the canonical grammar and cover it with tests. - Regenerate docs/parameters.yaml and input-main.md. Former bare "label"-style tags (e.g. "OFDFT", "Numerical atomic orbital basis") are now expressed as concrete conditions (esolver_type==ofdft, basis_type==lcao, ...) so they can be evaluated by validation/error tooling. mixing_tau has no hard gate, so its availability is empty (always available); its meta-GGA relevance is kept in the description.

AsTonyshment · 2026-08-08T03:42:55Z

I think this is a clever direction for making INPUT parameter constraints more structured and eventually improving consistency checks, but it may still be too early to merge in its current form. There are several design details that should be clarified first, such as how invalid or non-canonical availability expressions are handled, how to prevent developers from bypassing set_availability() and leaving the string and AST inconsistent, and whether each availability expression is expected to be a complete self-contained predicate. Since this metadata may later become a source of truth for validation and tooling, I think the implementation should be made more robust against malformed or incomplete definitions.

Given the scale and long-term impact of this change, I suggest preparing a short describing the intended design principles, grammar, invariants, and future validation workflow, and sharing it in the WeChat group “abacus开发信息对齐” for broader discussion before proceeding. It would also be helpful to update the developer documentation, including the corresponding Chinese guide (for example, something similar to https://mcresearch.github.io/abacus-user-guide/develop-addinp3.html), so that future contributors clearly understand the new conventions for defining availability.

Maybe you can have a look as well @ZhouXY-PKU ?

The strict C++ parse_availability() in Input_Item::set_availability() is now the single source of truth for the availability grammar (it throws on any non-canonical non-empty string). The legacy python classifier, its unit tests, and the --check-availability report plumbing in generate_input_main.py were an earlier design step and are no longer referenced by any workflow; remove them to keep a single grammar implementation.

Growl1234 · 2026-08-10T17:04:22Z

I fully support the intent; the implementation is questionable in some place. Please allow me some time to have a deeper look.

Growl1234

Thank you for your dedication! Most likely not all review comments are right, so correct me if I made mistakes or missed anything :)

Growl1234 · 2026-08-10T17:07:22Z


 - **Type**: Integer
- **Availability**: *Used only for nscf calculations with plane wave basis set.*
+- **Availability**: *calculation==nscf and basis_type==pw*


It would be better to use

- **Availability**: *calculation==nscf && basis_type==pw*

Or

- **Availability**: *`calculation==nscf`* and *`basis_type==pw`*

Similar to or.

Updated availability expressions to render as inline code.

Growl1234 · 2026-08-10T17:12:43Z

+    return value.substr(begin, end - begin);
+}
+
+std::vector<std::string> split_slash_values(const std::string& value)


param==a/b effectively introduces another spelling of membership in addition to param in [a, b], while also making / impossible to represent literally in an equality value. I'd suggest keeping == strictly single-valued and using in [...] whenever multiple alternatives are intended.

Removed the param==a/b special case. == now accepts one value, and alternatives use in [...].

Growl1234 · 2026-08-10T17:14:40Z

-    std::string availability;  ///< availability conditions (empty if always)
+    /// Set and validate the canonical availability expression. An empty value
+    /// means that the item is always available.
+    void set_availability(const std::string& value)


This currently normalizes non-canonical input rather than actually enforcing canonical input.

For example, anything accepted by the parser can be stored as parsed.to_string(), even if the original registration was not canonical. If the intended invariant is that C++ registrations themselves use the canonical grammar, could we explicitly reject a non-empty value when it differs from parsed.to_string()?

That would catch malformed or non-canonical metadata at development time instead of silently fixing it.

set_availability() now rejects non-canonical strings instead of normalizing them.

Growl1234 · 2026-08-10T17:18:04Z

        item.default_value = "0.5";
        item.unit = "";
-        item.availability = "Only used when relax_method is cg 2";
+        item.set_availability("relax_method in [cg 2]");


I'm not sure the semantics are well-defined here. relax_method is a Vector of string, while the parser tests explicitly distinguish vector containment (contains) from scalar membership (in). What exactly should relax_method in [cg 2] evaluate against: the whole vector, its first element, or one of its elements? Apparently this will cause confusion.

The vector-valued input form was introduced in PR #6517.

My concern is not why relax_method is vector-valued — #6517 explains that part. The question is how the new availability DSL defines the semantics of comparing such a vector. For example, what exactly does relax_method in [cg 2] mean? Does in compare the complete vector [cg, 2], test whether one of its elements belongs to the RHS set, or compare some serialized representation such as "cg 2"? This matters especially because this PR already defines contains specifically for vector containment. I think the DSL should make vector equality/membership unambiguous rather than relying on the historical INPUT spelling.

Updated the DSL: complete multi-token values are quoted and compared with ==, so this is now relax_method=="cg 2". in [...] is reserved for two or more alternatives; contains remains element containment.

Growl1234 · 2026-08-10T17:22:53Z

        item.default_value = "1";
        item.unit = "";
-        item.availability = "method_sto = 2 and out_dos = 1 or cal_cond = True";
+        item.set_availability("(method_sto==2 and out_dos==1) or cal_cond==true");


method_sto itself is only applicable when esolver_type==sdft but that prerequisite is not represented here. If availability expressions are evaluated independently, the default value of method_sto could make this condition true outside SDFT. Therefore, please double-check if dependent options are well encapsulated; if so then this might not be an issue.

Updated dependent availability expressions to include their enclosing prerequisites, such as esolver_type==sdft.

Growl1234 · 2026-08-10T17:23:53Z

+    AvailabilityExpr parse_condition()
+    {
+        AvailabilityExpr result;
+        result.condition.param = parse_identifier();


The parser currently validates only the syntax of the parameter name. Where will we validate that the referenced parameter actually exists, and that the operator/value are compatible with its declared type?

Added post-registration validation for referenced labels, operator/type compatibility, and literals.

Growl1234 · 2026-08-10T17:27:03Z

 if(BUILD_TESTING)
  if(ENABLE_MPI)
    add_subdirectory(test)
    add_subdirectory(test_serial)


Note here, test_serial is guarded with ENABLE_MPI. You might have added the unittests in the wrong place, as the new availability parser is (and must be) dependency-free.

Moved the availability tests outside ENABLE_MPI; they now build independently of MPI.

Stardust0831 · 2026-08-11T07:07:24Z

To improve readability, parameter names in the generated availability expressions are now linked to their corresponding parameter descriptions. For example, users can directly follow basis_type or ks_solver to see their meanings and usage, while the availability field remains a pure condition expression for parsing and validation.

Stardust0831 requested review from AsTonyshment and mohanchen August 5, 2026 14:34

Stardust0831 added the Refactor Refactor ABACUS codes label Aug 5, 2026

Stardust0831 force-pushed the upstream-ready branch from 9472f73 to 2050b54 Compare August 7, 2026 01:38

Stardust0831 force-pushed the upstream-ready branch from 2050b54 to 8b5d337 Compare August 7, 2026 01:47

AsTonyshment requested a review from ZhouXY-PKU August 8, 2026 03:43

Stardust0831 added 2 commits August 8, 2026 13:08

fix(input): enforce canonical availability metadata

caaf729

Growl1234 reviewed Aug 10, 2026

View reviewed changes

Stardust0831 added Compile & CICD & Docs & Dependencies Issues related to compiling ABACUS Input&Output Suitable for coders without knowing too many DFT details labels Aug 10, 2026

Stardust0831 added 5 commits August 11, 2026 02:45

fix(input): harden availability metadata

5bafaf3

docs(input): keep review context out of repository

c83c78d

docs(input): generalize availability guide

b02f369

fix(input): validate availability prerequisites

6afb1fa

fix(input): quote multi-token availability values

7cb8522

Uh oh!

Conversation

Stardust0831 commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reminder

Linked Issue

Unit Tests and/or Case Tests for my changes

What's changed?

Governance Notes

Uh oh!

AsTonyshment commented Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Growl1234 commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Growl1234 left a comment

Choose a reason for hiding this comment

Uh oh!

Growl1234 Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Growl1234 Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Stardust0831 commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Stardust0831 commented Aug 5, 2026 •

edited

Loading

AsTonyshment commented Aug 8, 2026 •

edited

Loading

Growl1234 commented Aug 10, 2026 •

edited

Loading

Growl1234 Aug 10, 2026 •

edited

Loading

Growl1234 Aug 10, 2026 •

edited

Loading