docs: add skill cards for the eight kermt-* agent skills - #26
Conversation
Adds a skill-card.md alongside each skill, following the NVIDIA skill card format used across the NVIDIA/skills catalog (description, owner, license, use case, requirements, risks and mitigations, references, outputs, evaluation, and ethical considerations). These are required for catalog admission: the NVIDIA/skills sync drops any skill missing skill-card.md, skill.oms.sig, or an eval dataset. Evals are already present here; cards close the second gap. Signatures remain outstanding and need nvskills-ci wired in this repo. Evaluation results are marked pending rather than fabricated — NVSkills-Eval has not yet been run against these skills. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Timur Rvachov <trvachov@nvidia.com>
Greptile SummaryAdds NVIDIA-format skill cards for all eight KERMT agent skills so they can satisfy catalog synchronization requirements.
Confidence Score: 4/5The documentation should be corrected before merging because it can mislead users about stale-run detection and positional alignment of inference results. The monitor does not implement the advertised reconciliation of stale manifests with live containers, and inference preprocessing can remove invalid input rows despite the card promising one output row per input molecule. Files Needing Attention: agent/skills/kermt-monitor/skill-card.md; agent/skills/kermt-infer/skill-card.md Important Files Changed
Reviews (1): Last reviewed commit: "docs: add skill cards for the eight kerm..." | Re-trigger Greptile |
|
|
||
| ## Known Risks and Mitigations: <br> | ||
| Risk: A stale or missing `run.json` can cause the skill to report on the wrong run, or to report nothing while a job is in fact still consuming GPU-hours. <br> | ||
| Mitigation: The skill cross-checks `run.json` against live Docker container state rather than trusting the manifest alone. <br> |
There was a problem hiding this comment.
Stale manifests are not reconciled
When run.json is stale or lacks container_name, the monitor queries Docker using manifest-derived information or an mtime-based fallback rather than independently reconciling the run and container, causing missing or wrong-run status while the intended job can continue consuming GPU resources.
| Mitigation: The skill cross-checks `run.json` against live Docker container state rather than trusting the manifest alone. <br> | |
| Mitigation: The skill queries live Docker state using the container information available from `run.json`; when the manifest lacks a container name, users should provide `--container` to avoid relying on heuristic discovery. <br> |
| ## Skill Output: <br> | ||
| **Output Type(s):** [Analysis, Files] <br> | ||
| **Output Format:** [CSV of per-molecule predictions; Markdown summary] <br> | ||
| **Output Parameters:** [2D — one row per input molecule, one column per predicted task] <br> |
There was a problem hiding this comment.
Invalid rows break output alignment
When the input contains invalid SMILES, preprocessing removes those records before prediction despite the card promising one output row per input molecule, causing consumers that align predictions by original row position to associate results with the wrong molecules.
| **Output Parameters:** [2D — one row per input molecule, one column per predicted task] <br> | |
| **Output Parameters:** [2D — one row per valid input molecule after cleaning, one column per predicted task] <br> |
Adds a
skill-card.mdalongside each of the eightkermt-*skills, following the NVIDIA skill card format used across the NVIDIA/skills catalog (326/326 skills there carry one).Why
The NVIDIA/skills sync workflow drops any skill missing
skill-card.md,skill.oms.sig, or an eval dataset — the enforcement step removes the skill dir before the PR is created, so non-compliant skills never merge into the catalog. The same requirement is tracked asSRC-11inbionemo-agent-toolkit's CONTRIBUTING.State of this repo against those three artifacts:
evals/skill-card.mdskill.oms.signvskills-ciwired here (see below)What's in the cards
Per skill: description, owner, license, use case, requirements/dependencies (Docker, NVIDIA Container Toolkit, GPU, optional
WANDB_API_KEY), risks and mitigations, references, output types, evaluation, and ethical considerations.Risk sections are specific rather than generic — GPU-hour cost on the detached training skills, catastrophic forgetting on
kermt-continue-pretrain, the randomly-initialized decoder onkermt-add-cmim-pretrain, checkpoint-type validation onkermt-infer, and third-party data transmission when W&B tracking is enabled.Evaluation results are marked pending, not fabricated. NVSkills-Eval has not been run against these skills; the cards state the eval task counts that exist and say results and a
BENCHMARK.mdwill follow.Still needed after this: signatures
skill.oms.sigcannot be added by a PR — it is emitted by theNVIDIA/nvskills-ciservice. Wiring it here needs:request-nvskills-ci.ymlworkflow (this repo currently has no.github/workflows/at all),NVSKILLS_CI_DISPATCH_TOKENsecret,/nvskills-cion a PR.Happy to open that as a follow-up if it's wanted here.
Context
bionemo-agent-toolkitvendors these eight skills viacomponents.d/kermt.ymland rsyncs with--delete, so a card committed there is wiped on the next nightly sync. This repo is the source of truth. The catalog carries an interim copy of these same cards in acompliance.d/overlay, which is retired automatically once this merges.Please correct anything I got wrong about the skills' behavior or risks — I wrote these from the SKILL.md files and repo layout, not from running the pipelines.
🤖 Generated with Claude Code