Skip to content

feat: make Holocron a trustworthy CV research workbench - #525

Open
frgfm wants to merge 19 commits into
mainfrom
codex/v0.3-research-workbench
Open

feat: make Holocron a trustworthy CV research workbench#525
frgfm wants to merge 19 commits into
mainfrom
codex/v0.3-research-workbench

Conversation

@frgfm

@frgfm frgfm commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Summary

Make Holocron v0.3 a trustworthy computer-vision research workbench:

  • discover models and checkpoint maturity through a public catalog
  • train with atomic full-state checkpoints, exact resume, RunResult, and schema-v1 run bundles
  • repair YOLOv4 training correctness and report COCO AP through optional pycocotools
  • expose a consistent classifier feature-extraction interface
  • export with dynamo ONNX verification and benchmark with synchronized, machine-readable timings
  • generate provenance-backed model cards and harden package/release artifact verification
  • add reproducible reference commands, migration docs, agent guidance, and preview iFormer T/S/M models

Trust boundaries

  • manifest.json remains the run-bundle completion marker and is written last.
  • Published typed checkpoints enforce their recorded SHA-256.
  • Legacy model-only checkpoints remain supported as warm restarts, not exact resume.
  • Detection stays experimental and unbenchmarked architectures stay preview.
  • No CUDA benchmark, competitive detection benchmark, SegNeXt, or YOLOv10 claim is made here.

Validation

  • 244 passed, 1 skipped across the aggregate CPU suite; skip is CUDA-only
  • 11 passed for the catalog after the final Ponytail simplification
  • make quality passes: Ruff, formatting, ty, and dependency-sync verification
  • documentation build passes with existing MkDocs/Griffe warnings
  • wheel and sdist build once, install independently, expose py.typed, and discover 62 models
  • representative ONNX export parity and latency JSON smokes pass on CPU

Migration

See docs/docs/v0.3-migration.md for catalog, checkpoint, experiment, and export changes.

@github-actions github-actions Bot added topic: docs Improvements or additions to documentation module: models func: build Related to build and installation ext: tests Related to test module: ops Related to ops ext: references module: trainer ext: scripts ext: docs ext: api labels Aug 12, 2026
@socket-security

socket-security Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatednumpy@​1.26.4 ⏵ 2.5.275 +110010010070
Updatedonnxruntime@​1.23.2 ⏵ 1.28.074 +110010010070
Updatedtorch@​2.9.0 ⏵ 2.13.073100 +410010070
Updatedonnx@​1.19.1 ⏵ 1.22.074 +1100 +40100100100
Updatedtorchvision@​0.24.0 ⏵ 0.28.078 -4100100100100
Updatedonnxscript@​0.5.0 ⏵ 0.7.181 -1100100100100
Addedpycocotools@​2.0.11100100100100100

View full report

@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.21939% with 61 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.66%. Comparing base (434ad5e) to head (5381b5c).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
holocron/models/model_card.py 82.31% 26 Missing ⚠️
holocron/trainer/core.py 88.11% 12 Missing ⚠️
holocron/models/catalog.py 91.57% 8 Missing ⚠️
holocron/trainer/detection.py 91.76% 7 Missing ⚠️
holocron/models/classification/_features.py 95.12% 2 Missing ⚠️
holocron/models/classification/iformer.py 98.03% 2 Missing ⚠️
holocron/trainer/experiment.py 97.29% 2 Missing ⚠️
holocron/models/detection/yolov4.py 98.36% 1 Missing ⚠️
holocron/models/utils.py 97.14% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #525      +/-   ##
==========================================
- Coverage   93.97%   93.66%   -0.31%     
==========================================
  Files          64       69       +5     
  Lines        4017     4674     +657     
==========================================
+ Hits         3775     4378     +603     
- Misses        242      296      +54     
Files with missing lines Coverage Δ
holocron/models/__init__.py 100.00% <100.00%> (ø)
holocron/models/checkpoints.py 98.07% <100.00%> (+0.20%) ⬆️
holocron/models/classification/__init__.py 100.00% <100.00%> (ø)
holocron/models/classification/convnext.py 97.72% <100.00%> (+0.02%) ⬆️
holocron/models/classification/darknet.py 100.00% <100.00%> (ø)
holocron/models/classification/darknetv2.py 100.00% <100.00%> (ø)
holocron/models/classification/darknetv3.py 85.93% <100.00%> (+0.22%) ⬆️
holocron/models/classification/darknetv4.py 100.00% <100.00%> (ø)
holocron/models/classification/mobileone.py 96.87% <100.00%> (+0.02%) ⬆️
holocron/models/classification/repvgg.py 86.36% <100.00%> (+0.12%) ⬆️
... and 15 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@frgfm

frgfm commented Aug 12, 2026

Copy link
Copy Markdown
Owner Author

Adversarial review follow-up

Ran six requested harnesses: five returned substantive reviews; the Opus harness exhausted its external monthly quota before analysis on both attempts.

Accepted and fixed:

  • save schema-v2 checkpoints after on_epoch_end (in finally) so callback RNG/state changes resume exactly; added an RNG-consuming callback oracle
  • exercise full-state resume with weights_only=True, matching every reference CLI
  • expose optional --run-dir bundles in classification, detection, and segmentation recipes
  • instantiate export/latency models through the public catalog
  • let declared checkpoint maturity override task defaults
  • document latest-checkpoint versus best-so-far metric semantics
  • install the optional evaluation extra in the detection reference
  • align API lock, source headers, and optional-import typing with CI
  • require immutable 40-character Hugging Face commit revisions for Hub downloads

Reviewed but intentionally retained:

  • artifact re-hashing during model-card generation: required trust-boundary verification
  • manifest-last writes and separate atomic file helpers: short, type-specific crash-safety paths
  • synthetic-character recipe: already-reviewed feat(references): add synthetic character training #524 scope, kept experimental with held-out font families
  • one latest resumable checkpoint instead of separate best/latest artifacts: exact resume remains the contract; best_metric is explicitly best-so-far

Post-review validation:

  • aggregate CPU suite: 244 passed, 1 CUDA-only skipped
  • focused Hub pinning/catalog suite: 12 passed
  • make quality: Ruff, format, ty, and dependency sync pass

@frgfm frgfm self-assigned this Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant