Skip to content

Add parametric validation framework - #90

Draft
ilkilic wants to merge 2 commits into
mainfrom
parametric-validation-framework
Draft

Add parametric validation framework#90
ilkilic wants to merge 2 commits into
mainfrom
parametric-validation-framework

Conversation

@ilkilic

@ilkilic ilkilic commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Introduces a composable, parametric validation framework alongside the existing validation.py. Users can define custom electrophysiology validations by composing Protocol + Measurement + Criterion — without modifying BlueCelluLab source code.

Motivation

The current validation.py contains hardcoded validation tests (fixed protocols, fixed eFEL features, fixed pass/fail logic). Adding a new validation requires writing a new function from scratch. This framework makes it trivial to define parametric validations that can be configured at runtime.

What's added

bluecellulab/validation/
├── __init__.py          # Package exports
├── base.py              # ValidationOutcome dataclass + ValidationTest ABC
├── protocol.py          # Protocol ABC + StepProtocol (IDRest at X% rheobase)
├── measurement.py       # Measurement ABC + EfelMeasurement (any eFEL feature)
├── criterion.py         # Criterion ABC + GreaterThan (numpy-array-safe)
├── parametric_test.py   # ParametricValidation: composes protocol+measurement+criterion
├── plotting.py          # plot_trace utility (extracted from validation.py)
└── validation.py        # UNCHANGED — backward compatible

Usage

from bluecellulab.validation import (
    ParametricValidation, StepProtocol, EfelMeasurement, GreaterThan,
)

# Define a custom validation: spike count > 0 at 130% rheobase
test = ParametricValidation(
    validation_name="Spiking Validation",
    protocol=StepProtocol(threshold_percentage=130.0),
    measurement=EfelMeasurement(feature_name="Spikecount"),
    criterion=GreaterThan(threshold=0),
)

outcome = test.run(cell.template_params, rheobase, out_dir)
# outcome.name, outcome.passed, outcome.details, outcome.figures

Design decisions

  • Existing validation.py is untouched — no breaking changes
  • numpy-safe criteriaGreaterThan handles both scalar and array-valued measurements (e.g., AP_amplitude returns one value per spike)
  • Composable — any combination of protocol, measurement, and criterion can be assembled without subclassing
  • Extensible — new protocols, measurements, and criteria can be added independently

@codecov

codecov Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.81268% with 18 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
bluecellulab/validation/protocol.py 67.92% 17 Missing ⚠️
bluecellulab/validation/criterion.py 94.73% 1 Missing ⚠️
Files with missing lines Coverage Δ
bluecellulab/validation/__init__.py 100.00% <100.00%> (ø)
bluecellulab/validation/base.py 100.00% <100.00%> (ø)
bluecellulab/validation/measurement.py 100.00% <100.00%> (ø)
bluecellulab/validation/parametric_test.py 100.00% <100.00%> (ø)
bluecellulab/validation/plotting.py 100.00% <100.00%> (ø)
tests/test_validation/test_parametric_framework.py 100.00% <100.00%> (ø)
bluecellulab/validation/criterion.py 94.73% <94.73%> (ø)
bluecellulab/validation/protocol.py 67.92% <67.92%> (ø)

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant