Add PTB-XL dataset support (ECG) - #1187
Open
AxelNoun wants to merge 4 commits into
Open
Conversation
Introduce PTBXLDataset (BaseDataset + YAML) with optional wfdb extra, root-keyed metadata cache, and a resolved cache YAML so file_path is correct before BaseDataset init. Co-authored-by: Cursor <cursoragent@cursor.com>
Add PTBXLSuperclassClassification and split_by_strat_fold (folds 1-8 / 9 / 10) in separate modules so they can move to benchmarks later. Co-authored-by: Cursor <cursoragent@cursor.com>
Cover helpers, metadata cache keyed by root, event reads via PTBXLDataset, waveform IO (wfdb), and the 5-superclass task path. Co-authored-by: Cursor <cursoragent@cursor.com>
Document the dataset/task Overview RST pages and a minimal ECG superclass example for CPBench users. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contributor: AxelNoun external contributor, no NetID
Contribution Type: New Dataset + Task
Description: Adds PTB-XL (12-lead ECG, PhysioNet open access) as a
PyHealth dataset, with a 5-superclass multi-label diagnosis task and a
split helper using the official stratified folds. Motivated by the CPBench
effort: it gives conformal prediction a multi-label ECG benchmark that needs
no credentialing, with metadata suited to covariate/label-shift splits.
Based on Wagner et al., Scientific Data 2020
(https://www.nature.com/articles/s41597-020-0495-6).
Files to Review:
pyhealth/datasets/ptbxl.pydataset implementationpyhealth/datasets/configs/ptbxl.yamltable config (template)pyhealth/tasks/ptbxl.py5-superclass multi-label taskpyhealth/datasets/splitter.pysplit_by_strat_foldhelpertests/core/test_ptbxl.pytests with synthetic fixturesdocs/api/datasets/pyhealth.datasets.ptbxl.rst,docs/api/tasks/pyhealth.tasks.ptbxl.rstexamples/ecg/ptbxl/ptbxl_superclass_quickstart.pyImplementation notes:
wfdbis an optional extra (pip install pyhealth[ptbxl]), lazily imported.Happy to make it a hard dependency if you'd prefer.
MODULE_CACHE_PATH, notroot. Filenames includesampling rate and a short hash of the resolved data root so different roots
never share a cache; reuse also validates
signal_file ⊆ data_root.BaseDatasetonly acceptsconfig_path, so PTB-XL writes a root+rate-keyed YAML next to the derived CSV(
ptbxl-config-{rate}hz-{root_hash}.yaml) beforesuper().__init__(), withfile_pathalready pointing at the derived metadata CSV. EEGBCI insteadpatches
self.config.tables[...].file_pathaftersuper(). Open to aligningwith maintainers' preferred pattern.
# noqa: E402on the EEGBCI import line inpyhealth/datasets/__init__.pyleft untouched (out of diff).(n_leads, n_samples), matching EEGBCI's convention.age_is_censored,distinct from
age_is_missing(written aspd.NA).drop_empty_labels.test-resources/ptbxl/is synthetic WFDB (.hea/.dat)plus tiny CSVs covering censored age, missing age, multilabel, and empty superclass.
Testing / CI:
pytest tests/core/test_ptbxl.py17 passed (withwfdbinstalled).wfdb/pyhealth[ptbxl], the 2 waveform I/O tests skip; metadata /task / e2e event-read tests still run.
wfdbsomake testallunder pixi covers waveform I/O;CI paths that do not install that extra will not exercise
load_ptbxl_record.Follow-up: 71-SCP-code classification and age regression are structured
for but not implemented here planned as a separate PR.
Test plan
pytest tests/core/test_ptbxl.py17 passed (withwfdb)python tools/check_pr_rules.py --base upstream/master --head HEADmake testall/ contribution-rules on the PRRecreates #1186, which was closed automatically on 12 Aug when the fork
hosting its branch was detached during an account cleanup on my side
my mistake, apologies for the noise.
The head commit is byte-identical to the original PR (
47aa3a41d7).All prior context and discussion: #1186