Add downloadable cells dataset via scverse-misc - #1149
Open
timtreis wants to merge 12 commits into
Open
Conversation
Expose spatialdata.datasets.cells() alongside blobs/raccoon. It downloads the cells example dataset and loads it as a SpatialData object, reusing the scverse-misc datasets infrastructure (parse_registry + fetch with the built-in spatialdata loader) rather than reimplementing a downloader. - ship src/spatialdata/datasets.yaml registry (base_url + cells.zip sha256) - add scverse-misc[datasets]>=0.0.10 dependency - bump requires-python and ruff target to 3.12 (scverse-misc requires >=3.12) - update CI matrix 3.11 -> 3.12 - docs + network-free registry test and a slow download test Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
for more information, see https://pre-commit.ci
timtreis
force-pushed
the
add-cells-dataset
branch
from
June 22, 2026 10:14
7d60be6 to
c453909
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1149 +/- ##
==========================================
+ Coverage 92.44% 92.45% +0.01%
==========================================
Files 51 51
Lines 7820 7836 +16
==========================================
+ Hits 7229 7245 +16
Misses 591 591
🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Member
|
Didn't dive deep but this looks great. One important thing is missing, for each dataset we must list the license, and if the license requires attribution, we need to list that. Could you please add this? |
The cells dataset is derived from the 10x Genomics Xenium Prime Cervical Cancer FFPE sample, released under CC BY 4.0 (attribution required). Record the license and attribution in the registry and document the expectation that every dataset lists its license.
Member
Author
|
@LucaMarconato added the license info 👌 |
ajkswamy
approved these changes
Aug 14, 2026
ajkswamy
left a comment
ajkswamy
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, here are some suggestions for improvements:
- Please add a link to the associated PR in
spatialdata-notebooks. Please also add a couple of lines describing the motivation for this PR along which what value it adds. I see the assocated PR inspatialdata-notebooksalready has a couple of good lines in this direction.
| def test_cells_download(tmp_path) -> None: | ||
| # Downloads ~3 MB from the scverse example data bucket; opt out with `-m "not slow"`. | ||
| sdata = cells(path=str(tmp_path)) | ||
| assert isinstance(sdata, SpatialData) |
Collaborator
There was a problem hiding this comment.
In case we want to write tests based on this dataset, it would be good if this test would verify some particular of the dataset similar to test_datasets above, like number of each type of elements expected, their shapes/sizes, etc.
|
|
||
| def cells(path: str | None = None) -> SpatialData: | ||
| """ | ||
| Cells dataset. |
Collaborator
There was a problem hiding this comment.
It would be good to include a brief description of what this dataset contains, to help developers adopt it.
| import pooch | ||
| from scverse_misc.datasets import fetch, parse_registry | ||
|
|
||
| cache_dir = Path(path) if path is not None else Path(pooch.os_cache("spatialdata")) |
Collaborator
There was a problem hiding this comment.
please also cover the "else" case here in tests.
| from scverse_misc.datasets import fetch, parse_registry | ||
|
|
||
| cache_dir = Path(path) if path is not None else Path(pooch.os_cache("spatialdata")) | ||
| registry = importlib.resources.files("spatialdata").joinpath("datasets.yaml") |
Collaborator
There was a problem hiding this comment.
I see the three lines here are also used in the unit test for this function. Would it make sense to make this into a separate function. Maybe it will be reused if we add more datasets that use this pattern from spatialdata-misc
Extract _shipped_registry() and _cache_dir() helpers, shared between cells() and the tests, removing the duplicated registry-parsing block. _cache_dir() isolates the path-vs-default branch so both sides are covered without a network call. Describe the dataset contents in the docstring, and assert the concrete element counts and shapes in test_cells_download. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
parse_registry is untyped in the mypy env, so returning its result directly tripped no-any-return; assign through annotated locals. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Exposes
spatialdata.datasets.cells()alongsideblobs/raccoon. It downloads thecellsexample dataset and loads it as aSpatialDataobject.Dataset fetching is delegated to
scverse-misc[datasets](>=0.1.0, the first released version with the datasets feature). The registry shipped insrc/spatialdata/datasets.yamlis parsed and fetched (downloaded, hash-verified, cached, loaded) at call time.The actual download is exercised by
test_cells_download, markedslow; the network-freetest_cells_registryvalidates the shipped registry on every run.