feat(connectors): add moss-connector-zeroentropy#490
Open
HarshaNalluru wants to merge 2 commits into
Open
Conversation
Migrate a ZeroEntropy collection into a Moss index, following the moss-data-connector _template pattern: a ZeroEntropyConnector whose __iter__ pages through documents.get_info_list and yields DocumentInfo, plus the shared ingest() helper. - Default mapping (path -> id, content -> text, metadata -> metadata) so a whole collection migrates with no mapper; overridable via mapper=. - Fetches parsed text per document via get_info(include_content=True) and skips documents that never parsed (no content) rather than indexing empties. - Coerces ZeroEntropy's str|list[str] metadata to Moss's str-only values. - Unit tests (mocked, 9 passing) + skippable live integration test. - Publish workflow, package README, and a row in the connectors table. Verified against the installed zeroentropy 0.1.0a11 SDK: method names, params, and response fields all match.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new Python data connector package under packages/moss-data-connector/ to migrate a ZeroEntropy collection into a Moss index, consistent with the existing connector template and publish workflow patterns used elsewhere in the repo.
Changes:
- Introduces
moss-connector-zeroentropypackage withZeroEntropyConnector+ingest()helpers for collection migration. - Adds mocked unit tests and a skippable live integration test (ZeroEntropy → Moss → query → delete).
- Wires the connector into the mono-repo docs and adds a dedicated publish workflow.
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/moss-data-connector/README.md | Adds moss-connector-zeroentropy to the connectors list/table. |
| packages/moss-data-connector/moss-connector-zeroentropy/uv.lock | Adds a uv lockfile for the new connector’s dev/runtime deps. |
| packages/moss-data-connector/moss-connector-zeroentropy/tests/test_zeroentropy.py | Mocked unit tests covering default/custom mapping, skipping empty content, prefix, auto-id, metadata coercion, etc. |
| packages/moss-data-connector/moss-connector-zeroentropy/tests/test_integration_zeroentropy_moss.py | Skippable live end-to-end migration + query + cleanup test. |
| packages/moss-data-connector/moss-connector-zeroentropy/src/ingest.py | Implements ingest() to create a Moss index from an iterable of DocumentInfo. |
| packages/moss-data-connector/moss-connector-zeroentropy/src/connector.py | Implements ZeroEntropyConnector, default mapping, and metadata coercion. |
| packages/moss-data-connector/moss-connector-zeroentropy/src/init.py | Re-exports the public connector API (ZeroEntropyConnector, ingest, etc.). |
| packages/moss-data-connector/moss-connector-zeroentropy/README.md | Package-level usage docs and migration guidance. |
| packages/moss-data-connector/moss-connector-zeroentropy/pyproject.toml | Declares the new package metadata, deps, and tool config. |
| .github/workflows/publish-moss-connector-zeroentropy.yml | Adds a publish workflow mirroring existing connector publish workflows. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codex reviewThe connector implementation is mostly coherent, but the release workflow has a release-integrity hazard around version reuse. |
- Copilot: reject include_content=False with the default mapper (it reads 'content', so it would silently index empty documents). Fail fast in __init__ with a clear error; added a unit test. - Codex (blocking): the publish step ran inside one matrix leg, so it could upload+tag before other Python versions finished. Split into a build-test matrix + a separate publish job that needs the whole matrix.
HarshaNalluru
commented
Jul 25, 2026
| TWINE_USERNAME: __token__ | ||
| TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} | ||
| run: | | ||
| twine upload --skip-existing packages/moss-data-connector/moss-connector-zeroentropy/dist/* |
There was a problem hiding this comment.
BLOCKING ```yaml
twine upload --skip-existing packages/moss-data-connector/moss-connector-zeroentropy/dist/*
`--skip-existing` makes a reused version look like a successful publish, so this job can still tag the current commit even when PyPI already contains artifacts built from a different commit. Remove `--skip-existing` and fail on existing versions, or pre-check the tag/PyPI version and only skip when you can verify the existing artifacts correspond to this exact commit.
ashvathsureshkumar
approved these changes
Jul 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A new data connector,
moss-connector-zeroentropy, that migrates a ZeroEntropy collection into a Moss index. Follows the establishedmoss-data-connector/_templatepattern exactly (same shape as the Supabase/MongoDB/DynamoDB connectors).How it works
ZeroEntropyConnector.__iter__walksdocuments.get_info_list(an auto-paginating cursor) and yields onemoss.DocumentInfoper document via amapper.path-> id, parsedcontent-> text,metadata-> metadata) means a whole collection migrates with no mapper; passmapper=to customize.documents.get_info(..., include_content=True). Documents that never parsed (nocontent) are skipped rather than indexed empty.Dict[str, str | list[str]]; the connector coerces list values to strings (Moss requiresDict[str, str]).path_prefix=scopes the migration;include_content=Falsemigrates metadata only.Testing
tests/test_zeroentropy.py— 9 mocked unit tests, all passing (default/custom mapper, skip-empty-content,include_content=False,path_prefix, auto_id, empty collection, metadata coercion).tests/test_integration_zeroentropy_moss.py— skippable live round trip (ZeroEntropy → Moss → query → delete), gated onZEROENTROPY_API_KEY/ZEROENTROPY_TEST_COLLECTION/MOSS_PROJECT_ID/MOSS_PROJECT_KEY.zeroentropy0.1.0a11 SDK by introspection: method names, keyword params (collection_name,path,include_content,path_prefix), theSyncGetDocumentInfoListCursorreturn, and every response field the connector reads all match.Also
publish-moss-connector-zeroentropy.yml(mirrors the other connectors' publish workflows).packages/moss-data-connector/README.md.CI is green (python-lint + python-sdk-test pass under the pinned ruff from #480).