[DCP Ingestion] Namespace support in preprocessor

[gmechali]

Standardize on provenance_id across graph nodes and enable per-provenance GCS handshakes

• Per-Provenance GCS Handshakes: Update dcpbridge mode to generate a per-provenance {workflow_id}.json handshake manifest where importList specifies each individual provenance folder (provenance/<name>) and its shard glob (**/*.jsonld).
• Standardize Data Model: Replace provenance_dir(s) with provenance_id(s) across Entity, Property, Provenance, and Source to match StatVar, EventType, and EntityType.
• Remove Upfront Normalization: Eliminate boilerplate config.import_name() and provenance_dir piping in importers (Observations, VariablePerRow, Events, Mcf) and runner IPC serialization.
• Centralize Shard Bucketing: Consolidate folder normalization (strip_namespace(prov_id)) to a single location at the JSON-LD export boundary in Nodes.triples_by_provenance_dir().

[codacy-production]

Not up to standards ⛔

🔴 Issues 3 medium · 4 minor

Alerts:
⚠ 7 issues (≤ 0 issues of at least minor severity)

Results:
7 new issues

Category	Results
UnusedCode	2 minor
Security	1 medium
CodeStyle	2 minor
Complexity	2 medium

View in Codacy

🟢 Metrics 54 complexity

Metric	Results
Complexity	54

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}

[gemini-code-assist]

Code Review

This pull request introduces parallel processing for CSV imports using a ProcessPoolExecutor in the Runner, along with changes to prevent file collisions by incorporating the file name into observation shard names. It also adds GCS upload retry logic, improves custom namespace handling across importers and validators, and optimizes custom dimension serialization. Feedback on these changes highlights a fragile design pattern in _write_observation_shard where the first parameter is overloaded to accept either a list or a tuple, and warns of a potential collision risk for node shards similar to the one resolved for observation shards.

[gmechali]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces changes to support grouping and exporting triples by their provenance directories, handling custom namespace prefixes during entity resolution, and preventing the duplication of global triples across multiple imports. It updates several importers, the node registry, and database interfaces to track and propagate provenance directories. A correctness bug was identified in events_importer.py where unresolved entity names are registered before they are resolved, which can lead to invalid registrations and missing triples.

[clincoln8]

Thank you!!!!!!

Nonblocking questions / comments. Can likely be addressed in followups if needed.

[gmechali]

Thanks Christie! Fixed like half the comments, the half regarding namespaces i think will need to be hanlded in the namespace work coming up, but agree fully on the cleanup needed :)