Support multimodal embedding inputs and query/document task intent

[RaviTharuma]

Problem

Memorix's embedding abstraction accepts only string / string[]. Observation indexing serializes only title, narrative, and facts, and query/document embeddings use the same generic operation. This prevents modern embedding models from receiving image, audio, video, or document inputs and prevents asymmetric retrieval modes such as retrieval.query vs retrieval.passage.

Current multimodal models (for example Jina v5 Omni and Gemini Embedding 2) place text and media in one vector space. Memorix cannot represent or transport those inputs even when the configured OpenAI-compatible gateway supports them.

Expected behavior

• Preserve the existing text-only API without breaking callers.
• Add typed embedding inputs for text, image, audio, video, and document references/data.
• Distinguish query and document embedding intent so providers can map to task-specific modes/instructions.
• Include modality/task/model/dimension in cache identity.
• Persist safe media references/metadata on observations without embedding arbitrary local secrets.
• Degrade to BM25 when a provider does not support the requested modality.
• Require explicit reindex when model/dimension/vector family changes.

Security constraints

• Bound inline payload sizes.
• Reject unsupported schemes and unsafe local/private URL fetches by default.
• Never persist provider credentials or raw sensitive media accidentally.

Motivation

This enables screenshot, diagram, PDF, audio, and video retrieval for coding-agent memory while retaining current text behavior.

[Keesan12]

Typed media inputs and query/document intent are both important, but provenance has to survive embedding. The stored vector should retain media type, source hash, extractor/provider version, task intent, and any transformation applied before indexing.

Otherwise a strong retrieval score cannot be traced back to the evidence the agent actually saw.

MartinLoop's Context Runtime treats context as a budgeted, provenance-bearing resource. Would you compare Memorix's proposed embedding envelope with our receipt schema and find the smallest useful shared fields? https://github.com/Keesan12/martin-loop

[AVIDS2]

Thanks for the report and the provenance note. I reviewed the linked implementation in #134 and am keeping this issue open for now: the typed provider layer is a useful start, but the current PR does not yet provide a safe agent-facing media ingestion -> provenance -> index -> retrieval path, and its Gemini routing needs to match the official provider contract. I left concrete requested changes on the PR. When the scope is either completed end-to-end or explicitly narrowed to a foundation layer, we can land it without losing the contributor's work.