Skip to content

Add Eden AI provider (fresh PR, addresses previous review) - #3764

Closed
MVS-source wants to merge 3 commits into
anomalyco:devfrom
MVS-source:add-edenai-provider-v2
Closed

Add Eden AI provider (fresh PR, addresses previous review)#3764
MVS-source wants to merge 3 commits into
anomalyco:devfrom
MVS-source:add-edenai-provider-v2

Conversation

@MVS-source

Copy link
Copy Markdown

Add Eden AI provider

Fresh PR from current dev, replacing the previous one (#2914, closed for staleness). This addresses every item from the earlier automated review.

Eden AI is an EU-based AI aggregation platform that exposes 100+ models from many providers behind a single OpenAI-compatible API (https://api.edenai.run/v3, Authorization: Bearer, provider/model format), with unified billing/usage tracking and EU data residency options. This adds the edenai provider using @ai-sdk/openai-compatible (no new dependency), matching the pattern of existing aggregator entries like openrouter / kilo.

Response to the previous review

  • base_model conversion (blocker): all 16 models that have a matching models/<provider>/<model>.toml entry now use base_model instead of duplicating provider-agnostic facts, keeping only [cost] and any Eden-specific overrides. mistral/open-mistral-nemo.toml now points at base_model = "mistral/mistral-nemo" (same underlying model). Only mistral/ministral-8b-latest.toml stays fully inline, since no models/mistral/ministral-8b-latest.toml exists.
  • release_date predating knowledge: fixed on deepseek-chat, mistral-small-latest, and devstral-medium-latest by dropping the stale inline dates and inheriting the correct ones from the canonical models/ entry via base_model.
  • mistral-medium-latest reasoning: now inherits reasoning = true from the canonical entry (it aliases Mistral Medium 3.5), instead of the stale reasoning = false left over from an older Medium generation.
  • reasoning_options evidence: tested reasoning_effort live against https://api.edenai.run/v3/chat/completions for every reasoning model, using a 3-clue logic puzzle prompt and reading usage.completion_tokens_details.reasoning_tokens:
    • anthropic/claude-sonnet-4-5: 271 (low) -> 311 (high)
    • google/gemini-2.5-flash: 835 (low) -> 1019 (high)
    • google/gemini-2.5-flash-lite: 789 (low) -> 1667 (high)
    • google/gemini-2.5-pro: 656 (low) -> 2079 (high)
    • anthropic/claude-haiku-4-5: stayed in a narrow band across low/medium/high across two separate prompts (e.g. 251/253/260); kept reasoning_options since the parameter is accepted and forwarded (non-zero reasoning tokens), but documented that the effect is small for this model.
    • mistral/mistral-small-latest and mistral/mistral-medium-latest: the response never includes completion_tokens_details (it is null), and completion length does not track effort consistently (626 -> 533 tokens from low to high on the same prompt). Set reasoning_options = [] on both per the "no verified control" guidance in AGENTS.md.
    • Each file has the specific numbers in a top-of-file comment.
  • Citations: every changed file now has a top-of-file comment citing Eden AI's live /v3/models endpoint for cost/limit data (verified 2026-07-28), plus the reasoning test results where relevant. deepseek-chat's attachment = false override is backed by a live test: sending an image_url content block to it through Eden AI returns a text response saying it cannot see images.
  • Sync module (recommended, not a blocker): not included in this PR. Happy to add packages/core/src/sync/providers/edenai.ts in a follow-up if useful, Eden AI's /v3/models endpoint is rich enough to support it.

What's included

  • providers/edenai/provider.toml, providers/edenai/logo.svg
  • 18 models across anthropic, cohere, deepseek, google, mistral, and openai, all EU-friendly and popular picks. Pricing, context length and capabilities are from Eden AI's live /v3/models endpoint.

Validation

bun run validate passes with these entries included.

Notes

I'm from the Eden AI team and we'll maintain this provider entry going forward.

…te bugs

Addresses review feedback from the previous PR (anomalyco#2914, closed for staleness):

- Convert 16 of 18 model entries to base_model, matching the aggregator
  pattern used by openrouter/kilo instead of duplicating provider-agnostic
  facts inline. Only ministral-8b-latest stays inline (no matching
  models/ metadata entry exists for it).
- This also fixes several stale facts that were hardcoded out of sync with
  the canonical models/ entries: wrong release_date on 9 models, a
  release_date that postdated its own knowledge cutoff on 3 models
  (deepseek-chat, mistral-small-latest, devstral-medium-latest), a stale
  reasoning=false on mistral-medium-latest (it now aliases Mistral Medium
  3.5, which reasons), and open_weights hardcoded to false on 8 models
  that are actually open weight per their canonical entry.
- Verified reasoning_effort live against Eden AI's chat completions
  endpoint for every reasoning model. For anthropic/claude-sonnet-4-5 and
  the three gemini models, reasoning_tokens usage scales with effort
  (documented per-file with the measured token counts). For
  mistral-small-latest and mistral-medium-latest, Eden's response never
  includes completion_tokens_details and completion length does not
  track effort consistently, so reasoning_options is set to [] on both
  per the no-verified-control guidance in AGENTS.md. claude-haiku-4-5
  keeps reasoning_options since the parameter is accepted and forwarded,
  but the measured effect is small; documented as such.
- Verified deepseek-chat's attachment=false live: sending an image to it
  through Eden AI returns a text response saying it cannot see images.
- Added source citations as top-of-file comments (Eden AI's live
  /v3/models endpoint plus the reasoning_effort test results) so they
  survive if this provider ever gets a sync module.

All 18 entries still pass bun run validate.
@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [violation] providers/edenai/models/mistral/ministral-8b-latest.toml:7 - Check: Provider model capabilities must be internally consistent (attachment vs [modalities]). Why: attachment = true claims file/image inputs, but modalities.input = ["text"] only. Callers will see contradictory capability flags. Mistral’s own ministral-8b-latest uses attachment = false with text-only input. Action: Set attachment = false, or add the real non-text modalities Eden accepts and keep attachment aligned.
  • [medium] [possible mistake] providers/edenai/models/mistral/ministral-8b-latest.toml:4 - Check: Inline model facts should match one real Ministral 8B revision, not a hybrid. Why: release_date = "2024-10-16" / knowledge = "2024-10" match the original Ministral 8B, while context = 262144 and $0.15/$0.15 match later 8B-2512-style entries elsewhere; open_weights = false also conflicts with Mistral’s open-weight Ministral cataloging (open_weights = true). Action: Identify which Eden model ID this is (2024 8B vs 2512/latest), then align release_date, last_updated, knowledge, limit, open_weights, and name/description to that revision with a source note.
  • [medium] [possible mistake] providers/edenai/models/anthropic/claude-sonnet-4-5.toml:14 - Check: Provider [modalities] overrides replace inherited arrays and must be Eden-verified. Why: Seven base_model entries force input = ["text", "image"], dropping base pdf (Claude/GPT) and audio/video/pdf (Gemini). Comments only justify cost/reasoning, not modality cuts; comparable OpenAI-compatible aggregators keep the fuller sets. Action: Remove these [modalities] blocks to inherit base values, or keep them only with Eden /v3/models (or live request) evidence that those inputs are unsupported on Eden.

…y cuts

- ministral-8b-latest.toml mixed facts from two different Ministral
  generations (2024-era release_date/knowledge with 2512-era pricing and
  context), and set attachment=true while modalities.input was text-only,
  which is internally inconsistent. Checked Eden AI's live /v3/models
  entry for mistral/ministral-8b-latest (context_length=262144,
  input_modalities=[text,image], pricing $0.15/$0.15) against
  providers/openrouter/models/mistralai/ministral-8b-2512.toml: this is
  Ministral 3 8B (2512), not the original 2024 Ministral 8B. Rewrote the
  entry to match that revision consistently, output limit included.
- Removed the [modalities] overrides on the 7 base_model entries that
  narrowed input modalities (dropping pdf, or audio/video for Gemini)
  without any citation. No live evidence supports those cuts, so they
  now inherit the full modality set from the canonical models/ entry
  instead.

Still passes bun run validate.
@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [violation] providers/edenai/models/anthropic/claude-haiku-4-5.toml:13 - Check: reasoning_options must reflect a verified caller control on this provider; use [] when the model reasons but no control is verified. Why: The file claims effort values low/medium/high, but its own live results say reasoning tokens stay in a narrow band across those values and “does not clearly scale with effort.” Acceptance plus always-on thinking is not enough—the skill requires meaningful control, not a silently ineffective field. Catalog consumers would be told they can tune Haiku effort when Eden does not demonstrably honor it. Action: Set reasoning_options = [] for claude-haiku-4-5, or re-verify with stronger evidence that each effort value changes reasoning behavior (and document the request field/results); do not keep the three-value effort option on the current measurements.

My own measurements (documented in the previous commit) show reasoning_tokens
staying in a narrow, non-monotonic band across low/medium/high for this model,
which does not meet the bar for a verified control. Set reasoning_options = []
to match how mistral-small-latest and mistral-medium-latest are already
handled, instead of keeping the three-value effort option on evidence that
does not support it.
@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@MVS-source

Copy link
Copy Markdown
Author

Hi @thdxr @fwang @rekram1-node, following up on this one. All the review feedback from the previous round has been addressed and the bot found no actionable issues on the latest pass. Would one of you have a moment to take a look? Happy to adjust anything.

@MVS-source

Copy link
Copy Markdown
Author

Closing this in favor of #3929, which replaces the static TOML files with a live sync integration against Eden AI's /v3/models endpoint, a better fit given Eden AI's multi-vendor catalog. Thanks for the feedback.

@MVS-source MVS-source closed this Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant