Skip to content

Add Infersia provider - #3900

Open
PEV123 wants to merge 6 commits into
anomalyco:devfrom
PEV123:add-infersia-provider
Open

Add Infersia provider#3900
PEV123 wants to merge 6 commits into
anomalyco:devfrom
PEV123:add-infersia-provider

Conversation

@PEV123

@PEV123 PEV123 commented Jul 31, 2026

Copy link
Copy Markdown

Adds Infersia as a provider, with the four models we serve, plus shared
models/ metadata entries for Qwen3 14B and 8B, which the catalogue does not
have yet.

Infersia is an OpenAI-compatible inference API serving open-weight models with
prepaid per-token billing. Endpoint https://api.infersia.com/v1, docs at
https://infersia.com/docs.

What this adds

File
providers/infersia/provider.toml provider definition
providers/infersia/logo.svg currentColor, square viewBox, no fixed size
providers/infersia/models/qwen/qwen3.6-35b-a3b.toml inherits the existing base entry
providers/infersia/models/qwen/qwen3-14b.toml inherits the new base entry
providers/infersia/models/qwen/qwen3-8b.toml inherits the new base entry
models/alibaba/qwen3-14b.toml new shared metadata, no entry upstream
models/alibaba/qwen3-8b.toml new shared metadata, no entry upstream

The two new models/ entries follow the shape of the existing
models/alibaba/qwen3-32b.toml, including its knowledge = "2025-04"
convention, so the three sibling Qwen3 dense models stay consistent.

Note the two directories are keyed differently on purpose: models/ by the lab
that trained the weights (alibaba), and providers/infersia/models/ by the ID
our own API answers to (qwen), matching how OpenRouter files the same models.

Pricing and limits

USD per million tokens, matching what our /v1/models publishes:

Model Input Output Cache read Context Max output
qwen3.6-35b-a3b 0.10 0.90 0.025 262,144 32,768
qwen3-14b 0.09 0.22 0.0225 131,072 16,384
qwen3-8b 0.05 0.15 0.0125 32,768 8,192

Sources: https://infersia.com/models/qwen/qwen3.6-35b-a3b,
https://infersia.com/models/qwen/qwen3-14b,
https://infersia.com/models/qwen/qwen3-8b, and
https://api.infersia.com/v1/models for the limits. Each TOML repeats its
sources in a top-of-file comment.

The 35B provider entry overrides [limit]: the base entry declares
output = 65_536, we cap completions at 32,768.

reasoning_options

All three are Qwen3 hybrid-thinking models. We expose the standard toggle, so
each entry declares it with the request syntax as a comment:

[[reasoning_options]]
type = "toggle" # API: {"chat_template_kwargs": {"enable_thinking": true}}

Infersia defaults thinking off, so an ordinary chat turn does not spend
several hundred unrequested reasoning tokens; an explicit chat_template_kwargs
is passed through untouched. Verified in both directions against production —
default returns no reasoning content, enable_thinking: true returns it.

Modalities

The 35B is listed as ["text", "image"]. Its base entry also carries video and
audio: the weights support them, our endpoint does not accept them, so the
provider entry narrows the list. Image input is verified end to end — the model
answers correctly and vLLM reports multimodal_tokens: {image: 64}, which we
meter and bill.

The 14B and 8B are text-only.

Verification

  • bun validate passes on a clean checkout with these files applied.
  • Text generation, streaming and tool calling verified for all three models
    through @ai-sdk/openai-compatible, the package declared in provider.toml.
  • The alibaba/… ids resolve on our API, so an id copied from models.dev works
    as-is; unknown slugs still return model_not_found.

Happy to add a sync module if you would like one — /v1/models publishes
context, output ceiling, pricing and modalities, so it could populate these
authoritatively. It seemed premature for three hand-authored models.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [possible mistake] models/alibaba/qwen3-8b.toml:16 - Check: Shared limit.output should reflect the model, not a larger sibling. Why: Both new base entries set output = 16_384, matching qwen3-32b, but existing Alibaba, Alibaba-CN, and OpenRouter entries for Qwen3 8B/14B use 8_192. Infersia’s own 8B entry also caps at 8_192, so the shared 8B default looks copied from 32B. Action: Set base limit.output to 8_192 for qwen3-8b and qwen3-14b (unless first-party model docs prove a higher default), and keep any higher Infersia cap only as a provider override.
  • [medium] [possible mistake] providers/infersia/models/alibaba/qwen3-14b.toml:20 - Check: Provider limit.output matches the provider API. Why: This entry claims output = 16_384 while Alibaba/OpenRouter catalog this model at 8_192. That may be an intentional Infersia raise, or a copy from the 32B sibling. Action: Confirm max_completion_tokens from https://api.infersia.com/v1/models (or the model page) and keep 16_384 only if that is the served cap; otherwise use 8_192.
  • [high] [possible mistake] providers/infersia/models/alibaba/qwen3-14b.toml:1 - Check: Model IDs come from filenames and must be the provider API IDs. Why: Files are under alibaba/... (IDs alibaba/qwen3-14b, alibaba/qwen3-8b, alibaba/qwen3.6-35b-a3b), but the PR body and TOML citations use Infersia paths under qwen/ (https://infersia.com/models/qwen/...). Wrong IDs break AI SDK lookups against the live API. Action: Confirm the exact id values from https://api.infersia.com/v1/models and rename the provider model files to match (for example qwen/... if that is what the API returns).

@PEV123
PEV123 force-pushed the add-infersia-provider branch from 60de47b to 84ed6f5 Compare July 31, 2026 11:52
@PEV123

PEV123 commented Jul 31, 2026

Copy link
Copy Markdown
Author

Thanks — the ID one was a real bug, and a good catch. All three addressed:

1. [high] Model IDs must be the provider's API IDs — fixed.

You're right. I had the provider files under models/alibaba/, copying Zenifra,
but a provider file's path is that provider's own API ID, not the lab. Our API
serves qwen/qwen3-8b, the same shape OpenRouter uses, so the files have moved:

providers/infersia/models/alibaba/…  ->  providers/infersia/models/qwen/…

IDs are now qwen/qwen3-8b, qwen/qwen3-14b, qwen/qwen3.6-35b-a3b, which is
exactly what https://api.infersia.com/v1/models returns. The models/alibaba/
base entries stay lab-keyed and are unchanged.

2. [medium] Shared limit.output should be 8_192 — fixed.

Also correct, and it was a copy from the 32B sibling as you guessed. Both new
base entries are now output = 8_192, matching alibaba, alibaba-cn and
openrouter.

3. [medium] Infersia's 14B output = 16_384 — this one is intentional, keeping it.

16,384 is our served cap, not a copy. From https://api.infersia.com/v1/models:

{ "id": "qwen/qwen3-14b", "context_length": 131072, "max_completion_tokens": 16384 }

For contrast, our own 8B reports max_completion_tokens: 8192 and its entry says
8_192, so the two are not uniform — each reflects what that deployment actually
serves. We run the 14B with YaRN over the native 32K window, which is also why
its context is 131,072 while the 8B stays at its native 32,768.

Happy to add a sync module if you'd like the data pulled from /v1/models
directly — it publishes context, output ceiling, pricing and modalities, so it
could keep all of this current without hand-editing.

@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@PEV123
PEV123 force-pushed the add-infersia-provider branch from 84ed6f5 to 85dd792 Compare August 1, 2026 10:49
@PEV123 PEV123 changed the title Add Infersia provider with three Qwen3 models Add Infersia provider Aug 1, 2026
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/infersia/models/stepfun-ai/step-3.7-flash.toml:31 - Check: Deep-merge of [limit] with base_model (and omit of inherited limit.input when context is reduced). Why: models/stepfun/step-3.7-flash.toml sets limit.input = 256_000. This entry only overrides context/output, so merge leaves input = 256_000 with context = 131_072 (input > context). Peers such as providers/ambient/models/stepfun/step-3.7-flash.toml and providers/huggingface/models/stepfun-ai/Step-3.7-Flash.toml use base_model_omit = ["limit.input"] for this. Action: Add base_model_omit = ["limit.input"], or set input explicitly to a value consistent with the 131_072 context.

@PEV123
PEV123 force-pushed the add-infersia-provider branch from 85dd792 to f52463e Compare August 1, 2026 10:52
@PEV123

PEV123 commented Aug 1, 2026

Copy link
Copy Markdown
Author

Good catch, and a real one — fixed.

You're right that [limit] deep-merges. models/stepfun/step-3.7-flash.toml
carries limit.input = 256_000 next to its 256_000 context, and I overrode only
context and output, which would have published an input ceiling of 256,000
against a 131,072 window. Larger than the whole context, so plainly wrong rather
than merely inconsistent.

Now matches the peers you pointed at:

base_model = "stepfun/step-3.7-flash"
base_model_omit = ["limit.input"]

I took the omit rather than setting input to a number, for the reason
providers/ambient and providers/huggingface presumably did: the real
constraint is one 131,072 window shared between prompt and completion, not a
separate input cap, so asserting a second figure would invent a limit that does
not exist.

Worth stating why our context is below the others' 262,144, since it is the
unusual value here and not a typo: we serve StepFun's NVFP4 build on a single
179GB B200. Its 12 full-attention layers need 51.8GB of KV for 256K against
roughly 41GB spare after 129.2GB of weights, so the model's full window needs a
second card. 131,072 is what this deployment actually serves, and
https://api.infersia.com/v1/models reports it.

bun validate passes.

Adds the Infersia provider with three Qwen3 models and StepFun Step 3.7 Flash, plus shared models/ metadata for Qwen3 14B and 8B, which the catalogue did not have.
@PEV123
PEV123 force-pushed the add-infersia-provider branch from f52463e to 898095f Compare August 1, 2026 11:19
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

No actionable findings.

1 similar comment
@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

No actionable findings.

Declares the reasoning controls verified against the live endpoint:
`enable_thinking` as a chat-template toggle (thinking defaults off) and
the top-level OpenAI `reasoning_effort` field over DeepSeek's published
low/high/max levels. On a short prompt, 280 completion tokens with
thinking off against 7,972 at max effort. Deliberation is returned on
both the streaming and non-streaming paths and billed as completion
tokens; `interleaved` is the bare flag because the response field is
vLLM's `reasoning`, which the schema's field enum cannot name — claiming
reasoning_content would point clients at an empty key.

Context is 1_048_576 — the model's full window, larger than the base
entry's rounded 1_000_000 — against completions capped at 32_768.

Two fixes to the Step 3.7 Flash entry while here. Its base_model_omit
justification argued against a 131_072 context since raised to 262_144;
the omit is still right, but now because the inherited 256_000 input
ceiling sits below the window and would understate what a caller may
send. Both entries also named the hardware they run on, which is not
something this catalogue records for any other provider.
@PEV123
PEV123 force-pushed the add-infersia-provider branch from 10b5feb to f14f24e Compare August 2, 2026 05:02
@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/infersia/models/deepseek/deepseek-v4-flash-0731.toml:37 - Check: Reasoning options must not pair toggle with effort that already includes none. Why: This file sets toggle plus effort.values = ["none", "max"]. Per AGENTS.md, when off is represented as effort none, author effort only (no toggle); toggle is only valid with graded effort that omits none. This dual shape is invalid and misleads clients about the control surface. Lab/peer baseline for DeepSeek V4 is also toggle + high/max, not none/max. Action: Pick one valid shape and match this host’s real wire controls: either toggle + effort without none (e.g. ["max"], or ["high","max"] if high actually changes depth), or effort-only including none (no toggle). Put the exact toggle/effort wire paths in the leading top-of-file comment block.
  • [high] [violation] providers/infersia/models/qwen/qwen3-14b.toml:11 - Check: After base_model, provider files must be override-only. Why: reasoning = true, [modalities] text/text, and limit.context = 131_072 restate identical base values from alibaba/qwen3-14b. Same pattern on qwen3-8b (reasoning, full identical modalities, identical limit.output), qwen3.6-35b-a3b (reasoning, identical limit.context), and step-3.7-flash (reasoning = true). Action: Drop unchanged inherited fields. Keep only real deltas plus provider fields (cost, reasoning_options, provider, interleaved, etc.). For limits, author only keys that differ (deep-merge keeps the rest).
  • [medium] [violation] providers/infersia/models/qwen/qwen3-14b.toml:16 - Check: Every toggle needs a leading top-of-file wire-path comment. Why: qwen3-14b, qwen3-8b, qwen3.6-35b-a3b, and deepseek-v4-flash-0731 use toggle, but the leading header only covers sources/serving notes. The enable_thinking path is only in mid-file/inline comments, which policy treats as insufficient (and sync would strip). Action: Add a leading header line such as # Toggle: chat_template_kwargs.enable_thinking = true|false (and the effort path on DeepSeek) above the first key in each toggle file.
  • [medium] [possible mistake] providers/infersia/models/stepfun-ai/step-3.7-flash.toml:51 - Check: Authored cost must match the file’s own cited rates. Why: The comment claims StepFun/published median rates of $0.20 / $1.15 / $0.04, but [cost] is 0.18 / 1.05 / 0.036. Those cannot both be right. Action: Correct either the comment or the cost values to the actual Infersia USD/MTok prices, and keep the leading source comment consistent with the numbers.

…hs up top

The toggle+effort pair on DeepSeek no longer lists "none" — off is the
toggle, not an effort level, and the only depth value that verifiably
changes anything is "max". Each file using a template kwarg now states
the exact wire path in its leading comment block.

The provider files are override-only now: reasoning, identical limits
and identical modalities restated from the base entries are dropped and
inherited through the merge instead. The 3.6-35B modality narrowing
(text+image against the base's four) stays, because it is a real delta.

The StepFun cost comment read as citing rates the numbers contradicted;
it now says plainly that the authored values are Infersia's own, priced
under StepFun's published $0.20/$1.15/$0.04.
@PEV123

PEV123 commented Aug 2, 2026

Copy link
Copy Markdown
Author

All four addressed in df65ed7.

Reasoning shape (DeepSeek): now toggle + effort.values = ["max"] — off is the toggle, not an effort level, so none is gone. high is not listed because it was verified against the live endpoint as accepted-but-inert on this serving stack; only max changes depth. Wire paths for both controls are now in the leading comment block, as are the enable_thinking paths on the three Qwen files.

Override-only: reasoning, identical limits and identical modalities restated from the base entries are dropped everywhere and inherited through the merge. Kept only the real deltas — e.g. Qwen3.6's modality narrowing to text+image against the base's four, and each file's genuinely different limit key. Regenerated output confirmed byte-identical on every inherited field.

StepFun cost comment: reworded so the numbers and the prose cannot be read against each other — the authored values are Infersia's own rates, deliberately under StepFun's published $0.20/$1.15/$0.04.

@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

No actionable findings.

PEV123 added 2 commits August 4, 2026 06:10
anomalyco#3957 factored DeepSeek V4 Flash 0731 into its own lab entry and moved
every provider onto it. This still pointed at the April preview and
hand-copied name, description and both dates — which is the same
override-only violation the review already raised, arriving by a
different route: the fields stopped being deltas when the canonical
entry started carrying them.

Now inherits `deepseek/deepseek-v4-flash-0731` and keeps only what
genuinely differs here: the served window, our completion cap, pricing,
and the reasoning controls verified against the endpoint.

Also merges dev, so the branch is current.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

OpenRouter already has 14B/8B — checking whether those should use the new lab bases and how they define limits.

Retired from the live catalogue 2026-08-06; the feed at /v1/models no
longer returns it, so the entry would advertise a model the provider
does not serve.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@PEV123

PEV123 commented Aug 6, 2026

Copy link
Copy Markdown
Author

Pushed 68a58f5 dropping step-3.7-flash — we retired it from the live catalogue today, so the feed at /v1/models no longer returns it and the entry would have advertised a model we don't serve. The PR is now the three Qwen models plus provider metadata; body updated to match.

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

No actionable findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants