Add sync for tensorx - #4035
Conversation
Action items
|
|
1. 2. GLM
Each list now carries only accepted values. 3. DeepSeek V4 effort — verified, not actionable. TensorX accepts 4. 5. |
Action items
|
|
1. Partial pricing — fixed. Confirmed: 2. GLM effort — fixed, and the prior fix had it backwards. You're right that acceptance isn't a distinct control. First-party 3. kimi-k3 — fixed. |
Action items
|
|
Both findings correctly identified a real contradiction — the header claimed no reasoning was observed while the file asserted a Root cause: I measured the wrong signal. My earlier runs used Re-measured against
So the side channel is real and the toggle is honored. All three proposed actions — dropping Fixed in Because that metric had misled me once, I re-audited every other "this control has no effect" claim in the PR using content length. All held: |
Action items
|
|
1–4. Token-count evidence — valid objection, all four re-probed on
kimi-k3 changed. On the content signal, Every header now cites the content-based measurement and the run counts, and notes that
|
Action items
|
Action items
|
|
1. Nemotron toggle wire path — fixed, path found. You were right that shipping a toggle with an admittedly unknown wire syntax claims a capability the file doesn't define. It turned out to be documented: Workers AI hosts this exact model and specifies 2. attachment vs non-image modalities — fixed. Correct, and the MiniMax-M3 case you named was the real hole: |
|
No actionable findings. |
Sync provider models from TensorX's own /v1/model/info endpoint. Capability flags are three-valued in that catalog: true, false, and null for "not published". Only real booleans are authored so a null leaves the field inherited from the lab entry rather than overriding it with false. supported_openai_params is deliberately not read. It is a static LiteLLM-style list — all 32 catalog entries advertise the same params, including temperature and response_format on the Whisper and embedding models — so it carries no per-model signal. temperature and structured_output stay lab metadata. The endpoint publishes no name, description, or release date, so an unknown model is reported through skippedNotice for hand-authoring instead of being created from invented metadata. Base models resolve through the shared resolveCanonicalBaseModel, which matches metadata filenames case-insensitively.
The sync re-serializes provider files through Bun.TOML.parse, which discards comments; only the leading block is re-attached. Trailing comments on `type = "toggle"` / `type = "effort"` lines were therefore lost on the first sync run. AGENTS.md requires a leading top-of-file comment carrying the exact wire path for every toggle, so the request syntax moves there and survives future runs. No data changes.
Source: https://api.tensorx.ai/v1/model/info - max_output_tokens is published per model and was previously missing or stale on ten entries (deepseek-v4-*, kimi-k2.6/k2.7-code/k3, glm-5.1, glm-5.2, minimax-m3, qwen3.5-9b) - deepseek-v4-flash-0731 cache_read corrected 0.06 -> 0.0625 cache_creation_input_token_cost is null for every model in the catalog, so cache_write is left at its authored value rather than being dropped: a null there means "not published", not "not charged".
|
Reconstructing the final sync module and checking a few peer comparisons that look suspicious. |
Not sure what happened to the bot here, but I think this PR is now ready to be merged. (Last bot run said no actionable findings, and then I only rebased) |
Adds a sync module for TensorX and corrects the existing
providers/tensorxcatalog against the provider's own API.Sources
GET https://api.tensorx.ai/v1/model/infocost.input,cost.output,cost.cache_read;limit.context,limit.output;mode,supports_reasoning,supports_vision,supports_tool_choice/supports_function_callingPOST https://api.tensorx.ai/v1/chat/completionsreasoning_optionsandinterleavedclaim below, and the enforced completion capproviders/zai/models/glm-5.2.tomlhigh|maxproviders/openrouter/models/…andproviders/nvidia/models/nvidia/…How reasoning controls were measured
usage.completion_tokens_details.reasoning_tokensis unreliable on this host. GLM-5-turbo and GLM-5v-turbo stream ~3,000 characters intoreasoning_contentwhile reportingreasoning_tokens = 0. Every claim below is therefore measured byreasoning_contentlength, with a nonce in each prompt to defeat caching, repeated runs, and an invalid effort value as a control for silently-ignored parameters.Reasoning controls
Measured against the live API:
togglelow,high,maxand an invalid value all overlapeffort ["none","high"][]nonerejected, no level grades depth — matches the OpenRouter entry[]thinking=falseandeffort=noneboth 0/6 emptytogglethinking=falseempties reasoning; effort levels and an invalid value overlapeffort ["none","low","high","max"]noneempties 9/9 and depth gradeslow<high<max; toggle dropped —thinking=falseleaked in 3/9[]nonerejected, levels overlap, both toggle fields no-opseffort ["none","high"]noneempties; levels overlap across 4 runs each; toggle fields no-opseffort ["none","high"]enable_thinkingis a no-op heretoggleenable_thinking=falseemptiesreasoning_contenttoggle+effort ["high","max"]Carried, not measured —
/v1/model/infois a per-key view and these four return 403key not allowed to access model, so nothing could be probed. Each says so in its leading comment, and none gets[interleaved]rather than assuming one:toggleeffort ["low","medium","high"]minimax-m3keeps its previously authoredthinking_modetoggle; only its side channel was re-verified here.interleaved: all 18 reachable reasoning models returnreasoning_content, and each now declares it. None usereasoning_details.Limits
limit.outputis the API'smax_output_tokens, and it is enforced. Streamingdeepseek-v4-flashwithmax_tokens: 200000stopped at exactly 64,000 withfinish_reason: "length". Its lab entry claims 384,000, so inheriting the lab value would overstate real capacity ~6×.Sync module
skipCreates: true— the catalog exposes no name, description, release date, reasoning controls or side channel, so new IDs are reported for hand-authoring rather than published with the runner's fallbackreasoning_options = [].deleteMissing: false— absence from/v1/model/infomeans the key lacks access (403, not 404), not that a model was retired.base_modelalready authored locally wins over re-derivation; resolution uses the sharedresolveCanonicalBaseModel, which matches metadata filenames case-insensitively.input/outputpair;cache_writeis preserved when the API reports null — it is null for every model, meaning "not published", not "not charged".attachmentandmodalities.inputare derived together fromsupports_vision, touching onlyimage.temperatureandstructured_outputare deliberately not derived:supported_openai_paramsis a static list identical across all 32 entries, including the Whisper and embedding models.Validation
bun validatepasses.bun models:sync tensorx --dry-runagainst the live API reports 0 created, 0 updated, 0 removed, 25 unchanged — every hand edit matches serializer output, so nothing is clobbered on the next automated run.Out of scope
deepseek-v3.2andqwen3-235b-a22b-2507remain full-inline because nomodels/lab entry exists for either. Pre-existing; adding those affects other providers (302ai, vivgrid and siliconflow all carry inlinedeepseek-v3.2) and belongs in its own PR.To do before merging
A maintainer should create a free account and API key at Tensorx.ai because the sync pipeline endpoint is authenticated.