feat(ai-proxy): add Anthropic LLM provider support#1435
Conversation
4 new issues
|
|
Coverage Impact ⬆️ Merging this pull request will increase total coverage on Modified Files with Diff Coverage (7)
🤖 Increase coverage with AI coding...🚦 See full report on Qlty Cloud » 🛟 Help
|
| tool_calls: msg.tool_calls.map(tc => ({ | ||
| id: tc.id, | ||
| name: tc.function.name, | ||
| args: JSON.parse(tc.function.arguments), |
There was a problem hiding this comment.
JSON.parse() can throw if tc.function.arguments contains malformed JSON. This call is outside the try-catch block (line 210), so errors would propagate as unhandled SyntaxError instead of being wrapped in AnthropicUnprocessableError.
OpenAI API can return malformed JSON in edge cases (documented issues).
Fix packages/ai-proxy/src/provider-dispatcher.ts:259: Wrap JSON.parse in try-catch or move convertMessagesToLangChain call inside the existing try-catch block at line 210
| }); | ||
| } | ||
|
|
||
| return new AIMessage(msg.content); |
There was a problem hiding this comment.
LangChain message constructors (SystemMessage, HumanMessage, AIMessage) fail with null content. OpenAI API allows content: null for assistant messages (API spec).
The OpenAIMessage interface at line 117 defines content: string, but should allow null. Line 255 handles this correctly with msg.content || '', but lines 249, 251, 264, 267, 271 pass content directly.
| return new AIMessage(msg.content); | |
| return new AIMessage(msg.content || '\); |
482f6b8 to
0fe24b9
Compare
e121d6a to
4fa181e
Compare
f938960 to
503736c
Compare
cabba2c to
4bd08ce
Compare
BREAKING CHANGE: isModelSupportingTools is no longer exported from ai-proxy - Add AIModelNotSupportedError for descriptive error messages - Move model validation from agent.addAi() to Router constructor - Make isModelSupportingTools internal (not exported from index) - Error is thrown immediately at Router init if model doesn't support tools This is a bug fix: validation should happen at proxy initialization, not at the agent level. This ensures consistent behavior regardless of how the Router is instantiated. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add comprehensive integration tests that run against real OpenAI API: - ai-query route: simple chat, tool calls, tool_choice, parallel_tool_calls - remote-tools route: listing tools (empty, brave search, MCP tools) - invoke-remote-tool route: error handling - MCP server integration: calculator tools with add/multiply - Error handling: validation errors Also adds: - .env-test support for credentials (via dotenv) - .env-test.example template for developers - Jest setup to load environment variables Run with: yarn workspace @forestadmin/ai-proxy test openai.integration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…adiness - Add multi-turn conversation test with tool results - Add AINotConfiguredError test for missing AI config - Add MCP error handling tests (unreachable server, auth failure) - Skip flaky tests due to Langchain retry behavior - Ensure tests work on main branch without Zod validation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Set maxRetries: 0 by default when creating ChatOpenAI instance. This makes our library a simple passthrough without automatic retries, giving users full control over retry behavior. Also enables previously skipped integration tests that were flaky due to retry delays. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Addresses all issues identified in PR review: - Add invoke-remote-tool success tests for MCP tools (add, multiply) - Strengthen weak error assertions with proper regex patterns - Fix 'select AI configuration by name' test to verify no fallback warning - Add test for fallback behavior when config not found - Add logger verification in MCP error handling tests Tests now verify: - Error messages match expected patterns (not just toThrow()) - Logger is called with correct level and message on errors - Config selection works without silent fallback Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The test was incorrectly checking for finish_reason: 'tool_calls'. When forcing a specific function via tool_choice, OpenAI returns finish_reason: 'stop' but still includes the tool_calls array. The correct assertion is to verify the tool_calls array contains the expected function name, not the finish_reason. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move the agent-specific message "Please call addAi() on your agent" from ai-proxy to the agent package where it belongs. - ai-proxy: AINotConfiguredError now uses generic "AI is not configured" - agent: Catches AINotConfiguredError and adds agent-specific guidance This keeps ai-proxy decoupled from agent-specific terminology. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The test was not properly waiting for the HTTP server to close. Changed afterAll to use a Promise wrapper around server.close() callback. This removes the need for forceExit: true in Jest config. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests typically complete in 200-1600ms. 30 second timeouts were excessive. - Single API calls: 10s timeout (was 30s) - Multi-turn conversation: 15s timeout (was 60s) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add support for Anthropic's Claude models in the ai-proxy package using @langchain/anthropic. This allows users to configure Claude as their AI provider alongside OpenAI. Changes: - Add @langchain/anthropic dependency - Add ANTHROPIC_MODELS constant with supported Claude models - Add AnthropicConfiguration type and AnthropicModel type - Add AnthropicUnprocessableError for Anthropic-specific errors - Implement message conversion from OpenAI format to LangChain format - Implement response conversion from LangChain format back to OpenAI format - Add tool binding support for Anthropic with tool_choice conversion - Add comprehensive tests for Anthropic provider Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… dispatcher - Move convertMessagesToLangChain inside try-catch to properly handle JSON.parse errors - Update OpenAIMessage interface to allow null content (per OpenAI API spec) - Add null content handling for all message types with fallback to empty string Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Mirror OpenAI integration tests for Anthropic provider. Requires ANTHROPIC_API_KEY environment variable. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Both root and ai-proxy resolve to the same hoisted version, so the local override is unnecessary. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Only ai-proxy uses @anthropic-ai/sdk, no need to configure it globally. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use SDK-native model types (OpenAI.ChatModel, Anthropic.Messages.Model)
instead of LangChain re-exports that added a double (string & {}).
Also Omit model from BaseAiConfiguration in AnthropicConfiguration
to prevent string intersection from erasing literal autocomplete.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
| export class AnthropicUnprocessableError extends AIUnprocessableError { | ||
| constructor(message: string) { | ||
| super(message); | ||
| this.name = 'AnthropicError'; |
There was a problem hiding this comment.
this.name should be the same as the class name no ?
| this.name = 'AnthropicError'; | |
| this.name = 'AnthropicUnprocessableError'; |
| ErrorClass: typeof OpenAIUnprocessableError | typeof AnthropicUnprocessableError, | ||
| providerName: string, |
There was a problem hiding this comment.
you should have only one of these params
and the other one is based on it
if you keep providerName: 'Anthropic' | 'OpenAi', you can get the error class from it
const errorClass = providerName === 'Anthropic' ? AnthropicUnprocessableError : OpenAIUnprocessableError| // disable_parallel_tool_use, but the Anthropic API supports it and LangChain passes it through | ||
| const model = enhancedTools?.length | ||
| ? this.anthropicModel.bindTools(enhancedTools, { | ||
| tool_choice: this.convertToolChoiceForAnthropic(toolChoice, parallelToolCalls) as any, |
There was a problem hiding this comment.
| tool_choice: this.convertToolChoiceForAnthropic(toolChoice, parallelToolCalls) as any, | |
| tool_choice: this.convertToolChoiceForAnthropic(toolChoice, parallelToolCalls) as AnthropicToolChoice, |
IMO we should cast it to AnthropicToolChoice or set the method response type to AnthropicToolChoice
| private convertMessagesToLangChain(messages: OpenAIMessage[]): BaseMessage[] { | ||
| return messages.map(msg => { | ||
| switch (msg.role) { | ||
| case 'system': | ||
| return new SystemMessage(msg.content || ''); | ||
| case 'user': | ||
| return new HumanMessage(msg.content || ''); | ||
| case 'assistant': | ||
| if (msg.tool_calls) { | ||
| return new AIMessage({ | ||
| content: msg.content || '', | ||
| tool_calls: msg.tool_calls.map(tc => ({ | ||
| id: tc.id, | ||
| name: tc.function.name, | ||
| args: ProviderDispatcher.parseToolArguments( | ||
| tc.function.name, | ||
| tc.function.arguments, | ||
| ), | ||
| })), | ||
| }); | ||
| } | ||
|
|
||
| return new AIMessage(msg.content || ''); | ||
| case 'tool': | ||
| if (!msg.tool_call_id) { | ||
| throw new AIBadRequestError('Tool message is missing required "tool_call_id" field.'); | ||
| } | ||
|
|
||
| if (err.status === 401) { | ||
| throw new OpenAIUnprocessableError(`Authentication failed: ${err.message}`); | ||
| return new ToolMessage({ | ||
| content: msg.content || '', | ||
| tool_call_id: msg.tool_call_id, | ||
| }); | ||
| default: | ||
| throw new AIBadRequestError( | ||
| `Unsupported message role '${msg.role}'. Expected: system, user, assistant, or tool.`, | ||
| ); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| throw new OpenAIUnprocessableError(`Error while calling OpenAI: ${err.message}`); | ||
| private static parseToolArguments(toolName: string, args: string): Record<string, unknown> { | ||
| try { | ||
| return JSON.parse(args); | ||
| } catch { | ||
| throw new AIBadRequestError( | ||
| `Invalid JSON in tool_calls arguments for tool '${toolName}': ${args}`, | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| private convertToolChoiceToLangChain( | ||
| toolChoice: ChatCompletionToolChoice | undefined, | ||
| ): 'auto' | 'any' | 'none' | { type: 'tool'; name: string } | undefined { | ||
| if (!toolChoice) return undefined; | ||
| if (toolChoice === 'auto') return 'auto'; | ||
| if (toolChoice === 'none') return 'none'; | ||
| if (toolChoice === 'required') return 'any'; | ||
|
|
||
| if (typeof toolChoice === 'object' && toolChoice.type === 'function') { | ||
| return { type: 'tool', name: toolChoice.function.name }; | ||
| } | ||
|
|
||
| throw new AIBadRequestError( | ||
| `Unsupported tool_choice value. Expected: 'auto', 'none', 'required', or {type: 'function', function: {name: '...'}}.`, | ||
| ); | ||
| } | ||
|
|
||
| /** | ||
| * Convert tool_choice to Anthropic format, supporting disable_parallel_tool_use. | ||
| * | ||
| * When parallel_tool_calls is false, Anthropic requires the tool_choice to be | ||
| * an object with `disable_parallel_tool_use: true`. | ||
| * LangChain passes objects through directly to the Anthropic API. | ||
| */ | ||
| private convertToolChoiceForAnthropic( | ||
| toolChoice: ChatCompletionToolChoice | undefined, | ||
| parallelToolCalls?: boolean, | ||
| ) { | ||
| const base = this.convertToolChoiceToLangChain(toolChoice); | ||
|
|
||
| if (parallelToolCalls !== false) return base; | ||
|
|
||
| // Anthropic requires object form to set disable_parallel_tool_use | ||
| if (base === undefined || base === 'auto') { | ||
| return { type: 'auto', disable_parallel_tool_use: true }; | ||
| } | ||
|
|
||
| if (base === 'any') { | ||
| return { type: 'any', disable_parallel_tool_use: true }; | ||
| } | ||
|
|
||
| if (base === 'none') return 'none'; | ||
|
|
||
| return { ...base, disable_parallel_tool_use: true }; | ||
| } | ||
|
|
||
| private static extractTextContent(content: AIMessage['content']): string | null { | ||
| if (typeof content === 'string') return content || null; | ||
|
|
||
| if (Array.isArray(content)) { | ||
| const text = content | ||
| .filter(block => block.type === 'text') | ||
| .map(block => ('text' in block ? block.text : '')) | ||
| .join(''); | ||
|
|
||
| return text || null; | ||
| } | ||
|
|
||
| return null; | ||
| } | ||
|
|
||
| private convertLangChainResponseToOpenAI(response: AIMessage): ChatCompletionResponse { | ||
| const toolCalls = response.tool_calls?.map(tc => ({ | ||
| id: tc.id || `call_${crypto.randomUUID()}`, | ||
| type: 'function' as const, | ||
| function: { | ||
| name: tc.name, | ||
| arguments: JSON.stringify(tc.args), | ||
| }, | ||
| })); | ||
|
|
||
| const usageMetadata = response.usage_metadata as | ||
| | { input_tokens?: number; output_tokens?: number; total_tokens?: number } | ||
| | undefined; | ||
|
|
||
| return { | ||
| id: response.id || `msg_${crypto.randomUUID()}`, | ||
| object: 'chat.completion', | ||
| created: Math.floor(Date.now() / 1000), | ||
| model: this.modelName, | ||
| choices: [ | ||
| { | ||
| index: 0, | ||
| message: { | ||
| role: 'assistant', | ||
| content: ProviderDispatcher.extractTextContent(response.content), | ||
| refusal: null, | ||
| tool_calls: toolCalls?.length ? toolCalls : undefined, | ||
| }, | ||
| finish_reason: toolCalls?.length ? 'tool_calls' : 'stop', | ||
| logprobs: null, | ||
| }, | ||
| ], | ||
| usage: { | ||
| prompt_tokens: usageMetadata?.input_tokens ?? 0, | ||
| completion_tokens: usageMetadata?.output_tokens ?? 0, | ||
| total_tokens: usageMetadata?.total_tokens ?? 0, | ||
| }, | ||
| }; | ||
| } | ||
|
|
There was a problem hiding this comment.
IMO since this is some kind of serialization / deserialization it should be in another file or service
…lity Anthropic only allows a single system message at position 0. Merge all system messages into one before dispatching. Also unify provider error classes into AIUnprocessableError, type tool_choice properly, bump @langchain/anthropic to 1.3.17 and add integration test for the case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract conversion logic from ProviderDispatcher into a dedicated LangChainAdapter class with clear separation between generic (convertMessages, convertResponse, convertToolChoice) and Anthropic-specific methods (mergeSystemMessages, withParallelToolCallsRestriction). Also fixes: AIToolUnprocessableError.name, wrapProviderError cause preservation, unknown provider guard in constructor. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move Anthropic-specific logic (system message merging, disable_parallel_tool_use) into a dedicated AnthropicAdapter that composes with the generic LangChainAdapter. This keeps LangChainAdapter purely generic and reusable for any LangChain-based provider. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…Choice Both parameters are optional, an object param is clearer than positional undefined values. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…pported list Add claude-3-5-haiku (EOL 2026-02-19) and claude-opus-4/4-1 (require streaming) to the unsupported models list. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace deprecated claude-3-5-haiku-latest with claude-haiku-4-5-20251001. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
claude-3-5-haiku is deprecated but still functional. Only Opus models that require streaming should be in the unsupported list. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… handling Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
55b40cc to
0d0574e
Compare
…r cause Move tool binding logic into AnthropicAdapter.bindTools(), making convertToolChoice private. Replace unsafe cast for error cause with a proper constructor option on AIUnprocessableError. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… safety - Narrow dispatchOpenAI try-catch to only wrap model.invoke(), preventing internal assertions from being routed through wrapProviderError - Add provider name and cause chain to 429/401 error wrapping - Make tool_call_id required on OpenAIToolMessage to match runtime contract - Clarify convertToolChoice JSDoc about strict false check Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Keep only model.invoke() inside the try-catch, consistent with the OpenAI dispatch path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…se JSDoc - Simplify isOpenAIModelSupported to single boolean return - Add explicit return type to enrichToolDefinitions - Type response variable in dispatchOpenAI for consistency with Anthropic path - Condense bindTools JSDoc to essential one-liner Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
| if (status === 429) { | ||
| return new AIUnprocessableError(`${providerName} rate limit exceeded: ${error.message}`, { | ||
| cause: error, | ||
| }); | ||
| } | ||
|
|
||
| if (status === 401) { | ||
| return new AIUnprocessableError(`${providerName} authentication failed: ${error.message}`, { | ||
| cause: error, | ||
| }); | ||
| } |
There was a problem hiding this comment.
use dedicated errors instead of generi ones
Currently all provider errors are wrapped as AIUnprocessableError, losing the original HTTP semantics (429 rate limit, 401 auth failure). Documents the plan to add proper error types in datasource-toolkit and ai-proxy for correct HTTP status mapping. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This model only supports v1/completions, not v1/chat/completions, causing integration test failures. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
# @forestadmin/ai-proxy [1.5.0](https://github.com/ForestAdmin/agent-nodejs/compare/@forestadmin/ai-proxy@1.4.3...@forestadmin/ai-proxy@1.5.0) (2026-02-17) ### Features * **ai-proxy:** add Anthropic LLM provider support ([#1435](#1435)) ([f748b19](f748b19))

Summary
@forestadmin/ai-proxyusing@langchain/anthropicname,provider,model,apiKey)Changes
@langchain/anthropicdependencyAnthropicConfigurationtype with all Anthropic-specific optionsANTHROPIC_MODELSconstant with supported Claude modelsAnthropicUnprocessableErrorfor error handlingTest plan
🤖 Generated with Claude Code