Skip to content

feat(ai-proxy): add Anthropic LLM provider support#1435

Merged
Scra3 merged 58 commits into
mainfrom
feat/prd-87-anthropic-support
Feb 17, 2026
Merged

feat(ai-proxy): add Anthropic LLM provider support#1435
Scra3 merged 58 commits into
mainfrom
feat/prd-87-anthropic-support

Conversation

@Scra3

@Scra3 Scra3 commented Jan 23, 2026

Copy link
Copy Markdown
Member

Summary

  • Add Anthropic provider support to @forestadmin/ai-proxy using @langchain/anthropic
  • Support all Claude models with tool calling capabilities
  • Maintain OpenAI-compatible response format for seamless integration
  • Unified configuration interface: both providers use same base fields (name, provider, model, apiKey)

Changes

  • Added @langchain/anthropic dependency
  • Added AnthropicConfiguration type with all Anthropic-specific options
  • Added ANTHROPIC_MODELS constant with supported Claude models
  • Implemented message conversion (OpenAI ↔ LangChain)
  • Implemented response conversion (LangChain → OpenAI format)
  • Added AnthropicUnprocessableError for error handling
  • Added comprehensive tests (38 new tests)

Test plan

  • All 86 tests pass
  • Build succeeds
  • Lint passes (only warnings for non-null assertions)

🤖 Generated with Claude Code

@linear

linear Bot commented Jan 23, 2026

Copy link
Copy Markdown

@qltysh

qltysh Bot commented Jan 23, 2026

Copy link
Copy Markdown

4 new issues

Tool Category Rule Count
qlty Structure Function with many returns (count = 5): convertToolChoice 3
qlty Structure Function with high complexity (count = 15): convertMessages 1

@qltysh

qltysh Bot commented Jan 23, 2026

Copy link
Copy Markdown

Qlty

Coverage Impact

⬆️ Merging this pull request will increase total coverage on main by 0.02%.

Modified Files with Diff Coverage (7)

RatingFile% DiffUncovered Line #s
Coverage rating: A Coverage rating: A
packages/ai-proxy/src/provider-dispatcher.ts100.0%
Coverage rating: A Coverage rating: A
packages/ai-proxy/src/supported-models.ts100.0%
Coverage rating: A Coverage rating: A
packages/ai-proxy/src/router.ts100.0%
Coverage rating: A Coverage rating: A
packages/ai-proxy/src/errors.ts100.0%
Coverage rating: A Coverage rating: A
packages/ai-proxy/src/index.ts100.0%
New file Coverage rating: A
packages/ai-proxy/src/anthropic-adapter.ts100.0%
New file Coverage rating: A
packages/ai-proxy/src/langchain-adapter.ts97.6%177
Total99.2%
🤖 Increase coverage with AI coding...

In the `feat/prd-87-anthropic-support` branch, add test coverage for this new code:

- `packages/ai-proxy/src/langchain-adapter.ts` -- Line 177

🚦 See full report on Qlty Cloud »

🛟 Help
  • Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.

  • Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.

  • File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)

    • Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

tool_calls: msg.tool_calls.map(tc => ({
id: tc.id,
name: tc.function.name,
args: JSON.parse(tc.function.arguments),

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSON.parse() can throw if tc.function.arguments contains malformed JSON. This call is outside the try-catch block (line 210), so errors would propagate as unhandled SyntaxError instead of being wrapped in AnthropicUnprocessableError.

OpenAI API can return malformed JSON in edge cases (documented issues).

Fix packages/ai-proxy/src/provider-dispatcher.ts:259: Wrap JSON.parse in try-catch or move convertMessagesToLangChain call inside the existing try-catch block at line 210

});
}

return new AIMessage(msg.content);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LangChain message constructors (SystemMessage, HumanMessage, AIMessage) fail with null content. OpenAI API allows content: null for assistant messages (API spec).

The OpenAIMessage interface at line 117 defines content: string, but should allow null. Line 255 handles this correctly with msg.content || '', but lines 249, 251, 264, 267, 271 pass content directly.

Suggested change
return new AIMessage(msg.content);
return new AIMessage(msg.content || '\);

@Scra3 Scra3 force-pushed the feat/prd-87-anthropic-support branch from 482f6b8 to 0fe24b9 Compare February 4, 2026 15:13
@Scra3 Scra3 changed the base branch from main to feat/ai-proxy-zod-validation February 4, 2026 15:21
@Scra3 Scra3 force-pushed the feat/prd-87-anthropic-support branch 2 times, most recently from e121d6a to 4fa181e Compare February 4, 2026 15:36
@Scra3 Scra3 force-pushed the feat/ai-proxy-zod-validation branch 2 times, most recently from f938960 to 503736c Compare February 5, 2026 11:15
Base automatically changed from feat/ai-proxy-zod-validation to main February 5, 2026 14:03
@Scra3

Scra3 commented Feb 5, 2026

Copy link
Copy Markdown
Member Author

@Scra3 Scra3 force-pushed the feat/prd-87-anthropic-support branch 5 times, most recently from cabba2c to 4bd08ce Compare February 7, 2026 10:50
alban bertolini and others added 12 commits February 13, 2026 14:47
BREAKING CHANGE: isModelSupportingTools is no longer exported from ai-proxy

- Add AIModelNotSupportedError for descriptive error messages
- Move model validation from agent.addAi() to Router constructor
- Make isModelSupportingTools internal (not exported from index)
- Error is thrown immediately at Router init if model doesn't support tools

This is a bug fix: validation should happen at proxy initialization,
not at the agent level. This ensures consistent behavior regardless
of how the Router is instantiated.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add comprehensive integration tests that run against real OpenAI API:

- ai-query route: simple chat, tool calls, tool_choice, parallel_tool_calls
- remote-tools route: listing tools (empty, brave search, MCP tools)
- invoke-remote-tool route: error handling
- MCP server integration: calculator tools with add/multiply
- Error handling: validation errors

Also adds:
- .env-test support for credentials (via dotenv)
- .env-test.example template for developers
- Jest setup to load environment variables

Run with: yarn workspace @forestadmin/ai-proxy test openai.integration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…adiness

- Add multi-turn conversation test with tool results
- Add AINotConfiguredError test for missing AI config
- Add MCP error handling tests (unreachable server, auth failure)
- Skip flaky tests due to Langchain retry behavior
- Ensure tests work on main branch without Zod validation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Set maxRetries: 0 by default when creating ChatOpenAI instance.
This makes our library a simple passthrough without automatic retries,
giving users full control over retry behavior.

Also enables previously skipped integration tests that were flaky
due to retry delays.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Addresses all issues identified in PR review:

- Add invoke-remote-tool success tests for MCP tools (add, multiply)
- Strengthen weak error assertions with proper regex patterns
- Fix 'select AI configuration by name' test to verify no fallback warning
- Add test for fallback behavior when config not found
- Add logger verification in MCP error handling tests

Tests now verify:
- Error messages match expected patterns (not just toThrow())
- Logger is called with correct level and message on errors
- Config selection works without silent fallback

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The test was incorrectly checking for finish_reason: 'tool_calls'.
When forcing a specific function via tool_choice, OpenAI returns
finish_reason: 'stop' but still includes the tool_calls array.

The correct assertion is to verify the tool_calls array contains
the expected function name, not the finish_reason.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move the agent-specific message "Please call addAi() on your agent"
from ai-proxy to the agent package where it belongs.

- ai-proxy: AINotConfiguredError now uses generic "AI is not configured"
- agent: Catches AINotConfiguredError and adds agent-specific guidance

This keeps ai-proxy decoupled from agent-specific terminology.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The test was not properly waiting for the HTTP server to close.
Changed afterAll to use a Promise wrapper around server.close() callback.

This removes the need for forceExit: true in Jest config.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests typically complete in 200-1600ms. 30 second timeouts were excessive.
- Single API calls: 10s timeout (was 30s)
- Multi-turn conversation: 15s timeout (was 60s)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add support for Anthropic's Claude models in the ai-proxy package using
@langchain/anthropic. This allows users to configure Claude as their AI
provider alongside OpenAI.

Changes:
- Add @langchain/anthropic dependency
- Add ANTHROPIC_MODELS constant with supported Claude models
- Add AnthropicConfiguration type and AnthropicModel type
- Add AnthropicUnprocessableError for Anthropic-specific errors
- Implement message conversion from OpenAI format to LangChain format
- Implement response conversion from LangChain format back to OpenAI format
- Add tool binding support for Anthropic with tool_choice conversion
- Add comprehensive tests for Anthropic provider

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… dispatcher

- Move convertMessagesToLangChain inside try-catch to properly handle JSON.parse errors
- Update OpenAIMessage interface to allow null content (per OpenAI API spec)
- Add null content handling for all message types with fallback to empty string

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Mirror OpenAI integration tests for Anthropic provider.
Requires ANTHROPIC_API_KEY environment variable.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
alban bertolini and others added 3 commits February 13, 2026 16:52
Both root and ai-proxy resolve to the same hoisted version,
so the local override is unnecessary.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Only ai-proxy uses @anthropic-ai/sdk, no need to configure it globally.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use SDK-native model types (OpenAI.ChatModel, Anthropic.Messages.Model)
instead of LangChain re-exports that added a double (string & {}).
Also Omit model from BaseAiConfiguration in AnthropicConfiguration
to prevent string intersection from erasing literal autocomplete.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment thread packages/ai-proxy/src/errors.ts Outdated
export class AnthropicUnprocessableError extends AIUnprocessableError {
constructor(message: string) {
super(message);
this.name = 'AnthropicError';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this.name should be the same as the class name no ?

Suggested change
this.name = 'AnthropicError';
this.name = 'AnthropicUnprocessableError';

Comment on lines +309 to +310
ErrorClass: typeof OpenAIUnprocessableError | typeof AnthropicUnprocessableError,
providerName: string,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should have only one of these params

and the other one is based on it

if you keep providerName: 'Anthropic' | 'OpenAi', you can get the error class from it

const errorClass = providerName === 'Anthropic' ? AnthropicUnprocessableError : OpenAIUnprocessableError

// disable_parallel_tool_use, but the Anthropic API supports it and LangChain passes it through
const model = enhancedTools?.length
? this.anthropicModel.bindTools(enhancedTools, {
tool_choice: this.convertToolChoiceForAnthropic(toolChoice, parallelToolCalls) as any,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
tool_choice: this.convertToolChoiceForAnthropic(toolChoice, parallelToolCalls) as any,
tool_choice: this.convertToolChoiceForAnthropic(toolChoice, parallelToolCalls) as AnthropicToolChoice,

IMO we should cast it to AnthropicToolChoice or set the method response type to AnthropicToolChoice

Comment on lines +156 to +306
private convertMessagesToLangChain(messages: OpenAIMessage[]): BaseMessage[] {
return messages.map(msg => {
switch (msg.role) {
case 'system':
return new SystemMessage(msg.content || '');
case 'user':
return new HumanMessage(msg.content || '');
case 'assistant':
if (msg.tool_calls) {
return new AIMessage({
content: msg.content || '',
tool_calls: msg.tool_calls.map(tc => ({
id: tc.id,
name: tc.function.name,
args: ProviderDispatcher.parseToolArguments(
tc.function.name,
tc.function.arguments,
),
})),
});
}

return new AIMessage(msg.content || '');
case 'tool':
if (!msg.tool_call_id) {
throw new AIBadRequestError('Tool message is missing required "tool_call_id" field.');
}

if (err.status === 401) {
throw new OpenAIUnprocessableError(`Authentication failed: ${err.message}`);
return new ToolMessage({
content: msg.content || '',
tool_call_id: msg.tool_call_id,
});
default:
throw new AIBadRequestError(
`Unsupported message role '${msg.role}'. Expected: system, user, assistant, or tool.`,
);
}
});
}

throw new OpenAIUnprocessableError(`Error while calling OpenAI: ${err.message}`);
private static parseToolArguments(toolName: string, args: string): Record<string, unknown> {
try {
return JSON.parse(args);
} catch {
throw new AIBadRequestError(
`Invalid JSON in tool_calls arguments for tool '${toolName}': ${args}`,
);
}
}

private convertToolChoiceToLangChain(
toolChoice: ChatCompletionToolChoice | undefined,
): 'auto' | 'any' | 'none' | { type: 'tool'; name: string } | undefined {
if (!toolChoice) return undefined;
if (toolChoice === 'auto') return 'auto';
if (toolChoice === 'none') return 'none';
if (toolChoice === 'required') return 'any';

if (typeof toolChoice === 'object' && toolChoice.type === 'function') {
return { type: 'tool', name: toolChoice.function.name };
}

throw new AIBadRequestError(
`Unsupported tool_choice value. Expected: 'auto', 'none', 'required', or {type: 'function', function: {name: '...'}}.`,
);
}

/**
* Convert tool_choice to Anthropic format, supporting disable_parallel_tool_use.
*
* When parallel_tool_calls is false, Anthropic requires the tool_choice to be
* an object with `disable_parallel_tool_use: true`.
* LangChain passes objects through directly to the Anthropic API.
*/
private convertToolChoiceForAnthropic(
toolChoice: ChatCompletionToolChoice | undefined,
parallelToolCalls?: boolean,
) {
const base = this.convertToolChoiceToLangChain(toolChoice);

if (parallelToolCalls !== false) return base;

// Anthropic requires object form to set disable_parallel_tool_use
if (base === undefined || base === 'auto') {
return { type: 'auto', disable_parallel_tool_use: true };
}

if (base === 'any') {
return { type: 'any', disable_parallel_tool_use: true };
}

if (base === 'none') return 'none';

return { ...base, disable_parallel_tool_use: true };
}

private static extractTextContent(content: AIMessage['content']): string | null {
if (typeof content === 'string') return content || null;

if (Array.isArray(content)) {
const text = content
.filter(block => block.type === 'text')
.map(block => ('text' in block ? block.text : ''))
.join('');

return text || null;
}

return null;
}

private convertLangChainResponseToOpenAI(response: AIMessage): ChatCompletionResponse {
const toolCalls = response.tool_calls?.map(tc => ({
id: tc.id || `call_${crypto.randomUUID()}`,
type: 'function' as const,
function: {
name: tc.name,
arguments: JSON.stringify(tc.args),
},
}));

const usageMetadata = response.usage_metadata as
| { input_tokens?: number; output_tokens?: number; total_tokens?: number }
| undefined;

return {
id: response.id || `msg_${crypto.randomUUID()}`,
object: 'chat.completion',
created: Math.floor(Date.now() / 1000),
model: this.modelName,
choices: [
{
index: 0,
message: {
role: 'assistant',
content: ProviderDispatcher.extractTextContent(response.content),
refusal: null,
tool_calls: toolCalls?.length ? toolCalls : undefined,
},
finish_reason: toolCalls?.length ? 'tool_calls' : 'stop',
logprobs: null,
},
],
usage: {
prompt_tokens: usageMetadata?.input_tokens ?? 0,
completion_tokens: usageMetadata?.output_tokens ?? 0,
total_tokens: usageMetadata?.total_tokens ?? 0,
},
};
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO since this is some kind of serialization / deserialization it should be in another file or service

alban bertolini and others added 12 commits February 16, 2026 15:50
…lity

Anthropic only allows a single system message at position 0. Merge all
system messages into one before dispatching. Also unify provider error
classes into AIUnprocessableError, type tool_choice properly, bump
@langchain/anthropic to 1.3.17 and add integration test for the case.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract conversion logic from ProviderDispatcher into a dedicated
LangChainAdapter class with clear separation between generic
(convertMessages, convertResponse, convertToolChoice) and
Anthropic-specific methods (mergeSystemMessages,
withParallelToolCallsRestriction).

Also fixes: AIToolUnprocessableError.name, wrapProviderError cause
preservation, unknown provider guard in constructor.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move Anthropic-specific logic (system message merging,
disable_parallel_tool_use) into a dedicated AnthropicAdapter that
composes with the generic LangChainAdapter. This keeps LangChainAdapter
purely generic and reusable for any LangChain-based provider.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…Choice

Both parameters are optional, an object param is clearer than
positional undefined values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…pported list

Add claude-3-5-haiku (EOL 2026-02-19) and claude-opus-4/4-1
(require streaming) to the unsupported models list.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace deprecated claude-3-5-haiku-latest with claude-haiku-4-5-20251001.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
claude-3-5-haiku is deprecated but still functional. Only Opus models
that require streaming should be in the unsupported list.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Scra3 Scra3 force-pushed the feat/prd-87-anthropic-support branch from 55b40cc to 0d0574e Compare February 16, 2026 16:38
alban bertolini and others added 6 commits February 16, 2026 18:50
…r cause

Move tool binding logic into AnthropicAdapter.bindTools(), making
convertToolChoice private. Replace unsafe cast for error cause with
a proper constructor option on AIUnprocessableError.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… safety

- Narrow dispatchOpenAI try-catch to only wrap model.invoke(), preventing
  internal assertions from being routed through wrapProviderError
- Add provider name and cause chain to 429/401 error wrapping
- Make tool_call_id required on OpenAIToolMessage to match runtime contract
- Clarify convertToolChoice JSDoc about strict false check

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Keep only model.invoke() inside the try-catch, consistent with the
OpenAI dispatch path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…se JSDoc

- Simplify isOpenAIModelSupported to single boolean return
- Add explicit return type to enrichToolDefinitions
- Type response variable in dispatchOpenAI for consistency with Anthropic path
- Condense bindTools JSDoc to essential one-liner

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment on lines +151 to +161
if (status === 429) {
return new AIUnprocessableError(`${providerName} rate limit exceeded: ${error.message}`, {
cause: error,
});
}

if (status === 401) {
return new AIUnprocessableError(`${providerName} authentication failed: ${error.message}`, {
cause: error,
});
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use dedicated errors instead of generi ones

alban bertolini and others added 2 commits February 17, 2026 11:36
Currently all provider errors are wrapped as AIUnprocessableError,
losing the original HTTP semantics (429 rate limit, 401 auth failure).
Documents the plan to add proper error types in datasource-toolkit
and ai-proxy for correct HTTP status mapping.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This model only supports v1/completions, not v1/chat/completions,
causing integration test failures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Scra3 Scra3 merged commit f748b19 into main Feb 17, 2026
28 checks passed
@Scra3 Scra3 deleted the feat/prd-87-anthropic-support branch February 17, 2026 10:49
forest-bot added a commit that referenced this pull request Feb 17, 2026
# @forestadmin/ai-proxy [1.5.0](https://github.com/ForestAdmin/agent-nodejs/compare/@forestadmin/ai-proxy@1.4.3...@forestadmin/ai-proxy@1.5.0) (2026-02-17)

### Features

* **ai-proxy:** add Anthropic LLM provider support ([#1435](#1435)) ([f748b19](f748b19))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants