[ENHANCEMENT] Isolate provider behavior behind capability registries

[WebMad]

Problem (one or two sentences)

Provider integration behavior is spread across parallel switch/case hubs for runtime construction, model defaults and selection, settings validation, model discovery and caching, profile handling, and CLI configuration. Adding or changing a provider requires coordinated edits across disconnected layers, increasing maintenance cost and allowing incomplete integrations or behavior drift.

Context (who is affected and when)

This primarily affects maintainers and contributors whenever a provider is added, changed, renamed, or retired. Each central branch knows implementation details for every provider, so a provider update can work in one environment while remain incomplete in another.

The current production hotspots concentrate these responsibilities:

Area	Responsibility concentrated in the central branch	Provider-specific knowledge carried there
API runtime	Chooses the concrete API handler, rejects retired providers, and handles special model-dependent routing	Every handler constructor and runtime implementation
Shared provider metadata	Resolves the default model	Provider model constants and regional rules
Webview model state	Reads provider-specific model fields, combines static and dynamic catalogs, and applies capability or pricing overrides	Model catalogs and provider-specific capability rules
Webview settings validation	Validates credentials, endpoints, selectors, authentication modes, and account state	Provider setting fields and user-facing validation messages
Profile validation	Extracts a model ID from differently named provider fields	The complete provider settings shape
Dynamic model discovery	Selects a model-list fetcher and adapts generic input to provider-specific arguments	Fetchers, key conventions, base URLs, and cache isolation rules
CLI settings construction	Maps generic CLI key and model input into provider-specific settings	Provider setting fields
CLI context-window resolution	Extracts the configured model ID	A duplicate of the profile model-field mapping
Embedding defaults	Chooses an embedding model default	Embedding-specific profiles and defaults
Embedding UI validation	Builds provider-specific indexing validation	Embedding credentials, endpoints, model fields, and validation rules

The highest-value API-provider hotspots are runtime construction, webview model resolution, webview validation, and dynamic model discovery because each commonly changes when a provider is introduced or its configuration evolves.

The duplicated model-ID extraction in profile validation and CLI context-window handling is a smaller concrete example: both locations independently encode which model field belongs to each provider.

Current dependency shape:

flowchart TB
    Provider[Provider identifier]
    Provider --> Runtime[API handler selection]
    Provider --> Defaults[Default model selection]
    Provider --> ModelState[Selected model resolution]
    Provider --> Validation[Settings validation]
    Provider --> Profiles[Profile model resolution]
    Provider --> Discovery[Model discovery and cache policy]
    Provider --> CLI[CLI settings and context]
    Runtime --> Handlers[All runtime handlers]
    Defaults --> Models[All provider model defaults]
    ModelState --> Catalogs[All model catalogs and capability rules]
    Validation --> Fields[All credential and endpoint fields]
    Profiles --> Fields
    Discovery --> Fetchers[All provider model fetchers]
    CLI --> Fields

Loading

These branches act as parallel registries. A provider is threaded through several layers, and every layer must understand details belonging to that provider.

Related work demonstrates the need for a broader architectural discussion:

• [ENHANCEMENT] Centralize provider identifiers and replace hardcoded provider-name literals #944 centralizes provider identifiers. It provides the identity foundation but intentionally preserves provider-specific branches.
• [ENHANCEMENT] Extract shared OpenAI-compatible chunk processing from RouterProvider subclasses #338 proposes extracting duplicated OpenAI-compatible streaming and completion processing when existing inheritance hierarchies cannot combine provider responsibilities cleanly.
• [BUG] Reasoning Content Leaks into Text Blocks Across Providers #750 reports the same reasoning-stream defect across multiple provider handlers, demonstrating the cost of duplicated provider behavior.
• feat(api): introduce RequestConfigBuilder for SDK-agnostic abort signal support #1007 proposes shared request-configuration infrastructure because providers independently repeat request setup.
• Add Novita AI provider #798 demonstrates how adding one provider currently requires registration across shared types, runtime handling, profiles, CLI, webview settings, model discovery, and model selection.

Desired behavior (conceptual, not technical)

Provider-specific decisions should be owned by provider modules and exposed through small, focused capabilities. Central orchestration should look up those capabilities through type-checked registries rather than contain exhaustive provider branches.

Use separate registries for each dependency boundary instead of one large provider base class:

• A shared descriptor registry owns provider identity, configured model resolution, and default model resolution.
• An extension runtime registry owns API handler construction, optional dynamic model discovery, and cache-isolation policy.
• A webview registry owns settings validation and selected-model resolution.
• A CLI registry owns mapping generic CLI input to provider-specific settings.
• A separate embedding-provider registry owns embedding defaults and indexing UI behavior.

Proposed dependency shape:

flowchart LR
    Settings[Provider settings] --> Id[Provider identifier]
    Id --> Shared[Shared descriptor registry]
    Id --> Runtime[Extension runtime registry]
    Id --> Webview[Webview registry]
    Id --> CLI[CLI registry]
    Shared --> SharedCapabilities[Model ID and defaults]
    Runtime --> RuntimeCapabilities[Handler, model discovery, cache policy]
    Webview --> WebviewCapabilities[Validation and selected-model resolution]
    CLI --> CLICapabilities[CLI input mapping]
    SharedCapabilities --> Modules[Provider-owned behavior]
    RuntimeCapabilities --> Modules
    WebviewCapabilities --> Modules
    CLICapabilities --> Modules

Loading

There may still be one lookup per environment, but it should be data-driven, exhaustive for the providers supported by that environment, and dependent on narrow capabilities. Provider modules should receive only the settings valid for that provider rather than the complete provider-settings union.

The current API-handler abstraction should remain responsible for inference operations such as creating messages, reporting model information, and counting tokens. The proposed runtime capability complements it by owning handler creation and surrounding provider integration policies. The existing public handler factory can remain as a compatibility facade while its dispatch moves behind registrations.

Validation capabilities should return stable domain outcomes rather than translated text. The webview can translate those outcomes, keeping shared provider behavior independent of UI and localization frameworks.

Chat/API providers and embedding providers should remain separate domains. Although some provider names overlap, their settings, model metadata, and validation behavior differ; combining them would create a broad interface dominated by optional capabilities.

Constraints / preferences (optional)

• Treat this as an architectural discussion proposal before beginning a repository-wide migration.
• Prefer composition and focused capabilities over one large inheritance hierarchy.
• Keep shared descriptors independent of extension-runtime, React, localization, and validation-framework dependencies.
• Preserve the canonical identifier registry introduced by [ENHANCEMENT] Centralize provider identifiers and replace hardcoded provider-name literals #944 as the source of provider identities; behavior registries complement it rather than replace it.
• Preserve serialized provider values and compatibility with existing settings, profiles, imports, and extension state.
• Preserve current behavior throughout incremental migration.
• Only providers supporting dynamic model discovery should expose that capability; avoid required no-op implementations.
• Keep model-fetch behavior and its cache discriminator together so key and base-URL isolation rules cannot drift.
• Migrate selected-model behavior provider by provider because it contains the largest amount of policy and has the highest regression risk.
• Keep API/chat and embedding provider registries separate.
• Add regression coverage at the narrowest test layer appropriate for each migrated responsibility.

Request checklist

• I have searched existing Issues and Discussions for duplicates
• This describes a specific problem with clear context and impact

Acceptance criteria (optional)

• A documented provider capability model and environment boundaries are agreed upon before broad implementation.
• Shared provider descriptors resolve configured and default model IDs without requiring consumers to maintain provider switches.
• Runtime handler construction is registered per provider while the existing public factory remains compatible.
• Dynamic model discovery and cache-isolation policy can be registered together for providers that support discovery.
• Webview settings validation and selected-model resolution can be moved behind provider-owned UI capabilities without introducing UI dependencies into shared packages.
• CLI provider input mapping reuses provider-owned metadata or capabilities instead of maintaining a parallel field mapping.
• Chat/API provider behavior and embedding provider behavior remain separate.
• Adding a representative provider requires registering its capabilities rather than editing each migrated central switch.
• Existing serialized settings and provider behavior remain compatible.
• Focused regression tests cover each responsibility as it is migrated.

Proposed approach (optional)

Use an incremental migration rather than replacing all branches at once:

1. Introduce the shared provider descriptor contract.
2. Consolidate duplicated configured-model resolution from profile and CLI logic.
3. Register runtime handler factories while retaining the existing public factory facade.
4. Attach dynamic model discovery and cache-isolation policy to runtime registrations.
5. Extract webview validation capabilities while keeping translation in the UI layer.
6. Migrate selected-model resolution provider by provider.
7. Move CLI settings mapping behind provider-owned capabilities or declarative metadata.
8. Introduce a separate embedding-provider registry for embedding defaults and indexing validation.

Trade-offs / risks (optional)

• A broad provider interface could become optional-heavy and reproduce the same coupling in another form; capabilities should remain small and environment-specific.
• Registries introduce indirection and require clear ownership and exhaustiveness tests.
• Moving high-policy model-selection behavior has significant regression risk and should happen incrementally.
• Shared packages cannot import runtime or webview implementations without creating dependency-direction or circular-dependency problems.
• Some provider-specific behavior may be better represented as shared protocol utilities rather than registry methods; work such as [ENHANCEMENT] Extract shared OpenAI-compatible chunk processing from RouterProvider subclasses #338 should remain complementary.
• During migration, registry-based and switch-based paths may temporarily coexist.

[WebMad]

Proposed implementation order

To keep this refactor reviewable and behavior-preserving, I suggest treating it as a sequence of small, independently testable changes rather than one repository-wide PR.

Phase 0 — Agree on boundaries

1. Confirm that the existing canonical provider identifier registry remains the source of provider identity.
2. Agree that the existing ApiHandler contract remains the runtime inference interface.
3. Agree on separate shared, extension-runtime, webview, and CLI registries instead of one cross-environment provider class.
4. Keep chat/API providers separate from embedding providers.
5. Decide how registry exhaustiveness will be checked at compile time and in tests.

Checkpoint: document the accepted contracts and dependency boundaries before migrating behavior.

Phase 1 — Introduce the shared descriptor

1. Define a small provider descriptor containing only framework-independent behavior:
   • provider identifier;
   • configured model-ID resolution;
   • default model-ID resolution.
2. Type each descriptor against the discriminated settings subtype for its provider, rather than the full provider-settings union.
3. Add exhaustiveness and representative behavior tests.
4. Do not move runtime, webview, localization, or validation-framework behavior into the shared package.

First migration target: consolidate the duplicated configured-model lookup used by profile validation and CLI context-window resolution. This is the smallest duplication hotspot and provides a low-risk proof of the registry design.

Phase 2 — Migrate shared default-model behavior

1. Move provider default-model decisions behind shared descriptors.
2. Preserve regional and provider-specific default rules exactly.
3. Keep the existing public default-model helper as a compatibility facade while callers migrate.

Checkpoint: adding a test provider descriptor should demonstrate model-ID and default-model behavior without editing the migrated central switches.

Phase 3 — Register runtime handler factories

1. Introduce an extension-runtime registry that maps provider identifiers to handler factories.
2. Preserve the existing public API-handler factory as a compatibility facade.
3. Move special routing rules, such as model-dependent handler selection, into the owning runtime adapter.
4. Keep ApiHandler focused on inference operations; the registry owns construction and integration policy.

Checkpoint: existing handler-selection and retired-provider tests must remain unchanged in observable behavior.

Phase 4 — Co-locate dynamic model discovery and cache policy

1. Add optional model-discovery capability only for providers that support it.
2. Move each provider's fetcher argument adaptation into its runtime registration.
3. Keep cache discrimination next to the corresponding fetch capability so API-key and base-URL isolation cannot drift.
4. Preserve cache keys, refresh behavior, request deduplication, and authentication boundaries.

Checkpoint: model discovery for a representative URL-scoped provider, key-scoped provider, and unscoped provider should work without central provider dispatch.

Phase 5 — Extract webview validation capabilities

1. Add a webview-only provider registry for settings validation.
2. Return stable validation outcomes or error codes from provider capabilities.
3. Keep translation and presentation in the webview layer.
4. Migrate providers incrementally, retaining the existing validation facade until coverage is complete.

Checkpoint: credentials, endpoints, selectors, authentication modes, and existing validation messages remain behaviorally compatible.

Phase 6 — Extract selected-model resolution

1. Move selected-model behavior one provider at a time because this is the highest-policy and highest-risk hotspot.
2. Keep static and dynamic model catalog merging, capability overrides, pricing information, and fallback behavior together for each provider.
3. Start with a simple static provider, then a dynamic provider, and only then migrate providers with complex Bedrock, Vertex, or Anthropic-specific rules.
4. Remove the central switch only after exhaustive provider coverage exists.

Checkpoint: representative static, dynamic, local, and model-dependent providers preserve model IDs and capability metadata.

Phase 7 — Migrate CLI mapping

1. Add CLI-specific provider capabilities or declarative field metadata.
2. Move generic key/model input mapping out of the central CLI switch.
3. Reuse the shared configured-model resolver for context-window lookup.
4. Preserve environment-variable names, serialized settings, and CLI compatibility.

Phase 8 — Handle embedding providers separately

1. Define a separate embedding-provider descriptor and webview adapter.
2. Migrate embedding defaults and indexing validation independently of chat providers.
3. Do not infer shared behavior solely from overlapping provider names.

Phase 9 — Cleanup and enforcement

1. Remove compatibility switches only after all consumers are migrated.
2. Audit non-switch provider conditionals and provider arrays discovered during migration.
3. Add registry exhaustiveness tests and contributor guidance for adding a provider.
4. Verify that a representative provider addition requires registering capabilities rather than editing every migrated orchestration layer.

Suggested PR slicing

Each phase should normally be one PR, with the selected-model phase split into several provider-group PRs. Every PR should:

• preserve serialized values and public behavior;
• include regression coverage at the narrowest appropriate layer;
• avoid unrelated provider cleanup;
• state which compatibility facade remains and when it can be removed;
• update the architecture notes if the agreed capability boundaries change.

This ordering deliberately starts with small duplicated shared behavior, proves the registry shape, then moves toward the highest-risk webview policy. It also allows the discussion to stop or adjust after any checkpoint without leaving a partially rewritten provider architecture.

[edelauna]

This proposed implementation order looks right to me. I would keep #1018 as the umbrella architecture/tracking issue and use it to coordinate smaller PRs that are reviewable on their own. The main adjustment I would suggest is to explicitly layer the already-open #944 work underneath this plan, so identifier cleanup and capability extraction do not get mixed together.

A practical PR sequence could look like this:

1. Finish the [ENHANCEMENT] Centralize provider identifiers and replace hardcoded provider-name literals #944 identifier-centralization track first or in parallel where it is purely mechanical: [ENHANCEMENT] Migrate provider default-model selection to canonical identifiers #954, [ENHANCEMENT] Migrate API handler selection to canonical provider identifiers #955, [ENHANCEMENT] Migrate profile model resolution to canonical provider identifiers #956, [ENHANCEMENT] Migrate model fetching and cache logic to canonical provider identifiers #957, [ENHANCEMENT] Migrate extension-side provider comparisons to canonical identifiers #958, [ENHANCEMENT] Migrate webview provider comparisons to canonical identifiers #959, [ENHANCEMENT] Replace remaining production provider literals and verify canonical identifier coverage #960, and [BUG] OpenAI Native fallback ignores the canonical default model #992. These should stay focused on canonical identifiers and behavior preservation, not introduce new capability registries. Tests should remain in the narrow package touched: packages/types provider/default-model tests, src/api/__tests__/index.spec.ts, src/shared/__tests__/ProfileValidator.spec.ts, webview utility tests, and model-cache/webview-message tests for [ENHANCEMENT] Migrate model fetching and cache logic to canonical provider identifiers #957.

2. Phase 1 proof point: introduce the shared provider descriptor with only configured model-id resolution first. Use it to remove duplicated model-id extraction from profile validation and CLI context-window handling, and optionally the webview validation helper if the dependency direction stays clean. This is the smallest useful proof of the descriptor shape. Tests: shared provider-descriptor tests for representative providers (openrouter, openai, vscode-lm, zoo-gateway, anthropic), plus existing ProfileValidator and CLI context-window tests.

3. Phase 2: move shared default-model resolution behind the same descriptor shape, unless [ENHANCEMENT] Migrate provider default-model selection to canonical identifiers #954/[BUG] OpenAI Native fallback ignores the canonical default model #992 already fully cover the behavior. Preserve all regional/provider-specific default rules, especially Z.ai and OpenAI Native fallback behavior. Tests should live in packages/types and include fallback/regression coverage.

4. Phase 3: introduce an extension-runtime handler factory registry behind the existing buildApiHandler(configuration) facade. Keep retired-provider handling and the current Anthropic fallback unchanged. Provider-specific routing such as Vertex Claude-vs-non-Claude selection should live in that provider factory. Tests: src/api/__tests__/index.spec.ts for representative normal handlers, special routing, retired providers, and fallback behavior.

5. Phase 4: co-locate dynamic model discovery and cache discrimination. This should build on [ENHANCEMENT] Migrate model fetching and cache logic to canonical provider identifiers #957 rather than duplicate it. Move provider-specific GetModelsOptions construction and flush/cache isolation rules out of the webview message handler into runtime discovery capabilities. Tests: model cache tests and webviewMessageHandler.routerModels.spec.ts, covering at least one key-scoped provider, URL-scoped provider, public endpoint provider, and auth-required provider.

6. Keep [ENHANCEMENT] Extract shared OpenAI-compatible chunk processing from RouterProvider subclasses #338/[BUG] Reasoning Content Leaks into Text Blocks Across Providers #750 as a complementary protocol-behavior track, not a registry track. Shared OpenAI-compatible streaming/reasoning extraction should reduce duplicated provider bugs, but it is better represented as shared protocol utility behavior than as a provider capability registry method. Tests should cover mixed reasoning/content deltas and the existing provider unit suites for LiteLLM, OpenCode Go, Vercel AI Gateway, DeepSeek, Requesty, and Unbound.

7. Keep feat(api): introduce RequestConfigBuilder for SDK-agnostic abort signal support #1007 as a separate request-configuration track. Add the builder/utilities and migrate one or two providers as proof before applying it broadly. Tests should focus on immutability, empty/undefined handling, abort signal merging, and provider-specific behavior for migrated providers.

8. Phase 5 and Phase 6 should stay later and be split more aggressively. Webview validation should return stable validation outcomes/codes while translation remains in the webview. Selected-model resolution should be split by risk: simple static providers first, then dynamic providers, then complex providers like Bedrock, Vertex, Anthropic-style paths, and OpenAI Native. Tests should primarily be webview-ui utility/component tests, with e2e only when the real extension/webview boundary is essential.

9. Phase 7 CLI mapping should reuse the shared configured-model resolver early, but full CLI provider input mapping can wait until the shared/runtime pieces are stable. Tests should be CLI utility tests and CLI command parsing/config tests.

10. Phase 8 embedding providers should remain explicitly out of scope until chat/API provider boundaries are proven. When it starts, it should be a separate descriptor/registry, not optional methods on the chat provider descriptor.

11. Phase 9 cleanup should include contributor guidance and an enforcement test showing that a representative provider addition requires registering capabilities rather than editing every migrated orchestration layer. Add Novita AI provider #798 is a good future pressure test for this, but I would not block the early architecture PRs on a full provider addition.

The main design constraint I would keep front and center: avoid one mega ProviderDescriptor with optional methods for every environment. The useful unit here is provider-owned behavior at explicit boundaries: shared descriptor, runtime factory/discovery, webview validation/model selection, CLI mapping, and embedding descriptors. That lines up with the proposed phases while keeping each PR small enough to review and test independently.