feat(js/plugins/compat-oai): add OpenAI Responses transport and non-streaming runner

[cabljac]

Part of #6002, closes #6003. Design rationale: design notes on the tracking issue.

Adds a second OpenAI transport to the existing openAI() plugin: the eight models OpenAI serves only over /v1/responses (gpt-5-pro, gpt-5-codex, gpt-5.1-codex, gpt-5.1-codex-mini, gpt-5.1-codex-max, codex-mini-latest, o1-pro, o3-pro) now register and run against that endpoint, in the same namespace with the same action-name scheme. They were unreachable via this plugin before.

• One RESPONSES_ONLY_MODELS const drives the runtime routing predicate and the openAI.model() type union; suffixed names (o3-pro-2025-06-10, gpt-5-pro-preview) route like their base, deliberately biased toward the transport that works.
• Non-streaming runner over client.responses.create. store is pinned to false unless set, keeping retention behavior identical to Chat Completions.
• supports.constrained: 'all', so output schemas go natively through text.format instead of genkit's simulated-constrained prompt injection. Prior model turns replay part by part, so structured-output history survives multi-turn.
• Fails loudly instead of degrading: genkit tools and function_call output items are rejected (INVALID_ARGUMENT/UNIMPLEMENTED) until the tool-calling slice; a failed response throws its error instead of returning an empty completion. Deliberate exception: a streaming caller gets the completed response as a single terminal chunk (the Dev UI always streams), with the event protocol in a follow-up.
• Model listing fix: the 'codex' substring in UNSUPPORTED_MODEL_MATCHERS was hiding the gpt-5-codex family; matchers are now anchored. Side effect: retired code-cushman-001 is now filtered.

Usage

import { genkit, z } from 'genkit';
import { openAI } from '@genkit-ai/compat-oai/openai';

const ai = genkit({ plugins: [openAI()] });

// Previously unreachable models now work by name:
const { text } = await ai.generate({
  model: openAI.model('gpt-5-pro'),
  prompt: 'Prove there are infinitely many primes.',
});

// Output schemas go natively through text.format:
const { output } = await ai.generate({
  model: openAI.model('gpt-5-pro'),
  prompt: 'Extract the invoice fields.',
  output: { schema: z.object({ total: z.number(), currency: z.string() }) },
});

Shared-layer note (affects deepSeek()/xai()/openAICompatible()): transport becomes a reserved config key in the Chat Completions body builder - stripped from the wire (Chat Completions 400s on unknown args), with transport: 'responses' rejected via a provider-neutral INVALID_ARGUMENT rather than silently ignored.

No behavior change for existing models: nothing in SUPPORTED_GPT_MODELS changes transport, raw shape, or config; gpt.ts is untouched. Fake-server tests assert endpoints on the wire for both transports (gpt-4o → /v1/chat/completions, gpt-5-pro → /v1/responses), and an end-to-end test through a real genkit() instance asserts the output schema reaches the wire as text.format.

Tests: 146 passing in js/plugins/compat-oai (+49 vs base); live smoke tests run when OPENAI_API_KEY is set.

Not in this slice (see #6002): streaming events, tool calling, encrypted reasoning round-trip, annotations, responsesModel() opt-in for dual-transport models, background models.

[gemini-code-assist]

Code Review

This pull request introduces support for the OpenAI Responses API (/v1/responses) in the compat-oai plugin, enabling integration with models served exclusively over this transport (such as gpt-5-pro and o3-pro). It refactors common utilities into utils.ts and adds comprehensive unit and live integration tests. The reviewer feedback suggests replacing generic Error instances with framework-specific GenkitError classes across several validation checks in responses.ts to ensure consistent error handling and telemetry integration.

[gemini-code-assist]

To ensure consistent error handling and better integration with Genkit's error reporting and telemetry, please throw a GenkitError with an appropriate status code (such as INVALID_ARGUMENT) instead of a generic Error.

        throw new GenkitError({
          status: 'INVALID_ARGUMENT',
          message: 'Invalid data URL format for media: ' + part.media.url.substring(0, 50) + '...'
        });

[cabljac]

Done in b468a84 - these throw GenkitError now (INVALID_ARGUMENT for the media/part cases, UNIMPLEMENTED for the unsupported-role branch, which only fires for the tool role deferred to the next slice).

[gemini-code-assist]

To ensure consistent error handling and better integration with Genkit's error reporting and telemetry, please throw a GenkitError with an appropriate status code (such as INVALID_ARGUMENT) instead of a generic Error.

    throw new GenkitError({
      status: 'INVALID_ARGUMENT',
      message: 'File URLs are not supported. Only base64-encoded files and image URLs are supported. Content type: ' + contentType
    });

[cabljac]

Done in b468a84 - these throw GenkitError now (INVALID_ARGUMENT for the media/part cases, UNIMPLEMENTED for the unsupported-role branch, which only fires for the tool role deferred to the next slice).

[gemini-code-assist]

To ensure consistent error handling and better integration with Genkit's error reporting and telemetry, please throw a GenkitError with an appropriate status code (such as INVALID_ARGUMENT) instead of a generic Error.

  throw new GenkitError({
    status: 'INVALID_ARGUMENT',
    message: 'Unsupported genkit part fields encountered for current message role: ' + JSON.stringify(part) + '.'
  });

[cabljac]

Done in b468a84 - these throw GenkitError now (INVALID_ARGUMENT for the media/part cases, UNIMPLEMENTED for the unsupported-role branch, which only fires for the tool role deferred to the next slice).

[gemini-code-assist]

To ensure consistent error handling and better integration with Genkit's error reporting and telemetry, please throw a GenkitError with an appropriate status code (such as INVALID_ARGUMENT) instead of a generic Error.

      throw new GenkitError({
        status: 'INVALID_ARGUMENT',
        message: 'Unsupported genkit part fields encountered for current message role: ' + JSON.stringify(part) + '.'
      });

[cabljac]

Done in b468a84 - these throw GenkitError now (INVALID_ARGUMENT for the media/part cases, UNIMPLEMENTED for the unsupported-role branch, which only fires for the tool role deferred to the next slice).

[gemini-code-assist]

To ensure consistent error handling and better integration with Genkit's error reporting and telemetry, please throw a GenkitError with an appropriate status code (such as UNIMPLEMENTED) instead of a generic Error.

        throw new GenkitError({
          status: 'UNIMPLEMENTED',
          message: 'role ' + message.role + ' is not supported by the OpenAI Responses API transport.'
        });

[cabljac]

Done in b468a84 - these throw GenkitError now (INVALID_ARGUMENT for the media/part cases, UNIMPLEMENTED for the unsupported-role branch, which only fires for the tool role deferred to the next slice).

[cabljac]

This fires for deepSeek/xai too - none of the providers on this builder can speak the responses transport, and stripping the key silently seemed worse than rejecting it. Hence the provider-neutral message.

[cabljac]

Weakest part of the slice: streaming callers get the final response as one chunk. Rejecting streaming outright would have made all eight models unusable in the Dev UI, which always streams. Event mapping is #6004.

[cabljac]

Note: unit and fake-server coverage is in (146 tests, including wire assertions for both transports and an end-to-end generate() with an output schema), but the live smoke tests against the real API have not been run yet. Keeping this as draft until that's done.