[API Proposal]: IOcrClient — an OCR / document-extraction capability for Microsoft.Extensions.AI

[luisquintanilla]

[API Proposal]: IOcrClient — an OCR / document-extraction capability for Microsoft.Extensions.AI

Updates

• 2026-07-23 — Pre-review spikes: per-page coordinate model, selective promotes, geometry primitives and GetService kept. SPIKE-07/08 moved OcrCoordinateUnit / OcrCoordinateOrigin back onto OcrPage (reported per page, not per document — engines emit different units for different pages in mixed image/PDF batches, per Google Page.Dimension and Azure DI DocumentPage.Unit), and grouped OcrPage.Width / Height into a new OcrPageDimensions value type. SPIKE-06's 14-engine raw-output inventory kept RawRepresentation / AdditionalProperties and promoted per-cell BoundingRegion / Confidence / RawRepresentation / AdditionalProperties onto OcrTableCell plus RawRepresentation onto OcrPage. SPIKE-09 kept the three OCR-owned geometry primitives (no cross-platform BCL type carries the page-scoped, rotation-capable OcrBoundingRegion polygon). SPIKE-04/05 kept IOcrClient.GetService (one optional seam for provider metadata, provider-SDK escape, and adapter unwrap). Surface is now 32 public types (adds OcrPageDimensions).
• 2026-07-22 — API-review reshape: reading-order element model + document-level coordinate model, grounded in a 12-engine provider survey. Collapsed the parallel OcrPage.Blocks/Tables/Images into one reading-order OcrPage.Elements over a new polymorphic OcrElement base (OcrBlock/OcrTable/OcrImage derive; project with OfType<T>()), and added optional nested OcrTableCell.Elements. Moved coordinates to the document level: OcrCoordinateUnit is now a closed enum (added Point) plus a new OcrCoordinateOrigin enum, both on OcrResult/OcrPageResult (removed from OcrPage). Renamed ExtractStreamingAsync → ExtractPagesAsync and OcrResponseUpdate → OcrPageResult (non-null Page, dropped Status); Markdown → Text; added OcrUsage token counts; added OcrTableCellKind.RowHeader/RowSection; dropped response ModelId, OcrPage.Confidence, and OcrOptions.IncludeImages; OcrBoundingRegion.FromRectangle now takes float. Surface is now 31 public types.
• 2026-07-15 — API-review alignment (family symmetry). Replaced the unary-only shape with the family's unary + streaming pair: added IAsyncEnumerable<OcrResponseUpdate> ExtractStreamingAsync(...) (the IChatClient.GetStreamingResponseAsync twin), OcrResponseUpdate, and an OcrResponseUpdateExtensions.ToOcrResult/ToOcrResultAsync reducer, and removed IProgress<OcrProgress> and OcrProgress (progress now rides on the streamed update). OcrBlock.Kind / OcrTableCell.Kind are now ChatRole-style open structs (OcrBlockKind / OcrTableCellKind), not raw strings. Added OcrPage.Width / Height + OcrCoordinateUnit so bounding-box coordinates are interpretable across engines. Removed the leaky OcrResult.OcrSource (ModelId + OcrClientMetadata.ProviderName carry provenance). Unsealed the result / data types to match ChatResponse / ChatOptions. Surface is now 29 public types.
• 2026-07-14 — Reformatted to the API-proposal template and refreshed the API Proposal speclet to match the surface implemented in PR Add IOcrClient OCR/document-extraction capability to Microsoft.Extensions.AI #7588: GetTextAsync → ExtractAsync; IOcrClient : IDisposable; typed geometry OcrPoint / OcrBoundingBox with OcrBoundingRegion.Polygon as IReadOnlyList<OcrPoint>; 1-based OcrPage.PageNumber; [Experimental("MEAI001")]; added OcrImage, OcrPage.Images, OcrOptions.Clone(), OcrClientMetadata, the OcrClientExtensions surface (incl. opt-in ExtractFromUriAsync), and the full builder / middleware / DI types. The sibling IDocumentAnalysisClient is scoped out to its own future proposal.

Background and motivation

Document parsing is a core RAG and ingestion building block, but Microsoft.Extensions.AI.Abstractions
does not yet have an OCR / document-extraction capability. Today, you either wire provider SDKs directly
or route OCR through IChatClient, which loses native document structure such as tables, bounding boxes,
confidence, polygons, and reading order.

This proposal adds IOcrClient as a provider-neutral capability interface in
Microsoft.Extensions.AI.Abstractions, with the same builder, middleware, and DI shape developers
already use across the Microsoft.Extensions.AI capability family.

What is OCR / document AI?

OCR is the process of extracting text from documents and images. Document AI goes further: it keeps
structure around that text, including pages, tables, blocks, regions, confidence scores, and reading
order.

For RAG and ingestion pipelines, that structure matters. A document reader should not only produce
markdown. It should also preserve enough page, region, table, confidence, and source metadata for
downstream chunking, retrieval, grounding, and evaluation.

Why an abstraction?

Microsoft.Extensions.AI.Abstractions ships a family of capability interfaces: IChatClient,
IEmbeddingGenerator, ISpeechToTextClient, ITextToSpeechClient, IImageGenerator,
IRealtimeClient, and IHostedFileClient. There is no OCR / document-extraction capability, even
though Microsoft.Extensions.DataIngestion (MEDI) already depends on document parsing. Its
IngestionDocumentReader roadmap, per MS Learn, includes LlamaParse and Azure Document Intelligence,
both hosted document-AI services that need a provider-agnostic seam.

Today, if you want to use document-AI models, you must:

1. Couple ingestion code directly to a provider SDK.
2. Model OCR as a chat prompt against IChatClient.
3. Add reader-mode flags for specific engines or hosts.
4. Rebuild retry, logging, DI, middleware, and test seams per provider.
5. Give up native structure when the abstraction cannot represent it.

CommunityToolkit/AI #3 is a representative example.
It introduced a PdfReadingMode.VisionOnly flag that routes whole-document transcription through a
vision LLM (IChatClient). That is a layer leak: a model choice hardened into a reader-mode flag,
with temporal coupling because the reader emits placeholders that are useless unless a specific enricher
runs. The cleaner shape is a capability client the reader composes, exactly how MEDI's enrichers
already compose an injected IChatClient.

OCR is not chat. Most OCR / document-AI engines emit structured output: tables, bounding boxes,
confidence, polygons, reading order. That does not fit ChatResponse. A vision LLM can transcribe by
prompt, but it is the lowest-fidelity path and loses native structure. Purpose-built engines (Mistral
OCR, Azure Document Intelligence, Azure AI Content Understanding) and local document VLMs
(granite-docling, PaddleOCR) beat it. Modeling OCR as "call IChatClient with an image" makes those
engines unrepresentable without discarding their value. That points to a separate capability
interface, independent of IChatClient.

Prototype validation

This proposal is not a sketch. It describes a working prototype spiked across four real engine
providers (three using no IChatClient at all) plus one vision-LLM adapter, composed with an
IChatClient-style builder pipeline, and validated end-to-end through a MEDI ingestion pipeline:

• FoundryMistralOcrClient: Azure AI Foundry mistral-ocr-4-0, keyless Entra (verified HTTP 200).
• MistralOcrClient: Mistral-direct, API key.
• AzureDocumentIntelligenceClient: Azure.AI.DocumentIntelligence (AnalyzeResult, native polygons + table cells).
• ContentUnderstandingClient: Azure.AI.ContentUnderstanding 1.1.0, keyless Entra (markdown path).
• VisionLlmOcrClient: the one adapter over IChatClient (gpt-4o / Gemini / local Ollama GLM-OCR), the lowest-fidelity path.

Every claim below ("one pipeline wraps all engines", "the polygon flows losslessly from DI and Mistral",
"streaming yields pages as they finish") is backed by code that builds and runs, not by assertion. The demo
is a public, runnable proof: one IOcrClient in front of four OCR engines, bridged into a MEDI RAG
pipeline, with the identical consumer loop across both provider archetypes (document-native and
image-per-page).

The design goal is provider-neutrality: one small set of composable primitives that every provider
maps onto equally, judged on interoperability, reusability, composition, extensibility. No provider
is privileged. Providers form a coverage matrix, not a hierarchy.

The same precedent that justifies splitting OpenAIClient / AzureOpenAIClient behind IChatClient
applies here. The interface is the portability guarantee; concrete classes split by engine and by
host where credential, route, or provider behavior leaks. The model/deployment id is a parameter,
never a type or a boolean flag.

Building-block symmetry with Microsoft.Extensions.AI

The goal is not a one-off OCR helper. The goal is another building block in the same capability family.

If you know one Microsoft.Extensions.AI capability, you should know the others: abstraction,
options/result types, delegating base, builder/middleware, provider implementation, and DI registration.

Capability	Abstraction	Options / result types	Delegating base	Builder / middleware	Example implementation	DI registration shape
Chat	IChatClient	ChatOptions, ChatResponse, ChatResponseUpdate	DelegatingChatClient	ChatClientBuilder, .Use(...), logging, OpenTelemetry, caching, function invocation	OpenAIChatClient	AddChatClient, AddKeyedChatClient
Embeddings	IEmbeddingGenerator<TInput,TEmbedding>	EmbeddingGenerationOptions, GeneratedEmbeddings<TEmbedding>	DelegatingEmbeddingGenerator<TInput,TEmbedding>	EmbeddingGeneratorBuilder<TInput,TEmbedding>, .Use(...), logging, OpenTelemetry, caching	OpenAIEmbeddingGenerator	AddEmbeddingGenerator, AddKeyedEmbeddingGenerator
Speech-to-text	ISpeechToTextClient	SpeechToTextOptions, SpeechToTextResponse, response updates	DelegatingSpeechToTextClient	SpeechToTextClientBuilder, .Use(...), logging, OpenTelemetry, options	OpenAISpeechToTextClient	AddSpeechToTextClient, AddKeyedSpeechToTextClient
Text-to-speech	ITextToSpeechClient	TextToSpeechOptions, TextToSpeechResponse, response updates	DelegatingTextToSpeechClient	TextToSpeechClientBuilder, .Use(...), logging, OpenTelemetry, options	OpenAITextToSpeechClient	AddTextToSpeechClient, AddKeyedTextToSpeechClient
Images	IImageGenerator	ImageGenerationOptions, ImageGenerationRequest, ImageGenerationResponse	DelegatingImageGenerator	ImageGeneratorBuilder, .Use(...), logging, options	OpenAIImageGenerator	AddImageGenerator, AddKeyedImageGenerator
Realtime	IRealtimeClient	RealtimeSessionOptions, client/server messages, sessions	DelegatingRealtimeClient	RealtimeClientBuilder, .Use(...), logging, OpenTelemetry, function invocation	OpenAIRealtimeClient	Register IRealtimeClient / keyed clients through DI
Hosted files	IHostedFileClient	HostedFileClientOptions, HostedFileDownloadStream	DelegatingHostedFileClient	HostedFileClientBuilder, .Use(...), logging, OpenTelemetry	OpenAIHostedFileClient	Register IHostedFileClient / keyed clients through DI
OCR / document extraction	IOcrClient	OcrOptions, OcrResult, OcrResponseUpdate, OcrPage, OcrBlock, OcrTable, OcrImage, OcrUsage	DelegatingOcrClient	OcrClientBuilder, .Use(...), logging, OpenTelemetry, configure-options	FoundryMistralOcrClient, MistralOcrClient, AzureDocumentIntelligenceClient, ContentUnderstandingClient, VisionLlmOcrClient	AddOcrClient, AddKeyedOcrClient

That symmetry is the main API shape. IOcrClient should feel like a natural next capability, not a
separate pattern you have to relearn.

Provider coverage

Providers are peers behind a provider-neutral contract, not a tier or hierarchy.

Provider	IOcrClient (markdown/structure)	IDocumentAnalysisClient (typed fields + grounding) — future sibling proposal, not in this PR
Foundry Mistral OCR	yes	—
Azure Document Intelligence	yes	yes (Documents[].Fields)
Content Understanding	yes (markdown path)	yes (fields{} + grounding)
Vision-LLM adapter	yes (lowest fidelity)	—
Local ONNX / Ollama (roadmap)	yes	—

No row is privileged. Some providers implement more of the family than others; the family is the
design, and coverage is a matrix. Content Understanding is the widest-surface conformance test (one
service exercises both interfaces with the same primitives), not an apex; it validates
provider-neutrality because the same polygon / confidence / builder primitives serve its two shapes,
Mistral OCR, Azure DI, and a vision LLM. The second column previews a future sibling capability
(IDocumentAnalysisClient, see Related and future work) and is shown only to illustrate that the same
region / confidence / builder primitives generalize; it is not part of this PR.

API Proposal

The surface below is the settled public API on PR #7588 (32 public types across
Microsoft.Extensions.AI.Abstractions and Microsoft.Extensions.AI), reshaped per the 2026-07-22 API
review and a 12-engine provider survey (see Validated design decisions). Signatures only, no method
bodies. All new surface ships [Experimental("MEAI001")] from day one (the STT/TTS precedent).

Core abstraction, options, and result types (Microsoft.Extensions.AI.Abstractions)

namespace Microsoft.Extensions.AI;

/// <summary>
/// A capability for OCR / document-extraction engines. Independent of <see cref="IChatClient"/>:
/// engines emit structured output (tables, bounding boxes, confidence, reading order) that does not
/// fit a chat response. One contract, many engines (Mistral OCR, Azure Document Intelligence, Content
/// Understanding, a local ONNX model, or a vision LLM behind an adapter).
/// </summary>
[Experimental("MEAI001")]
public interface IOcrClient : IDisposable
{
    /// <summary>Runs OCR / document parsing over a document stream and returns structured text + pages.</summary>
    Task<OcrResult> ExtractAsync(
        Stream document, string mediaType, OcrOptions? options = null,
        CancellationToken cancellationToken = default);

    /// <summary>
    /// Streams OCR / document parsing as <see cref="OcrPageResult"/> values — one per page as it finishes
    /// (the <see cref="IChatClient.GetStreamingResponseAsync"/> twin). Reassemble into an
    /// <see cref="OcrResult"/> via <see cref="OcrPageResultExtensions.ToOcrResultAsync"/>. Lets
    /// large-document RAG chunk/embed early pages while later pages are still being parsed.
    /// </summary>
    IAsyncEnumerable<OcrPageResult> ExtractPagesAsync(
        Stream document, string mediaType, OcrOptions? options = null,
        CancellationToken cancellationToken = default);

    /// <summary>Provider escape hatch (the <see cref="IChatClient.GetService"/> pattern).</summary>
    object? GetService(Type serviceType, object? serviceKey = null);
}

/// <summary>Normalized OCR result — "normalize the common, preserve the raw" (the ChatResponse pattern).</summary>
[Experimental("MEAI001")]
public class OcrResult
{
    public OcrResult(IReadOnlyList<OcrPage> pages);

    public IReadOnlyList<OcrPage> Pages { get; }
    public string Text { get; }                              // derived: page Text joined with blank lines
    public OcrUsage? Usage { get; set; }
    public object? RawRepresentation { get; set; }           // provider-native object — nothing is lost
    public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
}

[Experimental("MEAI001")]
public class OcrPage
{
    public OcrPage(int pageNumber, string text);

    public int PageNumber { get; }                           // 1-based
    public string Text { get; }
    public IReadOnlyList<OcrElement> Elements { get; set; }  // READING ORDER; OfType<T>() to project. default: empty
    public OcrPageDimensions? Dimensions { get; set; }       // page extent (width+height), when the engine reports it
    public OcrCoordinateUnit? CoordinateUnit { get; set; }   // per page — engines emit different units per page (image vs PDF batches)
    public OcrCoordinateOrigin? CoordinateOrigin { get; set; }
    [JsonIgnore] public object? RawRepresentation { get; set; } // provider-native page object; survives ToOcrResult reduction (SPIKE-06)
    public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
}

/// <summary>
/// The reading-order element base — a polymorphic ($type) shape (the AIContent pattern), designed to be
/// promotable to a future shared document-element type. OcrBlock / OcrTable / OcrImage derive from it.
/// </summary>
[Experimental("MEAI001")]
[JsonPolymorphic(TypeDiscriminatorPropertyName = "$type")]
[JsonDerivedType(typeof(OcrBlock), "block")]
[JsonDerivedType(typeof(OcrTable), "table")]
[JsonDerivedType(typeof(OcrImage), "image")]
public abstract class OcrElement
{
    protected OcrElement();

    public OcrBoundingRegion? BoundingRegion { get; set; }
    public double? Confidence { get; set; }
    [JsonIgnore] public object? RawRepresentation { get; set; }
    public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
}

[Experimental("MEAI001")]
public class OcrBlock : OcrElement
{
    public OcrBlock(string text);

    public string Text { get; }
    public OcrBlockKind? Kind { get; set; }                  // open struct: Paragraph / Title / Figure / ...
    // BoundingRegion, Confidence inherited from OcrElement
}

/// <summary>An image or figure extracted from a page (emitted normally; no request flag).</summary>
[Experimental("MEAI001")]
public class OcrImage : OcrElement
{
    public DataContent? Content { get; set; }
    public string? Caption { get; set; }
    // BoundingRegion, Confidence inherited from OcrElement
}

/// <summary>A single 2-D point in page coordinates (a polygon vertex).</summary>
[Experimental("MEAI001")]
public readonly record struct OcrPoint(float X, float Y);

/// <summary>An axis-aligned bounding box, for coarse filters / hit-testing.</summary>
[Experimental("MEAI001")]
public readonly record struct OcrBoundingBox(float Left, float Top, float Right, float Bottom);

/// <summary>
/// The SHARED, provider-neutral geometry primitive — a polygon of OcrPoint vertices (populated natively by
/// Azure DI, via FromRectangle for Mistral's rect, and reused for field grounding). GetBounds() gives a coarse box.
/// </summary>
[Experimental("MEAI001")]
public class OcrBoundingRegion
{
    public OcrBoundingRegion(int pageNumber, IReadOnlyList<OcrPoint> polygon);

    public int PageNumber { get; }
    public IReadOnlyList<OcrPoint> Polygon { get; }
    public static OcrBoundingRegion FromRectangle(
        int pageNumber, float left, float top, float right, float bottom);   // float (was double)
    public OcrBoundingBox? GetBounds();
}

/// <summary>Cells are the primary structured representation; MarkdownRepresentation is the fallback (Mistral).</summary>
[Experimental("MEAI001")]
public class OcrTable : OcrElement
{
    public OcrTable(
        int rowCount, int columnCount,
        IReadOnlyList<OcrTableCell>? cells = null, string? markdownRepresentation = null);

    public int RowCount { get; }
    public int ColumnCount { get; }
    public IReadOnlyList<OcrTableCell>? Cells { get; }
    public string? MarkdownRepresentation { get; }
    // BoundingRegion, Confidence inherited from OcrElement
}

[Experimental("MEAI001")]
public class OcrTableCell
{
    public OcrTableCell(int rowIndex, int columnIndex, string content);

    public OcrTableCellKind? Kind { get; set; }              // open struct: ColumnHeader / Content / RowHeader / RowSection
    public int RowIndex { get; }
    public int ColumnIndex { get; }
    public int RowSpan { get; set; }                         // default 1
    public int ColumnSpan { get; set; }                      // default 1
    public string Content { get; }                           // flat-text convenience (kept)
    public IReadOnlyList<OcrElement>? Elements { get; set; } // optional NESTED content (structured cells)
    // Positioned-node facet mirrored from OcrElement (NOT inheritance; reversible way-station, SPIKE-06):
    public OcrBoundingRegion? BoundingRegion { get; set; }   // per-cell geometry (5 engines: Textract/Google/DI/Adobe/Docling)
    public double? Confidence { get; set; }
    [JsonIgnore] public object? RawRepresentation { get; set; }
    public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
}

/// <summary>
/// A streamed OCR update — one completed page (the ChatResponseUpdate pattern). Reduce a sequence of these
/// into an OcrResult with OcrPageResultExtensions. Page is non-null (no sentinel update).
/// </summary>
[Experimental("MEAI001")]
public class OcrPageResult
{
    [JsonConstructor] public OcrPageResult(OcrPage page);    // Page is non-null (no sentinel update)

    public OcrPage Page { get; }
    public int? PagesProcessed { get; set; }                 // progress (absorbs the retired OcrProgress)
    public int? TotalPages { get; set; }
    public OcrUsage? Usage { get; set; }
    [JsonIgnore] public object? RawRepresentation { get; set; }
    public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
}

/// <summary>Reducers that assemble streamed page results back into one OcrResult (the ToChatResponseAsync pattern).</summary>
[Experimental("MEAI001")]
public static class OcrPageResultExtensions
{
    public static OcrResult ToOcrResult(this IEnumerable<OcrPageResult> updates);
    public static Task<OcrResult> ToOcrResultAsync(
        this IAsyncEnumerable<OcrPageResult> updates, CancellationToken cancellationToken = default);
}

[Experimental("MEAI001")]
public class OcrUsage
{
    public int? PagesProcessed { get; set; }
    public int? InputTokenCount { get; set; }                // vision-LLM path; classic OCR leaves these null
    public int? OutputTokenCount { get; set; }
    public int? TotalTokenCount { get; set; }
    public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
}

/// <summary>The kind of a text block — a ChatRole-style OPEN set (well-knowns + provider-specific kinds).</summary>
[Experimental("MEAI001")]
public readonly struct OcrBlockKind : IEquatable<OcrBlockKind>
{
    public OcrBlockKind(string value);                       // throws on null/whitespace
    public static OcrBlockKind Paragraph { get; }            // "paragraph"
    public static OcrBlockKind Title { get; }                // "title"
    public static OcrBlockKind Figure { get; }               // "figure"
    public string Value { get; }
    // == / != / IEquatable / GetHashCode / ToString + a JsonConverter (the ChatRole shape)
}

/// <summary>The kind of a table cell — a ChatRole-style OPEN set.</summary>
[Experimental("MEAI001")]
public readonly struct OcrTableCellKind : IEquatable<OcrTableCellKind>
{
    public OcrTableCellKind(string value);
    public static OcrTableCellKind ColumnHeader { get; }     // "columnHeader"
    public static OcrTableCellKind Content { get; }          // "content"
    public static OcrTableCellKind RowHeader { get; }        // "rowHeader"    (added)
    public static OcrTableCellKind RowSection { get; }       // "rowSection"   (added)
    public string Value { get; }
}

/// <summary>The unit for page dimensions + region coordinates — a CLOSED enum (units are physically bounded).</summary>
[Experimental("MEAI001")]
public enum OcrCoordinateUnit { Pixel, Point, Inch, Normalized }

/// <summary>Origin corner + y-axis direction of the coordinate space — a CLOSED enum.</summary>
[Experimental("MEAI001")]
public enum OcrCoordinateOrigin { TopLeft, BottomLeft }

/// <summary>Page extent (width + height), expressed in the page's OcrCoordinateUnit — a readonly record struct (atomic pair).</summary>
[Experimental("MEAI001")]
public readonly record struct OcrPageDimensions(float Width, float Height);

/// <summary>Request knobs — the ChatOptions pattern.</summary>
[Experimental("MEAI001")]
public class OcrOptions
{
    public string? ModelId { get; set; }                     // "GetChatClient(model)" analog
    public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
    public OcrOptions Clone();                               // shallow clone (the ChatOptions.Clone pattern)
}

/// <summary>Metadata about an <see cref="IOcrClient"/> (the *ClientMetadata pattern).</summary>
[Experimental("MEAI001")]
public class OcrClientMetadata
{
    public OcrClientMetadata(string? providerName = null, Uri? providerUri = null, string? defaultModelId = null);
    public string? ProviderName { get; }
    public Uri? ProviderUri { get; }
    public string? DefaultModelId { get; }
}

Extension methods (OcrClientExtensions)

/// <summary>Convenience helpers over <see cref="IOcrClient"/>.</summary>
[Experimental("MEAI001")]
public static class OcrClientExtensions
{
    public static TService? GetService<TService>(this IOcrClient client, object? serviceKey = null);

    // Extract from an in-memory DataContent (unary + streaming twin).
    public static Task<OcrResult> ExtractAsync(
        this IOcrClient client, DataContent document, OcrOptions? options = null,
        CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<OcrPageResult> ExtractPagesAsync(
        this IOcrClient client, DataContent document, OcrOptions? options = null,
        CancellationToken cancellationToken = default);

    // Extract from a UriContent. Handles self-contained data: URIs; throws NotSupportedException for
    // file:/http(s) (whether to download vs. hand the URL to the engine is a deliberate non-decision).
    public static Task<OcrResult> ExtractAsync(
        this IOcrClient client, UriContent document, OcrOptions? options = null,
        CancellationToken cancellationToken = default);
    public static IAsyncEnumerable<OcrPageResult> ExtractPagesAsync(
        this IOcrClient client, UriContent document, OcrOptions? options = null,
        CancellationToken cancellationToken = default);

    // Explicit, opt-in remote downloader: fetches http(s) bytes with a caller-supplied HttpClient
    // (caller owns handlers/auth/timeouts/lifetime), inlines data: URIs, then extracts.
    public static Task<OcrResult> ExtractFromUriAsync(
        this IOcrClient client, UriContent document, HttpClient httpClient, OcrOptions? options = null,
        CancellationToken cancellationToken = default);
}

Delegating base, builder, middleware, and DI

All Microsoft.Extensions.AI capabilities ship the same five-layer shape:
interface → Delegating<Cap> → <Cap>Builder → Add<Cap> (returns the builder) → .Use*()
middleware, with one composition primitive, Builder Use(Func<T, IServiceProvider, T>). IOcrClient
mirrors it exactly.

// ---- Microsoft.Extensions.AI.Abstractions ----

/// <summary>Optional base for an <see cref="IOcrClient"/> that passes calls through to an inner instance.</summary>
[Experimental("MEAI001")]
public class DelegatingOcrClient : IOcrClient
{
    protected DelegatingOcrClient(IOcrClient innerClient);
    protected IOcrClient InnerClient { get; }

    public void Dispose();
    public virtual Task<OcrResult> ExtractAsync(
        Stream document, string mediaType, OcrOptions? options = null,
        CancellationToken cancellationToken = default);
    public virtual IAsyncEnumerable<OcrPageResult> ExtractPagesAsync(
        Stream document, string mediaType, OcrOptions? options = null,
        CancellationToken cancellationToken = default);
    public virtual object? GetService(Type serviceType, object? serviceKey = null);
    protected virtual void Dispose(bool disposing);
}

// ---- Microsoft.Extensions.AI ----

[Experimental("MEAI001")]
public sealed class OcrClientBuilder
{
    public OcrClientBuilder(IOcrClient innerClient);
    public OcrClientBuilder(Func<IServiceProvider, IOcrClient> innerClientFactory);
    public IOcrClient Build(IServiceProvider? services = null);                     // first .Use is outermost
    public OcrClientBuilder Use(Func<IOcrClient, IOcrClient> clientFactory);
    public OcrClientBuilder Use(Func<IOcrClient, IServiceProvider, IOcrClient> clientFactory);  // THE primitive
}

[Experimental("MEAI001")] public class LoggingOcrClient : DelegatingOcrClient { }          // logging middleware
[Experimental("MEAI001")] public sealed class OpenTelemetryOcrClient : DelegatingOcrClient { } // OTel middleware
[Experimental("MEAI001")] public sealed class ConfigureOptionsOcrClient : DelegatingOcrClient { } // options middleware

[Experimental("MEAI001")]
public static class OcrClientBuilderOcrClientExtensions
{
    public static OcrClientBuilder AsBuilder(this IOcrClient innerClient);
}

[Experimental("MEAI001")]
public static class LoggingOcrClientBuilderExtensions
{
    public static OcrClientBuilder UseLogging(
        this OcrClientBuilder builder, ILoggerFactory? loggerFactory = null, Action<LoggingOcrClient>? configure = null);
}

[Experimental("MEAI001")]
public static class OpenTelemetryOcrClientBuilderExtensions
{
    public static OcrClientBuilder UseOpenTelemetry(
        this OcrClientBuilder builder, ILoggerFactory? loggerFactory = null, string? sourceName = null,
        Action<OpenTelemetryOcrClient>? configure = null);
}

[Experimental("MEAI001")]
public static class ConfigureOptionsOcrClientBuilderExtensions
{
    public static OcrClientBuilder ConfigureOptions(this OcrClientBuilder builder, Action<OcrOptions> configure);
}

[Experimental("MEAI001")]
public static class OcrClientBuilderServiceCollectionExtensions
{
    public static OcrClientBuilder AddOcrClient(
        this IServiceCollection serviceCollection, IOcrClient innerClient,
        ServiceLifetime lifetime = ServiceLifetime.Singleton);
    public static OcrClientBuilder AddOcrClient(
        this IServiceCollection serviceCollection, Func<IServiceProvider, IOcrClient> innerClientFactory,
        ServiceLifetime lifetime = ServiceLifetime.Singleton);
    public static OcrClientBuilder AddKeyedOcrClient(
        this IServiceCollection serviceCollection, object? serviceKey, IOcrClient innerClient,
        ServiceLifetime lifetime = ServiceLifetime.Singleton);
    public static OcrClientBuilder AddKeyedOcrClient(
        this IServiceCollection serviceCollection, object? serviceKey, Func<IServiceProvider, IOcrClient> innerClientFactory,
        ServiceLifetime lifetime = ServiceLifetime.Singleton);
}

IOcrClient ships logging, OpenTelemetry, and configure-options middleware in v1, matching the
ISpeechToTextClient template. It does not ship a built-in retry client: no Microsoft.Extensions.AI
capability does, because resilience belongs in the HTTP pipeline
(Microsoft.Extensions.Http.Resilience). The .Use(...) primitive still lets a consumer wrap a custom
retry or cache decorator when they want one.

API Usage

Every snippet below is drawn from the runnable demo (iocrclient-demo), which exercises this exact
surface across four engines and a MEDI RAG pipeline.

One interface, four engines — the payoff. Vision LLM, Mistral OCR, Azure Document Intelligence, and
Azure Content Understanding each speak a completely different wire protocol. Behind IOcrClient they are
IOcrClient; the consumer loop never changes when you add or swap a provider.

var clients = new (string Name, IOcrClient Client)[]
{
    ("vision-llm",                  new VisionLlmOcrClient(chatClient)),
    ("mistral-ocr",                 new FoundryMistralOcrClient(foundryEndpoint, cred)),
    ("azure-document-intelligence", new AzureDocumentIntelligenceClient(diEndpoint, cred)),
    ("azure-content-understanding", new ContentUnderstandingClient(cuEndpoint, cred)),
};

byte[] bytes = await File.ReadAllBytesAsync("report.pdf");
foreach (var (name, client) in clients)
{
    using (client)
    {
        using var stream = new MemoryStream(bytes, writable: false);
        OcrResult r = await client.ExtractAsync(stream, "application/pdf");   // identical for every engine

        int tables = r.Pages.Sum(p => p.Elements.OfType<OcrTable>().Count());
        Console.WriteLine($"{name}: {r.Pages.Count} pages, {tables} tables, {r.Text.Length} chars");
    }
}

Stream pages as they finish — the IChatClient.GetStreamingResponseAsync twin. Each
OcrPageResult carries one completed page (plus progress + usage), so a RAG pipeline can chunk and
embed early pages while later pages are still being OCR'd; ToOcrResultAsync reduces the stream back to
the same OcrResult the unary call would return.

await foreach (OcrPageResult update in client.ExtractPagesAsync(stream, "application/pdf"))
{
    OcrPage page = update.Page;
    Console.WriteLine($"page {page.PageNumber}/{update.TotalPages}: {page.Text.Length} chars");
}

// …or reduce the whole stream back into one OcrResult (the ToChatResponseAsync pattern):
OcrResult full = await client.ExtractPagesAsync(stream, "application/pdf").ToOcrResultAsync();

Compose middleware with the builder — the same shape as ChatClientBuilder: you compose a client,
you don't set flags.

IOcrClient ocr = new FoundryMistralOcrClient(endpoint, cred)
    .AsBuilder()
    .UseOpenTelemetry(loggerFactory)
    .UseLogging(loggerFactory)
    .Build();

Register with dependency injection — the consumer depends only on IOcrClient; swap the engine
without touching downstream code.

services.AddOcrClient(sp => new FoundryMistralOcrClient(endpoint, new DefaultAzureCredential()))
        .UseOpenTelemetry()
        .UseLogging();

// Later, swap the engine on one line — nothing downstream changes:
services.AddOcrClient(sp => new AzureDocumentIntelligenceClient(diEndpoint, cred)).UseLogging();

Configure per-request options through the pipeline (for example, a local Ollama GLM-OCR engine that
needs a task-prefix prompt injected even when the caller passes no options):

IOcrClient ocr = engine
    .AsBuilder()
    .ConfigureOptions(o => o.ModelId ??= "mistral-ocr-4-0")
    .Build();

Bridge into a MEDI ingestion / RAG pipeline. A hosted document-AI service is a reader (the
LlamaParse-as-reader shape). One provider-agnostic reader composes any IOcrClient:

public sealed class OcrDocumentReader(IOcrClient ocr, OcrOptions? options = null) : IngestionDocumentReader
{
    public override async Task<IngestionDocument> ReadAsync(
        Stream source, string identifier, string mediaType, CancellationToken ct = default)
    {
        OcrResult r = await ocr.ExtractAsync(source, mediaType, options, ct);
        // map r.Pages -> IngestionDocument elements, stamping page/region/confidence/model metadata
    }
}

// Usage: one section per OCR page, each stamped with its 1-based PageNumber for downstream chunking.
var reader = new OcrDocumentReader(ocr);
IngestionDocument doc = await reader.ReadAsync(fileStream, "report.pdf", "application/pdf");

Extract from a remote URI (opt-in download). ExtractAsync(UriContent) never touches the network;
ExtractFromUriAsync is the explicit counterpart that fetches http(s) bytes with a caller-owned
HttpClient:

using var http = new HttpClient();
OcrResult r = await ocr.ExtractFromUriAsync(new UriContent(url, "application/pdf"), http);

Alternative Designs

1. Reuse IChatClient with multimodal content + a prompt. Rejected: loses native
   tables/bbox/confidence, is nondeterministic and token-expensive, and cannot represent non-chat engines
   (Azure DI, CU, local ONNX) at all. A vision LLM is supported as one provider behind IOcrClient
   (VisionLlmOcrClient), not as the contract. The prototype demonstrates this: three of four real
   engines use no IChatClient.
2. A flag on the reader (the VisionOnly approach). Rejected: a model choice hardened into a reader
   mode; temporal coupling; not decoratable (no middleware); does not generalize across engines.
3. A transport-parameterized single class (new MistralOcrClient(isFoundry: true)). Rejected:
   smuggles host branching into the type; breaks composition. Follow the OpenAIClient/AzureOpenAIClient
   precedent: the interface is portable; concrete classes split by host where the host leaks.
4. One interface for OCR and field extraction (a flag/overload). Rejected: different output
   contract; modeled as the sibling IDocumentAnalysisClient peer (the STT/TTS precedent). See Related
   and future work.

Validated design decisions (resolved by the prototype + sources)

• Geometry = typed polygon; table = structured cells. Azure DI BoundingRegion.Polygon is a
  possibly rotation-skewed quad; an axis-aligned rect would be lossy. Resolved with
  OcrBoundingRegion.Polygon as IReadOnlyList<OcrPoint> (populated natively by DI, by FromRectangle
  for Mistral's rect, reused for field grounding); GetBounds() returns the OcrBoundingBox struct for
  coarse filters. Tables resolve to OcrTable{RowCount, ColumnCount, Cells?, MarkdownRepresentation?}
  (DI cells primary, Mistral markdown fallback). The prototype demonstrates this: the DI polygon and
  the Mistral rect→quad both flow losslessly into the same reader metadata.
  (Sources: Azure/azure-sdk-for-net DocumentTable/DocumentTableCell/BoundingRegion; Mistral OCR schema.)
• Typed OcrPoint over a flat float[]. The earlier open question (flat IReadOnlyList<float> vs a
  point shape) is now resolved: a typed OcrPoint list makes an odd/empty coordinate count
  unrepresentable and reads correctly without documentation, while still carrying DI's 4-vertex quad
  without loss.
• 1-based OcrPage.PageNumber. Page identity matches how documents, Azure DI, and downstream
  provenance number pages (1-based), removing the off-by-one that a 0-based Index forced on adapters.
• Unary ExtractAsync + streaming ExtractPagesAsync (updated in API review; IProgress retired).
  The proposal originally shipped unary-only with an IProgress<OcrProgress> hook and deferred streaming.
  API review aligned it to the family instead: every sibling ships a streaming twin
  (IChatClient.GetStreamingResponseAsync, ISpeechToTextClient.GetStreamingTextAsync), and on
  netstandard2.0 / net462 there are no default interface methods, so streaming cannot be added later
  without a new interface — it has to ship now. ExtractPagesAsync yields one OcrPageResult per
  page as it finishes (progress fields PagesProcessed / TotalPages ride on the update, replacing
  OcrProgress), letting large-document RAG chunk/embed early pages while later pages are still
  parsing; OcrPageResultExtensions.ToOcrResultAsync reduces the stream back to a single OcrResult
  (the ToChatResponseAsync pattern). The prototype demonstrates this: each engine surfaces per-page
  updates and the reducer reassembles the same OcrResult as the unary call.
• Home / process. Lives in Microsoft.Extensions.AI.Abstractions, namespace Microsoft.Extensions.AI,
  [Experimental("MEAI001")] from day one, area-ai, full stack (abstractions +
  Delegating/Logging/OpenTelemetry/ConfigureOptions middleware + DI) in one PR. First mover: no existing
  OCR/IOcrClient issue in dotnet/extensions.
• Reading-order element model over a polymorphic base (resolved by the survey). Every surveyed engine
  emits elements in a single reading order; consumers want "iterate the page top-to-bottom" without
  re-deriving order from geometry. OcrPage therefore exposes one Elements list over an
  OcrElement base ($type-discriminated, the AIContent shape), with OfType<OcrTable>() /
  OfType<OcrImage>() projections — replacing the parallel Blocks/Tables/Images lists. The base is
  shaped to be promotable to a shared document-element type (cross-team track), so this is not a
  redesign later. Tables can carry nested OcrTableCell.Elements for structured cell content, keeping
  the flat Content string as a convenience.
• Kind taxonomies are open structs; units/origins are closed enums (resolved by the survey). Block and
  cell kinds vary by engine and grow over time, so OcrBlockKind / OcrTableCellKind stay
  ChatRole-style open structs (well-knowns RowHeader/RowSection added to cells). Coordinate
  units and origins are physically bounded, so OcrCoordinateUnit (Pixel/Point/Inch/Normalized)
  and the new OcrCoordinateOrigin (TopLeft/BottomLeft) are closed enums.
• Coordinate metadata is per page (resolved by SPIKE-08). CoordinateUnit + CoordinateOrigin live
  on OcrPage, not the document. Primary sources refuted a "one unit per document" assumption: Google
  models Page.Dimension { width, height, unit } per page, and Azure DI's per-page DocumentPage.Unit is
  pixel for image inputs and inch for PDF — so one analyze call over a mixed batch returns different
  units on different pages. OcrPage.Width/Height group into a new OcrPageDimensions value type
  (the extent), kept as a sibling to unit + origin (which together describe the coordinate system).
  A continuous page-rotation angle is deferred to AdditionalProperties for v1 (element rotation is
  already carried losslessly by the polygon OcrBoundingRegion).
• Markdown → Text. Engines return plain text far more often than genuine Markdown; the property is
  renamed Text on OcrPage and OcrResult (dropping Markdown). A real Markdown property can be
  re-added later if a genuine-Markdown need appears (additive).
• OcrUsage carries optional tokens. The vision-LLM path reports token usage; classic OCR engines
  report only page counts. OcrUsage gains nullable InputTokenCount/OutputTokenCount/TotalTokenCount
  alongside PagesProcessed (mirrors MEAI UsageDetails), left null by engines that don't bill tokens.

Basis: a 12-engine cross-provider output-model survey — the five demo providers (Azure Document
Intelligence, Azure Content Understanding, Mistral OCR, Foundry Mistral, a vision-LLM adapter) plus an
exhaustive pass over LlamaParse, LiteParse, Microsoft MarkItDown, IBM Docling, Unstructured.io, AWS
Textract, Google Cloud Document AI, Upstage Document Parse, and Adobe PDF Extract — spanning cloud OCR,
agentic/VLM parsers, open-source libraries, and markdown-first converters, each with primary-source
citations.

Staged decisions / open questions (by track)

The reshape above is the "just-do" round that ships on #7588. The genuinely cross-team, tooling, and
empirical-validation questions run as parallel tracks that do not block the PR:

Cross-team socialization (does not block #7588)

• Shared document-model package + assembly home. OcrElement is shaped to be promotable to a shared
  document-element type co-owned with the data-ingestion (MEDI) work; and whether
  Microsoft.Extensions.AI.Abstractions is the right home vs. a dedicated document package is an open,
  cross-team question.
• Can Azure Document Intelligence implement IOcrClient directly? Depends on the family rename below
  (AI-terminology-free names); conversation to start with the DI owners.
• Should IChatClient advertise vision capability? Capability metadata vs. an early/compile-time
  throw when a non-vision model is handed an image — a MEAI-team discussion.
• Family rename. Ocr* is kept for v1; an acronym-free scheme is deferred to one mechanical pass
  downstream of the shared-model + DI-implementability questions.

Implementation spike (separate impl package, not Abstractions)

• GetService for v1 (RESOLVED — SPIKE-04/05). Kept. A vision-LLM
  IChatClient.AsOcrClient() prototype confirmed that with a non-public adapter type,
  GetService<IChatClient>() is the only public unwrap to the inner chat client. The shipping
  middleware's GetService<OcrClientMetadata>() use is conceded as partly biased evidence (copied
  from IChatClient's middleware), but the real need — a telemetry decorator discovering an
  arbitrary client's provider identity — remains; dropping GetService would only trade it for a
  mandatory Metadata property while losing the provider-SDK escape hatch and the adapter unwrap.
  One optional seam covers all three; reversible while [Experimental].

Validation before formal review (via the demo + a DataIngestion reader)

• Justify RawRepresentation / AdditionalProperties (RESOLVED — SPIKE-06). An exhaustive
  14-engine raw-output inventory shows every engine emits structured data with no first-class home
  (Content Understanding's geometry is a source string, unparseable into a polygon), so both escape
  hatches stay; AdditionalProperties guidance tightened to real provider signals only. Three fields
  cleared the promote bar: OcrTableCell gained per-cell geometry + confidence + raw + properties
  (mirrored from OcrElement), OcrPage gained RawRepresentation, and detected language is
  deferred to a standardized detectedLanguages key (typed OcrDetectedLanguage later).
• Group Width/Height into an OcrPageDimensions (RESOLVED — SPIKE-07). Done: OcrPageDimensions
  is folded into the surface above, as a per-page sibling to unit + origin.
• Confirm Pixel is the dominant coordinate unit (RESOLVED — SPIKE-08). Units are not
  Pixel-dominant: they spread across Pixel/Point/Inch/Normalized and are reported per page, so unit +
  origin moved onto OcrPage. See the reshape rationale above.
• Confirm no cross-platform BCL geometry type fits (RESOLVED — SPIKE-09). No BCL type carries
  the load-bearing shape — a page-scoped, rotation-capable polygon (OcrBoundingRegion).
  System.Numerics.Vector2 has no region type; System.Drawing.RectangleF is axis-aligned only (it
  cannot hold Azure DI's skewed quad) and the BCL has no polygon primitive at all. Reusing
  System.Drawing.PointF for OcrPoint alone would delete no type while coupling OCR coordinates to a
  drawing namespace. The three OCR-owned primitives stay; note the package already uses a BCL value
  type where one fits exactly (ImageGenerationOptions.ImageSize is System.Drawing.Size).

Tooling

• Adopt the new C# extension-members syntax for the OCR extensions — first verifying the public-API
  baseline/analyzer story (OCR would be the repo's first user), then adopting.

Still open from before

• Whether .UseDistributedCache(...) ships in v1 (OCR is expensive). Logging, OpenTelemetry, and
  configure-options middleware already ship, matching the ISpeechToTextClient template.
• Naming of the shared region primitive if grounding is reused by other capabilities: keep OCR-prefixed
  OcrBoundingRegion (lowest churn) vs. promote to a capability-neutral BoundingRegion. Recommendation:
  keep the prefix for v1.
• Page vs. chunk streaming unit — ExtractPagesAsync / OcrPageResult use Page for v1; revisit only
  if the shared-model track elevates types toward a chunk model.

Risks

• Shape creep. Keep IOcrClient focused on OCR / document extraction. Field extraction stays in the
  sibling IDocumentAnalysisClient proposal (not this PR).
• Middleware scope. Logging, OpenTelemetry, and configure-options ship in v1, validated in the
  prototype and matching the ISpeechToTextClient template. Retry is intentionally not a built-in
  client: no Microsoft.Extensions.AI capability ships per-capability retry, since resilience belongs in
  the HTTP pipeline (Microsoft.Extensions.Http.Resilience); the .Use(...) primitive can still wrap a
  custom retry decorator. Distributed caching can follow the same builder shape, but API review should
  decide whether it ships in v1.
• Mutable result lists. OcrPage.Elements (and OcrTable.Cells) are settable / init-populated so
  providers can fill them incrementally; consumers treat the returned result as read-only. API review may
  prefer init-only / ctor-set here (constrained by the netstandard2.0 / net462 targets, which rule out
  required / DIMs).

Related and future work

Future sibling: IDocumentAnalysisClient (separate proposal)

Field extraction (a document + a schema/analyzer → typed fields with confidence + grounding) is a
categorically different output contract than OCR→markdown, exposed by multiple providers (Azure
DI custom models, Content Understanding). It is best modeled as a peer interface, not a flag/overload
on IOcrClient — the same call M.E.AI made splitting ISpeechToTextClient / ITextToSpeechClient. Its
grounding would reuse the SAME OcrBoundingRegion polygon and compose via the SAME builder, which is why
this PR ships the shared region primitive with IOcrClient. It carries its own harder design debates
(schema/analyzer-id representation, the typed-field value model, lifecycle) and its own provider matrix,
so it is deliberately out of scope for this PR and will be its own proposal. Sketch (illustrative only):

[Experimental("MEAI001")]
public interface IDocumentAnalysisClient
{
    Task<DocumentAnalysisResult> AnalyzeAsync(
        Stream document, string mediaType, DocumentAnalysisOptions options,
        CancellationToken cancellationToken = default);
    object? GetService(Type serviceType, object? serviceKey = null);
}

public sealed class DocumentField
{
    public string Name { get; }
    public object? Value { get; set; }
    public string? ValueType { get; set; }
    public double? Confidence { get; set; }
    public OcrBoundingRegion? Grounding { get; set; }     // SHARED region primitive, reused
    public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
}

References

• CommunityToolkit/AI #3: Document Processing packages (MEDI); the VisionOnly flag this proposal replaces.
• area-data-ingestion IngestionPipeline (dotnet/extensions #7488): the downstream consumer of an OcrDocumentReader.
• M.E.AI ADRs cited for the design: STT/TTS split (dotnet/extensions #5838), capability-family precedents (IVideoGenerator dotnet/extensions #7420).
• Prototype + coverage matrix runnable offline: the public iocrclient-demo (one IOcrClient in front of four OCR engines, bridged into a MEDI RAG pipeline).

[luisquintanilla]

Consumer validation: a reference document-ingestion app now consumes IOcrClient directly from the Microsoft.Extensions.AI package end to end, which exercises the proposed surface as a real dependency rather than a copy.

The app previously carried an in-repo copy of the IOcrClient family. After packing this branch's Microsoft.Extensions.AI(.Abstractions) locally, the app deleted its in-app abstraction (~470 lines) and now depends on the package, keeping only the genuinely app-owned pieces:

• a provider client (Azure AI Foundry Mistral Document AI / OCR, keyless via DefaultAzureCredential) implementing IOcrClient;
• an app-owned UseRetry() pipeline stage (this proposal intentionally ships no retry — resilience is an HTTP-pipeline concern), composed over the package's DelegatingOcrClient + OcrClientBuilder;
• a MEDI IngestionDocumentReader bridge and a small OcrTable → markdown renderer.

The composition reads exactly like the IChatClient story: new FoundryMistralOcrClient(endpoint, credential, model).AsBuilder().UseRetry().UseLogging().Build(). Opting into the experimental surface is a one-line <NoWarn>$(NoWarn);MEAI001</NoWarn>. Builds clean (0 errors) with no local definitions of the package types remaining — confirming the abstraction is the seam, and providers + cross-cutting concerns compose around it.

Reference app branch: https://github.com/luisquintanilla/PdfAIngest/tree/feature/mistral-ocr-reader

[JohnGalt1717]

Is there a way to pass multiple images as streams with this? It looks like a single file. I.e. if you have a nasty pdf, the best way to use these models is to use pdfpig or similar and convert every page in the pdf to an image, then pass all images in for each page and have it process on the images. (normally with chunking and windowing of the bottom 1/6th of the previous page etc)

If not, is your intent to have it automatically recognize the file type and convert them? (seems like a lot of implicit overhead and optimization work...)

If not, then I guess we could just default to the windowing method, but being able to upload all of the pages as stream to process on would be better if supported.