[.NET 1.0.6][FFI] CreateSessionAsync can hang after successful startup in Linux/Kubernetes

[mit2nil]

Summary

With the .NET SDK 1.0.6 in-process transport (RuntimeConnection.ForInProcess()), CopilotClient.StartAsync() succeeds, but a later CreateSessionAsync() can remain incomplete indefinitely in a Linux container running on Kubernetes.

The same application/configuration works locally, and changing only the replacement transport from FFI to the default stdio child-process transport recovers session creation.

Environment

• GitHub Copilot SDK for .NET: 1.0.6
• Runtime: .NET 8
• Host: Linux container on Kubernetes
• Client mode: CopilotClientMode.Empty
• Transport: RuntimeConnection.ForInProcess()
• Session uses a custom provider, tools/hooks, and SessionFs

Observed behavior

1. Create the client with RuntimeConnection.ForInProcess().
2. StartAsync() completes successfully and the transport reports setup complete.
3. Call CreateSessionAsync(sessionConfig).
4. The returned task never completes or faults. The API has no cancellation token, so the caller cannot stop it.
5. Recreating another client with the same FFI transport can reproduce the wedge.
6. Recreating the client with Connection = null (stdio) allows the same session configuration to succeed.

Representative host-side timeout logs:

CreateSessionAsync did not return within 120000ms; abandoning session creation. The SDK transport is likely wedged.

Create failed with a dead/wedged transport; recreating the Copilot client and retrying once.

We also observed that disposal of sessions/client associated with the wedged FFI transport may not complete promptly, so recovery must not depend on unbounded DisposeAsync() calls.

Expected behavior

CreateSessionAsync() should either:

• complete successfully,
• fail with a transport/runtime exception, or
• accept cancellation/a timeout so the host can recover.

If the in-process runtime becomes unhealthy after startup, recreating the client should not recreate the same permanently wedged runtime state.

Current workaround

We keep FFI as the preferred transport, but on the first post-start session-create/send transport timeout:

1. mark the FFI transport unhealthy,
2. detach sessions associated with that client generation,
3. bound/abandon cleanup of the wedged runtime,
4. recreate the client with stdio, and
5. retry once with a rebuilt session configuration.

Related issue

#1934 tracks options/environment parity for the in-process transport. This appears distinct: startup succeeds, but the FFI transport later stops completing session operations.

Please let me know what additional native/runtime logging or a smaller container repro would be most useful.

[github-actions]

Investigation findings

I investigated this by reading dotnet/src/FfiRuntimeHost.cs and dotnet/src/Client.cs.

Root cause confirmed — this is a real bug

The hang is caused by a fundamental difference in how the FFI transport handles transport failure vs. the stdio/TCP transports:

Stdio child process: When the CLI process exits (or crashes), its stdout pipe closes → the JSON-RPC read loop sees EOF → ConnectionLostException is thrown → all pending RPC calls (including session.create) fault immediately.

TCP: Same — socket close propagates as an exception.

FFI / in-process (RuntimeConnection.ForInProcess()): The transport is a managed Channel<byte[]> inside CallbackReceiveStream (FfiRuntimeHost.cs). The native Rust side pushes frames by invoking FeedInbound → _channel.Writer.TryWrite(data). The channel only ever closes (sends EOF to JSON-RPC) when _receiveStream.Complete() is called — which only happens inside FfiRuntimeHost.Dispose().

If the native runtime becomes wedged after StartAsync() returns (stops delivering frames without the managed host being disposed), the channel stays permanently open with no new data. The JSON-RPC read loop blocks forever at _channel.Reader.WaitToReadAsync(...), and any pending RPC call — including session.create — waits indefinitely for a response that will never arrive. There is no built-in watchdog or health-check timeout that would close the channel.

On CreateSessionAsync and cancellation

Contrary to the description that "The API has no cancellation token", CreateSessionAsync does accept an optional CancellationToken (second parameter, default CancellationToken.None):

public async Task<CopilotSession> CreateSessionAsync(SessionConfig config, CancellationToken cancellationToken = default)

The token is forwarded to InvokeRpcAsync → JsonRpc.InvokeAsync, so passing a token with a deadline does allow the caller to unblock and recover. However:

• The parameter is optional and not prominently documented in the context of transport-health scenarios
• Using default (no token) means the call truly hangs with no exit
• The SDK provides no built-in timeout for this case

On "recreating the client reproduces the wedge"

PrepareNativeLibrary loads the cdylib once per process (singleton). Each new CopilotClient creates a new FfiRuntimeHost and calls NativeHostStart to get a fresh _serverId, but the underlying native runtime (Node.js embedded in Rust) is shared across all calls. If process-level state inside the native library is corrupted or if the native host/connection can't be re-initialized cleanly after the previous instance was abandoned without proper disposal, subsequent NativeHostStart calls may inherit the broken state.

Summary

Aspect	Status
CreateSessionAsync can hang indefinitely with FFI transport	✅ Confirmed bug
No built-in SDK timeout for wedged FFI transport	✅ Confirmed — stdio/TCP get natural EOF; FFI does not
CreateSessionAsync has no CancellationToken	❌ Incorrect — the parameter exists but is optional/defaulted
Re-creating client can reproduce wedge	⚠️ Plausible given shared native library state

The core fix needed is for the FFI transport to propagate native-side health degradation to the managed Channel (e.g., call _receiveStream.Complete() on native disconnect events), so the JSON-RPC layer can detect the failure and fault pending calls the same way stdio/TCP do naturally.

Generated by Bug Handler for issue #1958 · sonnet46 2M · ◷

[SteveSandersonMS]

@mit2nil Can you repro this on Linux without involving Kubernetes?