Proposal: Carrying error detail over the TCP/QUIC/WebSocket binary protocol

[jiengup]

Problem

Issue raised by #3735

When the Iggy server rejects a request, the binary protocol response carries only a
4-byte discriminant (e.g., 3007 for PartitionNotFound). The actual field values
(partition ID, topic ID, stream ID) are discarded before hitting the wire:

Server builds:    PartitionNotFound(5, topic_42, stream_1)
send_error_response → [3007_u32_le] [0x00000000]    ← empty payload
Client rebuilds:  PartitionNotFound(0, Identifier(0), Identifier(0))

Due to &[] payload in send_error_response<T> in core/server/src/sender/mod.rs here:

iggy/core/server/src/sender/mod.rs

Lines 199 to 207 in d9635b6

	pub(crate) async fn send_error_response<T>(
	stream: &mut T,
	error: IggyError,
	) -> Result<(), IggyError>
	where
	T: AsyncReadExt + AsyncWriteExt + Unpin,
	{
	send_response(stream, &error.as_code().to_le_bytes(), &[]).await
	}

Every data-bearing IggyError variant (StreamIdNotFound, TopicIdNotFound,
InvalidOffset, ClientNotFound, …) arrives with zeroed or empty fields. Two
genuinely different requests produce byte-identical errors. The HTTP transport is
unaffected (it serializes error.to_string() as JSON reason), proving the data
exists and is merely not transported.

Current SDK behavior

Each SDK handles the [status:4][length:4][body:N] response frame independently,
with widely varying treatment of the error body:

SDK	Wire parsing	Reads body on error?	Body treatment	Exception produced
C++	Rust FFI	Yes (inside Rust)	IggyError → formatted string	std::runtime_error
C#	Own (raw socket)	Yes	UTF-8 decoded → exception message	IggyInvalidStatusCodeException
Go	Own (raw socket)	Reads and discards	Unused	Typed IggyError structs (code only)
Java	Own (Netty)	Yes	UTF-8 decoded → exception reason	IggyServerException + typed subclasses
Node.js	Own (raw socket)	Frame includes body, but ignores body content	Unused	Generic Error
PHP	Rust FFI	Yes (inside Rust)	IggyError → PHP exception	Typed PHP exception hierarchy
Python	Rust FFI	Yes (inside Rust)	IggyError → RuntimeError(e.to_string())	RuntimeError (undifferentiated)

Three SDKS do their own parsing (C#, Go, Java). Four delegate to Rust (C++, Python,
PHP, Node via different approaches). The body contents are currently unused in
practice because the server sends an empty payload.

Proposed approaches

Option A — String payload (lightweight)

The server sends error.to_string() as the response body:

Server:   PartitionNotFound(5, topic_42, stream_1)
  └─ body = error.to_string()
     = "Partition with ID: 5 for topic with ID: 42 for stream with ID: 1 was not found."

Wire:     [status=3007][len=80]["Partition with ID: 5 for topic..."]

Client:   reads body → decodes as UTF-8 → displays to caller
           error type is determined by the status code alone (e.g., 3007 → PartitionNotFound)

Pros

• Minimal SDK changes: C# and Java already decode the body as UTF-8 string today.
  Go and Node.js need only to consume or surface the body bytes.
• The error code in the header preserves the semantic type of the error (callers can still inspect the status code to differentiate PartitionNotFound from InvalidOffset, etc.).
• No wire-format specification doc needed — the contract is "the body is a UTF-8 encoded human-readable error description."

Cons

• Callers cannot reconstruct a typed error with correct field values (e.g., a structured PartitionNotFound object with partition_id=5) from a flat string. For most use cases (logging, debug output), the human-readable string is sufficient.

---

Option B — Structured serialization (full fidelity)

Define a wire-format specification that carries the error's fields as structured binary data, enabling callers to reconstruct the original typed error with every field populated.

Two sub-options:

B1. Custom specification

Define a simple binary layout (e.g., field-order matches the variant's tuple fields, integers as LE bytes, identifiers as [kind:u8][len:u8][bytes]). This is what was prototyped in #3751.

Pros: No external dependency. Full control over the wire format.
Cons: Must be manually reimplemented in every SDK that does its own wire parsing (C#, Go, Java, Node.js). Requires a specification document. Every new error variant must be added to the spec and each implementation.

B2. Standard serialization framework

Use a cross-language framework such as protobuf (already used in the connectors module) or Apache Fury to serialize error payloads.

Pros: Code generation for all target languages. Schema serves as documentation. No hand-written parser maintenance.
Cons: Introduces a dependency to every SDK. Serialization overhead for small error payloads (~20–80 bytes). Protobuf/Avro schema must be kept in sync with IggyError variants.

---

Questions for discussion

1. Is full structured error reconstruction necessary? In practice, are callers inspecting error field values (e.g., the rejected partition ID) in application logic, or is the human-readable message sufficient?

2. If structured, B1 (custom) or B2 (framework)? Protobuf is already a transitive dependency via connectors. Would adding it to the error path be acceptable, or is a lightweight custom spec preferred?

3. Backward compatibility. Option A requires no changes to Java and C# (they already decode UTF-8 bodies). Option B requires changes to ALL self-parsing SDKs (C#, Go, Java, Node.js). Is the additional fidelity worth the per-SDK implementation cost?

Related Issues and PRs

• Figure out how to extract IggyError into an dedicated crate. #2524
• feat(csharp): add iggy error status codes #2519

[jiengup]

A sample protobuf message definition will look like this:

syntax = "proto3";

message ErrorPayload {
  uint32 schema_version = 1;
  string message = 2;

  oneof details {
    PartitionNotFoundDetails partition_not_found = 10;
    ResourceDetails resource = 11;
    ChecksumDetails checksum = 12;
  }
}

message PartitionNotFoundDetails {
  uint64 partition_id = 1;
  Identifier topic_id = 2;
  Identifier stream_id = 3;
}

message Identifier {
  oneof value {
    uint32 id = 1;
    string name = 2;
  }
}

[lukaszzborek]

Few quick thoughts:
Some time ago, there was idea for using flatbuffers #2524 to autogenerate errors for all languages.
I think that at this stage, we shouldn't worry about backward compatibility just yet.

In my opinion, we should go to option B. I'm not sure if we need Apache.Fury (I don't know it, I need to check it out). Server also has to send every field value and rendered message from server (like in http). This means we don't need to store message templates in every sdk, and we also have way to get all parts of field values like topic_id, stream_id (like in your example protobuf).
Also propably in C#/Java/Go we want to have some sync enum/typed-error list for error codes, instead of just numbers (like now in c#). It can be generated like in go (or my old pr 😅), but I think it should be also topic of that discussion to cover that

[jiengup]

Thanks for your context. I think flatbuffer is also a good choice. It's be used in some famous projects such as Apache Arrow as I know.

[jiengup]

After discussion with the core team, it has been decided that we will postpone the error protocol implementation until the VSR cluster feature is stable.

This proposal can be used for tracking and further discussion.

[hubcio]

I will prepare design doc and post it here. I actually have something in mind, to finally get rid of big nasty IggyError and have per-command individual errors + mechanism that allows cross-sdk maintainability of it.