feat: emit partial (pending) spans at span start - #37
Merged
Conversation
Opt-in (init(partial_spans=True) / GLASSFLOW_PARTIAL_SPANS, default OFF until the backend's unfinished-spans storage ships): every sampled span additionally exports a content-free snapshot at START — same trace/span/parent ids, name, and start timestamp as the final span, zero duration, marked glassflow.span.pending=true. The backend stores it as an unfinished row the real span replaces at end; a snapshot that is never replaced is the durable record of what a crashed agent was doing. Mechanics: PendingSpanProcessor's on_start builds a ReadableSpan snapshot and delegates it to the provider's existing BatchSpanProcessor (shared exporter/batching/retries/masking; on_start stays an in-memory enqueue — the never-block guarantee holds). Attributes are filtered by an identity ALLOWLIST (kind, operation, provider, tool name, plus the gen_ai.request.* prefix) so content can never ride a pending, whatever instrumentation set it. Sampled-out spans and disabled mode produce no snapshots. The marker key knowingly bends the convention-native rule: OTel has no pending-span mechanism to align with (spec #3732/#4646, semconv #2133 open, none planned); rationale recorded in semconv.py. Supporting change: the SDK's own span/generation helpers now attach identity attributes at span CREATION (kind_attributes/_creation_ attributes) instead of only immediately after — on_start-built snapshots would otherwise see empty attributes. Final spans are byte-identical (the same values were previously set post-creation). The _emit seam is the future debounce point (only emit if still open after N seconds — the network-volume escape valve for fast spans); designed for, deliberately not built (follow-up ticket).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements GLA2-195 (Partial spans project): the SDK half of the pending-spans design.
What
With
init(partial_spans=True)(orGLASSFLOW_PARTIAL_SPANS; default OFF until the backend'sFinished-column storage ships), every sampled span exports twice:trace_id/span_id/parent/name/start timestamp as the final span (the identity ClickHouse keys replacement on),end_time == start_time, markedglassflow.span.pending = true, carrying only identity attributes (kind, operation, model/provider via thegen_ai.request.*prefix).Flag off (the default): behavior is byte-identical to today — the processor isn't even registered.
Design points
PendingSpanProcessor.on_starthands the snapshot to the provider's existingBatchSpanProcessor— shared exporter, batching, retries, masking.on_startstays an in-memory enqueue, so the never-block guarantee holds.input.valueset at creation never reaching the snapshot.glassflow.span.pendingbends the convention-native rule knowingly — OTel has no pending-span mechanism (spec #3732/#4646, semconv #2133 all open, none planned; Logfire'slogfire.span_typeis equally vendor-namespaced). Rationale lives insemconv.pyper the ticket's AC.on_start, so late-set attributes are invisible to them. Final spans are unchanged; a test asserts the at-start visibility._emitis where a future "only emit if still open after N seconds" timer wraps (cancel via the processor's already-receivedon_end), cutting pending volume for fast spans without any wire-format change. Follow-up ticket covers it.Tests
TDD: 9 new tests written first and watched fail — identity mirroring (ids/name/start), zero duration, parent linkage, allowlist vs content, flag-off byte-identical, sampled-out and disabled produce nothing, env-var resolution, at-creation attribute visibility. Full suite 156 passed; ruff + format + mypy --strict clean.