Skip to content

fix(go/tracing): mark only originating span as failure source - #6047

Open
ktsoator wants to merge 2 commits into
genkit-ai:mainfrom
ktsoator:fix/go-failure-source
Open

fix(go/tracing): mark only originating span as failure source#6047
ktsoator wants to merge 2 commits into
genkit-ai:mainfrom
ktsoator:fix/go-failure-source

Conversation

@ktsoator

Copy link
Copy Markdown

Summary

  • mark only the span where an error originates with genkit:isFailureSource
  • propagate an internal handled marker through nested Genkit spans while keeping parent spans in the error state
  • preserve the application error type in OpenTelemetry events and restore the original error identity at the outermost Genkit span

Problem

RunInNewSpan currently sets IsFailureSource on every span that observes a returned error. As the same error propagates through nested actions and flows, each parent span therefore exports genkit:isFailureSource=true, which obscures the actual source and can overcount failures in monitoring.

The tracing package already contains markedError, markErrorAsHandled, and isErrorAlreadyMarked, but the marker was never applied in RunInNewSpan.

Implementation

The originating span now marks and wraps an unhandled error before returning it. Parent spans recognize the marker and omit genkit:isFailureSource, while still recording their normal OpenTelemetry error event and error status.

The internal wrapper is removed when recording the application error in OpenTelemetry and when the error exits the outermost Genkit span. This keeps exception.type, direct sentinel comparisons, and errors.Is/errors.As behavior compatible with the original error.

Testing

Added regression coverage that verifies:

  • the child/originating span exports genkit:isFailureSource=true
  • the parent span does not export the failure-source attribute
  • both child and parent spans remain in the error state
  • the original error identity and errors.Is behavior are preserved
  • the internal marker is removed without stripping application-added wrappers

Commands run:

go vet ./core/tracing
go test -race ./core/tracing
go test ./...

All passed.

Fixes #5153

Checklist

  • PR title follows Conventional Commits
  • Tested with unit, race, vet, and full Go test suite
  • Docs not applicable: no public API or user-facing behavior changed

Track handled errors as they propagate through nested Genkit spans so only the span where an error originates receives genkit:isFailureSource. Parent spans remain in the error state and continue to record OpenTelemetry error events and status.

Keep the internal marker out of OpenTelemetry exception types and unwrap it at the outermost Genkit span to preserve the application's original error identity and errors.Is/errors.As behavior.

Add regression coverage for nested span attributes, error states, error identity, and marker unwrapping.

Fixes genkit-ai#5153

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an unwrapMarkedError helper function to preserve the original error identity in telemetry, updates RunInNewSpan to correctly mark and unwrap errors, and enhances testing around failure sources and error unwrapping. Feedback on these changes includes adding a nil check during type assertion in unwrapMarkedError to prevent potential nil pointer dereferences, and checking if err != nil before unwrapping in RunInNewSpan to avoid unnecessary operations.

Comment thread go/core/tracing/tracing.go
Comment thread go/core/tracing/tracing.go Outdated
Handle typed-nil marked errors without dereferencing them and avoid marker unwrapping on successful root spans. Add regression coverage for the typed-nil case.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Go] Prevent genkit:isFailureSource attribute from propagating to parent spans

1 participant