Skip to content

fix(tracing): stop display sessions feeding back on the dashboard itself - #68

Merged
thiagoesteves merged 1 commit into
mainfrom
thiagoesteves/fix-flaky-tests
Jul 17, 2026
Merged

fix(tracing): stop display sessions feeding back on the dashboard itself#68
thiagoesteves merged 1 commit into
mainfrom
thiagoesteves/fix-flaky-tests

Conversation

@thiagoesteves

Copy link
Copy Markdown
Owner

What

Fixes the intermittent failures in the tracing/version test files (promised in #64) - and the root cause turned out to be a real product bug, not just test timing.

The bug

Display trace sessions send every message to the requesting LiveView, whose handle_info calls ObserverWeb.Common.uuid4() for stream ids (and plenty of Enum while rendering). Tracing any module the dashboard itself uses (ObserverWeb.Common, Enum, ...) therefore feeds back on itself: every rendered trace message generates the next traced call, in a storm that runs until max_messages aborts the session. In tests this raced the assertions (flaky by seed and machine load); for real users it silently kills exactly the kind of session "trace Enum" describes.

Tracer.Server.handle_trace/2 now drops display events originating from the request_pid. Tool sessions (count/duration/call-seq/flame-graph) only render at the end, so they keep the requester's calls.

Test hardening

  • Version server tests synchronize on the mocked RPC + :sys.get_state instead of Process.sleep(100); the :warning test now actually asserts the warning status (it previously asserted nothing and said so in a comment).
  • Tracing page/tracer tests wait for trace messages and session-state transitions with bounded polling or explicit messages instead of fixed sleeps; Request Function terminate synchronizes on session creation before waiting for the reset (its old 50ms sleep could also leak a running session into the next test).
  • Tracer unit tests invoke traced functions from a Task, since the requester's own calls are now (correctly) excluded.

Verification

  • 12/12 randomized-seed runs green on the previously flaky files (they failed roughly 1 in 3 before).
  • 5/5 randomized full-suite runs green (395 tests).

Risk assessment

  • Impact: display sessions no longer report calls made by the dashboard's own LiveView; all other origins unchanged.
  • Blast radius: one guard in Tracer.Server.handle_trace/2; everything else is test-only.
  • Regression risk: low - the excluded origin was self-observation noise that could abort sessions; coverage 96.0%, credo/sobelow/dialyzer/format clean.
  • Rollback plan: revert the commit.

Checklist

  • mix test green across repeated randomized runs
  • mix coveralls 96.0% (threshold 95%)
  • mix credo --strict, mix sobelow, mix dialyzer, mix format --check-formatted clean
  • Small focused diff, no leftover debug output

🤖 Generated with Claude Code

Root cause of the intermittent tracing/version test failures.

Display trace sessions delivered every message to the requesting
LiveView, whose handle_info calls ObserverWeb.Common.uuid4() (stream
ids) and plenty of Enum while rendering. Tracing any module the
dashboard itself uses therefore fed back on itself: every rendered
message generated the next traced call until max_messages aborted the
session mid-test (and mid-real-session for users). Tracer.Server now
drops display events originating from the request_pid; tool sessions
only render at the end and keep the requester's calls.

Test hardening for the remaining sleep-based races:
- version server tests synchronize on the mocked RPC plus
  :sys.get_state instead of fixed sleeps; the :warning test now
  asserts the warning status it previously only commented about.
- tracing page/tracer tests wait for trace messages and session state
  transitions with bounded polling or messages instead of sleeps; unit
  tests invoke traced functions from a Task since the requester's own
  calls are now (correctly) excluded.

Risk assessment:
- Impact: display trace sessions no longer report calls made by the
  observer dashboard's own LiveView; all other origins are unchanged.
- Blast radius: one guard in Tracer.Server.handle_trace plus test-only
  changes.
- Regression risk: low - the excluded origin was noise that could
  abort sessions; suite green 12/12 randomized runs on the previously
  flaky files and 5/5 full-suite runs (395 tests, 96.0% coverage).
- Rollback plan: revert the commit.
@thiagoesteves
thiagoesteves force-pushed the thiagoesteves/fix-flaky-tests branch from 9640745 to 4e0ae19 Compare July 17, 2026 18:08
@thiagoesteves
thiagoesteves merged commit 29bfadc into main Jul 17, 2026
3 checks passed
@thiagoesteves
thiagoesteves deleted the thiagoesteves/fix-flaky-tests branch July 17, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant