fix(tracing): stop display sessions feeding back on the dashboard itself - #68
Merged
Merged
Conversation
4 tasks
Root cause of the intermittent tracing/version test failures. Display trace sessions delivered every message to the requesting LiveView, whose handle_info calls ObserverWeb.Common.uuid4() (stream ids) and plenty of Enum while rendering. Tracing any module the dashboard itself uses therefore fed back on itself: every rendered message generated the next traced call until max_messages aborted the session mid-test (and mid-real-session for users). Tracer.Server now drops display events originating from the request_pid; tool sessions only render at the end and keep the requester's calls. Test hardening for the remaining sleep-based races: - version server tests synchronize on the mocked RPC plus :sys.get_state instead of fixed sleeps; the :warning test now asserts the warning status it previously only commented about. - tracing page/tracer tests wait for trace messages and session state transitions with bounded polling or messages instead of sleeps; unit tests invoke traced functions from a Task since the requester's own calls are now (correctly) excluded. Risk assessment: - Impact: display trace sessions no longer report calls made by the observer dashboard's own LiveView; all other origins are unchanged. - Blast radius: one guard in Tracer.Server.handle_trace plus test-only changes. - Regression risk: low - the excluded origin was noise that could abort sessions; suite green 12/12 randomized runs on the previously flaky files and 5/5 full-suite runs (395 tests, 96.0% coverage). - Rollback plan: revert the commit.
thiagoesteves
force-pushed
the
thiagoesteves/fix-flaky-tests
branch
from
July 17, 2026 18:08
9640745 to
4e0ae19
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fixes the intermittent failures in the tracing/version test files (promised in #64) - and the root cause turned out to be a real product bug, not just test timing.
The bug
Display trace sessions send every message to the requesting LiveView, whose
handle_infocallsObserverWeb.Common.uuid4()for stream ids (and plenty ofEnumwhile rendering). Tracing any module the dashboard itself uses (ObserverWeb.Common,Enum, ...) therefore feeds back on itself: every rendered trace message generates the next traced call, in a storm that runs untilmax_messagesaborts the session. In tests this raced the assertions (flaky by seed and machine load); for real users it silently kills exactly the kind of session "traceEnum" describes.Tracer.Server.handle_trace/2now drops display events originating from therequest_pid. Tool sessions (count/duration/call-seq/flame-graph) only render at the end, so they keep the requester's calls.Test hardening
:sys.get_stateinstead ofProcess.sleep(100); the:warningtest now actually asserts the warning status (it previously asserted nothing and said so in a comment).Request Function terminatesynchronizes on session creation before waiting for the reset (its old 50ms sleep could also leak a running session into the next test).Task, since the requester's own calls are now (correctly) excluded.Verification
Risk assessment
Tracer.Server.handle_trace/2; everything else is test-only.Checklist
mix testgreen across repeated randomized runsmix coveralls96.0% (threshold 95%)mix credo --strict,mix sobelow,mix dialyzer,mix format --check-formattedclean🤖 Generated with Claude Code