Reduce writer lock contention + unify the writer/medium queue - #576
Open
FZambia wants to merge 7 commits into
Open
Reduce writer lock contention + unify the writer/medium queue#576FZambia wants to merge 7 commits into
FZambia wants to merge 7 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #576 +/- ##
==========================================
- Coverage 84.64% 84.17% -0.48%
==========================================
Files 58 58
Lines 14996 14955 -41
==========================================
- Hits 12693 12588 -105
- Misses 1644 1703 +59
- Partials 659 664 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reworks the internal unbounded queue behind the per-connection message
writer and the per-channel channel medium:
goroutines (concurrent broadcasts/subscriptions) produce into one
connection's queue.
internal/queue.Queueandchannel_medium.go'spublicationQueue) into one genericqueue.Queue[T],removing ~150 lines of duplicate code.
channelMedium.close()(a writer goroutine couldpark forever on an empty queue if the queue wasn't closed).
The queue stays an unbounded, dynamically grown/shrunk ring buffer tuned for
MPSC (many producers, exactly one consumer — how both callers use it).
What changed
internal/queue/queue.goQueue→Queue[T any]with asizeFn func(T) intfor byte accounting.Item/ItemSizestay for the writer; the medium parameterises it withqueuedPublication+queuedPublicationSize.sync.Mutex(master usedsync.RWMutex). For thisworkload — short critical sections hit by many producers — a
sync.Mutexismeasurably lighter than
RWMutex, and it's the main source of thehigh-contention wins below.
Waitusessync.Cond(on that mutex). An earlier iteration ofthis branch used a channel doorbell; profiling showed channel send/recv take
the runtime hchan lock on every park/unpark and that dominates when the single
consumer parks frequently —
sync.Condis cheaper, so it was restored.cnt/sizeare plainint64under the mutex, not atomics (an atomic RMWper op bounces its cache line across all producer cores).
closed/cpstayatomic (written only on close/resize → free lock-free reads for
Closed/Cap).Add/AddManyreturn the post-insert byte size, so the writer'sMaxQueueSizecheck avoids a second synchronisedSize()call (master tooktwo locks per enqueue: Add + Size).
sizeFnis computed outside the critical section inAdd.waitingflag gatescond.Signalso producers skip the wakeup when theconsumer isn't parked. No lost wakeup (the flag is set/read under the same
mutex the consumer uses to check
cnt).writer.goenqueue/enqueueManyuse the size returned byAdd/AddMany.timerScheduledFlag atomic.Bool: in timer mode, concurrent producers skipthe writer mutex while a flush is already pending.
flushreduced from up to 3Len()calls to 1.channel_medium.goqueue.Queue[queuedPublication]; the bespokepublicationQueueis gone.Tests / benchmarks
idempotent close, the returned-size contract, and non-
Iteminstantiations.BenchmarkWriteMerge/BenchmarkWriteMergeDisabledreplaced by oneself-contained suite,
BenchmarkWriterQueue(writer_queue_bench_test.go),covering every writer path: all flush modes (immediate / merged / delay-batched
/ timer), every
maxMessagesInFrameregime (1, N, unlimited), both enqueueentry points, the shrink path, and 1 vs 8 producers. Uses only the writer API
so the identical file runs on master for comparison.
Correctness / safety
MPSC is a hard invariant (single consumer).
Close/CloseRemainingusecond.Broadcast. No lost wakeups (waiting-flag reasoning + close/wake racetests). Idempotent
Close. Whole suite passes-race(-count≥ 5),go vetand
gofmtclean.Benchmarks
Methodology — the benchmark models a real writer, which matters enormously:
master/branch, 12 rounds,
-benchtime=300ms) — sequential runs drift far toomuch to compare (±30% on master's own numbers).
resize is measured, not disabled.
write(2)syscall per frame to/dev/null. A real consumer spends its time in that syscall outside thequeue lock, so it cannot monopolise the lock.
enqueuing. Real broadcast producers are not infinitely-fast enqueue loops;
this pacing keeps the consumer able to keep up, so the queue stays shallow
(frames of tens–~128 messages, never thousands) — exactly how a healthy
connection behaves. (A connection slow enough to back the queue up to thousands
is disconnected as slow via
MaxQueueSize.)ns/opis per produced message. Apple M4.Reading the results
−16.16%, with the big wins under concurrent producers (the broadcast fan-in
this targets):
Simple/8p −68%,BatchedMany8/8p −52%,DelayBatch16/8p −32%,Batched16/8p −25%,TimerUnlimited/8p −17%,TimerShrink16/8p −16%.remove; the encode cost dominates per-message time there, which is realistic).
How we got here (so the design choices are documented, not guessed)
pprofplus controlled experiments pinned every choice:sync.Mutexis the source of the wins — switching the queue toRWMutex(master's type) erased them.
sync.Condbeats a channel doorbell for this MPSC use —pprofshowedchannel send/recv hammering the runtime hchan lock on every park/unpark.
sizeFnis not a factor — a parallel byte-size ring removedthe per-item indirect call from the drain and changed nothing (reverted).
transport made the consumer a lock-monopolising spin, and infinitely-fast
pure-loop producers forced the consumer to drain thousands of items under
the lock — neither happens in production. Modeling the per-frame write syscall
and the per-message producer encode (which keeps the queue shallow) made
the apparent large/unlimited-frame regressions disappear entirely.
Net
A clear win across the realistic operating range — biggest under concurrent
broadcast fan-in, neutral where there's little contention to remove — with no
regressions.