Route Master/Replica SCAN continuations to the issuing node #3287 - #3843
Open
lsh1215 wants to merge 3 commits into
Open
Route Master/Replica SCAN continuations to the issuing node #3287#3843lsh1215 wants to merge 3 commits into
lsh1215 wants to merge 3 commits into
Conversation
SCAN cursors are node-local state: a continuation request must be routed to the node that issued the cursor. ScanCursor now carries an optional source RedisURI so drivers can associate a cursor with its issuing node. The attribute is package-private and exposed across packages through ScanCursorAccessor (following the OrderingReadFromAccessor pattern), leaving the public ScanCursor API unchanged. The immutable INITIAL/FINISHED cursors reject setSource.
Scan command builders now copy the source-node hint from the input cursor to the cursor that carries the response, so reusing the returned cursor object for continuation calls retains the association (as ScanIterator and ScanStream already do). Applied uniformly across all SCAN/HSCAN/SSCAN/ZSCAN builders including streaming variants.
MasterReplicaChannelWriter now recognizes scan commands: continuation requests (a cursor carrying a source) are pinned to the issuing node, while the node selected for an initial request is stamped onto the returned cursor. MasterReplicaConnectionProvider gains a node-reporting overload and getPinnedConnectionAsync, which fails fast with a descriptive RedisException when the issuing node is no longer part of the topology instead of silently routing elsewhere. Read load-balancing for non-scan reads and for initial scan selection is unchanged. Adds unit coverage plus a {SCAN,HSCAN,SSCAN,ZSCAN} x {sync,async,reactive} integration matrix under ReadFrom.ANY with a ReadFrom.UPSTREAM control.
lsh1215
marked this pull request as ready for review
July 13, 2026 17:10
Collaborator
|
Hey @lsh1215 |
Collaborator
|
Hey again @lsh1215 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3287 — undefined SCAN-family results when
ReadFromspreads reads across nodes on a Master/Replica connection.Problem
SCAN/HSCAN/SSCAN/ZSCANcursors are node-local state: the cursor value is a physical position in that node's hashtable, so a continuation request (cursor != 0) must return to the node that issued it. On Master/Replica, every replica holds a full copy of the keyspace, soReadFromload-balances individual reads across nodes. That is correct for stateless reads, but for the stateful SCAN family it meansMasterReplicaConnectionProviderre-selects a read node per call, and a continuation can land on a different node that interprets the cursor against its own table — yielding undefined results (silently missing/duplicated elements).Measured with a 1000-field hashtable-encoded hash:
Fix
Carry the issuing node on the cursor and pin continuations to it (the opposite direction of the cluster "aggregate" scan — here we stick):
ScanCursorgains an optional, package-privatesource(RedisURI), exposed across packages via a newScanCursorAccessor(following the existingOrderingReadFromAccessorpattern), so the publicScanCursorAPI is unchanged. ImmutableINITIAL/FINISHEDrejectsetSource.RedisCommandBuilderpropagates the source hint from the input cursor to the cursor that carries the response, across all scan builders (incl. streaming). Reusing the returned cursor object — asScanIterator/ScanStreamalready do — carries it automatically.MasterReplicaChannelWriterrecognizes scan commands at the single routing choke point (covers sync/async/reactive): continuations pin to the issuing node; initial requests stamp the selected node onto the returned cursor.MasterReplicaConnectionProvideradds a node-reporting overload andgetPinnedConnectionAsync, which fails fast with a descriptiveRedisExceptionif the issuing node has left the topology, instead of silently re-routing (which would reintroduce undefined results).Read load-balancing for non-scan reads and for initial scan selection is unchanged; the cluster scan path is untouched.
Tests
MULTI/WRITE binding, batch stamping, and provider node-notification / pin / fail.MasterReplicaStickyScanIntegrationTests):{SCAN, HSCAN, SSCAN, ZSCAN} × {sync, async, reactive}underReadFrom.ANYmust return every element, plus aReadFrom.UPSTREAMcontrol, against the standard docker-env master/replica pair.Notes for reviewers (draft)
ScanCursorAccessorapproach over a publicScanCursor.setSourceto avoid widening the public API; happy to switch to public@since 7.7accessors if you prefer.findNodeByUrimatches by host:port; if a failover reassigns the same host:port to a different physical node, the pin would target it — a pre-existing Master/Replica assumption (ConnectionKey), not introduced here.Make sure that:
help-wanted/mre-available).mvn formatter:formattarget. No formatting-only changes are included.Note
Medium Risk
Changes read routing for a core command family on Master/Replica connections; behavior is scoped to scan commands with solid tests, but incorrect pinning would still cause subtle data-iteration bugs.
Overview
Fixes #3287: under Master/Replica with
ReadFromload-balancing,SCAN/HSCAN/SSCAN/ZSCANcontinuations could hit a different replica and return incomplete or duplicate results because cursors are node-local.ScanCursornow carries an optional internalRedisURIsource (viaScanCursorAccessor, public API unchanged).RedisCommandBuildercopies that hint from the request cursor onto the response cursor for all scan command builders (including streaming).MasterReplicaChannelWriterroutes initial scans through read selection while stamping the chosen node on the cursor, and routes continuations throughgetPinnedConnectionAsyncwhen a source is set; batched reads stamp unscanned cursors on the same selected node.MasterReplicaConnectionProvideradds a listener overload for reporting the selected node and fails fast if the pinned node left the topology.Unit tests cover propagation, pinning, transactions, and batches;
MasterReplicaStickyScanIntegrationTestsexercises full iterations across sync/async/reactive underReadFrom.ANY.Reviewed by Cursor Bugbot for commit 7df3592. Bugbot is set up for automated code reviews on this repo. Configure here.