Enhancement Task
Background
PD currently allows an unbounded number of scatter-range-scheduler instances to be added via pd-ctl scheduler add scatter-range-scheduler <start-key> <end-key>. Each active scatter-range scheduler runs its own periodic scheduling loop and region-fit computation. #3594 already showed that FitRegion cost is significant per scatter-range scheduler.
Problem
There is no upper bound on how many scatter-range schedulers can be created. When the total count grows too large (e.g. via automation or misconfiguration repeatedly adding key-range schedulers), the cumulative scheduling/FitRegion cost can drive PD CPU usage very high, degrading other latency-sensitive PD work (region heartbeat processing, leader election, TSO) and overall service quality.
Proposal
- Add a configurable limit on the total number of concurrently active scatter-range schedulers.
- When
scheduler add scatter-range-scheduler is invoked and the current count has already reached the limit, reject the creation with a clear error instead of silently accepting it.
- Expose the current count and limit (via
scheduler config output and/or metrics) so operators can observe usage before hitting the ceiling.
Alternatives considered
- Leave enforcement to operators manually tracking scheduler counts — no safety net, easy to overshoot.
- Throttle/rate-limit scatter-range scheduler execution instead of capping count — reduces CPU spikes but doesn't bound per-scheduler bookkeeping overhead, and is a larger change than a simple count check.
Enhancement Task
Background
PD currently allows an unbounded number of
scatter-range-schedulerinstances to be added viapd-ctl scheduler add scatter-range-scheduler <start-key> <end-key>. Each active scatter-range scheduler runs its own periodic scheduling loop and region-fit computation. #3594 already showed thatFitRegioncost is significant per scatter-range scheduler.Problem
There is no upper bound on how many scatter-range schedulers can be created. When the total count grows too large (e.g. via automation or misconfiguration repeatedly adding key-range schedulers), the cumulative scheduling/FitRegion cost can drive PD CPU usage very high, degrading other latency-sensitive PD work (region heartbeat processing, leader election, TSO) and overall service quality.
Proposal
scheduler add scatter-range-scheduleris invoked and the current count has already reached the limit, reject the creation with a clear error instead of silently accepting it.scheduler configoutput and/or metrics) so operators can observe usage before hitting the ceiling.Alternatives considered