Share one capacity limit across tenants, lend out what nobody is using, and give it back on a deadline.
import { FairShareCeiling } from 'fair-share-ceiling';
const ceiling = new FairShareCeiling({
ceiling: 100_000, // total capacity held concurrently
fairnessUnit: 'cost', // required, see below
tenants: [
{ id: 'acme', weight: 3 },
{ id: 'globex', weight: 1 },
],
reclaimHorizonMs: 250, // how long a returning tenant can be kept waiting
idleAfterMs: 1_000, // how long quiet counts as gone
startedAt: 0,
});
const result = ceiling.acquire({ tenant: 'acme', cost: 4_200, holdMs: 200, now });
if (result.ok) {
try {
await callModel();
} finally {
ceiling.release(result.lease.id, Date.now());
}
} else {
console.warn(result.reason, result.retryAfterMs, result.explanation);
}Every entry point takes an explicit now. There are no timers and no background sweeps, so a stall can be replayed exactly from a recorded sequence of calls.
The obvious design gives each tenant a bucket sized at its weight times the global limit. It is a few lines and it is wrong in both directions: the ceiling sits half empty whenever a tenant is quiet, and it is oversubscribed the moment two tenants are busy at once.
So you let active tenants borrow the unused share. That part is easy. Reclaim is the part that fails, and it fails quietly.
Concretely, with a ceiling of 100 split evenly between a and b:
t=0 b takes its own 50, then borrows a's idle 50 for a call that runs 1000ms
t=10 a comes back and asks for the 50 it is guaranteed
nothing is preempted, so a waits
t=1000 b finishes, a finally runs
a waited 990ms for capacity it never gave away permanently. Nothing errored. And if you measure fairness the usual way, by averaging each tenant's share over the window, the report says the split was even, because over a long enough window it was. The stall is real and the metric cannot see it.
A loan may not outlive the horizon. acquire requires holdMs, and a request that needs borrowed units is refused outright when holdMs exceeds reclaimHorizonMs. This is the rule that kills the trace above: with a horizon of 100ms, b cannot fund a 1000ms call out of a's share. It can still run that call inside its own guarantee, which is all it was ever entitled to. Refusing costs the borrower nothing it owned, and granting would have moved the cost onto a tenant whose only mistake was going quiet.
hold_exceeds_reclaim_horizon
Tenant "b" asked to hold 50 cost units for 1000ms, which exceeds the reclaim
horizon of 100ms, and the request needs to borrow capacity because only 0 of its
own guarantee is free. Borrowed capacity may not be committed for longer than the
horizon, because nothing here is preempted and a lender that comes back would
otherwise wait for the whole hold. Retry with holdMs of 100 or less, split the work
so it fits inside the tenant's own guarantee, or raise reclaimHorizonMs if the
tenants really can tolerate waiting that long for their share.
Asking counts, even when the answer is no. A refused acquire marks the tenant active. That single fact is the reclaim freeze: while a is waiting, a is not in the lender pool, so b cannot release its borrowed lease and immediately take out a fresh one against the same units. Without it, a borrower looping on acquire wins every time capacity frees up and the returner never gets in at all. This is the difference between the two traces in the test suite: with the demand signal a is served at t=100, without it b renews at t=100 and a is told to come back at t=200.
Quiet is not idle. A tenant becomes lendable only after idleAfterMs of no activity, where activity means any acquire attempt and any release. Set it above the natural gap between a tenant's bursts and a 200ms pause costs nothing. Idleness is measured from the release, not from the acquire, so a tenant in the middle of a long call is never mistaken for one that has left.
The constructor requires idleAfterMs to be strictly greater than reclaimHorizonMs, and refuses the configuration otherwise. The reason is a boundary case: the worst loan a returner can face was taken at the same instant it asked, for the full horizon, so it expires at demandedAt + reclaimHorizonMs. The freeze has to still be in force at that exact moment. If the two values are equal, the borrower releases and re-borrows in the same instant the protection lapses, and the advertised bound stops holding with nothing to show for it.
Together these give a bound rather than a hope: once a tenant asks, no lease holding its units extends past reclaimHorizonMs from that moment, and no new lease against them can be written in between. Provided holders release by the deadline they declared, the tenant is whole within the horizon.
Each borrowed slice is attributed to a specific tenant, not to an anonymous surplus pool. A pool can tell you how much is on loan but not by whom or until when, so the only reclaim estimate it can produce is "eventually", which is the answer that turns a wait into a stall. With per lender attribution the refusal is specific:
guarantee_on_loan retryAfterMs: 90 blockingLeases: ['b2']
Loans are also taken from the largest available lender first rather than spread evenly. Spreading a loan across every idle tenant guarantees that whichever one wakes up next finds part of its guarantee committed. Concentrating it means fewer tenants are exposed at all: in the four tenant test, c funds the entire loan and a and b reclaim instantly.
retryAfterMs is null when a blocking lease has already passed its declared deadline. That is not a shrug, it is the honest answer, because an overrunning holder is the one thing that breaks the bound. Overrunning leases stay counted against the ceiling and are listed in stats().overrunLeases. They are never silently freed: the work behind them is still running, and reissuing that capacity would be worse than the delay it papers over.
stats() reports two numbers that a share average cannot express.
deficitUnitMs integrates the gap between what a tenant was entitled to and what it actually held, over time, while it was short. worstReclaimMs records the longest wait that ended in the tenant getting its share.
In the test that pins this down, the trace ends with a and b each having run two calls of 50 units. Cost share is 0.5 each. Request share is 0.5 each. Skew against configured weight is 1.00. By every averaged measure it is textbook fairness. The deficit integral says a spent 90ms holding nothing while entitled to 50, and b spent none.
Two details keep those numbers honest. A tenant's stated demand is the largest single refused charge outstanding, not the sum, so a client retrying every 10ms cannot inflate its way to a bigger fair share. And the deficit integral is cut at the moment a want lapses rather than running to the present, so a tenant that asked once and left does not accumulate a deficit for the rest of the process lifetime.
fairnessUnit has no default and the constructor refuses to guess.
Max-min fairness over cost and max-min fairness over request count produce different admissible sets, and both are defensible. A tenant sending 200 token calls and a tenant sending 100k token calls split a request ceiling exactly evenly and a cost ceiling 500 to 1. Which is correct depends on what the limit protects: a request ceiling protects a connection pool, a cost ceiling protects a token budget. There is a test that runs one identical trace under both settings, and the same request is admitted under 'requests' and refused under 'cost'.
Whichever you choose, cost is always required and always recorded, and fairnessReport() returns both share vectors side by side with a unitsDisagree flag. An even split in requests is routinely a 500 to 1 split in cost, and a dashboard that only shows the configured unit will call that fair.
plan() returns the weighted max-min allocation for current demand, computed by progressive filling. It is the yardstick, not the admission rule: admission is online and cannot revoke a lease that was legitimate when granted, so the interesting number is how far held has drifted from target during a reclaim.
The bound depends on holders keeping their word. holdMs is a declaration, not a limit. Nothing is preempted, so a holder that runs long blocks the reclaim for as long as it runs. The module detects this, stops quoting a reclaim time, and lists the lease, but it cannot fix it. If your holds are wildly unpredictable, the horizon is aspirational.
Grants are all or nothing. A request for 50 units against 30 free is refused rather than partially filled. Partial fills would need the caller to be able to use less than it asked for, which is not true of a single model call.
Demand has to be re-signalled. A tenant that asks once, is refused, and never asks again has its reservation lapse after idleAfterMs. A client that wants its share must keep asking. The alternative, holding a reservation open indefinitely, leaves the ceiling permanently underused for a tenant that has gone.
Abandoned waits are not counted. When a want lapses, the in progress reclaim is dropped rather than recorded, so worstReclaimMs only covers waits that ended in the tenant actually being served. A tenant that gave up mid stall does not appear in that statistic.
Guarantees are fixed at construction. There is no way to add, remove, or reweight a tenant on a live instance. Reweighting mid flight would change guarantees underneath leases that were granted against the old ones, and there is no correct answer for what happens to a loan whose lender just shrank.
Concurrency is the model, not rate. The ceiling bounds what is held at one time. It does not bound throughput over a window. Pairing it with a token bucket, if you need both, is left to the caller.
Comparisons use a fixed tolerance of 1e-9. Guarantees are ceiling * weight / totalWeight and are almost never exactly representable, so an exact comparison would reject requests that fit by one part in 2^52. If your ceiling is large enough that 1e-9 is a meaningful quantity of capacity, scale your units down.
npm install
npm test # 93 tests: reclaim bounds, lender attribution, overruns, both fairness unitsThe reclaim tests are adversarial on purpose. The borrower gets first move at the instant capacity frees up, loops on acquire, and overruns its deadline, and there is a test that runs the same trace without the demand signal to show the starvation the freeze prevents.
MIT