perf: cache moq-api namespace lookups for a few seconds - #201
Open
thexeos wants to merge 1 commit into
Open
Conversation
`ApiCoordinator::lookup()` issued a fresh HTTP request to moq-api on every call. The relay looks a namespace up once per track it cannot serve locally, so a subscriber asking for several tracks of the same namespace produced that many identical requests, each one on the critical path of a cold subscribe. Put a small TTL cache in front of the lookup. It defaults to 2s and is configurable through `ApiCoordinatorConfig::with_lookup_cache_ttl()` and `--api-lookup-cache-ttl`, where 0 disables it. The TTL is deliberately far below the registration TTL, so a cached origin cannot outlive the registration it came from; it is also the only invalidation, since a stale origin is discovered by connecting, which the coordinator does not observe. Misses are cached for a tenth of that, long enough to collapse a burst but short enough that a namespace registered moments ago is not hidden. Failed requests are not cached, so a transient API error is retried by the next subscriber.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ApiCoordinator::lookup()issued a fresh HTTP request to moq-api on every call, and the relay looks a namespace up once per track it cannot serve locally — so a subscriber asking for several tracks of one namespace produced that many identical requests, each on the critical path of a cold subscribe. This adds a small TTL cache in front of the lookup, defaulting to 2s and configurable viaApiCoordinatorConfig::with_lookup_cache_ttl()/--api-lookup-cache-ttl(0 disables it); the TTL sits far below the registration TTL so a cached origin cannot outlive the registration it came from. Misses are cached for a tenth of that, long enough to collapse a burst but short enough that a namespace registered moments ago is not hidden, and failed requests are not cached at all. The cache uses only the standard library plus existing dependencies, and is covered by unit tests for hits, expiry, the shorter negative TTL, the disable switch, bounded growth, and concurrent use.