block: skip the blkcg walk in blk_cgroup_congested() when nothing is throttled - #1107
Open
blktests-ci[bot] wants to merge 3 commits into
Open
block: skip the blkcg walk in blk_cgroup_congested() when nothing is throttled#1107blktests-ci[bot] wants to merge 3 commits into
blktests-ci[bot] wants to merge 3 commits into
Conversation
Author
|
Upstream branch: 0d83957 |
blktests-ci
Bot
force-pushed
the
linus-master_base
branch
2 times, most recently
from
August 9, 2026 07:24
612ae31 to
863d43a
Compare
Author
|
Upstream branch: 06cf618 |
blktests-ci
Bot
force-pushed
the
series/1141695=>linus-master
branch
from
August 9, 2026 07:41
2a63076 to
67ec94b
Compare
blktests-ci
Bot
force-pushed
the
linus-master_base
branch
from
August 10, 2026 22:28
863d43a to
29ac21d
Compare
Author
|
Upstream branch: d58772d |
added 3 commits
August 10, 2026 22:48
io.latency can throttle a group which has no latency target of its own. When a sibling misses its target, check_scale_change() scales down its peers, and a peer that reaches queue depth one gets blkcg_use_delay() called on it on every further scale-down, even with min_lat_nsec == 0. iolatency_pd_offline() resets the target through iolatency_set_min_lat_nsec(), which clears the delay only on a nonzero to zero transition, so it never clears such a peer. Freeing the policy data then leaves blkg->use_delay set and blkcg->congestion_count elevated with nothing left that can drop it. blk_cgroup_congested() then returns true for every task in that cgroup and its descendants for as long as the cgroup lives: page_cache_sync_ra() cuts readahead to a single page, page_cache_async_ra() skips it altogether, and __folio_throttle_swaprate() takes swap_avail_lock and schedules a throttle on anonymous folio allocation. Clear the delay in iolatency_pd_free(). By then bio-held blkg references have drained, or the queue is frozen for policy deactivation, so check_scale_change() cannot re-arm it. The free callback can also see policy data which was never attached to a blkg, hence the pd->blkg check. Fixes: d706751 ("block: introduce blk-iolatency io controller") Signed-off-by: Usama Arif <usama.arif@linux.dev>
iocg_kick_delay() turns sufficiently large debt into an explicit block-cgroup delay with blkcg_set_delay(), setting blkg->use_delay to -1 and incrementing blkcg->congestion_count. Clearing it again depends on iocg_kick_delay() running from the period timer, the waitq timer or the issue path. ioc_pd_free() removes the iocg from active_iocgs and cancels its waitq timer, and no further bios can arrive, so once it has run nothing is left which can reduce the debt and clear the delay. The blkcg stays marked congested for the rest of its life. blk_cgroup_congested() then returns true for every task in that cgroup and its descendants: page_cache_sync_ra() cuts readahead to a single page, page_cache_async_ra() skips it altogether, and __folio_throttle_swaprate() takes swap_avail_lock and schedules a throttle on anonymous folio allocation. Clear it explicitly, after the list removal and the synchronous hrtimer_cancel() so that neither timer processing nor an I/O path can re-arm it. The free callback can also see policy data which was never attached to a blkg, hence the pd->blkg check. Fixes: 7caa471 ("blkcg: implement blk-iocost") Signed-off-by: Usama Arif <usama.arif@linux.dev>
blk_cgroup_congested() walks the current task's blkcg ancestor chain on every readahead decision and, once swap is in use, on every anonymous and shmem folio allocation. The answer is almost always "no", but finding that out costs two loads per level on two cold cache lines, plus an out-of-line kthread_blkcg() and an RCU read-side pair. On a fleet profile of hosts running containers with 5-10 level hierarchies it costs about as much as all of mutex_lock(), 99.4% of it under __folio_throttle_swaprate(). Gate the walk on a global count of blkcgs with a non-zero congestion_count. The counter only moves on the 0 <-> 1 transitions of each blkcg's congestion_count, so the extra atomic stays in the throttle arm/disarm paths and never appears in steady state. When something is throttled the counter is non-zero and the walk runs as before. Signed-off-by: Usama Arif <usama.arif@linux.dev>
blktests-ci
Bot
force-pushed
the
series/1141695=>linus-master
branch
from
August 10, 2026 22:48
67ec94b to
2e8ea84
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull request for series with
subject: block: skip the blkcg walk in blk_cgroup_congested() when nothing is throttled
version: 1
url: https://patchwork.kernel.org/project/linux-block/list/?series=1141695