Skip to content

Fix: wait only the remaining approval delay, not the full duration - #1173

Open
thc1006 wants to merge 1 commit into
nephio-project:mainfrom
thc1006:fix/approval-delay-remaining
Open

Fix: wait only the remaining approval delay, not the full duration#1173
thc1006 wants to merge 1 commit into
nephio-project:mainfrom
thc1006:fix/approval-delay-remaining

Conversation

@thc1006

@thc1006 thc1006 commented Aug 16, 2026

Copy link
Copy Markdown

Summary

manageDelay returned the full configured approval delay even when the PackageRevision had already existed for part of it, so a revision was requeued for the whole delay again on top of the time it had already waited.

Motivation

The approval reconciler delays approval by approval.nephio.org/delay measured from the revision's creationTimestamp. manageDelay computed whether the delay had elapsed but, when it had not, returned the full delay d as the requeue duration:

if time.Since(pr.CreationTimestamp.Time) > d {
    return 0, nil
}
return d, nil

So a revision created 50 minutes before reconciliation with a 1h delay was requeued for a further 1h — approval ended up delayed by roughly 1h50m instead of 1h. The overshoot grows with how long the revision has already waited.

Changes

controllers/pkg/reconcilers/approval/reconciler.go:

  • Return the time remaining until creationTimestamp + delay, not the full delay, so the total wait matches the configured duration:

    remaining := pr.CreationTimestamp.Time.Add(d).Sub(now)

    Taken from a deadline rather than as d - now.Sub(created). That subtraction overflows: with the largest valid duration and a creation timestamp one nanosecond in the future, d minus a negative elapsed wraps to MinInt64, reads as "already elapsed", and approves immediately. The delay is a user-supplied annotation, so that value is reachable. Time.Sub saturates instead of wrapping, so the same input now yields the maximum duration.

  • Refuse a revision with no creation timestamp rather than treating it as having waited forever.

  • Correct the README, which documented a two minute default and a thirty second floor.

    That text was accurate when it was written, and I got its history wrong in the first version of this description. Tracing it: #226 introduced the delay with a thirty second default and a thirty second floor; #252 raised the default to two minutes and updated the README to match; #329 then removed both, and said so plainly:

    There is now no longer a default delay; it only delays if you specifically give a delay annotation.

    The minimum delay is now 0.

    The README was not updated at that point and has been stale since. This PR aligns the documentation with the post-Faster approvals for the auto-approve controller #329 contract; it does not reintroduce the old policy, because nothing in the project sets this annotation on the revisions it creates and a default would delay every approval in every deployment.

  • Document both policy values. The README still said initial was the only supported one, but #806 added always and the reconciler has handled it since. The change-history above stays in this description; the README carries only the current contract.

  • A delay that has already elapsed (or is exactly met) still returns 0, and a malformed or negative annotation still returns an error — behaviour and the (time.Duration, error) signature are otherwise unchanged, so the caller (RequeueAfter: requeue) needs no change.

  • Split the wall-clock read into manageDelayAt(pr, now) (with manageDelay calling it with time.Now()) so the logic can be unit-tested without depending on the wall clock.

A future creationTimestamp (e.g. clock skew) is well defined and does not panic: the delay is measured from creation, so remaining is simply d plus the skew.

Testing

Added cases: the maximum duration with a future creation timestamp (which fails against the old subtraction), and a missing creation timestamp. Restoring d - now.Sub(created) fails only the overflow case, so it is pinned rather than merely observed.

Both mistakes here are about arithmetic across the whole int64 range rather than about any particular annotation, so the invariants are also pinned by a fuzz target: a requeue is never negative, 0 means the deadline has passed, and a positive result is exactly the time left.

$ go test -fuzz FuzzManageDelay -fuzztime=60s ./reconcilers/approval/
fuzz: elapsed: 1m0s, execs: 1458900 (27973/sec), new interesting: 6 (total: 10)
PASS

The seeds run under plain go test, which is what make unit invokes, so this gates in CI without anyone opting into fuzzing. Each seed was checked against the implementation it is meant to reject:

# with the full-delay return restored
delay_fuzz_test.go:50: requeued 1h0m0s, want 0s remaining (offset=-3600000000000 delay="1h")

# with the overflowing subtraction restored
delay_fuzz_test.go:55: approved early: deadline 2465-01-17 ... is after now 2026-08-19 ...
                       (offset=4611686018427387904 delay="2562047h47m16.854775807s")

TestManageDelay drives manageDelayAt with a fixed now and asserts exact durations:

  • partway through the delay (created 50m ago, 1h) → 10m
  • exactly at the boundary → 0
  • past the delay → 0
  • future creationTimestamp → well defined, no panic
  • no annotation → 0; zero delay (0s) → 0; malformed / negative annotation → error

It also checks that the manageDelay wrapper reads the wall clock, by bracketing the call between two time.Now() reads and requiring the result to lie between deadline.Sub(after) and deadline.Sub(before). That is exact rather than a tolerance, so a suspended runner cannot make it flake and a frozen clock cannot slip through: replacing time.Now() with time.Time{} in the wrapper fails it.

$ go test ./reconcilers/approval/...        # from controllers/pkg
ok  github.com/nephio-project/nephio/controllers/pkg/reconcilers/approval   (manageDelay/manageDelayAt 100%)

$ go test -race ./reconcilers/approval/...
$ golangci-lint run ./reconcilers/approval/...   # v2.8.0, .golangci.json → 0 issues
$ gosec ./reconcilers/approval/...               # no findings

@nephio-prow

nephio-prow Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign liamfallon for approval by writing /assign @liamfallon in a comment. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nephio-prow
nephio-prow Bot requested review from efiacor and johnbelamaric August 16, 2026 07:02
@nephio-prow

nephio-prow Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Hi @thc1006. Thanks for your PR.

I'm waiting for a nephio-project member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@thc1006
thc1006 force-pushed the fix/approval-delay-remaining branch 4 times, most recently from 5fd31d4 to c64f966 Compare August 19, 2026 23:21
manageDelay returned the full configured delay even when the
PackageRevision had already existed for part of it. A revision created
50 minutes before reconciliation with a 1h delay was requeued for a
further 1h, so approval was delayed by roughly 1h50m instead of 1h.

Return the time remaining until creationTimestamp + delay instead of the
full delay, so the total wait matches the configured duration. An
already-elapsed or exactly-met delay still returns 0, and a malformed or
negative annotation still returns an error.

Subtracting the elapsed time from the delay would overflow. With the
largest valid duration and a creation timestamp one nanosecond in the
future, d minus a negative elapsed wraps to MinInt64, which reads as
"already elapsed", so the revision is approved immediately instead of
waiting. The delay annotation is user supplied, so that value is
reachable:

    elapsed   = -1ns
    remaining = MaxInt64 - (-1ns) = MinInt64
    remaining <= 0 -> approve now

Take the difference from a deadline instead. Time.Sub saturates rather
than wrapping, so the same input now yields the maximum duration. A
revision with no creation timestamp is refused rather than treated as
having waited forever.

Split the wall-clock lookup into manageDelayAt(pr, now) so the behaviour
can be unit-tested deterministically, covering the remaining-time,
boundary, expired, zero-delay and future-timestamp cases, plus a bounded
check that manageDelay itself reads the wall clock.

Both mistakes are about arithmetic over the whole int64 range rather
than about any particular annotation, so they are also pinned by a fuzz
target on the invariants: a requeue is never negative, zero means the
deadline has passed, and a positive result is exactly the time left. Its
seeds run under plain go test, which is what make unit invokes, and each
seed was checked against the implementation it is meant to reject. The
full-delay return fails on seed 1 and the overflow on seed 2. A minute
of fuzzing beyond the seeds found nothing further in 1.4 million cases.

The README described a two minute default and a thirty second floor. It
was accurate when written and has been stale since 2023. nephio-project#226 introduced
the delay with a thirty second default and a thirty second floor, nephio-project#252
raised the default to two minutes and updated this file to match, and
nephio-project#329 removed both:

    There is now no longer a default delay; it only delays if you
    specifically give a delay annotation.

    The minimum delay is now 0.

The README was not updated then. It is corrected to the current contract
rather than the behaviour being restored, since nothing in the project
sets this annotation on the revisions it creates and reintroducing a
default would delay every approval everywhere. Both supported policy
values are documented too: the file still claimed initial was the only
one, but nephio-project#806 added always, which the reconciler has handled since.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
@thc1006
thc1006 force-pushed the fix/approval-delay-remaining branch from c64f966 to 7ae4e80 Compare August 20, 2026 09:19
@thc1006
thc1006 marked this pull request as ready for review August 20, 2026 15:14
@nephio-prow
nephio-prow Bot requested a review from liamfallon August 20, 2026 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant