Skip to content

fix(automation): run securely pinned review repair every hour - #704

Closed
seonghobae wants to merge 4 commits into
mainfrom
codex/hourly-review-fix-loop
Closed

fix(automation): run securely pinned review repair every hour#704
seonghobae wants to merge 4 commits into
mainfrom
codex/hourly-review-fix-loop

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Problem

The merge scheduler already revisits merge readiness every 15–30 minutes, but unresolved review-feedback autofix ran only every two hours and would not revisit the same current head for 24 hours. Review-addressable pull requests could therefore remain idle long after feedback appeared.

The reusable workflow also checked out scheduler code from mutable main, even when callers referenced the workflow at an immutable SHA. In a reusable workflow, the ordinary github context belongs to the caller, so the privileged implementation must be bound to the called workflow's own resolved source.

Change

  • run the bounded review-fix scheduler hourly at minute 23, away from the top-of-hour GitHub Actions load spike;
  • reduce the same-head retry floor from 24 hours to one hour;
  • check out job.workflow_repository at immutable job.workflow_sha;
  • retain canonical_ref only as an ignored deprecated compatibility input so existing pinned callers can upgrade without a coordinated break;
  • retain the existing one-dispatch-per-run default, current-head validation, concurrency control, least-privilege permissions, and fail-closed autofix workflow;
  • pin the cadence, retry bounds, immutable source, compatibility boundary, and permissions with focused contract tests.

Safety

The scheduler does not bypass reviews or checks. It only dispatches the existing bounded autofix workflow; merge eligibility continues to be decided by the independent merge scheduler and repository rules. The deprecated input is never read and cannot select trusted code.

Verification

  • focused cadence contract added before the workflow change;
  • exact cron, retry default, environment fallback, and one-dispatch bound are asserted;
  • reusable workflow source must resolve through job.workflow_repository and job.workflow_sha;
  • caller payloads, deprecated inputs, mutable refs, and the ordinary caller GitHub SHA cannot select privileged source;
  • current-head GitHub Actions are the authoritative workflow validation.

Summary by CodeRabbit

  • 개선 사항

    • PR 리뷰 수정 작업의 재배포 최소 대기 시간이 24시간에서 1시간으로 단축되었습니다.
    • 스케줄 실행 주기가 2시간마다에서 매시 23분으로 변경되었습니다.
    • 호출된 워크플로의 확인된 버전을 사용해 실행 안정성과 보안이 강화되었습니다.
    • 기존 canonical_ref 입력과의 호환성이 유지됩니다.
  • 테스트

    • 스케줄, 재시도 설정, 소스 검증, 권한 및 실행 제한에 대한 자동 검증을 추가했습니다.

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 27 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bbfe0594-534a-4715-8fcd-e37b640b5f46

📥 Commits

Reviewing files that changed from the base of the PR and between 1296408 and e01e52b.

📒 Files selected for processing (4)
  • .github/workflows/pr-review-fix-scheduler.yml
  • requirements-strix-ci.txt
  • tests/test_hourly_review_fix_workflow.py
  • tests/test_pr_review_fix_scheduler_source_pin.py
📝 Walkthrough

Walkthrough

재시도 최소 대기 시간을 24시간에서 1시간으로 변경했습니다. 스케줄을 매시 23분으로 변경했습니다. 체크아웃은 호출된 워크플로의 저장소와 확인된 SHA를 사용합니다. 관련 동작을 테스트로 검증합니다.

Changes

스케줄러 동작 및 소스 제어

Layer / File(s) Summary
시간별 스케줄 및 재시도 계약
.github/workflows/pr-review-fix-scheduler.yml, tests/test_hourly_review_fix_workflow.py
스케줄을 매시 23분으로 변경했습니다. retry_hoursRETRY_HOURS 기본값을 1시간으로 변경했습니다. 이벤트 페이로드와 입력 대체식을 테스트합니다.
호출된 워크플로 소스 고정
.github/workflows/pr-review-fix-scheduler.yml, tests/test_pr_review_fix_scheduler_source_pin.py
체크아웃에 job.workflow_repositoryjob.workflow_sha를 사용합니다. canonical_ref는 빈 기본값의 호환성 입력으로 유지하고 사용하지 않습니다. 권한 및 실행 제한 계약을 테스트합니다.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 리뷰 수정 스케줄러의 시간별 실행과 안전한 소스 고정이라는 주요 변경 사항을 명확하고 간결하게 설명합니다.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/hourly-review-fix-loop

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review August 3, 2026 03:06
@seonghobae seonghobae changed the title test(automation): pin hourly review-fix cadence fix(automation): run review-feedback repair every hour Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@seonghobae
seonghobae enabled auto-merge (squash) August 3, 2026 03:10
@seonghobae seonghobae changed the title fix(automation): run review-feedback repair every hour fix(automation): run securely pinned review repair every hour Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

Source-pin design independently verified against GitHub's current Actions context reference: for jobs defined by a reusable workflow, job.workflow_repository identifies the called workflow repository and job.workflow_sha identifies its immutable commit; GitHub's own example checks out co-located source with exactly those fields. This is stronger than caller-supplied canonical_ref, so competing #710 was closed as superseded. Reference: https://docs.github.com/en/actions/reference/workflows-and-actions/contexts#job-context

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review the exact current head of the hourly review-repair scheduler. Focus on immutable called-workflow source binding, one-dispatch bounds, same-head retry timing, least-privilege permissions, and fail-closed behavior.

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

@seonghobae I will review the exact current head. I will focus on immutable called-workflow source binding, dispatch bounds, retry timing, permissions, and fail-closed behavior.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 12964088023f48cf6a90089e2439226b1402b5e6.

  • Head SHA: 12964088023f48cf6a90089e2439226b1402b5e6

  • Workflow run: 30841879924

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: pr-review-fix-scheduler.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: pr-review-fix-scheduler.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Test (2 files)"]
  S2 --> I2["regression suite"]
  I2 --> R2["Review risk: Test (2 files)"]
  R2 --> V2["targeted test run"]
Loading

@opencode-agent

opencode-agent Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

  • Head SHA: e01e52b32d237f16ce614b1760469f9c6f64a3c4
  • Workflow run: 30900327985
  • Workflow attempt: 1
  • Gate result: REQUEST_CHANGES (approval step)

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head e01e52b32d237f16ce614b1760469f9c6f64a3c4.

  • Head SHA: e01e52b32d237f16ce614b1760469f9c6f64a3c4

  • Workflow run: 30900327985

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: pr-review-fix-scheduler.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: pr-review-fix-scheduler.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file: requirements-strix-ci.txt"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file: requirements-strix-ci.txt"]
  R2 --> V2["required checks"]
  Evidence --> S3["Test (2 files)"]
  S3 --> I3["regression suite"]
  I3 --> R3["Review risk: Test (2 files)"]
  R3 --> V3["targeted test run"]
Loading

@opencode-agent
opencode-agent Bot disabled auto-merge August 3, 2026 18:37
@seonghobae
seonghobae enabled auto-merge (squash) August 4, 2026 03:21
@opencode-agent
opencode-agent Bot disabled auto-merge August 4, 2026 03:22
@seonghobae
seonghobae enabled auto-merge (squash) August 4, 2026 06:00
@seonghobae
seonghobae dismissed opencode-agent[bot]’s stale review August 4, 2026 06:54

Superseded by current head df2eaf9. The review concerns failed coverage evidence on 1296408; current-head checks and a fresh independent review remain required before merge.

@opencode-agent
opencode-agent Bot disabled auto-merge August 4, 2026 06:59
@seonghobae
seonghobae force-pushed the codex/hourly-review-fix-loop branch from df2eaf9 to ab5b5f5 Compare August 4, 2026 08:41
@seonghobae
seonghobae enabled auto-merge (squash) August 4, 2026 08:43

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

1 similar comment

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head e01e52b32d237f16ce614b1760469f9c6f64a3c4.

  • Head SHA: e01e52b32d237f16ce614b1760469f9c6f64a3c4

  • Workflow run: 30900327985

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: pr-review-fix-scheduler.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: pr-review-fix-scheduler.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file: requirements-strix-ci.txt"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file: requirements-strix-ci.txt"]
  R2 --> V2["required checks"]
  Evidence --> S3["Test (2 files)"]
  S3 --> I3["regression suite"]
  I3 --> R3["Review risk: Test (2 files)"]
  R3 --> V3["targeted test run"]
Loading

@opencode-agent
opencode-agent Bot disabled auto-merge August 4, 2026 10:26

Copy link
Copy Markdown
Contributor Author

Closing this 10-commit pre-baseline branch to reduce source-repository workflow pressure. Its hourly immutable-source review-repair contract is first in the consolidated post-#731 replay queue and should be rebuilt immediately after #731 lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant