Skip to content
This repository was archived by the owner on Sep 5, 2025. It is now read-only.

feat: migrate to gobenchdata for continuous benchmarking with PR comments only#20

Closed
icedream with Copilot wants to merge 5 commits into
mainfrom
copilot/fix-44997c8a-f70c-4095-ba9f-7795eb136ce3
Closed

feat: migrate to gobenchdata for continuous benchmarking with PR comments only#20
icedream with Copilot wants to merge 5 commits into
mainfrom
copilot/fix-44997c8a-f70c-4095-ba9f-7795eb136ce3

Conversation

Copilot AI commented Aug 14, 2025

Copy link
Copy Markdown
Contributor

This PR migrates the repository from a custom benchmark comparison workflow to use the gobenchdata action for continuous benchmarking, configured to publish results only as PR comments rather than to GitHub Pages.

Changes Made

🔄 Workflow Migration

  • Replaced the existing benchmark.yml workflow with a gobenchdata-based implementation
  • Added benchmark-publish.yml to establish baseline benchmark data on main branch pushes
  • Configured gobenchdata to use checks mode for PR comments only (no GitHub Pages publishing during PR reviews)

🛠️ Comment Management

  • Migrated from custom GitHub script to peter-evans actions:
    • Uses peter-evans/find-comment@v3 for finding existing benchmark comments
    • Uses peter-evans/create-or-update-comment@v4 for creating/updating PR comments
  • Improved comment formatting with structured results and clear explanations

⚙️ Configuration

  • Added gobenchdata-checks.yml with performance regression thresholds:
    • Performance (ns/op): 20% maximum regression allowed
    • Memory allocation (B/op): 25% maximum increase allowed
    • Allocation count (allocs/op): 15% maximum increase allowed
  • Configured benchmark data retention (30 runs) and proper error handling

📚 Documentation

  • Updated README.md to document the new continuous benchmarking system
  • Replaced references to the old benchstat-based approach with gobenchdata information

💯 Conventional Commits

  • Used conventional commit format throughout the implementation
  • Configured gobenchdata to use conventional commits for benchmark data updates

Benefits

  • More robust benchmark comparison with configurable performance thresholds
  • Better error handling for scenarios with missing baseline data
  • Cleaner PR comments with structured, easy-to-understand results
  • Historical tracking of benchmark performance over time (via baseline publishing to gh-pages)
  • Reduced maintenance overhead compared to the custom solution
  • PR-only workflow that keeps benchmark comparisons in PR discussions without cluttering GitHub Pages

Testing

  • ✅ Verified gobenchdata configuration works correctly with CLI testing
  • ✅ Confirmed benchmark field names match actual JSON structure
  • ✅ Tested that existing build and test workflows remain unaffected
  • ✅ Added graceful handling for first-run scenarios without baseline data
  • ✅ Removed GitHub Pages publishing from PR workflow per user request

The new workflow establishes baseline data when changes are merged to main, and subsequent PRs show performance comparisons in PR comments only without publishing to GitHub Pages during the review process.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

@icedream icedream changed the title [WIP] Use continuous benchmarking action for Go Use action https://github.com/marketplace/actions/continuous-benchmarking-for-go instead of our current benchmark PR comment solution. However, make it post its results to a comment on a PR (and ideally keep t... Use continuous benchmarking action for Go Aug 14, 2025
Copilot AI and others added 3 commits August 14, 2025 12:30
- Replace custom benchmark workflow with gobenchdata action
- Add gobenchdata-checks.yml configuration for performance monitoring
- Use peter-evans actions for PR comment management
- Update README.md to document new continuous benchmarking system
- Configure thresholds for ns/op (20%), B/op (25%), and allocs/op (15%)

Co-authored-by: icedream <807772+icedream@users.noreply.github.com>
- Use Mem.BytesPerOp and Mem.AllocsPerOp to match actual benchmark JSON structure
- Tested with gobenchdata CLI to verify checks work correctly

Co-authored-by: icedream <807772+icedream@users.noreply.github.com>
- Add benchmark-publish.yml workflow to create baseline data on main branch pushes
- Configure workflow to publish to gh-pages with conventional commit messages
- Add continue-on-error to benchmark checks to handle missing baseline gracefully
- Use PRUNE_COUNT to limit stored benchmark history to 30 runs

Co-authored-by: icedream <807772+icedream@users.noreply.github.com>
Copilot AI changed the title Use continuous benchmarking action for Go feat: migrate to gobenchdata for continuous benchmarking Aug 14, 2025
Copilot AI requested a review from icedream August 14, 2025 12:36
@icedream

Copy link
Copy Markdown
Owner

@copilot Can you make it so using actions peter-evans/find-comment@v3 and peter-evans/create-or-update-comment@v4 we do not publish benchmark comparisons to GitHub Pages but instead to a comment on a PR again?

Co-authored-by: icedream <807772+icedream@users.noreply.github.com>

Copilot AI commented Aug 14, 2025

Copy link
Copy Markdown
Contributor Author

@copilot Can you make it so using actions peter-evans/find-comment@v3 and peter-evans/create-or-update-comment@v4 we do not publish benchmark comparisons to GitHub Pages but instead to a comment on a ...

Done! I've updated the benchmark workflow to remove GitHub Pages publishing from the PR workflow. Now the benchmark.yml workflow only creates PR comments using peter-evans actions and doesn't publish to GitHub Pages.

The baseline benchmark data will still be published to the gh-pages branch when changes are merged to main (via benchmark-publish.yml), which provides the historical data needed for comparisons. Commit: 80153db

Copilot AI changed the title feat: migrate to gobenchdata for continuous benchmarking feat: migrate to gobenchdata for continuous benchmarking with PR comments only Aug 14, 2025
@icedream icedream closed this Sep 5, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants