Skip to content

ci: add sandbox-capabilities workflow for GitHub Actions smoke test - #322

Merged
kisernl merged 6 commits into
masterfrom
devin/sandbox-capabilities-workflow-smoketest
Aug 14, 2026
Merged

ci: add sandbox-capabilities workflow for GitHub Actions smoke test#322
kisernl merged 6 commits into
masterfrom
devin/sandbox-capabilities-workflow-smoketest

Conversation

@kisernl

@kisernl kisernl commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Workflow-only PR so the sandbox-capabilities GitHub Actions pipeline can be exercised independently.

  • Contains only .github/workflows/sandbox-capabilities.yml (mirrored from Add sandbox capabilities matrix benchmark and weekly workflow #321, but with the schedule trigger removed so it doesn't run weekly before the benchmark implementation lands).
  • Supports workflow_dispatch with provider, iterations, concurrency, and publish inputs.
  • Renders the provider × feature pass/fail matrix as a GitHub Step Summary even when publish is false.
  • All uses: references are pinned to full commit SHAs, the collect job has no vault access, and user inputs are passed through env: to avoid shell template injection.

The benchmark implementation (benchmarks/sandbox/capabilities.bench.ts, result writer, merge logic, and bench:capabilities script) lives in #321. Merging that PR before the functional scheduled runs is required; this PR is for workflow syntax/matrix/smoke validation.

Link to Devin session: https://app.devin.ai/sessions/f98952f689154db886036dda3dd56bd1
Requested by: @kisernl


Open in Devin Review

Co-Authored-By: Noah Kiser <noah@computesdk.com>
@kisernl kisernl self-assigned this Aug 14, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@open-cla

open-cla Bot commented Aug 14, 2026

Copy link
Copy Markdown

Contributor License Agreement

All contributors are covered by a CLA.

@superagent-security superagent-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superagent found 3 security concern(s).

Comment thread .github/workflows/sandbox-capabilities.yml Outdated
Comment thread .github/workflows/sandbox-capabilities.yml Outdated
Comment thread .github/workflows/sandbox-capabilities.yml Outdated

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 4 potential issues.

Open in Devin Review

Comment thread .github/workflows/sandbox-capabilities.yml
Comment thread .github/workflows/sandbox-capabilities.yml
Comment thread .github/workflows/sandbox-capabilities.yml Outdated
Comment thread .github/workflows/sandbox-capabilities.yml Outdated
devin-ai-integration Bot and others added 5 commits August 14, 2026 19:24
Co-Authored-By: Noah Kiser <noah@computesdk.com>
Co-Authored-By: Noah Kiser <noah@computesdk.com>
Co-Authored-By: Noah Kiser <noah@computesdk.com>
Co-Authored-By: Noah Kiser <noah@computesdk.com>
Co-Authored-By: Noah Kiser <noah@computesdk.com>
@kisernl
kisernl merged commit a952896 into master Aug 14, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant