fix!: migrate to the current Copilot metrics API, harden CI, correct docs

[austenstone]

The problem

This action has been broken for every user since GitHub sunset the Copilot usage/metrics endpoints.

I verified this against a live org (octodemo) with a token that has full access, so these are sunsets rather than permission errors:

Endpoint	Status
GET /orgs/{org}/copilot/usage	404
GET /orgs/{org}/copilot/metrics	404
GET /orgs/{org}/team/{team_slug}/copilot/metrics	404

The replacement is a different shape entirely: report endpoints that return signed download links to NDJSON files rather than returning metrics inline.

What this does

Migrates the data layer to the report endpoints. New src/report.ts handles fetch + download + NDJSON parse; new src/types.ts holds types derived from real downloaded payloads rather than from the docs.

Wires up enterprise. It has been declared in action.yml and silently ignored. It now works.

Rebuilds team support. The new API has no team endpoint, so team figures are derived by joining the user report against the user-teams report.

Fixes an async error-handling bug. src/index.ts wrapped an async run() in a synchronous try/catch, so every rejection escaped as an unhandled rejection instead of failing the step. Confirmed with a standalone repro, and confirmed fixed during an end-to-end run.

Fixes a latent 1MiB overflow. In my e2e run the seat table was 318KB of a 327KB summary at ~1500 seats. GitHub caps job summaries at 1MiB, so a ~5000-seat org would have lost the entire report including the 8KB of usage data. Capped at 1000 rows with a notice pointing at the json input.

Moves to node24. Node 20 is EOL as of April 2026 and node20 removal from the runners is estimated for fall 2026, at which point node20 actions fail outright.

Hardens CI. Third-party actions pinned by SHA, timeout-minutes everywhere, least-privilege permissions:, and a new check that fails if the committed dist/ is out of sync with source. I confirmed ncc output is byte-for-byte deterministic before adding that gate so it will not be flaky.

Corrects the README, which documented dead endpoints, wrong token scopes, and was missing three inputs and three outputs.

Verification

Beyond tsc / lint / 13 passing tests, I ran the built dist/index.js end-to-end against the live API with real credentials. It fetched 28 real days, wrote the job summary, and emitted all six outputs. Tests run against a real 460KB NDJSON report captured from the API rather than a hand-written mock.

Some things real data taught me that the docs did not:

• day_totals arrives unsorted, so sorting is required for correctness, not cosmetics.
• There is no totals_by_language or totals_by_model, only the paired variants. The unpaired names return null.
• totals_by_ide contains dirty values in production ("1.133.0:" next to "vscode"), now bucketed to unknown.
• Pie charts contained zero-value slices, now filtered.
• user-teams-28-day/latest does not exist. Only the 1-day variant does.

Breaking

This warrants a v6. The result / csv / xml / json field names follow the new schema (date is now day, total_suggestions_count is now code_generation_activity_count, and so on), and team now requires a token that can read user-level metrics. Migration notes are in the README.

Not verified

I could not exercise the enterprise path end-to-end — my token lacks admin:enterprise and enterprise-28-day returns 403. It is written defensively but untested against a live enterprise.

---

Added after opening

All dependencies to latest; npm audit 25 → 0. Majors: @actions/core 1→3, @actions/github 6→9, @actions/artifact 2→6, eslint 9→10, vitest 3→4, @types/node 22→26, globals 16→17, dotenv 16→17.

@actions/artifact v6 is ESM-only now, which unit tests would not have caught, so I re-ran the built dist/index.js end to end to confirm ncc still bundles it.

I did not take TypeScript 7. It compiles, but breaks both typescript-eslint and ncc's ts-loader, and buys nothing at runtime. Staying on 5.9.

The dependency agent had reached for skipLibCheck: true to silence a @octokit/request-error type error. That was a misdiagnosis: ErrorOptions ships in lib.es2022, and the real cause was a target: ES2020 that predates it. Raising the target fixes it properly without disabling lib checks repo-wide.

Actionable error messages. A run that fails auth reported Bad credentials, which tells nobody which token or scope is wrong. 401/403/404 now name the required scope, call out that GITHUB_TOKEN cannot read Copilot metrics, and specifically detect the disabled Copilot usage metrics policy. Verified against a real 401.

The lint script was checking zero files. eslint src/**/*.ts expands to nothing locally without globstar, so lint passed vacuously. Pointing it at src linted all seven files and immediately surfaced a real rethrow that dropped its cause. Both fixed.

Remaining action pins updated, covering what Dependabot had open: codeql-action v3→v4, dependency-review-action v4.7.1→v5.0.0, harden-runner v2.12.0→v2.21.0, action-send-mail v5→v18 (verified all nine inputs still exist first), plus checkout/upload-artifact/scorecard pins still on v4-era SHAs.

Supersedes

#325, #324, #321, #318, #316, #311, #309, #284, #282, #266, #229, #131 — every one is either upgraded past or made moot here (ts-jest in #311 was removed outright as it was unused).

Why one PR rather than several

The batches are interdependent: the dist-sync gate needs the rebuilt bundle, which needs the dependency upgrade, which needs the ES2022 target. Splitting produces intermediate commits with failing CI. Each commit is individually focused and reviewable.

Needs you

secrets.TOKEN on this repo is expired — the Usage workflow fails with a 401 independent of these changes. That is why the Usage check is red. Rotating it is the one thing I cannot do.

[austenstone-datadog]

⚠️ Warnings

🚦 1 Pipeline job failed

Usage | Copilot Usage

View in Datadog · View in GitHub Actions

Error: The github-token is invalid or expired. Copilot metrics cannot be read with the default GITHUB_TOKEN.

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a80f835 | Docs | View more details | Give us feedback!}

[austenstone]

Post-merge checklist

Three things I can't do from here:

1. Rotate secrets.TOKEN. It's the only red check. The token 401s, which is why the Copilot Usage workflow fails — it has nothing to do with this PR. Worth noting the failure output is the new error message doing its job:

##[error]Error: The github-token is invalid or expired. Copilot metrics cannot be read with the
default GITHUB_TOKEN, so supply a PAT with read:org (organization) or
manage_billing:copilot / read:enterprise (enterprise).

Needs read:org (classic) or View Organization Copilot Metrics (fine-grained).

2. Tag v6 / v6.0.0 after merge, and move the floating major. There is currently no v5 tag at all — v1–v4 exist, v5.0–v5.2 exist, but nothing consumers can pin as @v5. My recommendation is don't backfill v5: it would point at code the API sunset already broke, so it'd hand people a friendlier name for a 404. Ship v6 and let the README's migration section carry the story.

3. The 12 open Dependabot PRs are all superseded by the dependency upgrade in here (#325, #324, #321, #318, #316, #311, #309, #284, #282, #266, #229, #131). Every one targets a package this PR takes past the version it proposes, so they should close themselves on merge. I left them alone rather than spamming close comments first.

Unverifiable with my credentials

• Enterprise paths. My token lacks admin:enterprise, so /enterprises/{enterprise}/copilot/metrics/reports/... returns 403 for me. The code paths are symmetric with the org ones that I did verify against live data, but they're untested against a real enterprise.
• Org-level users-28-day. This is what backs the rebuilt team support. It works with my octodemo access; whether a typical consumer's token clears it is unknown.

[austenstone]

Two more commits after the checklist above, both addressing item 2 from it.

e01656a adds a release workflow — there was no release automation in the repo at all, which is precisely why v5 went missing. On a published vX.Y.Z release it rebuilds dist/, refuses to proceed if the bundle is stale, then moves the floating major tag. Prereleases are skipped so an -rc can't hijack the stable tag. Least-privilege contents: write, SHA-pinned, timeout-minutes, and the tag name goes through env: rather than being interpolated into a run: block. actionlint and CodeQL's Actions analyzer both pass on it.

5fa8592 documents the tag policy in the README, including why @v5 does not resolve, so anyone who tries it gets an answer instead of a broken ref.

This means the only manual step left post-merge is cutting the v6.0.0 release itself. The v6 tag will place itself.