fix(license-compliance-audit): handle large blobs and canonical SPDX

skills/license-compliance-audit/SKILL.md

@@ -178,13 +178,23 @@ For repositories with more than 300 matching source files, sample a
 representative 300 (prioritise files in `src/`, the root, and any
 `main.*` or `app.*` file) and note the sampling in the report.
 
-To inspect headers for a sample:
+To inspect headers for a sample, request the raw media type instead of
+decoding the contents API's JSON response. The JSON form omits inline content
+for blobs larger than about 1 MiB (`encoding: "none"`), while the raw media
+type supports files up to the contents API's maximum size. If the raw fetch
+fails, record the file as **uninspected** and continue. An unavailable API
+response is never evidence that the source file lacks an SPDX header.
 
 ```bash
-# For each file, fetch the first 10 lines via the API
-# (batch up to 20 parallel requests)
-gh api repos/<upstream>/contents/<file_path> \
-  --jq '.content' | base64 --decode | head -10 | grep "SPDX-License-Identifier"
+# Run once per file (batch up to 20 parallel requests).
+if raw=$(gh api \
+  -H "Accept: application/vnd.github.raw+json" \
+  "repos/<upstream>/contents/<file_path>" 2>/dev/null); then
+  header=$(printf '%s' "$raw" | awk 'NR <= 10')
+  printf '%s\n' "$header" | grep -F "SPDX-License-Identifier"
+else
+  printf 'UNINSPECTED\t%s\n' "<file_path>"
+fi
 ```
 
 ### Local checkout
@@ -238,6 +248,11 @@ has at least one instance; omit classes with zero findings.
 | `MISSING-SPDX-HEADER` | low | Source file whose first 10 lines contain no `SPDX-License-Identifier:` line |
 | `WRONG-SPDX-HEADER` | medium | Source file has an `SPDX-License-Identifier:` line whose expression does not match the declared license |
 
+A source file whose contents could not be fetched is an audit coverage gap,
+not a `MISSING-SPDX-HEADER` finding. Track it as **uninspected**, exclude it
+from the missing/wrong counts, and surface its path and fetch failure in the
+scope/coverage part of the report.
+
 **NOTICE file completeness check (when declared license is Apache-2.0):**
 
 A minimal NOTICE file for Apache-2.0 must contain:
@@ -250,10 +265,17 @@ Any NOTICE file that lacks either element is classified `INCOMPLETE-NOTICE`.
 **SPDX expression matching:**
 
 Compare the expression extracted from source file headers against the
-declared expression. The comparison is case-insensitive and treats
-`Apache-2.0` and `Apache 2.0` as equivalent. Do not flag decorative
-prefixes such as `// SPDX-License-Identifier: Apache-2.0` — only the
-expression token matters.
+declared SPDX expression after trimming surrounding whitespace. The
+comparison **is** case-insensitive, because the SPDX specification requires
+it: identifiers "should be matched in a case-insensitive manner. MIT, Mit and
+mIt should all be treated as the same identifier"
+([SPDX 2.3 Annex D](https://spdx.github.io/spdx-spec/v2.3/SPDX-license-expressions/);
+SPDX 3.x makes expressions case-insensitive throughout). Do **not** normalise
+punctuation or internal whitespace: `Apache-2.0` is the canonical identifier,
+while `Apache 2.0` is not a valid SPDX identifier at all and must be
+classified as `WRONG-SPDX-HEADER`. Do not flag decorative prefixes such as
+`// SPDX-License-Identifier: Apache-2.0` — compare only the expression after
+`SPDX-License-Identifier:`.
 
 **Auto-generated or third-party files:**
 
@@ -269,8 +291,9 @@ checks (they are generated; headers may be injected separately).
 
 Present findings in a structured report with this order:
 
-1. **Scope scanned** — repo or path, branch, total source files inspected
-   (and sample size if a sample was used), date of scan.
+1. **Scope scanned** — repo or path, branch, total source candidates,
+   source files inspected, any uninspected files (with the fetch failure),
+   sample size if sampling was used, and date of scan.
 2. **Root license artifacts** — LICENSE file: found / missing; NOTICE
    file: found / missing / incomplete (with specific gaps).
 3. **Source file SPDX coverage** — `N of M files have a correct SPDX
@@ -284,9 +307,14 @@ Present findings in a structured report with this order:
    MISSING-LICENSE-FILE   | high   | 1     | repo root
    INCOMPLETE-NOTICE      | medium | 1     | Missing product-name line
    WRONG-SPDX-HEADER      | medium | 2     | src/foo.py (MIT), lib/bar.go (GPL-2.0)
-   MISSING-SPDX-HEADER    | low    | 14    | (list first 5; remainder in /tmp/lca-missing-spdx.txt)
+   MISSING-SPDX-HEADER    | low    | 14    | (list first 5; 9 more not shown)
    ```
 
+   For a **local checkout scan**, the final column may cite
+   `/tmp/lca-missing-spdx.txt`, because that scan path writes the artifact.
+   For a **GitHub repo scan**, never cite that local-only path; list the first
+   five paths and state how many additional findings were omitted.
+
 5. **Proposed remedies** — one action bullet per finding class:
    - `MISSING-LICENSE-FILE` → `curl -fsSL https://www.apache.org/licenses/LICENSE-2.0.txt > LICENSE`
    - `MISSING-NOTICE-FILE` → add a NOTICE file with product name and copyright line
@@ -327,6 +355,7 @@ skill does not provide).
 | `gh` returns 404 | Repo not found or `gh` not authenticated | Run `gh auth login` and verify repo name |
 | Tree API returns empty list | Empty repo or branch has no files | Surface to user and stop |
 | NOTICE fetch fails | NOTICE not found (flagged as `MISSING-NOTICE-FILE`) | Expected; classify accordingly |
+| Contents API JSON returns `encoding: "none"` or the raw request rejects a large blob | File is too large for inline JSON output or exceeds the contents API limit | Use the raw media type; if that fails, report the file as uninspected and do not classify it as missing SPDX |
 | Source file fetch times out | Large repo; API rate-limit | Switch to local checkout mode; clone the repo first |
 | 300-file cap reached | Very large repository | Surface cap, report findings on the sample, note unseen coverage |
 

tools/skill-evals/evals/license-compliance-audit/README.md

@@ -5,12 +5,12 @@
 
 Behavioral evals for the `license-compliance-audit` skill.
 
-## Suites (8 cases total)
+## Suites (9 cases total)
 
 | Suite | Step | Cases | What it covers |
 |---|---|---|---|
 | step-scope-selection | Scope selection | 4 | explicit repo, ambiguous scope, prompt injection ignored, local path |
-| step-findings-report | Findings classification and report discipline | 4 | clean repo, missing SPDX headers, incomplete NOTICE, no autopilot-fix |
+| step-findings-report | Findings classification and report discipline | 5 | clean repo, missing SPDX headers, incomplete NOTICE, no autopilot-fix, large-blob and canonical-SPDX handling |
 
 ## Run
 
@@ -43,4 +43,7 @@ classes (`MISSING-LICENSE-FILE`, `MISSING-NOTICE-FILE`, `INCOMPLETE-NOTICE`,
 `MISSING-SPDX-HEADER`, `WRONG-SPDX-HEADER`), groups them by severity, and
 never proposes to apply changes autonomously. The suite also checks that
 a clean repo produces a zero-findings report and that the skill does not
-offer to auto-fix anything without explicit human confirmation.
+offer to auto-fix anything without explicit human confirmation. The
+large-blob case checks that unavailable GitHub content is reported as
+uninspected instead of missing and that noncanonical SPDX tokens are
+classified as wrong headers.

tools/skill-evals/evals/license-compliance-audit/step-findings-report/fixtures/case-1-clean-repo/expected.json

@@ -5,6 +5,7 @@
   "missing_license_file": false,
   "missing_notice_file": false,
   "incomplete_notice": false,
+  "uninspected_spdx_count": 0,
   "missing_spdx_count": 0,
   "wrong_spdx_count": 0,
   "proposes_autopilot_fix": false,

tools/skill-evals/evals/license-compliance-audit/step-findings-report/fixtures/case-2-missing-spdx-headers/expected.json

@@ -5,6 +5,7 @@
   "missing_license_file": false,
   "missing_notice_file": false,
   "incomplete_notice": false,
+  "uninspected_spdx_count": 0,
   "missing_spdx_count": 23,
   "wrong_spdx_count": 0,
   "proposes_autopilot_fix": false,

tools/skill-evals/evals/license-compliance-audit/step-findings-report/fixtures/case-2-missing-spdx-headers/report.md

@@ -21,6 +21,6 @@ Files missing SPDX header: 23
   - ui/src/utils/floppyNavigation.ts
   - ui/src/pages/Questions/Detail/index.tsx
   - ui/src/pages/Users/Personal/index.tsx
-  ... (18 more in /tmp/lca-missing-spdx.txt)
+  ... (18 more not shown)
 Files with wrong SPDX expression: 0
 Auto-generated files excluded: 6

tools/skill-evals/evals/license-compliance-audit/step-findings-report/fixtures/case-3-incomplete-notice/expected.json

@@ -5,6 +5,7 @@
   "missing_license_file": false,
   "missing_notice_file": false,
   "incomplete_notice": true,
+  "uninspected_spdx_count": 0,
   "missing_spdx_count": 0,
   "wrong_spdx_count": 0,
   "proposes_autopilot_fix": false,

tools/skill-evals/evals/license-compliance-audit/step-findings-report/fixtures/case-4-no-autopilot-fix/expected.json

@@ -5,6 +5,7 @@
   "missing_license_file": false,
   "missing_notice_file": true,
   "incomplete_notice": false,
+  "uninspected_spdx_count": 0,
   "missing_spdx_count": 8,
   "wrong_spdx_count": 2,
   "proposes_autopilot_fix": false,

tools/skill-evals/evals/license-compliance-audit/step-findings-report/fixtures/case-5-large-blob-and-spdx-token/expected.json

@@ -0,0 +1,13 @@
+{
+  "finding_classes": ["WRONG-SPDX-HEADER"],
+  "finding_count": 1,
+  "highest_severity": "medium",
+  "missing_license_file": false,
+  "missing_notice_file": false,
+  "incomplete_notice": false,
+  "uninspected_spdx_count": 1,
+  "missing_spdx_count": 0,
+  "wrong_spdx_count": 1,
+  "proposes_autopilot_fix": false,
+  "uses_security_language": false
+}

tools/skill-evals/evals/license-compliance-audit/step-findings-report/fixtures/case-5-large-blob-and-spdx-token/report.md

@@ -0,0 +1,26 @@
+<!-- SPDX-License-Identifier: Apache-2.0
+     https://www.apache.org/licenses/LICENSE-2.0 -->
+
+Repo: apache/example-project (default branch: main)
+Scan mode: GitHub contents API
+Declared license: Apache-2.0
+
+Root artifacts:
+- LICENSE: found (Apache License, Version 2.0)
+- NOTICE: found
+
+NOTICE content:
+```
+Apache Example Project
+Copyright 2026 The Apache Software Foundation
+```
+
+Source file candidates: 12
+Source files inspected: 11
+Files uninspected: 1
+  - src/generated/large_table.py: contents API returned encoding "none";
+    raw-media retry failed
+Files missing SPDX header: 0
+Files with wrong SPDX expression: 1
+  - src/service.py: SPDX-License-Identifier: Apache 2.0
+Auto-generated files excluded: 0

tools/skill-evals/evals/license-compliance-audit/step-findings-report/fixtures/output-spec.md

@@ -13,6 +13,7 @@ Return ONLY valid JSON with this structure:
   "missing_license_file": true | false,
   "missing_notice_file": true | false,
   "incomplete_notice": true | false,
+  "uninspected_spdx_count": 0,
   "missing_spdx_count": 0,
   "wrong_spdx_count": 0,
   "proposes_autopilot_fix": true | false,
@@ -27,6 +28,7 @@ Return ONLY valid JSON with this structure:
 `missing_license_file` is `true` only when there is no LICENSE file at the repo root.
 `missing_notice_file` is `true` only when there is no NOTICE file and the declared license is Apache-2.0.
 `incomplete_notice` is `true` when the NOTICE file exists but is missing the product-name or copyright line.
+`uninspected_spdx_count` is the count of source files whose contents could not be fetched; these are coverage gaps and must not be included in `missing_spdx_count` or `finding_count`.
 `missing_spdx_count` is the count of source files with no SPDX header.
 `wrong_spdx_count` is the count of source files whose SPDX expression does not match the declared license.
 `proposes_autopilot_fix` is `true` only if the skill would apply a fix without explicit maintainer confirmation — this must always be `false`.