feat(storage): reject uploads whose extension, declared type and bytes disagree

[pyramation]

Summary

html-as-jpg.jpg — the fixture that has been sitting in this repo waiting to be rejected — could still be stored. The streamed lane sniffed the bytes but then relabelled the upload with what it found, so a .jpg full of HTML was accepted as text/html and (for an image column with the default allowlist) blocked only incidentally; the presigned lane never compared the two claims it does have. Both lanes now enforce one rule:

declared MIME ≡ magic bytes ≡ filename extension   (or reject)

Disagreement is refusal, not renaming: an object served back under a type it isn't is the whole attack.

New shared helper checkTypeAgreement (uploads/mime-bytes) decides agreement, with three deliberate softenings so the rule is enforceable rather than merely strict:

• silence is not contradiction — a missing declared type, an unknown extension, or application/octet-stream never rejects on its own. Only two specific and incompatible types do.
• text is a family — leading bytes distinguish text from binary, not CSV from JSON from Markdown; treating text/csv vs text/plain as a mismatch would reject most legitimate text uploads.
• containers are a family — a .docx is a zip, an .m4a is an MP4; CONTAINER_FAMILIES maps each container to the types it legitimately carries.

Violations are structured (EXTENSION_BYTES_MISMATCH / DECLARED_BYTES_MISMATCH / DECLARED_EXTENSION_MISMATCH) so the caller can put the reason in an error message or a rejection_reason.

Where each lane checks

Streamed multipart (graphile-settings/src/upload-resolver.ts) — the server has the bytes, so it checks in transit, before staging:

  const detected = await s3.detectContentType({ readStream, filename });
+ const agreement = checkTypeAgreement({
+   filename,
+   declaredMime: upload.mimetype,
+   detectedMime: detected.magic?.type   // NOT detected.contentType
+ });
+ if (!agreement.ok) { detected.stream.destroy(); throw new Error('UPLOAD_TYPE_MISMATCH: …'); }
  const staged = stagingKey();

Compared against magic.type rather than contentType on purpose: the detector's contentType may have been refined using the very extension under suspicion, so using it would let the filename vouch for itself.

Presigned — the bytes go straight from client to S3, so the check splits in two:

1. At the mutation (plugin.ts), the free half: declared type vs filename extension, no bytes needed. payload.html claiming image/jpeg is refused before a row or a URL exists.
2. At confirmation (new confirm-upload.ts), the bytes: readObjectPrefix does a ranged GET of the leading 4KB — validating a 2GB video costs the same as an icon — and returns a verdict rather than executing it:

confirmUploadedBytes({ s3, key, declaredMime, filename })
  → { outcome: 'uploaded' | 'rejected' | 'expired', … }

uploaded is the gate every process node already fires on, so validating before that transition means nothing downstream is ever handed unvalidated bytes. Absent object → expired (the client walked away) rather than rejected (there is nothing to reject). The transition itself stays with the worker that owns storage:confirm_upload, which calls the generated <files>_confirm_uploaded / _reject_file / _expire_file (constructive-db#2891) — this repo defines the decision, not the state change, and the verdict is therefore testable without a database.

Context: the full lifecycle design is constructive-planning#1488; SSG/site uploads are untouched — they are path-addressed presigned PUTs, and a site publish carrying no filename states nothing for the extension leg to contradict.

Link to Devin session: https://app.devin.ai/sessions/739f6e40a0dc44c4a49b8d84390a2268
Requested by: @pyramation

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring