Skip to content

feat(audit): consume signed events, classify integrity_status (PR2) - #5

Merged
man4ish merged 1 commit into
mainfrom
feature/audit-event-signing-pr2-consumer
Aug 14, 2026
Merged

feat(audit): consume signed events, classify integrity_status (PR2)#5
man4ish merged 1 commit into
mainfrom
feature/audit-event-signing-pr2-consumer

Conversation

@man4ish

@man4ish man4ish commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

PR2 of the audit:events integrity remediation (PR1 #4, merged — the sign/verify helper, previously unwired anywhere). Wires classification into the worker without requiring any producer to sign yet.

What changes

  • AuditConfig.EVENT_SIGNING_SECRET — reuses the existing JWT_SECRET env var and "change-me" fallback, same convention audit/jwt_verify.py already uses. No new secret mechanism. audit/jwt_verify.py itself is untouched.
  • consumers/processor.py::classify_event_integrity(service, signature, data, secret) -> str — new, separate function. parse_audit_event()'s signature and its 3 existing callers are unchanged.
  • worker/main.py::handle_message — classifies after a successful parse, using the exact raw fields["data"] string (never a re-serialization), persists via a purely additive consumers/sink.py lookup, then ACKs.
  • db/models.py + alembic/versions/0002_integrity_status.py — exactly one new column, integrity_status (String(16), NOT NULL, server_default="unsigned"). No index — nothing queries it yet, and the table has none on any column today.

Classification behavior

if not signature: return "unsigned"
return "valid" if verify_audit_event(service, data, signature, secret) else "invalid"

The missing-signature check happens before verify_audit_event() — verified by a dedicated ordering test, not just asserted.

ACK/retry behavior

Case Persisted? ACK?
Valid yes, "valid" yes
Unsigned (100% of current traffic) yes, "unsigned" yes
Invalid/malformed sig yes, "invalid" yes + distinct SIGNATURE INVALID print
Malformed JSON/schema no unchanged — still unacked/retried
DB failure no unchanged — still unacked/retried

Invalid signatures are persisted and ACKed rather than left pending: a signature verdict can't change on retry, so leaving it pending would only accumulate a permanent poison-pill entry, whereas persisting it preserves the forged attempt as durable evidence. A dedicated test proves the distinct log line never includes the secret, the signature, or the raw event data.

Migration

0002_integrity_status, down_revision="0001_audit_events" (21-char revision id, under the installed alembic's 32-char alembic_version.version_num limit — verified against source). Tests prove: a row inserted before this migration existed backfills to "unsigned" on upgrade (real backfill via server_default, not just schema presence), and downgrading exactly one revision removes only this column, distinct from the existing full-downgrade-to-base test.

Tests — 32 new

12 classification (valid/invalid/malformed/missing/empty, service+data+signature mismatches, exact-raw-string preservation via a reordered-JSON-keys case, missing-check ordering), 3 migration, 4 sink, 10 worker (including the secret/signature/data non-leakage security test), 3 real Redis+MySQL integration tests extending the existing PR-B0 fixture — signing with whatever AuditConfig.EVENT_SIGNING_SECRET actually resolves to in the running test process, never assuming which value it holds.

Full suite: 423 passed (391 baseline + 32 new), 0 regressions
ruff:       16 findings, all pre-existing or matching 0001's own Union/Sequence migration style — 0 new avoidable findings
git diff --check: clean

Compatibility

Every existing producer keeps working unmodified — none write sig today; the real integration test proves today's exact traffic shape ({"data": ...}, no sig) classifies as "unsigned" and persists/ACKs against real Redis+MySQL. No API Gateway, Control Center, or other-repo changes. audit/signing.py — zero-line diff.

Follow-up (deliberately not fixed here)

The dev-compose security-audit-worker container currently has JWT_SECRET unset (falls back to "change-me") while other platform services have the real shared secret configured. Harmless today since no producer signs yet, but must be corrected before any producer-signing rollout, or valid producer signatures will be classified as invalid.

🤖 Generated with Claude Code

PR2 of the audit:events integrity remediation (PR1 #4, merged: the
sign/verify helper, previously unwired anywhere). Wires classification
into the worker without requiring any producer to sign yet -- every
producer today writes only {"data": ...}, no `sig` field, and this PR
keeps that fully compatible: unsigned traffic persists and ACKs exactly
as before, now explicitly tagged integrity_status="unsigned" rather than
untagged.

AuditConfig.EVENT_SIGNING_SECRET reuses the existing JWT_SECRET env var
and "change-me" fallback -- same convention audit/jwt_verify.py already
uses, no new secret mechanism. audit/jwt_verify.py itself is untouched.

consumers/processor.py::classify_event_integrity(service, signature, data,
secret) is a new, separate function -- parse_audit_event()'s signature
and its 3 existing callers are unchanged. The missing-signature check
happens before verify_audit_event() is ever called (verified by a
dedicated ordering test), and verification is always performed against
the exact raw fields["data"] string, never a re-serialization of the
parsed event -- audit/signing.py's own MAC covers the exact transmitted
bytes, and reusing it unmodified was a hard constraint of this PR.

worker/main.py::handle_message now classifies after a successful parse
and persists the result via consumers/sink.py's new, purely additive
integrity_status=event.get("integrity_status", "unsigned") lookup.
Invalid/malformed signatures are persisted and ACKed, not left pending:
retrying a signature verdict can never change it, so treating it like a
transient parse/DB failure would only accumulate a permanent poison-pill
pending entry, whereas persisting it preserves the forged attempt as
durable evidence. A distinct "SIGNATURE INVALID" print fires only for
that case; a dedicated test proves it never includes the secret, the
signature, or the raw event data. Parse failures, DB failures, and
duplicate-event_id dedup are all unchanged -- confirmed by re-running the
pre-existing tests for those paths unmodified, byte-for-byte, alongside
the new ones.

db/models.py adds exactly one column, integrity_status (String(16),
NOT NULL, server_default="unsigned"), via new migration
0002_integrity_status (down_revision=0001_audit_events, revision id
21 chars -- under the 32-char limit the installed alembic's
alembic_version.version_num column enforces). No index: nothing queries
this column yet, and the table has none on any column today, including
ones already actively filtered on. Migration tests prove a row inserted
under 0001, before this column existed, backfills to "unsigned" on
upgrade to head -- not just that the column exists -- and that
downgrading exactly one revision (head -> 0001) removes only this
column, distinct from the existing full-downgrade-to-base test.

32 new tests: 12 classification (valid/invalid/malformed/missing/empty,
service+data+signature mismatches, exact-raw-string preservation via a
reordered-JSON-keys case, ordering of the missing-check before
verify_audit_event()), 3 migration, 4 sink, 10 worker (including the
secret/signature/data non-leakage security test), 3 real Redis+MySQL
integration tests extending the existing PR-B0 fixture -- signing with
whatever AuditConfig.EVENT_SIGNING_SECRET actually resolves to in the
running test process rather than assuming which value it holds. Full
suite 423 passed (391 baseline + 32 new), 0 regressions. ruff: 16
findings, all pre-existing or matching 0001's own established Union/
Sequence migration-file style; 0 new avoidable findings. git diff
--check clean.

Follow-up, deliberately not fixed here: the dev-compose
security-audit-worker container currently has JWT_SECRET unset (falls
back to "change-me") while other platform services have the real shared
secret configured. Harmless today since no producer signs yet, but must
be corrected before any producer-signing rollout, or valid producer
signatures will be classified as invalid.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@man4ish
man4ish merged commit 7892749 into main Aug 14, 2026
1 check failed
@man4ish
man4ish deleted the feature/audit-event-signing-pr2-consumer branch August 14, 2026 03:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant