Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 16 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,11 +141,19 @@ warns if neither is set. After that, every release publishes with an automatic
Slack alert.

Canonical UK exact-k builds also require a stable, base64-encoded 32-byte
`POPULACE_UK_TERMINAL_GATE_SIGNING_KEY`. Source `tools/release.env` before the
national build as well as publication. The terminal-gate aggregator authenticates
the complete report, canonical release id, and exact calibration-diagnostics
digest with HMAC-SHA256; the persistence seam
cannot sign caller-composed gate results, and publication independently verifies
the report from the same out-of-band key. If the key is missing or malformed,
the writer first persists an unsigned failed report and then raises, and
publication rejects it.
release key. Source `tools/release.env` before the national build as well as
publication. Two variables carry it during the report-format migration —
export both from the same key material:

- `MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY` — what the national build signs
with (the gate-battery executor) and what schema-4 report verification
reads.
- `POPULACE_UK_TERMINAL_GATE_SIGNING_KEY` — what schema-3 (legacy-format)
report verification reads; retires with the legacy format.

The gate battery authenticates the complete report, canonical release id, and
exact calibration-diagnostics digest with HMAC-SHA256; the persistence seam
cannot sign caller-composed gate results, and publication independently
verifies the report from the same out-of-band key. If the key is missing or
malformed, a full-scale build persists the unsigned report and then refuses
to stage, and publication rejects unsigned reports.
1 change: 1 addition & 0 deletions changelog.d/611-uk-battery-consumer.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The UK national build runs its gates through the shared `GateBatteryRun` executor: preflight before the frame loads, terminal immediately before the staging writer, one schema-4 report at the existing `terminal_gates.json` path with every declared entry present — evidence the build cannot supply is a named `evidence_absent` gap that blocks only under the new `--release-candidate` posture (refused on a sampled rung, the #627 coupling). Gate failures raise the typed `GateBatteryBlockedError` in place of the `"Release gates failed:"` string — any downstream consumer matching that prefix breaks silently and must switch to catching the typed error — preflight refusals now leave a persisted report, the signing key moves to `MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY` (unsigned full-scale builds still refuse to stage; rungs proceed with an honest `shippable: false`), and `calibration_diagnostics_sha256` rides the new signed `release_evidence` slot plus the build record. One exclusion-expiry clock is threaded to every exclusion-consuming gate — resolved when the battery is armed, before the stages, where the legacy aggregator resolved it after them — and a `--degenerate-exclusions` override is digested into the report's `evidence_sha256` so an overridden run self-describes; the override label follows the records' content, so re-supplying the committed register is not a deviation. The legacy `uk_terminal_gate_report` remains solely as the differential-test oracle; `_UKGateEvidence`, the duplicated `_evaluate_gate`, and the driver's error string-match are deleted. The schema-3 release verifier cannot read schema-4 reports — the schema-4 verification path must land before any exact-k release assembly.
6 changes: 5 additions & 1 deletion docs/gate-battery-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,11 @@ stretches on declaration alone:
| criticality mix | 8 blocking + 1 diagnostic | all blocking |

The differences are entirely in the two country inputs — the spec file
and the registry — which is the point.
and the registry — which is the point. The UK national build is the
executor's production consumer: it constructs one `GateBatteryRun` per
build and runs both phases under `BLOCKS_ARTIFACT` (preflight before the
frame loads, terminal immediately before the staging writer), while BE
remains spec-only.

## The reference rule

Expand Down
39 changes: 33 additions & 6 deletions packages/microcosm-build/src/microcosm/build/gate_battery.py
Original file line number Diff line number Diff line change
Expand Up @@ -520,11 +520,12 @@ def failures(self) -> tuple[str, ...]:
def _evaluate_gate(name: str, evaluator: Callable[[], GateResult]) -> GateResult:
"""Run one evaluator, failing closed on any misbehaviour.

Lifted verbatim from the UK terminal battery: a raising evaluator
becomes a failed result (the batch must keep evaluating — a crash that
masked the remaining gates would hide exactly the failures the battery
exists to surface), and a result under the wrong name fails rather than
letting one gate impersonate another.
The one fail-closed wrapper for every battery, shared with the legacy UK
terminal report: a raising evaluator becomes a failed result (the batch
must keep evaluating — a crash that masked the remaining gates would
hide exactly the failures the battery exists to surface), and a result
under the wrong name fails rather than letting one gate impersonate
another.
"""

try:
Expand Down Expand Up @@ -749,6 +750,11 @@ class GateBatteryRun:
cannot excuse absent evidence; dev builds record it and
continue.
registry: Gate bindings; defaults to :data:`DEFAULT_REGISTRY`.
release_evidence: Digests of release inputs the gates themselves do
not consume but the release contract links (for the UK, the
calibration-diagnostics digest). Carried in the report and the
signed attestation, so the linkage survives the build that
attested it.
"""

def __init__(
Expand All @@ -759,15 +765,25 @@ def __init__(
report_path: Path | str,
release_candidate: bool,
registry: Mapping[str, GateBinding] = DEFAULT_REGISTRY,
release_evidence: Mapping[str, str] | None = None,
) -> None:
if not isinstance(release_id, str) or not release_id.strip():
raise ValueError("release_id must be a non-empty string.")
validate_gate_parameters(gates, registry)
evidence = dict(release_evidence or {})
for key, value in evidence.items():
if not isinstance(key, str) or not key.strip():
raise ValueError("release_evidence keys must be non-empty strings.")
if not isinstance(value, str) or not value.strip():
raise ValueError(
f"release_evidence[{key!r}] must be a non-empty string digest."
)
self._gates = gates
self._release_id = release_id
self._report_path = Path(report_path)
self._release_candidate = bool(release_candidate)
self._registry = dict(registry)
self._release_evidence = dict(sorted(evidence.items()))
self._gates_manifest_sha256 = _canonical_sha256(
_gates_manifest_payload(self._gates)
)
Expand All @@ -789,6 +805,16 @@ def blocked_at_phase(self) -> str | None:
def phases_evaluated(self) -> tuple[str, ...]:
return tuple(self._phase_reports)

def phase_report(self, phase: str) -> GatePhaseReport:
"""The evaluated report for ``phase``; refuses a phase that has not run."""

try:
return self._phase_reports[phase]
except KeyError:
raise ValueError(
f"phase {phase!r} has not run; evaluated: {list(self._phase_reports)}."
) from None

def _next_phase(self) -> str | None:
for phase in self._gates.phases:
if phase not in self._phase_reports:
Expand Down Expand Up @@ -922,7 +948,6 @@ def report_payload(self) -> dict[str, object]:
and self._blocked_at_phase is None
and blocking_ok
and signing_key is not None
and self._gates_manifest_sha256 is not None
)
attestation: dict[str, object] = {
"schema_version": GATE_BATTERY_ATTESTATION_SCHEMA_VERSION,
Expand All @@ -936,6 +961,7 @@ def report_payload(self) -> dict[str, object]:
"phases": list(self._gates.phases),
"phases_evaluated": list(self._phase_reports),
"blocked_at_phase": self._blocked_at_phase,
"release_evidence": dict(self._release_evidence),
"evidence_sha256": dict(evidence_sha256),
"gate_outcomes_sha256": _canonical_sha256(gates_payload),
"signature_algorithm": GATE_BATTERY_SIGNATURE_ALGORITHM,
Expand All @@ -961,6 +987,7 @@ def report_payload(self) -> dict[str, object]:
"shippable": shippable,
"gates": gates_payload,
"policy_sha256": policy_sha256,
"release_evidence": dict(self._release_evidence),
"evidence_sha256": dict(evidence_sha256),
"attestation": attestation,
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,17 @@
with the declared one. Any *other* unexpected name passes through
untouched so that check keeps biting.

The evidence surface handed to the legacy gate modules mirrors the
national build's adapter (the three entity tables plus period,
weight-kind, and mass-log metadata); the two copies consolidate when the
national build swaps onto the battery executor and the legacy
orchestration path retires.
The evidence surface handed to the legacy gate modules (the three entity
tables plus period, weight-kind, and mass-log metadata) lives here as the
single copy: the national build's adapter consolidated into it when the
orchestration swapped onto the battery executor.
"""

from __future__ import annotations

from collections.abc import Callable, Mapping
from dataclasses import dataclass, replace
from datetime import date, datetime
from typing import Any

import pandas as pd
Expand Down Expand Up @@ -67,7 +67,9 @@
UK_INPUT_MASS_EXCLUSION_REGISTER_RESOURCE,
UK_INPUT_MASS_REFERENCE_EVIDENCE_SHA256,
UK_QRF_TAIL_EXCLUSION_REGISTER_RESOURCE,
UKReviewedExclusion,
_input_mass_reference_evidence_sha256,
coerce_reviewed_exclusions,
uk_dataset_input_mass_totals,
uk_input_mass_parity_gate,
uk_qrf_tail_concentration_columns,
Expand Down Expand Up @@ -230,6 +232,58 @@ def _stage_names_evidence(
}


def _exclusion_clock(context: EvidenceContext) -> date:
"""The one expiry clock every exclusion-consuming gate shares.

Exclusion receipts carry approval and expiry dates, and the release
contract requires every gate in one report to evaluate them on the same
date. A per-gate default could straddle midnight, so the clock is a
required artifact and anything but a plain ``date`` is refused.
"""

clock = context.artifacts["exclusions_evaluated_on"]
if isinstance(clock, datetime) or not isinstance(clock, date):
raise ValueError(
"exclusions_evaluated_on must be a datetime.date, got "
f"{type(clock).__name__}; expiry must be evaluated on one "
"shared clock."
)
return clock


def _resolve_degenerate_exclusions(
context: EvidenceContext,
) -> tuple[Mapping[str, UKReviewedExclusion], str]:
"""The exclusion records the degenerate gate runs, and their source.

The committed register is the reviewed policy of record (#630/#610);
a supplied ``reviewed_degenerate_exclusions`` artifact is the loud
review-time override — the evidence hook digests whichever resolved,
so an overridden run self-describes in the signed report. The label
follows the *content*, not the artifact's presence: records identical
to the committed register are the committed policy whichever route
delivered them, so an override cannot masquerade as a deviation (or a
caller re-supplying the register as a false one).
"""

committed = uk_default_degenerate_reviewed_exclusions()
override = context.artifacts.get("reviewed_degenerate_exclusions")
if override is None:
return committed, "committed"
resolved = coerce_reviewed_exclusions(
override, label="UK degenerate-surface policy"
)
committed_payload = {
name: record.policy_payload() for name, record in committed.items()
}
resolved_payload = {
name: record.policy_payload() for name, record in resolved.items()
}
if resolved_payload == committed_payload:
return committed, "committed"
return resolved, "override"


def _evaluate_degenerate_release_surface(
context: EvidenceContext, parameters: Mapping[str, Any]
) -> GateResult:
Expand All @@ -240,16 +294,33 @@ def _evaluate_degenerate_release_surface(
f"uk/gates.json names exclusion register {register!r} but the "
f"runtime loads {UK_DEGENERATE_EXCLUSION_REGISTER_RESOURCE!r}."
)
# The committed register is the reviewed policy of record (#630/#610):
# the battery must run the same exclusions the legacy terminal report
# resolves for None, or the two paths diverge on dormant/expired state.
resolved, _source = _resolve_degenerate_exclusions(context)
return uk_degenerate_release_surface_gate(
_uk_gate_surface(context.frame),
reviewed_exclusions=uk_default_degenerate_reviewed_exclusions(),
reviewed_exclusions=resolved,
now=_exclusion_clock(context),
**kwargs,
)


def _degenerate_exclusions_evidence(
context: EvidenceContext, parameters: Mapping[str, Any]
) -> object:
resolved, source = _resolve_degenerate_exclusions(context)
# ``exclusions_policy`` answers "which register content governed this
# run" — deliberately distinct from the build record's
# ``degenerate_exclusions_override_supplied``, which answers "did the
# operator invoke the override path". The two can honestly disagree
# (a review file byte-identical to the committed register), so they
# carry different names.
return {
"exclusions_policy": source,
"reviewed_exclusions": {
name: record.policy_payload() for name, record in sorted(resolved.items())
},
}


def _evaluate_zero_weight_strata(
context: EvidenceContext, parameters: Mapping[str, Any]
) -> GateResult:
Expand Down Expand Up @@ -366,6 +437,7 @@ def _evaluate_input_mass_parity(
uk_dataset_input_mass_totals(_uk_gate_surface(context.frame)),
context.artifacts["input_mass_reference"],
policy=context.artifacts["input_mass_policy"],
now=_exclusion_clock(context),
**kwargs,
)

Expand Down Expand Up @@ -397,6 +469,7 @@ def _evaluate_tail_concentration(
weights,
policy=context.artifacts["qrf_tail_policy"],
surface=surface,
now=_exclusion_clock(context),
**kwargs,
)

Expand Down Expand Up @@ -442,6 +515,8 @@ def _evaluate_tail_concentration(
name="degenerate_release_surface",
evaluator=_evaluate_degenerate_release_surface,
parameter_keys=frozenset({"reviewed_exclusions_resource"}),
artifact_keys=frozenset({"exclusions_evaluated_on"}),
evidence=_degenerate_exclusions_evidence,
),
"zero_weight_strata": UKGateBinding(
name="zero_weight_strata",
Expand Down Expand Up @@ -489,14 +564,16 @@ def _evaluate_tail_concentration(
"candidate_name",
}
),
artifact_keys=frozenset({"input_mass_reference", "input_mass_policy"}),
artifact_keys=frozenset(
{"input_mass_reference", "input_mass_policy", "exclusions_evaluated_on"}
),
evidence=_input_mass_reference_evidence,
),
"tail_concentration": UKGateBinding(
name="tail_concentration",
evaluator=_evaluate_tail_concentration,
parameter_keys=frozenset({"reviewed_exclusions_resource"}),
artifact_keys=frozenset({"qrf_tail_policy"}),
artifact_keys=frozenset({"qrf_tail_policy", "exclusions_evaluated_on"}),
legacy_name="qrf_tail_concentration",
),
}
Loading
Loading