Retarget the UK build to FRS 2024-25: re-pin raw vintages, regenerate parity instruments, re-measure gate baselines

[juaristi22]

Closes #723. Workstream E of the UK-migration master epic #665, child of #145. Retargets the UK build to FRS 2024-25 (survey/base 2024, calibration year 2025) per the vintage decision recorded on the issue: the parity bar re-freezes on the incumbent's live artifact enhanced_frs_2024_25.h5 @ the v1.56.14 tag (= incumbent-at-pin ebf733c). Donor vintages (SPI 2022-23, WAS r8, LCFS 2023-24, ETB) are unchanged. The 2023 instruments are preserved in git history at the pre-#723 pins (#687's model).

What moved

1. Release object (ex-E11 discipline) — new uk/frs_release.json + load_uk_frs_release() loader: survey/base/calibration years, time_period, vintage token, SN 9563, DOI 10.5255/UKDA-SN-9563-1, UKDS tab-zip sha, and the HF acquisition pins (repo, revision a2039519… = tag 1.56.14, zip sha 05dd0069… — the HF zip is byte-identical to the UKDS EUL zip). Lockstep tests bind TIME_PERIOD, every frs_table artifact vintage, and take_up_contract.build_year to it.
2. Raw-vintage re-pin — all 21 frs_table artifact entries re-pinned (sha256/size for the 2024-25 tabs, vintage: "2024_25") and upgraded to the SPI-stage convention (tax_year_start: 2024, ukds_study_number: 9563, doi); survey/source prose updated on all FRS stages; typed spec/sources.yaml moved in lockstep (its artifact schema gains the three keys).
3. Build period → "2024" — TIME_PERIOD, HMRC_SPI_BUILD_PERIOD, HMRC_CGT_BUILD_PERIOD; the take-up contract's build_year: 2024 flips two date-keyed rates (tax_free_childcare 0.586→0.88 @2024-04-06, tv_licence_evasion 0.1058→0.1252 @2024-01-01); both drivers stamp source_vintages: {frs: 2024_25} into build sidecars.
4. HMRC published surface re-mapped, not uprated (adjudicated 2026-08-20) — the 2023-24 Tables 3.6/3.7 ODS keeps its bytes, values, and vintage label; mapped_build_period: 2024 with the new explicit token period_mapping: "latest_published_tax_year". The frozen hmrc_income_source_stages.json and its sha pin are untouched; the frozen-copy lockstep test's named-rewrite whitelist carries the re-map; the default HMRC source-contract validator now reconstructs the live canonical payload from source_stages.json (frozen base_candidate retained). Fact-value alignment to the target year stays with Adopt the UK national calibration contract (re-homed from chronicle#164 per the #166 ruling) #707/Consume Ledger UK facts: uk_target_references, mapping module, and a compile-parity gate #622/Adjudicate the UK spine grain at the ingest/calibration seam #688.
5. Parity instruments regenerated from enhanced_frs_2024_25.h5 (sha 97a07f9c…, time_period "2024") — efrs_parity_reference.json (145 columns — surface unchanged; engine 2.89.0) and release_input_coverage_manifest.json (145 required / 0 exclusions; candidate side stays the certified populace_uk_2023 until Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686). Known gaps: none added, none removed. New test pins the record-count identity: household 52,846 = (16,288 raw + 10,000 SPI) × 2 + 270 CGT band donors (30 × 9 HMRC bands; 181 FRS-side + 89 SPI-side). Two vintage findings: no composition-rule drop at 2024-25 (2023-24 dropped 390 raw households) and the donor count moved 780 → 270 (band-file min-lower filter).
6. Gate-side reference pins — the efrs-post-calibration registry descriptor replaced in place with the 2024-25 identity and totals_sha256 e70a4538… (131-column register, width unchanged); mirrors re-pinned in gates.json, weighted_integrity.py, and the data-shard contract. No thresholds move — Retarget the UK build to FRS 2024-25: re-pin raw vintages, regenerate parity instruments, re-measure gate baselines #723 measures, Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686 arms. UK_REFERENCE_DATASET_NAME → enhanced_frs_2024_25_recalibrated.
7. Smoke rung on the ingest driver (adjudicated 2026-08-20) — --sample-fraction (Doctrine: the scale ladder — dev-scale runs before production spend ("boring before billed") #624 rungs) subsamples households post-frs_spine via the generic frame-sampling helpers; sampled rungs refuse checkpoints, record a sampling receipt + rung token, and adopt the named-edge abort. Three full-scale fences gain an explicit receipt-posture on sampled rungs (leaves raw-coverage, SPI stack scaling, ultra-sparse effective-mass restoration); f100 keeps every fence strict.

Signed differences and fixed defects

• HMRC period re-map (above) — signed nearest-available-vintage declaration; no 2024-25 SPI-based HMRC edition exists (~2-year lag). The 208-fact replay honestly becomes "2023-24 published facts replayed against a 2024 build".
• SN 9252 prose defect (fixed and signed) — six FRS stages cited the FRS 2022-23 study number in source prose; all now cite SN 9563 with the DOI, and the release object is the single source.
• WAS bridge-donor pin defect on merged main (fixed and signed) — lcfs_consumption declared the WAS tab with locator: <filename> while was_wealth pins locator: "caller-supplied local input"; _artifact_pins() refused every full-roster licensed run (introduced by the E6 merge-time provenance fixup, after E6's acceptance twins ran; invisible to secrets-free PR CI). Locator aligned; hermetic pin-coherence regression test added.
• Take-up rate flips — semantics of the ported date-keyed rate_at() at build_year 2024; values verified against the committed contract keys.
• Consequence note — the national staging driver derives its build period from the input frame, so staging runs of the certified 2023 line now require the pre-Retarget the UK build to FRS 2024-25: re-pin raw vintages, regenerate parity instruments, re-measure gate baselines #723 checkout (consistent with Legacy 2023-vintage build of the microcosm UK spine (deferred) #687's git-history model); Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686's control build should pin accordingly.

Licensed acceptance (all data local; disclosure-safe evidence only, data/ukds/acceptance/723-frs-2024-25/)

• Raw-vintage audit (raw_vintage_audit_2024_25.json): staged 25-tab set byte-identical to the UKDS zip @ the pinned HF revision; provenance.json written (E6 schema). No consumed-column regressions (benunit's 178-column material-deprivation drop, covesa1-4, month everywhere — all unread). househol.tab confirmed sernum-scrambled (the E2 sort guard now binds for real); SALSAC domain 216/3,743/12,732 of 16,691 (asked-state capture unchanged); EDUCQUAL codes 1–87 both vintages, no map change.
• Twin f100 builds payload-identical (twin_payload_compare.json, exit 0) on the full 21-stage roster.
• Per-stage identity receipts green: e4 (recompute scoped to the survey channel per the post-Create the SPI income channel on the microcosm spine with the FRS-only second stage #717 rule — take-up targets are unweighted int(rate × n_units)), e5, e6.
• Smoke rung f001 end-to-end PASS (spine-smoke-f001.h5 + sampling sidecar + Logbook row) — no abort receipt needed.
• Nonzero shares vs the re-frozen reference: 139/145 columns overlap; 27 beyond ±0.02, all in the already-signed method classes — E6 consumption (donor-faithful vs the incumbent's zero-inflation collapse, e.g. dfe_education_spending ours 0.225 vs ref 0.0003), E5 wealth-QRF, E7 SPI-channel QRF — plus E8-pending salary-sacrifice. Missing columns: capital_gains/student_loan_plan (E8 Port capital gains structure, salary sacrifice, and student loan stages #684), three E9-class derived-benefit columns, household_weight (bookkeeping). Weighted-distributional comparison binds at Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686.
• owned_land stability re-derived at 2024-25 (owned_land_stability_receipt.json): 71.4% national / 112.9% max-region seed swing — the uk-data#448 class persists (37.7% at 2023-24), evidencing the exclusion renewal. Seed0 cross-check annotated as the Carry manifest rewrites into the shared Stage/StageRecord producer graph #721 in-place-rewrite class (in-build determinism = the payload-identical twins).
• Baselines recorded (uk_weighted_integrity_baselines_2024_25.json): 174 input-mass totals + 47 QRF tail grids at spine grain; incumbent-side totals evidence (uk_input_mass_reference_2024_25.json, 131 columns) digest-bound into the gate registry. These are the candidate thresholds Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686 arms.

Adjudications (@juaristi22, 2026-08-20 — confirmed in session)

1. Input-mass exclusion re-signings confirmed: charitable_investment_gifts (expires 2027-02-20) and owned_land (expires 2026-09-20, fresh 2024-25 stability receipt above). The registry scope notes were tightened afterward to state the precise fact (the 2024-25 incumbent carries the SPI-synthetic rows structurally but no admin-restored mass in the channel-exclusive columns); the approved exclusion reasons are verbatim-untouched.
2. UK_REFERENCE_DATASET_NAME = "enhanced_frs_2024_25" — the _recalibrated suffix was dropped per adjudication: the label now names the pinned artifact exactly (no recalibrated variant exists at this vintage; the June report strings keep their own label).
3. Shares-divergence classification confirmed: the 27 columns rest on the existing signed method classes; no new class minted at this seam (the weighted-distributional comparison at Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686 is where one would be, if residuals survive the E8 composition close).

Adversarial review (Codex, 2026-08-20) — dispositions

Two high findings, both dispositioned in Adversarial-review dispositions (last commit):

• Accepted — evidence fields must bind to the bytes their hash names: the family-coverage block rendered the re-mapped period fields from the canonical manifest while hashing only the frozen mirror. Now carries a dual pin (source_manifest = the frozen June identity; canonical_source_manifest = the bytes the re-mapped fields come from), with a test binding each field set to the sha256 of its actual source and asserting the frozen mirror still declares the June mapping.
• Rejected with armor — do not regenerate the June replay report: the committed hmrc_income_replay_report.json deliberately records mapped_build_period: 2023; it is evidence for the grandfathered June release, not the 2024 line, and retires with the frozen manifest after Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686 (Legacy 2023-vintage build of the microcosm UK spine (deferred) #687's disposition). Regenerating it would destroy the June anchor. Instead, a new assertion binds it to the frozen manifest's declared mapping, making the freeze partition self-describing. (This is the one additive touch to test_uk_hmrc_replay_artifacts.py; every pre-existing assertion is unchanged.)

Also in this round: rebased onto main past #729 (uk-national-calibrate) — conflicts resolved by keeping #729's structures with our values applied, and the gate-battery digests re-cut over the union from the live producer payload.

UK-specificity note

UK-specific: the release object and its loader (FRS/UKDS identity), the manifest re-pins, the HMRC period re-map, take-up contract year. General: the ingest-driver rung support (generic frame_sampling helpers), the sampled-rung fence postures (any country's ladder can use the pattern), the pin-coherence regression test shape, and the three added artifact-schema keys.

Implemented by Codex under /codex-implement-plan (4 recorded deviations, all reviewed: uv cache relocation; coverage-manifest regenerated in two commits; HMRC contract validator reconstruction — kept, it is the mechanism for signed re-maps over a frozen original; frs_release.json allowlisted in the immutable-reference naming test). Claude review added the WAS-pin fix, the sampled-rung fence postures, and the e4-receipt scoping — all licensed-run findings PR CI cannot see.

🤖 Generated with Claude Code