How to build, test, install, and debug the Codex compiler. For the
runtime memory model, register conventions, and allocator internals,
see docs/ArchitectsSketchbook.md.
The build script (build/build.ps1) runs the full verification
pipeline. Each phase must pass before the next begins.
Traps in the surrounding tooling, every one of which presents as something other than what it is:
tools/codex-vm.exeis a versioned Perforce binary.build-vm.ps1cannot relink it untilp4 edit tools/codex-vm.exe; without that you getLNK1104, which reads like a missing file rather than a read-only one. Open BOTH the.cand the.exe, and after any merge-down that touched the.c, relink before running anything: a resolve takes main's binary wholesale, so the exe on disk is no longer the one your source builds and every measurement from it is about somebody else's revision.build/run-plug.ps1takes-InFile, not-Input($Inputis a PowerShell automatic variable and shadows it). The request it sends is framed and the reply is not, so strip a header only when the length field proves one is there.- PowerShell's
-band 0xFFFFFFFFis not a mask.0xFFFFFFFFparses as Int32-1, so the AND is the identity and the value reaches the next[int]cast intact and overflows. In hand-assembly likecdx-to-pe.ps1, take the bytes from[BitConverter]::GetBytesinstead. - Variable names are CASE-INSENSITIVE, so
$emand$EMare ONE variable. A scan holding its needle in$EMand its per-file counter in$emsearches for whatever the counter last held. Cost an hour on 2026-07-28: the counter reset to 0 each file, so from the second file onward the scan looked for[char]0, every real hit vanished, and binaries decoded as UTF-8 matched their NUL bytes and reported a uniform 10. The uniform count across unrelated files was the tell -- a real measurement does not come out identical on a.glb, a.wasmand an.elf. Never distinguish a constant from a counter by case, and note that-replaceis case-insensitive too, so renaming$EMalso renames every$em. Measure-Object -Linecounts only NON-BLANK lines, and nothing at the call site says so. Measured 2026-07-28 it reported the compiler at 48,456 lines against a true 57,440 fromReadAllLines, a 16 per cent undercount, heading into a public report. Any line count wants two methods before it is published; the disagreement is the instrument, not an annoyance.-match/-notmatchagainst an ARRAY are filters, not tests. They return the matching elements, and a non-empty result is truthy, soif ($lines -notmatch 'x')is true whenever ANY line lacksx-- nearly always. Join to one string before testing. This silently disabled a whole guard inmerge-down-all.ps1.
-
Clean: Remove
build-output/and temporary files. -
Source concat:
build/concat-codex-self.ps1scanscodex/compiler/for.codexfiles, resolves foreword dependencies transitively, and emits a single concatenated source file. Foreword chapters are prefixed withForeword--. Compiler chapters in subdirectories are prefixed with the directory name (e.g.,Emit--). -
CDX build: Boot the seed (
seed/Codex.cdx) in the VM (codex-vm by default), feed it the concatenated source over serial, receive the compiled CDX binary. This produces the SUT (System Under Test). -
Sign: Compile and run an inline Ed25519 signing program to embed a signature in the CDX header (bytes 40-135).
-
Canary: Compile
codex/test/factorial.codexwith the SUT and verify runtime output matchescodex/test/factorial.expected. This confirms the SUT can compile and run a simple program. -
Semantic equivalence: The SUT emits the source in TEXT mode (stage1.codex).
build/compare-codex-semantic.ps1parses both source and stage1, normalizes whitespace/parens/operator aliases, and compares every definition body. Mismatches indicate the emitter lost information.Its operator precedence table is a COPY of the compiler's and has to track it.
Strip-RedundantParensdecides whether a paren is load-bearing by comparing precedences, so a table that disagrees withoperator-precedenceincodex/compiler/Syntax/ParserCore.codexmakes the leg either reject correct output or accept wrong output. Measured 2026-08-11: the shipped table gave&and|the SAME precedence where the compiler gives Ampersand 3 and Pipe 2, so(a | b) & cnormalized toa | b & cand the two compared EQUAL. The table now carries the compiler's own numbers. If you changeoperator-precedence, change this table incodex/build/comparecodexsemanticScript.codexin the same CL.Be precise about what that cost, because it is narrower than it sounds. The compiler parses these correctly and the text emitter preserves the parens, so no verdict this leg ever returned was wrong. The old table stripped the parens from the source side AND the stage1 side equally, so the two still matched. What it produced was a BLIND SPOT: measured 2026-08-11 over the compiler's own source, four definitions normalize differently under the corrected table --
join-title-partsandscan-class-instance-defs(Syntax/Parser.codex:1245,:1522),normalize-list-insert-atandnormalize-list-prim(Types/TypeChecker.codex:309,:250). All four mix boolean|and&. For those four, the leg was comparing a form whose grouping had already been destroyed, so an emitter that reassociated|against&there would not have been caught.Two things it still does NOT know, both pre-existing and both unmeasured against real output: its tokenizer has no
|>(it lexes as|then>), and no===,~or~0, all of which the compiler's table ranks. Nothing has yet shown these matter on the compiler's own source, which is the only corpus the leg runs on. -
Text fixed point: The SUT emits stage1.codex, then emits stage1.codex again to produce stage2.codex. SHA-256 of stage1 must equal SHA-256 of stage2. This proves the text emitter is idempotent.
-
CDX fixed point: The SUT compiles source → stage1.cdx. Then stage1.cdx compiles source → stage2.cdx. SHA-256 of stage1.cdx must equal SHA-256 of stage2.cdx. This proves the compiler is a fixed point of itself -- the binary it produces is identical to itself.
-
Test battery:
build/test.ps1runs all samples incodex/test/. Each sample has a sidecar (.expectedfor success,.failingfor expected errors,.skipfor skipped). Runs parallel VM instances. -
Plug gates: the five native backends (riscv, arm64, elf, pe, img) must rebuild clean with the just-proven compiler (
plug-binary), and a representative transpiler subset (typescript, python, rust, ptx) must run end-to-end -- SUT IR over the framed TCP wire through the plug VM to non-empty target text (plug-smoke). Missing plug CDX builds once and caches; a failing smoke run gets one rebuild-and-retry before failing the build. The full 53-plug matrix (codex/plugs/test-plugs.ps1) remains a manual sweep.
build/test.ps1 # Sample battery (~2-5s per sample)
build/test.ps1 -Jobs 8 # Parallel test (8 batch slots; 8 is the standard)
build/test.ps1 -All # Include foreword + app tests
build/test.ps1 -Fatal # Include fatal (GPF/exception) tests
build/build.ps1 # Full pipeline (all gates)
build/desk.ps1 # The desktop, in a window, ~1.5s to paintMost of build/ is emitted by a Shell DSL: 49 generators under
codex/build/*Script.codex, each printing one script. build/build.ps1,
build/test.ps1, build/compile.ps1 and build/bvt.ps1 are all in that set.
They have not been the source of truth for a long time. Do not regenerate
one to fix a bug in it. Measured 2026-08-03 by compiling every generator
against the depot seed and diffing its output against what ships: 39 of the
40 generators with a live target have drifted, about 6,300 lines in total.
Only vm-config still matches. Nine more generators emit scripts that no
longer exist anywhere. Nothing has ever regenerated or diffed any of them, so
the drift accumulated silently while the shipped scripts were maintained by
hand.
The shipped script is the maintained side in every case measured, and the generator is the abandoned one, so a regeneration is a downgrade rather than a refresh. Two examples, because the shape matters more than the count:
compile.ps1's generator declares 3 parameters against the shipped script's 30 (measured 2026-08-03: it hasSrc,OutandLogand nothing else), and itsStage0is the hardcoded literalbuild-output\bare-metal\Codex.cdxwith no-Kernelpath at all. A regeneration would delete-Kernel,-Text,-Measureand-Decks, whichbuild/build.ps1,deck-floor-test.ps1and most recipes in this document all pass. The gate would go red on the first run and the cause would read as a compiler regression. This is the sharpest instance and fester found it.test.ps1's generator does not emit-ApprovedByat all. Regenerating it removes the stop that keeps the full battery from being launched by an agent, and removes it silently.stress-sweep.ps1was the same until it was backported.- Every unconverted generator still defaults to
-Jobs 4. That number was overturned on 2026-08-02; the shipped scripts say 8.
So the generators encode at least two rulings that have since been reversed.
Until a generator is backported, hand-editing the .ps1 is the correct
thing to do and regenerating it is not.
build/check-generated-scripts.ps1 # drift table for all 49
build/check-generated-scripts.ps1 -Only test # one, by emitted name
build/check-generated-scripts.ps1 -Diff test # the actual drift, line by lineIt exits 1 when anything has drifted and takes about 40 seconds for the whole
set (one VM boot compiles them all). It is deliberately not wired into
build.ps1 or the battery, and it deliberately has no write mode: with the
generators in their current state, a bulk regenerate would destroy the working
scripts. Use -Diff and port the change back into the generator by hand.
When a generator is backported and -Only <name> reports match, that script
flips: the generator becomes the place to edit, and hand-editing the .ps1
becomes the thing that reintroduces drift. stress-sweep and
lint-unused-cites have flipped; vm-config reports match but is still
almost entirely ScRaw, so it is not yet a place to edit either. Everything
else is still hand-edited.
build/desk.ps1 # interactive window at 1600x900
build/desk.ps1 -Width 1920 -Height 1080 # any mode codex-vm will give you
build/desk.ps1 -Force # recompile even if the CDX is current
build/desk.ps1 -Shot shot.bmp # headless, one frame, then exit
build/desk.ps1 -Keys '4000:4' # scancode timeline: 33 = f, 4 = 3, 1 = Esc
build/desk.ps1 -Rtc 2026-07-30T06:00:00 # freeze the taskbar clock
build/desk.ps1 -Disk seed/Codex.img # give the Files pane a real ESPTwo scales, and they step at different widths. ui-scale drives the
hand-drawn surfaces (topbar, welcome window, clock, cursor) and doubles at
1024. ui-wscale drives the WIDGET layer and doubles at 1600, because the
widget tree is laid out in LOGICAL pixels at w / ui-wscale: doubling it
does not enlarge the chrome so much as halve the ROOM, and every app tree in
the depot is written against 1024x768. Measured 2026-08-11: at scale 2 a
1024-wide panel lays out in 512x384 and the calculator loses four of its
fifteen keys off the bottom, so the widget step stays at 1600 where half the
panel is still at least as much room as 1024x768 ever gave. Below 1600 the
widget layer is scale 1 and the desk is pixel-for-pixel what it always was.
-Disk copies the image to build-output/ and attaches the copy.
codex-vm writes back and flushes to the host, so attaching a depot
artifact directly would let a guest edit it. With seed/Codex.img the
Files pane browses ESP:/ and lists EFI/, SOURCE.SRC and
CODEX.CDX, which is the compiler reading its own boot medium. Without a
disk the pane says no FAT ESP on the boot medium and the font falls
back; both are correct and neither is a fault.
An F12 shot from inside the desk takes 16 to 110 SECONDS in the bed, and a
short deadline renders that as "F12 does not fire". med-selected is unset
under DeskVm, so every sector operation of the ~3 MB BMP write pays the
AHCI+NVMe probe chain before falling through to IDE. Any capture deadline
under two minutes reports a working key as a dead one -- twice filed as a
finding here and twice refuted. Give it -screenshot-delay 120000 or more,
and verify by scanning the disk copy (build-output/desk-Codex.img) for the
SH*.BMP FAT directory entry (S, H, six digits, B, M, P) rather
than trusting the on-glass verdict. The chaser that settles "did the key
arrive" without waiting: inject S after F12; if shadows do not toggle the
loop is inside the shot, meaning the key landed and the arm is merely slow.
GopDesk normally reaches the glass through an Option A boot image, which
means flashing a stick or driving OVMF. apps/works/DeskVm.codex is the
dev-box entry point instead: it reads codex-vm's own GOP cells (0x7C4,
0x7C8, 0x7E0, framebuffer at 0xBF000000) rather than the boot stub's cells
at 0x8000, so the desktop comes up from a plain CDX with no image, no ESP
and no firmware in front of it. Measured 2026-07-30: the desk paints
1.5 seconds after launch (nothing at 1.0s, chrome at 1.5s), so a key
pressed in the first second is gone.
f opens Files, 3 opens the 3D View, Esc leaves a view for the desk.
Editor, Terminal, Monitor and Settings are inert chrome. There is no disk
attached, so desk-font falls back to the CBF bitmap face and the Welcome
window says so; that is the fallback working, not a fault.
Nothing leaves the desktop by key. desk-loop does not return, so Esc
at the desk does nothing and the guest cannot be ended by a stray keypress
(fester, main 12350). Shutdown, bottom-left, is the way out, and it is
a deliberate click. Closing the window or Stop-Process -Id also works.
The launcher hands back the codex-vm PID and the Stop-Process -Id line
to close it. Use it: several agents run guests on this box and killing
codex-vm by name takes down all of them (see the section below).
Phase 1 -- Batch compile. One VM per job slot, REPL loop reuse.
Each slot boots seed/Codex.cdx once, then compiles multiple test
sources sequentially over the persistent serial connection. The
compiler resets its heap between compilations automatically.
Phase 2 -- Run. Each compiled CDX with a .expected sidecar is
booted in its own VM. Serial output is captured and compared
byte-for-byte against the expected output.
| File | Meaning |
|---|---|
foo.expected |
Compile must succeed; runtime output must match |
foo.failing |
Compile must fail with listed CDX error codes |
foo.diag |
Compile must succeed and emit each listed CDX code at any severity (warning/info/error); one per line. Tests warnings/infos. |
foo.skip |
Skipped entirely (first line = reason) |
foo.slow |
Skipped unless -Slow (first line = reason) |
foo.fatal |
Skipped unless -Fatal (kills VM at runtime) |
foo.flags |
First line appended to the compile mode line: prose, passes=+name, decks=N. Read by the batch harness only. See docs/ExaminersAssay.md |
foo.stdin |
Pumped to VM serial after boot (runtime input) |
foo.keys |
Scancode timeline (t:scancode per line, t = ms since boot) passed as -keys-file. This is the keyboard; .stdin is the serial ring. A keyboard read (uefi-read-key / poll-key) reads the PS/2 key cell and no .stdin reaches it -- pick by what the code reads. See docs/ExaminersAssay.md |
foo.disk |
Attached as IDE disk image (primary master) |
foo.disk2 |
A second image, attached as the primary slave and reached by block-select 1. A test whose subject is WHICH drive it reached needs two: with one image behind every position, a working drive-select and a missing one produce identical output. codex/test/block-select-drives is the worked example. Copied to a writable temp like .disk, which matters more here because the case it exists for is one drive writing to another |
foo.smp |
Core count; the test boots with -smp N |
foo.vmargs |
Extra codex-vm flags (whitespace-separated, # comments). For tests whose subject is the machine: a bus topology, an absent device. See docs/ExaminersAssay.md |
Results are written to test-output/_results/, one file per test.
Each file contains a tab-separated line: STATUS\tNAME\tDETAIL.
| Status | Meaning |
|---|---|
| PASS_EXPECTED | Compiled, ran, output matched |
| PASS_FAILING | Compiled failed with expected error codes |
| PASS_UNVERIFIED | Compiled but no .expected sidecar |
| FAIL_COMPILE | Compile failed unexpectedly |
| FAIL_OUTPUT | Ran but output didn't match |
| FAIL_RUNTIME | VM crashed or timed out |
| FAIL_EXPECTED_BUT_COMPILED | Expected failure but compile succeeded |
| FAIL_WRONG_DIAGNOSTIC | Failed but wrong error codes |
| SKIPPED | Skipped (with reason) |
All scripts use build/vm-config.ps1 for shared VM setup.
tools/codex-vm.exe -- a ~12,800-line C program (measured 2026-08-05) using Windows Hypervisor
Platform (WHP). Build with tools/build-vm.ps1.
A guest CAN detect codex-vm through the HYPERVISOR bit since 2026-08-18.
CPUID leaf 1 ECX carries bit 31 (the conventional "am I virtualised" answer)
and nothing else; leaves 8000_0002h..04h answer the brand string
codex-vm virtual CPU (WHP). Until 2026-08-18 leaf 1 ECX read ZERO and the
brand leaves were absent, so a guest was told it was on bare silicon with no
name. Otherwise unchanged: the vendor string is GenuineIntel, the maximum
basic leaf is 1, and leaf 1 EDX carries a real feature set (FPU TSC MSR PAE
APIC CMOV MMX FXSR SSE SSE2). The corollary for anything decoding CPUID: a
feature line with nothing but the hypervisor bit on this host is the host,
not a broken decode. The diagnostic ladder's cpu stage reads hypervisor
here by design.
Never kill codex-vm by process name. Several agents run this box at once,
each booting VMs out of its own D:\Projects\Cobblestone-<agent>\, so
Get-Process codex-vm | Stop-Process -Force takes out every agent's VMs and
not just yours. A gate or a battery that dies for no reason someone can explain
is what that looks like from the other end. Filter on the executable's path:
Get-CimInstance Win32_Process -Filter "Name='codex-vm.exe'" |
Where-Object { $_.ExecutablePath -like 'D:\Projects\Cobblestone-<agent>\*' } |
ForEach-Object { Stop-Process -Id $_.ProcessId -Force }The same caution applies to reading a stray VM as evidence: a codex-vm you did
not start is usually another agent working, not a leak of yours. Check
ExecutablePath before concluding anything about it, and before killing it.
Build note: the linker fails with LNK1104 if a codex-vm is holding
tools/codex-vm.exe open, so stop your own VMs (as above) before
tools/build-vm.ps1.
codex-vm: unrecognised argument '-x' on stderr, exit 2, on the first one it
meets. Before that it dropped them in silence, and that was the defect under
the one below: a typo, a renamed flag, a flag deleted from the emulator and a
flag that was never built all presented to the caller as a clean launch, so a
green boot was not evidence that a flag took effect. The parse loop ran 121
else if arms and ended with no final else; it has one now.
Only a leading - reaches the refusal, because every flag's value is consumed
by its own arm. -run-list and -run-list-wall are answered in an earlier
loop that returns before this one runs, so they are unaffected.
If a command line that used to work now exits 2, the flag it names was never
doing anything. That is what it found on its own first run: -serial stdio -timeout <n> in build/test-exception-handler.ps1, neither parsed in any
revision, with Wait-Process -Timeout beside them doing the work -timeout
appeared to do. Both dropped; the harness still passes 5 of 5.
The instrument, when a flag's effect is in doubt, still works and no longer has
to be reached for first: the parse sites are all strcmp(argv[i], "-..."), so
grepping that pattern yields the complete set codex-vm accepts. It answered 120
flags on 2026-08-24 and 121 on 2026-08-27; re-measure rather than quoting
either (L-COUNT).
No revision of tools/codex-vm.c has ever parsed either flag (#1 through
#110 checked 2026-08-24, and the flag table below is the full set it does
parse). They are out of the usage banner as of 2026-08-27 and were not
implemented: with the refusal above in place, advertising them would advertise
two flags the program answers with exit 2. Passing either now says so.
Until then, because unknown flags were dropped in silence, the command line looked accepted: the guest booted, found nothing on the wire, and halted inside 500 ms.
That kills Start-VmRun's codex-vm path in build/vm-config.ps1, which builds
exactly those two flags. Start-CodexVmRun reads the fast exit as a failed
launch, retries four times and returns $null, so a caller reports
FAIL: VM did not start and the real cause is a flag that was never there.
Wherever codex-vm is present -- every box -- these are unrunnable:
codex/plugs/elf/extract-x86-output.ps1, build/test-disk-compile.ps1,
tools/sim-test.ps1, build/gdb-watchpoint.ps1. The QEMU fallback branch does
carry a real serial wire, so the path works only where codex-vm is ABSENT,
which is nowhere.
tools/test-codex-vm.ps1 passes the same two flags directly and also invokes
codex.build\sample-compile-selfhost.ps1, a path that no longer exists.
Whether this gets the serve mode built or the callers deleted is unclaimed.
codex-vm -kernel file.cdx [options]
| Flag | Default | Description |
|---|---|---|
-kernel <file> |
(required) | CDX or multiboot kernel to boot |
-mem <MB> |
3072 | Guest RAM in megabytes. Binaries compiled by seeds older than CL 7209 require more than 2048 (their boot stack lands in the demand-paged range below 2 GB); current seeds boot at any size from ~128 MB up. The size REPORTED to the guest is capped at 3040 MB, so anything above that changes nothing without -mem-nocap: see "The guest heap ceiling is 3040 MB" |
-mem-nocap |
off | Report the real -mem to the guest instead of the 3040 MB cap. Only for a run that draws nothing: the heap may then grow through the GPU and GOP windows, which are at fixed GPAs |
-input <file> |
-- | Pre-load file into serial ring buffer (source input). The image needs a TRAILING ZERO BYTE after the source -- see below |
-output <file> |
-- | Capture serial output to file |
-disk <file> |
-- | Attach IDE disk image as the primary channel's MASTER (read/write, flushed to host) |
-disk2 <file> |
-- | Attach a second image as the primary channel's SLAVE, reached by block-select 1. A position with no image behind it answers the floating bus (0xFF on the signature registers, 0x00 on status) so the guest's own detect reports it absent. Before this existed there was one image behind every drive position, dm-enumerate-drives reported four drives on a machine with one disk, and "Install Codex to Drive" pointed at any of them repartitioned the boot disk. Status answers 0x00 rather than 0xFF deliberately: emit-ata-bring-up runs a bounded BSY wait before it reads the signature, and a floating 0xFF there costs a million port INs, which is a million VM exits, on every diskless boot |
-headless |
off | Suppress VGA/GOP display window |
-board-mmio |
off | Commit and map host RAM at the three board register windows that sit above the RAM ceiling: RP2040 SIO (0xD0000000), Cortex-M PPB/SCB (0xE0000000), BCM2711 peripherals (0xFE000000, 9 MB). A board driver's register access then reads back what it wrote -- the same fidelity the six sub-3GB boards get from falling inside guest RAM. Opt-in, because it shadows the HDA and xHCI BARs: audio and USB are dead while it is on, which a board test does not care about. Required by the pi4 / rp2040 / stm32l4 driver tests; build/boards-test.ps1 passes it. |
-xhci-no-root-kbd |
off | Unplug the HID keyboard on xHCI root port 2, leaving the high-speed hub on root port 4 as the only route to a keyboard. The bus walk takes the first keyboard it finds and the root port comes first, so this is what lets a test drive the hub path with an unmodified guest binary. |
-xhci-hub-tiers <N> |
1 | How many hubs to stack on xHCI root port 4. 1 is a high-speed hub with a full-speed keyboard below it. 2 inserts a full-speed hub in between, which is what tells apart a driver that reads the transaction translator off the immediate parent from one that carries the nearest high-speed ancestor's down the walk -- a full-speed hub has no translator of its own, so the keyboard below it is still served by the high-speed hub two tiers up. Route strings run to two nibbles (0x11). A monitor hub in front of a keyboard hub is this topology. |
-xhci-ports <N> |
4 | Root ports the controller REPORTS (HCSPARAMS1 and the Supported Protocol capability), 4 to 32. The four modelled devices stay on ports 1-4; ports above them are empty and POWERED, so PORTSC reads non-zero there, which is what a real wide controller looks like. Use it on anything that INDEXES A TABLE BY PORT NUMBER: xhci-diag-ports wrote 20 + port into a band that ends at 27 and laid PORTSC over the handback flag gating kbd-pump. Four ports here and eight in QEMU both sat on or under that boundary, so neither bed could show it; the ASUS reports 26. |
-usb-setcfg-fault-once <N> |
0 | As -usb-setcfg-fault, but the fault applies to the FIRST SET_CONFIGURATION only and then clears. A TRANSIENT failure and a permanent one want opposite fixes in the host, so a bed that can only produce the permanent kind cannot show that a reading distinguishes them. Pair with the probe's retry: field: permanent gives sc1=4 sc2=4, transient gives sc1=4 sc2=1. |
-usb-bot-drop <N> |
0 | Swallow the Nth transfer event on a BULK endpoint (EP0 enumeration untouched), counting from 1. The data still moves and the completion code is still success; only the EVENT goes missing, so the guest spins its xhci-fuel out and reads no completion. This is the worksflight 2026-08-09 signature exactly: a 32 KB data phase answering msc-ce-no-event with no stall and a volume left clean on all four FAT questions. Nothing else can reach a guest's timeout path, and everything behind it -- Stop Endpoint recovery, the latch drain, BOT Reset Recovery, the chunk retry -- was unexecuted code before this flag existed. Also implemented with it: Bulk-Only Mass Storage Reset (bmRequestType 0x21, bRequest 0xFF), which clears the model's in-flight CBW; without it a retried command arrived while bot.active was still set and was consumed as write data. |
-usb-bot-drops <K> |
1 | How many CONSECUTIVE transfer events -usb-bot-drop swallows, starting at N. One drop is always recovered -- the driver resets, re-issues, and the write completes (rty=3), so with the default K no bed can produce a REFUSED write at all, and the shape the 2026-08-19 metal sitting showed had no bed reproduction. K=4 at index 500 reaches it on diag.img: sink state=write-refused wr=0 cc=256 lba=3574 rty=2 ph=1 after=0, the medium refusing a small write after the big one as well. The index decides which state you get and the states are distinct, so sweep it rather than assuming one value covers the path (measured 2026-08-19: 300 recovers, 500 refuses, 1000 writes and fails the readback). |
-usb-bot-drop-len <N> |
0 | Swallow the transfer event for any bulk WRITE while the BOT command in flight carries N bytes or more. The size-keyed sibling of -usb-bot-drop, and the reason to prefer it: an ordinal is not a property of the thing under test. -usb-bot-drop 500 names the 500th event since boot, so inserting a stage anywhere upstream moves the drop into a different phase silently. Measured 2026-08-20: adding the xhci stage moved the sink arm's drop out of the data phase into the MOUNT, and the arm reported mount-fail, a plausible word rather than a nonsense one. Keyed to dCBWDataTransferLength the same flag means the same thing whatever the guest does first, and it gives a bed a THRESHOLD for an instrument to reproduce: with -usb-bot-drop-len 16384 -usb-bot-drop-len-max 16384 the sink ladder must answer rung=32, and at 8192 it must answer rung=16. Two arms whose answers move with the bed are what separate an instrument that measures from one that always says the same thing. WRITES ONLY, and that is load-bearing: the first version refused reads too, the mount reads more than 16 KB, and the stage under test reported no-medium instead of a threshold. |
-usb-bot-drop-len-max <M> |
0 (no upper bound) | Bound -usb-bot-drop-len above, so the lever refuses a BAND of command sizes rather than everything from N upward. It exists because the diagnostic bank writes 32,768-byte commands of its own: without it, every threshold small enough to refuse a rung also refuses the bank, the bank dies before the stage runs, and the row reads no-medium. With N == M the lever refuses exactly one command size, which is what a control wants. It cannot rescue an ordinal-keyed arm whose target is the same size as the bank's writes: the 2.7 MB sink stage writes 32,768-byte commands and so does the bank, so no threshold separates them, and diag-arm.ps1's sink-drop stays ordinal-keyed for that reason. |
-usb-bot-revive-on-reset |
off (the death is LATCHED) | Pairs with -usb-bot-die-len / -usb-bot-die-lba, which kill the BOT target at a write of a given size at or above a given LBA and leave it dead for the rest of the run. With this switch the target ANSWERS AGAIN after a Bulk-Only Mass Storage Reset, and the die trigger is SPENT rather than merely lifted. It keys on the BOT class reset and not a port reset because that is the one the driver issues: measured off sink-dies, the guest sends four Mass Storage Resets after the death and no port reset at all. Without the trigger being spent the driver retries the write that killed it, the length and LBA still match, and it dies again forever. This is what makes msc-cell-retry reach msc-retry-ok (3): the latched default can only ever reach msc-retry-failed (2), so the driver's recovery had a bed for its failure and none for its success |
-usb-bot-die-on-nic |
off | The BOT target stops answering FOREVER at the first bulk WRITE issued after the e1000 model sees the first observable of NIC bring-up. Latched like -usb-bot-die-lba, and spent by -usb-bot-revive-on-reset in the same way. It injects a symptom, not a mechanism, exactly as -usb-setcfg-fault does: nothing here claims to know why a part would behave this way, and no reading taken under it is evidence that it does. It exists because the other two medium-death levers key on a property of the WRITE (its length, its LBA) and so can only express "the medium dies at a certain kind of write"; the candidate sitting 11 raised is that the write is innocent and its TIMING is the whole of it, which no length and no LBA can separate. Arms on either the CTRL write that sets SLU or the RCTL write that sets EN, whichever comes first, and the stderr line NAMES the register rather than assuming one: measured 2026-08-21 on the diag ladder it is RCTL.EN, written by nicinit. Its arm is nic-kills-msc and its control is b3-pass |
-usb-no-unit-attention |
off (the condition is ON) | Restore the old always-ready storage target. By default the device now presents the power-on UNIT ATTENTION every conforming SCSI target presents: the first command after a controller reset answers CHECK CONDITION with sense key 0x06 / ASC 0x29, and the condition persists until REQUEST SENSE reads it. A host that skips that handshake sees its first real command fail on real hardware and used to pass here; msc-wait-ready's retry loop had never executed. The condition is armed in the RESET path, not at init, because the guest issues HCRST during bring-up and would wipe it -- a sabotage arm that should have failed and did not is what found that. |
-usb-disk-port <N> |
1 | Carry the mass-storage device to root port N; its old port goes dark rather than answering as well. Pair with -xhci-ports to reproduce a device sitting where no reader can see it. No bed could put a connected device above root port 7 before this: the model had four ports, and QEMU refuses attachment above its eighth whatever HCSPARAMS1 claims. The ASUS answered with the boot stick on port 9, past the probe's eight PORTSC rows, so a count of connected ports named none of them. -xhci-ports 26 -usb-disk-port 10 reproduces the board's port=9 speed=4. Before believing a wide-port PASS, run the arm that must FAIL: -xhci-ports 8 -usb-disk-port 10 seats the disk past the declared port count, so the walk's own bound cannot reach it and the run has to report connect=FAILED (or ok=0 on a usb-attach probe). Without that third arm a passing wide-port run is indistinguishable from a flag the emulator ignored. Better still, make the probe PRINT the port the walk settled on, so a vacuous run says port=0 and convicts itself. |
-usb-cfgval <N> |
1 | The mass-storage device numbers its configuration N and refuses any other value with a STALL (USB 2.0 9.4.7 makes a bad configuration value a request error, and a control pipe reports one by stalling). bConfigurationValue is not an index and is not obliged to be 1. Use it on any driver that sends SET_CONFIGURATION: msc-open-endpoints sent a hardcoded 1 while every sibling driver read descriptor byte 5, and no bed could refuse it -- this model reported 1, and QEMU's usb-storage reports 1 and then accepts anything. |
-usb-setcfg-fault <N> |
0 | The mass-storage device answers SET_CONFIGURATION with completion code N whatever value is sent: 6 STALL, 4 USB Transaction Error. Reproduces the ASUS 2026-08-03 reading (connect=FAILED, rung 2) on the desk so the host's handling of a refusal can be built here. It injects a symptom, not a cause. Never use it to confirm a diagnosis -- a check fed a fault you selected will agree with you. |
-xhci-calibrate-periodic |
off | Skew the model's EXPECTED Interval and Max ESIT Payload by one, so a correct driver is reported as MISMATCH. The periodic value check runs always and reports once per slot and DCI on stderr; this is the arm that shows it can say no. Use it whenever you are about to believe a MATCH. |
-xhci-psi |
off | Declare Protocol Speed ID dwords in the Supported Protocol capability (PSIC = 4) and report NON-DEFAULT Port Speed values in PORTSC: full speed as 5, low as 6, high as 7, super as 8. Table 7-13's familiar 1/2/3/4 are only the defaults, and xHCI 7.2.2.1.1 makes them conditional on PSIC being zero. Use this on any driver that decides a speed CLASS from the PORTSC speed field. The model reports the true class and the slot context's claim side by side on stderr. |
-xhci-intel |
off | Present the xHCI as an Intel Lynx Point PCH (8086:8C31) with the USB2 ports still owned by the companion EHCI. XUSB2PRM reads 0xE (ports 1-3 routable), XUSB2PR resets to 0, and an unrouted port reads PORTSC as all zero -- dark, not merely disconnected, which is what a port left with the companion looks like to the xHCI. The guest must route the ports before it sees any device on them. Writes to XUSB2PR are traced on stderr. |
-xhci-intel-lock |
off | As -xhci-intel, and additionally make XUSB2PR read-only, modelling a part whose ports firmware pinned to the companion. A correct driver then finds nothing, which is what proves the routing gate rather than something else is deciding. |
-xhci-csz |
off | Advertise CSZ=1 (HCCPARAMS1 bit 2) and hold every context to the 64-byte stride, the way Intel PCH silicon does. Every bed before this flag reported CSZ=0, so the driver's 64-byte context path shipped in every image and executed nowhere but on real Intel parts. A driver that hardcodes 32 writes its slot context inside the input control context and its ring pointers where the controller will not look. |
-xhci-scratch <N> |
0 | Declare N scratchpad buffers in HCSPARAMS2 and REFUSE the first ENABLE_SLOT (completion code 9, stderr verdict) unless DCBAA[0] points at a 64-byte-aligned array of N page-aligned, in-RAM page pointers. QEMU and this model declared zero forever; Intel parts demand real pages, and a missing array is silent corruption on metal. |
-no-hpet |
off (the HPET answers) | The HPET window at 0xFED00000 is dead: every register reads zero and every write is dropped. What decides is the capability period at offset 04, which a guest divides into a second to get its tick rate, so a zero period is the box saying it has no clock to offer. Hpet.codex turns that into a rate of zero, and E1000e and NicAsde above it fall back to counting reads instead of clocking a deadline -- the path every network timeout on a box without an HPET actually takes, and one that had no bed arm at all until 2026-08-19. Four readers in the diagnostic ladder say so under it (nicsit, nicinit and nicring state no-hpet, and the scene stage's frame row reads plain=no-clock), which is the nic-nohpet arm. The xHCI MFINDEX is unaffected: that runs off host time inside the model rather than off a register the guest was offered. |
-hpet-frozen |
off (the HPET answers) | The HPET window is there and UNDECODED: every register reads all-ones and every write is dropped, which is the other dead clock and the one -no-hpet cannot express. A period of 0xFFFFFFFF derives a rate of 232830 Hz, bogus but nonzero, so every reader takes its CLOCKED wait path, and the counter at F0/F4 is 0xFFFFFFFF and never moves, so each of those waits is bounded only by its read fuel (e1000-await-link-clocked is 100000 batches of 4096 STATUS reads). The diagnostic ladder's b3 stage reads the counter across 100000 reads at entry and answers clock-stuck before bring-up; that is the b3-clockstuck arm, whose cfg turns the three nic stages off so nothing ahead of b3 spends its fuel on the same clock first. (root, 2026-08-21, the shape blu asked for.) |
-no-smbios |
off (SMBIOS is published) | With -uefi: leave the SMBIOS entries out of the ConfigurationTable, so a guest that goes looking is told "none offered". The arm for a reader's no-table state; the diagnostic ladder's smbios stage answers no-table and its box row unnamed under it. |
-no-edid |
off (an EDID is offered) | With -uefi: LocateProtocol answers NOT_FOUND for the EDID protocols. The ladder's edid stage answers absent. |
-edid-bad |
off | With -uefi: the offered EDID has a wrong checksum byte. The ladder's edid stage answers bad-checksum, which is what shows the checksum is read rather than assumed. |
-uefi-conout-remode |
off | With -uefi: the first ConOut ClearScreen switches the GOP to 1024x768 stride 1024 and updates Mode->Info, modelling AMI Aptio V's GraphicsConsole activation. A stub that reads the GOP geometry BEFORE its first ConOut use hands its payload the splash mode's numbers for a scanout that has since changed, which is the ASUS display corruption of 2026-08-02 exactly. |
-xhci-evt-flood <N> |
0 | Post N extra Port Status Change events at the Run transition, on top of the per-port connect and reset-completion PSC events the model now always posts. The event ring is 64 TRBs: N at or above it marches the producer through the ring WRAP and into the FULL condition (drop + one stderr report, per real silicon), the two paths a 4-port bed can never reach and a 26-port Intel reaches before enumeration begins. Both paths measured PASS on the shipping driver at N=30/50/100. |
-hid-nak |
off | The HID keyboard NAKs every interrupt IN forever: no DMA, no transfer event, the pending TD stays in progress and Stop Endpoint answers FSE code 26 (Stopped) with the residual. Reproduces the ASUS 2026-08-03 flight signature (EPINT=0, dq parked, est=1, f1=1a) on the desk; the arm every silent-keyboard hypothesis is tested against. |
-hid-idle-quirk |
off | The HID keyboard NAKs every interrupt IN after the guest sends SET_IDLE duration 0 (the over-honored "report only on change", HID 1.11 7.2.4 against the F.3 every-poll default). A driver that skips SET_IDLE never triggers it; one that sends it goes silent -- the arm that separates the v15 fix from the v14 behavior. |
-hid-root-silent |
off | The root-port keyboard completes every interrupt IN with SUCCESS and eight zero bytes (GET_REPORT answers zeros too) while the hub-attached keyboard carries the injected keys. The ASUS TUF 2026-08-04 desktop signature -- SUCCESS on the driver's own endpoint, buffer untouched, key held -- which -hid-nak cannot produce (it models no event at all). The arm the bind-every-keyboard fix is proven against (usb-kbd-silent). |
-hid-keys |
off | Scripted keys (-keyt/-keys-file) feed ONLY the USB HID held-key set: no PS/2 queue, no host-side key-cell write. Without it injected keys reach the guest through the PS/2 emulation whatever the USB stack does, so no bed run could ever prove a scancode travelled the interrupt-IN DMA path. The honest model of the ASUS, which has no PS/2 controller at all. |
-hid-combo |
off | The root HID device carries TWO interfaces: boot keyboard on EP1 IN and boot mouse on EP2 IN -- the wireless-dongle topology. Boot mouse reports are built as deltas from the -mouse timeline's absolute samples (clamped to the signed byte the protocol carries), so a scripted pointer drives the USB pipe exactly as scripted keys do under -hid-keys. The arm the per-interface classification is proven against (usb-hid-combo): a whole-device classifier hands the combo to the keyboard driver and the mouse half is unreachable. |
-hid-nak-unchanged |
on | HID interrupt IN endpoints NAK until they have news: a TRB completes only when its report would differ from the last one delivered on that endpoint (the combo mouse: only when a sample arrived since the last report); otherwise the TD stays pending and is re-rung when input state changes. The default since 2026-08-06 -- the flag is still accepted so existing sidecars keep parsing, and passing it is now a no-op. The arm the per-endpoint event latch is proven against (usb-hid-steal). CODEX_VM_HIDNAK_TRACE=1 restores the HIDNAK: narration, which used to appear whenever the flag was passed and would otherwise now be on every bed's stderr. |
-hid-instant-complete |
off | Restores the pre-2026-08-06 default: every interrupt IN TRB completes at doorbell time with whatever the input state is then. Completions become so plentiful that a defect fed by their SCARCITY -- one endpoint's waiter consuming a sibling's rare completion and starving it, the Unifying-receiver mouse failure of 2026-08-04 -- cannot be expressed, which is the reason to reach for it. The cost is that it drops any keystroke narrower than the guest's poll interval, silently; see the scripted-input section below. |
-uefi |
off | UEFI firmware mode (ConOut/ConIn, GOP, Block I/O, memory map, runtime services, auto-extract PE from GPT images) |
-gop |
off | Activate GOP framebuffer (default 640x480) |
-gop-width <N> |
640 | GOP framebuffer width (implies -gop) |
-gop-height <N> |
480 | GOP framebuffer height (implies -gop) |
-gop-max-mode <N> |
-- | Cap the GOP mode table's MaxMode. The UEFI GOP enumerates modes 0..2 (640x480, 800x600, 1024x768) plus, as mode 3, the -gop-width x -gop-height you set when it matches none of them, so a bed at 1600x900 is a firmware whose largest mode is what the display supports; Mode->Mode names the current one. -gop-max-mode 1 is a firmware with nothing to enumerate, the fallback arm for a stub that picks the largest mode (build/gop-mode-arm.ps1). QueryMode answers into a scratch info block, not the current mode's; a mode past MaxMode is EFI_INVALID_PARAMETER; SetMode commits the framebuffer before it clears it (a runtime mode set from a headless boot faulted the HOST until it did). |
-smp [N] |
1 | Enable multi-core: N virtual processors (1-16, default 4 if N omitted). Creates WHP VPs, LAPIC, MADT with per-core entries. Core count written to GPA 0xFF8; boot code reads it to decide whether to send INIT/SIPI. |
-portfwd [udp:]<host:guest> |
-- | Port forwarding from host to guest NIC (repeatable, max 8). TCP by default; udp: forwards datagrams instead, giving each host client a synthetic gateway source port so the guest's replies route back. Examples: -portfwd 8080:80, -portfwd udp:15683:5683 |
-natmap <guestdest:hostport> |
-- | Remap an OUTBOUND destination port (repeatable, max 16, TCP only). The opposite direction to -portfwd: when the guest dials guestdest, the NAT connects to the host on hostport instead of the port the guest asked for. Exists because a plug's port is compiled into it from build/plug-ports.ps1, so N copies of one plug all needed the same host listener and could not run at once. With this, each worker owns a private host port while running the same unmodified plug binary -- which is what lets codex/plugs/recheck/sweep-all.ps1 go N-wide. Unmapped ports are untouched, so every existing invocation means what it always did. Example: -natmap 9134:9250 |
-debug |
off | Interactive debugger shell on breakpoints and single-step |
-break <name> |
-- | Patch INT3 at named function entry (implies -debug, repeatable) |
-map <file> |
auto | Symbol map file for address resolution. Auto-probed: <kernel>.map, then seed/Codex.map |
-watch <0xADDR> |
-- | Hardware watchpoint via page protection |
-watch-size <N> |
8 | Watchpoint region size (max 64 bytes) |
-hbreak <fn>[:<reg>=<val>] |
-- | Conditional execution breakpoint through the debug registers (repeatable, max 4, one DR each, shared with -wcet; needs -map). The condition is evaluated at the host's #DB exit on the guest's own register file. Writes no guest byte, and an unmatched hit RESUMES instead of halting, so this is the way to reach the Nth call of a hot helper. Prints HBREAK-OBS: <fn> hits=<n> matched=<m> on exit -- both counts, always, because a run reporting only matches cannot be told from one whose breakpoint was never reached. A conditional breakpoint on INT3 cannot work in this guest and -break's is not offered: vector 3 belongs to the guest, whose handler runs before the host sees the trap, so the condition would be read off the handler's registers (rdi is 0x33 every time) |
-hwwatch <0xADDR> |
-- | Data watchpoint through DR0, trapping the exact linear address regardless of guest paging, so it does not collide with the guest's own demand-paging #PF handler the way -watch's page-protection exits do. -hwwatch-rw widens it to reads, -hwwatch-len <N> sets 1, 2, 4 or 8 bytes. It did not fire host-side until 2026-07-28: DR0 was armed correctly but nothing enabled ExtendedVmExits.ExceptionExit, so the #DB was delivered to the guest IDT, which dumped !EXC=01 and halted on the first hit. Any conclusion drawn from an older -hwwatch run saw at most one write |
-hwwatch-log |
off | With -hwwatch, print one line per write and continue, instead of a full crash report. This is what names a corrupter: it is the SECOND write to a cell that matters, and the crash report is both too heavy and, in practice, terminal |
-wcet <name> |
-- | Observe a function's per-invocation dynamic instruction count (repeatable, max 4 -- one DR0-DR3 exec breakpoint each; needs -map). Prints WCET-OBS: <fn> max=<n> calls=<k> on exit. Observation only: no guest byte is modified. |
-mouse <script> |
-- | Scripted pointer: t:x,y,btn events separated by ; (t = ms from boot, btn bit 0 left / 1 right / 2 middle). Injected straight into the guest, so no host cursor moves and no window takes focus. Works headless. The x,y are NOT screen coordinates. The host tracks a position starting at 0,0, each event's x,y sets that tracked position, and what reaches the guest is the DELTA from the previous event, clamped to +-127 per sample (a HID boot mouse reports one signed byte per axis). Negative values are accepted and are how you move left or up. So a guest pointer that starts centred at 800,450 reaches 420,857 with four samples of about -95,+102 each, ending at -380,407, and a single 420,857 event moves it 127 right and 127 down instead of putting it there. Cost several boots to find; see the worked walk in ModernDesk.md stage 14. |
-mouse-file <file> |
-- | Same, read from a file (one event per line, # comments). Use for drags, which run to dozens of samples. |
-keys-file <file> |
-- | Timeline keyboard: t:scancode per line, on the same clock as -mouse. Lets a script interleave typing with clicks (the older -keys fires on a fixed start+interval). The event separators are ; and newline ONLY. inject_keyt_parse skips leading ;, newline, \r, space and tab, but after each event its trailing skip runs to the next ; or newline, so a comma-joined timeline silently injects its FIRST event and discards the rest of the line with no error printed. Cost four boots to find. Re-derive any probe arm that rode a multi-key timeline before building on it. |
-rtc <stamp> |
host clock | Freeze the emulated CMOS RTC at YYYY-MM-DDTHH:MM:SS (the T may be a space). Day-of-week is computed from the date, not accepted. This is what makes a guest that paints the time comparable against a recorded frame -- without it the clock is host state the test cannot twist, which is why GuiOS was believed to be un-goldenable. It also turns the update-in-progress simulation OFF (a frozen clock cannot express UIP), so it is for frames and never for testing the RTC itself: anything asserting on clock behaviour must run without it. It stops the HPET main counter as well, pinned to (minute*60 + second) * HPET_HZ so the two clocks tell the same time and moving the stamp forward a second moves an HPET-driven animation forward a second. That was added when the desk's 3D orbit moved to the HPET: a guest animating off a hardware counter is exactly as uncomparable as one painting the wall clock, and freezing only the RTC would have left every existing capture recipe reproducing a different frame each run. Frozen-clock frames across that change were required to be byte-identical, and were. The counter therefore does not advance under this flag, so anything that WAITS on the HPET must run without it -- today that is E1000e and NicAsde, and no test sidecar passes -rtc. |
-screenshot <file> |
-- | Save GOP framebuffer as BMP on exit |
-screenshot-delay <ms> |
0 | Delay before screenshot capture. Measured 2026-08-15 on a build-option-a UEFI probe image: 0 and 2000 capture; 5000 and 9000 give code=-1 and NO BMP. RE-MEASURED after the GOP rewrite at main 15469 and unchanged, which is worth saying because that rewrite touched QueryMode, SetMode and the framebuffer commit and would reasonably be assumed to have invalidated this row. The 120000 recommended above is for the DeskVm BMP-write path and was not re-measured here. Verified with a control, because the first reading looked like a broken payload: the KNOWN-GOOD image fails the same way at the same delay. If a capture goes missing, bisect the delay before suspecting the guest. |
-args <string> |
-- | Boot arguments string (accessible to guest) |
-trace-file <file> |
-- | Write execution trace to file |
Environment: CODEX_VM_NO_TIMER=1 disables PIT timer interrupts.
codex-vm -run-list list.txt [-run-list-wall MS]
One line per run, each line carrying exactly the flags that run would take on
its own command line. Tokens are whitespace-separated and may be double-quoted,
so a path with a space survives; # comments and blank lines are ignored.
-kernel beside -run-list is REFUSED rather than ignored, so the two modes
never look combinable.
It spawns a fresh codex-vm child per line. It does not reuse the process.
That is the whole design and it is worth knowing before anyone tries to
"optimise" it: a batch is then byte-identical to N single runs by construction,
because each line gets a genuinely new process, not a partition delete and a
hopeful reset. Measured 2026-08-22, one kernel on a twelve-core box:
| path | per kernel |
|---|---|
pwsh child running build/test-run.ps1 |
574.8 ms |
codex-vm.exe directly, fresh process each |
73.7 ms |
bare codex-vm.exe process start alone |
12.6 ms |
The pwsh child is 501 of the 575, so what a batch mode has to remove is the
SCRIPT per test and not the exe. Reusing one process would buy the 12.6 ms and
require resetting 376 file-scope statics between runs: deleting a partition
does not reset them, because the device models are host state, and one missed
static makes a test's result depend on what preceded it in the batch.
Per line it writes RUN-LIST BEGIN [i/N] <kernel>, then that child's stderr
verbatim, then:
RUN-LIST END [i/N] <kernel> exit=<code|TIMEOUT|SPAWNFAIL> output=<bytes> dropped=<n> ms=<elapsed>
output is the child's own Output: N bytes line, or -1 when it wrote none.
dropped is read from that child's SERIAL: N guest serial byte(s) DROPPED
lines, so L-SHORT's drop signal survives batching and is attributed to the
kernel that dropped it. It is keyed on that whole phrase and not on the word
DROPPED, which the GPU triangle-cap warning also prints.
Do not read exit as a verdict. A healthy test exits
(debug_exit_code << 1) | 1, so 1 is the normal answer and build/test-run.ps1
has always ignored it. The supervisor's OWN exit code is 0 when it ran the
list and 1 when a line timed out or could not be spawned.
The per-kernel wall budget is the same 60 s test-run.ps1 keeps. A timeout
signals the child's own Global\CodexVmShutdown_<pid> event first and only
falls back to TerminateProcess, because killing a child mid-hypervisor-call
is what corrupts vid.sys. -run-list-wall MS overrides the budget: it exists
so the timeout path can be exercised at all, and build/check-run-list.ps1
uses it at 2000 ms.
Sidecar preparation stays with the caller. test-run.ps1 copies a depot
-disk image to a writable temp because a synced sidecar is read-only; a list
line carries the FINAL paths, exactly as they would be passed to a single run.
build/check-run-list.ps1 is the runner: byte-identity against single runs
(with an ablation proving that comparison can fail), a corrupt kernel mid-list
not taking its neighbours, the wall budget stopping one line alone, and the
drop count landing on its own line with the GPU line as a negative control.
There is no -timeout flag, and an unknown flag is ignored in silence.
Measured 2026-08-11: -timeout appears zero times in tools/codex-vm.c, so
-timeout 20 parses as two arguments the loop above matches nothing against
and drops. The run is UNBOUNDED. Every recipe in this file that carries one --
including the disk-image reproduction below -- ended when its guest crashed,
which is why nobody noticed; point one at a guest that does not crash and it
runs until the host kills it. The wall budget lives in the harness instead:
build/test-run.ps1 enforces $wallBudgetMs. For a bounded run by hand, use
-screenshot <file> -screenshot-delay <ms>, which exits after the capture and
gives a visual record as well. This cost a browser soak an hour of wall clock
on 2026-08-11, and the trap is the shape of it: the flag looks accepted, the
command works, and it is only wrong on the runs you most want bounded.
CPU and SMP. WHP-accelerated x86-64 (long mode, full hardware
virtualization). Shadow register file works around WHP GPR corruption.
Multi-core via -smp N: each AP gets its own WHP virtual processor
and host thread. INIT/SIPI startup sequence: the guest copies a real-mode
trampoline into the page at GPA 0x1000, seeds the core-id counter at cell
36256, writes the per-core stack addresses to a stack table at GPA 0xF00,
then writes the LAPIC ICR -- an INIT IPI followed by two start-up IPIs
carrying vector 1, delivered to all cores excluding self. That ICR
write is what launches the APs.
An AP starts the way silicon starts one, at vector<<12 in real mode
with reset control registers, an empty GDT and nothing in RDI. Everything
after that is the guest's trampoline: protected mode, PAE, CR3, EFER, long
mode, a core id taken with a locked exchange-add on cell 36256, a stack
claimed from the table by that index, the runtime GDT and IDT, its own task
register, and one added to the ready count at cell 4080. The BSP spins on
that count (bounded -- it gives up and continues single-core rather than
hanging) before carrying on.
The vector field is the only thing an AP is told, here as on a physical
machine. docs/ArchitectsSketchbook.md has the three guest-side
requirements that follow from it.
An AP runs processes. It goes to __idle_dispatch, the same routine
the boot processor goes to when it runs out of work, claims a READY slot
out of the process table with a LOCK CMPXCHG, and resumes it -- on that
process's own stack, with that process's own R10. Six children spawned at
-smp 4 execute across four cores. codex/test/smp-dispatch.codex pins
it, and pins the right thing: every claim by a core whose id is not
zero bumps cell 36200, and the boot processor's id is always zero, so a
count above zero is evidence a core other than the BSP took a process out
of the table and executed it. Six children finishing would prove
nothing -- one core does that.
A process on an AP is preempted. The PIT's IRQ reaches the boot
processor alone, so each AP arms its own LAPIC
timer at bring-up -- periodic, on vector 48 -- and raises IF, so a
scheduling tick arrives on every core. codex-vm emulates the timer
per-core (lapic_timers[]) and injects the vector from each AP's own
thread; the kicker cancels the AP's VP every PIT period, because a
compute-bound core never leaves WHvRunVirtualProcessor on its own and a
tick that is never injected is not a clock.
codex/test/smp-preempt.codex pins it, and pins the right thing: every
timer interrupt taken on a core whose id is not zero bumps cell 36216, and
the BSP's id is always zero. Six children finishing proves nothing here
either -- and neither does six children finishing on six cores, because an
AP that runs its child to completion undisturbed has still never been
preempted. The claim is about who can be interrupted, so the evidence is
about who was interrupted.
An idle core halts rather than spinning on pause, and affinity is
honoured: __idle_dispatch skips a process whose affinity field names a
different core, -1 meaning any. codex/test/smp-halt.codex and
codex/test/smp-affinity.codex pin the two. Work stealing exists in the OS
scheduler model but not in the bare-metal dispatcher, which scans a shared
process table rather than per-core queues.
Proc 0 does not migrate. Three guards forbid it: __idle_dispatch starts
each core's scan at its own id so an AP never reaches slot 0, and both
preemption scans skip slot 0 when the claiming core is not the boot
processor. Its affinity field says 0 to match.
codex/test/smp-proc0-pinned.codex
pins it, and pins it in the only way that means anything: slot 0's core stamp
must still be the boot processor's after a run in which an AP demonstrably
claimed a process, an AP was demonstrably preempted, and some other slot's
stamp is demonstrably above zero.
An SMP test must assert that an AP executed guest code, not that the
work finished -- one core can finish the work. smp-cores.codex reads
back the ready count, which only an AP ever increments;
smp-dispatch.codex reads cell 36200, which only a core whose id is not
zero ever bumps. Both are evidence a different core ran. Before those
tests existed, codex-vm was launching the APs from the host without ever
creating a thread to run them and incrementing the guest's ready count on
their behalf: no AP had executed a single guest instruction, and nothing
in the battery could tell.
An AP can be heard. Each application processor has its own TSS and
IST1 emergency stack, so a fault on one produces the standard !EXC dump
(tagged with its core id in R15) instead of a silent triple fault. codex-vm
serves COM1 only on AP threads: handle_io() drives stateful devices
through VP 0's shadow register file and is not safe to re-enter from
another thread. An AP has no business touching the IDE controller; it has
business reporting that it died.
Do not step RIP past an MMIO instruction by a guessed length. WHP
does not hand you the length: VpContext.InstructionLength reads 0 on
these exits and MemoryAccess.InstructionByteCount is the size of the
16-byte instruction buffer, not the instruction. Both were taken as
lengths at various points, and both resume mid-instruction -- the
signature is a !EXC=06 (invalid opcode) at an address a byte or two
inside a real instruction. mmio_insn_len() decodes the bytes WHP
provides, and returns 0 rather than guessing when it cannot.
LAPIC (0xFEE00000). Per-core local APIC: ID register, SIVR, EOI, ICR (lo/hi). SIPI delivery creates AP threads. Used for SMP boot and inter-processor interrupts.
IOAPIC (0xFEC00000). 24 redirection entries. IRQ routing from PCI devices and ISA sources to the BSP.
PCI Bus. 3 devices on bus 0:
| Slot | Device | Class | BAR | IRQ |
|---|---|---|---|---|
| 0 | Bochs VGA (1234:1111) | Display | 0xFD000000 | -- |
| 1 | xHCI USB (1033:0194) | USB 3.0 | 0xFE800000 | 10 |
| 2 | Intel HDA (8086:2668) | Audio | 0xFE000000 | 11 |
Config space read/write via ports 0xCF8/0xCFC.
Display. Three modes:
- VGA text (80x25) via port 0x3D4/0x3D5, text buffer at 0xB8000
- Bochs VBE via ports 0x1CE/0x1CF (index/data), guest-initiated mode switch
- GOP framebuffer at GPA 0xBF000000 (in RAM -- fast writes, no MMIO trap). Guest writes 32-bit XRGB pixels directly. The VM renders from a shadow copy to a Win32 window on a separate thread. Three preset modes: 640x480 (0), 800x600 (1), 1024x768 (2). Custom sizes via
-gop-width/-gop-height. Window title changes to "Codex Spark" in GOP mode.
GPU Triangle Rasterizer. Host-side native-speed rasterizer accessed via I/O ports 0x400-0x40F:
| Port | Direction | Function |
|---|---|---|
| 0x400 | OUT | Rasterize N triangles from guest command buffer |
| 0x401 | OUT | Clear framebuffer to XRGB color |
| 0x402 | OUT | Clear depth buffer |
| 0x403 | IN | GPU rasterizer present (returns 1) |
| 0x404-0x406 | OUT | Set light direction (x/y/z, fixed-point /1000) |
| 0x407 | OUT | Set eye direction X (Y/Z copied from light) |
| 0x408 | OUT | Set texture guest address |
| 0x409-0x40A | OUT | Set texture width/height |
| 0x40B | OUT | Commit texture upload, and the value IS the wire format (0 or 1) |
| 0x40C | OUT | Asset load: guest address of a null-terminated host path |
| 0x40D | OUT | Asset load: guest destination address |
| 0x40E | OUT | Fade-clear framebuffer toward an XRGB color |
| 0x417 | OUT | Asset load: execute (reads the host file into guest RAM) |
| 0x40E | IN | Asset load: low 32 bits of bytes loaded |
| 0x40F | IN | Asset load: high 32 bits of bytes loaded |
The asset-load execute is 0x417 and not 0x40E, which the OUT chain matches first as the fade-clear: an asset load fired at 0x40E silently faded the sky and reported zero bytes read. 0x40E OUT (fade) and 0x40E IN (size) do not collide, so the size read-back keeps its port.
Includes depth buffering, per-vertex normals, diffuse+specular lighting, texture mapping with bilinear filtering, procedural Earth texture generation, and atmospheric glow post-processing.
The commit value at 0x40B declares the texture, not just the doorbell, and
the two modes are not interchangeable. 0 is the original wire: three packed
RGB bytes per pixel, which is what apps/globe/TerrainGen.codex writes with
poke-byte, sampled bilinear with both axes inverted and shaded by the
earth-globe shader. 1 is one 32-bit 0x00RRGGBB word per pixel, which is
what a Codex EngineTexture holds and what gpu-mem-write writes, sampled
nearest with both axes wrapping and shaded by a plain modulate of the
interpolated vertex colour, matching Renderer3D's r3d-tex-px.
The globe shading is not a filter that can be left on. It reads the UVs as
SPHERICAL COORDINATES, fabricates a per-pixel sphere normal from them, lights
that, and adds a Fresnel atmosphere rim worth up to +140 blue. On a ground plane
it turns a correctly sampled tan texel (6E5F4B) into 405A9C, a blue that
looks like some other object's albedo rather than like a shading bug, which cost
four wrong hypotheses before a probe was pointed at it.
A viewport-confined frame is double buffered; a full-screen one is not. A frame arrives as a clear on one port and a draw on another, and the rasterizer writes into the GOP framebuffer the display thread is reading, so between those two writes the target holds nothing but the clear colour. Measured on the desk pane at the instant between them: 1,123,200 of 1,123,200 pixels, every frame. That window is what a headless capture occasionally returns as a pane of pure sky with the label still drawn beside it, and what reads on the glass as the pane flickering while it animates.
So when a viewport is armed the frame is built in a host back buffer and presented in one copy at the end of the rasterize; the same reading afterwards is 333,663, which is the previous complete frame's real sky. A program that owns the whole screen has no viewport armed and still draws directly, so it keeps this window -- the globe demo will hand back a bare-background frame now and then for exactly this reason, and that is the capture, not the renderer.
GPU_SHADOW_SLOPE is 16 and the number was swept, twice. The first sweep
used a CUBE FACE and settled on 6; flat geometry cannot show what a curved
surface does, and a sphere in the desk scene still carried visible acne at that
setting. Re-swept measuring acne as the sphere's difference from the same frame
with shadows off, against the ground shadow measured the same way: 6 gives 350
acne pixels and 132,287 of shadow, 16 gives 0 and 131,691, 24 gives 0 and
131,200, 40 gives 0 and 130,137. 16 is the knee rather than the largest value
that works. The software renderer reads 0 acne at every setting, so this was the
host path alone off parity.
Single-pixel rasterizer probe. Set CODEX_GPU_PROBE=x,y and codex-vm prints
one line per triangle that covers that screen pixel, whether it wins or loses the
depth test:
GPUPROBE f=8 tri=0 tex=1 uv=(0,0 1000,1000 1000,0) c=(C2C0B8 C2C0B8 C2C0B8)
interp=(716,303) texel=6E5F4B mode=1 d=982321 pixel=534736
It answers "which triangle drew here, from what UVs, sampling what, and what did it emit" in one run, and it costs nothing when the variable is unset. It is the right first instrument for any "wrong colour in the pane" question: a census of the frame tells you a colour is wrong, and only this tells you which triangle and which stage produced it.
The same switch prints one line per rasterize carrying the triangle count, the screen and depth extents of the whole submitted batch, and how much of the VISIBLE framebuffer is currently the clear colour. The extents separate "the guest culled everything" from "the guest submitted geometry that landed somewhere unexpected" without a rebuild, and the last figure is what turns an intermittent flicker into a deterministic reading.
Serial I/O. Ring buffer at GPA 0x500000 (1 MB). Source input is
pre-loaded from -input file. Output captured from guest UART writes
to -output file. Ports 0x3F8-0x3FD (COM1). Protocol: guest reads
input from ring buffer; writes output bytes; harness captures until
EOT or VM exit. The capture buffer starts at 16 MB and doubles; if a
doubling fails, every byte that does not fit is COUNTED, the first prints
SERIAL: output buffer growth failed at N bytes ... on stderr and the exit
dump adds SERIAL: K guest serial byte(s) DROPPED ... <file> is SHORT
(2026-08-18; before that the drop was silent and -output was simply
short with both ends reporting success). Exit status is unchanged by a
drop, so a harness that wants to refuse a short capture greps stderr for
SERIAL:. The BLIT path (bulk output) reports its own growth failure as
BLIT: output buffer growth failed.
A -input image needs a TRAILING ZERO BYTE after the source, so the
image is header, source, then one 0x00. Without it the run produces exactly
one byte of output, 0x01 -- the leading marker with nothing behind it -- and
that reads as the compiler dying rather than as an empty read, which is a
diagnosis two steps away from the cause. codex/plugs/wasm/build-page.ps1
builds header + src + 1 and is the worked example. The expensive half is
that wasmtime does not care, because fd_read's zero return is its own
terminator: the same malformed image runs clean on the wasm target and fails
on x86, so the two disagree in the direction that makes wasm look healthy and
the emulator look broken. (Distinct from the trailing 0x04 in the
two-compiler comparison recipe further down, which is a mode-line protocol for
that procedure and does not cover this.)
The compile payload has three ways out, and the guest picks by probing
(root, 2026-08-21). __write_binary and __write_binary_buf first read port
0x511: 0xB7 means codex-vm and the whole buffer leaves by the blit doorbell in
one exit. Otherwise they read port 0xE9: 0xE9 read back means QEMU's Bochs
debug console (isa-debugcon) is wired, and the whole buffer leaves as ONE
rep outsb to that port, no LSR poll and no 16-byte FIFO bound, because the
console is a byte sink with no status register. Anywhere else the port floats
0xFF (QEMU without the device, codex-vm itself, real hardware) and the COM1
loop runs as before: poll LSR once, burst up to 16 bytes. Only the QEMU compile
fallback in vm-config.ps1 (Invoke-VmCompileFallback, generated from
codex/build/vmconfigScript.codex) wires the device, to a file chardev, and
it accepts the payload from EITHER channel: complete means the console holds
SIZE:'s byte count, or holds nothing and the serial wire does. A kernel older
than the probe (the gate's stage0 is the depot seed) still answers on serial and
is read unchanged; a console holding some bytes but not all is still streaming
whatever the serial count says, since a MAP trailer can outweigh a small
payload. The payload is spliced back where it sat on the wire, so compile.ps1
reads OutputFile the same way from either host, and the fallback prints
qemu fallback: N payload bytes via the debug console on stderr when that path
was taken. No other Start-VmRun consumer gets the device: each reads the
serial wire alone, and a device they did not ask for would move their payload
somewhere they do not look.
Measured 2026-08-21 on the landing, one unit (codex/test/text-helper-spec,
105 KB of output): the same kernel produces a byte-identical binary through
codex-vm's blit and through QEMU's debug console (one SHA-256 for both), which
is the claim that matters; the gate's vm-differential phase makes the same
comparison on every seed land. Wall clock under the QEMU fallback at that size
was 4.0/3.9 s with the pre-change kernel on the COM1 bursts and 3.6/4.0 s with
the new one on the console: inside noise, because the 16-byte bursts had
already taken the exits out of the serial path (blu, main 17213) and a compile
that size is boot and compile time, not output time. The path's own signature
is two in al,dx; cmp rax,0xE9 sites in the seed, one per helper: 0 in the
pre-change seed, 0 in the pre-convergence Sut, 2 in the converged seed, the
4.3a shape exactly. A larger payload was attempted (GopBoot, 1.2 MB of
output) and the PRE-CHANGE kernel's run under the fallback did not complete:
36 minutes, 19 s of CPU, no log line, and the fallback's own deadline never
fired, so the QEMU path on a unit that size has a hang of its own that this
change neither causes nor measures. Not pursued; named here so the next
person who wants the big-payload number starts there rather than at the
console.
NE2K NIC. NE2000-compatible ISA NIC at I/O base 0x300. User-mode
NAT stack with IP 10.0.2.15, gateway 10.0.2.2, DNS 10.0.2.3. Handles
ARP (responds for gateway), DHCP (offers 10.0.2.15/24), DNS, TCP
forwarding, and UDP forwarding (see below). Port forwarding via
-portfwd for host-to-guest TCP connections.
DHCP is answered. The server answers DISCOVER with an OFFER and REQUEST
with an ACK carrying mask, router, DNS and a 3600-second lease.
codex/test/dhcp-acquire is the guest that asks, and it is what keeps this
row honest: Dhcp.codex builds the messages and parses the replies but
performs no I/O of its own, so without a caller neither the model nor this
document has anything testing it.
-dhcp-lease <seconds> sets the lease the NAT offers; the default is
3600. It exists because the default cannot be tested: a client renews at
half its lease, so nothing observes a renewal inside a run that lasts
seconds. codex/test/dhcp-renew asks for four, which puts the renewal two
seconds in, and it is the only reason that test can exist.
An empty receive poll costs 540x more on the NE2000 than on the e1000, and
a guest that counts polls is counting two different things. Measured
2026-08-14, one million empty polls through net-driver-recv-frame: 15.52 s
on the NE2000, 0.029 s on the e1000. An NE2000 poll is a port IN and
therefore a VM exit; an e1000 poll reads a descriptor the model already wrote
into guest RAM, which is also what the real I219-V does. Anything in a guest
that uses a poll count as a stand-in for elapsed time is therefore calibrated
against whichever card it was written on, and NetIO's tick was calibrated
against the NE2000 -- see net-driver-poll-cell in ArchitectsSketchbook.md.
This is a property of the two device models and not of any one guest, so
expect it to bite anything else that counts polls.
The test for whether a spin count in this tree is secretly a duration, and the audit that came out of it (2026-08-14). The question is not how big the number is, it is what the loop body touches, because that decides whether the bed and the metal agree:
| the loop polls | cost per iteration | bed vs metal | verdict |
|---|---|---|---|
| a register, MMIO or port I/O | a VM exit in the bed, a bus read on metal | comparable, within a small factor | a count is a poor duration but it will not surprise you |
| memory the device DMAd (a descriptor status byte, a completion-queue phase bit, a flag another core sets) | a RAM read, no exit at all | hundreds of times apart | the count is a hidden duration and it is probably wrong |
| nothing external (a recursion or parse guard) | irrelevant | irrelevant | not a duration, leave it alone |
Measured on this box for the third row: one million iterations of a bare
peek-byte status poll cost 605 microseconds. That is the number to price
any RAM-polling budget against.
Swept 2026-08-14 over codex/** and apps/**: 36 named budgets of 1000 or
more. Almost all are honest give-up caps where the value never had to mean a
duration. The RAM-polling ones, which are the class above, are:
e1000-tx-fuel(E1000e.codex) WAS one and is FIXED. At 1,000,000 it was a 605 microsecond budget against a 1500-byte frame that takes 1200 microseconds on the wire at 10 Mb/s, ande1000-send-framereturns the result directly, so every send on a slow link would have reported failure while the frame went out anyway. Now bounded by HPET at 20 ms with the spin as the no-clock fallback;codex/test/e1000-tx-deadlinepins it with the old path as its own control, in a band that cannot overlap.arm64-net-io-tick-interval(Arm64NetIO.codex) is one and is NOT fixed, because this lane has no counter read. What is missing is written out in that chapter's own Poll Clock section, including the derivedsysreg-cntvct-el0value and the warning that half of it is seed-affecting.nvme-fuel(GopNvme.codex) polls a completion-queue phase bit, which is DMAd memory. Not measured, not mine, named here because it is the same shape.xhci-fuel(GopXhci.codex) is already open as WORKS-9, which states the problem exactly: it is a spin count nobody has converted to a duration on that box, and no bed can, because codex-vm completes every transfer before the guest spins once.smp-wait-fuel(X86_64Boot.codex) polls a RAM flag another core sets. Boot-time only and it has never failed, but it is the same shape.
ahci-fuel, ide-poll-fuel, rtc-uip-fuel and the other e1000-*-fuel
constants poll registers and sit in the first row, so they are not urgent.
The guest-to-host byte census, and how to read it. Any run in which the guest sent TCP payload prints one line to stderr at exit:
NAT TX BYTES: seg=N queued=N sent=N drop-noconn=0 drop-badstate=0 drop-oom=0 drop-freed=0 (reap=0 exit=0)
seg is what the guest's data segments carried and the four drop-*
counters plus sent must add up to it. It exists because a short arrival
was invisible from both ends: the guest's own accounting says complete, and
net-send-raw answers the frame length unconditionally after kicking TXP
without ever reading the NE2000's transmit status, so the guest cannot
report a transmit failure even in principle. A nonzero drop- counter
names the site; drop-freed splits into reap (the half-closed reaper
collected a connection mid-drain) and exit (VM exit stopped waiting for a
reader, bounded by NAT_EXIT_DRAIN_MS).
A slow reader does not provoke the loss this census was built for; a STALLED one does. The guest sends at roughly 120 KB/s, so a uniformly throttled host reader is never the bottleneck and the socket never fills. What fills it is a reader that stops near the end of a bulk transfer, so the send buffer is full at the moment the guest's FIN arrives. Measured 2026-08-18: a one-shot 15 s stall at 16.0 MB of a 16 MB send lost half a megabyte every time, where the unaided failure rate was about one run in five.
The IDE census, and how to read it. Any run that touched the disk prints one line to stderr at exit:
IDE CENSUS: pio-exits=N str-exits=N words=N reg-exits=N out-batch(hits=N words=N) in-batch(hits=N words=N) flush-entries=N flush-calls=N flush-bytes=N flush-ms=N refused(nopath=0 nodata=0 oob=0 openfail=0)
pio-exits counts exits on the data port at 0x1F0 and words counts the
words that crossed it, so their RATIO is the question the line exists to
answer: 1.0 is one exit per word and 1/256 is one exit per sector.
reg-exits counts the task-file registers at 0x1F1-0x1F7 and 0x3F6, which
is the driver's polling around the transfer rather than the transfer. The
out-batch and in-batch pairs are the string arms in handle_io
consuming a whole REP OUTSW/REP INSW in one exit.
Read the refused counters before quoting a time from any disk run. A
Copy-Item of a Perforce file leaves the copy READ-ONLY, every write-back
is then refused, the guest still reports the save complete, and the run
comes back about ten times faster. That silent loss is latched, counted and
reported here; clear the read-only bit rather than trusting the number.
Measured 2026-08-20 on docs/Probes/fat-write-phases.codex writing 2,896,050
bytes (5,657 sectors), a 16 MB image, 3 GB headless: 6,018 exits for 1,540,608
words, out-batch 5,865 hits over 1,501,440 words, whole probe 3.3 s wall. The
same probe against the compiler of the previous day took 1,540,098 exits for
the identical 1,540,608 words and 8.2 s, because the guest driver issued a
plain OUT per word and the host batch went unexercised.
A retransmitted SYN is handled, and before 2026-08-14 it was not. A SYN
for a connection the NAT already had fell through the new-connection branch,
opened a second host socket over the first and leaked it, and the guest's
payload then went to a socket no listener had accepted -- the guest saw its
data vanish and the peer saw a reset. It now answers a duplicate SYN with
silence while the host connect is still in flight (nat_poll_connect sends
the SYN+ACK when the socket is writable) and by resending the SYN+ACK once the
connection is up. Nothing in the tree had ever made a guest retransmit a SYN,
which is why it survived: it took running TCP over the e1000, where the poll
cost above puts the guest's first retransmit at 8.6 ms.
The NAT is one wire, and -e1000-nat moves it to the Intel card. The
e1000 model is absent unless a flag selects it, and by default its transmit
path sums the frame rather than sending it. With -e1000-nat the frame goes
to the NAT and replies are delivered into the e1000 receive ring instead of
the NE2000's, so a guest that binds the Intel part can hold a real TCP
conversation. Both cards drain one queue, so the NE2000 injection is turned
off while the flag is set; a guest that stays on the NE2000 under this flag
transmits and never hears back. -e1000-strict-filter makes the model refuse
promiscuous mode, so a frame addressed to neither the station address nor
broadcast is dropped and named on stderr -- the metal failure a stack
sourcing the wrong MAC produces, which our driver's own RCTL.UPE hides.
A SECOND RECEIVER BRING-UP IN ONE BOOT USED TO STOP DELIVERY FOR GOOD, and
what that costs is a class of old bed reading rather than anything still
broken. Fixed 2026-08-20 (blu, main 18011). e1000_rx_cursor is the head
both delivery paths use, and nothing reset it: there was no
case E1000_REG_RDH in e1000_write at all, so a write to RDH fell through
to the default store and the cursor only ever advanced. e1000-setup-rx
programs RDH=0 and RDT=15 on a 16-descriptor ring, and both paths declare
"ring full, leave it queued" when the cursor equals RDT, so a guest bringing
the receiver up a second time had one chance in sixteen per bring-up of
resuming with the cursor already on 15 -- and from there nothing was ever
delivered again. RNBC counted up, the host queue grew without bound, and the
guest polled a ring the model refused to fill. Measured on the diagnostic
ladder, which brings the receiver up four times in one boot: 138 ARP replies
sat in rx_queue while the stage reported no-arp. RDH is device state the
driver initialises, so writing it now moves the cursor, and reprogramming
RDBAL/RDBAH/RDLEN resets it.
Which readings this voids, stated narrowly on purpose. It was a BED
defect, so metal readings are untouched -- the sittings stand because
they are metal. It is also not "all bed evidence": the fault needs TWO
receiver bring-ups in ONE boot and the cursor landing on RDT, so a test
that boots, brings the receiver up once and exits could never reach it. All
22 e1000 and net tests pass identically before and after the fix, and no
published arm value changes. What IS void is one inference: "the part did
not deliver", drawn from an empty receive row taken after a second bring-up
in the same boot. In practice that is the diagnostic ladder's nicring and
b3 rows and nothing else. A bed reading of arrived-but-invisible from that
shape corroborated nothing, which matters because arrived-but-invisible is
exactly what NIC-4 was chasing on metal, and a bed that can produce it for a
reason the metal does not have is worse than one that cannot.
The PHY, reached through MDIC at 0x0020. The model answers MDIC with a 32-register PHY file at address 1; a read of any other address returns the error bit rather than zero, because that is what a bus with nothing on it does and a driver must be able to tell the two apart. BMCR reset and auto-negotiation restart are self-clearing, and a device reset drops the negotiated state.
| Flag | Effect |
|---|---|
-e1000-no-phy |
MDIC never reports ready: a PHY that is not answering |
-e1000-phy-err |
MDIC reports the error bit on every transaction |
-e1000-phy-link |
STATUS.LU requires auto-negotiation complete, not merely CTRL.SLU |
-e1000-mdio-window |
MDIC answers nothing for 10 ms after CTRL.RST (I219 datasheet 9.2) |
-e1000-mdio-slow |
MDIO reads answer E until page 769 register 16 bit 10 is set (I219 9.2) |
-e1000-ctrl-ro |
CTRL is READ-ONLY: writes discarded whole, register keeps the firmware value |
-i219-ulp-armed |
PHY page 779 register 16 powers up with STICKY_ULP (bit 4) and EN_ULP_LANPHYPC (bit 10) SET, on top of the board's measured 0x0800 |
779.16 is modelled and 779.17 is deliberately not. ULP Configuration 1
(I219 datasheet 9.5.7.1) answers reads and writes under -i219; ULP
Configuration 2 at register 17 on the same page still answers E, which is
the control that shows the page dispatch is not blanket-answering everything
on page 779.
The power-up value is the BOARD's, not the datasheet's. Every field in
that register documents a default of 0b, and Damian's I219 reads 0x0800
(bit 11, inside the reserved field) on two independent flights, sittings 8
and 9. The model powers up at the measured value.
-i219-ulp-armed exists because both ULP-entry bits are CLEAR on that board,
so a driver's entry-disable write has nothing to change and no arm can see it
happen. Armed, 779.16 powers up at 0x0C10 and the write has something to
clear. Without the flag a reaching arm and a skipping arm are the same run.
xdiag-put/xdiag-get (GopXhci) index one flat array of 32-bit cells at
xhci-diag. Every subsystem that wants a breadcrumb takes cells out of it,
nothing arbitrates, and no compiler or gate ever looked. Two owners of one
cell is green everywhere and surfaces as a wrong number on a photograph of a
boot screen.
| cells | owner |
|---|---|
| 0..19 | xHCI scalars (GopXhci's own map, literal offsets) |
| 20..27 | xHCI port cells, 20 + port, xhci-port-cells = 8 |
| 28..46 | xHCI ownership, release and BAR scalars |
| 47 | usb-hid-cell-count, the HID bind counter |
| 48..63 | xHCI controller table, 48 + i*4, four controllers |
| 64..69 | xhci-ep-base block, six words |
| 70..91 | MSC and FAT cells (msc-cell-*, gfat-cell-*; 88 is gfat-cell-written, the sectors the data run actually wrote; 89 is gfat-cell-lastsize, the size read back from the directory entry after the last bank write; 90 is gfat-cell-banklost, the stage index plus one of the first bank append that did not land, zero while every one has; 91 is msc-cell-chunk, the MSC transfer chunk in sectors for this boot, zero for the default of 64, set by sink chunk=N in DIAG.CFG) |
| 92..93 | diag b3 bank-loss cells (db3-cell-notes, the stage's notes counted as the serial trail counts them, and db3-cell-lost, the ordinal of the first note whose write was refused with a bank open, zero while none was; both zeroed at db3-run entry because metal RAM is not, and the second is what bank-lost-note=N on b3's first row reports) |
| 96..111 | usb-hid-note block, usb-hid-cell-base + idx*4, four devices |
msc-cell-end = 80 is an EXCLUSIVE bound for xdiag-zero, not a stored cell,
which is why it may equal another owner's first cell.
The HID block was at 80 until 2026-08-13, four cells deep into storage
territory, overlapping gfat-cell-stage 80, msc-cell-lastcc 81,
msc-cell-phase 82, gfat-cell-write 83, xhci-cell-fuel-left 84,
msc-cell-fuel-lo 85, msc-cell-fail-lba 86 and msc-cell-retry 87. It
stayed invisible because the writers run in order and neither reads the other:
usb-attach binds HID and fills 80..95, then the mount and any disk I/O
overwrite 80..87. So the storage diagnostics were correct and DeskBoot's
first two HID table rows were showing storage state under device-descriptor
labels. In the other direction a cell the storage path did not write on a
given boot kept its HID value, which is how an F12 mount failure came to
report gfat-cell-write as 1964712320 -- not a write stage, but HID device
0's report buffer address.
build/check-xdiag-cells.ps1 is the runner. It fails on an overlap between
declared blocks, on two named constants claiming one cell, and on a named
constant outside every declared block. A subsystem taking a RANGE must
declare it in that script's table, because a computed base is invisible to a
constant scan. It deliberately does not parse the ~90 literal xdiag-put 19
calls in GopXhci; those are that chapter's private map.
| -e1000-asde | STATUS answers SPEED and ASDV, and CTRL.ASDE picks which source SPEED comes from (82583V 12349, 12590) |
-e1000-phy-link is the one worth knowing about. It is off by default,
so every run that predates it keeps the SLU-only link it was measured
against; with it on, a driver that never brings the PHY up gets no link,
which is the I219's real behaviour and a failure the default bed cannot
show. codex/test/e1000-phy runs under it and
codex/test/e1000-phy-absent runs under -e1000-no-phy.
-e1000-mdio-window is also off by default and is the arm for the settle
the I219 datasheet requires between a reset and the first MDIO access
(section 9.2). A closed window answers with neither R nor E, so it is
indistinguishable from -e1000-no-phy to the driver, on purpose.
codex/test/e1000-mdio-window runs under it together with
-e1000-phy-link, which is the pair that reproduces the metal symptom on
the desk: a link that never comes up while every MAC register reads
exactly as it should.
-e1000-asde is the fourth and also off by default, and it exists because
the fields it fills were dead. STATUS carried no SPEED and no ASDV, so
na-line printed both off a register nothing ever wrote and every arm ever
run reported 10 Mb/s. With the flag on, SPEED follows the PHY's negotiated
speed when CTRL.ASDE is clear and the MAC's own detection when it is set,
which is the divergence 82583V 12349 describes. The datasheet says the bit
"must be set to 0b" and does not say what happens when software disobeys, so
the model invents no failure for it: the MAC's detection resolves to 10 Mb/s
because this bed's MAC has nothing to sense. codex/test/e1000-asde-speed
runs under it with -e1000-phy-link. It does not reproduce the metal wedge
and must not be read as evidence about it.
-e1000-ctrl-ro is the newest and it is measured on metal, not invented.
Damian's Intel I219-V at 00:1f.6, 2026-08-13: the driver cleared CTRL.SLU
and read CTRL straight back, four rows across two flights, and got
0x180240 -- the firmware value, SLU still set -- every time. CTRL.ASDE
refused to set on the same flights. MDIC writes work on that part, so
this is CTRL specifically and not the CSR write path: e1000-phy-write
writes MDIC at 0x0020, polls it ready, and succeeds on every arm.
Two consequences, and the second is the one that cost a day. The link comes
up anyway, over MDIO, because na-phy-kick reaches the PHY and the CTRL
write was never load-bearing. And with CTRL discarded, e1000-reset never
sets RST, so e1000-await-reset sees it clear on its first read and answers
settled=1 -- a reset that never happened, reporting exactly like one that
completed. On 2026-08-13 that read as "the reset works on a warm part", and
a whole cold-versus-warm hypothesis got built on an event that never
occurred. codex/test/e1000-ctrl-ro pins all of it, including a row that
deliberately reads the same with the arm and without it, because no aggregate
can tell those two resets apart.
Nothing public in this tree states that CTRL is read-only to the host, so the flag models the OBSERVATION and claims no mechanism. What the datasheet does say is that the I219 is the PHY and the MAC lives in the PCH, whose link configuration belongs to the integrated LAN controller and the ME (I219 datasheet Table 5-1, and 9.2's note that the LAN controller configures the LCD registers).
-e1000-mdio-slow is the third of these and also off by default. The PHY
model has pages now: registers 0-15 are the IEEE set and answer in every
page, registers 16-31 at page 0 keep the flat behaviour they always had,
and page 769 register 16 is modelled with the reset value its field table
gives. Any other page answers E rather than pretending to hold something.
With the flag on, ordinary MDIO reads answer E until slow mode is set, and
the page register and 769.16 itself stay reachable so the bootstrap can
happen at all -- that exemption is ours rather than the datasheet's,
because 9.2 read strictly forbids the very writes that would satisfy it.
A PHY reset clears the paged state, so a driver that sets slow mode before
resetting the PHY loses it; codex/test/e1000-mdio-slow catches that
ordering specifically.
DNS is answered. A query to port 53 is handled by nat_handle_dns: it
walks the QNAME, resolves it with getaddrinfo -- the host's own resolver, so
the hosts file, the search domain and whatever DNS the host actually
uses all apply, and no packet leaves the process -- and dresses the
answer as a DNS response the guest's resolver parses. Only QTYPE=A/IN
is answered; anything else returns NXDOMAIN rather than a lie.
General UDP forwarding is implemented. A guest-originated datagram opens a UDP flow -- a host socket remembered by (guest port, destination port, destination address) -- and its replies are framed back to the port the guest sent from. A flow is not a connection: there is no handshake and nothing to retransmit, so it is reaped on 30 seconds of idleness rather than on any close, and 32 of them are live at once.
The gateway address is the host. 10.0.2.2 is translated to
127.0.0.1 when a host socket is opened, for TCP as well as UDP. This is
the convention every user-mode NAT uses and the one this emulator already
advertised in DHCP and answered ARP for -- but nothing translated it, so
a guest addressing the gateway had its packet handed to the host stack as
a literal destination and it went nowhere. A service running on the box
codex-vm runs on is reached at 10.0.2.2 from inside the guest.
build/mqtt-interop-test.ps1 and build/coap-interop-test.ps1 are the
worked examples, against mosquitto and aiocoap.
Inbound UDP is forwarded too, via -portfwd udp:<host>:<guest> (the
prefix is optional and the default stays TCP, so every existing invocation
means what it always did). A datagram from a host client is put in front of
the guest from a synthetic gateway port that names which client it came
from; the guest answers that port as it would any peer, and the reply is
routed back by it. Without the synthetic port there is nothing in the
guest's reply that says which of several host clients it is for.
build/coap-serve-test.ps1 is the worked example: aiocoap's client against
a CoAP server running in the guest.
IDE Disk. PIO-mode IDE controller at ports 0x1F0-0x1F7, 0x3F6. Supports IDENTIFY, READ SECTORS, WRITE SECTORS, and FLUSH CACHE. Writes are flushed to the host image file (durable disk writes).
xHCI USB 3.x Controller. Full command ring, event ring, and transfer ring processing. Three device slots:
- Slot 1: Mass storage (bulk IN/OUT, SCSI READ/WRITE to RAM disk)
- Slot 2: HID keyboard (interrupt IN, generates scan codes)
- Slot 3: UVC camera (isochronous transfers). Delivers a 160x120 YUYV colour-bar test pattern: eight vertical bars 20 px wide, luma alternating 16 and 235 by bar. An isochronous TRB that sets IOC gets a transfer event (Short Packet when the frame is smaller than the buffer).
- Root port 4: a high-speed hub with a full-speed keyboard behind
its one downstream port (hub class descriptor, GET_PORT_STATUS,
SET/CLEAR_PORT_FEATURE). Device personality keys off the root port AND
the route string, because with a hub on the bus two devices share a root
port and only the route tells them apart. Under
-xhci-hub-tiers 2a full-speed hub is inserted between the two, so the keyboard sits at route0x11and is served by a translator that its immediate parent does not own.
A full- or low-speed device below a high-speed hub is serviced only if its slot context names a TRANSACTION TRANSLATOR. Real silicon reaches such a device through the hub's TT, and is told which one by the TT Hub Slot ID / TT Port Number in the slot context; leave them zero and there is no split schedule to place the endpoint in. codex-vm refuses to service that endpoint and says so, rather than delivering anyway. The failure this models is asymmetric and worth recognising: control transfers to endpoint zero keep working, because the hub handles those itself, while the periodic endpoint is silent forever -- a keyboard that enumerates perfectly and then never sends a keystroke. That is the reported shape of the failure on real Intel xHCI, and it is reproducible here with no hardware.
Which translator is the part that is easy to get wrong, and
-xhci-hub-tiers 2 is what makes the difference observable. A translator
lives in a high-speed hub; a full-speed hub has none, so everything below a
full-speed hub is served by whichever high-speed hub sits further up. A
walk that reads the translator off the device's immediate parent is right
for one tier and writes zero for two. codex/test/apps/usb-kbd-hub and
usb-kbd-hub2 are the two cases, and they differ by nothing but the
topology the emulator presents.
Intel HDA Audio. CORB/RIRB command interface, output stream DMA. 48 kHz 16-bit stereo PCM. Host-side playback via Windows waveOut API.
HPET. High Precision Event Timer at 0xFED00000. Main counter driven by QueryPerformanceCounter. Comparator 0 with periodic mode and interrupt generation.
Timers and Interrupts. Dual 8259 PIC (master 0x20, slave 0xA0). PIT channel 0 at port 0x40 (host-driven periodic tick). CMOS RTC at port 0x70/0x71 (real host time). PC speaker via Windows Beep().
PS/2 Keyboard. Port 0x60/0x64 with a scan code queue.
Scripted input (-mouse, -mouse-file, -keys-file) writes the same guest
state the window proc writes -- press latch included -- so a guest cannot
distinguish it from a hand on the mouse. This is what build/test-gui.ps1
drives GOP applications with; see docs/ExaminersAssay.md.
Under -hid-instant-complete a keystroke narrower than the guest's poll
interval does not exist. That model completes every interrupt IN TRB at
doorbell time, so the guest's armed TD is consumed with whatever the
held-key set held then and nothing stays armed for a later report to land
in. Measured on usb-kbd-multi, hold width the only variable, three runs
per cell:
| hold | -hid-instant-complete |
default (NAK) |
|---|---|---|
| 1 ms | got=0 3/3 |
got=30 3/3 |
| 2 ms | got=0 3/3 |
got=30 3/3 |
| 10 ms | got=30 3/3 |
got=30 3/3 |
| 600 ms (shipped) | got=30 3/3 |
got=30 3/3 |
It fails silently: the VM logs the key event and the report reaching the endpoint either way, so a bed reads as working right up to the width where it stops being. That silence is why the default moved rather than the guidance.
Re-ringing the endpoint on input change does NOT rescue instant-complete, and was measured not to: with a trace on the re-ring it fires (twice, make and break) and produces no report, because that model already drained the ring at the guest's last doorbell. There is nothing to deliver into. Only leaving the TD pending works, which is what the NAK model is.
Separator trap, same family: -keys-file and -mouse accept ; and
newline ONLY. A comma-joined timeline silently injects its FIRST event and
prints no error, because the parser skips to the next ; or newline and
discards the rest of the line.
HID: first key event sc=NN prints NN in HEX, and nothing on the line
says so. Read it as decimal and an injected 80 reads back as sc=50,
which looks like the wrong key arrived when it is the right one. On
2026-08-11 that cost a false backlog entry: four injected arrow keys
changed nothing on screen, the trace was read as decimal, and the
keyboard path was filed as the suspect. The keys were being delivered
perfectly and the page simply had nothing to scroll. When a scripted key
seems not to arrive, convert the trace before blaming the route, and
prefer a chord with an unmistakable effect -- -keys 29,20,157 is
Ctrl+T, and a browser tab either appears or does not.
Mouse (absolute, I/O ports 0xE1-0xE4). The guest reads the mouse through four ports rather than shared memory (which WHP does not keep coherent between the window thread and the vCPU).
| Port | Meaning |
|---|---|
| 0xE1 | Button mask (bit 0 left, 1 right, 2 middle). The live level, OR the presses latched since the last read. Reading it consumes the latch. |
| 0xE2 | Absolute x (16-bit, client coords) |
| 0xE3 | Absolute y (16-bit). Reading it consumes 0xE4. |
| 0xE4 | An edge: a new mouse event has arrived since 0xE3 was last read |
0xE1 is a level and stays valid between events; 0xE4 gates only the
position reads. Never gate the button read on 0xE4 -- that makes a
held button legible for exactly one poll per host mouse message and
reads as released on every other poll, so no drag can ever be seen.
(This was the circuits mouse bug; see codex/foreword/ui/InputSource.codex
for the correct poll sequence.) The press latch on 0xE1 means a click
whose down and up both land between two guest polls is still delivered,
exactly once, so a slow-rendering guest does not drop clicks.
The 3-byte PS/2 packet buffer at GPA 28684 (read by
codex/os/kernel/Mouse.codex) is inert: codex-vm flushes it but never
populates it, so it always reads as 'no packet'.
ACPI. RSDP at 0xE0000, RSDT, FADT (SCI on port 0x2000, PM timer at 0x2004), MADT (LAPIC entries per core + IOAPIC), DSDT stub.
SMBIOS. The 2.1 entry point (_SM_) at 0xE0780 and the structure table
at 0xE0800: Type 0 (BIOS "Codex 1.0 05/23/2026"), Type 1 (System "Codex
Project" / "Codex VM" / "1.0"), Type 2 (baseboard "Codex Virtual Board"),
Type 17 (one memory device sized from -mem, added 2026-08-18) and Type 127.
Under -uefi a 3.0 entry point (_SM3_) at 0xF0B00 names the same table and
the ConfigurationTable publishes both under their GUIDs (entries 2 and 3;
-no-smbios leaves them out). Also under -uefi: EFI_EDID_ACTIVE and
EFI_EDID_DISCOVERED answer from LocateProtocol with a 128-byte EDID 1.4 block
at 0xF0B80 (manufacturer CDX, product 1, one 1920x1080 detailed timing,
monitor name "codex-vm dsp"); -no-edid removes them and -edid-bad breaks
the checksum. Measured 2026-08-18: the type 0 and type 1 string sets carried a
stray third NUL, which a spec walker reads as a zero-length structure and
stops on; fixed the same day, and the ladder's smbios stage is the arm.
UEFI Firmware Emulation (when -uefi or booting a GPT image).
Trap-page dispatch at GPA 0xF1000 (HLT opcodes -- guest calls trap,
VM intercepts the HLT). Supported protocols:
| Protocol | Functions |
|---|---|
| ConIn | Reset, ReadKeyStroke, ReadKeyStrokeEx |
| ConOut | Reset, OutputString, TestString, QueryMode, SetMode, SetAttribute, ClearScreen, SetCursorPosition, EnableCursor |
| Boot Services | AllocatePages, FreePages, GetMemoryMap, AllocatePool, FreePool, ExitBootServices, Stall, SetWatchdogTimer, HandleProtocol, LocateHandle, LocateProtocol |
| Runtime Services | GetTime (host RTC) |
| GOP | QueryMode, SetMode, Blt |
| Block I/O | Reset, ReadBlocks, WriteBlocks, Flush |
| Simple File System | OpenVolume, Open, Close, Read, GetInfo, SetPosition |
Auto-extracts PE from GPT images: scans for EFI System Partition,
locates EFI/BOOT/BOOTX64.EFI, loads the PE into memory.
Guest-to-Host Communication. Guest reads host-provided values from fixed GPAs:
| GPA | Width | Content |
|---|---|---|
| 0xFE8 | 8 bytes | Guest RAM size in bytes (for dynamic RSP) |
| 0xFF0 | 8 bytes | Boot arguments string pointer |
| 0xFF8 | 4 bytes | SMP core count (0 or 1 = single-core) |
Run with -debug (and optionally -break <fn> and -map <file>)
for a command shell on breakpoints. See the Native Debugging Toolkit
section for commands and workflows.
Set $env:CODEX_VM_HOST='qemu' to force QEMU. It is the only switch that
does anything: vm-config.ps1 decides the host with
$script:UseCodexVm = (codex-vm.exe exists) -and ($env:CODEX_VM_HOST -ne 'qemu'), and it reads that variable and no other.
USE_QEMU is not a switch and never was. This section told the reader
to set it until 2026-08-18, and gdb-watchpoint.ps1 set it too, which is
how the GDB watchpoint workflow came to be silently dead on every box that
has codex-vm.exe: the host stayed codex-vm and -gdb tcp::1234 -S went
to a VM with no GDB stub. Measured by dot-sourcing vm-config.ps1 both
ways, USE_QEMU=1 leaves UseCodexVm True.
$env:CODEX_VM_HOST = 'qemu'
build/test.ps1 -Jobs 8CODEX_ACCEL overrides the accelerator (whpx on Windows, tcg on Linux).
kernel-irqchip=off required for bare-metal operation under QEMU.
Where a QEMU compile's time goes, and it is mostly not the guest.
Measured 2026-08-18 on plug-oracle-arith (103,645 bytes of output) under
WHPX, through vm-config's own boot path: QEMU process start 565 ms, guest
boot to READY 404 ms, input send 260 ms, compile-and-output 1,419 ms,
teardown 17 ms. Two traps sit in that profile:
- Do not time a QEMU compile with a bespoke
qemu-system-x86_64invocation. A hand-rolled one that does not honour the boot contract (the0xfe8RAM-size loader cell, the two chardevs) produces a few hundred bytes instead of the full binary because the guest never ran, and the run still looks like it finished. Drive it throughcompile.ps1withCODEX_VM_HOST=qemu. Invoke-VmCompileFallbackused to pay a flat 2 s per compile. Once SIZE's declared payload was complete it setReadTimeout = 2000and did one more blocking read to drain a trailer that in practice is never there. Now 250 ms. If a QEMU timing looks like it has a constant floor in it, this is the shape of thing to look for.
The output path itself is no longer per-byte: __write_binary_buf polls
LSR once and rep outsb a burst of up to 16 through the 16550 transmit
FIFO, so a burst is one VM exit rather than sixteen. codex-vm handles the
string I/O in one exit as well (codex-vm.c, the StringOp arm on 0x3F8);
without that arm a burst would write one byte out of RAX and skip the rest.
Source to booted guest in one command: IR, ARM64 codegen, PE through the plug,
GPT disk image, QEMU. -NoBoot stops after the image.
build/boot-arm64.ps1 -Src codex\test\arm64-web-server.codex # interactive
build/boot-arm64.ps1 -Src codex\test\arm64-web-server.codex -NoBoot # build only
build/boot-arm64.ps1 -Src codex\test\arm64-web-server.codex -TimeoutSec 45 # as an arm-TimeoutSec defaults to 0, which is the interactive console: foreground
-nographic, no deadline, Ctrl+C to stop. Any positive value launches QEMU
monitored, sends the UART to build/arm64-output/arm64-serial.log, and kills it
at the deadline.
A TIMEOUT EXITS 0, deliberately, and the printed line is what tells you
which run you had. For a server payload the deadline IS the intended end, so a
kill is not a failure and a nonzero code would cry wolf on every normal run. The
cost is that the exit code alone cannot separate a healthy guest from a wedged
one: QEMU exited on its own, code N and TIMEOUT at Ns: ... the run did NOT finish are the discriminator. Do not script a verdict off the exit code
here -- read the line, and read the serial log knowing it may be a partial.
The host-forward port is derived per workspace in 18080..18279, and a held
port is a REFUSAL. It was a fixed 8080, so two agents booting arm64 answered
each other's requests. Same hash shape as build/boot/test-ovmf.ps1's monitor
port and for the same reason (L-SHARED). It refuses rather than hopping to the
next free number, because a run that quietly moves its port answers on an
address the caller was never told. -HostPort N overrides. The chosen port is
printed as hostfwd tcp::N -> guest :80.
To drive requests at the guest, docs/Probes/arm64-two-requests.ps1 is the
caller: it banks guest serial, QEMU stderr and a per-request verdict, and
refuses to report request results unless the guest announced it was listening.
Renode v1.16.1 provides cycle-accurate simulation for ARM64 and
RISC-V 64 targets. Install the runtime once per box -- extract the
renode-1.16.1.windows-portable-dotnet.zip to C:\Renode (so
C:\Renode\renode.exe exists), or set $env:CODEX_RENODE_HOME to any
Renode install dir. Every workspace finds it via build/renode-config.ps1
(Get-RenodeExe), which probes CODEX_RENODE_HOME, then C:\Renode,
then %LOCALAPPDATA%\Renode, then a legacy tools/renode/ copy. Only the
board .repl files under tools/renode/codex/ are tracked per-workspace;
the ~120 MB runtime is not.
Board definitions (in tools/renode/codex/):
| Board | CPU | UART | RAM |
|---|---|---|---|
codex-arm64.repl |
Cortex-A53 (GICv3) | PL011 @ 0x09000000 | 1GB @ 0x40000000 |
codex-riscv64.repl |
RV64GC (PLIC/CLINT) | NS16550 @ 0x10000000 | 1GB @ 0x80000000 |
Quick test (requires plugs built first):
build/test-boards.ps1 # Both boards
build/test-boards.ps1 -Arch arm64 # ARM64 only
build/test-boards.ps1 -Arch riscv64 # RISC-V onlyPipeline: source → compile.ps1 (IR) → arm64/riscv plug → wire protocol → compile-arm64/riscv.ps1 (ELF) → Renode (UART capture).
Setup from scratch:
- Download the Renode v1.16.1
windows-portable-dotnetzip from GitHub releases - Extract box-wide to
C:\Renode(or set$env:CODEX_RENODE_HOME) - The
.replboard files undertools/renode/codex/are already tracked - Build plugs:
codex/plugs/arm64/build.ps1,codex/plugs/riscv/build.ps1 - Run
build/test-boards.ps1
Key fixes for Renode compatibility (vs QEMU):
- ARM64: exception vector table required (VBAR_EL1/EL3), PL011 needs explicit UARTLCR_H + UARTCR init, 32-bit UART writes
- ARM64: all runtime helpers must use
a64-emit-block(nota64-emit) - RISC-V: heap register must be S1/x9 (not t3/x28 which collides with temp allocator)
A harness that polls a running codex-vm's redirected stderr or stdout to
watch for progress must open those files sharing write. Plain
Get-Content does not, and the writer's next write into the locked file
fails, which takes the VM down.
What it looks like is not a file error. It is a guest that dies partway
through, silently, always around the same place. Measured 2026-08-10 on
build/sink-arm.ps1: its pass arm ran a 2.7 MB FAT write and died after the
same rung on three consecutive runs, with no fault line, no EXC, and
nothing in either stream. The identical image invoked by hand -- same flags,
same -output, no polling -- reached its last rung every time. Two earlier
runs of the same harness had passed, and the only thing that changed between
them was the addition of the poll.
The tell is that shape: an arm that fails under the harness and passes by
hand is the harness perturbing the run, not a flaky guest. -output was
the obvious suspect and was wrong -- adding it to the by-hand run changed
nothing, which is what ruled it out.
Read a live log like this instead:
$fs = [IO.File]::Open($path, [IO.FileMode]::Open, [IO.FileAccess]::Read,
([IO.FileShare]::ReadWrite -bor [IO.FileShare]::Delete))
$sr = New-Object IO.StreamReader($fs)
$text = $sr.ReadToEnd(); $sr.Close(); $fs.Close()Reading the streams AFTER the process is gone, which is what
build/ladder-arm.ps1 does, is unaffected -- there is no writer left to
lock out. This only bites a harness that watches progress while the VM runs,
and the reason to watch is real: without it, an arm that holds its colour by
repainting never exits and costs its whole deadline.
-screenshot works, and a payload's paint DOES show up in it. Verified
2026-08-11 across four payloads: a minimal paint probe and blockladder.img
both render FFFFFF, sinkladder.img renders FF8000 mid-write, and
a5flight2.img renders FFFFFF.
A ladder that reports painted fb=1 against an apparently black BMP is
almost always the reader. This expression is the usual cause:
($b[$i+2] -shl 16) -bor ($b[$i+1] -shl 8) -bor $b[$i] # WRONG$b[$i] indexes a byte[], and PowerShell shifts a [byte] at BYTE width, so
both shifted terms are 0 and the result is the blue channel alone. Orange
(FF 80 00) decodes as 000000 and reads as black; white decodes as 0000FF
and reads as "not white".
Cast, or read the channels separately and do not combine them:
$b = [IO.File]::ReadAllBytes($bmp)
$off = [BitConverter]::ToInt32($b,10); $w = [BitConverter]::ToInt32($b,18)
$h = [BitConverter]::ToInt32($b,22)
$stride = [int][math]::Floor(($w * 24 + 31) / 32) * 4
$i = $off + ($h - 1) * $stride # BMP is bottom-up; this is the TOP row
'R={0:X2} G={1:X2} B={2:X2}' -f $b[$i+2], $b[$i+1], $b[$i]The general shape is worth more than the recipe: a decoder that renders two
different inputs as the same glyph is the peek-32 sentinel trap again, and
it cost a false section in this file plus a harness built around the belief
that the bed could not see a colour at all.
-screenshot <file> -screenshot-delay <ms> also EXITS after the capture, which
is the only bounded way to run a payload that holds its colour by repainting.
The section above is one instance of a general rule, and the rest of the instances cost a session each. When a check you wrote in the moment disagrees with the product, distrust the check first, and prefer the real harness to a reimplementation of it. Three false readings in one session were all the apparatus; the product was fine every time.
[uint32]0xFFFFFFFFTHROWS, because PowerShell parses0xFFFFFFFFas Int32 minus one. Measured 2026-08-15 writing a CRC32 helper to check GPT fixtures: the seed line$crc = [uint32]0xFFFFFFFFthrew on every call, the catch-free loop reported every one of 26 fixtures as failing both CRCs, and that read exactly like a corpus-wide finding. Write the constant in decimal (4294967295, and3988292384for the polynomial). Validate any checksum helper against a published vector before believing a single answer from it:CRC32("123456789")is0xCBF43926, and one line asserting that turned the false corpus-wide finding into a real 21-verify / 4-stub split.[int]ROUNDS in PowerShell, it does not truncate.[int](16895 / 512)is 33, not 32, so the usual(n + d - 1) / dceiling idiom comes out one high. Use[Math]::Floor. It landed aFirstUsableLBAone sector past where it belonged in four disk fixtures, which is harmless and wrong, and the same idiom in Codex is correct because Codex integer division truncates.- PowerShell's error formatting ECHOES THE SOURCE LINE before the message,
and gutters the wrapped continuation with
|. So the FIRST occurrence of your own message text in captured output is the uninterpolated source, not the message: scoring refusal arms againstREFUSED: ([^.]+)matched the literal$whyfrom thethrowstatement and failed four arms that were working. Strip the gutters, collapse the wrap, and take the LAST match. And clamp a captured string for DISPLAY only -- scoring against the clamped copy failed the one arm whose phrase sat past column 74. - Array splatting binds POSITIONALLY, and against
test-run.ps1it DESTROYS the file you name.$a = @('-Kernel', $k, '-OutFile', $o)then& test-run.ps1 @aputs the literal string-Kernelin$Kerneland the KERNEL PATH in$OutFile; the no-output path then writes an empty file to$OutFile, so every run truncates the kernel to 0 bytes. It reads as a broken environment rather than a bad call, because the run "fails" and the next run fails differently now that the kernel is empty. Use a HASHTABLE splat (@{ Kernel = ...; OutFile = ... }), and when a test fails assert the kernel still has content BEFORE believing the result: a 0-byte kernel and a failing test are indistinguishable downstream. 2>&1does not captureWrite-Host. Use*>&1. Most build scripts here write progress that way, so$t = & .\build.ps1 2>&1 | Out-Stringleaves$tEMPTY while the output scrolls past on the console. A pass test of the formif ($t -notmatch 'OK:')then reports every arm failed; 44 plugs that had all printed OK came back as 44 failures. The dangerous direction is the mirror image:-match 'FAIL'over an empty string is False, so a real failure passes silently. Check the exit code, which is never empty, and if you must match text assert$t.Length -gt 0first.-matchand-notmatchare case-INSENSITIVE, and a cleaning filter is where that bites.-notmatch 'HEAP-'also drops everyheap-line, so a text cleaned for comparison silently lost four emittedheap-*/stack-*lines while the exact-match consumer downstream still expected them. The failure is quiet in the worst direction: the filter looks like it did its job and the diff that follows is against a text nobody wrote. Use-cnotmatch/-cmatchwherever a filter has to agree with a case-sensitive reader. Related, same family:"\$$n"in a double-quoted string expands$$as an automatic variable rather than giving you a literal$followed by$n.Measure-Object -Linesilently DROPS EMPTY LINES, so it is the wrong instrument for a file size and it never says so. Measured 2026-08-15 ondocs/Agents/PerforceProcess.md:(Get-Content $f).Count= 500,Measure-Object -Line= 405, and the file has exactly 95 blank lines. The gap is the blanks, every time. It is dangerous precisely because 405 is a plausible number for that file, and the usual place it lands is a CL description quoting a size or a percentage cut, where nothing will ever re-measure it (L-COUNT). Count with(Get-Content $f).Count, or count the line-ending bytes; two instruments agreeing is the check.- A pipeline that greps a command's output discards that command's error
message, so a failed command reports as a clean result.
p4 diff -du -c <CL> 2>&1 | Select-String '^\+' | Measure-Objectreturns 0 for a CL with 164 added lines, because the2>&1foldsInvalid option: -c.into the stream and the^\+filter eats it. The command was never silent; the pipeline silenced it.$LASTEXITCODEsurvives the pipeline and was 1 the whole time, so test that rather than the filtered count. The Perforce instance isP-DIFFCindocs/Agents/PerforceProcess.md, but the shape is general and this tree is full of2>&1 | Select-String. - A one-element array unrolls to a scalar, and indexing a scalar STRING
yields a character. The loud instance reads
unresolved symbol 'm'for"main". The silent instance is the one to fear: a list of mnemonics derived each as its initial letter and printed a clean, plausible, entirely wrong opcode table. Force@(...)on the assignment or type the variable[string[]]. The registry harness section inExaminersAssay.mdhas the same trap costing a green that could not have passed. - Name a helper carefully; a short name can shadow an alias. A comparison
helper called
Hbound to theGet-Historyalias, both hashes came back empty, and the check printed EQUAL. Assert the expected length before comparing -- two empty strings are equal, which is the exact shape of a check that passes because it never ran. - A substring match fails on wrapped prose. Checking that a merge kept both sides, with a phrase that spans a line break in the file, answers False for both. Grep a short distinctive fragment, or the line number.
- Do not filter a long run's output at the pipe.
build.ps1 | Select-Object -Last 26leaves a log holding 26 lines, and a later search of that whole file is not the wide check it feels like: the file is already the narrowed thing. It has twice produced a confident wrong answer -- once reporting that the gate prints no one-pass line when it prints it atbuild/build.ps1:620, and again on 2026-08-27 when a filtered gate capture showed adeck-headroomFAIL with the offending unit's name and the entire derivation table absent, so the visible text named a cause the run had not established. Let the log land whole and filter at read time; when the question is "does X ever appear", search the SOURCE that would print X, not a capture of it. The truncation cannot be undone and costs a rerun to recover.
Stop-Process -Name codex-vm -Force, and every spelling of it
(Get-Process codex-vm | Stop-Process), is machine-wide, not
workspace-scoped. Several agents share this box, and every one of their
gates runs the compiler inside codex-vm, so a name-based kill reaches
into other workspaces and terminates compiles that have nothing to do
with you.
What it looks like on the receiving end is not a dead VM. It is a compiler that produced nothing, in a different phase every time. Measured 2026-07-28, four gate runs on one unchanged tree while a peer was sweeping name-based kills between its own compiles:
| run | symptom |
|---|---|
| 1 | FAIL: text round-trip -- stage2 wrote 0 bytes, hashing to the sha of the empty string |
| 2 | green |
| 3 | FAIL: BVT -- four unrelated tests failing at (compile) |
| 4 | FAIL: semantic equivalence -- stage1 does not match source |
Three different failures, none reproducible, none real. There is no crash and no diagnostic: the phase simply yields a truncated or empty artifact. So the agent on the receiving end reverts to a clean tree, gets one green, and concludes the fault is in their own change. That happened here, and the clean-tree green was itself a one-sample coincidence that made the wrong conclusion look confirmed.
Kill by PID, or filter on the command line. Start-Process hands the
process back; keep it and stop that one. If you must sweep, scope it to
your own workspace path:
# Yours only -- the path is what makes it yours
Get-CimInstance Win32_Process -Filter "Name='codex-vm.exe'" |
Where-Object { $_.CommandLine -match [regex]::Escape($PWD.Path) } |
ForEach-Object { Stop-Process -Id $_.ProcessId -Force }The same applies to pwsh, which carries every agent's session as well
as every build worker: a name-based kill there takes out other agents'
Claude sessions, not just their builds.
If a gate fails in a compile-heavy phase and will not reproduce,
re-run before diagnosing. text-stage1, text-stage2, sem-equiv
and the BVT's compile steps are the long codex-vm runs and therefore the
wide targets. A failure that vanishes on re-run was never in your tree,
and the expensive mistake is bisecting your own change to find it.
codex/plugs/*/build-output/*-plug.cdx is not a depot file --
build-output/ is in .p4ignore -- so each workspace has its own, and
they are only as fresh as the last hand run of that plug's build.ps1.
A merge-down brings you new plug SOURCE and leaves your binaries where
they were.
pwsh codex\plugs\rust\build.ps1 # one plug, about 3 seconds
Get-ChildItem codex\plugs -Directory | ForEach-Object {
$b = Join-Path $_.FullName 'build.ps1'
if (Test-Path $b) { & pwsh -NoProfile -File $b *> $null }
} # all of them, about 150 spwsh build/check-plug-builtins.ps1 # OK, or the gaps + exit 1
pwsh build/check-plug-builtins.ps1 -Update # rewrite the baselineIt compiles codex/test/plug-oracle-arith to IR under -Passes 'text-plug',
which is the pipeline a source plug actually receives, takes every
(name "X") whose X is a declared builtin, and asks each wired plug's table
for an arm. A FAIL line reads <plug> <builtin>: that builtin arrives on
that plug's wire and the plug has no arm, so it emits a call to a function the
target never defines and only the target's own toolchain would ever notice.
Deliberately not wired into build.ps1 (it costs a compiler run) and it
covers the WIRED plugs only. pascal, fortran and others register builtins
in a shape it does not model, so it refuses on a thin extraction rather than
listing phantom gaps against them. The compiler as a subject is not re-checked
here either: that case is the DDC witness, which already fails on a missing
builtin.
Do this after any change under codex/plugs/common/, which every plug
bundles. On 2026-08-11 a workspace held 44 plug binaries from 08-06 against
a shared IRTextParser.codex from 08-08, and 37 of 38 runnable plugs died
with an invalid opcode in parse-type-record the moment the IR contained a
record construction. The gate does not catch it: plug-smoke rebuilds a
plug only when its CDX is MISSING, and tests four of them on one
record-free program. Symptom to recognise: the plug VM exits code=-1 and
the harness reports plug produced no output (exit 6).
Timestamps are the fast check -- compare
Get-ChildItem codex\plugs\*\build-output\*-plug.cdx against
codex\plugs\common\*.codex.
It purges orphaned qemu-system-x86_64, codex-vm and wsl processes
after a SIGKILL-style abort (a taskkill /F, a harness OOM, a wedged WSL
VM) left the traps in the test, build and 3stage harnesses unable to run.
It is scoped by process NAME, which is exactly what the section above
warns about, and that is deliberate: an orphan has no workspace left to
scope it to.
So it is safe between test runs and destructive during one. It kills any VM regardless of which agent started it, and the receiving agent sees the truncated-artifact failures tabulated above rather than anything that names a kill. Do not run it while another agent may be mid-gate.
powershell -NoProfile -File build/clean-zombies.ps1It never calls wsl --shutdown unless vmmemWSL is already up, and
that fast path is not a micro-optimisation. wsl.exe SPINS THE VM UP
IN ORDER TO SHUT IT DOWN, so the unconditional call costs minutes on a
box where WSL was simply idle. The same fast-path rule covers the other
two names: a process that is not running is skipped rather than asked
about.
This doctrine had no home but the script's own header comment until 2026-08-11, when adopting the generator dropped the header. Nothing else in the tree described the tool.
build/compile.ps1 boots the compiler kernel in a VM and
communicates over serial:
- Wait for
READYon control channel (ch1). - Send mode header (
CDX,TEXT,IR, etc.) on data channel. - Send foreword library bytes (transitively resolved).
- Send source bytes.
- Send EOT (0x04).
- Read diagnostic lines until
SIZE:<n>(success) orCODEGEN-HALTED(failure). - On success, read
nraw bytes of binary output.
True ON STDOUT IS NOT THE VERDICT. compile.ps1 prints True on a
refused compile exactly as it does on a clean one; the verdict is the EXIT
CODE (4 on codegen errors) and the -Log file, which carries
CODEGEN-ERRORS:<n>, the diagnostics themselves, and CODEGEN-HALTED: errors in bag; no binary emitted. A clean log holds only info lines. The other
tell is that no -Out file appears, which is easy to miss when a stale one
from an earlier run is still sitting there. Measured 2026-08-19: a refusal
read as a pass off that True, and the wrong reading was published before the
log was opened.
IN -Text AND -IrUni, EXIT 4 IS THE SUCCESS PATH AND THE EXIT CODE MEANS
NOTHING. -IrCce IS NOT LIKE THEM. The paragraph above holds for CDX,
where a clean compile exits 0. The modes do NOT agree with each other, and
assuming they do is how this entry got written wrong the first time. Measured
2026-08-27 against seed 0634584EF849D297, same source and same invocation
shape:
| mode | exit | -Out |
-Log |
|---|---|---|---|
-IrUni |
4 | never written | the wire, bracketed by IR-BEGIN / IR-END |
-Text |
4 | never written | the text, no IR-BEGIN |
-IrCce |
0 | written (1,109 bytes here) | diagnostics only, no IR-BEGIN |
-Text and -IrUni emit no SIZE: line, so the scan that looks for one runs
off the end of the output and the script falls through to the bare exit 4 at
the bottom of the compile loop, having written the whole guest output into
-Log. In those two a clean emit, a crash and a codegen refusal all carry the
SAME exit code, so the verdict is the -Log file: IR-BEGIN / IR-END bracket
the wire and error CDX.... lines carry the diagnostics. -IrCce takes the
binary path instead and behaves like CDX, so there the exit code IS the
verdict.
The first version of this entry said all three behaved alike, on a reading of
compile.ps1's control flow rather than a measurement of each mode, and a
naive-reader probe caught it the same day. Reading one branch of a dispatch and
generalising to its siblings is the same error as reading one tier and
concluding about three. The half that was right is worth keeping: a harness
gating on the exit code in -IrUni reports every run as a failure with the
correct IR sitting in the log, which is L-ACCEPTED pointed at a RESULT rather
than an argument.
When a plug emits a wrong type for a value and the program compiles clean, the question is whether the compiler worked the type out and failed to put it on the wire, or whether the program never constrained it. Those need different fixes and they look identical from inside a plug, which erases or guesses either way.
build\ir-fidelity\ir-fidelity.ps1 -Grade # grade the arm, then run every case
build\ir-fidelity\ir-fidelity.ps1 -Only lambda-param-type # one case
build\ir-fidelity\ir-fidelity.ps1 -Passes # audit the OPTIMIZED program insteadA case is three programs and one wire position: a and b differ in one
respect and both compile clean, knows is a program the checker must REFUSE
with a named diagnostic, and path names the cell to compare. CARRIED means
the checker knew and the wire carried it, so a wrong type downstream is the
plug's. DROPPED means the checker knew and the wire did not, so it is
upstream. UNCONSTRAINED means the knows arm did not refuse, so the case
proved nothing and wants fixing before its cell is read. UNSUPPORTED means
the reader could not find the cell; expect it under -Passes where the
optimizer inlined the subject away. The last two are deliberately not passes.
The reader (ir-wire.ps1) has no plug opinion in it and shares no code with
codex/plugs/common/IRTextParser.codex. Run it from the depot root. Cost is
about 0.3 s per compile and three compiles per case. It is NOT wired into any
gate; that is Damian's ruling. codex/plugs/plugs-backlog.md 1.96 carries the
standing verdicts and what they mean.
| Mode | Output |
|---|---|
CDX |
CDX binary |
CDX repl |
CDX binary, REPL loop enabled |
TEXT |
Codex source text |
IR |
IR text dump |
MEASURE |
Phase metrics |
Append flags: TEXT prose, CDX repl, CDX poison
Container formats (ELF, PE, IMG) are produced by post-compile
plugs in codex/plugs/.
A source that quotes a work by digest normally has to carry it in a
%%QUOTED-WORKS%% blob or find it in an attached store (-DiskFile).
-Peer <host>:<port> fetches it instead:
build/compile.ps1 -Src x.codex -Out x.cdx -Log x.log -Peer 127.0.0.1:9300compile.ps1 scans the source for every digest it quotes and does not already
carry, asks the peer for each over TCP, and prepends the answers to the blob
before the compiler is booted. The peer is tools/cdx-serve, listening on guest
port 9300; the client is build/work-wire.ps1.
The compiler is not involved and never will be. Library Rule 2 fixes
codex.foreword → codex → codex.os → apps, so the compiler citing the net stack
inverts it -- and it does not need to. It hashes the content, checks the
signature, checks the key the source pinned and checks the trust floor itself,
which is exactly what lets the messenger be untrusted. The host carries works;
it never carries trust, and work-wire.ps1 emits WORK lines and never a KEY
line. A peer that answers with the wrong work is refused by arithmetic.
The consequence to know: the resolution order lives in the host, so this works only when a host drives the build. A bare-metal stick has no fetcher.
A miss is not an error -- a peer that does not hold a work simply does not, and the compile then fails at the gate if nothing else supplies it. An unreachable peer is an error.
The canonical seed is seed/Codex.cdx -- the signed, self-sustaining
CDX binary, bootable via codex-vm or QEMU multiboot.
Pre-conditions:
- All source changes are submitted
- The change justifies a rebuild. The test is whether the compiler REACHES
the code you changed, not which chapter you edited -- whole-program DCE
prunes definitions the compiler never calls, so a new function in a cited
chapter can cost the binary nothing while a new
citesline, a live registry arm, or the body of a definition the compiler calls all change it. Do not predict which case you are in: compare the hash ofbuild/output/Sut.cdxagainstseed/Codex.cdxafter the gate. Equal means no seed is needed.build.ps1proves the SUT is a fixed point of itself and never checks it against the depot seed. Full table and the two measured cases:docs/DevelopersGuide.md, Seed Rebuild Procedure. - The SIZE of a seed-to-seed difference carries no information, so do not read a change's reach off it in either direction. Measured 2026-08-15 over the last seven seeds on main, consecutive pairs differ by 97 bytes, 2,354,783, 2,282,139, 1,211,880, 2,395,976, 364,032 and 2,536,593 -- four orders of magnitude, and the first difference sits at offset 8 every time because that is the content hash. A big diff is not evidence of a big change and a small one is not evidence of a safe change. The hash compare above answers whether a seed is needed; the gate answers whether the compiler still agrees with itself. Nothing else is a reading.
Steps:
- Run the full build:
build/build.ps1. All phases must PASS. - Install new seed:
Copy-Item build/output/Sut.cdx seed\Codex.cdx -ForceUsebuild/output/Sut.cdx-- the signed SUT. Do NOT usebuild-output/bare-metal/Codex.cdx(unsigned boot kernel). - Self-verify:
build/test-self-verify.ps1. Must print "THE SEED VERIFIES ITSELF". - Capture digest:
Get-FileHash -Algorithm SHA256 seed\Codex.cdx - Submit to Perforce.
Rules:
- Never skip the full build. Never skip self-verify.
- One seed per CL. CDX is primary.
- The bootable image (
seed/Codex.img) is a separate distribution artifact built bybuild/build-boot-img.ps1. It is NOT part of the seed rebuild. - Signing is automatic.
- Never
git add -A. Never force-push.
Codegen changes need a one-pass fixed point. Step 2's "install
Sut.cdx" is correct ONLY when the build prints
(SUT === stage1 -- hard fixed point in one pass). A change to code
generation (e.g. emit-prologue) built from a pre-change seed is
two-pass: the stage0 seed lacks the new codegen, so its Sut carries
the change only in its emit logic, not yet in its own function bodies --
Sut != stage1, and the real fixed point is stage1 (NewSeed.cdx, which
is unsigned). Installing Sut.cdx there leaves a seed that is one pass
short: it compiles correctly but is not byte-identical to itself
(seed != seed-compiles-seed), so it must not be trusted or copied up. The
fix is simply to rebuild again from that once-built seed -- the second
build converges one-pass with the change baked into the seed's own
prologues, and Sut.cdx is then the signed fixed point. Always rebuild
until the build reports one pass before submitting a codegen seed.
But a codegen change can be VERIFIED without any of that. A gate followed by rebuilding whatever app you care about compiles that app with the changed compiler, and no seed is written or submitted anywhere. The seed rebuild above is how a codegen change LANDS, not how you find out whether it works. Worth knowing before queueing for a seed cycle to answer a question one gate already answers.
Pass -Kernel when you do, because the default is not what you think.
build-output/bare-metal/Codex.cdx is not the SUT, and it is not the seed
either.
Build-Cdx and Build-Text in build.ps1 each copy their own kernel
over that path before running, so it ends up holding whichever kernel the
LAST compile phase used, not whichever artifact the run produced.
Measured after each of two green build.ps1 runs on 2026-08-11, its
content hash equalled seed/Codex.cdx exactly (AF4E14D9703985AC),
because the phases after plug-binary compile against the seed.
That is the dangerous direction. Verifying a codegen change with the
default kernel boots the OLD compiler, the emitted wire is unchanged, and
the honest reading of that is "my fix did nothing" when the fix was never
in the binary under test. It has already produced a wrong answer: a
per-chapter sweep of apps/works reported roughly 80 of 84 chapters
compiling where the real figure against the depot seed was about 55,
because build-output held a kernel from before the file-exists
builtin was removed and every chapter calling it still resolved. The
sweep was measuring a compiler that no longer existed.
So compile.ps1 prints the kernel and its digest on stderr on every run,
and warns when you did not ask for one and it is not the seed. Read the
line. If you are measuring anything at all, name the kernel:
build/compile.ps1 -Src x.codex -Out x.cdx -Log x.log -Kernel build/output/Sut.cdxbuild/output/Sut.cdx is what a gate just built; seed/Codex.cdx is the
shipped compiler. Compare the printed digest against the one you think
you are testing.
And the default is resolved RELATIVE TO THE CURRENT DIRECTORY, so the same
command works from the repo root and reports a missing kernel anywhere else.
compile.ps1:58 sets $Stage0 = Join-Path 'build-output' 'bare-metal' 'Codex.cdx', a relative path, and prints MISSING: build-output\bare-metal\ Codex.cdx - run build.ps1 first. Run from a subdirectory, or from a
background call that inherited a different location, that reads as the
COMPILE failing rather than as the path not resolving, and three subjects in
a row reported as failing to compile is convincing. cd to the repo root
before any compile, or pass -Kernel with an absolute path.
(The CDX fixed-point check compares content ignoring the signature bytes,
so a signed Sut and the unsigned stage1 still register as one pass when
their code is identical. On copy-up, the gate is Sut === seed rebuilt on
the target workspace -- see docs/Agents/PerforceProcess.md.)
Building the compiler by hand needs -Repl, and without it you get a
different binary that looks like a codegen difference. Invoke-BuildCdx
in build.ps1 compiles every stage with compile.ps1 -Repl, and nothing
outside that function does. Measured 2026-08-09 against the same source and
the same kernel: -Repl gives 2,753,304 bytes and is BYTE-IDENTICAL to the
gate's build/output/stage1.cdx, while omitting it gives 2,753,312. So a
hand-built compiler compared against stage1.cdx or the seed disagrees by
8 bytes for a reason that has nothing to do with the change under test.
Pass -Repl whenever the artifact is meant to BE the compiler; leave it off
when compiling an ordinary program. L-SAMEVER: confirm the two things you
are diffing were built the same way before concluding anything about either.
That difference is real codegen and is meant to stay. X86_64Chapter
ends the chapter on st.exit-mode: Exit emits the shutdown epilogue
(li rax,0, li rdx,244, out dx,al, hlt, jmp self) and anything else
emits a single jmp repl-loop. A REPL binary that shut the machine down
would be the defect. Do not try to make the two byte-identical.
The quarter of a million differing bytes between the two modes are that
one stub and nothing else (COMPILER-3, closed 2026-08-16). The item was
filed because a positional byte compare of the two arms reports ~9 per cent
of the file different, first difference at offset 8, spread the whole
length, which reads as an emitter that answers differently in the two
modes. It is an artifact of comparing at absolute offsets. Measured against
seed 55983566 on the 2,817,743-byte concatenation of codex/compiler,
-Repl at 2,798,023 bytes against 2,798,031 without it, 258,186 bytes
differing:
| region | size | differing | what it is |
|---|---|---|---|
| header | 276 | 37 | the content hash at bytes 8-39, plus 5 length and offset fields |
| code | 2,390,823 | 3,744 | 3,641 sites, every one a 32-bit absolute data address, all exactly +8; nothing unclassified |
| data | 254,626 | 126,893 | identical under a +8 shift but for 9,136 embedded 64-bit pointers, themselves +8 |
| MAP1 and tail | 152,298 | 127,512 | identical under a +8 shift but for ONE byte: __start's size, 9545 against 9556 |
__start is the last function in the code section, so the 11 bytes the
Exit epilogue adds over jmp repl-loop push everything after it;
alignment padding absorbs 3 and the rest of the file relocates by 8. The
two embedded maps agree on all 5,194 names and offsets apart from that one
size. Every differing byte in the code section is an address that moved,
not an instruction that changed: the emitter answers identically in both
modes. Control, and run it before believing any number here: the same arm
compiled twice is byte-identical, so nothing in this is nondeterminism.
What was NOT meant to stay was map riding on the same switch, which
is why a compiler build emitted no symbol map. Fixed 2026-08-14; the two
flags are independent in compile.ps1 now, as they always were in
emit-cdx.
Both are silent, both look like a result, and both have produced a published number that was not about the change under test.
build/compile.ps1 prints True on a FAILED compile. The True is the
script's own success, not the compiler's verdict, so a probe that reads it
goes on to run whatever .cdx was already at that path. Delete the output
first, gate on the log, and check the file exists afterwards:
Remove-Item probe.cdx -ErrorAction SilentlyContinue
build/compile.ps1 -Src probe.codex -Out probe.cdx -Log probe.log -Kernel seed/Codex.cdx
if (Select-String -Path probe.log -Pattern "error CDX" -Quiet) { throw "compile failed" }Match the number form as well as the CDX form when a harness does this for
you: error 3010: Unresolvable cite carries no CDX prefix, and a checker
matching only error CDX served a four-day-old IR file as a fresh verdict
twice in one session. Comparing the output's LastWriteTime against the
clock is the habit that catches the whole class.
tools/codex-vm.exe -kernel X.cdx -headless by hand prints NO program
output, and does not error. Measured 2026-08-08: one probe read as a crash
(EXC=06) and a variant read as a clean silent exit, and a trivial
print-line-uni (show 15) control run the same way ALSO printed nothing --
so both readings were void, and the silence was the invocation rather than
the subject. Run programs through build/test-run.ps1, which supplies
-output and the comparison the battery uses. The control is what caught it,
and it cost two cycles before one was run (L-SIDECAR).
The fixed point proves the seed is a stable fixed point of itself. It does not prove the seed is honest. A compiler carrying a Thompson trojan is a perfectly stable fixed point too: it recognises its own source, re-injects the payload into every compiler it emits, and the payload never appears in any source you can read. Self-consistency cannot detect this, and neither can reproducible builds, because a poisoned compiler is deterministic.
The answer is Wheeler's diverse double-compiling. Take the compiler under
suspicion A with source sA, and an unrelated compiler B:
X = A(sA) the official build -- our seed/Codex.cdx
stage1 = B(sA) rebuild A's source with the unrelated compiler
stage2 = stage1(sA) compile A's source once more with that rebuild
compare stage2 against X
Honest A gives stage2 == X byte for byte, because A and stage1 are
the same program built two ways. A poisoned A injects into X but not
into stage2, and the comparison goes red.
A different target architecture adds nothing. Emitting the compiler to
RISC-V and compiling back to x86 looks like an independent path and is not
one, because the RISC-V binary is A(sA, target=riscv) -- the seed built
it, and the seed had its chance to inject on the way out. The same holds
for a different host, a different board, or a different emulator. Only a
different implementation of the source-to-binary function buys
independence. Cross-lane runs are worth having as differential codegen
tests; they are not a trusting-trust defense and must not be described as
one.
codex/plugs/csharp/ renders Codex IR into a single C# file. Built by
Roslyn, whose lineage has no relationship to ours, it is the only B
available that stays current as the compiler evolves. The retired reference
compiler under old/ was a genuine hand-written second implementation and
is the formally cleaner witness, but it is frozen at an April language
version and can only ever check a seed nobody ships. Do not delete the C#
plug as unused. Its job is this.
codex/plugs/csharp/emit-compiler.ps1 already runs most of the pipeline:
codex/compiler/*.codex --concat--> Codex.codex
--seed----> compiler.ir
--csharp-plug--> Codex.cs
All three further steps are now taken, and the result is byte-identical.
Compiled in the mode the gate actually uses, the Roslyn arm reproduces
build/output/NewSeed.cdx (the gate's own unsigned stage1) with zero
differing bytes across the whole file. Against the shipped seed/Codex.cdx
the only difference is the 96-byte signature region at offsets 40..135,
which is zeros in an unsigned build and is stamped in place by the sign
phase rather than emitted by the compiler. Header bytes 0..39 and 136..223
agree and the entire body from 224 agrees. The source -> IR arm matches
too: both compilers run IR-UNI over the whole compiler source and emit
byte-identical IR. Nothing in build/ invokes any of this on the gate; it
is run on demand.
Five rules make the comparison mean anything. Each was learned by a false result.
- ALWAYS compile the compiler with
CDX replwhen comparing against the seed.build.ps1passes-Replandcompile.ps1turns that into theCDX replmode line; without-Replit appendsmapinstead. Exit mode is a real codegen difference, soCDX,CDX mapandCDX replare three different binaries from one source. Two content hashes disagreeing is not evidence of a defect; it is evidence that two different things were compiled. - Run
emit-compiler.ps1in FULL before any comparison.-SkipIrreuses the previouscompiler.irand therefore theCodex.codexit was assembled from, and a merge-down movescodex/compiler/underneath you with no signal. That is how a stale bundle passed for a compiler defect through two sessions, surviving six individually sound eliminations that were all aimed at the transpiler while the untested assumption -- that both arms were the same program -- was the only wrong one (L-SAMEVER). CompareLastWriteTimeonbuild-output/Codex.codexagainst the newest file undercodex/compiler/before believing any disagreement. Reserve-SkipIrfor iterating on the plug, where the compiler source is genuinely fixed. - Give both compilers the same input, and do NOT use
compile.ps1to do it. It assembles a unit (cites resolved, chapter lines rewritten) and that is not the file. Write the mode line, the source and a trailing0x04as UTF-8 without BOM, feed that to the .NET build's stdin and tocodex-vm -input. The stderr lineall_consumed=1confirms the guest read every byte. Both sides take the mode on stdin as one line, then the source. - The symbol map is the instrument for CDX, not the byte diff. Pass
CDX mapand compare theMAP:block. One early size difference shifts every later absolute address, so a raw byte diff reports near-total disagreement and localises nothing; the map gives per-helper sizes at relative offsets. opening's Integer return does NOT surface as the codex-vm exit code. A control returning 7 also reports 0, so reading an answer out of an exit code can "find" exactly the bug you are hunting. Print through the console and-output.
One known structural limit, not a defect to chase. The C# build prints
warning CDX9003: pmap-walk self-test FAILED. pmap-self-test builds
records and walks their raw memory; on .NET records are CLR objects rather
than bytes in the arena, so the walk sees nothing. It is a diagnostic and
does not alter emitted code, which is why the CDX still matches byte for
byte while the warning prints.
This is a release gate (release skill, step 4).
The pass has two conditions: the arm's output is the same LENGTH as
seed/Codex.cdx, and ZERO bytes differ outside the signature region at
offsets 40..135. How many differ inside it is not a criterion and must
not be quoted as one -- 96 is the region's WIDTH, and two unrelated
signatures agree at a given byte about one time in 256, so a run differing
in 95 of the 96 is ordinary. Measured 2026-08-10 against seed AF4E14D9:
96. Measured 2026-08-12 against seed 527C2C75: 95, on both arms
independently.
$R = 'D:\Projects\Cobblestone-<agent>-main'
# 1. Emit. Do NOT pass -SkipIr on a release run.
& "$R\codex\plugs\csharp\emit-compiler.ps1" -Kernel "$R\seed\Codex.cdx" `
-Out "$R\build-output\Codex.cs"
# 2. Build the arm with Roslyn. Scaffold once: a net9.0 Exe csproj beside
# Codex.cs, Nullable disable, LangVersion latest.
dotnet build "$R\build-output\ddc-arm\CodexCs.csproj" -c Release
# 3 and 4 are build/ddc-witness.ps1. It builds ONE input file and feeds it to
# BOTH arms, slices each at the CDX1 magic and applies the two conditions.
build/ddc-witness.ps1 -Repo $R -Source "$R\codex\plugs\csharp\build-output\Codex.codex"Steps 3 and 4 are scripted rather than prose, because doing them by hand
re-derives the one proof that does not take the compiler's word for
anything. The part that is easy to get wrong is
reconstructing the exact bytes compile.ps1 feeds the compiler (the mode
line is CDX repl, because the seed is built -Repl), and a wrong
reconstruction makes the C# arm disagree for a reason that has nothing to do
with trust. So the runner hands the SAME input to both arms and treats the
Codex arm reproducing the shipped seed as the precondition for the C# arm's
answer meaning anything; if it does not, the run is INCONCLUSIVE rather
than red.
build-output/ does not survive a gate run, and both DDC prerequisites
live there: the ddc-arm csproj scaffold and csharp-plug.cdx. Expect to
rebuild the plug (plugs/csharp/build.ps1 -Force) and re-create the csproj
on any release where a gate has run since the last one. The plug's builder
pins -Kernel to <repo>/seed/Codex.cdx, resolved from the repo root rather
than from the process working directory (plug-build-lib.ps1:149), so the
seed under audit has to BE the one in seed/; staging a kernel anywhere else
does nothing.
Four things that will give you a false answer, three of them measured here on 2026-08-10.
- The plug must be built with the seed under audit, and
seed/Codex.cdxis the only seed it will use.codex/plugs/common/plug-build-lib.ps1:149 passes-Kernel <repo>/seed/Codex.cdxexplicitly; neitherplugs/<name>/build.ps1norbuild/plug-build.ps1takes a-Kernelto override it. Auditing any other seed means installing it at that path first. Read thekernel:digest the compile prints and check it against the seed you meant to audit. (This paragraph said the opposite until 2026-08-20, when the builder was measured rather than remembered.) - The C# arm writes diagnostics to stdout AHEAD of the binary. The run
above put 67,380 bytes of warning text before the CDX. Find the
CDX1magic and slice from there; comparing the raw stream reports total disagreement and localises nothing. - Do not trust
plugs/csharp/build.ps1's exit code alone; check that the plug binary is actually newer than the bundle. On 2026-08-10 it reported "FAIL: compile errors" twice on a bundle that compiles clean, naming abuild.logthat was not at the path it printed, then succeeded on an identical third invocation. The cause is NOT established and both earlier failures were in runs chained behind another codex-vm, so file contention is a suspect and nothing more. What is certain is that the emit step will happily use a stale plug and produce a confidently wrong answer. - A new compiler builtin needs a matching
BuiltinEmitterin the plug, or the emitted C# references a name that does not exist and Roslyn refuses it. Ordinary upkeep, the same as teaching any transpiler a new primitive: add the entry beside the others incodex/plugs/csharp/CSharpEmitterExpressions.codex; the stub is"0L"wherever a hosted C# build has no such device. Running the DDC on the release is what surfaces it if the entry was missed.
A DDC that only ever comes back green is the exact instrument L-FALSIF warns about. It has been falsified deliberately (2026-08-10), and the procedure is worth keeping because the result names the boundary.
Poison the compiler in one source site, build A' from the poisoned
source WITH THE CLEAN SEED, then REVERT the source before using A' -- so
A' is a compiler binary whose payload appears in no source in the tree,
which is Thompson's setup. Run X = A'(clean source) and the witness
chain (A' -> IR -> C# -> stage1 -> stage2), then compare stage2 against
both X and the clean seed.
Two sites were tried, each semantically inert so the compiler still works and only the bytes move:
- Code generator (
X86_64Helpers.codex, one extra0x90in a wait loop): stage2 differed fromXby 2,232,990 bytes and matched the clean seed to the signature. The payload never reached the IR (its size was byte-for-byte the clean IR's), so the C# arm built an honest stage1. - Lowering (
fold-constants-in-chapterstubbed toch): stage2 differed fromXand again matched the clean seed. The payload DID reach the IR this time, but stage2 still reconstructed the honest compiler, becauseA'merely failed to fold -- it did not teach its output to fail to fold. - Frontend IR emission (
X86_64Helpers.codexemit-read-line-helper, the same one[144]nop as the code-generator arm, PLUSIRTextEmitter.codexir-emit-defreturning that function's trojaned IR def verbatim whenever it emits that function): stage2 ==Xbyte-for-byte (SHAEEDFECC9...) and both differ from the clean seed by 2,104,373 bytes. This one is NOT caught by the byte-comparison. The hook makesA'write the trojanedemit-read-line-helperinto the IR of clean source, so the C# arm builds a trojaned stage1 and stage2 carries the same payload asX. It is NOT self-reproducing:ir-emit-defsubstitutes a static def and stage1 never re-emits IR. Its own control is the code-generator arm above -- the same nop with no hook, which IS caught -- so the hook is the only variable that flips the verdict from caught to survived. Measured 2026-08-11.
The boundary is "reaches the readable intermediate" versus not, not
"self-reproducing" versus not (corrected 2026-08-11). The frontend-IR arm
survives with no self-reproduction, so the earlier "the one payload that
would survive is a self-reproducing quine" was too narrow. What holds: a
payload living only in the binary is caught anywhere (arms one and two),
because the rebuild starts from clean source; a payload survives the
byte-comparison only if A' writes it into the IR that becomes the C#.
And that is the neutralization, not a leak. A survivor is, by
construction, sitting AS READABLE TEXT in the IR and in the emitted C#:
measured, the injected st7z / [144] was greppable in both. So the two
halves cover the whole space. The byte-comparison catches anything hidden
only in the binary (arms one and two, the classic Thompson shape). Anything
that slips past it had to reach the IR that becomes the C#, and is therefore
visible as text that can be grepped, diffed across re-emissions, and
content-addressed. Thompson's attack depends on the payload being invisible
because a binary is unreadable; here there is no invisible surviving form. A
trojan hidden in the binary is caught by the rebuild; a trojan that survives
the rebuild is not hidden. That readable-intermediate property, not the
byte-comparison alone, is what defends against the Thompson attack here. See
"The residual hole, stated plainly" below for the two upstream seed positions
this covers.
If you add a DDC sabotage arm to a harness, confirm the probe FIRES first (stage2 must differ from a clean build) before trusting a red verdict -- an inert edit that never reaches the output is a vacuous arm.
jonquil: FAIL -- no IR-BEGIN/IR-END usually means the guest DIED, not
that anything is wrong with the IR. The phase compiles the whole compiler
to IR under the SUT and greps the result; when the compile dies the log is
near-empty and the grep reports the missing markers, which reads like a
finding about the quine check. Look at the size of
build/output/quine-check/compiler.ir.log before believing the message: a
real run is about 15 MB, and the failure that produced this note was 5 bytes.
Measured 2026-08-16 on a box with four agents gating at once; re-running the
phase alone against the same Sut.cdx scanned 4,709 defs and passed, with
nothing changed in between, and two later full gates were green. So it is a
contention flake, and the cheap repair when someone owns this script is for
the phase to distinguish an empty log from a real answer rather than
attributing it to the IR.
Two source sites read as dead bindings and are NOT. Leave them alone.
X86_64Helpers.codex:1539 binds st12c = emit-list-tail st12b and the next
line continues from st12b; X86_64Chapter.codex:450 binds
st45j = patch-jcc-at st45i ... and the next line continues from st45i.
Both are correct ONLY because __record-set mutates in place, and
check-stale-reads (TypeChecker.codex:3114) exists to warn about exactly
this shape. A tidy-up there is seed-affecting and would break the seed.
The seed sits upstream of this pipeline in two places, not one.
- The
source -> IRstep iscompile.ps1, which is the seed. A poisoned seed could inject into the IR that becomes the C#. csharp-plug.cdxis itself a Codex program the seed compiled. The emitter that renders IR into C# iscodex/plugs/csharp/*.codex, built by the same seed under suspicion, soAalso controls the translation intoB's input language. This is easy to miss because Roslyn's independence is real and draws the eye to the last step.
As written, the pipeline is not independent. Two things narrow it:
- Have the transpiled compiler do its own
source -> IRstep.Codex.csis the whole compiler, parser included, so once Roslyn has built it, it should readCodex.codexitself rather than consuming IR the seed produced. That leavesAresponsible only for generating the C# text. - What remains must survive rendering into readable C#. Thompson's attack works because binaries are unreadable; a payload forced through ~300 KB of generated source has to exist as text that can be grepped and diffed across re-emissions.
That is narrower than real DDC and should be reported as narrower. The full
claim requires a B the seed never touched.
Two independent sampling profilers, both driven by the PIT tick (~18 Hz). codex-vm now delivers timer interrupts to a compute-bound guest (a 55 ms kicker thread cancels the VP each period), so a self-compile is sampled -- before 2026-07-07 the timer only fired while the guest was halted, and long compute ran blind.
Host sampler (recommended -- bias-free). Set CODEX_VM_PROFILE=<file>
before booting; on each timer kick codex-vm records the guest RIP at the
point of cancellation and writes one HPROF:<hex-rip> line per sample on
exit. No guest cooperation, no injection-delivery skew.
$env:CODEX_VM_PROFILE = "prof.txt"
build/compile.ps1 -Src build/output/Codex.codex -Out out.cdx -Log c.log
$env:CODEX_VM_PROFILE = ""
build/prof-report.ps1 -Log prof.txt -Map seed/Codex.map -Top 25Guest sampler. prof-start/prof-dump builtins bracket a region; the
timer ISR records the interrupt-frame RIP into a ring at cell 0x60000.
compile.ps1 -Profile wraps the CDX pipeline in these (CDX profile
mode). It emits PROF:<count> then one PROF:<rip> per sample. Skews
toward hot call targets (WHP delivers the injected interrupt at the next
instruction boundary), so prefer the host sampler for accurate weights.
build/prof-report.ps1 -Log <capture> -Map <symbol.map> resolves either
format's *PROF: lines into a hot-function histogram. Compile it the
way the thing you are profiling was compiled and use the <out>.map
sidecar beside it. That is now possible in both modes: until 2026-08-14
compile.ps1 appended the map flag only when -Repl was ABSENT, so a
compiler build emitted no map at all and this line told you to mint one
NON-repl instead. It no longer does, and you no longer need to.
The coupling was in the driver, not the compiler. emit-cdx reads repl
and map as two independent flags, and emit-binary-tail-with-map writes
the same bytes as emit-binary-tail and then prints the MAP: lines --
the map never enters the binary. Measured 2026-08-14: CDX repl and
CDX repl map are byte-identical (84,607 both) and the second emits a
4,786-byte map. The symbol map is built into EmitChapterResult
regardless, so map only decides whether it is printed; it was being
computed and thrown away on every compiler build.
How much a mismatched map actually costs, since the docs did not say.
Measured over the same 158-symbol program: the non-repl and repl maps
differ on exactly ONE line, __start's SIZE (9784 against 9773), and
every symbol ADDRESS is identical. So a mode-mismatched map is not the
scary case. The stale case is -- seed/Codex.map describing an older
binary is what put a crash in sorted-builtin-names when the faulting
function was find-effect-op-addr (2026-07-16, an hour). Now that a
-Repl build emits <out>.map, codex-vm's -map auto-probe finds the
right file first and never reaches the seed/Codex.map fallback.
Re-concatenate before you mint, or the fresh map describes stale
source. build/output/Codex.codex is written by build.ps1's source
phase, so it is only as current as the last full build in THAT workspace.
Compiling it gives a genuinely fresh map of whatever the tree looked like
then, which is the worst of both: it carries no staleness marker and every
symbol in it is confident. Run
build/concat-codex-self.ps1 -OutFile <path> first and compile that.
Measured 2026-08-04 in a workspace whose last build was five days old: the
checked-in build/output/Codex.codex was 3007248 bytes against 2993576
from a fresh concatenation of the same tree.
Two things a fresh map is NOT. It is not a replacement for
seed/Codex.map: that file must describe the binary that actually ships,
and installing a map minted from a compiler source ahead of the seed makes
every backtrace it resolves wrong in a way nothing announces. And the
symbol-count delta between the two is a measure of how far compiler source
has moved ahead of the seed, not of anything being broken -- source landing
without a seed cycle is normal.
The sample buffer lives at 0x60000 (profiler) / 0x70000 (alloc trace), in the free low-memory band above the AP stacks; earlier it sat inside the page tables and enabling it destroyed them after ~88 samples.
build/wcet-validate.ps1 empirically validates punctual WCET claims:
it compiles a program, parses the CDX6010 static counts and budgets
from the compile log, then runs the binary under codex-vm -wcet
(hardware execution breakpoint at each function entry + TF
single-step; callee instructions excluded, matching the static count's
per-body semantics) and gates on observed <= budget per
invocation. CODEX_VM_NO_TIMER=1 is set for the observation runs.
build/wcet-validate.ps1 # default driver (codex/test/wcet-probe.codex)
build/wcet-validate.ps1 -Src path/to/prog.codex # any punctual programVerdicts: PASS (observed within budget and static claim), WARN
(function never entered -- the inliner consumed every call site),
FAIL (observed > budget, observed > static, or instrumentation
failure). Exit 0 unless FAIL. The static CDX6010 count is a decode
of the function's finished bytes (X86_64InsnCount.codex, taken
after NOP compaction), so it is exact: punctual code cannot loop,
every dynamic path is a subset of the body, and an observation above
the static count means the accounting is broken. An encoding outside
the decode vocabulary raises CDX6012 (count unavailable) instead of
reporting a partial number.
Note: -wcet is the only mode that host-intercepts exceptions
(ExtendedVmExits.ExceptionExit + bitmap narrowed to #DB). In every
other mode the bitmap is inert and all exceptions go to the guest IDT
(the !EXC dump protocol) -- INT3/vector 3 always belongs to the guest.
tools/status-server.ps1 serves a single-page dashboard at
http://localhost:8080/ with live data from the workspace and Perforce.
pwsh tools/status-server.ps1 # default port 8080
pwsh tools/status-server.ps1 -Port 9090 # custom portDisplays: seed CDX info, test battery results, module counts by quire, recent changelists. Auto-refreshes every 30 seconds.
seed/Codex.map is a text file mapping code addresses to function
names, emitted during each seed build. Format:
# Codex Symbol Map
# Address Size Name
0x00100114 7 __alloc
0x0010011B 251 __str_concat
...
0x002F56FB 614 lookup-expr-type
Each line: hex address, decimal byte size, function name. Use it to
identify crash locations from !EXC diagnostic output:
# Look up a crash RIP in the map
$rip = 0x2f588b
$lines = Get-Content seed\Codex.map | Where-Object { $_ -match '^0x' }
foreach ($line in $lines) {
if ($line -match '^(0x[0-9a-fA-F]+)\s+(\d+)\s+(.+)$') {
$addr = [Convert]::ToInt64($matches[1], 16)
if ($addr -le $rip -and ($addr + [int]$matches[2]) -gt $rip) {
"$($matches[3]) at $($matches[1]) (offset +$($rip - $addr))"
}
}
}The map is rebuilt whenever the seed is rebuilt. It tracks the
compiler's own functions -- not compiled test programs. Test CDX
binaries emit their own MAP block in the build log (visible between
MAP: and MAP-END lines).
All debugging uses codex-vm and the PowerShell harness. No GDB, no WSL, no external tools. The compiler embeds a binary MAP1 symbol map in every CDX (2600+ functions, ~79KB), and the harness resolves addresses automatically.
Resolve a fault with the map belonging to the binary that faulted.
codex-vm's backtrace loads seed/Codex.map whatever it actually booted, so
every symbol in a boot-PE or app fault is confident nonsense. Re-resolve
against that binary's own map (build-output/boot.map for the boot PE).
MMIO: cannot size instruction at RIP=... is not a demand-paging
message. It fires only when the faulting instruction cannot be SIZED. To
tell a livelock from a runaway allocation, group the reported GPAs: repeats
mean a livelock, unique and ascending means the guest is allocating.
A UEFI PE stub is not __start and gets none of its side effects. The
stub calls opening directly, so everything emit-start establishes -- the
SystemTable at uefi-systab-addr, the process table, capability grants, the
IDT, the PIC -- is simply absent unless the stub does it too. The failures
are all action-at-a-distance: a null SystemTable presents as a call to
address 0 from whichever uefi-* helper happens to lack a null guard.
When a crash occurs during batch compilation, the harness prints a resolved crash report:
CRASH in peek-qword+0x58 (page fault, CR2=0x000100000000)
RIP 0x00107DFE peek-qword+0x58
callR 0xBDFFFFC8
R10 0x00600000 (heap @ 0 MB)
Stack trace (heuristic):
S[58] 0x0010AB54 opening+0x2C
S[68] 0x001116A3 __start+0xDE2
The !EXC line from the guest's exception handler includes RIP, all
callee-saved registers, CR2, callR, and 16 stack qwords.
Format-CrashReport (vm-config.ps1) resolves every value in the code
range (0x100000-0x400000) to a function name.
The dump prints the syscall MSRs, read back off the virtual processor rather than out of codex-vm's own shadows:
EFER=0000000000000d01 STAR=0000000800000000 LSTAR=0000000000147098 SFMASK=0000000000000200
Without them, a guest that jumped somewhere absurd through an unprogrammed
LSTAR and one that got there any other way produce the identical picture, and
the fix for the first is invisible when it works. That ambiguity is exactly what
made the UEFI boot path look unfixed after it had been fixed.
A guest rdmsr is answered from a host-side shadow, not from the
processor. handle_msr keeps msr_efer, msr_star, msr_lstar,
msr_sfmask, msr_cstar, msr_kernel_gs_base and msr_apic_base, and
anything not in that list reads zero. So any place codex-vm sets one of those
registers on the VP directly has to set the shadow with it: a guest doing the
ordinary rdmsr; or rax,1; wrmsr to add SCE would otherwise read 0, write 1,
and clear LME and LMA in the middle of long mode. The UEFI entry path and the
UEFI trampoline fixup both set EFER=0xD01, and both now set msr_efer too.
callR is the interrupted RSP, not a return address, whatever its
name suggests. The dump reads it from offset 64 of the handler's own
stack, which is the RSP slot of the CPU's interrupt frame. It is
normally a stack address near the top of RAM and resolves to no
symbol; if it ever does resolve, that is a coincidence of the value
landing inside a function's byte range, not a caller.
Only part of the S[] dump is the interrupted program's stack. The
guest dumps sixteen qwords starting at the handler's RSP, and the
first eleven are the five registers the handler pushed (RDI, RSI, RDX,
RCX, RAX) followed by the five-word interrupt frame (RIP, CS, RFLAGS,
RSP, SS). Real stack begins at S[0x50]. This is why the trace is
labelled heuristic: the harness cannot tell a return address from a
saved register or a stale slot, so it prints whatever resolves to a
symbol and drops the rest.
Resolve-Rip and Resolve-Name are functions in build/vm-config.ps1.
Dot-source it, then call them:
. build/vm-config.ps1
Resolve-Rip -Rip 0x2748af # -> "function+0xNN"
Resolve-Name -Name "lookup-expr-type" # -> 0x2F56FB (address)Both read seed\Codex.map by default; pass -MapFile to point at
another map.
From a symbol to the emitted bytes. compile.ps1 writes a <out>.map
sidecar of 0xADDR <size> <name> rows, and the file offset of that address
inside the CDX is addr - 0x100000 + 224 (load address, then header). Dump
with [IO.File]::ReadAllBytes and decode. Reading the bytes is faster than
another round of ablation once the conditions are narrowed: ablation tells
you which conditions matter, the bytes tell you what happened. One decoding
trap worth naming, because it misreads as a different register: REX.R and
REX.B extend the modrm fields, so 4c 8b 4d 90 is mov r9,[rbp-0x70] and
not rcx.
Resolve a fault with the map belonging to the binary that faulted. The
-map auto-probe falls back to seed/Codex.map whatever was booted, so a
boot-PE backtrace resolved against it is confident nonsense. The MAP1 map
embedded in the CDX is authoritative, and its names are CCE -- which is why
an ASCII grep of a .cdx for a symbol answers False for symbols that are
certainly present.
Patch INT3 at a function's entry point. The guest exception handler
fires !EXC=03 (vector 3) and dumps register state.
build/compile.ps1 -Src foo.codex -Out foo.cdx -Log foo.log `
-Break "lookup-expr-type"Output:
BREAK: patched INT3 at lookup-expr-type+0x0 (0x2F56FB, orig=0x4C)
CRASH in lookup-expr-type+0x1 (breakpoint)
RIP 0x002F56FC lookup-expr-type+0x1
R10 0x00c00000 (heap @ 6 MB)
RDI 0x00000010
Exit code 5 = breakpoint hit (vs 4 = real crash).
crash_filter in tools/codex-vm.c catches a fault in codex-vm ITSELF,
prints HOST CRASH: to stderr, and exits 0xC0DE. The distinction
matters because a harness that only captures -output sees an empty
serial log and a nonzero exit and reads it as a guest that produced
nothing. Redirect stderr or the diagnosis is invisible.
FIXED 2026-08-18 (root). A failed demand-commit was reported as a guest
fault, and the crash reporter then read the page it was describing.
guest_mem is committed in 2 MB chunks on the guest's first touch; when
VirtualAlloc(MEM_COMMIT) or WHvMapGpaRange refused (the box's commit
charge: several agents' VMs, each committing whatever its guest touches of a 3 GB reservation), the run loop fell through
to the Unmapped MMIO GPA=... crash report as though the guest had walked
off the map, and dbg_dump_mem read the uncommitted GPA to print "Memory at
fault address" and killed the process: HOST CRASH ... at 0x...7C35. That
is the intermittent brotli-interop crash of 2026-08-16 (Address: 0x70a00000 Access: WRITE then HOST CRASH), reproduced by refusing the commit past
0x70000000 in an ablation build: same two lines, same low address bits.
Now the failure prints HOST: cannot back guest RAM at GPA 0x...: VirtualAlloc(MEM_COMMIT) failed, err=1455 (ERROR_COMMITMENT_LIMIT) -- the host is out of commit charge, this is not a guest fault (or the
WHvMapGpaRange hr), the report's reason line says Guest RAM commit failed, and every host read the reporter makes (dbg_dump_mem,
dbg_backtrace, dbg_dump_stack, dbg_disasm_at) asks VirtualQuery
first and prints host page not committed instead of dying; the backtrace
had a second instance one frame past __start (RBP chain ending at
0xbe000000). Same CL: Code at RIP had printed EMPTY in every report on a
3 GB guest because (int)(guest_mem_size - addr) went negative for any RIP
under 1 GB; the clamp now happens before the narrowing. If you see the
HOST: line, the fix is on the box (free memory or fewer concurrent VMs),
not in the guest.
FIXED 2026-08-10 (blu). Setting a VBE mode activated the framebuffer at runtime without committing the guest region the emulator reads back. The same defect as the oversized-disk crash below, other direction: that one wrote past the commit, this one read out of it.
The startup path commits 0xBE000000..GOP_FB_ADDR+fb only when
gop_active is ALREADY set, and the failing run's own log line says it is
not (gop=0), so the region stayed MEM_RESERVE. The Bochs VBE handler
then sets gop_active and vbe_active at runtime and allocates only the
HOST buffer. sync_shadow_buffers bounds its memcpy against
guest_mem_size -- 0xBF000000 + fb is inside 3 GB, so the bound passes --
and reads reserved uncommitted address space. guest_commit_range is the
existing remedy and the GPU texture and asset upload paths already call it
from this same I/O handler; the VBE handler did not.
The read runs on an exit counter (exits % 64), and that is what makes
this expensive to probe. A guest that sets the mode and returns exits
before the counter comes round and passes on a broken emulator: the first
probe written for this did exactly that, twice, and the second version
added 8 filler lines that bought 8 exits because serial is ring-buffered
rather than a port write per byte. codex/test/vbe-mode-set calls
vbe-read 200 times for the port pairs, and sums every value into its
answer so nothing can elide them. Measured both arms on one binary: depot
codex-vm.exe#92 faults the host (exit 49374, no serial), the fixed build
prints 1 2 0 204800 1024 at 612 exits.
This test is easy to disarm silently. It once took its exits from
drawing, and when gfx-put-pixel stopped
banking (below), the exits fell from 1011 to 211, the pre-fix binary stopped
crashing, and the recorded output did not move a single byte -- a guard
reporting exactly what a working one reports. If you change what this test
draws, re-run the old binary before believing it still guards anything.
The original measurement, unchanged:
Measured 2026-08-09 (blu): setting VBE mode crashes the
host. apps/browser/opening.codex compiles clean (493057 bytes off seed
A1EBA5A03016A128) and dies immediately on boot:
VBE: mode set 1024x768 fb=0xfd000000
HOST CRASH: codex-vm faulted (code=0xC0000005) at 0x7FF7F24D5A0D reading memory 0x22715520000
Reproduced three times; the faulting code address is identical every run
and the host address it reads varies, which is what a per-run mapping base
does. One byte of guest serial appears before the fault, so the guest never
reaches its first print-line.
The log line above it is the lead: RAM cap: guest_mem_size=0xc0000000 effective=0xbe000000 gop=0, and the VBE framebuffer is handed out at
0xfd000000, which is above effective. Raising -mem does not raise
effective -- at -mem 4096 the line reads guest_mem_size=0x100000000 effective=0xbe000000 and the fault is unchanged, so the cap is not derived
from the requested size. That refutes "give it more RAM" as a fix and
points at the framebuffer mapping rather than at the guest.
Whatever the guest did, a guest cannot be allowed to fault the emulator: the correct behaviour is a guest fault, not a host access violation.
codex/test/apps/browser-keys passes and is not a counter-example: it
never sets a video mode, so the whole display path is outside what the
browser's gate observes. That is why the browser reads green and does not
run.
FIXED 2026-08-10 (blu), main 14494. A disk image larger than 31 MB overran the pre-committed guest region. Measured by reek, kept in full below because the measurement is what made it findable in one sitting.
guest_mem is MEM_RESERVE with only the first 32 MB committed up
front (codex-vm.c:8997), and load_kernel's raw path memcpys the WHOLE
file to guest_mem + LOAD_ADDR (0x100000) bounded against
guest_mem_size -- 3 GB -- rather than against what is committed. A 32 MB
image therefore writes 1 MB..33 MB and walks off the end of the commit into
reserved address space. guest_commit_range exists for exactly this and its
own comment describes the symptom ("a direct memcpy out of the same region
does crash, and takes the VM process with it"); load_kernel was not among
its callers. It now commits first.
The threshold is where the mechanism says it is, which is how it was
pinned rather than argued. LOAD_ADDR + size of exactly 32 MB
(32,505,856-byte image) boots; one 64 KB step past it faults. Both crashing
sizes boot after the fix and the 30 MB control is unchanged.
Two corrections to the notes below, both worth keeping. The absence of
both GPT lines was not a clue about the directory walk -- the fault is
upstream of the walk entirely, in a copy that runs before it, so neither
line could have printed and buffering was never involved. And the
unbounded memcpy at the extracted-PE copy is real but is not this
crash: it is bounded against the image and not against guest RAM, so it
would need a BOOTX64.EFI larger than guest RAM, while the crash needs
only a 32 MB disk. It is bounded and committed now as well.
The original measurement, unchanged:
CLOSED. Re-verified 2026-08-10 (reek) against the fix above: a 32 MB
image (33,554,432 bytes, 65536 sectors) built from the same source now boots
and runs to completion under -uefi -disk, with no host fault. The entry
below is the original measurement, kept because it is what made the cause
findable in one sitting. It said the cause was NOT pinned, and that was
true when written; it is pinned now and the account is above.
The -timeout 60 below is a transcript of what was typed, not a recipe:
that flag does not exist and was silently ignored, exactly as the section
above records. Do not copy this line. To bound a run, use
-screenshot with -screenshot-delay, which exits after the capture.
tools/codex-vm.exe -kernel X.img -uefi -disk X.img -headless -timeout 60
IDE: X.img (33554432 bytes, 65536 sectors)
UEFI mode / ACPI / SMBIOS lines, then:
HOST CRASH: codex-vm faulted (code=0xC0000005) at 0x7FF62F685F9E writing memory 0x227397B0000
The control is the same payload and the same source in a 32768-sector
image, built by the same build/build-img.ps1 invocation with only
-TotalSectors changed: it boots, prints
GPT: extracted BOOTX64.EFI (2591232 bytes, FAT16) from partition at LBA 2048,
and compiles. So the variable is image size, not the payload.
Two facts to start from rather than re-measure. The failing run prints
neither the GPT: extracted line nor the GPT: ESP ... has an unusable BPB
line that is its else-branch, which either puts the fault inside the
directory walk or means those fprintf(stderr, ...) calls are lost to
buffering while the SEH handler's own writes survive -- establish which
before reasoning from the absence, because they imply different faults.
And the fault is a WRITE, while every access in the walk itself is a read.
Not on the A5 flight path: a5flight2.img and the 2.7 MB source arm are
both 32768 sectors. It bites whoever first needs an image above 16 MB,
which the drive installer will.
One thing this fix does NOT change, recorded so nobody reads it as
covered. A GPT disk image under -uefi is copied into guest RAM twice:
once whole by the raw path above, then again as the extracted PE. The first
copy is discarded and now commits the whole image, so a multi-gigabyte ESP
will commit multiple gigabytes of host RAM before booting. That is waste,
not a crash, and skipping the raw copy for a GPT image is a separate change
with its own fallback question.
Run codex-vm directly with -debug (and optionally -map <file>.map)
to get an interactive debug shell on breakpoints and single-steps.
tools/codex-vm.exe -kernel build-output/bare-metal/Codex.cdx ^
-input input.tmp -output out.tmp -mem 3072 -headless ^
-debug -map build/output/reg.map
When a breakpoint fires (-Break patch or runtime b command), the
debugger prints registers with symbol resolution and waits for commands:
--- Break at 0x12f1de <desugar-def+0> ---
RIP=0x12f1de <desugar-def+0>
RSP=00000000007ffa08 RBP=00000000007ffa30 ...
R13=00000000006ff325
dbg> _
Commands:
| Command | Description |
|---|---|
s / step |
Single-step one instruction (sets TF) |
c / continue |
Resume until next breakpoint |
r / regs |
Dump all registers with symbol lookup |
m <addr> [len] |
Hex+ASCII memory dump (default 64 bytes) |
x <addr> |
Read one qword at address |
bt / backtrace |
Walk RBP chain (resolved frames) |
stack |
Dump 16 stack slots with symbol lookup |
b <fn|0xaddr> [if reg=val] |
Set breakpoint. The if condition does not discriminate -- see below; use -hbreak |
w <addr> [size] |
Set memory watchpoint |
sym <name> |
Look up symbol address |
q / quit |
Exit VM |
The shell's if reg=val condition does NOT discriminate. Use -hbreak
instead. The form parses and arms, and it breaks whatever the register
holds -- proven with a condition that can never match, which broke anyway.
There are two reasons and the second is fatal to the whole approach. The
check in dbg_command_loop runs only for vec == 1, while an INT3
breakpoint arrives as vec == 3, so it is never consulted. And even
wired to vector 3 it could not work: vector 3 belongs to the guest.
Its handler runs before the host sees the trap, so the condition would be
evaluated against the handler's registers rather than the callee's --
measured, rdi reads 0x33 every time.
An unconditional breakpoint is also terminal here: the guest's
handler dumps !EXC and halts, so a run reports exactly one break
however many times the address is executed. Counting breaks does not
count calls.
-hbreak exists because of all three. It arms a debug register instead,
so the #DB is intercepted by the host and never reaches the guest IDT,
the condition sees the real register file, and an unmatched hit resumes.
Workflow: tracing a field corruption
- Break at the function that builds the record (
-Break "desugar-def") regsto see the input parameter (rdi = Def pointer)x <rdi+0>to read the Def's first field (ann in CCE order)stepthrough the function, watching how field values flow- After the record constructor,
x <r10_before+16>to verify the declared-type field was stored correctly - Or use
w <addr>to set a memory watchpoint on the field slot
Build codex-vm with debugger:
tools/build-vm.ps1 # standard build includes debuggerThe debugger compiles into the same binary -- -debug activates it.
Without -debug, behavior is unchanged (breakpoints print and continue
as before).
Emit phase markers during compilation. When a crash occurs, the
last DBG: line in the log identifies which phase was active.
build/compile.ps1 -Src foo.codex -Out foo.cdx -Log foo.log `
-DebugModeLog output:
DBG:frontend src=1204413
PH:LEX
PH:PARSE
PH:DESUGAR
PH:SCOPE
PH:CHECK
PH:LOWER
PH:FRONTEND
DBG:emit defs=412
SIZE:2176384
A PH: line means that phase COMPLETED, so a crash names the phase
after the last line printed. This is the only thing that identifies the
phase when a starved deck corrupts the heap: the crash site is a shared
runtime helper (__linked_list_to_list, copy-sx-text) that half the
frontend calls, and four -Decks values that all die in DESUGAR name
three different helpers between them. Reach for this before theorising
about which phase ran out of room.
-
Read the crash report. The harness prints resolved function names. Start with the RIP function and the heuristic stack trace.
-
Read the code.
Resolve-Rip(vm-config.ps1) gives you the function. Read that function in the compiler source. Form a theory. -
Set a breakpoint. Use
-Break "suspect-function"to confirm the function is reached and inspect register state at entry. -
Use debug mode.
-DebugModeshows phase progression. If the crash is in emit, the lastDBG:emitline narrows the window. -
Run the poison build. If you suspect uninitialized memory, build a poison seed (see Poison-Alloc section below) and run the tests.
CR2=0xCDCDCDCDCDCDCDCD= uninitialized field.
__out_of_memory is reached from cmp rsp, r10; jb __out_of_memory in every
non-leaf prologue, so it fires on a stack/heap COLLISION, not on a heap
allocation failure. codex/test/exc-stack-heap is the deliberate collision and
its whole expected output is the one line. The message names the wrong resource
and has repeatedly been read as heap exhaustion.
The handler opens by saving both sides of the comparison and then prints neither:
mov rbx, rsp ; RBX = the faulting RSP
mov r12, r10 ; R12 = the deck pointer at the trip
It ends cli; hlt; jmp -6, so it spins with both intact and any register dump
taken after the message reads them:
# under OVMF
pwsh build/boot/test-ovmf.ps1 -Img <img> -Out <png> -Seconds 20 -MonCmds 'info registers'
# then, to see the recursion cycle, dump the stack at RBX and resolve the
# return addresses against the .map that compile.ps1 wrote for THAT binary
pwsh build/boot/test-ovmf.ps1 -Img <img> -Out <png> -Seconds 20 -MonCmds 'xp /32gx <RBX>'Subtract the heap base the loader installed (cdx-to-pe.ps1 writes it to
deck-pos-addr, 0x7030) and the split says which side ran away. Measured on
seed/Codex.img 2026-07-30: stack 489.1 MB, heap 22.9 MB, meeting with a
32-byte gap inside a 512 MB region. A repeating frame carrying the SAME pointer
every time is mutual recursion; the map resolved it to
dev-console-poll / dev-console-serial-poll / dev-console-idle.
Do not read deck-pos-addr or heap-hwm-addr post-mortem to decide this.
Both are the wrong instrument on a UEFI boot and both answer plausibly:
heap-hwm-addr(0x7038) CANNOT MOVE there. Its only updater isemit-update-heap-hwm, whose one call site is insideemit-startafteropeningreturns, and the UEFI stub callsopeningdirectly so__startnever runs. It holds whatever the loader wrote, so "hwm equals the deck position" is a tautology on that path rather than evidence.deck-pos-addr(0x7030) is stale, because the live deck pointer is R10 and the cell is not written on every allocation. It read as the base while R10 was 22.9 MB above it.
codex-vm -kernel build\boot\diag.img -disk build\boot\diag.img looks like the
obvious way to try a diagnostic image by hand. It is not: the ladder BANKS, and
banking means the payload writes DIAG.TXT onto the ESP of whatever medium it
was given. Point -disk at the depot file and the depot file is what it edits.
Measured 2026-08-20 (blu): one probe boot moved diag.img from A6A00B79 to
0D69C8FC, and the next diag-arm.ps1 run duly reported the mutated image as
its subject.
Boot a COPY. That is what diag-arm.ps1 does for every arm it runs
(New-Copy / New-Variant), and the reason is this and not tidiness. The
repair when it happens is build/boot/build-diag.ps1, which is deterministic:
the same sources give the same hash back, so a rebuild returns the artifact
rather than merely a new one.
Two ways this bites beyond the wasted rebuild. A CL that carries the mutated
image ships an artifact with somebody's test run baked into it, and the hash in
the rehearsal record will not be the hash of the thing anyone else builds. And
build/check-shipping-images.ps1 will NOT catch it, because it refuses a
DIAG.CFG and this writes a DIAG.TXT.
| port | emitter | who reads it | |
|---|---|---|---|
| COM1 | 0x3F8 data, 0x3FD LSR | emit-serial-wait-and-send |
codex-vm -output, so test-run.ps1, so every .expected |
| COM2 | 0x2F8 data, 0x2FD LSR | emit-control-wait-and-send |
the compile protocol only. NOT captured to -output |
Both are in codex/compiler/Emit/X86_64IO.codex. A diagnostic written to COM2
is emitted and discarded, so the code under it looks like code that never
runs. __out_of_memory printed there for its whole life (fixed main 11837) and
a correct, firing stack guard read as "the guard never fires" for as long as it
did. Check which emitter a message goes through before concluding the path is
dead. emit-control-wait-and-send also carries a 10000-spin bail-out that the
serial one does not; the serial wait blocks until THR-empty.
Copy this (emit-cpu-exception-dump, X86_64Boot.codex); do not invent one:
emit-serial-drain-delay
cli
halt-pos = code-len
hlt
jmp (halt-pos - (code-len + 5))
hlt with IF=1 is woken by the timer and the loop spins forever. codex-vm
exits on Guest halted with IF=0, that never comes, and the harness kills the
run at its wall budget -- so a diagnostic that halted correctly is reported as a
hang, which is a different bug with a different owner. Any halt you add to a
guest path needs the cli.
| Flag | Purpose |
|---|---|
-Break "name" |
INT3 at function entry; use with codex-vm -debug for interactive shell |
-DebugMode |
Phase markers (DBG:frontend, DBG:emit) |
-Poison |
0xCD fill in __alloc (catches uninitialized fields) |
-Repl |
REPL loop (for batch compilation) |
-Decks <N> |
Scale every phase deck floor to N% of the BuildSettings defaults (100 = defaults). Sends decks=N on the mode line. |
Symptom: you edit a file, compile, and the output binary is byte-identical to the one before your edit. It reads exactly like "my change had no effect", which is the most expensive possible false reading.
Cause: compile.ps1 resolves cites with -Repo '.', and .NET file APIs
resolve . against [Environment]::CurrentDirectory -- which PowerShell's
Set-Location does not update. So in any workspace other than the shell's
default, a relative -Src and every cited quire silently resolve to the
other workspace. An absolute -Src fixes the source and leaves the quires
still pointing at the wrong tree, which is worse: the compile succeeds, most
of the program is yours, and one cited chapter is somebody else's copy.
Fix: set both, every time:
Set-Location D:\Projects\Cobblestone-<agent>-main
[Environment]::CurrentDirectory = 'D:\Projects\Cobblestone-<agent>-main'
build/compile.ps1 -Src codex\test\apps\foo.codex -Out out.cdx -Log out.log -Kernel seed\Codex.cdxDetect: hash the output. Two compiles of genuinely different source that produce the same SHA-256 have not read your file. This is the cheap check and it is the only one that fires before you start debugging the wrong program.
Symptom: you dump a program's IR with -IrUni because -IrCce is not
readable text, read the shape off it, and reason about what the plug does with
it. The reasoning is sound and the conclusion is wrong.
Cause: the two front doors run different pipelines.
emit-ir-cce (codex/compiler/opening.codex:1707) calls lift-lambdas at
:1717 before pruning; emit-ir-uni (:1683) does not lift at all. That
second call landed at main 19558 (2026-08-25) to fix a wasm defect. So a
program with a lambda in it has TWO different IRs depending on which flag you
pass, and the plug only ever sees the lifted one: where -IrUni shows
(lambda (params (param "x" ...)) ...), the wire carries
(apply (name "__lam_0") (name <captured>)), a partial application.
Cost so far: COMPILER-29's cause was recorded wrong twice, both times off
an -IrUni dump, and the second write-up named an innocent IR reader.
Detect: if a claim is about what a PLUG receives, do not settle it from
-IrUni. Force a marker constant into the emitted code from inside the arm you
believe runs, rebuild the plug, and read it off the target -- and put the same
marker in the arm you believe does NOT run, as the control. That pair is what
finally answered COMPILER-29; each round is a plug rebuild plus one filtered
test-cross-batch.ps1 run, about ninety seconds.
Symptom: pwsh .\build.ps1 inside codex\plugs\<p>\ exits 5 with
FAIL: compile errors, and prints a log full of CDX3005 shadowing
warnings about is-letter, is-digit and is-whitespace. There are no
errors anywhere in it. The same bundle compiles clean by hand.
Cause: the message is wrong twice over. Something in the compile
resolves a path against the CURRENT WORKING DIRECTORY, so from anywhere but
the repo root the compile exits non-zero and nothing is compiled at all.
Re-measured 2026-08-20 from C:\Users\Damian: it is now the CITE that fails,
error 3010: Unresolvable cite: Foreword chapter 'ListUtils' ... (expected .\codex\foreword\core\ListUtils.codex), not the kernel -- the builder pins
-Kernel to <repo>/seed/Codex.cdx now. Same fix, different message, so do
not use the old text to decide whether this is your failure.
Build-PlugCdx (codex\plugs\common\plug-build-lib.ps1:148)
swallows that output with | Out-Null, sees a non-zero exit, reports it as
"compile errors", and then dumps the FIRST TEN LINES OF THE PREVIOUS RUN'S
build.log. Those warnings are stale and belong to a compile that succeeded.
Fix: run it from the repo root.
cd D:\Projects\Cobblestone-<agent>
pwsh codex\plugs\csharp\build.ps1 # OK: ...\csharp-plug.cdx (371061 bytes)Detect: the success line is [<plug>-plug] OK: <path> (<n> bytes). If you
do not see it, nothing was written, whatever the log appears to say. Check the
log's timestamp against the run before believing a word of it.
Phase deck floors live in BuildSettings.codex and are compiled into
the compiler. build/check-constants.ps1 hashes them and warns when the
source and the seed disagree -- but it is only a warning, and build.ps1
refreshes the hash at the end of every build.
That creates a deadlock. If a source grows past the floor, the compile
fails with CDX9002 -- and raising the constant does not help, because
the seed doing that compile enforces its own baked-in floor. You
would need the new seed in order to build the new seed.
-Decks is the way out:
# The source outgrew the floors the current seed was built with.
build/compile.ps1 -Src big.codex -Out big.cdx -Log big.log -Decks 200Turn the floors up for the one compile that needs it, build the seed that carries the higher default, then turn the knob back off. Without it, the only escape is a two-stage bootstrap through an intermediate seed.
-EscapeCheck on the selfhost needs -Decks 200 or it dies in a silent
#GP in copy-sx-pos. The check allocates inside the walk, so it outgrows
the stock floors on a unit the size of the compiler. Not new, and it
reproduces on old seeds, so a fresh #GP there is not a regression you
introduced.
Turning the knob DOWN is sharp. An under-reserved floor does not
raise CDX9002 -- the parse keep-deck copy writes past the floor into
the scratch it is still reading and the compile dies in a #GP
(!EXC=0d) with no diagnostic, because the post-copy overflow check
never runs. That is a pre-existing property of under-reserved floors,
documented in BuildSettings.codex; the knob simply makes it reachable
on purpose. decks=5 on the compiler's own source demonstrates it.
The survey-multiplier system (and its -Survey override) was deleted
2026-07-07: phase decks are fixed generous floors and the heap range
[6 MB, 2 GB) is demand-paged -- physical memory commits on first touch.
compile.ps1 still accepts -Survey and ignores it for one transition
cycle.
Hardened 2026-07-06 (val CLs 7207-7211): only
not-present faults grow the heap (protection faults dump), demand
mappings carry NX, the demand-range top derives from actual RAM (any
-mem from ~128 MB boots; the top 64 MB stays present for the stack),
a TSS/IST1 emergency stack turns stack-overflow triple-faults into
!EXC dumps, and the touched-page count lives at cell 30688.
Measured 2026-08-03. codex-vm caps the RAM size it reports to the guest at
GPU_CMD_ADDR (0xBE000000, 3040 MB) so the boot stack, which starts at the
reported size and grows down, cannot land in the GPU command buffer, the
depth buffer or the GOP framebuffer. Those three sit at fixed GPAs. The cap
binds unconditionally, so -mem 8192 reports 3040 MB:
RAM cap: guest_mem_size=0x200000000 effective=0xbe000000 gop=0
The consequence is that compile.ps1's crash retry cannot recover
anything. On an !EXC it prints crash with 3072MB, retrying with 8192MB
and runs a byte-identical machine. A whole-compiler -IrCce emit crashes
with the same RIP, the same interrupted RSP and the same heap frontier at
both sizes, four runs out of four. Do not read that retry as having ruled
memory out.
The failure looks like this, and the shape is worth knowing because the crash site is innocent:
CRASH in __str_concat+0xF9 (general protection)
callR 0xBDFFFA48 interrupted RSP
R10 0xBE00367E heap frontier, ABOVE the reported RAM top
RBX 0x111C49161510180D CCE text, restored from a clobbered stack slot
The heap frontier has passed the stack. Registers holding text rather than
pointers are the fingerprint: saved registers reloaded from stack slots the
heap has overwritten. Any !EXC=0d whose R10 is near 0xBE000000 is this and
not a codegen defect.
-mem-nocap reports the real size instead. It is opt-in, and it must stay
opt-in: the guest can turn GOP on after boot (UEFI GOP, VBE) by which time
the RAM size is long written, and an uncapped heap that grows past
0xBE000000 then overwrites the framebuffer it is about to scan out. Pass it
only for a run that draws nothing. compile.ps1 -MemNoCap forwards it, and
-TimeoutSec raises the 600 s VM kill for the one job that outruns it.
Above ~3 GB nothing helps, because the address space is baked into the
binary. bare-metal-ram-size in codex/compiler/Emit/X86_64State.codex
is the constant 3221225472, and the page tables the seed emits map exactly
that much RAM plus one device gigabyte above it. Raising the guest ceiling
past 3 GB is a change to that constant, which is seed-affecting.
codex-vm identity-maps one 2 MB-page PD per GB of guest RAM, minimum 4 GB and
maximum 64 GB, and CR3 points at it before the guest builds its own tables.
It tracks -mem because GetMemoryMap advertises conventional RAM at and
above 0x100000000 whenever -mem exceeds 4 GB, and AllocateAnyPages
allocates top-down from guest_mem_size - 1 MB: a guest that believes
either one touches memory the map has to cover.
A UEFI fault of that shape is the bed, not the payload. Earlier builds
pinned this at a fixed 4 GB, and before that 2 GB, so the emulator advertised
memory it did not map: at -mem 8192 it returned 0x1f7f00000 from its own
allocator and then triple-faulted the guest on that address. An older account
calling a UEFI run above -mem 4096 meaningless is describing that, and it
does not hold now.
Past 64 GB the map stops growing and says so on stderr, naming the address above which memory is advertised and not mapped. It does not fault silently.
Any memory claim about a flight artifact measured in codex-vm at the
default -mem is measured against an arena roughly 24x too large. The
two paths do not share a memory model:
| Path | Arena |
|---|---|
build/boot/build-option-a.ps1 (the boot image, what flies) |
ONE region of HeapPages * 4096 bytes, heap at its base (r10) and stack at its top (RSP), every prologue checking cmp rsp, r10. -AllocPages 32768 is the shipping value: 128 MB for heap and stack together |
build/desk.ps1, any codex-vm -kernel *.cdx |
the bare-metal model above: heap 6 MB up to -mem, default 3072 |
-AllocPages is cdx-to-pe.ps1's -HeapPages under another name; the
name is kept because probe commands in docs/Hardware/HardwareSitting.md pass it.
A payload that hosts a VMX guest needs a bigger one, and 131072 (512 MB) is
measured. vm-prepare-guest allocates the guest's region out of this same
arena, so 32768 cannot carry a 256 MB guest and vm-guest-sizing refuses every
time. Measured 2026-08-15 under OVMF: an Option A image built
-AllocPages 131072 -Ebs -Uefi boots on real edk2 firmware in a 2048 MB
machine and paints the desk. The control is the same image in a 256 MB
machine, which never reaches the paint at all -- one flat colour against five
-- so the bed can say no and the green means something. What this does NOT
establish is the ASUS: OVMF choosing to satisfy a 512 MB AllocateMaxAddress
below the aperture is exactly the class of spec freedom L-FREEDOM is about, and
AMI is free to answer differently. The stub raises H on a failed heap
allocation, so a board that refuses says so rather than wandering.
-AllocPages above 524287 crashes cdx-to-pe.ps1 rather than building.
$ebsBackoff = [Math]::Max(0, 0x10000000 - ($HeapPages * 4096)) + 0x100000
picks the Int32 overload from the literal 0, and past a 2 GB arena the
subtraction no longer fits: Cannot convert argument "val2", with value: "-2952790016". It fails loudly at build time and ships nothing wrong, and
$HeapCeiling is 3 GB anyway, so the reachable band is 512 MB to 2 GB. 0L
would lift it if anyone ever needs more.
This is what hid the A6 F12 regression for a week. Six bed arms across two agents could not express a 4.6 MB per-visit leak, and the post-mortem concluded the remaining space was real hardware behavior the emulator does not model. It was the heap.
You cannot shrink the bed to match with -mem. codex-vm's GOP
framebuffer sits at GPA 0xBF000000 (3056 MB), so a small -mem leaves it
outside RAM and the guest dies in gop-rect-rows (measured at 256 MB).
OVMF or metal are the only routes to the real arena.
The interactive debugger (-debug) covers breakpoints, single-step,
register/memory inspection, conditional breaks, and backtraces. GDB
under WSL with QEMU TCG is a last resort for hardware watchpoints
(DR0-DR3) on specific memory addresses. Rule 6 permits Unix tools
for this purpose only. See build/gdb-watchpoint.ps1.
On real hardware there is no serial port. Consumer boards of this era
expose no UART header, and the no-borrowed-substrate doctrine means there
is no dmesg, no lsusb, and no kernel log -- nothing between the
framebuffer and silence. Worse, the obvious workaround (write the findings
to a file on the boot stick) routes the telemetry through USB mass
storage, which on the machines this matters for is itself one of the
broken subsystems. That mistake cost a week; the accident report is
docs/PM/Active/Stories/TheSilentKeyboard.md, and its first recommendation is
that no hardware campaign may launch without an output channel that does
not depend on the subsystem under test.
Since 2026-08-05 the standing probe telemetry channel is GopShot's F12
screenshot-to-stick: probe-halt was replaced by shot-wait, and the frame
lands on the ESP as SHhhmmss.BMP behind the medium-select CODEX.CDX
lock. QR plus a photograph remains the honest fallback when the disk
stack is itself the subsystem under test, or below rung 6, where the
shot cannot land through a broken disk path. This section is that
fallback.
The guest paints QR codes. apps/works/GopQr.codex is a QR encoder in
Codex -- version 5, error level L, mask 0, 106 bytes per code, chunked
i/n; across as many codes as the report needs. It draws them straight to
the GOP framebuffer. It needs no disk, no serial, no network, and no
working USB. If the machine can paint pixels, it can report.
The human photographs the screen. A phone, hand-held, is fine.
The host reads the photograph.
pwsh tools/qr-read.ps1 -Path D:\20260713_131201.jpg
pwsh tools/qr-read.ps1 -Path shot.jpg -Save findings.txt # keep the bytes
pwsh tools/qr-read.ps1 -Path shot.jpg -ShowDebug # when it will not readIt prints the exact bytes the machine emitted, reassembling the i/n;
chunks in order and warning loudly if a chunk is missing rather than
quietly handing back a truncated report. Reed-Solomon correction is real:
a photograph of a lit screen loses modules to glare and blur, and up to 13
corrupt codewords per code are recovered.
qr-read.ps1 is the decoder half of GopQr and is written as its exact
inverse -- same reserved-module map, same zigzag, same mask, same
generator (roots from alpha^0). Only the image decode is borrowed, from
Windows itself; the QR decode is ours. Do not reach for a Python library
or a QR app on your phone: the phone app will silently give you a
rendering of the bytes, and this pipeline is the one that gives you the
bytes.
Reading the output. -ShowDebug prints the timing patterns of every
symbol it sampled. Timing must read 101010... and report 21/21. That
number is the honest test of whether the geometry was found:
21/21and no decode -- the grid is right and the payload is damaged. Re-shoot with less glare.- anything less -- the grid is wrong, and no amount of error correction will save it. Re-shoot straighter and fill the frame with the codes.
When you add a probe that tests the disk stack or runs below rung 6, render its findings as QR; everywhere else, let it screenshot itself via F12. A number that only a human can read off a monitor is a number that arrives wrong, arrives late, or -- as happened -- dies with the framebuffer at power-off, having cost a person a walk across the building and a flash cycle to obtain.
__alloc zeroes every block it returns (calloc semantics). A poison
build replaces that zero fill with 0xCD, so any field read before it
is written dereferences 0xCDCDCDCDCDCDCDCD -- a non-canonical address,
and therefore an immediate page fault instead of a plausible zero.
This is a release gate, not a routine one. Run it before publishing the seed to the public mirrors. If the battery passes against a poison seed, the compiler has no uninitialized-field dependencies -- the zero fill is a safety net and not a patch holding something together.
Why it matters: the REPL loop resets R10, deck-pos and heap-hwm between compilations but does not zero the freed memory. Without the calloc, the second compile in a batch allocates records on top of the first compile's live pointers and type tags. The first boot looks clean because hardware zeroed the heap; the second one does not.
# 0. Save the kernel you are about to overwrite. Step 3 replaces it.
Copy-Item -Force build-output/bare-metal/Codex.cdx build-output/kernel-backup.cdx
# 1. Concat compiler source
build/concat-codex-self.ps1 -CodexDir codex/compiler -OutFile build/output/Codex.codex
# 2. Compile a poison seed (0xCD fill instead of zero)
build/compile.ps1 -Src build/output/Codex.codex `
-Out build/output/poison-seed.cdx `
-Log build/output/poison-build.log -Repl -Poison
# 3. Run the battery against the poison seed. Name the tiers you want:
# a bare invocation is the `lang` tier only, which is NOT the full battery.
build/test.ps1 -CodexCdx build/output/poison-seed.cdx -Tier all -Jobs 8 -ApprovedBy damian
# 4. PUT YOUR KERNEL BACK. Step 3 does not.
Copy-Item -Force build-output/kernel-backup.cdx build-output/bare-metal/Codex.cdx
# Zero failures. Any failure means an uninitialized field was read during
# compilation. The crash CR2 will be 0xCDCDCDCDCDCDCDCD -- look up RIP in
# the symbol map to find the function that dereferenced the bad pointer.Steps 0 and 4 are not tidiness. -CodexCdx does not point the battery at
another compiler, it copies that compiler over
build-output/bare-metal/Codex.cdx (test.ps1:170-173) and never restores
it. So the poison seed stays installed as the kernel every later
compile.ps1 boots by default, and the next thing you compile is compiled by
the poison compiler while you believe otherwise. Measured 2026-07-28: after a
poison battery the kernel was still 3AF5763C, the poison seed, and nothing
said so. compile.ps1 prints the kernel and its digest on every run for
exactly this class of mistake -- read that line.
-Jobs 8, and a release run is not an exception. A -Jobs 4 in any
older recipe is the dead XMP workaround described under "The parallelism
default" in ExaminersAssay.md: the box's DDR5 was running a profile it was
not stable at, and that was fixed 2026-07-22. Do not copy the lower number
forward. Measured
2026-08-02 on the poison run that carried it: 977 s in the compile phase at 4
slots on a 12-core box. Damian's ruling is that 8 is the standard everywhere,
including release proofs. The contention classes that motivated the caution are
crash-shaped (FAIL_RUNTIME with no uart, exit 4 with zero diagnostics) and
both retry paths already catch them -- Invoke-StandaloneRetry here, and the
sweep re-runs its own no-diagnostic units alone.
A bare build/test.ps1 is the lang tier, not the full battery. The
command above says -Tier all on purpose. A poison run over lang proves
lang, and the apps, forewords and lib tiers allocate shapes it never
reaches.
Do not expect a specific pass count here. The lang tier
alone measured 674 total / 649 pass / 25 skip on 2026-07-28. The number to
check is fail=0, and the rollup's own run-over-run delta (newly red) is
a better instrument than any count written down in a document.
-Poison fills every __alloc block with 0xCD, so a field read before it
is written faults instead of reading a plausible zero.
-PoisonCompact is the other axis: it poisons each phase's scratch after
that phase compacts, with a distinct byte per phase -- 161 lex, 162 parse,
163 desugar, 164 scope, 165 check, 166 lower, 167 resolve, 168 lift, 169
frontend (codex/compiler/opening.codex). A use-after-compact therefore
faults with a byte that names the phase whose memory was reused, which
-Poison cannot tell you. Run both; a clean -Poison says nothing about
phase discipline.
These steps run ONLY when publishing the seed to the public mirrors (GitHub, GitLab) -- never on routine seed rebuilds or copy-to-main. The day-to-day gates (text + CDX fixed point, sample battery) already prove correctness; the items here are public-facing polish and are not usually needed internally.
-
Poison build passes (above) -- the seed has no uninitialized-field dependencies.
-
Refresh
seed/Codex.map, and take it from the build. The gate already produces the right one.compile.ps1appendsmapto every CDX mode,-Replincluded (compile.ps1:139-141), andInvoke-BuildCdxcarries the sidecar with the binary it describes, so a gate run leavesbuild/output/Sut.mapbesidebuild/output/Sut.cdx. Install that:Copy-Item -Force build/output/Sut.map seed/Codex.map
It is the shipping compiler's own map rather than a re-derivation. Measured 2026-08-15 against the seed's embedded MAP1: 5,192 of 5,192 names, zero address differences, zero size differences.
Nothing outside this step refreshes it, and the drift is total rather than partial. The same measurement against the
seed/Codex.mapin the tree that day, minted at the previous release against seed8D405FDF: every name still resolved and 5,140 of 5,187 rows sat at the wrong address. A stale map does not look stale. It looks like a map.So validate rather than assume, and validate the MAP and not the binary: read the seed's embedded MAP1 with
Get-Map1Symbols(build/vm-config.ps1, whoseAddralready carries the0x100000load base) and require every text-map row to match by name, address and size. That check fails loudly on the case that actually hurts, a map minted from compiler source that has moved ahead of the seed, which is confident nonsense that nothing else announces.The embedded MAP1 in each CDX is authoritative for crash reports regardless -- the text map only feeds
-BreakandResolve-Rip.