Skip to content

experiment: add final overnight stability synthesis - #11

Merged
waple0820 merged 1 commit into
mainfrom
experiment/gpt55-overnight-stage4-20260714
Jul 13, 2026
Merged

experiment: add final overnight stability synthesis#11
waple0820 merged 1 commit into
mainfrom
experiment/gpt55-overnight-stage4-20260714

Conversation

@waple0820

Copy link
Copy Markdown
Collaborator

Summary

  • add the Stage 4 answer-first Helm HDOC synthesis for the GPT-5.5 LexBench overnight campaign
  • preserve Stage 1 population, Stage 2 selected mechanism holdout, and Stage 3 capacity evidence as separate layers instead of pooling unlike designs
  • document c32 as the current complete formal boundary and gate c64 on two successful task-shaped 28 EN + 36 ZH admissions or provider allocation evidence
  • add a campaign index and link the final synthesis from repository indexes

Evidence

  • 12 formal arms; 576 planned, completed, and valid judged trajectories
  • Stage 1: 82/128 vs 80/128, cluster CI [-9.375, +12.5]pp; no general winner
  • Stage 2: 16/18 selected holdout directions retained; bidirectional site-specific effects
  • Stage 3: balanced 64 observed, task-shaped 64 failed 0/2, c32 formal arms complete
  • Helm document: gpt55-lexbench-stage4-synthesis-2026-07-14
  • report SHA-256: 0562f2a90379ccb1c24dd1c01cb01d5eb5fb2ba554a88a94c3fc12c165ae01ac

Self-review

  • checked every synthesis field against the stage machine-readable evidence
  • independently summed all 12 arm summaries to reproduce the 576-record ledger and token/cost totals
  • verified the report does not claim a general accuracy winner, causal IP/proxy mechanism, arbitrary-profile c64 reliability, or end-to-end cloud savings
  • inspected desktop and mobile layouts; all five tables are contained in horizontal scroll wrappers with no page overflow
  • scanned changed artifacts for credentials, host details, and placeholders; no matches

Validation

  • uv run pytest (62 passed)
  • uv run ruff check .
  • bash -n scripts/run_gpt55_lexbench.sh
  • git diff --cached --check
  • Helm HDOC/1.0 validation (0 warnings)
  • final report hash rechecked after Helm submission

@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown

Claude finished @waple0820's task in 2m 2s —— View job


PR Review

  • Gather context and understand the request
  • Review changed files for bugs and security issues
  • Post findings

No bugs or security issues found.

Checks performed:

  • report.html: The only <script> tag uses type="application/json" (non-executable Helm HDOC metadata). No event handlers, no external resource loads, no dynamic content. Static HTML is safe.
  • SHA-256 integrity: report.html hash verified on disk — matches manifest.json report_sha256 field (0562f2a9…) exactly.
  • Cross-file hash consistency: Stage 1/2/3 report_sha256 values are identical between manifest.json and synthesis.json.
  • Arithmetic consistency: Key deltas check out — Stage 1 lexmount_minus_local = (82−80)/128 = 0.015625 ✓; c32 (40−41)/64 = −0.015625 ✓; Stage 2 rate 16/18 ≈ 0.888889 ✓.
  • source_commit in manifest.json (6dcd388…) correctly points to the Stage 3 parent commit.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Claude review completed. Review run: https://github.com/lexmount/LexBrowserEnv/actions/runs/29286306969. Please check Claude's inline feedback and summary comment for details.

@waple0820
waple0820 merged commit fc63d7c into main Jul 13, 2026
1 check passed
@waple0820
waple0820 deleted the experiment/gpt55-overnight-stage4-20260714 branch July 13, 2026 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant