Publish the learned-heuristic results: a Research view in the workbench (#161)

jupyddl.learn shipped in 2.3.0 with its results living only in .docs/ and a
video, so the published site said nothing about the largest addition in the
release. The workbench now carries a Research view with the measured run:
imitation against hff and goalcount, the per-instance spread behind the
mean, the logistics loss and the exact reason for it, and the three claims
that turned out to be wrong.

Its numbers come from promo/rl-data.json, the cache the RL video renders
from, so the page and the video quote one measured run and cannot drift
apart. Tests pin that, and pin capabilities.json to the live registries.

Rendering the view exposed a defect it did not cause: <main> was hidden
until the worker reported ready, so every page of prose, the requirement
matrix and the new view sat behind a spinner waiting for a 10 MB runtime
none of them use. The shell now renders immediately from the committed
bundle and only the run controls are gated on state.ready.

Also carried: the GitHub Release job no longer requires the PyPI upload to
have succeeded, and renovate no longer proposes bumping the workflows'
Python pin.

Author: guilyx

.github/workflows/release.yml

@@ -178,9 +178,20 @@ jobs:
     # Either a tag was pushed, or someone asked for a real release from the
     # Actions tab. A TestPyPI dry run deliberately creates neither tag nor
     # release -- it exists to rehearse, not to leave traces.
+    #
+    # Deliberately NOT gated on the publish succeeding. A GitHub Release
+    # records that a version was cut from a verified commit; PyPI is a
+    # downstream channel that can fail for reasons the code has nothing to do
+    # with -- auth not configured, an outage, a rate limit. Losing the tag and
+    # the release because of that would mean no record of the version and a
+    # full re-run to get one. `build` succeeding is the gate that matters:
+    # that is where every check lives. A failed publish stays red and visible,
+    # and re-running just that job finishes the job.
     if: >-
-      startsWith(github.ref, 'refs/tags/')
-      || github.event.inputs.target == 'pypi'
+      always()
+      && needs.build.result == 'success'
+      && (startsWith(github.ref, 'refs/tags/')
+          || github.event.inputs.target == 'pypi')
     runs-on: ubuntu-latest
     permissions:
       contents: write  # create the tag, the release, and attach the artifacts

AGENTS.md

@@ -75,7 +75,13 @@ native build step, and the core has zero runtime dependencies.
 - `jupyddl/viz/` — everything that imports matplotlib. Nothing in the core may
   import this package.
 - `web/` — the Pyodide playground; `tools/build_web.py` bundles the package
-  sources and demos into `web/dist` (committed).
+  sources and demos into `web/dist` (committed). It also writes
+  `capabilities.json` (the registries) and `research.json` (distilled from
+  `promo/rl-data.json`, so the page and the RL video quote one measured run).
+  Those two are rendered **before** Pyodide loads — the app shell is never
+  hidden, and only the run controls are gated on `state.ready` — so a stale
+  bundle briefly states something untrue rather than merely lagging.
+  `tests/test_web_bundle.py` pins both.
 - `tools/make_promo.py` — renders the main promo video from measured runs.
 - `tools/make_learn_promo.py` — the learned-heuristic/RL video. It re-measures
   everything including both failure modes, so it cannot drift from `.docs/`;

CHANGELOG.md

@@ -4,6 +4,27 @@ All notable changes to this project are documented in this file. The format is
 based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this
 project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+- **A Research view in the workbench.** `jupyddl.learn` shipped in 2.3.0 with
+  its results only in `.docs/` and a video, which meant the published site said
+  nothing about the biggest addition in the release. The view carries the
+  measured run: imitation against `hff` and `goalcount`, the per-instance
+  spread behind the mean, the logistics loss and why the feature space causes
+  it, and the three claims that turned out to be wrong. Its numbers are built
+  from `promo/rl-data.json` — the same cache the RL video renders from — so the
+  page and the video cannot drift apart, and a test pins them together.
+
+### Fixed
+- **Reading the workbench no longer costs a 10 MB download.** `<main>` was
+  hidden until Pyodide reported ready, so every page of prose, the requirement
+  matrix and the new Research view sat behind a full-screen spinner waiting for
+  a runtime none of them use. The shell now renders immediately from the
+  committed bundle, the boot notice is a status bar rather than a splash
+  screen, and only the controls that actually run a planner stay disabled until
+  the interpreter arrives.
+
 ## [2.3.0] - 2026-07-31
 
 ### Added

README.md

@@ -255,7 +255,7 @@ core is stdlib-only there is no wheel to resolve: the package sources are handed
 straight to the interpreter. Everything is computed in your tab; nothing is
 uploaded.
 
-Four views:
+Five views:
 
 - **Solve** — edit the PDDL, pick a planner, a heuristic and a budget, then watch
   the cost curves and the search wavefront animate while it works. Or ground
@@ -265,9 +265,17 @@ Four views:
   sorts on any column, and the whole run exports to CSV or JSON.
 - **PDDL support** — the requirement matrix, filterable by support level, read
   straight out of the library rather than transcribed.
+- **Research** — what `jupyddl.learn` measured: the imitation result, what the
+  reinforcement stage added, the domain where the whole approach loses and the
+  exact reason, and the three claims that turned out to be wrong.
 - **Generate** — produce a reproducible instance from a *(kind, size, seed)* and
   open it in Solve.
 
+The pages that are only text and measurements — **PDDL support** and
+**Research** — render immediately from the committed bundle. Only the controls
+that actually run a planner wait for the interpreter, so reading the workbench
+never costs a 10 MB download.
+
 <div align="center">
 
 <img src="promo/workbench-dark.png" alt="The jupyddl workbench: a PDDL editor beside live cost-estimate charts, a radial search wavefront, and the resulting validated plan." width="880">

docs/RELEASING.md

@@ -47,9 +47,15 @@ publish unless it also runs in that environment.
 Repeat on <https://test.pypi.org> with environment name `testpypi` if you want
 the dry run below to work.
 
-Until this exists the `pypi` job fails with an OIDC error. Everything before it
-still succeeds, so a tag pushed early leaves you with verified artifacts and no
-partial publish — re-run the job once the publisher is configured.
+Until this exists the `pypi` job fails with an OIDC error. Nothing else is
+blocked by it: the build still verifies, the tag is still created and the
+GitHub Release is still cut with the artifacts attached. Re-running the failed
+job once the publisher is configured completes the release.
+
+That decoupling is deliberate. A GitHub Release records that a version was cut
+from a verified commit; PyPI is a downstream channel that can fail for reasons
+the code has nothing to do with. Losing the release because the upload failed
+would leave no record of the version and force a full re-run to get one.
 
 ### Optional: require a human to approve each publish
 

renovate.json

@@ -1,5 +1,21 @@
 {
   "extends": [
-    "config:base"
+    "config:recommended"
+  ],
+  "packageRules": [
+    {
+      "description": [
+        "The `python-version` pins in build.yml, pages.yml and release.yml are",
+        "not a dependency to keep current -- they choose which interpreter",
+        "builds the wheel, bundles web/dist and cuts the release. `tests`",
+        "already runs the whole supported range (3.9-3.14) on every push, and",
+        "the wheel is py3-none-any, so building on the newest interpreter buys",
+        "nothing and ties the release pipeline to whatever is newest on the",
+        "runner. Moving that pin should be a deliberate edit, not a bot PR."
+      ],
+      "matchManagers": ["github-actions"],
+      "matchDepNames": ["python"],
+      "enabled": false
+    }
   ]
 }

tests/test_web_bundle.py

@@ -90,10 +90,94 @@ def test_bundle_is_ordered_independently_of_the_filesystem(sources):
     assert list(sources) == sorted(sources)
 
 
+def test_capabilities_bundle_agrees_with_the_registries():
+    """The page renders the support matrix *before* Python loads.
+
+    It does that from ``capabilities.json``, which means a stale bundle no
+    longer merely lags — it states something untrue about the library to every
+    visitor, and keeps stating it for the seconds before the runtime arrives
+    and overwrites it.
+    """
+    path = os.path.join(DIST, "capabilities.json")
+    if not os.path.exists(path):
+        pytest.skip("web bundle not built")
+    with open(path, encoding="utf-8") as handle:
+        caps = json.load(handle)
+
+    from jupyddl.generator import describe_generators
+    from jupyddl.heuristics import HEURISTICS
+    from jupyddl.requirements import as_rows, summary
+    from jupyddl.search import describe_planners
+
+    assert caps["requirements"] == as_rows()
+    assert caps["requirement_summary"] == summary()
+    assert caps["planners"] == describe_planners()
+    assert caps["heuristics"] == sorted(HEURISTICS)
+    assert caps["generators"] == describe_generators()
+
+
+def test_research_bundle_quotes_the_measured_run():
+    """The Research view must not invent numbers.
+
+    ``collect_research`` distils ``promo/rl-data.json`` — the cache the RL
+    video renders from — so page and video quote one measured run and cannot
+    drift apart. When that file is absent the builder emits ``{}`` and the
+    view says so; that is the only other acceptable state.
+    """
+    path = os.path.join(DIST, "research.json")
+    if not os.path.exists(path):
+        pytest.skip("web bundle not built")
+    with open(path, encoding="utf-8") as handle:
+        research = json.load(handle)
+
+    measured = os.path.join(REPO_ROOT, "promo", "rl-data.json")
+    if not research:
+        assert not os.path.exists(measured)
+        return
+    with open(measured, encoding="utf-8") as handle:
+        data = json.load(handle)
+
+    assert research["corpus"] == data["corpus"]["count"]
+    assert research["parameters"] == data["imitation"]["parameters"]
+    assert research["after"]["learned"]["expanded"] == round(
+        data["transfer_after"]["learned"]["mean_expanded"], 1
+    )
+    assert research["after"]["hff"]["expanded"] == round(
+        data["transfer_after"]["hff"]["mean_expanded"], 1
+    )
+    # A learned heuristic is not admissible, so the claim that survives is
+    # coverage, not cost. Pin it: the view leads with it.
+    assert research["after"]["learned"]["coverage"] == 1.0
+
+
+def test_static_views_do_not_wait_for_the_runtime():
+    """Reading the page must not cost a 10 MB WebAssembly download.
+
+    Most of the workbench is prose, a support matrix and measurements, none of
+    which need Python. Hiding ``<main>`` until Pyodide reports ready made all
+    of it unreachable behind a spinner, which is how this regressed once.
+    """
+    with open(os.path.join(WEB, "index.html"), encoding="utf-8") as handle:
+        markup = handle.read()
+    assert '<main id="app">' in markup, "the app shell must render immediately"
+
+    with open(os.path.join(WEB, "app.js"), encoding="utf-8") as handle:
+        script = handle.read()
+    # The controls, and only the controls, are what the runtime gates.
+    assert "state.ready" in script
+    assert "dist/capabilities.json" in script
+
+
 def test_builder_is_reproducible(tmp_path):
     """Running the builder again must not change the committed bundle."""
     before = {}
-    for name in ("jupyddl-sources.json", "demos.json", "build.json"):
+    for name in (
+        "jupyddl-sources.json",
+        "demos.json",
+        "build.json",
+        "capabilities.json",
+        "research.json",
+    ):
         path = os.path.join(DIST, name)
         if not os.path.exists(path):
             pytest.skip("web bundle not built")

tools/build_web.py

@@ -201,6 +201,69 @@ def collect_capabilities() -> dict:
     }
 
 
+def collect_research() -> dict:
+    """Distil the learned-heuristic measurements for the Research view.
+
+    Read from ``promo/rl-data.json`` — the cache the RL promo video renders
+    from — so the page and the video quote the same measured run and cannot
+    drift apart. Returns ``{}`` when that file is absent, and the view then
+    says so rather than showing numbers from nowhere.
+    """
+    path = os.path.join(ROOT, "promo", "rl-data.json")
+    if not os.path.exists(path):
+        return {}
+    with open(path, encoding="utf-8") as handle:
+        data = json.load(handle)
+
+    def rows(summary):
+        return {
+            name: {
+                "expanded": round(entry["mean_expanded"], 1),
+                "seconds": round(entry["mean_seconds"], 4),
+                "cost": round(entry["mean_cost"], 1),
+                "coverage": round(entry["coverage"], 2),
+            }
+            for name, entry in summary.items()
+        }
+
+    spread = data.get("spread", {})
+    per_instance = [
+        {
+            "instance": name,
+            "imitation": spread.get("imitation", {}).get(name),
+            "tuned": spread.get("hi", {}).get("per_instance", {}).get(name),
+            "solved": spread.get("imitation_solved", {}).get(name, True),
+        }
+        for name in spread.get("instances", [])
+    ]
+
+    return {
+        "train_sizes": data.get("space", {}).get("train_sizes"),
+        "eval_sizes": data.get("space", {}).get("eval_sizes"),
+        "instances": data.get("space", {}).get("train_instances"),
+        "features": data.get("space", {}).get("features"),
+        "predicates": data.get("space", {}).get("predicates", []),
+        "corpus": data.get("corpus", {}).get("count"),
+        "parameters": data.get("imitation", {}).get("parameters"),
+        "mae": data.get("imitation", {}).get("mae"),
+        "top1": data.get("imitation", {}).get("top1"),
+        "train_seconds": data.get("imitation", {}).get("seconds"),
+        "cem_seconds": data.get("cem", {}).get("seconds"),
+        "before": rows(data.get("transfer_before", {})),
+        "after": rows(data.get("transfer_after", {})),
+        "per_instance": per_instance,
+        "flat": data.get("flat", {}),
+        "sigma": {
+            "lo": spread.get("lo", {}).get("sigma"),
+            "hi": spread.get("hi", {}).get("sigma"),
+            "lo_mean": round(spread.get("lo", {}).get("mean", 0), 1),
+            "hi_mean": round(spread.get("hi", {}).get("mean", 0), 1),
+        },
+        "logistics": data.get("logistics", {}),
+        "budget": spread.get("budget"),
+    }
+
+
 def version() -> str:
     namespace: dict = {}
     init = os.path.join(PACKAGE, "__init__.py")
@@ -224,6 +287,9 @@ def main() -> int:
     capabilities = collect_capabilities()
     with open(os.path.join(OUT, "capabilities.json"), "w", encoding="utf-8") as fh:
         json.dump(capabilities, fh, indent=1, sort_keys=True)
+    research = collect_research()
+    with open(os.path.join(OUT, "research.json"), "w", encoding="utf-8") as fh:
+        json.dump(research, fh, indent=1, sort_keys=True)
     with open(os.path.join(OUT, "build.json"), "w", encoding="utf-8") as fh:
         json.dump({"version": version(), "modules": len(sources)}, fh)
 
@@ -238,6 +304,11 @@ def main() -> int:
         f"{len(capabilities['planners'])} planners, "
         f"{len(capabilities['generators'])} generators -> web/dist/capabilities.json"
     )
+    print(
+        "  learned-heuristic measurements -> web/dist/research.json"
+        if research
+        else "  ! no promo/rl-data.json; the Research view will say so"
+    )
     return 0