docs: replace MkDocs with Zensical#3073
Conversation
📚 Documentation preview
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 15 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Note for reviewers: the red "Docs Preview" check is expected on this PRThe
The authoritative signal is One follow-up called out in the PR description: the combined build serves v1 (MkDocs) + v2 (Zensical), and this branch's Generated by Claude Code |
Migrate the v2 docs build from MkDocs Material to Zensical, the Material team's successor (as Starlette and uvicorn have done). Zensical natively re-implements search, glightbox and mkdocstrings, but — unlike MkDocs — runs no arbitrary plugins or hooks. This SDK's docs relied on three of those, so they are reimplemented as standalone build steps around `zensical build`: - API reference: `mkdocs-gen-files` + `mkdocs-literate-nav` become `scripts/docs/gen_ref_pages.py` (writes the `docs/api/` mkdocstrings stubs) and `scripts/docs/build_config.py` (splices the nested API nav into a generated `mkdocs.gen.yml`, since Zensical has no literate-nav). - llms.txt: the former MkDocs hook becomes `scripts/docs/llms_txt.py`, a post-build step over the source tree, with no dependency on build internals. `mkdocs.yml` stays the source config, with the MkDocs-only YAML tags (`!relative`, `!ENV`, `!!python/name:`) translated to the string forms Zensical understands and the emoji functions pointed at `zensical.extensions.emoji`. The generators moved out of `docs/` (Zensical publishes everything under docs_dir) into `scripts/docs/`, and `scripts/build-docs.sh` now picks the toolchain per worktree so the combined v1 (MkDocs) + v2 (Zensical) build stays correct. Adds `scripts/serve-docs.sh` for local preview. The docs dependency group drops mkdocs, mkdocs-material[imaging] and the gen-files/glightbox/literate-nav plugins for `zensical`. mkdocstrings is floored at 1.0.4 / mkdocstrings-python 2.0.5: Zensical's compatibility layer targets that API, and the older line renders API pages with an unregistered-autorefs error. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CTkPwxyKkLTnqZdeiCMCzK
The wrapper forwards `"$@"` straight to `zensical serve`, so extra args are passed directly; the `--` in the usage example would be forwarded literally. Show args as direct forwarding instead. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CTkPwxyKkLTnqZdeiCMCzK
The three-step build (materialise config, zensical build --strict, generate llms.txt) was copy-pasted in shared.yml, docs-preview.yml, and build-docs.sh. Move it into scripts/docs/build.sh and call that everywhere. docs-preview.yml now picks the toolchain from the checkout: pull_request_target runs the workflow file from the base branch, so hardcoding the Zensical steps would break previews for older heads and v1.x-targeted PRs (and previews of this branch before it merges).
Zensical ships a nav entry for a nonexistent page as a broken link with no diagnostic, even under --strict; MkDocs strict aborted the build. Validate the materialised nav in build_config.py so the guarantee survives the migration, and reject an empty generated API nav while at it.
The social plugin is not handled by the scripts under scripts/docs/ — it was
dropped (Zensical has no social-card support; the cards were gated on an env
var no workflow set). The NavItem alias excluded the {title: url} page shape
it mostly holds.
mkdocstrings renders a member re-exported across a package boundary as a full duplicate of its canonical documentation whenever the target package happens to be loaded already, and silently omits it when it isn't - which of the two the api/mcp/ index got depended on page rendering order (255 headings under Zensical vs 123 under MkDocs, from the same stub). Pin the semantics in the generated stub instead: every export with a canonical page elsewhere under the API reference becomes a compact link to it, and only exports documented nowhere else (re-exports from private modules) keep their full body via an explicit members list. Packages whose re-exports never cross a package boundary (mcp_types) are deterministic already and keep the plain stub. griffelib moves from a transitive to a declared docs dependency since the generator now imports it directly.
Zensical natively supports the watch key; dropping it broke serve live-reload for docs_src/ snippet edits. The deploy-docs banner env only applied to mkdocs-material builds, and the v1.x fallback arm already sets it inline.
Unresolvable cross-references render as literal bracket text without a diagnostic, even under --strict, and link validation skips non-markdown targets entirely (a missing image ships silently; MkDocs failed both). Grep the built site for the literal-bracket signature after the build, and validate relative asset targets while llms_txt walks the prose. Also fail when a generated package index is missing from llms.txt's Optional section, so adding a package to gen_ref_pages cannot silently under-publish.
Docstring summaries took the first line, not the first paragraph, so wrapped one-sentence docstrings shipped cut off mid-sentence on the package index. The private-module filter only looked at the path leaf, so a private subpackage would have published its whole subtree; and an export resolving outside the documented packages now fails with a named error instead of a raw griffe traceback.
build_config's --site-dir/--output flags had no callers and contradicted the fixed site/ output contract; nav validation now skips external URLs, which are valid nav entries. The griffelib comment preempts the reasonable suspicion that the name is a typo for griffe.
Rendering the stub resolves its cross-package aliases in mkdocstrings' own collection. On a warm incremental rebuild the target package's pages can all be cache hits, so nothing loads it and rendering the changed index crashes with AliasResolutionError — breaking the serve/edit loop for any src change that alters the generated index. Cold builds were unaffected. preload_modules pins the targets into the collection.
maxisbey
force-pushed
the
claude/replace-mkdocs-zensical-5m0ltl
branch
from
7c7b6f6 to
d629160
Compare
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| def _missing_nav_pages(nav: list, docs_dir: Path) -> list[str]: | ||
| """Collect nav page references that don't exist under the docs dir. | ||
|
|
||
| Zensical (0.0.48) ships a nav entry for a nonexistent page as a broken | ||
| link without any diagnostic, even under --strict — MkDocs aborted the | ||
| build. Validating here keeps that guarantee: the concrete config never | ||
| leaves this script referencing a page that isn't there. | ||
| """ | ||
| missing: list[str] = [] | ||
| for entry in nav: | ||
| value = next(iter(entry.values())) if isinstance(entry, dict) else entry | ||
| if isinstance(value, list): | ||
| missing.extend(_missing_nav_pages(value, docs_dir)) | ||
| elif "://" not in value and not (docs_dir / value).is_file(): | ||
| missing.append(value) | ||
| return missing |
There was a problem hiding this comment.
🟡 The old pipeline's validation: omitted_files: warn combined with strict: true made a docs page that exists under docs/ but is missing from the nav abort the build (MkDocs' default for omitted_files is info, so this was a deliberate strengthening, not a restated default) — the new pipeline only validates the reverse direction (nav entry → file exists), so an orphaned page now builds green under zensical build --strict, gets published unreachable from navigation, and is silently absent from llms.txt/llms-full.txt. A small reverse docs/ → nav completeness check alongside _missing_nav_pages would restore the lost guard.
Extended reasoning...
What the bug is. The pre-PR mkdocs.yml set strict: true plus validation: omitted_files: warn. MkDocs' default severity for nav.omitted_files is info (mkdocs 1.6.1, config/defaults.py), which is invisible to strict mode — so raising it to warn was a genuine, deliberate strengthening: under --strict, warnings become errors, and a markdown page under docs/ that was not referenced by the nav aborted the build. The PR description justifies deleting the validation: block as "a severity filter that at best restates defaults", which is factually wrong for this key. The guard was real, and it is now gone.
Why nothing in the new pipeline covers it. The PR's sidecar scripts validate only the forward direction. build_config.py's _missing_nav_pages() checks that every nav entry points to an existing file under docs/ (nav → file); nothing checks that every docs/**/*.md file appears in the nav (file → nav). scripts/docs/llms_txt.py walks only the nav (_walk_nav/_collect_pages), so an orphaned page never enters prose and is silently excluded from llms.txt, llms-full.txt, and the per-page markdown renditions. The PR's own inventory of guarantees ported to sidecars — missing nav target, unresolved cross-reference, missing asset link, llms.txt package coverage — does not include omitted files. Zensical itself does not fill the gap: verifiers built a minimal project with an orphaned docs/orphan.md on zensical 0.0.48 and zensical build --strict printed 'No issues found', exited 0, and published the orphan into the site.
Step-by-step proof (reproduced empirically by a verifier against this branch):
- Create
docs/orphan-test-page.mdwith any content; do not add it to thenav:inmkdocs.yml. - Run
bash scripts/docs/build.sh. build_config.pypasses —_missing_nav_pagesonly iterates nav entries, and every nav entry still resolves to a real file.zensical build -f mkdocs.gen.yml --strictexits 0 with 'No issues found' and publishessite/orphan-test-page/index.html.llms_txt.pypasses — it walks the nav, never sees the orphan, and emitsllms.txt/llms-full.txtwithout it.- Net result: green build, a published page no navigation links to, and a page invisible to LLM consumers of llms.txt. On the pre-PR pipeline, step 2's equivalent (
mkdocs build --strict) aborted with the omitted-files warning promoted to an error.
Impact. The failure mode when triggered is silent — unlike the other failure classes this PR ports to sidecars, which all fail loudly. A contributor adding docs/servers/new-feature.md but forgetting the nav entry (a classic mistake, and exactly what the old guard existed to catch) ships an unreachable page with no error at any stage. This is the same failure class (silently incomplete published docs) that the PR's llms.txt coverage check exists to prevent for API packages.
How to fix. A few lines in build_config.py next to _missing_nav_pages: collect every docs/**/*.md (excluding the generated api/ tree and any non-page files like .overrides), collect the set of page paths referenced by the final nav, and SystemExit if any file is unreferenced. Equivalently the check could live in llms_txt.py, but build_config.py runs first and is where the other nav validation already lives.
Why nit rather than blocking. Nothing concretely breaks by merging this PR as-is — there is no orphaned page in the tree today, and the regression only manifests on a future contributor mistake. It is worth fixing (here or in a fast follow-up) because the removal was justified by an incorrect claim about MkDocs defaults and the PR's stated goal is preserving exactly this kind of strict-mode guarantee, but it does not need to hold up the merge.
| # Every generated package index must be listed: a package added to | ||
| # gen_ref_pages.PACKAGES without an entry here would be published on the | ||
| # site but silently missing from llms.txt. | ||
| generated = {f"api/{path.name}/index.md" for path in (DOCS / "api").iterdir() if path.is_dir()} | ||
| if unlisted := generated - {src_uri for src_uri, _, _ in _OPTIONAL_PAGES}: | ||
| raise _BuildError(f"llms_txt: generated package indexes missing from _OPTIONAL_PAGES: {sorted(unlisted)}") | ||
| for src_uri, title, description in _OPTIONAL_PAGES: | ||
| if not (DOCS / src_uri).exists(): | ||
| raise _BuildError(f"llms_txt: optional page {src_uri} not found (run gen_ref_pages first)") |
There was a problem hiding this comment.
🟡 Running the documented standalone invocation (python scripts/docs/llms_txt.py --site-dir site) on a checkout where docs/api/ doesn't exist crashes with a raw FileNotFoundError traceback from (DOCS / "api").iterdir(), because main() only catches _BuildError — the friendly "run gen_ref_pages first" message a few lines below is unreachable for exactly the missing-directory case it describes. Guarding the iterdir() (or converting OSError there into _BuildError) would restore the designed clean SystemExit message; the canonical scripts/docs/build.sh pipeline always runs gen_ref_pages first, so CI is unaffected.
Extended reasoning...
The bug. generate() in scripts/docs/llms_txt.py enumerates the generated package indexes with (DOCS / "api").iterdir() (line 202). docs/api/ is a gitignored build artifact (this PR adds /docs/api/ to .gitignore) that only exists after scripts/docs/gen_ref_pages.py has run. When the directory is absent, Path.iterdir() raises FileNotFoundError — an OSError, not the script's _BuildError — so main()'s except _BuildError doesn't catch it and the user gets a raw traceback instead of the clean SystemExit message the script's error-handling design promises.
The code path. The module docstring documents standalone invocation: python scripts/docs/llms_txt.py --site-dir site. On a fresh checkout, nothing earlier in generate() fails first: the prose pages, snippets, and assets all exist in the repo, and the API Reference: api/ nav entry is skipped by _collect_pages (it doesn't end in .md). So line 202 is the first failure point.
Why the designed error is shadowed. Lines 206–207 contain a friendly check for exactly this situation: 'llms_txt: optional page {src_uri} not found (run gen_ref_pages first)'. But that loop runs after the iterdir(), so when docs/api/ is entirely missing the check can never fire — it remains reachable only in the narrower case where docs/api/ exists but a listed package directory is missing.
Step-by-step proof.
- Fresh clone of the repo;
docs/api/is absent (gitignored, only created bygen_ref_pages.py). - Run
python scripts/docs/llms_txt.py --site-dir siteper the module docstring (assumesite/exists from a prior build, or the earlierwrite_textcalls fail similarly — either way, no clean message). generate()processes all prose pages successfully, then reachesgenerated = {... for path in (DOCS / "api").iterdir() ...}.Path.iterdir()raisesFileNotFoundError: [Errno 2] No such file or directory: '.../docs/api'.main()'sexcept _BuildErrordoesn't match (FileNotFoundErroris anOSError), so the exception propagates as a full traceback — the(run gen_ref_pages first)guidance is never shown.
Impact and severity. This is a nit, not a blocker: the canonical pipeline (scripts/docs/build.sh, called by shared.yml, docs-preview.yml, and scripts/build-docs.sh) always runs build_config.py/gen_ref_pages before llms_txt.py, so CI and every supported flow never hit this. When a contributor does hit it, the failure is still loud and the traceback names the missing docs/api path — nothing breaks silently and no incorrect artifact is produced. Only the error-message ergonomics the script deliberately designed (_BuildError → clean SystemExit) are lost. The same reasoning applies to other OSError paths, e.g. a nav page listed in mkdocs.yml but deleted from docs/ raises at the read_text on line 172.
The fix. One small change: guard the section with an existence check (if not (DOCS / "api").is_dir(): raise _BuildError("llms_txt: docs/api not found (run gen_ref_pages first)")), or catch FileNotFoundError around the iterdir() and re-raise as _BuildError. Optionally, main() could also catch OSError to cover the other file-read paths.
…e canvas Zensical's UI keeps Material's md-* class names but ships different stock nav styling (8px-radius pill rows with 7px 16px padding), which broke the sidebar design in two ways: section labels sat indented past their guide line (the label's inner anchor carries the pill padding), and the vertical rhythm doubled. Re-express the same design against the new stock rules, keeping Zensical's pill highlight for the active page. The dark scheme also swaps Zensical's near-black canvas back for the Material slate blue-grey this site shipped with; code blocks keep Zensical's own surfaces.
The stubs have no H1 of their own, so their pages fell back to the title Index — which is what navigation.prune shows on collapsed nav rows (every collapsed API subtree rendered as 'Index'), and what browser tabs and search results display.
| # Code examples live as complete, importable, tested files under `docs_src/` | ||
| # and are included into pages with `--8<-- "docs_src/<chapter>/tutorialNNN.py"` | ||
| # (resolved against the repo root regardless of the build's working | ||
| # directory; the extension's default base_path is the CWD). | ||
| # `check_paths: true` + `strict: true` turn a renamed/deleted example into a | ||
| # build failure instead of a silently empty code block. | ||
| # (resolved against the repo root, which is the build's working directory). | ||
| # `check_paths: true` turns a renamed/deleted example into a build failure | ||
| # instead of a silently empty code block. | ||
| - pymdownx.snippets: | ||
| base_path: !relative $config_dir | ||
| base_path: [.] | ||
| check_paths: true | ||
| - pymdownx.tilde | ||
| - pymdownx.inlinehilite | ||
| - pymdownx.highlight: |
There was a problem hiding this comment.
🟡 Replacing pymdownx.extra with a bare pymdownx.superfences entry silently drops the four other extensions the extra meta-extension enabled that aren't re-listed here: footnotes, def_list, abbr, and pymdownx.betterem. Nothing on the site uses them today, so nothing breaks at merge — but a future footnote or definition list will render as literal [^1]/: term text with no error even under --strict. Either re-add the four extensions explicitly or note the deliberate narrowing in the comment block that already documents every other dropped capability.
Extended reasoning...
What changed. The old config enabled pymdownx.extra (with the mermaid custom-fence options nested under it). pymdownx.extra is a meta-extension: in the locked pymdown-extensions 11.0.1, pymdownx/extra.py defines extra_extensions = [pymdownx.betterem, pymdownx.superfences, markdown.extensions.footnotes, attr_list, def_list, tables, abbr, md_in_html]. This PR replaces it with a direct pymdownx.superfences entry carrying the custom fences, and the surviving explicit list only covers tables, attr_list, md_in_html, and superfences (which were already listed pre-PR). Net effect: footnotes, def_list, abbr, and pymdownx.betterem are no longer registered at all.\n\nWhy nothing catches it. Markdown extensions don't fail when syntax they would have handled appears in a page — the syntax just stops being special. A page containing [^1] and a matching [^1]: note renders the literal bracket text into the published HTML; zensical build --strict sees nothing wrong because there is no diagnostic to promote. None of this PR's sidecar guards cover it either: build_config.py validates nav targets, the build.sh grep only catches unresolved [<code> autorefs artifacts, and llms_txt.py copies raw source markdown (which is also why the byte-identical llms.txt verification in the PR description couldn't detect this — it compares source markdown, not rendered HTML).\n\nStep-by-step proof.\n1. On the old toolchain, pymdownx.extra loads footnotes (per extra_extensions in pymdown-extensions 11.0.1), so Some claim.[^1] + [^1]: The source. renders as a superscript link and a footnotes block.\n2. On this branch, no entry in markdown_extensions loads footnotes — the explicit list is tables, admonition, attr_list, md_in_html, various pymdownx.* entries, and sane_lists; none is footnotes or a meta-extension that pulls it in.\n3. A contributor adds that footnote to, say, docs/servers/tools.md and runs bash scripts/docs/build.sh: build_config.py passes, zensical build --strict exits 0, the crossref grep finds no [<code>, llms_txt.py passes.\n4. The published page shows the literal text Some claim.[^1] and a stray [^1]: The source. paragraph. Same story for term + : definition (def_list) and *[MCP]: Model Context Protocol (abbr).\n\nWhy this doesn't bite today. I re-verified by grep that no page under docs/ currently uses footnote, definition-list, or abbr syntax (the one footnote-looking hit is a [^/]+ regex inside an inline code span in docs/migration.md, which the footnotes extension never processes). The betterem drop is the only piece touching existing prose — it changes emphasis-parsing edge cases for nested/intra-word */_ — but no concrete rendering difference on current pages was demonstrated, and Python-Markdown 3.x already smart-handles intra-word underscores by default.\n\nWhy it's still worth a comment. The drop looks like an oversight rather than a decision: the old config nested only the superfences options under pymdownx.extra, suggesting the meta-extension's other seven members were incidental side effects nobody inventoried, and this PR otherwise carefully documents every dropped capability (social cards, validation: block, gen-files, literate-nav, the llms_txt hook) in the mkdocs.yml comment block and the PR description — but not this one. It's also exactly the silent-failure class the PR's sidecar guards exist to prevent elsewhere.\n\nFix. Four lines: add footnotes, def_list, abbr, and pymdownx.betterem to markdown_extensions (all work under Zensical — it depends on the real pymdown-extensions package, and pymdownx.extra itself still exists in the locked 11.0.1 if you'd rather keep the meta-extension). Alternatively, if the narrowing is intentional, one sentence in the comment block documenting it. Marked nit because the built site is byte-for-byte unaffected today; the cost only lands on a future contributor.
| def _rewrite_links(markdown: str, src_uri: str, site_url: str, prose: dict[str, str | None]) -> str: | ||
| src_dir = posixpath.dirname(src_uri) | ||
|
|
||
| for match in _ASSET_LINK.finditer(markdown): | ||
| target = match.group(1) | ||
| if "://" in target: | ||
| continue | ||
| if not (DOCS / posixpath.normpath(posixpath.join(src_dir, target))).exists(): | ||
| raise _BuildError(f"llms_txt: cannot resolve asset link target {target!r} in {src_uri}") | ||
|
|
||
| def rewrite(match: re.Match[str]) -> str: | ||
| opening, target, anchor, title, closing = match.groups() | ||
| if "://" in target: | ||
| return match.group(0) | ||
| linked = posixpath.normpath(posixpath.join(src_dir, target)) | ||
| if not (DOCS / linked).exists(): | ||
| raise _BuildError(f"llms_txt: cannot resolve link target {target!r} in {src_uri}") | ||
| # Pages without a markdown rendition (the api/ stubs) link to their HTML instead. | ||
| url = _dest_md_uri(linked) if linked in prose else _page_url(linked) | ||
| return f"{opening}{site_url}{url}{anchor or ''}{title or ''}{closing}" | ||
|
|
There was a problem hiding this comment.
🟡 The relative-link validators in _rewrite_links check (DOCS / normpath(...)).exists() with no containment check, so a target that escapes docs/ but lands on a real repo file (e.g. ../../docs_src/extensions/tutorial004.py from a page under docs/advanced/) passes validation yet 404s on the published site — and per the PR description this check is the sole guard for non-md links in the pipeline. Mirroring the is_relative_to containment check that _resolve_snippets in the same file already does (against DOCS) closes the hole; nothing in the tree trips it today, so this is a hardening fix rather than a blocker.
Extended reasoning...
What the bug is. _rewrite_links in scripts/docs/llms_txt.py validates every relative link and asset target with a bare filesystem existence check — (DOCS / posixpath.normpath(posixpath.join(src_dir, target))).exists() — for both the asset-link loop and the .md-link rewrite closure. There is no check that the normalized path stays under docs/. A relative target that climbs out of docs/ but happens to land on a real file elsewhere in the repo passes validation, even though that file is never copied into the built site/, so the published link 404s.
Concrete proof (reproduced against this checkout). Suppose a contributor writes in docs/advanced/extensions.md: [full runnable example](../../docs_src/extensions/tutorial004.py) — a natural mistake, since every page already references these files via --8<-- includes.
src_dir = 'advanced';posixpath.normpath(posixpath.join('advanced', '../../docs_src/extensions/tutorial004.py'))→'../docs_src/extensions/tutorial004.py'.DOCS / '../docs_src/extensions/tutorial004.py'resolves to the real repo-rootdocs_src/extensions/tutorial004.py, so.exists()isTrue(verified) and the_ASSET_LINKbranch passes — asset links are validated but never rewritten, so the escaping relative link ships verbatim into the.mdrendition.docs_src/is never published intosite/, so the link 404s on the deployed site. The whole build stays green:zensical build --strictpasses (its link validation only covers.mdtargets, per the PR description), andllms_txt.pypasses.
The .md variant is similar: [readme](../README.md) from a top-level page passes the existence check (DOCS / '../README.md' exists) and _page_url('../README.md') emits the literal broken absolute URL https://py.sdk.modelcontextprotocol.io/v2/../README/ into the renditions and llms-full.txt. A leading-slash target is worse in a different direction: Path('docs') / '/foo.png' discards the docs prefix entirely and checks the filesystem root (verified: DOCS / '/etc/hostname' → /etc/hostname).
Why existing code doesn't prevent it. The PR description explicitly assigns this failure class to this script ("Missing non-markdown link target … llms_txt.py validates relative asset links while walking the prose") and states Zensical's own validation only covers .md targets — so for non-md links this check is the only guard anywhere in the pipeline, and it accepts exactly the class of link it was built to reject. The old pipeline caught this: mkdocs strict + unrecognized_links failed the build, and the old hook's files.get_file_from_path() (which only knew docs_dir files) returned None → PluginError. The asymmetry is clearly unintentional: _resolve_snippets in the same file guards escapes with resolved.is_relative_to(ROOT.resolve()) and raises _BuildError, mirroring the snippets extension's restrict_base_path — the link validators just never got the equivalent.
Impact. When triggered, the failure is fully silent — green zensical build --strict, green llms_txt step, broken link on the published site and in the llms.txt artifacts — precisely the silent-failure class this PR's sidecar guards were written to eliminate.
How to fix. Resolve the joined path and require containment, e.g. resolved = (DOCS / posixpath.normpath(posixpath.join(src_dir, target))).resolve() then if not (resolved.is_relative_to(DOCS.resolve()) and resolved.exists()): raise _BuildError(...) in both the asset loop and the rewrite closure (or equivalently, reject normpath results starting with ../ or targets starting with /).
Why nit rather than blocking. No page under docs/ currently contains an escaping relative link (verified by all four independent verifiers), so merging as-is breaks nothing today — the hole only manifests on a future contributor mistake. But since this guard is one of the PR's advertised deliverables, the two-line containment check is well worth adding while the file is fresh.
The section-label overrides tied the density rule on specificity and lost on source order, re-indenting labels; and the density padding also hit the md-nav__container wrapper (stock 0), double-indenting group rows that link to an index page. Order the density rules first and densify the container's inner anchor instead of the wrapper. The slate canvas override moves under @media screen like the stock rule it replaces, so dark-mode printing keeps a light canvas; stub titles gain YAML quoting.
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Top-level pages outside any section kept the pill's left padding plus a stock margin, sitting 14px right of the section labels; pull their box left by the pill inset so text lines up and the active pill overhangs the gutter. The API Reference label (the one section without an index page, rendered as a label element rather than a container div) was still caught by the density rule via :not() specificity — exclude section items from the density rules outright instead of relying on source order.
| # `zensical build --strict` fails on broken links / missing nav targets and | ||
| # `pymdownx.snippets: check_paths: true` fails on a deleted `docs_src/` | ||
| # include; the llms_txt step additionally re-resolves every snippet and link. | ||
| # Until this job existed the docs were only ever built post-merge by | ||
| # `deploy-docs.yml`, so those failures went green on the PR and broke the next | ||
| # deploy of main. This is the check path; `deploy-docs.yml` stays the deploy | ||
| # path. |
There was a problem hiding this comment.
🟡 This rewritten comment credits zensical build --strict with failing on missing nav targets, but the PR's own empirical finding (documented in build_config.py's _missing_nav_pages docstring) is the opposite: Zensical 0.0.48 ships a nav entry for a nonexistent page as a broken link with no diagnostic even under --strict — build_config.py is the actual enforcement point. Consider rewording to attribute each failure class to its actual owner, e.g. "scripts/docs/build.sh fails on broken links (zensical --strict), missing nav targets (build_config.py), and deleted docs_src/ includes (check_paths)", so a future maintainer doesn't remove the "redundant" nav validation or invoke zensical directly believing nav validation is covered.
Extended reasoning...
What the comment says vs. what the PR established. The rewritten comment above the docs job reads "zensical build --strict fails on broken links / missing nav targets and pymdownx.snippets: check_paths: true fails on a deleted docs_src/ include". For the missing-nav-target half, this contradicts the PR's own empirically verified finding. scripts/docs/build_config.py's _missing_nav_pages docstring states: "Zensical (0.0.48) ships a nav entry for a nonexistent page as a broken link without any diagnostic, even under --strict — MkDocs aborted the build. Validating here keeps that guarantee." The PR description restates the same thing in its guarantees inventory: "Missing nav target → ships as a broken nav link with zero diagnostics, even under --strict → build_config.py validates every nav entry."
Why the pipeline still passes/fails correctly today. The job's build step runs bash scripts/docs/build.sh, which invokes build_config.py before zensical build -f mkdocs.gen.yml --strict. build_config.py raises SystemExit on any nav entry that doesn't resolve to a file under docs/, so the docs job as a whole does fail on a missing nav target — the observable CI behavior matches the comment. The inaccuracy is purely in which component the comment names as the enforcement point.
Step-by-step proof of the misattribution. (1) Add - Ghost page: ghost.md to mkdocs.yml's nav without creating docs/ghost.md. (2) Run scripts/docs/build.sh: build_config.py's _missing_nav_pages collects ghost.md and the script exits with build_config: nav references pages that don't exist under docs/: ['ghost.md'] — zensical never runs. (3) Now bypass the sidecar and run zensical build --strict against a hand-written mkdocs.gen.yml with the same ghost entry: per the PR's own verification, the build exits 0 and publishes a broken nav link with zero diagnostics. So the failure the comment attributes to zensical is in fact produced by build_config.py; zensical alone would ship the breakage silently.
Why this matters despite being comment-only. Workflow comments like this one are exactly what a future maintainer reads when deciding what is safe to change. Trusting this comment, they could reasonably (a) treat build_config.py's _missing_nav_pages validation as redundant belt-and-braces and delete it, or (b) invoke zensical build --strict directly (skipping build.sh) believing nav validation is covered. Either action silently reintroduces the exact broken-nav failure mode this PR went to some length to close — and the PR even lists "missing-nav-target detection" as an upstream bug report worth filing against Zensical, underscoring that the tool does not do this today.
How to fix. A one-line reword that names the pipeline and attributes each failure class to its owner, e.g.: "scripts/docs/build.sh fails on missing nav targets (build_config.py), broken links (zensical build --strict), deleted docs_src/ includes (check_paths: true), and unresolved cross-references / asset links (the grep + llms_txt.py steps)." That keeps the comment's purpose (explaining why this check job exists) while making it a safe guide for future changes.
Severity. Nit: there is no runtime effect — the job fails on every failure class the comment lists, just via a different component than named. All three verifiers independently confirmed the factual contradiction and agreed it should not block merge; the fix is a comment reword.
| # Relative link/image targets with a non-markdown file extension. Zensical's | ||
| # link validation only covers .md targets (a missing image builds green even | ||
| # under --strict; MkDocs failed the build), so these are validated here. | ||
| _ASSET_LINK = re.compile(r'\]\(([^)\s#]+\.(?!md[)#\s])[a-zA-Z0-9]{1,4})(?:#[^)\s]*)?( +"[^"]*")?\)') |
There was a problem hiding this comment.
🟡 The _ASSET_LINK regex caps the file extension at [a-zA-Z0-9]{1,4}, so a relative link to an asset with a 5+ character extension (.jsonl, .woff2, .webmanifest, ...) matches neither _ASSET_LINK nor _MD_LINK and is never existence-checked — and per the PR description this validator is the sole guard for non-markdown link targets, so such a broken link would ship as a silent 404. Widening the quantifier (or dropping the cap and relying on the (?!md...) lookahead) closes the hole; nothing in docs/ trips it today.
Extended reasoning...
What the bug is. The asset-link validator in scripts/docs/llms_txt.py uses _ASSET_LINK = re.compile(r'\]\(([^)\s#]+\.(?!md[)#\s])[a-zA-Z0-9]{1,4})...'). The extension component is capped at 1–4 characters, so any relative link target whose extension is 5+ characters long simply never matches the pattern. Since backtracking can't help (there is only one dot and the closing paren must immediately follow the extension), the match fails outright rather than partially.
Empirical proof. With the exact compiled pattern from the file:
>>> _ASSET_LINK.findall('[data](files/data.jsonl)')
[]
>>> _ASSET_LINK.findall('[cfg](../app.webmanifest)')
[]
>>> _ASSET_LINK.findall('[font](fonts/inter.woff2)')
[]
>>> _ASSET_LINK.findall('[img](img/x.png)')
[('img/x.png', '')]
>>> _ASSET_LINK.findall('[img](img/x.webp)')
[('img/x.webp', '')]Four-character extensions (.webp, .avif, .webm) match; five-character ones (.jsonl, .woff2, .avifs, .webmanifest) don't.
The code path. In _rewrite_links, a link target is checked in exactly two places: the _ASSET_LINK loop (existence check for non-markdown targets) and the _MD_LINK rewrite closure (existence check + URL rewrite for .md targets). A target like files/data.jsonl fails both — it doesn't end in .md, and its extension exceeds the {1,4} cap — so it flows through _MD_LINK.sub untouched and is never validated against docs/. Nothing downstream catches it either: the PR description states Zensical's own link validation only covers .md targets ('Missing non-markdown link target (images etc.) → Zensical's link validation only covers .md targets → llms_txt.py validates relative asset links while walking the prose'), so this loop is the sole guard for the whole non-md class.
Impact. If a contributor adds [data](files/data.jsonl) (or a .woff2 font, a .webmanifest, etc.) pointing at a missing file, scripts/docs/build.sh runs green end to end — build_config.py passes, zensical build --strict passes, llms_txt.py passes — and the broken relative link ships verbatim into the published site and the .md renditions as a silent 404. That is precisely the silent-failure class this validator was written to eliminate. The old pipeline caught all extensions: MkDocs strict-mode unrecognized_links and the old hook's files.get_file_from_path were extension-agnostic.
Why nit rather than blocking. No relative link with a 5+ character extension exists anywhere under docs/ today (verified by grep — the only long-extension references are external URLs, which the :// guard skips), so merging as-is breaks nothing. The cost only lands on a future contributor, and even then the failure is a broken link, not a build or data problem. This is a coverage hole in an advertised guard, worth patching while the file is fresh.
How to fix. One-character change: widen the quantifier (e.g. {1,10}), or drop the length cap entirely — [a-zA-Z0-9]+ — and rely on the existing (?!md[)#\s]) negative lookahead to keep .md targets routed to _MD_LINK. Targets inside ](...) are already path-shaped, so widening costs nothing in false positives.
Distinct from the already-posted comment on this validator. The existing inline comment on line 151 covers the missing docs/ containment check (escaping ../ paths that land on real repo files). That is a different mechanism in the same function: this finding is about targets that never reach the existence check at all because of the extension-length cap.
| (site_dir / md_uri).parent.mkdir(parents=True, exist_ok=True) | ||
| (site_dir / md_uri).write_text(markdown, encoding="utf-8") | ||
|
|
||
| title = _title(src_uri, prose[src_uri], markdown) | ||
| index.append(f"- [{title}]({site_url}{md_uri})") | ||
|
|
||
| body, h1_found = re.subn(r"\A\s*# .+\n", "", markdown) | ||
| if not h1_found: | ||
| raise _BuildError(f"llms_txt: page {src_uri} does not start with an H1") |
There was a problem hiding this comment.
🟡 The new llms_txt.py reads raw page source and never strips YAML frontmatter, unlike the deleted hook which ran at on_page_markdown (post-frontmatter-extraction) — so a prose page with valid --- frontmatter (a pattern this PR's own generated api/ stubs use) fails the anchored H1 check with the misleading error "does not start with an H1", and the old hook's page.meta description tail in llms.txt index entries is silently dropped without appearing in the PR's inventory of dropped capabilities. No docs page has frontmatter today, so nothing breaks at merge; the fix is to strip/parse a leading frontmatter block before processing (optionally restoring the description feature), or at least name frontmatter in the error message.
Extended reasoning...
What the bug is. The deleted hook (docs/hooks/llms_txt.py) ran as an on_page_markdown MkDocs hook, so it always received markdown with the YAML frontmatter already extracted into page.meta — and it used page.meta.get("description") to append a description tail to each llms.txt index entry (tail = f": {description}" if description else ""). The new scripts/docs/llms_txt.py reads the raw source file with (DOCS / src_uri).read_text() and never parses or strips a leading frontmatter block. Two consequences follow.
Consequence 1: misleading build failure on frontmattered pages. The H1 check at the end of the per-page loop is re.subn(r"\A\s*# .+\n", "", markdown). \A\s* requires the H1 to be the first non-whitespace content of the file; - is not whitespace, so a file beginning with a --- frontmatter fence yields h1_found == 0 and the build aborts with llms_txt: page ... does not start with an H1 — even when a perfectly valid H1 sits immediately after the closing ---. This was empirically reproduced by the verifiers. Zensical itself renders such a page fine (frontmatter is a supported authoring pattern in both MkDocs and Zensical — indeed this PR's own gen_ref_pages.py writes title: frontmatter into every generated api/ stub), so this is a false-positive build failure introduced by the sidecar, with an error message pointing at the wrong cause. And had the check somehow not fired, the .md rendition written just above would ship the raw --- block verbatim into the published markdown and llms-full.txt.
Consequence 2: the description feature is silently gone. The new index.append(f"- [{title}]({site_url}{md_uri})") has no meta lookup at all. The only way the old pipeline's per-page description could be authored was via frontmatter — exactly the input the new script can no longer handle. This drop is absent from the PR's otherwise very thorough inventory of dropped capabilities (social cards, the validation: block, gen-files, literate-nav).
Why the PR's verification could not catch this. The PR description reports llms.txt, llms-full.txt, and all 50 per-page renditions byte-identical to the MkDocs pipeline. That holds because no page under docs/ currently has frontmatter (verified by grep across docs/**/*.md by multiple independent verifiers) — so no description tails existed to compare, and no page trips the H1 check. The regression is latent, not present.
Step-by-step proof.
- A contributor adds
---\ndescription: How to write MCP tools.\n---\n\n# Tools\n...to the top ofdocs/servers/tools.md— valid MkDocs/Zensical frontmatter, and on the old pipeline this would have produced a description tail in llms.txt. - Run
bash scripts/docs/build.sh.build_config.pypasses andzensical build --strictrenders the page correctly (title/description metadata honoured). llms_txt.pyreads the raw file;re.subn(r"\A\s*# .+\n", ...)sees the file starting with---, matches nothing,h1_found = 0.- The build aborts:
llms_txt: page servers/tools.md does not start with an H1— but the page does have an H1; the contributor is sent hunting for a nonexistent heading problem while the actual cause (unstripped frontmatter) is never named.
Impact and severity. Nothing in the tree triggers this at merge, and when triggered the failure is loud (the docs CI job aborts naming the page) rather than shipping a silently wrong artifact — only the error message's stated cause is wrong. The cost lands on a future contributor, which is why this is a nit rather than a blocker.
How to fix. Strip (and optionally parse) a leading ---\n...\n---\n block in generate() before snippet resolution — a small regex or yaml.safe_load on the fenced block. Parsing it would also restore the description feature (meta.get("description") → the old : {description} tail), which is likely the better move since the goal of this migration was byte-identical semantics. At minimum, extend the H1 error message to mention frontmatter as a possible cause.
The previous rules chased text alignment with per-element compensations (zeroed paddings, a negative margin), which clipped the active pill flat against the scroll container's paint edge, left label pills hugging their text, and fought the list's stock flex gap with per-link margins. Derive everything from the pill inset instead: every row is a direct .md-nav__link child with the same 10px horizontal padding (inner anchors inside container wrappers zeroed), so all text shares one column and pills always paint with breathing room inside the clip edge; section rules carry typography only, eliminating the specificity coordination; and the flex gap becomes the single density knob (2px, restoring the ~30px row pitch the site shipped with on Material).
Zensical paints nav hover/active pills on .md-nav__link[href] anchors and [for] labels, never on the .md-nav__container wrapper — which is where the previous commit put the padding, leaving container-type section labels with a pill hugging their text. Move the inset onto the paint elements (leaf links, bare labels, and a wrapper's inner anchor) and keep wrappers geometry-neutral like stock. The 10px inset also keeps the pill's corner radius (0.4rem) from crowding the glyphs.
| raise _BuildError(f"llms_txt: cannot resolve link target {target!r} in {src_uri}") | ||
| # Pages without a markdown rendition (the api/ stubs) link to their HTML instead. | ||
| url = _dest_md_uri(linked) if linked in prose else _page_url(linked) | ||
| return f"{opening}{site_url}{url}{anchor or ''}{title or ''}{closing}" | ||
|
|
||
| return _MD_LINK.sub(rewrite, markdown) | ||
|
|
||
|
|
||
| def _title(src_uri: str, nav_title: str | None, body: str) -> str: | ||
| if nav_title is not None: | ||
| return nav_title | ||
| match = re.search(r"^\s*# (.+)$", body, flags=re.MULTILINE) | ||
| if match is None: | ||
| raise _BuildError(f"llms_txt: page {src_uri} has no nav title and no H1") | ||
| return match.group(1).strip() | ||
|
|
||
|
|
||
| def generate(site_dir: Path) -> None: | ||
| config = yaml.safe_load((ROOT / "mkdocs.yml").read_text(encoding="utf-8")) |
There was a problem hiding this comment.
🟡 The new link validators in _rewrite_links run over raw markdown with only a "://" in target external-link guard, producing two classes of false-positive build failures on valid content: a mailto: link whose address ends in a common TLD (e.g. mailto:support@example.com — .com fits _ASSET_LINK's 1–4-char extension) bypasses the guard and is existence-checked as a relative path under docs/, and an illustrative asset link inside a fenced code block or inline code span (e.g. a tutorial showing [screenshot](images/setup.png)) is existence-checked even though fenced content is inert in the rendered HTML — either aborts the whole build with cannot resolve asset link target. Nothing under docs/ trips either today; the fixes are to treat any scheme-prefixed target (re.match(r'[a-zA-Z][a-zA-Z0-9+.-]*:', target)) as external in both the _ASSET_LINK loop and the rewrite closure, and to strip fenced code blocks/inline code spans before scanning.
Extended reasoning...
What the bug is. Both halves share one root cause: _rewrite_links scans the raw post-snippet markdown, and its only notion of "external" is if "://" in target: continue. That guard is too narrow in two directions. First, RFC-3986 scheme URIs without :// — mailto:support@example.com — sail past it, and because .com happens to fit _ASSET_LINK's [a-zA-Z0-9]{1,4} extension component, the whole address is captured as an "asset target" and existence-checked as a relative path under docs/. Second, the scan has no awareness of fenced code blocks or inline code spans, so example markdown shown inside a fence — the natural way to document link/image syntax — has its illustrative target existence-checked as if it were a real link.
The code path (empirically verified by four independent verifiers). For the mailto case: _ASSET_LINK.findall('[email us](mailto:support@example.com)') returns [('mailto:support@example.com', '')]. The target contains no ://, so the guard doesn't fire; posixpath.normpath(posixpath.join('advanced', 'mailto:support@example.com')) yields advanced/mailto:support@example.com, which can never exist under docs/, so _rewrite_links raises _BuildError('llms_txt: cannot resolve asset link target ...') and scripts/docs/build.sh (and with it the shared.yml docs job and the deploy) fails on a perfectly valid contact link. For the fence case: running the exact compiled regex over a page containing a fenced example like [screenshot](images/setup.png) returns the fenced targets — each gets the same existence check and the same fatal error, even though code-fence content is escaped in the built HTML and never rendered as a link.
Step-by-step proof (mailto). (1) A contributor adds Questions? [Email us](mailto:support@example.com). to docs/advanced/extensions.md — valid markdown that Zensical renders as a working mail link. (2) bash scripts/docs/build.sh runs: build_config.py passes, zensical build --strict passes (it treats scheme-bearing targets as external). (3) llms_txt.py reaches _rewrite_links: _ASSET_LINK matches mailto:support@example.com (extension .com), the :// guard is False, the joined path docs/advanced/mailto:support@example.com doesn't exist. (4) _BuildError aborts the build with a message pointing the contributor at a "missing file" that was never supposed to exist. The fenced-code proof is analogous: a markdown-authoring tutorial with \``markdown\nscreenshot\n```` fails step (3) the same way.
Why the old pipeline didn't have this. The deleted hook (docs/hooks/llms_txt.py) only validated .md-suffixed targets, so mailto:...com never entered its rewrite path, and MkDocs' own link validation recognizes mailto: as an external scheme. For fences, the old pipeline validated non-md links on the rendered page, where fence content is escaped and never treated as a link; the new script scans source text with no prose/code distinction. Two scope notes for precision: tel:+123 has no dot-extension and matches neither regex, so plain tel: links are merely (harmlessly) unvalidated rather than build-breaking; and the exotic _MD_LINK variant (mailto:x@foo.md, a .md TLD) would have failed the old hook's identical :// logic too, so that narrow sub-case is pre-existing — the common _ASSET_LINK mailto case and the fence case are squarely introduced by this PR.
Impact and severity. Grep confirms no mailto:/tel: links and no relative asset-style links inside fences exist anywhere under docs/ or docs_src/ today, so merging as-is breaks nothing — both failure modes are latent, land on a future contributor, and when triggered are loud (a clear _BuildError naming the page and target) rather than silently corrupting an artifact. That said, both produce a bogus failure with a misleading message on valid content, which is the inverse of this validator's purpose. This is distinct from the two already-posted comments on this function (the missing docs/ containment check and the {1,4} extension cap): those are under-validation holes where broken links slip through; this is over-validation where valid content is rejected.
How to fix. Two small changes in _rewrite_links: (1) replace the "://" in target checks (in both the _ASSET_LINK loop and the rewrite closure) with a scheme test, e.g. re.match(r'[a-zA-Z][a-zA-Z0-9+.-]*:', target) — matching how MkDocs classifies external links; (2) strip fenced code blocks and inline code spans from the markdown before running the asset/md link scans (a small regex pass over \``...```regions and backtick spans), so only prose regions are validated. Note the stripping should apply to the *validation* scans; the.md`-link rewrite's fence-blindness is pre-existing behavior from the old hook and can be left as-is or fixed in the same pass.
| # Zensical renders an unresolvable [`name`][identifier] cross-reference as | ||
| # literal bracket text and stays green even under --strict (mkdocs-autorefs | ||
| # used to warn, and strict mode failed). The generated API index relies on | ||
| # such references, so catch the failure mode here. | ||
| if grep -rn --include='*.html' -F '[<code>' site/ > /dev/null; then | ||
| echo "error: unresolved cross-references rendered as literal text:" >&2 | ||
| grep -rn --include='*.html' -Fo -m 1 '[<code>' site/ | head -20 >&2 | ||
| exit 1 | ||
| fi |
There was a problem hiding this comment.
🟡 The unresolved-crossref guard grep -F '[<code>' only fires when an unresolved [text][identifier] reference's text begins with a code span — a plain-text ref like [the Client API][mcp.Cleint] (or one whose code span isn't first) renders as literal bracket text with no [<code> marker and ships silently, and this grep is the sole guard for that class in the new pipeline. Nothing trips it today (the generated API index entries are all code-span-first), but with relative_crossrefs: true inviting prose/docstring crossrefs, broadening the check — e.g. also matching the trailing ][identifier] form in the built pages' visible text, scoped to exclude <pre>/<code> blocks — would close the hole.
Extended reasoning...
What the bug is. scripts/docs/build.sh guards against Zensical's silent handling of unresolved [text][identifier] cross-references with a literal match: grep -rn --include='*.html' -F '[<code>' site/. That string only appears when the reference's link text begins with a code span. [Client][mcp.Client] renders (unresolved) as literal [<code>Client</code>][mcp.Client] and is caught. But two whole classes of unresolved refs render as literal bracket text without that marker: (1) plain-text refs — [the Client API][mcp.Cleint] renders verbatim with no <code> anywhere; (2) refs whose code span is not the first character — [the Client class][mcp.Cleint] renders as [the <code>Client</code> class][mcp.Cleint], where [ is followed by t, so the substring [<code> never occurs. Both pass the guard and ship to the published site.
Why this is a coverage regression, not just a narrow check. Under the old pipeline, mkdocs-autorefs emitted a WARNING for every unresolved mandatory [text][identifier] reference — in any page or docstring, regardless of the text's shape — and strict: true promoted it to a build failure. The PR description's guarantees inventory advertises this grep as the replacement owner of the whole failure class ("Unresolved [name][identifier] cross-reference → renders as literal bracket text, build stays green → build.sh greps the built site for the rendered signature and fails"). No other guard covers it: llms_txt.py's _MD_LINK only matches inline ](target.md) syntax, never reference-style [text][id], and build_config.py validates nav entries only. So the text-agnostic guarantee narrowed to code-span-first-only, silently.
Step-by-step proof.
mkdocs.ymlenablesrelative_crossrefs: trueprecisely so authors can write cross-references in docstrings and prose.- A contributor writes
See [the Client class][mcp.client.Cleint] for details.(note the typo) in a docstring or a page underdocs/. zensical build -f mkdocs.gen.yml --strictexits 0 — per this PR's own verified finding, Zensical renders the unresolved ref as literal text with no diagnostic even under--strict.- The built HTML contains
[the Client class][mcp.client.Cleint]as visible text.grep -F '[<code>'finds nothing: there is no<code>in the text at all (and in the mid-text variant,[is followed byt). llms_txt.pypasses (_MD_LINKnever matches reference syntax). The build is green end to end, and the published page shows raw bracket text — the exact silent-failure class this guard was added to eliminate.
Why it doesn't bite today. The only [text][id] autorefs currently in the tree are the generated API index entries from gen_ref_pages.py, which are uniformly code-span-first (- [name][target]) and fully covered; grep finds zero such refs in docs/ prose or src/ docstrings. So nothing breaks at merge — this is a latent hole that lands on a future contributor's typo with zero diagnostics.
How to fix. Broaden the check to also catch the generic rendered form — unresolved mandatory refs keep their trailing ][identifier] literal, so something like grep -E '\]\[[A-Za-z_][A-Za-z0-9_.]*\]' over the built pages catches both missed classes. One caveat: a naive regex over raw HTML can false-positive on ][ sequences inside Pygments-rendered code blocks (chained subscripts like x[a][b] in examples), so the broadened pattern should be scoped to exclude <pre>/<code> content — or the check could move into llms_txt.py, which already walks the prose with more context. Alternatively, if the narrow literal is a deliberate precision tradeoff (the build.sh comment does frame the check around the generated API index), a sentence documenting that the guard only covers code-span-first refs would at least make the narrowing visible.
Severity. Nit: nothing in the tree trips this at merge, and the cost only lands on a future contributor. Same family as the _ASSET_LINK extension-cap finding — an advertised guard with a silent coverage gap, worth patching or documenting while the file is fresh, not blocking.
| title, value = next(iter(entry.items())) if isinstance(entry, dict) else (None, entry) | ||
| if isinstance(value, list): | ||
| pages.extend(_collect_pages(value, prose)) | ||
| elif value.endswith(".md") and not value.startswith("api/"): | ||
| prose[value] = title | ||
| pages.append(value) |
There was a problem hiding this comment.
🟡 An external nav entry whose URL ends in .md (e.g. - Contributing: https://github.com/.../CONTRIBUTING.md) is classified as a prose page by _collect_pages — unlike _missing_nav_pages and _rewrite_links, it has no :// exemption — so generate() tries to read_text() the URL as a local path and the build dies with a raw FileNotFoundError traceback instead of a clean _BuildError message. Nothing in today's nav triggers this; a one-line '://' not in value check mirroring the two sibling guards closes the hole.
Extended reasoning...
What the bug is. _collect_pages in scripts/docs/llms_txt.py decides what counts as a prose page with only value.endswith(".md") and not value.startswith("api/"). Unlike the two sibling link/nav handlers in this PR — build_config._missing_nav_pages explicitly skips values containing ://, and _rewrite_links in this same file skips :// targets — it has no external-URL exemption. An external nav entry whose URL happens to end in .md is therefore recorded as a local prose page.
The code path. External nav links are a supported MkDocs/Zensical feature, and this pipeline's own validator blesses them: build_config._missing_nav_pages sees the :// and skips validation, so zensical build --strict succeeds and renders the entry as an external nav link. Then llms_txt.py runs: _collect_pages records the full URL in prose, and generate() executes (DOCS / src_uri).read_text() with src_uri set to the URL. Path('docs') / 'https://github.com/...' joins as a relative path (docs/https:/github.com/...), which doesn't exist.
Why nothing catches it. read_text() raises FileNotFoundError — an OSError, not the script's _BuildError — so main()'s except _BuildError doesn't match and the exception propagates as a raw traceback. The designed clean-SystemExit error handling is bypassed entirely, and the traceback points at a mangled docs/https:/... path rather than naming the actual cause (an external URL in the nav). Even if the read somehow succeeded, the entry would be wrongly embedded in llms.txt/llms-full.txt as if it were a local page.
Why this is a regression, not pre-existing. The deleted MkDocs hook iterated nav.pages, and MkDocs models external nav entries as Link objects, never Pages — so the old pipeline never saw them and llms.txt correctly omitted them. The nav re-implementation in this PR reintroduced the class of input without the exemption its two siblings carry.
Step-by-step proof.
- Add
- Contributing: https://github.com/modelcontextprotocol/python-sdk/blob/main/CONTRIBUTING.mdtomkdocs.yml's nav — a natural shape when linking to a markdown file on GitHub. - Run
bash scripts/docs/build.sh.build_config.pypasses ('://' in value→ skipped) andzensical build --strictexits 0, rendering the external link fine. llms_txt.py's_collect_pageshits theelif:'https://github.com/.../CONTRIBUTING.md'.endswith('.md')isTrue,startswith('api/')isFalse→ the URL entersprose.generate()reaches(DOCS / src_uri).read_text(encoding="utf-8"); pathlib collapses the join todocs/https:/github.com/modelcontextprotocol/python-sdk/blob/main/CONTRIBUTING.mdandread_textraisesFileNotFoundError.main()only catches_BuildError;FileNotFoundErrorescapes and the docs build (including the CIdocsjob, which runs the samebuild.sh) dies with a raw traceback.
How to fix. Mirror the sibling guards: in _collect_pages, skip values containing :// (e.g. extend the elif to elif "://" not in value and value.endswith(".md") and ...). That restores the old pipeline's behavior of simply omitting external entries from llms.txt.
Why nit. No nav entry in the tree is an external URL today, so nothing breaks at merge. When a future contributor does add one, the failure is loud (the build aborts) — just with a misleading raw traceback instead of the clean message the script's error handling was designed to produce. Distinct from the already-posted comments on this file: the line-207 comment covers the docs/api iterdir() crash (different code path), and the _rewrite_links comments cover page-body links, not nav collection.
| # what pruned nav rows, browser tabs, and search results show. | ||
| stubs[API_DIR / doc_path] = f'---\ntitle: "{parts[-1]}"\n---\n\n::: {ident}\n' | ||
| if len(parts) == 1: | ||
| package_index[ident] = API_DIR / doc_path | ||
|
|
||
| node = root | ||
| for part in parts: | ||
| node = node.child(part) | ||
| node.url = f"api/{doc_path.as_posix()}" | ||
|
|
||
| # Load every package before inspecting any of them: aliases only resolve | ||
| # once the module they point at is in the loader's collection. | ||
| loader = griffe.GriffeLoader(search_paths=[str(package.parent) for package in PACKAGES]) | ||
| modules = {ident: loader.load(ident) for ident in package_index} | ||
| for ident, doc_path in package_index.items(): | ||
| module = modules[ident] | ||
| assert isinstance(module, griffe.Module) | ||
| if body := _compact_index(module, documented): | ||
| stubs[doc_path] = f'---\ntitle: "{ident}"\n---\n\n{body}' |
There was a problem hiding this comment.
🟡 The compact-index treatment (and its preload_modules emission) is gated on len(parts) == 1, but cross-package re-exports also exist in subpackages: mcp.server.mcpserver re-exports Icon from mcp_types in its __all__, so its plain ::: mcp.server.mcpserver stub inherits the same load-order-dependent duplicate-or-silently-omit rendering this PR's compact index eliminated for the top-level packages. Today that lands on "duplicate" (Icon's full canonical mcp_types docs repeated on the mcpserver page); a rendering-order flip would silently drop it with no diagnostic from --strict or the build.sh grep. Extending the cross-package-alias detection to all documented modules (or at minimum emitting preload_modules on such stubs) would close the gap.
Extended reasoning...
What the bug is. gen_ref_pages.py only feeds top-level packages into package_index — the if len(parts) == 1 gate at scripts/docs/gen_ref_pages.py:160 — so only the mcp and mcp_types indexes go through _compact_index, which is this PR's mechanism for pinning the semantics of members re-exported across a package boundary (compact linked entries + preload_modules). But cross-package re-exports aren't limited to top-level indexes, and one exists in the tree today: src/mcp/server/mcpserver/__init__.py line 3 does from mcp_types import Icon and lists Icon in __all__. Its generated page stays a plain ::: mcp.server.mcpserver stub with no compact handling and no preload_modules.
The mechanics — established by this PR itself. Per _compact_index's own docstring (and the PR description), mkdocstrings renders a member re-exported across a package boundary as a full duplicate of its canonical documentation when the target package happens to be loaded in the collection, and silently omits it when it isn't — which of the two you get depends purely on page rendering order. That is exactly the "accidental instead of deliberate" behavior the PR's headline fix eliminated for the two top-level indexes; the mcpserver stub keeps it, one level down.
Step-by-step proof.
generate()walkssrc/mcp/server/mcpserver/__init__.py→parts = ('mcp', 'server', 'mcpserver'),len(parts) == 3, so the module never enterspackage_indexand never reaches_compact_index. Its stub is the plain::: mcp.server.mcpserver.- During the full build,
mcp_typesis loaded in mkdocstrings' collection (its own pages render, and the compactmcpindex explicitly emitspreload_modules: [mcp_types]). So whenapi/mcp/server/mcpserver/index.mdrenders, theIconalias resolves and mkdocstrings inlines Icon's full canonicalmcp_typesdocumentation onto the MCPServer package page — duplicating the docs that already live at their canonicalapi/mcp_types/location. This is the current published-site behavior. - If a future mkdocstrings/Zensical rendering-order change flips the accident the other way (the empirically observed alternative — this same flip is what took the
mcpindex from 123 headings under MkDocs to 255 under Zensical),Iconis instead silently omitted from the page: no unresolved-ref bracket text is emitted, so thegrep -F '[<code>'guard inscripts/docs/build.shcan't see it, andzensical build --strictstays green. The member just disappears with zero diagnostics.
Why existing safeguards don't cover it. _compact_index scans only package.exports of the package_index entries (the two top-level packages); nothing walks subpackage __init__ exports for cross-package aliases; and neither of the two failure modes (duplicated content, silently missing member) is detectable by --strict or the crossref grep, since both produce well-formed HTML.
One correction to the original report. The claim that a warm incremental rebuild (scripts/serve-docs.sh) would crash this stub with AliasResolutionError is likely overstated: the crash the PR reproduced was specific to the compact index, whose explicit members: list and generated [name][target] crossrefs force alias resolution. For a plain stub, the PR's own documented mechanics say an unresolvable cross-package alias produces the silent omission, not a crash — so the realistic warm-rebuild failure mode is Icon transiently vanishing from the page, which is still the same undiagnosable class.
Why nit rather than blocking. The cold CI build is green, nothing crashes on any supported path, and the concrete effect at merge is content-quality: Icon's canonical docs duplicated on one subpackage page (arguably even consistent with that page's other full-body exports), with a latent order-dependent flip to silent omission. It's also pre-existing in kind — the deleted docs/hooks/gen_ref_pages.py wrote the identical plain stub — this PR just deliberately fixed it for the two indexes where ~50 re-exports made it egregious. Worth closing while the generator is fresh, not worth holding the merge.
How to fix. Detect cross-package aliases in every documented module's exports rather than only len(parts) == 1 entries, applying the compact treatment (or at least emitting preload_modules) wherever one is found — the _compact_index machinery already handles the general case; it just needs to be pointed at all documented modules with exports. A narrower alternative for the single in-tree case is to include mcp.server.mcpserver in package_index.
| # Zensical renders an unresolvable [`name`][identifier] cross-reference as | ||
| # literal bracket text and stays green even under --strict (mkdocs-autorefs | ||
| # used to warn, and strict mode failed). The generated API index relies on | ||
| # such references, so catch the failure mode here. | ||
| if grep -rn --include='*.html' -F '[<code>' site/ > /dev/null; then | ||
| echo "error: unresolved cross-references rendered as literal text:" >&2 | ||
| grep -rn --include='*.html' -Fo -m 1 '[<code>' site/ | head -20 >&2 | ||
| exit 1 | ||
| fi |
There was a problem hiding this comment.
🟡 The unresolved-crossref guard grep -F '[<code>' also fires on valid rendered content that contains no cross-reference: any prose with a literal bracket immediately followed by an inline code span — e.g. dict[str, Any] or result[0] in a docs page or any src/ docstring rendered by mkdocstrings — renders as [<code>..., so the build aborts with a misleading 'unresolved cross-references' error on content the author intended literally. Nothing in the tree trips it today; anchoring the pattern to the actual unresolved-autorefs rendering (require the trailing ][identifier] on the same match) would fix this false-positive direction and the false-negative coverage gap flagged in the existing line-35 comment at once.
Extended reasoning...
What the bug is. scripts/docs/build.sh guards against Zensical's silent rendering of unresolved [name][identifier] cross-references with a literal substring match: grep -rn --include='*.html' -F '[<code>' site/. But that substring is not unique to unresolved autorefs — it appears whenever rendered HTML contains a literal [ immediately followed by an inline code span. Ordinary, valid markdown produces exactly that: prose like the mapping is dict[str, Any] or use result[0] (bracket-then-backtick outside a code fence). Python-Markdown leaves a bare [str] with no matching reference definition as literal text, so the built page contains [<code>str</code>..., the grep matches, and the build exits 1 with "error: unresolved cross-references rendered as literal text" pointing at a page that has no cross-reference at all.\n\nThe code path. build.sh runs the grep over every built HTML page after zensical build --strict succeeds. The trigger surface is therefore not just docs/ prose but every docstring under src/ rendered by mkdocstrings — any of the codebase's docstrings gaining a bracket-then-code-span expression trips it. Escaped syntax examples also fire: a contributor documenting the crossref syntax itself with \\[Name\\][id] still renders with the literal [<code> prefix.\n\nStep-by-step proof (verified empirically against the locked markdown/pymdown-extensions versions):\n\n1. A contributor writes The handler returns dict[str, Any]. in a docs page or an SDK docstring — valid prose where the brackets are part of the type expression, intended literally.\n2. Python-Markdown's reference processor finds no reference definition for [str, Any], so it leaves the bracket text literal; the inline-code processor still converts the backtick spans. The rendered HTML is dict[<code>str</code>, <code>Any</code>] — visually exactly what the author wanted.\n3. zensical build --strict exits 0 (nothing is wrong with the page).\n4. grep -F '[<code>' matches the substring at dict[<code>, and build.sh aborts the whole docs build (and the CI docs job, and every deploy) blaming "unresolved cross-references" on a page that contains none.\n\nWhy existing checks don't prevent it. The grep is the only consumer of this signature and it is a bare substring match with no anchoring to the shape an actual unresolved autoref leaves behind. Resolved crossrefs render as <a> elements and don't fire; Pygments-highlighted fenced blocks emit <span>-wrapped tokens inside <pre><code> with the leading [ escaped separately, so they don't fire either — the guard is only wrong on this bracket-then-code-span prose class, which is uncommon but perfectly legitimate.\n\nImpact. Nothing in the tree trips it today (all four verifiers confirmed zero instances under docs/ or src/, matching this PR's green builds), so this is latent. When it does trip, the failure is loud and grep -n names the offending file:line — only the diagnosis in the message is wrong, so the cost is a future contributor's debugging time chasing a nonexistent cross-reference, plus the temptation to rewrite valid prose to appease the guard.\n\nHow to fix. Anchor the pattern to the rendering an unresolved mandatory autoref actually produces: the reference keeps its trailing ][identifier] literal, so requiring that suffix on the same match (e.g. grep -E '\\[<code>[^][]*</code>\\]\\[[A-Za-z_][A-Za-z0-9_.]*\\]') eliminates this false positive. Note this is the complement of the false-negative gap already flagged in the existing comment on line 35 (unresolved refs whose text is not code-span-first slip past the grep) — anchoring on the trailing ][identifier] form addresses both directions of the same guard in one change, so they're worth fixing together. Alternatively, a comment documenting the precision tradeoff would at least make the narrowing visible.\n\nSeverity. Nit: nothing breaks at merge, the failure when triggered is loud and CI-time rather than shipping a wrong artifact, and the contributor workaround (put the whole expression in one code span) is trivial.
The inline-code chip is an absolute dark surface designed for the page canvas; on an admonition's tinted panel it sits as an opaque slab (Zensical's modern theme has the same defect on its own docs — classic Material mitigated it with a hairline ring on title code, which modern dropped). Scope a tone-on-tone translucent chip to code inside admonitions, the pattern Starlight and Docusaurus ship for code inside callouts; prose chips keep the block-matching dark surface.
|
|
||
| _SNIPPET_LINE = re.compile(r'^(?P<indent>[ \t]*)--8<-- "(?P<path>[^"\n]+)"$', flags=re.MULTILINE) | ||
| _MD_LINK = re.compile(r'(\]\()([^)\s]+\.md)(#[^)\s]*)?( +"[^"]*")?(\))') | ||
| # Relative link/image targets with a non-markdown file extension. Zensical's | ||
| # link validation only covers .md targets (a missing image builds green even | ||
| # under --strict; MkDocs failed the build), so these are validated here. | ||
| _ASSET_LINK = re.compile(r'\]\(([^)\s#]+\.(?!md[)#\s])[a-zA-Z0-9]{1,4})(?:#[^)\s]*)?( +"[^"]*")?\)') |
There was a problem hiding this comment.
🟡 An internal link target with no file extension — the pretty-URL authoring forms [see tools](../servers/tools/), (tools/), (/servers/tools/), (/servers/tools) — matches neither _MD_LINK (requires .md) nor _ASSET_LINK (requires a 1–4-char dot extension), so _rewrite_links neither existence-checks nor rewrites it: the link ships verbatim into the built HTML (where a source-relative directory link resolves off-by-one against the page's directory URL and an absolute one misses the /v2/ prefix — both 404) and into the .md renditions/llms-full.txt. The old pipeline rejected all four forms (strict: true + unrecognized_links/absolute_links: warn), and Zensical only validates .md targets, so this link shape now has no owner anywhere in the pipeline; a third scan rejecting scheme-less targets whose final segment has no dot (directing authors to link the .md source) restores the old semantics. Nothing under docs/ uses these forms today, so this is latent — a coverage hole in an advertised guard, not a blocker.
Extended reasoning...
What the bug is. The two link scanners in scripts/docs/llms_txt.py partition internal links by file extension: _MD_LINK requires the target to end in .md, and _ASSET_LINK requires a dot-extension of 1–4 alphanumeric characters. A target with no extension — the natural pretty-URL authoring forms — matches neither. Verified empirically against the exact compiled patterns from the file:
'[see tools](../servers/tools/)' MD=False ASSET=False
'[see tools](tools/)' MD=False ASSET=False
'[see tools](/servers/tools/)' MD=False ASSET=False
'[see tools](/servers/tools)' MD=False ASSET=False
'[see tools](../servers/tools.md)' MD=True
So _rewrite_links neither existence-checks nor rewrites these targets: they flow through the whole pipeline untouched, into the built HTML, the per-page .md renditions, and llms-full.txt.
Why this is a real breakage, not just a validation gap. With directory URLs, a source-relative directory-style link is off by one level on the published site. Zensical's own link rewriting bails early on non-.md paths (zensical 0.0.48 extensions/links.py: if not path.endswith(".md"): return), so the target ships verbatim. A contributor writing [see tools](../servers/tools/) in docs/handlers/context.md (source-relative, the same base every other link on the page uses) gets a page built at /v2/handlers/context/; the browser resolves ../servers/tools/ against that to /v2/handlers/servers/tools/ — a 404. An absolute (/servers/tools/) misses the /v2/ prefix. .md-suffixed links don't have this problem because they are rewritten to the correct depth.
Why nothing catches it. (1) Zensical: per this PR's own verified finding, its link validation only covers .md targets, so the extensionless link builds green even under --strict. (2) llms_txt.py: neither regex matches (proven above), so neither existence check fires — and this script is the PR's designated owner of non-.md link validation. (3) build_config.py validates nav entries only; the build.sh grep only catches [<code> autoref artifacts.
Why this is a regression from the old pipeline. The pre-PR mkdocs.yml (visible in this diff) set strict: true plus validation: absolute_links: warn and unrecognized_links: warn. MkDocs flags a relative link that doesn't resolve to a file in docs_dir as an unrecognized-link warning and flags /...-absolute links via absolute_links; strict mode promoted both to build failures with a message telling the author to link to the .md source file. All four forms above aborted the old build. This PR removes the validation: block, and none of the sidecar guards inherited this class.
Step-by-step proof. (1) A contributor cross-links two docs pages using the URL they see in the browser — [Deploy & scale](/run/deploy/) or [tools page](../servers/tools/) — a common habit on sites with pretty URLs. (2) bash scripts/docs/build.sh runs green end to end: build_config.py passes (nav only), zensical build --strict passes (non-.md target, never validated), the crossref grep finds nothing, llms_txt.py passes (neither regex matches). (3) The published page carries a 404 link (relative form: off-by-one depth; absolute form: missing /v2/), and the .md rendition and llms-full.txt carry the broken target verbatim. On the old pipeline, step (2)'s equivalent aborted the build.
How to fix. Either (a) add a third scan for scheme-less ](...) targets whose final path segment has no dot, raising _BuildError directing the author to link the .md source file — a few lines, restoring the old semantics — or (b) resolve directory-style targets against the nav's page set (tools/ → tools.md/tools/index.md) and rewrite them like .md links. Option (a) matches the old pipeline's behavior.
Distinct from the existing comments on this validator, and why nit. The posted line-50 comment covers 5+-char extensions (its fix — widening the quantifier — does not cover extensionless targets, which have no extension to match); line 151 covers containment escapes of targets that do match; line 165 covers mailto/fence false positives. Those targets all enter a scanner; extensionless targets enter neither, and this finding additionally covers the lost absolute_links guard. Severity nit: grep confirms zero extensionless internal links under docs/ today (consistent with the PR's byte-identical verification), so nothing breaks at merge — the cost lands on a future contributor as a silent broken link, worth patching while the file is fresh.
| uv sync --frozen --group docs | ||
| uv run --frozen --no-sync python scripts/docs/build_config.py | ||
| uv run --frozen --no-sync zensical build -f mkdocs.gen.yml --strict |
There was a problem hiding this comment.
🟡 Zensical 0.0.48's --strict only counts its own diagnostics — ERROR-level log records from the mkdocstrings compatibility layer (e.g. Couldn't load inventory ... objects.inv) never fail the build, so a transient network failure or upstream outage during a deploy now publishes a site with every inventory-driven external cross-reference silently unlinked, where MkDocs strict mode aborted loudly (verified empirically: all three inventory fetches 403'd, zensical printed the errors, then 'No issues found' and exited 0). Capturing zensical's output and failing on the mkdocstrings error signature — mirroring the crossref grep this script already does — would restore the lost guarantee.
Extended reasoning...
What the bug is. mkdocs.yml configures three mkdocstrings inventories (docs.python.org, docs.pydantic.dev, typing-extensions objects.inv) plus signature_crossrefs: true and show_signature_annotations: true, precisely so stdlib/pydantic/typing-extensions types in every API signature link to their upstream docs. When an inventory URL is unreachable, mkdocstrings 1.0.4 swallows the failure with a bare log record (_internal/handlers/base.py:645-648: except Exception: _logger.error("Couldn't load inventory %s ...")) — no exception propagates. Under the old pipeline that was still fatal: MkDocs 1.6.1 strict mode attaches a CountHandler for every log record ≥ WARNING and raises Abort('Aborted with N warnings in strict mode!') when the count is nonzero. Zensical 0.0.48 has no such mechanism — the package contains no Python logging capture at all, and --strict is passed into the Rust runtime, which counts only its own diagnostics. A Python log record from the compat layer structurally cannot fail the build.
Empirically reproduced end-to-end in this review environment (which happens to 403-block the inventory hosts — exactly the claimed trigger): a cold zensical build -f mkdocs.gen.yml --strict printed three mkdocstrings: Couldn't load inventory https://... objects.inv through handler 'python': HTTP Error 403: Forbidden lines, then No issues found, and exited 0. Contrast: a hard exception does fail (a ::: mcp.totally_nonexistent_module stub raises and zensical exits 1) — the gap is specifically the logged warning/error class that MkDocs strict used to promote.
Why none of this PR's sidecar guards catches it — verified on the degraded build:
build_config.pypasses (it runs before zensical and validates nav entries, not rendered output).- The
grep -F '[<code>'crossref guard finds zero matches: inventory-driven signature/annotation crossrefs degrade to plain unlinked text spans, never literal bracket text. llms_txt.pypasses (it walks source markdown, never rendered HTML).- Net:
bash scripts/docs/build.shruns green end to end. The degraded site contains exactly 2docs.python.orglinks — both hand-written prose URLs — while every inventory-driven type link across the entire API reference is silently gone.
Realistic trigger. deploy-docs.yml builds in fresh worktrees with no persisted zensical cache, so every deploy performs three live inventory fetches. A transient CI egress failure, a docs.python.org/ReadTheDocs outage, or ReadTheDocs rate-limiting CI runner IPs (a known intermittent occurrence — and 403 is exactly what this sandbox got) used to fail the docs check or deploy loudly and was fixed by a re-run; now it silently publishes the degraded site, with the only signal an easily-missed log line inside a green job. The same swallow applies to every mkdocstrings/griffe WARNING/ERROR log record MkDocs strict used to promote (griffe docstring-parse warnings, deprecated-option warnings) — the inventory failure is just the instance with an external trigger and site-wide impact.
How to fix. Mirror the post-build grep pattern this script already uses for unresolved crossrefs: capture zensical's output and fail on the mkdocstrings error signature, e.g. zensical build -f mkdocs.gen.yml --strict 2>&1 | tee /tmp/zensical-build.log followed by if grep -E "Couldn't load inventory|mkdocstrings.*error" /tmp/zensical-build.log; then exit 1; fi (mind pipefail so zensical's own exit code still propagates). Alternatively, assert the built site contains at least one inventory-driven link per inventory host. If accepting degraded-but-green builds on network failure is a deliberate trade, documenting it alongside the PR's lost-guarantees inventory would at least make the narrowing visible.
Why nit rather than blocking. Nothing breaks at merge — CI has network today and the current build ships correct links. The regression is latent: it manifests only on a future environmental failure during a docs build, and the result is a degraded (recoverable) docs site rather than data loss or broken code. That said, it is the strongest of this PR's latent-guard findings, since it requires no contributor mistake — only a network event — and the fix is a few lines in this script. Distinct from the existing build.sh:35 comments: those concern the [<code> grep's precision/recall for autorefs bracket text; this is a different root cause (--strict ignoring embedded Python log records) and a different failure surface (missing external links produce no HTML marker at all).
3 participants
Migrates the v2 docs build from MkDocs Material to Zensical, the Material team's successor — the same move Starlette and uvicorn made. Original migration by @Kludex; taken over to fold in review findings and empirical validation.
Why this isn't a drop-in swap
Zensical natively re-implements
search,mkdocstringsandglightbox, but runs no arbitrary MkDocs plugins or hooks and its module/extension API hasn't shipped. This repo's docs relied on three plugin/hook-based pieces, which become standalone build steps aroundzensical build(the sidecar pattern):mkdocstrings,search,glightbox, emojimkdocs.yml(unchanged semantics)mkdocs-gen-files(theapi/tree)scripts/docs/gen_ref_pages.pymkdocs-literate-nav(theapi/nav)scripts/docs/build_config.py→mkdocs.gen.ymlhooks:(llms_txt.py)scripts/docs/llms_txt.py(post-build)socialcardsscripts/docs/build.shis the single owner of the whole recipe (dependency sync → generate config/API stubs →zensical build --strict→ llms.txt):shared.yml,docs-preview.yml(via the toolchain detection merged in #3081) andscripts/build-docs.shall call it.mkdocs.ymlstays the source config;mkdocs.gen.ymlanddocs/api/are gitignored build artifacts.Guarantees that did not survive the toolchain swap — and where they moved
Zensical builds some failure classes green that MkDocs strict-mode failed, so the sidecar scripts own them now (each verified by inducing the failure and observing the loud error):
--strict→build_config.pyvalidates every nav entry againstdocs/(external URLs exempt).[name][identifier]cross-reference → renders as literal bracket text, build stays green →build.shgreps the built site for the rendered signature and fails..mdtargets →llms_txt.pyvalidates relative asset links while walking the prose.llms_txt.pyfails if a generated package index is missing from its Optional section.Config-level
strict: trueis parsed but not honored by Zensical (verified: only the CLI flag aborts), so the--strictflag inbuild.shis the enforcement point; the oldvalidation:block is a severity filter that at best restates defaults, at worst masks issues, and stays removed. Broken-anchor detection is on by default in Zensical and verified working.The API package index is now deliberate instead of accidental
mkdocstrings renders a member re-exported across a package boundary (
mcp/__init__re-exportingmcp_types) as a full duplicate of its canonical docs when the target package happens to be loaded, and silently omits it when it isn't — soapi/mcp/had 123 headings under MkDocs (the ~50mcp_typesre-exports were just absent) and 255 under Zensical (all duplicated), from the same stub, purely by rendering order.gen_ref_pages.pypins the semantics: themcpindex renders each export as a compact one-line entry linking to its canonical page, inlining full docs only for exports documented nowhere else. This also surfacedClientResult, which was invisible on both old sites despite being in__all__. The declarative alternative (mkdocstrings'summaryoption) was tested in four variants and can't do this: it produces either an empty page or full duplicate bodies with same-page anchors, dropsClientResultin every variant, and inherits the same load-order accident.preload_modulesis emitted for the cross-package targets so warm incremental rebuilds can't hitAliasResolutionError(reproduced, fixed, re-verified).Visual parity and design
Zensical's modern UI keeps Material's
md-*DOM but ships different stock styling, sodocs/extra.csswas rebuilt against it (validated by measurement on the live preview, light and dark):navigation.prunelabels collapsed rows with the page title, which fell back to "Index"; browser tabs too).Verification
All exercised on real builds in a clean environment (plus the docs CI job on this branch):
bash scripts/docs/build.sh: strict build green with zensical 0.0.48; ~35s cold, ~3s warm incremental (a capability MkDocs didn't have). MkDocs baseline was ~33s.mcplogo icon, palette toggles,--8<--snippet includes, tabs and admonitions render correctly; search index atsite/search.json.KeyError).griffelibis griffe's successor distribution by the same author — not a typo — and is imported directly by the generator.Breaking changes
None for SDK users. For docs contributors:
mkdocs servebecomesscripts/serve-docs.sh; local one-shot builds arebash scripts/docs/build.sh.Follow-up (out of scope)
[v1.x]backport of the toolchain-detectingscripts/build-docs.sh, so a v1.x-triggered Pages deploy can still build main's half of the combined site. Until it lands, deploys triggered from v1.x pushes fail (main-triggered deploys, previews and CI are unaffected).--strict.AI Disclaimer