Version: 0.8.11
Rust: rushdown v0.18.0 (CommonMark 0.31.2 + GFM)
Python: 3.9+
Bindings: PyO3 0.29
A fast CommonMark + GFM Markdown parser and renderer for Python, powered by the rushdown Rust library.
- Architecture — Full architecture documentation
- Quick Reference — Python bindings quick reference
RuleMetadataexport fixed —RuleMetadatais now properly exported from themordantpackage (from mordant import RuleMetadataworks). Previously it was only returned bylint_rules()but not importable.LintConfig.from_dict— CLI--confignow works correctly;LintConfig.from_dict()parses.markdownlint.jsoninto a config object.- Stub file corrected —
__init__.pyihas accurate signatures for all public functions (parse,render_math,lint,fix,lint_many,fix_many) and correct class names (EmojiParserOptions,EmojiHtmlRendererOptionswithoutPyprefix). MarkdownChunkerAPI documented — Stub file now lists all actual methods (get_chunks,get_all_chunks,get_chunks_with_context,get_bare_chunks,compute_overlap_payloads) instead of the non-existentchunkmethod.
- Server-side Mermaid rendering — Mermaid diagrams now render as inline SVG via the
mermaid-rs-renderercrate (~3ms server-side vs ~2s client-side). No browser/CDN dependency. Three render modes:server(default, inline SVG),client(legacy, Mermaid.js ESM),hybrid(try server, fallback to client) - Render mode API —
DiagramHtmlRendererOptions(render_mode="server"|"client"|"hybrid", mermaid_url=...) - Customizable Mermaid themes — Mermaid color schemes derived from code-highlighting (syntect) themes.
DiagramHtmlRendererOptions(theme="Dracula")themes server-side SVG (viamermaid-rs-renderer) and client-side rendering (viamermaid.initialize+themeVariables). A singletheme=kwarg onmarkdown_to_htmlthemes both code and diagrams; native mermaid themes (modern/dark/forest/neutral) are also supported.
- Chunker GFM + Diagram parity —
MarkdownChunkernow uses the same parser extensions asparse()andmarkdown_to_html(): GFM tables (TableAstTransformer,TableParagraphTransformer) and Mermaid diagrams (DiagramAstTransformer) are correctly classified asBlockType::TableandBlockType::Diagramrespectively - Chunker Diagram block type —
BlockType::Diagramadded to the chunker's type system; mermaid code blocks yield as"Diagram"instead of"CodeBlock"or being silently dropped - Diagram source position fix —
DiagramAstTransformernow copies the original code block'spos()to the newDiagramnode so the chunker can slice raw source correctly
- Lint engine — 25 lint rules (MD001, MD003, MD009, MD010, MD012, MD013, MD018–MD022, MD024, MD025, MD026, MD031, MD032, MD034, MD040, MD042, MD045–MD048, MD049, MD050) with diagnostics, fix engine, and configuration
- Batch API —
lint_many()andfix_many()for parallel file processing viarayon, with GIL release for the entire batch - CLI —
python -m mordantwith--fix,--dry-run,--format(human/json/github),--config,--enable,--disable,--default-language, glob/directory recursion - Phase 8 accuracy polish — emoji text in heading comparison (MD024), frontmatter
title:support (MD025), fragment anchor validation for links (MD042) - Document chunking —
MarkdownChunkerlazy AST-based chunk iterator yielding bare chunks (no heading prefix), withget_chunks(),get_all_chunks(),get_chunks_with_context(),get_bare_chunks(),ExtractedChunk(withblock_type/start_offset/end_offset),get_delimiter(),compute_overlap_payloads() - Inline suppression —
<!-- markdownlint-disable MD001 -->comments supported - VSCode JSON theme support — Custom themes from
.jsonfiles viaadd_custom_theme()and user directory~/.mordant/themes/ - 1297 tests passing (up from 1161)
- Blazing fast. One of the fastest Markdown parsers for Python — up to 55x faster than python-markdown on large documents.
- Full AST access. Parse markdown to a
Documentwith complete tree traversal — navigate parent, children, siblings, access all node kinds. - CommonMark + GFM. Fully compliant with CommonMark 0.31.2 and GitHub Flavored Markdown (tables, task lists, strikethrough; autolink disabled by default, enable with
GfmOptions.all()). - YAML frontmatter. Extract metadata from YAML frontmatter with full type preservation (null, bool, int, float, str, list, dict).
- Multi-threaded. Parse and render release the GIL — scale ~4.0x linearly with thread count.
- Emoji support. 😂
:heart::smile:— shortcode-style emoji rendering with blacklist and custom templates. - Math support. LaTeX math via KaTeX — fenced
math/latex blocks, inline$...$/$$...$$math, standalonerender_math()function. - Mermaid diagrams.
graph LR,sequenceDiagram— render Mermaid diagrams from code blocks, server-side as inline SVG by default. Customizable color schemes derived from code-highlighting themes (a singletheme=kwarg themes both code and diagrams). - Footnotes. PHP Markdown Extra style footnotes (
[^1],[^hello]) with<sup>references,<div class="footnotes">endnotes, and backlinks. - Document chunking.
MarkdownChunker— lazy, low-copy AST-based chunk iterator with heading-context propagation,from_file()andfrom_file_mmap()constructors. - Extensible. Custom node types, parsers, transformers, and renderers via Rust extensions.
pip install mordantOr from source:
cd mordant-py
cargo build --release
pip install -e .import mordant
# Parse + render in one call
html = mordant.markdown_to_html("# Hello\n\n**World**")
# '<h1>Hello</h1>\n<p><strong>World</strong></p>\n'
# GFM support (tables, strikethrough, task lists enabled by default)
html = mordant.markdown_to_html("~~deleted~~")
# '<p><del>deleted</del></p>\n'
# Autolink (disabled by default; enable with GfmOptions.all())
html = mordant.markdown_to_html(
"https://example.com",
gfm_opts=mordant.GfmOptions.all()
)
# '<p><a href="https://example.com">https://example.com</a></p>\n'
# Full AST access
doc = mordant.parse("# Hello\n\n**World**")
print(doc.kind) # "Document"
print(doc.children) # [Heading, Paragraph]
print(doc.text) # "HelloWorld"
# Emoji support
html = mordant.markdown_to_html("I'm :joy: and :heart:")
# '<p>I'm 😀 and ❤️</p>\n'
# Emoji blacklist
opts = mordant.EmojiParserOptions(blacklist="joy")
html = mordant.markdown_to_html(":joy: :heart:", emoji_parse_opts=opts)
# ':joy:' passes through; :heart: renders as ❤️
# Math support
html = mordant.markdown_to_html("""```math
\\int_0^\\infty e^{-x^2} dx = \\frac{\\sqrt{\\pi}}{2}
```""")
# '<span class="katex katex-display">...</span>'
# Standalone math rendering
result = mordant.render_math(r"\alpha + \beta", display=True, output="both")
# Footnotes (always enabled)
html = mordant.markdown_to_html("Text[^1]\n\n[^1]: The footnote.")
# '<p>Text<sup id="fnref:1"><a href="#fn:1" class="footnote-ref">1</a></sup></p>\n<div class="footnotes" role="doc-endnotes">\n<hr>\n<ol><li id="fn:1">The footnote. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">↩︎</a></li></ol></div>'
# Custom footnote options
opts = mordant.FootnoteHtmlRendererOptions(
link_class="my-ref",
backlink_class="my-back",
backlink_html="↑ back",
)
html = mordant.markdown_to_html("Text[^1]", footnote_render_opts=opts)
# Mermaid diagrams
html = mordant.markdown_to_html("""```mermaid
graph LR
A --- B
```""")
# '<pre class="mermaid">\ngraph LR\n A --- B\n</pre>\n<script type="module">...'
# Mermaid with custom URL
opts = mordant.DiagramHtmlRendererOptions(mermaid_url="https://cdn.example.com/mermaid.mjs")
html = mordant.markdown_to_html("""```mermaid
graph TD
A --> B
```""", diagram_render_opts=opts)
# Themed Mermaid diagram — color scheme derived from a code-highlighting theme
opts = mordant.DiagramHtmlRendererOptions(render_mode="server", theme="Dracula")
html = mordant.markdown_to_html("""```mermaid
graph TD
A --> B
```""", diagram_render_opts=opts)
# Server-rendered SVG uses Dracula's palette (background #282a36, pink edges, ...)
# Single `theme=` kwarg themes BOTH code blocks and Mermaid diagrams
html = mordant.markdown_to_html(
"# Title\n```mermaid\ngraph LR\n A---B\n```\n```python\nx=1\n```",
theme="Dracula",
)
# Code block and diagram share Dracula's colors
# YAML frontmatter
md = """---
title: My Doc
author: Jane
tags: [rust, markdown]
---
Body
"""
doc = mordant.parse(md)
print(doc.metadata)
# {'title': 'My Doc', 'author': 'Jane', 'tags': ['rust', 'markdown']}
# Document chunking
chunker = mordant.MarkdownChunker("# Section\n\nPara one\n\n## Sub\n\nPara two")
for chunk in chunker:
print(chunk)
# Para one
# Para two
# (bare chunks — no heading prefix)
# get_chunks() returns ExtractedChunk with metadata
for chunk in chunker.get_chunks():
print(chunk.block_type, chunk.text, chunk.start_offset, chunk.end_offset)
# Paragraph Para one 9 17
# Paragraph Para two 27 35
# get_all_chunks() includes headings
for chunk in chunker.get_all_chunks():
print(chunk.block_type, chunk.text)
# Heading # Section
# Paragraph Para one
# Heading ## Sub
# Paragraph Para two
# get_chunks_with_context() adds heading prefix
for chunk in chunker.get_chunks_with_context():
print(chunk.text)
# # Section\n\nPara one
# ## Sub\n\nPara two
# compute_overlap_payloads() for embedding
payloads = chunker.compute_overlap_payloads(2)
# [{"chunk:0": "Para one"}, {"chunk:1": "one\n\nPara two"}]Split a document into bare chunks — each chunk is the raw block content with no heading prefix. OKF injects heading context at embed time for better embeddings. Headings update a current_header context; thematic breaks and other non-body nodes are skipped without resetting context.
import mordant
# Basic chunking — bare chunks (no heading prefix)
chunker = mordant.MarkdownChunker("# Section\n\nPara one\n\n## Sub\n\nPara two")
chunks = list(chunker)
assert len(chunks) == 2
assert chunks[0] == "Para one" # bare, no heading prefix
assert chunks[1] == "Para two" # bare, no heading prefix
# current_header still tracks the last heading seen
assert chunker.current_header == "## Sub"
# get_chunks() returns ExtractedChunk with metadata
for chunk in chunker.get_chunks():
print(chunk.block_type, chunk.text, chunk.start_offset, chunk.end_offset)
# Paragraph Para one 9 17
# Paragraph Para two 27 35
# get_all_chunks() includes headings as separate chunks
for chunk in chunker.get_all_chunks():
print(chunk.block_type, chunk.text)
# Heading # Section
# Paragraph Para one
# Heading ## Sub
# Paragraph Para two
# get_chunks_with_context() adds heading prefix for display
for chunk in chunker.get_chunks_with_context():
print(chunk.text)
# # Section\n\nPara one
# ## Sub\n\nPara two
# get_delimiter() for document reconstruction
mordant.MarkdownChunker.get_delimiter("List", "List") # "\n"
mordant.MarkdownChunker.get_delimiter("Blockquote", "Blockquote") # "\n> "
mordant.MarkdownChunker.get_delimiter("Paragraph", "CodeBlock") # "\n\n"
# compute_overlap_payloads() for embedding context continuity
chunker = mordant.MarkdownChunker("# Title\n\nFirst para second para third para.\n\n## Sub\n\nMore text here.")
payloads = chunker.compute_overlap_payloads(2)
# [{"chunk:0": "First para second para third para."},
# {"chunk:1": "third para.\n\nMore text here."}]
# from_file reads from disk
chunker = mordant.MarkdownChunker.from_file("/path/to/doc.md")
for chunk in chunker:
print(chunk) # bare chunks, no heading prefix
# from_file_mmap for zero-copy large files
chunker = mordant.MarkdownChunker.from_file_mmap("/path/to/large.md")
# Nested headings inside blockquotes never leak as context
chunker = mordant.MarkdownChunker("# Outer\n\n> # Nested\n\n> Quote text.")
chunks = list(chunker)
# current_header is "# Outer" (not "# Nested" which is nested)
assert chunker.current_header == "# Outer"See QUICKREF.md for full API reference.
doc = mordant.parse("# Title\n\n**Bold** and *italic*")
# Navigate tree
heading = doc.children[0]
print(heading.level) # 1
print(heading.text) # "Title"
# Walk all nodes
for node in doc.walk("depth"):
print(f"{node.kind}: {node.text}")
# Find by kind
links = [n for n in doc.walk("depth") if n.kind == "Link"]# Parse options
parse_opts = mordant.ParseOptions(
attributes=False,
auto_heading_ids=False,
escaped_space=False,
meta_table=False,
)
# Render options
render_opts = mordant.RenderOptions(
hard_wraps=False,
xhtml=False,
allows_unsafe=False,
escaped_space=False,
)
# GFM options (default: tables + strikethrough + task lists; linkify disabled)
import mordant
gfm_opts = mordant.GfmOptions()
# Enable all features including linkify
gfm_opts = mordant.GfmOptions.all()
# Granular feature selection
gfm_opts = mordant.GfmOptions(features=[
mordant.GfmFeature.Table,
mordant.GfmFeature.Strikethrough,
])
html = mordant.markdown_to_html(
"Hello\nWorld",
gfm_opts=gfm_opts,
parse_opts=parse_opts,
render_opts=render_opts,
)from concurrent.futures import ThreadPoolExecutor
import mordant
# GIL is released during parse + render — safe for concurrent use
with ThreadPoolExecutor(max_workers=4) as pool:
results = list(pool.map(mordant.markdown_to_html, markdown_docs))
# ~4.0x linear scaling vs single-threaded| Fixture | mordant | mistune | markdown-it-py | python-markdown |
|---|---|---|---|---|
| Small (400B) | 0.039ms | 0.430ms | 0.475ms | 2.301ms |
| Medium (5.4KB) | 0.155ms | 2.448ms | 3.940ms | 6.455ms |
| Large (26.7KB) | 0.410ms | 8.611ms | 16.743ms | 31.304ms |
| Data (202KB) | 2.763ms | 38.152ms | 65.736ms | 621.295ms |
| Library | 1-thread | 4-threads | Scaling |
|---|---|---|---|
| mordant | ~1,000 docs/s | ~4,000 docs/s | 4.0x |
| python-markdown | ~59 docs/s | ~257 docs/s | 4.35x |
| mistune | ~133 docs/s | ~542 docs/s | 4.07x |
| markdown-it-py | ~83 docs/s | ~337 docs/s | 4.06x |
| Kind | Type | Example |
|---|---|---|
| Document | block | Root node |
| Paragraph | block | Hello world |
| Heading | block | # Title |
| ThematicBreak | block | --- |
| CodeBlock | block | ```python ... ``` |
| Blockquote | block | > quoted |
| List | block | - item |
| ListItem | block | - [x] done |
| HtmlBlock | block | <div>...</div> |
| Text | inline | Plain text |
| CodeSpan | inline | `code` |
| Emphasis | inline | *italic* |
| Strong | inline | **bold** |
| Link | inline | [text](url) |
| Image | inline |  |
| RawHtml | inline | <span> |
| LinkReferenceDefinition | block | [ref]: url |
| Table | block | ` |
| TableHeader | block | Header row |
| TableBody | block | Body rows |
| TableRow | block | <tr> |
| TableCell | block | <td> |
| Strikethrough | inline | ~~text~~ |
| Diagram | block | ```mermaid ... ``` |
| FootnoteReference | inline | [^1], [^hello] |
| FootnoteDefinition | block | [^1]:, [^hello]: |
| Extension | any | Custom nodes |
The meta parser uses lookahead to distinguish --- (thematic break) from frontmatter:
# Thematic break
mordant.parse("---").metadata == {}
# Frontmatter
mordant.parse("---\ntitle: Test\n---").metadata["title"] == "Test"
# Five dashes is thematic break
mordant.parse("-----").metadata == {}import mordant
try:
doc = mordant.parse("---\ninvalid: yaml: [broken")
doc.metadata # Raises ValueError on access
except ValueError as e:
print(e) # YAML parsing error messageMordant wraps the rushdown Rust library (CommonMark 0.31.2 + GFM) via PyO3 bindings:
- Rust core: rushdown v0.18.0 — arena-allocated AST, priority-based parser dispatch, HTML renderer
- Python bindings: PyO3 0.29 —
Document,Node,Walkerclasses with sharedRc<RefCell<Arena>>andRc<str>source memory model (refcount bump on node creation instead of deep source copy) - GIL release: Parse and render release the GIL via
Python::detach()for multi-threaded parallelism - Frontmatter: YAML parsing via
yaml-pegwith thematic break conflict resolution
YAML frontmatter support is provided by rushdown-meta, which has been directly incorporated into mordant. The original rushdown-meta crate is available in extensions/rushdown-meta-main/.
Key features of the integrated meta parser:
- Thematic break conflict resolution:
---alone is a thematic break;---\n+ YAML-like content is frontmatter - Full YAML subset: null, bool, int, float, str, list, dict (via
yaml-peg) - AST table rendering: Optional
meta_tableoption renders metadata as an HTML table in the AST - Error handling: YAML parse errors are inserted as HTML comments in the AST; Python raises
ValueErrorondoc.metadataaccess
See ARCHITECTURE.md §6 for full details.
Emoji shortcode support (:joy:, :heart:, :smile:, etc.) is provided by rushdown-emoji, which has been directly incorporated into mordant. The original rushdown-emoji crate is available in extensions/rushdown-emoji-main/.
Key features of the integrated emoji extension:
- Shortcode parsing:
:joy:→ 😀,:heart:→ ❤️, 1,500+ emojis from theemojiscrate (v0.8.0) - Blacklist support:
EmojiParserOptions(blacklist="joy,heart")— blacklisted shortcodes pass through as literal text - Custom HTML templates:
EmojiHtmlRendererOptions(template='<img src="{shortcode}.png" />')— render emojis as<img>tags or any custom format - Template placeholders:
{emoji}(Unicode char),{shortcode}(e.g."joy"),{name}(e.g."grinning face with smiling eyes") - Code span protection: Emojis inside
`code`are not parsed —:joy:stays literal in code spans - AST node access: Emoji nodes expose
emoji,shortcode, andnameproperties via theExtensionnode kind - Error handling: Unknown shortcodes pass through as-is (
:invalid:→:invalid:)
See ARCHITECTURE.md §7.10 for full details.
Diagram support is provided by rushdown-diagram, which has been directly incorporated into mordant. The original rushdown-diagram crate is available in extensions/rushdown-diagram-main/.
rushdown-diagram supports two diagram formats:
- MermaidJS — client-side rendering via the Mermaid.js ESM module
- PlantUML — server-side rendering (requires a
plantumlcommand)
Mordant currently implements Mermaid support only. Key features:
- Code block detection:
```mermaidcode blocks are automatically detected and converted to diagram nodes via an AST transformer - Client-side rendering: Diagrams render as
<pre class="mermaid">with automatic Mermaid.js ESM script injection (single script tag for all diagrams) - Custom Mermaid URL:
DiagramHtmlRendererOptions(mermaid_url="https://cdn.example.com/mermaid.mjs")— use a custom Mermaid.js CDN or local file - Customizable themes:
DiagramHtmlRendererOptions(theme="<name>")derives Mermaid colors from a code-highlighting (syntect) theme — server-side SVG viarender_with_options, client-side viamermaid.initialize+themeVariables. Built-in mermaid themes (modern/dark/forest/neutral) are used natively. A singletheme=kwarg onmarkdown_to_htmlthemes both code and diagrams; explicit per-param args override it. - Parser options:
DiagramParserOptions(mermaid_enabled=False)— disable diagram transformation to keep code blocks as regular fenced code blocks - AST node access: Diagram nodes expose
diagram_type("mermaid") anddiagram_value(source content) properties via theDiagramnode kind - Multiple diagrams: Multiple Mermaid blocks in one document all render correctly with a single script tag
- GFM compatible: Works alongside other GFM features (tables, task lists, strikethrough; autolink disabled by default, enable with
GfmOptions.all()) - Frontmatter compatible: Works alongside YAML frontmatter
See ARCHITECTURE.md for full details.
Math support is provided by the pure-Rust katex-rs crate, incorporated directly into mordant.
Key features:
- Fenced math blocks:
```mathand```latexcode blocks render to KaTeX markup - Inline math:
$...$for inline,$$...$$for display mode - Standalone
render_math():mordant.render_math(r"\alpha + \beta", display=True, output="both")— renders LaTeX independently of the Markdown AST - Output formats:
"both"(HTML+MathML, default),"html", or"mathml" - Error handling: Invalid LaTeX produces an error span (
<span class="katex-error">...) instead of crashing - Caching: Rendered markup is memoized on
(display, output, latex)for repeated formulas - GIL released: Math rendering runs with the GIL released for multi-threaded parallelism
See ARCHITECTURE.md §7.12 for full details.
Footnote support is provided by rushdown-footnote, which has been directly incorporated into mordant. Footnotes are always enabled — no parser options to disable them.
Syntax (PHP Markdown Extra):
Text with a footnote.[^1]
Text with a named footnote.[^hello]
[^1]: The footnote.
[^hello]: The named footnote.Output:
<p>Text with a footnote.<sup id="fnref:1"><a href="#fn:1" class="footnote-ref">1</a></sup></p>
<div class="footnotes" role="doc-endnotes">
<hr>
<ol>
<li id="fn:1">The footnote. <a href="#fnref:1" class="footnote-backref" role="doc-backlink">↩︎</a></li>
</ol>
</div>Key features:
- Inline references:
[^1],[^hello]— rendered as<sup><a href="#fn:N">N</a></sup> - Block definitions:
[^1]:followed by content — rendered in<div class="footnotes">at end of document - Named footnotes:
[^hello]— label preserved in ID - Multiple refs: Multiple
[^1]to same[^1]:— each gets a superscript ref, definition rendered once - Backlinks: Each definition has a backlink anchor (
↩︎) to return to the reference - Accessibility:
role="doc-endnotes",role="doc-noteref",role="doc-backlink"ARIA attributes - Custom options:
FootnoteHtmlRendererOptionsfor custom CSS classes, backlink HTML, and ID prefixes - AST node access:
node.footnote_label,node.footnote_index,node.footnote_referencesproperties - No parser options: Footnotes are always enabled (matches math extension pattern)
See ARCHITECTURE.md §7.14 for full details.
Run benchmarks:
cd mordant-py
python benchmarks.py # All fixtures, 50 iterations
python benchmarks.py -f medium -n 100 # Specific fixture, custom count
python benchmarks.py -o results.json # Save JSONcd mordant-py
python -m pytest tests/ -v1233 Python tests passing (Core, AST, GFM, Options, YAML Frontmatter, Emoji, Mermaid Diagrams, Math, Lint engine, CLI, batch API, Phase 8 accuracy, VSCode theme, Chunker, OKF chunker methods, Extracted Chunk, Mixed Features) + 64 Rust tests (Unit tests, AST, CommonMark spec, Extensions, GFM, Options, Doc-tests).
Themes are loaded from multiple sources:
- Embedded themes — Bundled in
mordant/themes/, loaded at import time - User themes — Place
.jsonor.tmThemefiles in~/.mordant/themes/(or%APPDATA%/mordant/themes/on Windows) for auto-loading - Built-in themes — Loaded from
syntect-assets(bat's updated themes) - Custom themes — Use
add_custom_theme(name, content)to register themes from JSON or XML content
Both VSCode JSON and Sublime .tmTheme formats are supported. VSCode JSON themes are automatically converted to the syntect format via the parse_vscode_theme_jsonc → vscode_theme_to_syntect pipeline, allowing you to use any VSCode theme file directly.
See QUICKREF.md for details.
MIT
Mordant: Python bindings by opticsWolf Rushdown: Rust core by Yusuke Inuzuka