lucid-url

A dependency-free WHATWG URL parser written in Rust, with a Rust adaptation of Ada's URL interface.

The implementation uses only Rust's standard library. Ada is not linked, vendored, or used by the parser; its current public interface is the compatibility target and its parser is used only by the optional comparison benchmark.

Usage

use lucid_url::{UrlAggregator, parse};

let url = parse::<UrlAggregator>(
    "https://user:pass@example.com:8080/path?query=yes#fragment",
    None,
)?;

assert_eq!(url.get_protocol(), "https:");
assert_eq!(url.get_hostname(), "example.com");
assert_eq!(url.get_pathname(), "/path");

Parse into Url when the Ada-style owned getter surface is useful:

use lucid_url::{Url, parse};

let mut url = parse::<Url>("https://example.com/", None)?;
url.set_hostname("example.org");
url.set_pathname("/account");

assert_eq!(url.get_href(), "https://example.org/account");

Relative URLs take a parsed base of the same representation:

use lucid_url::{UrlAggregator, parse};

let base = parse::<UrlAggregator>("https://example.com/a/b", None)?;
let url = parse::<UrlAggregator>("../c", Some(&base))?;

assert_eq!(url.get_href(), "https://example.com/c");

Interface

The parser mirrors Ada's URL-facing concepts:

• parse::<UrlAggregator>(input, base) and parse::<Url>(input, base)
• can_parse(input, base)
• UrlAggregator, backed by one serialized buffer and component offsets
• Url, with Ada-compatible owned and borrowed getter return types
• component getters, setters, presence checks, get_origin, and get_components
• href_from_file, set_max_input_length, and get_max_input_length

UrlSearchParams and UrlPattern are not part of the initial parser crate.

Conformance

The repository includes Ada's parser fixtures at a pinned test revision. A fresh checkout runs them through both Url and UrlAggregator as part of ordinary cargo test; missing or malformed fixtures are hard failures.

• URL parsing, serialization, getters, and can_parse: 919 cases
• URL setters: 296 cases
• IdnaTestV2 through the host parser: 2670 representable cases
• Ada ToASCII success cases: 68 cases
• percent encoding: 7 cases
• DNS/domain-length validation: 17 cases

Run the mandatory suite directly with:

cargo test --test ada_conformance --locked

The fixtures can be refreshed to another Ada revision with ./scripts/update-ada-fixtures.sh <commit>. A daily workflow also runs the suite against Ada's current main and fails when upstream parser tests change. The exact coverage and justified exclusions are documented in tests/ADA_TEST_SCOPE.md.

The parser has direct paths for validated canonical URLs and common normalization work, including percent encoding, path normalization, IPv4, IPv6, and IDNA. Inputs outside those conservative paths fall through to the complete WHATWG state machine.

Benchmarks

Run the default, pinned comparison against Ada:

./benchmarks/run.sh

This command always fetches and builds both implementations, then runs them in the same Google Benchmark executable over the same corpus. It prints the operating system, compiler versions, Ada options, Ada revision, and dataset revision before reporting results. A missing compiler or failed Ada build is a hard failure rather than a Lucid-only fallback.

The comparison fetches Ada's 100,025-URL dataset at commit 9749b92c13e970e70409948fa862461191504ccc. Ada is pinned to commit 0a371d6b82c282948597d80f63e856862c8ce667.

Both parsers are built with peak local throughput defaults:

• Ada: Release, ADA_USE_SIMDUTF=OFF, -O3, native CPU, amalgamated into the harness TU (not linked as libada.a)
• Lucid: opt-level=3, codegen-units=1, target-cpu=native
• LTO matched on both sides: off on macOS (Apple Clang linker abort), on for Linux. See the note in Cargo.toml.

ADA_USE_SIMDUTF=ON ./benchmarks/run.sh
ADA_ENABLE_LTO=OFF ./benchmarks/run.sh

Override Ada/C++ flags wholesale with ADA_CXX_FLAGS (space-separated).

For Lucid-only development regressions, without making a comparison claim:

cargo bench --bench parse

These microbenchmarks exercise focused canonical, normalization, Unicode, IDNA, and long-input paths. They are regression aids rather than published Ada comparisons. Each section prints the dataset it runs (# source, # urls=… bytes=…, and the URL list). Cap listing with LUCID_URL_BENCH_DATASET_PRINT=N. The large real-world corpus is Ada's url-dataset out.txt; mixed top sites match Ada's default url_examples_default from benchmarks/bench.cpp.

Official Ada benchmark protocol

The default benchmark uses the Google Benchmark protocol from the Ada v4 release benchmark. It runs the official parse-plus-href and can_parse operations and reports the mean of five repetitions. It also refuses to run if the parsers disagree about which corpus inputs are valid or how any accepted input is serialized.

Results from the improved ./benchmarks/compare-ada.sh harness on an Apple M5 Max running macOS 26.5 (matched no-LTO builds, Ada amalgamated into the harness, ADA_USE_SIMDUTF=OFF, Ada 0a371d6b82c282948597d80f63e856862c8ce667). Prefer ./benchmarks/run.sh for the single-process Google Benchmark protocol.

Microbenchmarks

Workload	Lucid UrlAggregator	Lucid Url	Ada url_aggregator	Ada url
Canonical ASCII	48.50	66.94	70.26	86.91
Normalization-heavy	92.70	110.06	156.03	147.73
Unicode and IDNA	730.70	751.74	397.01	407.75
Long canonical scans	54.36	73.57	66.63	95.14

Values are ns/URL. Microbenchmark numbers above are from the previous full run; re-run ./benchmarks/compare-ada.sh after harness changes if you need a matched micro set.

Real-world corpora

Corpus	Operation	Lucid ns/URL	Ada ns/URL	Lucid speedup
Clean HTTP (24)	can_parse	6.64	10.15	1.53×
Benchdata (100,025)	UrlAggregator	52.04	51.98	1.00×
Benchdata (100,025)	Url	113.29	79.21	0.70×
Benchdata (100,025)	can_parse	10.35	12.33	1.19×

Corpora are split on purpose:

• Mixed top sites measure parse+href only (not shown above; the 11-URL set is too small for stable headline numbers and includes IPv4/IPv6/non-special paths). They remain in the harness for local smoke checks.
• Clean HTTP measures can_parse on already-canonical special URLs that both fast paths target. A mean over mixed top sites previously overstated the can_parse gap because three slow-path URLs dominated eleven samples.
• Benchdata is the large 100k corpus for published parse and can_parse claims.

The Url path materializes an owned href (matching Ada's get_href() protocol) so the compiler cannot replace it with a length lookup. Ada micro/real-world harnesses materialize inputs on the heap and opaque the pointer/length so the amalgamated TU cannot constant-fold known string literals.

Release artifact size

For a native-code comparison, both libraries were built at optimization level 3 without LTO so the Apple Mach-O size tool could inspect their complete objects:

Release object	Lucid	Ada
Native object sections	462.1 KiB	318.4 KiB
Machine-code __text	92.3 KiB	226.9 KiB

Lucid's complete native object is 45% larger, while its machine code is 59% smaller. Most of Lucid's remaining footprint is its dependency-free Unicode and IDNA data. Raw compiler archives are not comparable: Rust's .rlib also contains 1.52 MiB of compiler metadata and LLVM input used for downstream generic compilation and LTO, while Ada's .a is a conventional native archive.

License

MIT