You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+48-7Lines changed: 48 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -31,13 +31,54 @@ Modern `nogil` Python bindings for the PCRE2 library with `stdlib.re` API compat
31
31
32
32
## Why PyPcre ⚡
33
33
34
-
PyPcre is a modern PCRE2 binding designed to be both fast and thread-safe in a `GIL=0` world. In the era of the global interpreter lock, Python had real threads but often only limited concurrency, aside from a handful of low-level APIs and packages. As Python moves toward a fuller `GIL=0` design, true multi-threaded concurrency becomes practical and brings Python closer to parity with other modern languages.
35
-
36
-
Many Python regular expression packages either segfault under `GIL=0` or suffer suboptimal performance because they were not designed with threaded execution in mind.
37
-
38
-
PyPcre is fully CI-tested. Every API and PCRE2 flag is exercised in a continuous development environment backed by the ModelCloud.AI team. Fuzz (clobber) tests are also run to catch memory safety, accuracy, and memory leak regressions.
39
-
40
-
For safety, PyPcre preferentially links against the OS-provided `libpcre2` package so it can benefit from upstream security patches. You can force a full source build with the `PYPCRE_BUILD_FROM_SOURCE=1` environment variable.
34
+
PyPcre gives Python a familiar `re`-shaped API on top of the real `PCRE2` engine. That means you keep the ergonomics of the standard library while unlocking a far more capable regex engine, optional JIT, explicit threading support, and a binding that is designed and tested for free-threaded Python. 🧠⚡
35
+
36
+
### Big Wins 🏆
37
+
38
+
- 🧬 **Full power of PCRE2**: this is the actual `PCRE2` engine, not a look-alike. You get its native compile options, semantics, JIT, and upstream tuning.
39
+
- 🔥 **Much more powerful regex syntax**: `PCRE2` supports advanced constructs that go beyond stdlib `re`, including atomic groups `(?>...)`, possessive quantifiers `++`, branch-reset groups `(?|...)`, richer lookarounds, and backtracking control verbs like `(*SKIP)(*FAIL)`.
40
+
- 🧵 **Thread-safe all the way into `nogil`**: PyPcre is built for `PYTHON_GIL=0`, with CI coverage, lock-aware caches, reusable match/JIT resources, and `parallel_map()` for multi-subject fan-out.
41
+
- ⚡ **Fast on real workloads**: `PCRE2` JIT plus cached compiled patterns can make PyPcre as fast as, or faster than, `re` and `regex` on many common scans, especially multiline searches, lookaround-heavy patterns, and thread-heavy execution.
42
+
- 🛡️ **Operationally safer**: PyPcre prefers the system `libpcre2-8` shared library so normal OS package updates can bring security and bug-fix benefits without a bundled fork.
43
+
- ✅ **Validated hard**: this project runs API tests, fuzz tests, memory-safety checks, local `valgrind` leak checks, and `massif` heap profiles. Recent local profiling found `0` definite leaks and `0` possible leaks in both the public API and raw binding paths.
44
+
45
+
### Quick Comparison 🥊
46
+
47
+
| Area | PyPcre |`stdlib.re`|`regex`|
48
+
| --- | --- | --- | --- |
49
+
| Engine | Full `PCRE2` ✅ | CPython stdlib engine | Separate engine, not `PCRE2`|
50
+
|`PCRE2` syntax and flags | Full access ✅ | No | No |
51
+
| Advanced syntax surface | Very rich ✅ | More limited | Rich, but different from `PCRE2`|
52
+
| JIT execution |`PCRE2` JIT ✅ | No | No |
53
+
|`re`-shaped API | Yes ✅ | Native | Similar, but not the main goal |
54
+
| Free-threaded `PYTHON_GIL=0` focus in this project | Yes ✅ | No PyPcre-style threading layer | Not a project focus here |
55
+
| Built-in threaded subject fan-out |`parallel_map()` ✅ | No | No |
These tables are intentionally selective: they show representative local workloads where PyPcre was fastest or effectively tied. Environment: `Python 3.14.3` free-threaded build on x86_64 Linux, compiled-pattern reuse, best-of-5 timing, lower is better.
| First `ERROR` line in a multiline log buffer |`8`|`25.34 ms`|`38.83 ms`|`40.34 ms`|`1.53x` vs `re`, `1.59x` vs `regex`|
78
+
| Extract only `WARN` / `ERROR` lines |`8`|`28.58 ms`|`65.54 ms`|`73.55 ms`|`2.29x` vs `re`, `2.57x` vs `regex`|
79
+
| Per-line full-name extraction |`8`|`31.68 ms`|`123.44 ms`|`164.80 ms`|`3.90x` vs `re`, `5.20x` vs `regex`|
80
+
81
+
PyPcre is not trying to pretend stdlib `re` is bad. `re` is excellent. The point is different: PyPcre lets you keep that familiar Python shape while stepping into the `PCRE2` world with more syntax, more engine power, more explicit threading support, and many real workloads where it is already extremely competitive or outright faster. 🚀
0 commit comments