Evaluate AI agents with Unix-style pipeline commands. Schema-driven adapters for any CLI agent, trajectory capture, pass@k metrics, and multi-run comparison.

Codebase for Orthogonal Diverse Diffusion. We present a lightweight, training free method for improving sampling diversity and Pass@k in Diffusion Language Models.

ckl's personal benchmark for doc writing, infra code, and paper reading — one-click evaluation of the latest models via TUI coding CLIs with an interactive dashboard.

Offline-first Python framework for prompt-based program synthesis, test-based evaluation, and iterative repair

Evaluate LLMs the right way — confidence intervals, unbiased pass@k, significance testing, bias-controlled LLM-as-judge, contamination checks. A drop-in agent skill with a numpy stats core validated against ground truth.

The pass@k crossover in RLVR is a theorem, not a finding: pass@k is the Laplace transform of difficulty, so sharpening forces base and RL curves to cross exactly once (Karlin variation-diminishing), with a closed-form pivot and a one-bit certificate for the 'does RL expand reasoning?' debate.

Deterministic reliability-floor metrics for long-horizon agents — pass^k, reliability decay curve, meltdown onset. Zero dependencies.

Evaluation harness for coding agents - repeated runs, pass@k, bootstrap/McNemar significance, tamper detection, cost. Single vs Planner-Coder-Reviewer multi-agent on CLOVA.

Panel-of-experts scaffold vs <think> on Qwen3-30B-A3B — diversity finding + olympiad RLVR follow-up

Local-first reproducible LLM benchmark suite: 87 tasks, 16 scoring modes, multi-backend discovery, result diffs, and arena ELO artifacts.

A coding agent that doesn't give up: writes code, runs it, reads the traceback, and retries with the error as feedback until tests pass. Includes a benchmark proving the feedback loop (91%) beats blind retry/pass@k (67%) and one-shot (25%). Stdlib-only core; works with any OpenAI-compatible model.