Libra

Benchmarking framework for Clojure

Installation

Core

Leiningen/Boot:

[net.totakke/libra "0.1.1"]

Clojure CLI:

net.totakke/libra {:mvn/version "0.1.1"}

Tools

Leiningen plugin:

:plugins [[net.totakke/lein-libra "0.1.2"]]

Boot task:

[net.totakke/boot-libra "0.1.0" :scope "test"]

CLI runner:

net.totakke/libra-runner {:git/url "https://github.com/totakke/libra"
                          :sha "6d2da78300438cb212b58d17675fc9e5d405ad49"
                          :deps/root "libra-runner"}

Getting started

Libra provides clojure.test-like functions and macros for benchmarking. For example, defbench defines a benchmark and run-benches measures defined benchmarks in the namespace.

(require '[libra.bench :refer :all])

(defn slow-inc [n]
  (Thread/sleep 10)
  (inc n))

(defbench slow-inc-bench
  (is (dur 10 (slow-inc 100))))

(run-benches)
;; Measuring user
;;
;; slow-inc-bench (:xx)
;;
;;   time: 11.725818 ms, sd: 1.073600 ms
;;=> nil

Basics

Basic usage is writing benchmarks in a separate directory (e.g. bench) from src and running them with command-line. See example project and try running benchmark.

The project consists of the following files.

example/
├── project.clj or build.boot or deps.edn
├── src/
│   └── example/
│       └── core.clj
└── bench/
    └── example/
        └── core_bench.clj

Locate your awesome codes in src/example/core.clj as usual, and write benchmarking programs in bench/example/core_bench.clj.

(ns example.core-bench
  (:require [libra.bench :refer :all]
            [example.core :refer :all]))

(defbench primes-with-trial-div-bench
  (is (dur 10 (doall (primes-with-trial-div 100000)))))

(defbench primes-with-eratosthenes-bench
  (is (dur 10 (doall (primes-with-eratosthenes 100000)))))

With Leiningen

To run the benchmark with Leiningen,

$ lein libra

lein-libra looks benchmark files in bench directory by default. You can change this by placing the following in project.clj:

:libra {:bench-paths ["path/to/bench"]}

You can supply predicates to determine whether to run a benchmark or not, which takes defbench metadata as argument:

:libra {:bench-selectors {:default (complement :slow)
                          :slow :slow}}

With Boot

To run the benchmark with Boot,

$ lein benchmarking libra

boot-libra provides libra task. Benchmark directory needs to be included in the classpath, so that you should add a profile task for benchmarking:

(require '[libra.boot :refer [libra]])

(deftask benchmarking []
  (set-env! :source-paths #(conj % "bench"))
  identity)

With Clojure CLI

Include a dependency on libra-runner in deps.edn.

:aliases {:libra {:extra-paths ["bench"]
                  :extra-deps {net.totakke/libra-runner {:git/url "https://github.com/totakke/libra"
                                                         :sha "<git commit sha>"
                                                         :deps/root "libra-runner"}}
                  :main-opts ["-m" "libra.runner"]}}

Then, invoke libra alias with CLI.

$ clj -Alibra

You may supply additional options:

-d, --dir DIR           Name of the directory containing benchmarks, default "bench".
-n, --namespace SYMBOL  Symbol indicating a specific namespace to run benchmarks.
-i, --include KEYWORD   Run only benchmarks that have this metadata keyword.
-e, --exclude KEYWORD   Exclude benchmarks with this metadata keyword.

Criterium integration

Libra can be used with a famous benchmarking library, Criterium. libra.criterium provides wrapper macros of Criterium.

(require '[libra.criterium :as c])

(defbench primes-with-eratosthenes-bench
  (is (c/quick-bench (doall (primes-with-eratosthenes 100000)))))

License

Copyright © 2017-2019 Toshiki Takeuchi

Distributed under the MIT License.