package bench

  1. Overview
  2. Docs
package bench
Legend:
Library
Module
Module type
Parameter
Class
Class type

Benchmarking toolbox, based on the Criterion library by Bryan O'Sullivan

  • author Edgar Friendly <thelema314\@gmail.com>
Easy Benchmarking

The following function suffices for most uses of this library. To benchmark the (unit -> unit) functions f,g,and h, the following will run the benchmark and print a comparison: 

Bench.bench ["f",f; "g",g; "h",h]

The number of iterations to run per sample and number of samples to take will be computed automatically.

val bench : (string * (unit -> 'a)) list -> unit

Benchmark many unit functions.

Internal result format
module Bootstrap : sig ... end
type results = {
  1. desc : string;
  2. times : float array;
  3. mean : Bootstrap.estimate;
  4. stdev : Bootstrap.estimate;
  5. ov : float;
}

The results of running a test

Detailed benchmarking functions
val bench_arg : (string * ('a -> 'b) * 'a) list -> results list

bench fs benchmarks the named functions with argument given in fs

let res = Bench.bench_arg ["div", ( /. ) 3., 4.;
                          "mul", ( *. ) 3., 0.25] in
Bench.run_outputs res
val bench_args : ('a -> 'b) -> (string * 'a) list -> results list

Benchmark one function with many arguments.

let () = Bench.run_outputs (Bench.bench_args List.rev [
    "10", Array.to_list (Array.create 10 0);
    "100", Array.to_list (Array.create 100 0);
    "1000", Array.to_list (Array.create 1000 0);
    ])
val bench_funs : (string * ('a -> 'b)) list -> 'a -> results list

Benchmark many functions with a fixed parameter. let l = Array.to_list (Array.init 1000 (fun _ -> Random.float 20.)) in let res = Bench.bench_funs "listsort", List.sort compare; "arrsort", (fun l -> let a = Array.of_list l in Array.sort compare a; Array.to_list a); l in Bench.run_outputs res

val bench_n : (string * (int -> 'a)) list -> results list

This function benchmarks functions that take a single parameter of how many repetitions to run. This is slightly more accurate benchmark than others, as the function isn't wrapped by an extension function that calls it many times.

val bench_throughput : (int -> 'a) -> int list -> results list

Benchmark a function for throughput. The function's argument is the block size it will work over, and the block sizes to test are given in a list. The results are in terms of throughput - the time taken divided by the size of the block.

val summarize : float -> results list -> unit

The function that summarizes the results by comparing the results linearly. The first parameter is the alpha (type I error rate) for the "same time" test, the second is the list of results from the tests.

Configuration API
type config = {
  1. mutable verbose : bool;
    (*

    bench will print much more progress information if this is true. Default=true.

    *)
  2. mutable samples : int;
    (*

    The minimum number of samples to measure Default=1000

    *)
  3. mutable gc_between_tests : bool;
    (*

    Whether or not to call Gc.compact between tests Default=false

    *)
  4. mutable resamples : int;
    (*

    The number of resamples to use when computing confidence intervals Default=1000

    *)
  5. mutable confidence_interval : float;
    (*

    How big a confidence interval to estimate for mean and stdev Default: 95% (0.95)

    *)
  6. mutable output : (results list -> unit) list;
    (*

    Output functions to use. Default: summarize 0.05

    *)
}

The following global configuration parameters are available for bench

val config : config
val init_environment : unit -> unit

Initialize the benchmark environment by measuring the clock resolution and cost. This function is called automatically if these measurements are needed, and calling it a second time does nothing.