package qcheck-alcotest
QuickCheck inspired property-based testing for OCaml
Install
Dune Dependency
Authors
Maintainers
Sources
0.9.tar.gz
md5=7782c8cfce30a5fb766d933e99129ee7
sha512=e1007b4a3be338406d855efcf8d13ac4961963a6c77b794ab83973950e21c777cb66ab6020a0a714f96866597bc8bf7a76a84243e5062dab5b41978e78422e0b
README.adoc.html
README.adoc
= QCheck :toc: macro :toclevels: 4 :source-highlighter: pygments QuickCheck inspired property-based testing for OCaml, and combinators to generate random values to run tests on. The documentation can be found https://c-cube.github.io/qcheck/[here]. This library spent some time in https://github.com/vincent-hugot/iTeML[qtest], but is now standalone again! Note that @gasche's http://gasche.github.io/random-generator/doc/Generator.html[generator library] can be useful too, for generating random values. toc::[] image::https://travis-ci.org/c-cube/qcheck.svg?branch=master[alt="Build Status", link="https://travis-ci.org/c-cube/qcheck"] == Use See the documentation. I also wrote http://cedeela.fr/quickcheck-for-ocaml.html[a blog post] that explains how to use it and some design choices; however, be warned that the API changed in lots of small ways (in the right direction, I hope) so the code will not work any more. <<examples>> is an updated version of the blog post's examples. == Build $ make You can use opam: $ opam install qcheck == License The code is now released under the BSD license. [[examples]] == An Introduction to the Library First, let's see a few tests. Let's open a toplevel (e.g. utop) and type the following to load QCheck: [source,OCaml] ---- #require "qcheck";; ---- NOTE: alternatively, it is now possible to locally do: `dune utop src` to load `qcheck`. === List Reverse is Involutive We write a random test for checking that `List.rev (List.rev l) = l` for any list `l`: [source,OCaml] ---- let test = QCheck.Test.make ~count:1000 ~name:"list_rev_is_involutive" QCheck.(list small_nat) (fun l -> List.rev (List.rev l) = l);; (* we can check right now the property... *) QCheck.Test.check_exn test;; ---- In the above example, we applied the combinator `list` to the random generator `small_nat` (ints between 0 and 100), to create a new generator of lists of random integers. These builtin generators come with printers and shrinkers which are handy for outputting and minimizing a counterexample when a test fails. Consider the buggy property `List.rev l = l`: [source,OCaml] ---- let test = QCheck.Test.make ~count:1000 ~name:"my_buggy_test" QCheck.(list small_nat) (fun l -> List.rev l = l);; ---- When we run this test we are presented with a counterexample: [source,OCaml] ---- # QCheck.Test.check_exn test;; Exception: QCheck.Test.Test_fail ("my_buggy_test", ["[0; 1] (after 23 shrink steps)"]). ---- In this case QCheck found the minimal counterexample `[0;1]` to the property `List.rev l = l` and it spent 23 steps shrinking it. Now, let's run the buggy test with a decent runner that will print the results nicely (the exact output will change at each run, because of the random seed): ---- # QCheck_runner.run_tests [test];; --- Failure -------------------------------------------------------------------- Test my_buggy_test failed (10 shrink steps): [0; 1] ================================================================================ failure (1 tests failed, 0 tests errored, ran 1 tests) - : int = 1 ---- For an even nicer output `QCheck_runner.run_tests` also accepts an optional parameter `~verbose:true`. === Mirrors and Trees `QCheck` provides many useful combinators to write generators, especially for recursive types, algebraic types, and tuples. Let's see how to generate random trees: [source,OCaml] ---- type tree = Leaf of int | Node of tree * tree let leaf x = Leaf x let node x y = Node (x,y) let tree_gen = QCheck.Gen.(sized @@ fix (fun self n -> match n with | 0 -> map leaf nat | n -> frequency [1, map leaf nat; 2, map2 node (self (n/2)) (self (n/2))] ));; (* generate a few trees, just to check what they look like: *) QCheck.Gen.generate ~n:20 tree_gen;; let arbitrary_tree = let open QCheck.Iter in let rec print_tree = function | Leaf i -> "Leaf " ^ (string_of_int i) | Node (a,b) -> "Node (" ^ (print_tree a) ^ "," ^ (print_tree b) ^ ")" in let rec shrink_tree = function | Leaf i -> QCheck.Shrink.int i >|= leaf | Node (a,b) -> of_list [a;b] <+> (shrink_tree a >|= fun a' -> node a' b) <+> (shrink_tree b >|= fun b' -> node a b') in QCheck.make tree_gen ~print:print_tree ~shrink:shrink_tree;; ---- Here we write a generator of random trees, `tree_gen`, using the `fix` combinator. `fix` is *sized* (it is a function from `int` to a random generator; in particular for size 0 it returns only leaves). The `sized` combinator first generates a random size, and then applies its argument to this size. Other combinators include monadic abstraction, lifting functions, generation of lists, arrays, and a choice function. Then, we define `arbitrary_tree`, a `tree QCheck.arbitrary` value, which contains everything needed for testing on trees: - a random generator (mandatory), weighted with `frequency` to increase the chance of generating deep trees - a printer (optional), very useful for printing counterexamples - a *shrinker* (optional), very useful for trying to reduce big counterexamples to small counterexamples that are usually more easy to understand. The above shrinker strategy is to - reduce the integer leaves, and - substitute an internal `Node` with either of its subtrees or by splicing in a recursively shrunk subtree. A range of combinators in `QCheck.Shrink` and `QCheck.Iter` are available for building shrinking functions. We can write a failing test using this generator to see the printer and shrinker in action: [source,OCaml] ---- let rec mirror_tree (t:tree) : tree = match t with | Leaf _ -> t | Node (a,b) -> node (mirror_tree b) (mirror_tree a);; let test_buggy = QCheck.Test.make ~name:"buggy_mirror" ~count:200 arbitrary_tree (fun t -> t = mirror_tree t);; QCheck_runner.run_tests [test_buggy];; ---- This test fails with: [source,OCaml] ---- --- Failure -------------------------------------------------------------------- Test mirror_buggy failed (6 shrink steps): Node (Leaf 0,Leaf 1) ================================================================================ failure (1 tests failed, 0 tests errored, ran 1 tests) - : int = 1 ---- With the (new found) understanding that mirroring a tree changes its structure, we can formulate another property that involves sequentializing its elements in a traversal: [source,OCaml] ---- let tree_infix (t:tree): int list = let rec aux acc t = match t with | Leaf i -> i :: acc | Node (a,b) -> aux (aux acc b) a in aux [] t;; let test_mirror = QCheck.Test.make ~name:"mirror_tree" ~count:200 arbitrary_tree (fun t -> List.rev (tree_infix t) = tree_infix (mirror_tree t));; QCheck_runner.run_tests [test_mirror];; ---- === Preconditions The functions `QCheck.assume` and `QCheck.(==>)` can be used for tests with preconditions. For instance, `List.hd l :: List.tl l = l` only holds for non-empty lists. Without the precondition, the property is false and will even raise an exception in some cases. [source,OCaml] ---- let test_hd_tl = QCheck.(Test.make (list int) (fun l -> assume (l <> []); l = List.hd l :: List.tl l));; QCheck_runner.run_tests [test_hd_tl];; ---- === Long tests It is often useful to have two version of a testsuite: a short one that runs reasonably fast (so that it is effectively run each time a projet is built), and a long one that might be more exhaustive (but whose running time makes it impossible to run at each build). To that end, each test has a 'long' version. In the long version of a test, the number of tests to run is multiplied by the `~long_factor` argument of `QCheck.Test.make`. === Runners The module `QCheck_runner` defines several functions to run tests, including compatibility with `OUnit`. The easiest one is probably `run_tests`, but if you write your tests in a separate executable you can also use `run_tests_main` which parses command line arguments and exits with `0` in case of success, or an error number otherwise. === Integration within OUnit http://ounit.forge.ocamlcore.org/[OUnit] is a popular unit-testing framework for OCaml. QCheck provides a sub-library `qcheck-ounit` with some helpers, in `QCheck_ounit`, to convert its random tests into OUnit tests that can be part of a wider test-suite. [source,OCaml] ---- let passing = QCheck.Test.make ~count:1000 ~name:"list_rev_is_involutive" QCheck.(list small_nat) (fun l -> List.rev (List.rev l) = l);; let failing = QCheck.Test.make ~count:10 ~name:"fail_sort_id" QCheck.(list small_nat) (fun l -> l = List.sort compare l);; let _ = let open OUnit in run_test_tt_main ("tests" >::: List.map QCheck_ounit.to_ounit_test [passing; failing]) ---- NOTE: the package `qcheck` contains the module `QCheck_runner` which contains both custom runners and OUnit-based runners. === Integration within alcotest https://github.com/mirage/alcotest/[Alcotest] is a simple and colorful test framework for OCaml. QCheck now provides a sub-library `qcheck-alcotest` to easily integrate into an alcotest test suite: [source,OCaml] ---- let passing = QCheck.Test.make ~count:1000 ~name:"list_rev_is_involutive" QCheck.(list small_int) (fun l -> List.rev (List.rev l) = l);; let failing = QCheck.Test.make ~count:10 ~name:"fail_sort_id" QCheck.(list small_int) (fun l -> l = List.sort compare l);; let () = let suite = List.map QCheck_alcotest.to_alcotest [ passing; failing] in Alcotest.run "my test" [ "suite", suite ] ---- === Compatibility notes Starting with 0.9, the library is split into several components: - `qcheck-core` depends only on unix and bytes. It contains the module `QCheck` and a `QCheck_base_runner` module with our custom runners. - `qcheck-ounit` provides an integration layer for `OUnit` - `qcheck` provides a compatibility API with older versions of qcheck, using both `qcheck-core` and `qcheck-ounit`. It provides `QCheck_runner` which is similar to older versions and contains both custom and Ounit-based runners. - `qcheck-alcotest` provides an integration layer with `alcotest` Normally, for contributors, `opam pin https://github.com/c-cube/qcheck` will pin all these packages.
sectionYPositions = computeSectionYPositions($el), 10)"
x-init="setTimeout(() => sectionYPositions = computeSectionYPositions($el), 10)"
>