package alcotest-lwt

  1. Overview
  2. Docs
Lwt-based helpers for Alcotest

Install

Dune Dependency

Authors

Maintainers

Sources

alcotest-1.7.0.tbz
sha256=812bacdb34b45e88995e07d7306bdab2f72479ef1996637f1d5d1f41667902df
sha512=4ae1ba318949ec9db8b87bc8072632a02f0e4003a95ab21e474f5c34c3b5bde867b0194a2d0ea7d9fc4580c70a30ca39287d33a8c134acc7611902f79c7b7ce8

Description

Lwt-based helpers for Alcotest

Published: 27 Feb 2023

README

README.md

A lightweight and colourful test framework.


Alcotest exposes a simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run. See the manpage for details.

The API documentation can be found here. For information on contributing to Alcotest, see CONTRIBUTING.md.


Examples

A simple example (taken from examples/simple.ml):

Generated by the following test suite specification:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let lowercase = String.lowercase_ascii
  let capitalize = String.capitalize_ascii
  let str_concat = String.concat ""
  let list_concat = List.append
end

(* The tests *)
let test_lowercase () =
  Alcotest.(check string) "same string" "hello!" (To_test.lowercase "hELLO!")

let test_capitalize () =
  Alcotest.(check string) "same string" "World." (To_test.capitalize "world.")

let test_str_concat () =
  Alcotest.(check string) "same string" "foobar" (To_test.str_concat ["foo"; "bar"])

let test_list_concat () =
  Alcotest.(check (list int)) "same lists" [1; 2; 3] (To_test.list_concat [1] [2; 3])

(* Run it *)
let () =
  let open Alcotest in
  run "Utils" [
      "string-case", [
          test_case "Lower case"     `Quick test_lowercase;
          test_case "Capitalization" `Quick test_capitalize;
        ];
      "string-concat", [ test_case "String mashing" `Quick test_str_concat  ];
      "list-concat",   [ test_case "List mashing"   `Slow  test_list_concat ];
    ]

The result is a self-contained binary which displays the test results. Use dune exec examples/simple.exe -- --help to see the runtime options.

Here's an example of a of failing test suite:

By default, only the first failing test log is printed to the console (and all test logs are captured on disk). Pass --show-errors to print all error messages.

Using Alcotest with opam and Dune

Add (alcotest :with-test) to the depends stanza of your dune-project file, or "alcotest" {with-test} to your opam file. Use the with-test package variable to declare your tests opam dependencies. Call opam to install them:

$ opam install --deps-only --with-test .

You can then declare your test and link with Alcotest: (test (libraries alcotest …) …), and run your tests:

$ dune runtest

Selecting tests to execute

You can filter which tests to run by supplying a regular expression matching the names of the tests to execute, or by passing a regular expression and a comma-separated list of test numbers (or ranges of test numbers, e.g. 2,4..9):

$ ./simple.native test '.*concat*'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[SKIP]     string-case            1   Capitalization.
[OK]       string-concat          0   String mashing.
[OK]       list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 tests run.

$ ./simple.native test 'string-case' '1..3'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[OK]       string-case            1   Capitalization.
[SKIP]     string-concat          0   String mashing.
[SKIP]     list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Lower case or Capitalization), you must filter by test name & number instead.

See the examples directory for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your preferred number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Lwt_main.run @@ Alcotest_lwt.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck does random generation and property testing (e.g. Quick Check);

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, i.e. they take advantage of the AFL support the OCaml compiler;

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (6)

  1. logs
  2. lwt
  3. alcotest = version
  4. ocaml >= "4.05.0"
  5. fmt
  6. dune >= "3.0"

Dev Dependencies (3)

  1. odoc with-doc
  2. cmdliner with-test & >= "1.1.0"
  3. re with-test

  1. ambient-context-eio
  2. ambient-context-lwt
  3. ask
  4. ask-integrator
  5. azure-cosmos-db >= "0.1.3"
  6. capnp-rpc-lwt = "0.3"
  7. capnp-rpc-mirage >= "0.6.0"
  8. capnp-rpc-unix >= "0.6.0"
  9. caqti-lwt >= "1.7.0"
  10. carton
  11. carton-git
  12. carton-lwt
  13. chamelon
  14. chamelon-unix
  15. current >= "0.4"
  16. current_git >= "0.6.4"
  17. equinoxe
  18. equinoxe-cohttp
  19. equinoxe-hlc
  20. git >= "3.0.0"
  21. git-cohttp
  22. git-cohttp-mirage
  23. git-cohttp-unix
  24. git-mirage >= "3.0.0"
  25. git-unix >= "3.0.0"
  26. gitlab-unix < "0.1.1"
  27. guardian
  28. http-mirage-client
  29. irmin >= "2.4.0"
  30. irmin-containers
  31. irmin-graphql >= "2.2.0"
  32. irmin-pack != "2.3.0" & != "2.6.1"
  33. irmin-server
  34. irmin-test >= "3.0.0"
  35. ledgerwallet-tezos >= "0.4.0"
  36. letters
  37. mimic
  38. mirage-block-partition
  39. mirage-vnetif-stack
  40. multipart_form-lwt
  41. nbd >= "4.0.3"
  42. nbd-tool
  43. nbd-unix < "6.0.1"
  44. obuilder
  45. ocluster
  46. opium >= "0.19.0"
  47. opium-graphql
  48. opium-testing
  49. otoggl
  50. paf
  51. paf-cohttp
  52. pgx_lwt_unix
  53. piaf
  54. prometheus-app >= "1.2"
  55. resto >= "0.9"
  56. rpclib-lwt >= "7.1.0"
  57. SZXX < "4.0.0"
  58. server-reason-react
  59. sihl != "0.3.0~rc1"
  60. sihl-cache
  61. sihl-core
  62. sihl-email >= "0.3.0~rc1"
  63. sihl-facade
  64. sihl-persistence >= "0.3.0~rc1"
  65. sihl-queue >= "0.3.0~rc1"
  66. sihl-session
  67. sihl-storage >= "0.3.0~rc1"
  68. sihl-token
  69. sihl-type
  70. sihl-user >= "0.3.0~rc1"
  71. sihl-web >= "0.3.0~rc1"
  72. tar-mirage >= "2.4.0"
  73. terminus
  74. terminus-cohttp
  75. terminus-hlc
  76. tezos-008-PtEdo2Zk-test-helpers
  77. tezos-009-PsFLoren-test-helpers
  78. tezos-010-PtGRANAD-test-helpers
  79. tezos-011-PtHangz2-test-helpers
  80. tezos-012-Psithaca-test-helpers
  81. tezos-013-PtJakart-test-helpers
  82. tezos-014-PtKathma-test-helpers
  83. tezos-alpha-test-helpers
  84. tezos-baking-011-PtHangz2 >= "12.0"
  85. tezos-baking-012-Psithaca
  86. tezos-baking-013-PtJakart
  87. tezos-baking-014-PtKathma
  88. tezos-baking-015-PtLimaPt
  89. tezos-baking-016-PtMumbai < "17.1"
  90. tezos-baking-alpha >= "12.0" & < "17.1"
  91. tezos-base-test-helpers < "17.1"
  92. tezos-clic >= "8.0" & < "17.1"
  93. tezos-client-004-Pt24m4xi >= "8.0" & < "9.0"
  94. tezos-client-005-PsBabyM1 >= "8.0" & < "10.2"
  95. tezos-client-006-PsCARTHA >= "8.0" & < "9.0"
  96. tezos-client-007-PsDELPH1 >= "8.0" & < "13.0"
  97. tezos-client-008-PtEdo2Zk < "13.0"
  98. tezos-client-008-PtEdoTez
  99. tezos-client-009-PsFLoren < "13.0"
  100. tezos-client-010-PtGRANAD < "13.0"
  101. tezos-client-011-PtHangz2 < "17.1"
  102. tezos-client-012-Psithaca >= "13.0" & < "17.1"
  103. tezos-client-013-PtJakart < "17.1"
  104. tezos-client-014-PtKathma < "17.1"
  105. tezos-client-015-PtLimaPt < "17.1"
  106. tezos-client-016-PtMumbai < "17.1"
  107. tezos-client-alpha >= "8.0" & < "17.1"
  108. tezos-client-base-unix >= "13.0" & < "17.1"
  109. tezos-context < "17.1"
  110. tezos-crypto >= "8.0" & < "9.2" | >= "10.2" & < "17.1"
  111. tezos-dal-node-lib < "17.1"
  112. tezos-error-monad >= "8.0" & < "12.0"
  113. tezos-layer2-store < "17.1"
  114. tezos-legacy-store
  115. tezos-lwt-result-stdlib < "17.1"
  116. tezos-micheline >= "8.0" & < "13.0"
  117. tezos-micheline-rewriting
  118. tezos-mockup >= "10.2" & < "17.1"
  119. tezos-p2p >= "8.0" & < "17.1"
  120. tezos-protocol-environment >= "8.0" & < "17.1"
  121. tezos-protocol-plugin-012-Psithaca < "13.0"
  122. tezos-protocol-plugin-012-Psithaca-tests
  123. tezos-protocol-plugin-013-PtJakart-tests
  124. tezos-protocol-plugin-alpha >= "12.0" & < "13.0"
  125. tezos-protocol-plugin-alpha-tests
  126. tezos-proxy < "17.1"
  127. tezos-proxy-server-config < "17.1"
  128. tezos-requester >= "8.0" & < "17.1"
  129. tezos-rpc-http-server >= "10.2" & < "17.1"
  130. tezos-sapling < "17.1"
  131. tezos-scoru-wasm-helpers < "17.1"
  132. tezos-shell >= "8.0" & < "17.1"
  133. tezos-shell-context-test
  134. tezos-shell-services >= "11.0" & < "13.0"
  135. tezos-shell-services-test-helpers >= "12.0"
  136. tezos-signer-backends >= "8.0" & < "17.1"
  137. tezos-stdlib >= "8.0" & < "17.1"
  138. tezos-storage >= "8.0"
  139. tezos-store < "17.1"
  140. tezos-test-helpers < "12.0"
  141. tezos-test-services
  142. tezos-workers >= "14.0" & < "17.1"
  143. tidy_email_mailgun
  144. tidy_email_sendgrid
  145. tidy_email_smtp
  146. wayland < "2.0"

Conflicts

None