package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework.

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (40)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-0.8.4.tbz
sha256=8643b39e74317e2ec49be9a0986e73acd4fcd1277a6d59fb1423c3a6ebdfaf61
md5=c940f89a2bb2e23d3f1422ce61d1396b

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Oct 2018

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. jbuilder >= "1.0+beta10"
  7. ocaml >= "4.02.3"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. ambient-context
  6. ambient-context-eio
  7. ambient-context-lwt
  8. angstrom >= "0.7.0"
  9. ansi >= "0.6.0"
  10. anycache >= "0.7.4"
  11. anycache-async
  12. anycache-lwt
  13. archetype >= "1.4.2"
  14. archi
  15. arp
  16. arp-mirage
  17. arrakis
  18. art
  19. asak >= "0.2"
  20. asli >= "0.2.0"
  21. asn1-combinators >= "0.2.2"
  22. atd >= "2.3.3"
  23. atdgen >= "2.10.0"
  24. atdpy
  25. atdts
  26. base32
  27. base64 >= "2.1.2"
  28. bechamel >= "0.5.0"
  29. bigarray-overlap
  30. bigstring >= "0.3"
  31. bigstring-unix >= "0.3"
  32. bigstringaf
  33. bitlib
  34. blake2
  35. bloomf
  36. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  37. bls12-381-hash
  38. bls12-381-js >= "0.4.2"
  39. bls12-381-js-gen >= "0.4.2"
  40. bls12-381-legacy
  41. bls12-381-signature
  42. bls12-381-unix
  43. blurhash
  44. builder-web
  45. bulletml
  46. bytebuffer
  47. ca-certs
  48. ca-certs-nss
  49. cactus
  50. calendar >= "3.0.0"
  51. callipyge
  52. camlix
  53. camlkit
  54. camlkit-base
  55. capnp-rpc < "0.6.0"
  56. capnp-rpc-lwt < "0.3"
  57. carray
  58. carton
  59. cborl
  60. ccss >= "1.6"
  61. cf-lwt
  62. chacha
  63. channel
  64. charrua-client
  65. charrua-client-lwt
  66. charrua-client-mirage < "0.11.0"
  67. checked_oint < "0.1.1"
  68. checkseum >= "0.0.3"
  69. cid
  70. clarity-lang
  71. class_group_vdf
  72. cohttp >= "0.17.0"
  73. cohttp-curl-async
  74. cohttp-curl-lwt
  75. cohttp-eio >= "6.0.0~beta2"
  76. colombe
  77. color
  78. conan
  79. conan-cli
  80. conan-database
  81. conan-lwt
  82. conan-unix
  83. conduit = "3.0.0"
  84. conex < "0.10.0"
  85. conex-mirage-crypto
  86. conex-nocrypto
  87. cookie
  88. cow >= "2.2.0"
  89. css
  90. css-parser
  91. cstruct >= "3.3.0"
  92. cstruct-sexp
  93. ctypes-zarith
  94. cuid
  95. curly
  96. current_incr
  97. cwe_checker
  98. data-encoding < "1.0.0"
  99. datakit >= "0.12.0"
  100. datakit-bridge-github >= "0.12.0"
  101. datakit-ci
  102. datakit-client-git >= "0.12.0"
  103. decompress >= "0.8" & < "1.5.3"
  104. depyt
  105. digestif >= "0.7"
  106. dispatch >= "0.4.1"
  107. dkim
  108. dkim-bin
  109. dkim-mirage
  110. dns >= "4.0.0"
  111. dns-cli
  112. dns-client >= "4.6.0"
  113. dns-forward < "0.9.0"
  114. dns-forward-lwt-unix
  115. dns-resolver
  116. dns-server
  117. dns-tsig
  118. dnssd
  119. dnssec
  120. docfd >= "2.2.0"
  121. dog < "0.2.1"
  122. domain-name
  123. dream
  124. dream-pure
  125. duff
  126. dune-release >= "1.0.0"
  127. duration >= "0.1.1"
  128. emile
  129. encore
  130. eqaf >= "0.5"
  131. equinoxe
  132. equinoxe-cohttp
  133. equinoxe-hlc
  134. eris
  135. eris-lwt
  136. ezgzip
  137. ezjsonm >= "0.4.2" & < "1.3.0"
  138. ezjsonm-lwt < "1.3.0"
  139. FPauth
  140. FPauth-core
  141. FPauth-responses
  142. FPauth-strategies
  143. faraday != "0.2.0"
  144. farfadet
  145. fat-filesystem >= "0.12.0"
  146. ff
  147. ff-pbt
  148. fiat-p256
  149. flex-array
  150. fsevents-lwt
  151. functoria >= "2.2.0"
  152. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  153. geojson
  154. geoml >= "0.1.1"
  155. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  156. git-mirage < "3.0.0"
  157. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  158. gitlab-unix
  159. glicko2
  160. gmap >= "0.3.0"
  161. gpt
  162. graphql
  163. graphql-async
  164. graphql-cohttp >= "0.13.0"
  165. graphql-lwt
  166. graphql_parser
  167. graphql_ppx >= "0.7.1"
  168. h1_parser
  169. h2
  170. hacl
  171. hacl_func
  172. hacl_x25519 >= "0.2.0"
  173. highlexer
  174. hkdf
  175. hockmd
  176. html_of_jsx
  177. http
  178. http-multipart-formdata < "2.0.0"
  179. httpaf >= "0.2.0"
  180. hvsock
  181. icalendar
  182. imagelib >= "20200929"
  183. index
  184. inferno >= "20220603"
  185. influxdb-async
  186. influxdb-lwt
  187. inquire < "0.2.0"
  188. interval-map
  189. iomux
  190. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  191. irmin-bench >= "2.7.0"
  192. irmin-chunk < "2.0.0" | >= "2.3.0"
  193. irmin-cli
  194. irmin-containers
  195. irmin-fs < "2.0.0" | >= "2.3.0"
  196. irmin-git < "2.0.0" | >= "2.3.0"
  197. irmin-http < "2.0.0"
  198. irmin-mem < "2.0.0"
  199. irmin-pack >= "2.4.0" & != "2.6.1"
  200. irmin-pack-tools
  201. irmin-test < "2.2.0"
  202. irmin-tezos
  203. irmin-tezos-utils
  204. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  205. irmin-watcher
  206. jekyll-format
  207. jerboa
  208. jitsu
  209. jose
  210. json-data-encoding >= "0.9"
  211. json_decoder
  212. jsonxt
  213. junit_alcotest
  214. jwto
  215. ke >= "0.2"
  216. kkmarkdown
  217. lambda-runtime
  218. lambdapi >= "2.0.0"
  219. lambdoc >= "1.0-beta4"
  220. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  221. lmdb >= "1.0"
  222. logical
  223. logtk >= "1.5.1"
  224. lp
  225. lp-glpk
  226. lp-glpk-js
  227. lp-gurobi
  228. lru
  229. lt-code
  230. luv
  231. mbr-format >= "1.0.0"
  232. mdx >= "1.6.0"
  233. mec
  234. mechaml >= "1.0.0"
  235. merge-queues >= "0.2.0"
  236. merge-ropes >= "0.2.0"
  237. metrics
  238. mirage >= "4.0.0~beta1"
  239. mirage-block-partition < "0.2.0"
  240. mirage-block-ramdisk >= "0.3"
  241. mirage-channel >= "4.0.0"
  242. mirage-channel-lwt
  243. mirage-crypto-ec
  244. mirage-flow >= "1.0.2" & < "1.2.0"
  245. mirage-flow-unix
  246. mirage-fs-mem
  247. mirage-fs-unix >= "1.2.0"
  248. mirage-kv >= "2.0.0"
  249. mirage-kv-mem
  250. mirage-kv-unix
  251. mirage-logs >= "0.3.0"
  252. mirage-nat
  253. mirage-net-unix >= "2.3.0"
  254. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  255. mirage-tc
  256. mjson
  257. mnd
  258. monocypher
  259. mrmime
  260. mrt-format
  261. multibase
  262. multihash
  263. multihash-digestif
  264. multipart-form-data
  265. multipart_form
  266. multipart_form-eio
  267. multipart_form-lwt
  268. named-pipe
  269. nanoid
  270. nbd >= "4.0.3"
  271. nbd-tool
  272. nloge
  273. nocoiner
  274. non_empty_list
  275. OCADml >= "0.6.0"
  276. ocaml-r >= "0.5.0"
  277. ocaml-version >= "3.1.0"
  278. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  279. ocamlformat-rpc < "removed"
  280. ocamline
  281. odoc >= "1.4.0" & < "2.1.0"
  282. ohex
  283. oidc
  284. opam-0install
  285. opam-file-format >= "2.1.1"
  286. opentelemetry >= "0.6"
  287. opentelemetry-client-cohttp-lwt >= "0.6"
  288. opentelemetry-client-ocurl >= "0.6"
  289. opentelemetry-cohttp-lwt >= "0.6"
  290. opentelemetry-lwt >= "0.6"
  291. opium >= "0.15.0"
  292. opium-graphql
  293. opium-testing
  294. opium_kernel
  295. orewa
  296. ortac-core
  297. osx-acl
  298. osx-attr
  299. osx-cf
  300. osx-fsevents
  301. osx-membership
  302. osx-mount
  303. osx-xattr
  304. otoggl
  305. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  306. owl-base < "0.5.0"
  307. owl-ode >= "0.1.0" & != "0.2.0"
  308. owl-symbolic
  309. passmaker
  310. patch
  311. pbkdf
  312. pecu >= "0.2"
  313. pf-qubes
  314. pg_query >= "0.9.6"
  315. phylogenetics
  316. piaf
  317. polyglot
  318. polynomial
  319. ppx_blob >= "0.3.0"
  320. ppx_deriving_cmdliner
  321. ppx_deriving_rpc
  322. ppx_deriving_yaml
  323. ppx_graphql >= "0.2.0"
  324. ppx_mysql
  325. ppx_parser
  326. ppx_protocol_conv >= "5.0.0"
  327. ppx_protocol_conv_json >= "5.0.0"
  328. ppx_protocol_conv_jsonm >= "5.0.0"
  329. ppx_protocol_conv_msgpack >= "5.0.0"
  330. ppx_protocol_conv_xml_light >= "5.0.0"
  331. ppx_protocol_conv_xmlm
  332. ppx_protocol_conv_yaml >= "5.0.0"
  333. ppx_subliner
  334. ppx_units
  335. ppx_yojson >= "1.1.0"
  336. pratter
  337. prc
  338. preface
  339. pretty_expressive
  340. prettym
  341. proc-smaps
  342. producer < "0.2.0"
  343. prometheus < "1.2"
  344. prometheus-app
  345. protocell
  346. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  347. protocol-9p-unix
  348. psq
  349. qcheck >= "0.18"
  350. qcheck-alcotest
  351. qcheck-core >= "0.18"
  352. quickjs
  353. radis
  354. randii
  355. reason-standard
  356. reparse >= "2.0.0" & < "3.0.0"
  357. reparse-unix < "2.1.0"
  358. resp < "0.10.0"
  359. resp-unix
  360. rfc1951 < "1.0.0"
  361. routes < "2.0.0"
  362. rpc >= "5.9.0"
  363. rpclib
  364. rpclib-async
  365. rpclib-lwt
  366. rubytt
  367. SZXX >= "4.0.0"
  368. salsa20
  369. salsa20-core
  370. sanddb >= "0.2"
  371. scaml >= "1.5.0"
  372. scrypt-kdf
  373. secp256k1 >= "0.4.1"
  374. secp256k1-internal
  375. semver >= "0.2.1"
  376. sendmail
  377. sendmail-lwt
  378. sendmsg
  379. server-reason-react
  380. session-cookie
  381. session-cookie-async
  382. session-cookie-lwt
  383. sherlodoc
  384. slug
  385. sodium-fmt
  386. spin >= "0.6.0"
  387. squirrel
  388. ssh-agent
  389. ssl >= "0.6.0"
  390. stramon-lib
  391. styled-ppx
  392. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
  393. tdigest < "2.1.0"
  394. terminal_size >= "0.1.1"
  395. terminus
  396. terminus-cohttp
  397. terminus-hlc
  398. terml
  399. textrazor
  400. tezos-base-test-helpers < "13.0"
  401. tezos-client-base < "12.0"
  402. tezos-lmdb
  403. tezos-test-helpers < "11.0"
  404. tftp
  405. timedesc
  406. timere
  407. tls >= "0.12.0"
  408. toc
  409. topojson
  410. topojsone
  411. transept
  412. twostep
  413. type_eq
  414. type_id
  415. typebeat
  416. typeid >= "1.0.1"
  417. tyre >= "0.4"
  418. tyxml >= "4.0.0"
  419. tyxml-jsx
  420. tyxml-ppx >= "4.3.0"
  421. tyxml-syntax
  422. uecc
  423. ulid
  424. universal-portal
  425. unix-dirent
  426. unix-errno >= "0.3.0"
  427. unix-fcntl >= "0.3.0"
  428. unix-sys-resource
  429. unix-sys-stat
  430. unix-time
  431. unstrctrd
  432. user-agent-parser
  433. uspf
  434. uspf-lwt
  435. uspf-unix
  436. utop >= "2.13.0"
  437. validate
  438. validator
  439. vercel
  440. vpnkit
  441. wcwidth
  442. websocketaf
  443. x509 >= "0.7.0"
  444. xapi-rrd >= "1.8.2"
  445. xapi-stdext-date
  446. xapi-stdext-encodings
  447. xapi-stdext-std >= "4.16.0"
  448. yaml < "3.2.0"
  449. yaml-sexp
  450. yocaml
  451. yocaml_yaml
  452. yuscii >= "0.2.0"
  453. zar
  454. zed >= "3.2.2"
  455. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct