package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (40)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-0.8.5.tbz
md5=2db36741c413ab93391ecc1f983aa804

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Jan 2019

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Screenshots

The following screenshots demonstrate the HTML testing output from the odoc project.

All tests passed Some tests failed Failed test with custom diffing
ok err diff

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. ocaml >= "4.02.3"
  7. dune >= "1.1.0"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. alg_structs_qcheck
  6. ambient-context
  7. ambient-context-eio
  8. ambient-context-lwt
  9. angstrom >= "0.7.0"
  10. ansi >= "0.6.0"
  11. anycache >= "0.7.4"
  12. anycache-async
  13. anycache-lwt
  14. archetype >= "1.4.2"
  15. archi
  16. arp
  17. arp-mirage
  18. arrakis
  19. art
  20. asak >= "0.2"
  21. asli >= "0.2.0"
  22. asn1-combinators >= "0.2.2"
  23. atd >= "2.3.3"
  24. atdgen >= "2.10.0"
  25. atdpy
  26. atdts
  27. base32
  28. base64 >= "2.1.2"
  29. bechamel >= "0.5.0"
  30. bigarray-overlap
  31. bigstring >= "0.3"
  32. bigstring-unix >= "0.3"
  33. bigstringaf
  34. bitlib
  35. blake2
  36. bloomf
  37. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  38. bls12-381-hash
  39. bls12-381-js >= "0.4.2"
  40. bls12-381-js-gen >= "0.4.2"
  41. bls12-381-legacy
  42. bls12-381-signature
  43. bls12-381-unix
  44. blurhash
  45. builder-web
  46. bulletml
  47. bytebuffer
  48. ca-certs
  49. ca-certs-nss
  50. cactus
  51. caldav
  52. calendar >= "3.0.0"
  53. callipyge
  54. camlix
  55. capnp-rpc < "0.6.0"
  56. capnp-rpc-lwt < "0.3"
  57. carray
  58. carton
  59. cborl
  60. ccss >= "1.6"
  61. cf-lwt
  62. chacha
  63. channel
  64. charrua-client
  65. charrua-client-lwt
  66. charrua-client-mirage < "0.11.0"
  67. checked_oint < "0.1.1"
  68. checkseum >= "0.0.3"
  69. cid
  70. clarity-lang
  71. class_group_vdf
  72. cohttp >= "0.17.0"
  73. cohttp-curl-async
  74. cohttp-curl-lwt
  75. cohttp-eio >= "6.0.0~beta2"
  76. colombe
  77. color
  78. conan
  79. conan-cli
  80. conan-database
  81. conan-lwt
  82. conan-unix
  83. conduit = "3.0.0"
  84. conex < "0.10.0"
  85. conex-mirage-crypto
  86. conex-nocrypto
  87. cookie
  88. cow >= "2.2.0"
  89. css
  90. css-parser
  91. cstruct >= "3.3.0"
  92. cstruct-sexp
  93. ctypes-zarith
  94. cuid
  95. curly
  96. current_incr
  97. cwe_checker
  98. data-encoding < "1.0.0"
  99. datakit >= "0.12.0"
  100. datakit-bridge-github >= "0.12.0"
  101. datakit-ci
  102. datakit-client-git >= "0.12.0"
  103. decompress >= "0.8" & < "1.5.3"
  104. depyt
  105. digestif >= "0.7"
  106. dispatch >= "0.4.1"
  107. dkim
  108. dkim-bin
  109. dkim-mirage
  110. dns >= "4.0.0"
  111. dns-cli
  112. dns-client >= "4.6.0"
  113. dns-forward < "0.9.0"
  114. dns-forward-lwt-unix
  115. dns-resolver
  116. dns-server
  117. dns-tsig
  118. dnssd
  119. dnssec
  120. docfd >= "2.2.0"
  121. dog < "0.2.1"
  122. domain-name
  123. dream
  124. dream-pure
  125. duff
  126. dune-release >= "1.0.0"
  127. duration >= "0.1.1"
  128. emile
  129. encore
  130. eqaf >= "0.5"
  131. equinoxe
  132. equinoxe-cohttp
  133. equinoxe-hlc
  134. eris
  135. eris-lwt
  136. ezgzip
  137. ezjsonm >= "0.4.2" & < "1.3.0"
  138. ezjsonm-lwt < "1.3.0"
  139. FPauth
  140. FPauth-core
  141. FPauth-responses
  142. FPauth-strategies
  143. faraday != "0.2.0"
  144. farfadet
  145. fat-filesystem >= "0.12.0"
  146. ff
  147. ff-pbt
  148. fiat-p256
  149. flex-array
  150. fsevents-lwt
  151. functoria >= "2.2.0"
  152. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  153. geojson
  154. geoml >= "0.1.1"
  155. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  156. git-mirage < "3.0.0"
  157. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  158. gitlab-unix
  159. glicko2
  160. gmap >= "0.3.0"
  161. gobba
  162. gpt
  163. graphql
  164. graphql-async
  165. graphql-cohttp >= "0.13.0"
  166. graphql-lwt
  167. graphql_parser
  168. graphql_ppx >= "0.7.1"
  169. h1_parser
  170. h2
  171. hacl
  172. hacl_func
  173. hacl_x25519 >= "0.2.0"
  174. highlexer
  175. hkdf
  176. hockmd
  177. html_of_jsx
  178. http
  179. http-multipart-formdata < "2.0.0"
  180. httpaf >= "0.2.0"
  181. hvsock
  182. icalendar
  183. imagelib >= "20200929"
  184. index
  185. inferno >= "20220603"
  186. influxdb-async
  187. influxdb-lwt
  188. inquire < "0.2.0"
  189. interval-map
  190. iomux
  191. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  192. irmin-bench >= "2.7.0"
  193. irmin-chunk < "2.0.0" | >= "2.3.0"
  194. irmin-cli
  195. irmin-containers
  196. irmin-fs < "2.0.0" | >= "2.3.0"
  197. irmin-git < "2.0.0" | >= "2.3.0"
  198. irmin-http < "2.0.0"
  199. irmin-mem < "2.0.0"
  200. irmin-pack >= "2.4.0" & != "2.6.1"
  201. irmin-pack-tools
  202. irmin-test < "2.2.0"
  203. irmin-tezos
  204. irmin-tezos-utils
  205. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  206. irmin-watcher
  207. jekyll-format
  208. jerboa
  209. jitsu
  210. jose
  211. json-data-encoding >= "0.9"
  212. json_decoder
  213. jsonxt
  214. junit_alcotest
  215. jwto
  216. ke >= "0.2"
  217. kkmarkdown
  218. lambda-runtime
  219. lambdapi >= "2.0.0"
  220. lambdoc >= "1.0-beta4"
  221. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  222. lmdb >= "1.0"
  223. logical
  224. logtk >= "1.5.1"
  225. lp
  226. lp-glpk
  227. lp-glpk-js
  228. lp-gurobi
  229. lru
  230. lt-code
  231. luv
  232. mbr-format >= "1.0.0"
  233. mdx >= "1.6.0"
  234. mec
  235. mechaml >= "1.0.0"
  236. merge-queues >= "0.2.0"
  237. merge-ropes >= "0.2.0"
  238. metrics
  239. minicaml >= "0.3.1"
  240. mirage >= "4.0.0~beta1"
  241. mirage-block-partition < "0.2.0"
  242. mirage-block-ramdisk >= "0.3"
  243. mirage-channel >= "4.0.0"
  244. mirage-channel-lwt
  245. mirage-crypto-ec
  246. mirage-flow >= "1.0.2" & < "1.2.0"
  247. mirage-flow-unix
  248. mirage-fs-mem
  249. mirage-fs-unix >= "1.2.0"
  250. mirage-kv >= "2.0.0"
  251. mirage-kv-mem
  252. mirage-kv-unix
  253. mirage-logs >= "0.3.0"
  254. mirage-nat
  255. mirage-net-unix >= "2.3.0"
  256. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  257. mirage-tc
  258. mjson
  259. mmdb
  260. mnd
  261. monocypher
  262. mrmime
  263. mrt-format
  264. msgpck >= "1.6"
  265. multibase
  266. multihash
  267. multihash-digestif
  268. multipart-form-data
  269. multipart_form
  270. multipart_form-eio
  271. multipart_form-lwt
  272. named-pipe
  273. nanoid
  274. nbd >= "4.0.3"
  275. nbd-tool
  276. nloge
  277. nocoiner
  278. non_empty_list
  279. OCADml >= "0.6.0"
  280. ocaml-r >= "0.5.0"
  281. ocaml-version >= "3.1.0"
  282. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  283. ocamlformat-rpc < "removed"
  284. ocamline
  285. odoc >= "1.4.0" & < "2.1.0"
  286. ohex
  287. oidc
  288. opam-0install
  289. opam-file-format >= "2.1.1"
  290. opentelemetry >= "0.6"
  291. opentelemetry-client-cohttp-lwt >= "0.6"
  292. opentelemetry-client-ocurl >= "0.6"
  293. opentelemetry-cohttp-lwt >= "0.6"
  294. opentelemetry-lwt >= "0.6"
  295. opium >= "0.15.0"
  296. opium-graphql
  297. opium-testing
  298. opium_kernel
  299. orewa
  300. ortac-core
  301. osx-acl
  302. osx-attr
  303. osx-cf
  304. osx-fsevents
  305. osx-membership
  306. osx-mount
  307. osx-xattr
  308. otoggl
  309. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  310. owl-base < "0.5.0"
  311. owl-ode >= "0.1.0" & != "0.2.0"
  312. owl-symbolic
  313. passmaker
  314. patch
  315. pbkdf
  316. pecu >= "0.2"
  317. pf-qubes
  318. pg_query >= "0.9.6"
  319. phylogenetics
  320. piaf
  321. polyglot
  322. polynomial
  323. ppx_blob >= "0.3.0"
  324. ppx_deriving_cmdliner
  325. ppx_deriving_rpc
  326. ppx_deriving_yaml
  327. ppx_graphql >= "0.2.0"
  328. ppx_mysql
  329. ppx_parser
  330. ppx_protocol_conv >= "5.0.0"
  331. ppx_protocol_conv_json >= "5.0.0"
  332. ppx_protocol_conv_jsonm >= "5.0.0"
  333. ppx_protocol_conv_msgpack >= "5.0.0"
  334. ppx_protocol_conv_xml_light >= "5.0.0"
  335. ppx_protocol_conv_xmlm
  336. ppx_protocol_conv_yaml >= "5.0.0"
  337. ppx_subliner
  338. ppx_units
  339. ppx_yojson >= "1.1.0"
  340. pratter
  341. prc
  342. preface
  343. pretty_expressive
  344. prettym
  345. proc-smaps
  346. producer < "0.2.0"
  347. prom < "0.2"
  348. prometheus < "1.2"
  349. prometheus-app
  350. protocell
  351. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  352. protocol-9p-unix
  353. psq
  354. qcheck >= "0.18"
  355. qcheck-alcotest
  356. qcheck-core >= "0.18"
  357. quickjs
  358. radis
  359. randii
  360. reason-standard
  361. reparse >= "2.0.0" & < "3.0.0"
  362. reparse-unix < "2.1.0"
  363. resp < "0.10.0"
  364. resp-unix
  365. rfc1951 < "1.0.0"
  366. routes < "2.0.0"
  367. rpc >= "5.9.0"
  368. rpclib
  369. rpclib-async
  370. rpclib-lwt
  371. rubytt
  372. SZXX >= "4.0.0"
  373. salsa20
  374. salsa20-core
  375. sanddb >= "0.2"
  376. scaml >= "1.5.0"
  377. scrypt-kdf
  378. secp256k1 >= "0.4.1"
  379. secp256k1-internal
  380. semver >= "0.2.1"
  381. sendmail
  382. sendmail-lwt
  383. sendmsg
  384. server-reason-react
  385. session-cookie
  386. session-cookie-async
  387. session-cookie-lwt
  388. sherlodoc
  389. slug
  390. sodium-fmt
  391. spin >= "0.6.0"
  392. squirrel
  393. ssh-agent
  394. ssl >= "0.6.0"
  395. stramon-lib
  396. styled-ppx
  397. syslog-rfc5424
  398. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "8.0.0"
  399. tdigest < "2.1.0"
  400. terminal_size >= "0.1.1"
  401. terminus
  402. terminus-cohttp
  403. terminus-hlc
  404. terml
  405. textrazor
  406. tezos-base-test-helpers < "13.0"
  407. tezos-client-base < "12.0"
  408. tezos-crypto < "8.0"
  409. tezos-lmdb
  410. tezos-signer-backends < "8.0"
  411. tezos-stdlib < "8.0"
  412. tezos-test-helpers < "11.0"
  413. tftp
  414. timedesc
  415. timere
  416. tls >= "0.12.0"
  417. toc
  418. topojson
  419. topojsone
  420. transept
  421. twostep
  422. type_eq
  423. type_id
  424. typebeat
  425. typeid >= "1.0.1"
  426. tyre >= "0.4"
  427. tyxml >= "4.0.0"
  428. tyxml-jsx
  429. tyxml-ppx >= "4.3.0"
  430. tyxml-syntax
  431. uecc
  432. ulid
  433. universal-portal
  434. unix-dirent
  435. unix-errno >= "0.3.0"
  436. unix-fcntl >= "0.3.0"
  437. unix-sys-resource
  438. unix-sys-stat
  439. unix-time
  440. unstrctrd
  441. user-agent-parser
  442. uspf
  443. uspf-lwt
  444. uspf-unix
  445. utop >= "2.13.0"
  446. validate
  447. validator
  448. vercel
  449. vpnkit
  450. wcwidth
  451. websocketaf
  452. x509 >= "0.7.0"
  453. xapi-rrd >= "1.8.2"
  454. xapi-stdext-date
  455. xapi-stdext-encodings
  456. xapi-stdext-std >= "4.16.0"
  457. yaml < "3.2.0"
  458. yaml-sexp
  459. yocaml
  460. yocaml_yaml
  461. yojson >= "1.6.0"
  462. yuscii >= "0.2.0"
  463. zar
  464. zed >= "3.2.2"
  465. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events Blog Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct