package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (40)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-0.8.5.tbz
sha256=6b3b638fc7c6f4c52617b0261dc9f726ce21bb0c485e05bab6fe41a57697fc6b
md5=2db36741c413ab93391ecc1f983aa804

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Jan 2019

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Screenshots

The following screenshots demonstrate the HTML testing output from the odoc project.

All tests passed Some tests failed Failed test with custom diffing
ok err diff

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. ocaml >= "4.02.3"
  7. dune >= "1.1.0"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. alg_structs_qcheck
  6. ambient-context
  7. ambient-context-eio
  8. ambient-context-lwt
  9. angstrom >= "0.7.0"
  10. ansi >= "0.6.0"
  11. anycache >= "0.7.4"
  12. anycache-async
  13. anycache-lwt
  14. archetype >= "1.4.2"
  15. archi
  16. arp
  17. arp-mirage
  18. arrakis
  19. art
  20. asak >= "0.2"
  21. asli >= "0.2.0"
  22. asn1-combinators >= "0.2.2"
  23. atd >= "2.3.3"
  24. atdgen >= "2.10.0"
  25. atdpy
  26. atdts
  27. base32
  28. base64 >= "2.1.2"
  29. bechamel >= "0.5.0"
  30. bigarray-overlap
  31. bigstring >= "0.3"
  32. bigstring-unix >= "0.3"
  33. bigstringaf
  34. bitlib
  35. blake2
  36. bloomf
  37. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  38. bls12-381-hash
  39. bls12-381-js >= "0.4.2"
  40. bls12-381-js-gen >= "0.4.2"
  41. bls12-381-legacy
  42. bls12-381-signature
  43. bls12-381-unix
  44. blurhash
  45. builder-web
  46. bulletml
  47. bytebuffer
  48. ca-certs
  49. ca-certs-nss
  50. cactus
  51. caldav
  52. calendar >= "3.0.0"
  53. callipyge
  54. camlix
  55. camlkit
  56. camlkit-base
  57. capnp-rpc < "0.6.0"
  58. capnp-rpc-lwt < "0.3"
  59. carray
  60. carton
  61. cborl
  62. ccss >= "1.6"
  63. cf-lwt
  64. chacha
  65. channel
  66. charrua-client
  67. charrua-client-lwt
  68. charrua-client-mirage < "0.11.0"
  69. checked_oint < "0.1.1"
  70. checkseum >= "0.0.3"
  71. cid
  72. clarity-lang
  73. class_group_vdf
  74. cohttp >= "0.17.0"
  75. cohttp-curl-async
  76. cohttp-curl-lwt
  77. cohttp-eio >= "6.0.0~beta2"
  78. colombe
  79. color
  80. conan
  81. conan-cli
  82. conan-database
  83. conan-lwt
  84. conan-unix
  85. conduit = "3.0.0"
  86. conex < "0.10.0"
  87. conex-mirage-crypto
  88. conex-nocrypto
  89. cookie
  90. cow >= "2.2.0"
  91. css
  92. css-parser
  93. cstruct >= "3.3.0"
  94. cstruct-sexp
  95. ctypes-zarith
  96. cuid
  97. curly
  98. current_incr
  99. cwe_checker
  100. data-encoding < "1.0.0"
  101. datakit >= "0.12.0"
  102. datakit-bridge-github >= "0.12.0"
  103. datakit-ci
  104. datakit-client-git >= "0.12.0"
  105. decompress >= "0.8" & < "1.5.3"
  106. depyt
  107. digestif >= "0.7"
  108. dispatch >= "0.4.1"
  109. dkim
  110. dkim-bin
  111. dkim-mirage
  112. dns >= "4.0.0"
  113. dns-cli
  114. dns-client >= "4.6.0"
  115. dns-forward < "0.9.0"
  116. dns-forward-lwt-unix
  117. dns-resolver
  118. dns-server
  119. dns-tsig
  120. dnssd
  121. dnssec
  122. docfd >= "2.2.0"
  123. dog < "0.2.1"
  124. domain-name
  125. dream
  126. dream-pure
  127. duff
  128. dune-release >= "1.0.0"
  129. duration >= "0.1.1"
  130. emile
  131. encore
  132. eqaf >= "0.5"
  133. equinoxe
  134. equinoxe-cohttp
  135. equinoxe-hlc
  136. eris
  137. eris-lwt
  138. ezgzip
  139. ezjsonm >= "0.4.2" & < "1.3.0"
  140. ezjsonm-lwt < "1.3.0"
  141. FPauth
  142. FPauth-core
  143. FPauth-responses
  144. FPauth-strategies
  145. faraday != "0.2.0"
  146. farfadet
  147. fat-filesystem >= "0.12.0"
  148. ff
  149. ff-pbt
  150. fiat-p256
  151. flex-array
  152. fsevents-lwt
  153. functoria >= "2.2.0"
  154. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  155. geojson
  156. geoml >= "0.1.1"
  157. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  158. git-mirage < "3.0.0"
  159. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  160. gitlab-unix
  161. glicko2
  162. gmap >= "0.3.0"
  163. gobba
  164. gpt
  165. graphql
  166. graphql-async
  167. graphql-cohttp >= "0.13.0"
  168. graphql-lwt
  169. graphql_parser
  170. graphql_ppx >= "0.7.1"
  171. h1_parser
  172. h2
  173. hacl
  174. hacl_func
  175. hacl_x25519 >= "0.2.0"
  176. highlexer
  177. hkdf
  178. hockmd
  179. html_of_jsx
  180. http
  181. http-multipart-formdata < "2.0.0"
  182. httpaf >= "0.2.0"
  183. httpun
  184. hvsock
  185. icalendar
  186. imagelib >= "20200929"
  187. index
  188. inferno >= "20220603"
  189. influxdb-async
  190. influxdb-lwt
  191. inquire < "0.2.0"
  192. interval-map
  193. iomux
  194. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  195. irmin-bench >= "2.7.0"
  196. irmin-chunk < "2.0.0" | >= "2.3.0"
  197. irmin-cli
  198. irmin-containers
  199. irmin-fs < "2.0.0" | >= "2.3.0"
  200. irmin-git < "2.0.0" | >= "2.3.0"
  201. irmin-http < "2.0.0"
  202. irmin-mem < "2.0.0"
  203. irmin-pack >= "2.4.0" & != "2.6.1"
  204. irmin-pack-tools
  205. irmin-test < "2.2.0"
  206. irmin-tezos
  207. irmin-tezos-utils
  208. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  209. irmin-watcher
  210. jekyll-format
  211. jerboa
  212. jitsu
  213. jose
  214. json-data-encoding >= "0.9"
  215. json_decoder
  216. jsonxt
  217. junit_alcotest
  218. jwto
  219. ke >= "0.2"
  220. kkmarkdown
  221. lambda-runtime
  222. lambdapi >= "2.0.0"
  223. lambdoc >= "1.0-beta4"
  224. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  225. lmdb >= "1.0"
  226. logical
  227. logtk >= "1.5.1"
  228. lp
  229. lp-glpk
  230. lp-glpk-js
  231. lp-gurobi
  232. lru
  233. lt-code
  234. luv
  235. mbr-format >= "1.0.0"
  236. mdx >= "1.6.0"
  237. mec
  238. mechaml >= "1.0.0"
  239. merge-queues >= "0.2.0"
  240. merge-ropes >= "0.2.0"
  241. metrics
  242. minicaml >= "0.3.1"
  243. mirage >= "4.0.0~beta1"
  244. mirage-block-partition < "0.2.0"
  245. mirage-block-ramdisk >= "0.3"
  246. mirage-channel >= "4.0.0"
  247. mirage-channel-lwt
  248. mirage-crypto-ec
  249. mirage-flow >= "1.0.2" & < "1.2.0"
  250. mirage-flow-unix
  251. mirage-fs-mem
  252. mirage-fs-unix >= "1.2.0"
  253. mirage-kv >= "2.0.0"
  254. mirage-kv-mem
  255. mirage-kv-unix
  256. mirage-logs >= "0.3.0"
  257. mirage-nat
  258. mirage-net-unix >= "2.3.0"
  259. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  260. mirage-tc
  261. mjson
  262. mmdb
  263. mnd
  264. monocypher
  265. mrmime
  266. mrt-format
  267. msgpck >= "1.6"
  268. multibase
  269. multihash
  270. multihash-digestif
  271. multipart-form-data
  272. multipart_form
  273. multipart_form-eio
  274. multipart_form-lwt
  275. named-pipe
  276. nanoid
  277. nbd >= "4.0.3"
  278. nbd-tool
  279. nloge
  280. nocoiner
  281. non_empty_list
  282. OCADml >= "0.6.0"
  283. ocaml-r >= "0.5.0"
  284. ocaml-version >= "3.1.0"
  285. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  286. ocamlformat-rpc < "removed"
  287. ocamline
  288. odoc >= "1.4.0" & < "2.1.0"
  289. ohex
  290. oidc
  291. opam-0install
  292. opam-file-format >= "2.1.1"
  293. opentelemetry >= "0.6"
  294. opentelemetry-client-cohttp-lwt >= "0.6"
  295. opentelemetry-client-ocurl >= "0.6"
  296. opentelemetry-cohttp-lwt >= "0.6"
  297. opentelemetry-lwt >= "0.6"
  298. opium >= "0.15.0"
  299. opium-graphql
  300. opium-testing
  301. opium_kernel
  302. orewa
  303. ortac-core
  304. osx-acl
  305. osx-attr
  306. osx-cf
  307. osx-fsevents
  308. osx-membership
  309. osx-mount
  310. osx-xattr
  311. otoggl
  312. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  313. owl-base < "0.5.0"
  314. owl-ode >= "0.1.0" & != "0.2.0"
  315. owl-symbolic
  316. passmaker
  317. patch
  318. pbkdf
  319. pecu >= "0.2"
  320. pf-qubes
  321. pg_query >= "0.9.6"
  322. phylogenetics
  323. piaf
  324. polyglot
  325. polynomial
  326. ppx_blob >= "0.3.0"
  327. ppx_deriving_cmdliner
  328. ppx_deriving_rpc
  329. ppx_deriving_yaml
  330. ppx_graphql >= "0.2.0"
  331. ppx_mysql
  332. ppx_parser
  333. ppx_protocol_conv >= "5.0.0"
  334. ppx_protocol_conv_json >= "5.0.0"
  335. ppx_protocol_conv_jsonm >= "5.0.0"
  336. ppx_protocol_conv_msgpack >= "5.0.0"
  337. ppx_protocol_conv_xml_light >= "5.0.0"
  338. ppx_protocol_conv_xmlm
  339. ppx_protocol_conv_yaml >= "5.0.0"
  340. ppx_subliner
  341. ppx_units
  342. ppx_yojson >= "1.1.0"
  343. pratter
  344. prc
  345. preface
  346. pretty_expressive
  347. prettym
  348. proc-smaps
  349. producer < "0.2.0"
  350. prom < "0.2"
  351. prometheus < "1.2"
  352. prometheus-app
  353. protocell
  354. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  355. protocol-9p-unix
  356. psq
  357. qcheck >= "0.18"
  358. qcheck-alcotest
  359. qcheck-core >= "0.18"
  360. quickjs
  361. radis
  362. randii
  363. reason-standard
  364. reparse >= "2.0.0" & < "3.0.0"
  365. reparse-unix < "2.1.0"
  366. resp < "0.10.0"
  367. resp-unix
  368. rfc1951 < "1.0.0"
  369. routes < "2.0.0"
  370. rpc >= "5.9.0"
  371. rpclib
  372. rpclib-async
  373. rpclib-lwt
  374. rubytt
  375. SZXX >= "4.0.0"
  376. salsa20
  377. salsa20-core
  378. sanddb >= "0.2"
  379. scaml >= "1.5.0"
  380. scrypt-kdf
  381. secp256k1 >= "0.4.1"
  382. secp256k1-internal
  383. semver >= "0.2.1"
  384. sendmail
  385. sendmail-lwt
  386. sendmsg
  387. server-reason-react
  388. session-cookie
  389. session-cookie-async
  390. session-cookie-lwt
  391. sherlodoc
  392. slug
  393. sodium-fmt
  394. spin >= "0.6.0"
  395. squirrel
  396. ssh-agent
  397. ssl >= "0.6.0"
  398. stramon-lib
  399. styled-ppx
  400. syslog-rfc5424
  401. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
  402. tdigest < "2.1.0"
  403. terminal_size >= "0.1.1"
  404. terminus
  405. terminus-cohttp
  406. terminus-hlc
  407. terml
  408. textrazor
  409. tezos-base-test-helpers < "13.0"
  410. tezos-client-base < "12.0"
  411. tezos-crypto < "8.0"
  412. tezos-lmdb
  413. tezos-signer-backends < "8.0"
  414. tezos-stdlib < "8.0"
  415. tezos-test-helpers < "11.0"
  416. tftp
  417. timedesc
  418. timere
  419. tls >= "0.12.0"
  420. toc
  421. topojson
  422. topojsone
  423. transept
  424. twostep
  425. type_eq
  426. type_id
  427. typebeat
  428. typeid >= "1.0.1"
  429. tyre >= "0.4"
  430. tyxml >= "4.0.0"
  431. tyxml-jsx
  432. tyxml-ppx >= "4.3.0"
  433. tyxml-syntax
  434. uecc
  435. ulid
  436. universal-portal
  437. unix-dirent
  438. unix-errno >= "0.3.0"
  439. unix-fcntl >= "0.3.0"
  440. unix-sys-resource
  441. unix-sys-stat
  442. unix-time
  443. unstrctrd
  444. user-agent-parser
  445. uspf
  446. uspf-lwt
  447. uspf-unix
  448. utop >= "2.13.0"
  449. validate
  450. validator
  451. vercel
  452. vpnkit
  453. wcwidth
  454. websocketaf
  455. x509 >= "0.7.0"
  456. xapi-rrd >= "1.8.2"
  457. xapi-stdext-date
  458. xapi-stdext-encodings
  459. xapi-stdext-std >= "4.16.0"
  460. yaml < "3.2.0"
  461. yaml-sexp
  462. yocaml
  463. yocaml_yaml
  464. yojson >= "1.6.0"
  465. yojson-five
  466. yuscii >= "0.2.0"
  467. zar
  468. zed >= "3.2.2"
  469. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct