package lwt-pipeline

  1. Overview
  2. Docs
Pipeline library for Lwt

Install

Dune Dependency

Authors

Maintainers

Sources

lwt_pipeline-v0.2.tar.gz
md5=06f3e61266b930f2f68c1a6649497411
sha512=4e3f8f1ed26ab70b9d6a25432b3a9fcbc33a827028801cf631fa246f8a76588fa4365572f02052c769293eec26a0213c0e561fb6288bc60a269a06c182c724ff

README.html

README

# Pipeline

Pipeline is a library to build and execute processing pipelines with Lwt-based
concurrency.

--------------------------------------------------------------------------------

A pipeline (`('a, 'b) pipe`) is a series of steps (`('c, 'd) step`) which are
daisy-chained in a type-safe way
(`cons: ('a, 'b) step -> ('b, 'c) pipe -> ('a, 'c) pipe`).

A step can either be:
- synchronous (`sync: ('a -> 'b) -> ('a, 'b) step`) in which case its
  transformation is applied immediately to the data that traverses the pipeline,
- asynchronous-sequential (`async_s: ('a -> 'b Lwt.t) -> ('a, 'b) step`) in
  which case its transformation is applied as an Lwt promise, but only one such
  promise is ever not resolved at a time, or
- asynchronous-parallel (`async_p: ('a -> 'b Lwt.t) -> ('a, 'b) step`) in which
  case its transformation is applied as an Lwt promise, but multiple such
  promises cat be not resolved at the same time.

When a pipeline is executed (`run: ('i, 'o) pipe -> 'i list -> 'o list Lwt.t`)
each element of the input list goes through each of the steps one-by-one, and
the order is always preserved. In addition, an optional `pool` argument can be
used to limit the number of promises that can be not resolved at any given time.
OCaml

Innovation. Community. Security.