package rpc_parallel

  1. Overview
  2. Docs
Type-safe parallel library built on top of Async_rpc

Install

Dune Dependency

Authors

Maintainers

Sources

rpc_parallel-v0.13.0.tar.gz
sha256=df42adb1702525e92b6809b00c8106f03eeb79c49d82ab9cd0cfe0efe1fc69ef
md5=1a2e33449294291a3290b5a9b0142554

Description

Rpc_parallel offers an API to define various workers and protocols, spawn workers as separate processes, and communicate with them using Async Rpc.

Published: 20 Nov 2019

README

README.org

# Mental Model

- =Worker.t= identifies a worker rpc server
- =spawn= (=serve=) starts a worker Rpc server in another process (the same
  process)
- =client= connects to a worker Rpc server
- =run= dispatches on a connection to a worker Rpc server

# Top-level

It is highly recommended for =Rpc_parallel.start_app= and =Rpc_parallel.Make=
calls to be top-level. But the real requirements are:

1) The master's state is initialized before any calls to =spawn=. This will be
   achieved either by =Rpc_parallel.start_app= or
   =Rpc_parallel.Expert.start_master_server_exn=.

2) Spawned workers (runs of your executable with a certain environment variable
   set) must start running as a worker. This will be achieved either by
   =Rpc_parallel.start_app= or =Rpc_parallel.Expert.worker_command=.

3) Spawned workers must be able to find their function implementations when they
   start running as a worker. These implementations are gathered on the
   application of the =Rpc_parallel.Make= functor.

4) The worker implementations must be defined completely and in the same order,
   regardless of master and worker code paths. This is necessary for the masters
   and workers to agree on certain generated ids.

# Monitoring your workers

Uncaught exceptions in workers will always result in the worker shutting down.
The master can be notified of these exceptions in multiple ways:

- If the exception occured in a function implementation =f= before =f= is
  determined, the exception will be returned back to the caller. E.g. the caller
  of =spawn= or =run= will get an =Error.t= describing the exception.

- If the exception occured after =f= is determined, =on_failure exn= will be
  called (in =Monitor.current ()= at the time of =spawn=) in the spawning
  process.

- If =redirect_stderr= specifies a file, the worker will also write its
  exception to that file before shutting down.

# Optional Rpc Settings

The master's Rpc server will be started with the =max_message_size=,
=heartbeat_config=, and =handshake_timeout= settings passed in to
=start_app= / =init_master_exn=.

Each worker's Rpc server will be started with the settings passed in upon
=spawn= or =serve= of that worker.

All =client= calls will use the corresponding Rpc settings for the given
worker/master that it is connecting to.

Dependencies (7)

  1. dune >= "1.5.1"
  2. sexplib >= "v0.13" & < "v0.14"
  3. ppx_jane >= "v0.13" & < "v0.14"
  4. core_kernel >= "v0.13" & < "v0.14"
  5. core >= "v0.13" & < "v0.14"
  6. async >= "v0.13" & < "v0.14"
  7. ocaml >= "4.08.0"

Dev Dependencies

None

Used by

None

Conflicts

None

OCaml

Innovation. Community. Security.