octez-libs 19.1 · OCaml Package

TEZOS_CONTEXT is the module type implemented by all storage implementations. This is the module type that the shell expects for its operation. As such, it should be a strict superset of the interface exposed to the protocol (see module type S above and Tezos_protocol_environment.Environment_context_intf.S).

The main purpose of this module type is to keep the on-disk and in-memory implementations in sync.

Generic interface

type index

A block-indexed (key x value) store directory.

val equal_config : Config.t -> Config.t -> bool

include VIEW with type key = string list and type value = bytes

type t

The type for context views.

type key = string list

The type for context keys.

type value = bytes

The type for context values.

type tree

The type for context trees.

Getters

val mem : t -> key -> bool Lwt.t

mem t k is an Lwt promise that resolves to true iff k is bound to a value in t.

val mem_tree : t -> key -> bool Lwt.t

mem_tree t k is like mem but for trees.

val find : t -> key -> value option Lwt.t

find t k is an Lwt promise that resolves to Some v if k is bound to the value v in t and None otherwise.

val find_tree : t -> key -> tree option Lwt.t

find_tree t k is like find but for trees.

val list : t -> ?offset:int -> ?length:int -> key -> (string * tree) list Lwt.t

list t key is the list of files and sub-nodes stored under k in t. The result order is not specified but is stable.

offset and length are used for pagination.

val length : t -> key -> int Lwt.t

length t key is an Lwt promise that resolves to the number of files and sub-nodes stored under k in t.

It is equivalent to let+ l = list t k in List.length l but has a constant-time complexity.

Setters

val add : t -> key -> value -> t Lwt.t

add t k v is an Lwt promise that resolves to c such that:

k is bound to v in c;
and c is similar to t otherwise.

If k was already bound in t to a value that is physically equal to v, the result of the function is a promise that resolves to t. Otherwise, the previous binding of k in t disappears.

val add_tree : t -> key -> tree -> t Lwt.t

add_tree is like add but for trees.

val remove : t -> key -> t Lwt.t

remove t k v is an Lwt promise that resolves to c such that:

k is unbound in c;
and c is similar to t otherwise.

Folding

val fold : 
  ?depth:depth ->
  t ->
  key ->
  order:[ `Sorted | `Undefined ] ->
  init:'a ->
  f:(key -> tree -> 'a -> 'a Lwt.t) ->
  'a Lwt.t

fold ?depth t root ~order ~init ~f recursively folds over the trees and values of t. The f callbacks are called with a key relative to root. f is never called with an empty key for values; i.e., folding over a value is a no-op.

The depth is 0-indexed. If depth is set (by default it is not), then f is only called when the conditions described by the parameter is true:

Eq d folds over nodes and values of depth exactly d.
Lt d folds over nodes and values of depth strictly less than d.
Le d folds over nodes and values of depth less than or equal to d.
Gt d folds over nodes and values of depth strictly more than d.
Ge d folds over nodes and values of depth more than or equal to d.

If order is `Sorted (the default), the elements are traversed in lexicographic order of their keys. For large nodes, it is memory-consuming, use `Undefined for a more memory efficient fold.

Configuration

val config : t -> Config.t

config t is t's hash configuration.

module Proof : PROOF

type node_key

type value_key

type kinded_key = [

| `Node of node_key
| `Value of value_key

]

The type of references to tree objects annotated with the type of that object (either a value or a node). Used to build a shallow tree with Tree.shallow

module Tree : sig ... end

type ('proof, 'result) producer :=
  index ->
  kinded_key ->
  (tree -> (tree * 'result) Lwt.t) ->
  ('proof * 'result) Lwt.t

produce r h f runs f on top of a real store r, producing a proof and a result using the initial root hash h.

The trees produced during f's computation will carry the full history of reads. This history will be reset when f is complete so subtrees escaping the scope of f will not cause memory leaks.

Calling produce_proof recursively has an undefined behaviour.

type ('proof, 'result) verifier :=
  'proof ->
  (tree -> (tree * 'result) Lwt.t) ->
  (tree * 'result,
    [ `Proof_mismatch of string
    | `Stream_too_long of string
    | `Stream_too_short of string ])
    result
    Lwt.t

verify p f runs f in checking mode. f is a function that takes a tree as input and returns a new version of the tree and a result. p is a proof, that is a minimal representation of the tree that contains what f should be expecting.

Therefore, contrary to trees found in a storage, the contents of the trees passed to f may not be available. For this reason, looking up a value at some path can now produce three distinct outcomes:

A value v is present in the proof p and returned : find tree path is a promise returning Some v;
path is known to have no value in tree : find tree path is a promise returning None; and
path is known to have a value in tree but p does not provide it because f should not need it: verify returns an error classifying path as an invalid path (see below).

The same semantics apply to all operations on the tree t passed to f and on all operations on the trees built from f.

The generated tree is the tree after f has completed. That tree is disconnected from any storage (i.e. index). It is possible to run operations on it as long as they don't require loading shallowed subtrees.

The result is Error (`Msg _) if the proof is rejected:

For tree proofs: when p.before is different from the hash of p.state;
For tree and stream proofs: when p.after is different from the hash of f p.state;
For tree proofs: when f p.state tries to access invalid paths in p.state;
For stream proofs: when the proof is not consumed in the exact same order it was produced;
For stream proofs: when the proof is too short or not empty once f is done.

raises Failure
if the proof version is invalid or incompatible with the verifier.

type tree_proof := Proof.tree Proof.t

The type for tree proofs.

Guarantee that the given computation performs exactly the same state operations as the generating computation, *in some order*.

val produce_tree_proof : (tree_proof, 'a) producer

produce_tree_proof is the producer of tree proofs.

val verify_tree_proof : (tree_proof, 'a) verifier

verify_tree_proof is the verifier of tree proofs.

type stream_proof := Proof.stream Proof.t

The type for stream proofs.

Guarantee that the given computation performs exactly the same state operations as the generating computation, in the exact same order.

val produce_stream_proof : (stream_proof, 'a) producer

produce_stream_proof is the producer of stream proofs.

val verify_stream_proof : (stream_proof, 'a) verifier

verify_stream is the verifier of stream proofs.

type context = t

type memory_context_tree

memory_context_tree is a forward declaration of the type of an in-memory Irmin tree. This type variable is to be substituted by a concrete type wherever the TEZOS_CONTEXT signature is used.

val index : context -> index

val init : 
  ?patch_context:(context -> context Tezos_base.TzPervasives.tzresult Lwt.t) ->
  ?readonly:bool ->
  ?index_log_size:int ->
  string ->
  index Lwt.t

Open or initialize a versioned store at a given path.

The indexing_strategy, which determines whether newly-exported objects by this store handle should also be added to the store's index, is set to `Minimal by default.

val close : index -> unit Lwt.t

Close the index. Does not fail when the context is already closed.

val compute_testchain_chain_id : 
  Tezos_base.TzPervasives.Block_hash.t ->
  Tezos_base.TzPervasives.Chain_id.t

val compute_testchain_genesis : 
  Tezos_base.TzPervasives.Block_hash.t ->
  Tezos_base.TzPervasives.Block_hash.t

val empty : index -> t

Build an empty context from an index. The resulting context should not be committed.

val is_empty : t -> bool

Returns true if the context is empty.

val commit_genesis : 
  index ->
  chain_id:Tezos_base.TzPervasives.Chain_id.t ->
  time:Tezos_base.TzPervasives.Time.Protocol.t ->
  protocol:Tezos_base.TzPervasives.Protocol_hash.t ->
  Tezos_base.TzPervasives.Context_hash.t Tezos_base.TzPervasives.tzresult Lwt.t

val commit_test_chain_genesis : 
  context ->
  Tezos_base.TzPervasives.Block_header.t ->
  Tezos_base.TzPervasives.Block_header.t Lwt.t

val to_memory_tree : t -> string list -> memory_context_tree option Lwt.t

Extract a subtree from the Tezos_context.Context.t argument and returns it as a Tezos_context_memory.Context.tree (note the the type change!). *

val merkle_tree : 
  t ->
  Proof_types.merkle_leaf_kind ->
  key ->
  Proof_types.merkle_tree Lwt.t

merkle_tree t leaf_kind key returns a Merkle proof for key (i.e. whose hashes reach key). If leaf_kind is Block_services.Hole, the value at key is a hash. If leaf_kind is Block_services.Raw_context, the value at key is a Block_services.raw_context. Values higher in the returned tree are hashes of the siblings on the path to reach key.

val merkle_tree_v2 : 
  t ->
  Proof_types.merkle_leaf_kind ->
  key ->
  Proof.tree Proof.t Lwt.t

merkle_tree_v2 t leaf_kind key returns an Irmin Merkle proof for key (i.e. a proof that *something* is in the context at key). The proof is supposed to be produced by Irmin's produce_proof, and consumed by Irmin's verify_proof. The value embedded in the proof depends on leaf_kind. If leaf_kind is Block_services.Raw_context, the embeded value is the complete subtree in the context at key. If leaf_kind is Block_services.Hole, the embedded value is the hash of the subtree described above.

Accessing and Updating Versions

val exists : index -> Tezos_base.TzPervasives.Context_hash.t -> bool Lwt.t

val checkout : 
  index ->
  Tezos_base.TzPervasives.Context_hash.t ->
  context option Lwt.t

val checkout_exn : 
  index ->
  Tezos_base.TzPervasives.Context_hash.t ->
  context Lwt.t

val hash : 
  time:Tezos_base.TzPervasives.Time.Protocol.t ->
  ?message:string ->
  t ->
  Tezos_base.TzPervasives.Context_hash.t

val commit : 
  time:Tezos_base.TzPervasives.Time.Protocol.t ->
  ?message:string ->
  context ->
  Tezos_base.TzPervasives.Context_hash.t Lwt.t

val gc : index -> Tezos_base.TzPervasives.Context_hash.t -> unit Lwt.t

gc index commit_hash removes from disk all the data older than the commit_hash. Operations needing to checkout commits greater or equal to commit_hash will continue to work. Calling checkout h' on GC-ed commits will return None.

From the irmin point of view, a successful gc call on a commit_hash will result in a new prefix file containing that commit_hash as a root commit. This prefix file is considered as standalone as all the data referenced by that commit is contained in that file.

val wait_gc_completion : index -> unit Lwt.t

wait_gc_completion index will return a blocking thread if an Irmin GC is currently ongoing.

Warning: this currently only applies to GC started in the Irmin instance referenced as index; other opened instances will always return instantly.

val sync : index -> unit Lwt.t

Sync the context with disk. Only useful for read-only instances. Does not fail when the context is not in read-only mode.

val is_gc_allowed : index -> bool

is_gc_allowed index returns whether or not it is possible to run a GC on the given context tree. If false is returned, it means that the context was run, at least once, with the indexing strategy mode "always", which is not suitable for GC.

val split : index -> unit Lwt.t

split index creates a new suffix file, also called "chunk", into the irmin's file hierarchy.

To be optimal, the split function is expected to be called directly after committing, to the context, a commit (of hash context_hash) that will be a future candidate of a GC target. Thus, the commit commit_hash is the last commit stored on a given chunk. The GC called on that commit_hash will be able to extract that commit_hash into a new prefix file, and then, drop the whole chunk.

If the last commit of a chunk appears not to be the candidate of a future GC, it may result in keeping chunks containing partially needed data. This is not an issue, but it should be avoided to prevent storing unnecessary data and thus, to minimize the disk footprint.

val export_snapshot : 
  index ->
  Tezos_base.TzPervasives.Context_hash.t ->
  path:string ->
  unit Lwt.t

export_snapshot index context_hash ~path exports the context corresponding to context_hash, if found in index, into the given folder path. As the export uses the GC's behaviour to extract a single commit into a standalone fresh store, it is not possible to export a snapshot while a GC is running. This call will hang until the GC has finished.

Note: there is no associated import_snapshot function as the import consist in copying the exported store.

val set_head : 
  index ->
  Tezos_crypto.Hashed.Chain_id.t ->
  Tezos_base.TzPervasives.Context_hash.t ->
  unit Lwt.t

val set_master : index -> Tezos_base.TzPervasives.Context_hash.t -> unit Lwt.t

Hash version

val get_hash_version : 
  context ->
  Tezos_base.TzPervasives.Context_hash.Version.t

Get the hash version used for the context

val set_hash_version : 
  context ->
  Tezos_base.TzPervasives.Context_hash.Version.t ->
  context Tezos_base.TzPervasives.tzresult Lwt.t

Set the hash version used for the context. It may recalculate the hashes of the whole context, which can be a long process. Returns an Error if the hash version is unsupported.

Predefined Fields

val get_protocol : context -> Tezos_base.TzPervasives.Protocol_hash.t Lwt.t

val add_protocol : 
  context ->
  Tezos_base.TzPervasives.Protocol_hash.t ->
  context Lwt.t

val get_test_chain : 
  context ->
  Tezos_base.TzPervasives.Test_chain_status.t Lwt.t

val add_test_chain : 
  context ->
  Tezos_base.TzPervasives.Test_chain_status.t ->
  context Lwt.t

val remove_test_chain : context -> context Lwt.t

val fork_test_chain : 
  context ->
  protocol:Tezos_base.TzPervasives.Protocol_hash.t ->
  expiration:Tezos_base.TzPervasives.Time.Protocol.t ->
  context Lwt.t

val clear_test_chain : 
  index ->
  Tezos_base.TzPervasives.Chain_id.t ->
  unit Lwt.t

val find_predecessor_block_metadata_hash : 
  context ->
  Tezos_base.TzPervasives.Block_metadata_hash.t option Lwt.t

val add_predecessor_block_metadata_hash : 
  context ->
  Tezos_base.TzPervasives.Block_metadata_hash.t ->
  context Lwt.t

val find_predecessor_ops_metadata_hash : 
  context ->
  Tezos_base.TzPervasives.Operation_metadata_list_list_hash.t option Lwt.t

val add_predecessor_ops_metadata_hash : 
  context ->
  Tezos_base.TzPervasives.Operation_metadata_list_list_hash.t ->
  context Lwt.t

package octez-libs