Persistent block store with arborescent history
The floating block store is an append-only store where blocks are stored arbitrarily. This structure possess an indexed map Block_hash.t -> (offset × predecessors) which points to its offset in the associated file along with an exponential list of predecessors block hashes (as implemented in Block_store.compute_predecessors). The structure access/modification is protected by a mutex (Lwt_idle_waiter) and thus can be manipulated concurrently. Stored blocks may or may not contain metadata. The instance maintains an opened file descriptor. Therefore it must be properly closed or it might lead to a file descriptor leak.
Four different kind of instances are allowed to co-exist for an identical path: - RO, a read-only instance; - RW, a read-write instance - RO_TMP, RW_TMP, read-write instances; - Restore is a generic instance used to wrap leftover instances while fixing a crashed storage. See Block_store.
This store is expected to respect the following invariant:
The floating block store is composed of the following files:
Block_repr.t:| <block> * |
where <kind> is RO(_TMP), RW(_TMP) (see Naming) and <block>, a Block_repr.t value encoded using Block_repr.encoding (thus prefixed by the its size).
The type for the kind of floating store opened.
val kind : t -> floating_kindkind floating_store returns the floating store's kind.
val mem : t -> Tezos_crypto.Block_hash.t -> bool Lwt.tmem floating_store hash tests whether hash is stored in floating_store.
val find_predecessors :
t ->
Tezos_crypto.Block_hash.t ->
Tezos_crypto.Block_hash.t list option Lwt.tfind_predecessors floating_store block_hash reads from the index the list of block_hash's predecessors if the block is stored in floating_store, returns None otherwise.
val read_block : t -> Tezos_crypto.Block_hash.t -> Block_repr.t option Lwt.tread_block floating_store hash reads from the file the block of hash if the block is stored in floating_store, returns None otherwise.
val read_block_and_predecessors :
t ->
Tezos_crypto.Block_hash.t ->
(Block_repr.t * Tezos_crypto.Block_hash.t list) option Lwt.tread_block_and_predecessors floating_store hash same as read_block but also returns the block's predecessors. Returns None if it fails to resolve the given hash.
val append_block :
?flush:bool ->
t ->
Tezos_crypto.Block_hash.t list ->
Block_repr.t ->
unit Lwt.tappend_block floating_store ?flush preds block stores the block in floating_store updating its index with the given predecessors preds and flushing if flush is set to true (defaults to true).
val append_all :
t ->
(Tezos_crypto.Block_hash.t list * Block_repr.t) Seq.t ->
(unit, Tezos_error_monad.TzCore.error list) result Lwt.tappend_all floating_store chunk stores the chunk of (predecessors × blocks) in floating_store updating its index accordingly.
val iter_s_raw_fd :
(Block_repr.t -> (unit, Tezos_error_monad.TzCore.error list) result Lwt.t) ->
Lwt_unix.file_descr ->
(unit, Tezos_error_monad.TzCore.error list) result Lwt.titer_s_raw_fd f fd unsafe sequential iterator on a file descriptor fd. Applies f on every block encountered.
Warning: should be used for internal use only (e.g. snapshots).
val fold_left_s :
('a ->
Block_repr.block ->
('a, Tezos_error_monad.TzCore.error list) result Lwt.t) ->
'a ->
t ->
('a, Tezos_error_monad.TzCore.error list) result Lwt.tfold_left_s f e floating_store sequential fold left on the floating_store using f on every block and e as initial element. The function f is given the last read block.
val fold_left_with_pred_s :
('a ->
(Block_repr.block * Tezos_crypto.Block_hash.t list) ->
('a, Tezos_error_monad.TzCore.error list) result Lwt.t) ->
'a ->
t ->
('a, Tezos_error_monad.TzCore.error list) result Lwt.tfold_left_with_pred_s f e floating_store sequential fold left on the floating_store using f on every block and e as initial element. The function f is given the last read block along with its predecessors.
val iter_s :
(Block_repr.t -> (unit, Tezos_error_monad.TzCore.error list) result Lwt.t) ->
t ->
(unit, Tezos_error_monad.TzCore.error list) result Lwt.titer_s f floating_store sequential iterator on the floating_store. Applies f on every block read.
val iter_with_pred_s :
((Block_repr.t * Tezos_crypto.Block_hash.t list) ->
(unit, Tezos_error_monad.TzCore.error list) result Lwt.t) ->
t ->
(unit, Tezos_error_monad.TzCore.error list) result Lwt.titer_with_pred_s f floating_store sequential iterator on the floating_store. Applies f on every block with its predecessors read.
val init :
[ `Chain_dir ] Naming.directory ->
readonly:bool ->
floating_kind ->
t Lwt.tinit ~chain_dir ~readonly kind creates or load an existing floating store at path chain_dir with kind.
Warning If readonly is not set, a RO instance is writable.
close floating_store closes floating_store by closing the index and the associated opened file descriptor.
swap ~src ~dst closes src and dst then overwrites dst files with src's ones.
Warning Non-atomic swap.
val append_floating_store :
from:t ->
into:t ->
(unit, Tezos_error_monad.TzCore.error list) result Lwt.tappend_floating_store ~from ~into takes two opened floating block stores and appends all blocks contained in from to into.
Integrity checks
val all_files_exists :
[ `Chain_dir ] Naming.directory ->
floating_kind ->
bool Lwt.tall_files_exists ~chain_dir kind checks that the files to the associated kind of floating store are present in the chain_dir directory.
val fix_integrity :
[ `Chain_dir ] Naming.directory ->
floating_kind ->
(unit, Tezos_error_monad.TzCore.error list) result Lwt.tfix_integrity ~chain_dir kind error fixes the integrity of the floating block stores by removing corrupted data and restoring a consistent state.