github.com/mirage/irmin ↗

About mirage/irmin

Irmin is an OCaml library for building mergeable, branchable distributed data stores based on the design principles of Git. It applies version-control system concepts to large-scale distributed data, providing Git-like operations such as clone, push, pull, branch, and rebase, but optimized for automatic programs performing high-speed operations with fully-automated conflict handling. The library is highly customizable, allowing developers to define custom data types, storage backends (in-memory, disk, Redis, browser storage, etc.), and event-driven behaviors for programmable dynamic dataflow pipelines.

Irmin was created at the University of Cambridge in 2013 as the default storage layer for MirageOS applications. Rather than a complete database engine, it is a collection of composable libraries designed to address challenges raised by the CAP Theorem. The core consists of well-defined low-level data structures for persistence and synchronization, with efficient synchronization algorithms and higher-level data structures. Some components have formal semantics and support Conflict-free Replicated Data-Types (CRDTs). Irmin makes minimal OS assumptions, making it highly portable across environments from Linux to web browsers to Xen unikernels.

The project has seen substantial real-world adoption across three categories: as a portable structured key-value store (used by XenStore, Cuekeeper, Caldav, Tezos, and others), as a distributed database backend (IMAP servers, ARP caching, Cassandra-based caching), and as an event-driven dataflow engine (Datakit CI, OCurrent). Irmin is primarily maintained by Tarides and includes packages for various storage backends like Git, filesystem, chunked storage, pack format, and GraphQL interfaces.