Subtree has released Dotmesh, a container-friendly application state snapshotting tool that provides a git-like CLI for manipulating and sharing the captured data. The core use case for Dotmesh is the ability to capture, manage and share state from cloud-native and microservice-based applications in order to facilitate the debugging and exploration of problematic states seen within QA and production environments.
Dotmesh is a snapshotting tool for databases and other filesystem states, and provides a "git-like CLI for capturing, organising and sharing application states". These application states are captured in "datadots", and can be stored and shared via a centralised "dothub" repository. Dotmesh can capture the state of multiple databases, each one in a "subdot", within a single atomic commit -- this can be useful for capturing state (at an arbitrary point in time) when working with a microservice-based application where the state of an environment is distributed over several components e.g. multiple data stores, caches and queues.
The documentation provides a simple example, and demonstrates how start a Dotmesh-backed PostgreSQL container:
docker run -d --volume-driver dm -v myapp:/var/lib/postgresql/data --name postgres postgres:9.6.6
This creates a datadot called "myapp", creates the writeable filesystem for the default master branch in that datadot, mounts the writeable filesystem for the master branch into /var/lib/postgresql/data within the postgres container, and starts the PostgreSQL container. A commit of a datadot can be made by running dm commit -m "empty state". This creates a point-in-time snapshot of the state of the filesystem on the current branch for the current dot.
Data can be added to the PostgreSQL database, and another commit made. An engineer can confirm the commit by running dm log, and can roll back to the first commit with dm reset --hard HEAD^ (the HEAD^ syntax should be familiar to engineers that have used git, and means "one commit before the latest commit on the current branch"). Rollbacks and fast forwards can also be made by commit id. Engineers can push data to a centralised "dothub" Dotmesh repository (registration is required to utilise the current Dothub SaaS offering), and can also clone and pull from other available dots.
Microservice-based applications often have more than one stateful component (sometimes even for a single service) for example, multiple databases, caches and queues. A datadot can capture all of those states in a single, atomic and consistent commit. Subdots can be thought of as different "partitions" of the master branch's writable filesystem, in the sense that they divide up the filesystem so that different containers can use different independent parts. Commits and branches of a datadot apply to the entire datadot, not specific subdots. This means that datadot commits can represent snapshots of the state of an entire application, not the individual data services.
Dotmesh stores datadot content in ZFS, a combined file system and logical volume manager that was originally designed by Sun Microsystems. The features of ZFS include protection against data corruption, support for high storage capacities, efficient data compression, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones. Many Linux container implementations support mounting ZFS volumes directly, including Docker -- the ZFS on Linux (ZoL) port is healthy and maturing -- however, at this point in time it is not recommended to use the zfs Docker storage driver for production use unless "you have substantial experience with ZFS on Linux". Dotmesh aims to provide the required expertise captured within their tooling.
Dotmesh nodes are organised into clusters, and the "mesh" that weaves the nodes together within a cluster is etcd. Each node also runs the Dotmesh server -- the Dotmesh documentation currently includes instructions for installing the Dotmesh server on Docker, a generic Kubernetes cluster, and a Google Container Engine (GKE) cluster. Any node within a cluster sees and operates upon the same list of Dots, and Dotmesh will move the physical data underlying the datadots between nodes in a cluster as they are needed.
All the commits on all the branches get replicated to every node automatically. The only thing that isn't replicated is the uncommitted "dirty" state of each subdot on each branch -- this is stored on a single node, known as the "master node" for that branch. Commiting the dirty state ensures that the data is replicated to multiple nodes.
Communication between server processes within the cluster is via two means: shared state in etcd, which communicates between nodes using port 42380; and via HTTP using port 6969. The documentation states that communications via HTTP on port 6969 aren't encrypted or protected from attackers in any meaningful sense. Accordingly, these ports must be locked down within a cluster, and a VPN used if a cluster is to be extended over untrusted networks.
The Dotmesh FAQ includes information on how the tool differs from other container and cloud-native storage offerings, such as Portworx and Rook:
Portworx and Dotmesh are really tackling different problems. We believe that Portworx is tackling storage for containers in production, with synchronously replicated block storage, the adoption of which will typically be an operations decision. This is great and it's a necessary component for many customers, especially on-prem where you don't have technology like EBS or GCE PD available (or where that technology is lacking).
The FAQ states that the Dotmesh team are not attempting to build another synchronously replicated block storage system, and are instead "tackling the broader, more workflow-focused problem of getting the data you need to the right place throughout the software development life cycle". Their vision is to be able to "capture, organise and share application states, i.e. snapshots of entire applications", with a particular focus on the developer workflow with building microservice-based applications that embrace polyglot persistence to manage state.
More information on Dotmesh can be found on the project's website and GitHub repository, and developers can experiment with the tooling via an interactive Katacoda tutorial.
