Developed at thatDot, Quine is an open source streaming graph solution aimed at high-volume event processing. Quine combines graph data and streaming technologies to enable the creation of real-time, complex event processing workflows at scale, says thatDot.
Building an event-driven microservice for stream processing is a complex task, which combines hard challenges from both the database and distributed systems domains. These include consensus, concurrency, transactional logic, clustering behavior, fault tolerance, scalability, balancing read and write performance trade-offs, and more. Quine, the outcome of seven years of development, aims to simplify that.
What sets Quine apart from other stream processing solutions, says thatDot, is a set of three design choices that lie at its foundations: a graph-structured data model, an asynchronous actor-based graph computational model, and standing queries.
Quine's graph model can be thought of as a sort of graph database. Indeed, Quine uses Cypher, a popular graph query language originally developed at Neo4j. Similarly to SQL, Cypher is declarative in nature, thus allowing its users to express queries by specifying what they want to extract from a graph, rather than how to do it. In a graph, nodes hold key-value pairs, while edges connect node pairs and represent the relationships among nodes. Traversing the edge connecting two nodes is analogous to computing a join across tables in a relational database, only that join is pre-calculated.
The second key component in Quine is its computational model.
Ingested data, queries, or other instructions are inserted into the graph and propagate through the network of nodes and edges to compute the appropriate answer or trigger an action.
This is accomplished by means of Akka-based actors that process events in a highly parallel fashion. Actors in Quine are dispatched using a "fork-join' thread pool, which is the default dispatcher in Akka.
What makes Quine really innovative, though, says thatDot, is its third component: standing queries.
Built on the pillars of the graph data and computational models, standing queries in Quine eliminate the time-based challenges otherwise inherent to distributed systems.
Standing queries are persisted within the graph and are able to trigger actions whenever new data comes in. In particular, standing queries provide a solution to out-of-order data arrival.
According to thatDot, Quine architecture provides high read and write performance thanks to its actor model and to "semantic caching", which extracts information from the graph to know which data is worth keeping in the cache.
To help developers try out Quine, thatDot has created three recipes, including real-time blockchain back-propagation, CDN cache efficiency, and Kubernetes event observability.
Quine is available on GitHub under the MIT with Commons Clause license.