Apache Ignite is an in-memory computing platform with transactional support, that supports both key-value persistence as well as streaming and complex-event processing. Ignite was open-sourced by GridGain in late 2014 and accepted in the Apache Incubator program.
InfoQ interviewed Nikita Ivanov, CTO of GridGain, to find out more about Apache Ignite.
InfoQ: Apache Ignite offers a plethora of in-memory functionality, from key-value store to stream processing. What are the flagship use cases?
Nikita Ivanov: The flagship use case for Apache® Ignite™ is any high-performance task in financial services, especially financial services and fintech companies. For these companies, there is a direct link between increased revenue and application performance and scalability. They derive immediate value from the high performance, low latency and scalability that GridGain and Ignite offer. Within these companies, there’s a wide range of applications that Ignite is supporting, including automated trading, real-time regulatory compliance, spread betting, fraud detection, back testing, analytics and more.
Apache Ignite is also frequently integrated into third party software or SaaS solutions. Two verticals that are growing rapidly are SaaS and IoT vendors, which have business models requiring the highest levels of performance and scalability in order to deliver an optimal user experience or meet SLAs for web-scale applications or data-intensive Internet of Things applications.
InfoQ: How does Apache Ignite complement popular in-memory computing frameworks such as Apache Spark, and how does it compare to Alluxio?
Ivanov: Apache Ignite complements Apache Spark in several ways. Apache Spark is a non-transactional, read-only system that is used primarily for analytics, especially machine learning. Apache Ignite is used primarily as a high performance transaction-based system. When used together, Apache Ignite can serve as an in-memory storage layer for Spark RDDs, allowing users to share state between Spark jobs. In addition, Ignite indexes any Spark data stored in-memory. When a SQL query is run on Spark, it performs a full scan of the data while the Ignite index allows much faster performance.
As far as Alluxio, we have never seen it in a competitive engagement so it’s very difficult to assess how it compares.
InfoQ: In December of 2016, Hollow, a disseminated cache from Netflix, was made open-source to embed caching capabilities into clients and reduce request latency and RPCs required. Can something similar be achieved with Apache Ignite?
Ivanov: While we have also never seen Hollow in a competitive engagement, I am more familiar with its capabilities, and nearly everything that Hollow offers is already supported by Apache Ignite.
InfoQ: One of the big advantages of Apache Ignite is to provide transactionality and consistency guarantees. Can you comment on Ignite's design decisions around CAP theorem?
Ivanov: We believe that products that support only the CAP theorem are inferior to Ignite. When you have support for only eventually consistent (EC) transactions, you can never get to full consistency. You can work to improve the EC – by getting faster – but you can never get to full consistency. So, if you have a classic transactional system, say you need to move money or manage accounts, EC solutions like Apache Cassandra, MongoDB, Apache CouchDB, etc., simply can’t be used because they don’t offer full consistency.
Ignite starts with full-blown, two-phase commit transactions for full consistency – indistinguishable from what Oracle does. But users can configure the system to relax the two-phase commit protocol and get back all the performance of an EC solution. With Ignite you get the best of both worlds – the ability to configure the full transactional behavior of a standard database or trade off some consistency for greater performance. According to our customer feedback, this flexibility is one of the greatest strengths of GridGain.
InfoQ: Does Apache Ignite support optimized in-memory columnar formats better suited for analytical workloads?
Ivanov: No, Apache Ignite does not support columnar format.
InfoQ: What’s on the roadmap for Apache Ignite?
Ivanov: The two most important developments on the Apache Ignite roadmap are, first, much improved and much stronger disk-based storage for use cases where the entire database will not fit in RAM, and second, a major push to improve support and compatibility for machine and deep learning capabilities. The specifics for these will come in a couple of major announcements.