Esper is an event stream processing (ESP) and event correlation engine (CEP) unveiled this week with a 1.0 launch on Codehaus. Esper is a real time engine that triggers actions when event conditions occurs among event streams. It is designed for high volume event correlation where millions of events coming in would make it impossible to store them all to later query them using a classical database architecture.
Esper acts as a message consumer in a Java application, consuming immutable plain java beans sent by the application and acting upon them when matching conditions have occured. To evaluate the java beans, Esper compares against developer-written Event Query Language (EQL) statements which are matched against events as they come in and used to trigger POJO listener/handler classes. EQL allows the expressing of complex matching conditions that include temporal windows, and joining different event streams, filtering, sorting, grouping, etc:
Esper was founded by Thomas Bernhardt and emerged out of protypes built by Thomas for a large financial institution that had asked him to evaluate different rules engines for use in an application for monitoring a trading system. The rules engines proved cumbersome and slow for this usecase, what was really needed was real time event correlation engine. InfoQ spoke to Alexandre Vasseur who is helping Thomas grow the Esper community. Many may remember Alexandre as one of three committers on the AspectWerkz project (which has since merged with AspectJ), who later joined BEA.
Alexandre provided an example of computing the average stock price of the BEA tick on a sliding window of 30 seconds. Given a StockTick event bean with a price and symbol property and the EQL "select avg(price) from StockTick.win:time(30) where symbol='BEA'", a POJO would get notified as tick come in. Statements can be much more complex, and also be combined together with "followed by" conditions.
On the differences between Esper and messaging or rules engines:
Esper acts as a message consumer in a Java application, consuming immutable plain java beans sent by the application and acting upon them when matching conditions have occured. To evaluate the java beans, Esper compares against developer-written Event Query Language (EQL) statements which are matched against events as they come in and used to trigger POJO listener/handler classes. EQL allows the expressing of complex matching conditions that include temporal windows, and joining different event streams, filtering, sorting, grouping, etc:
Esper was founded by Thomas Bernhardt and emerged out of protypes built by Thomas for a large financial institution that had asked him to evaluate different rules engines for use in an application for monitoring a trading system. The rules engines proved cumbersome and slow for this usecase, what was really needed was real time event correlation engine. InfoQ spoke to Alexandre Vasseur who is helping Thomas grow the Esper community. Many may remember Alexandre as one of three committers on the AspectWerkz project (which has since merged with AspectJ), who later joined BEA.
Alexandre provided an example of computing the average stock price of the BEA tick on a sliding window of 30 seconds. Given a StockTick event bean with a price and symbol property and the EQL "select avg(price) from StockTick.win:time(30) where symbol='BEA'", a POJO would get notified as tick come in. Statements can be much more complex, and also be combined together with "followed by" conditions.
On the differences between Esper and messaging or rules engines:
Today, event based system are often confused with message based system such as JMS and alike backends or Enterprise Service Bus (ESB). Messaging system are event based but usually do not correlate events or process event streams at all. They simply process unitary events - and it becomes quickly complex if you need to correlate messages... [Esper] is not to be confused with classical rules engines...The internals of Esper are made up of fairly complex algorithms primarily relying on state machines and delta networks in which only changes to data are communicated across object boundaries when required.On the best uses and mis-uses for Esper, Alexandre replied:
Esper is best suited for realtime event driven applications. Typical application areas are business process management and automation, finance, network and application monitoring and sensor network applications. Esper take much of the complexity out of developing applications that detect patterns among events, filter events, aggregate time or length windows of events, join event streams, trigger based on absence of events etc.According to Alexandre, Esper is the only open source Event Stream Processing solution; but the initial version lacks high availability features in commercial ESP engines such as StreamBase or Coral8. Esper can run standalone, the 1.0 release assumes events to be POJOs. The Esper team is currently working on providing native XML format support.
Esper is not designed for storing and retrieval of fairly static data - that is better left to conventional databases. In-memory databases may be better suited to CEP applications than traditional relational databases as they generally have good query performance. Yet they are not optimized to provide immediate, real-time query results required for CEP and event stream analysis.
