Spring Integration in Action book, authored by Mark Fisher, Jonas Partner, Marius Bogoevici and Iwein Fuld, covers the Spring Integration framework which provides an implementation of the well-known Enterprise Integration Patterns based on the Spring programming model. It enables lightweight messaging in Spring-based applications and supports application integration using declarative adapters.
Authors start the discussion of Spring Integration framework with the core components of asynchronous messaging architecture: Message, Channel, and Message Endpoint. As they say in the first chapter, Spring Integration is where “Enterprise Integration Patterns meet Inversion of Control”. Enterprise Integration Patterns and how Spring Integration framework implements these patterns is discussed in detail with sample application and code examples.
The book also covers the support Spring Integration provides for Email and File adapters, Web services, and batch processing of files using the Spring Batch framework. To complete the discussion, authors talk about the monitoring and management of messaging components using JMX. Finally, the discussion in the book concludes with the Testing topic on how to test an asynchronous system and messaging components by mocking the services.
InfoQ spoke with Mark Fisher and Iwein Fuld about the book, Spring Integration framework, its strengths and limitations.
InfoQ: There is a growing trend to use the application integration as a cloud based service. Can you talk this new concept of "Integration as a Service" and how Spring Integration framework helps?
Iwein: Spring Integration in itself is not anything "as a Service". This is a good thing. As a developer, you are interested in integrating with other *aaS API's and having an ESB like service in the mix usually doesn't simplify things. It would be easy enough to use Spring Integration in a service architecture, and why not in the cloud. The sweet spot of SI is integration, not service. That said, Cloud Foundry has native support for Spring apps and thus Spring Integration, and it provides services that Spring Integration supports - such as RabbitMQ, Redis, MongoDB, and relational DBs - so the possibility and examples are at your fingertips.
InfoQ: Are NoSQL data stores a better choice than the relational databases to use in messaging architectures, for example to store messages in Queue Channels or Messages Stores? What are the pros and cons of using NoSQL databases for the persistence purposes in messaging applications?
Iwein: If you're using a NoSQL store it is usually a bit better optimized for writes and simple 'by key' queries. This fits more natural with the messaging model. That said, it is not so hard to optimize a relational database for such a usage, so the choice of using a NoSQL store should really not be driven by the choice of Spring Integration as a framework for messaging.
Mark: Also, NoSQL stores are not one-size-fits-all, and so the more interesting discussion is about Spring Integration supporting a wide-variety of use-cases by providing adapters for a number of different NoSQL stores. For example, you can easily update scores in a Redis ZSET in a message-driven way using a Channel Adapter we added in version 2.2.
InfoQ: Big Data, Data Mining and Data Processing are getting lot of attention lately to help process and analyze different types of business data in an efficient and effective manner. How does Spring Integration along with Spring Batch framework fits into the Big Data space?
Iwein: If you want a quick result on throw-away code, Spring Integration and Spring Batch are probably not for you. However, if you want a maintainable solution they are. The Spring portfolio products shine in a long lived application where maintainability trumps quick time to market. In the Big Data space things are very volatile. Often solutions are meant to be done quick, return on their investment and be thrown away. If that applies to your solution, it might be sensible to skip Spring Integration.
Mark: However, we are beginning work on version 3.0 where one of the main themes is Big Data support. That will include not only adapters for HDFS and seamless integration with the Spring for Apache Hadoop project, but it might also include such features as applying the disruptor model to our socket level connection factories. That combined with Spring Integration's low-overhead should meet the throughput demands of many Big Data solutions. Also, one thing we have noticed while exploring "real world" Big Data scenarios is that the ultimate problem to solve is usually an integration problem: combining data from multiple sources, defining flows across different systems, creating pipelines of multiple processing steps, and so on. Obviously the same tools and frameworks can be used there as with the "classic" Enterprise Integration.
InfoQ: Security is an important aspect of enterprise application development especially in integration use cases. Can you discuss the support Spring Integration provides to secure various messaging components? What are the best practices in this area?
Iwein: Security in messaging is either done on transport level, or on message level, or both. It's kind of cool that Spring Integration manages to side step these issues on both fronts. Since below the SSL https is just http, SI doesn't need to know. On the message level Spring Integration delegates to Spring WS' security components. These are fine, so again we've dodged the bullet. In my opinion, if you care about security, you secure the message. That leaves you to make sure the trust between ultimate receiver and ultimate sender is clean, instead of taking responsibility for the total network topology. Many organizations go a different route and ignore the message level security entirely because 'it's all taken care of on the transport level'. This is as stupid as assuming that the guy installing your OS doesn't install a keylogger; it is a wide-spread practice. Best practice I know in security is always to keep your wits and not trust in best practices.
Mark: Spring Integration does provide a Channel Interceptor that simply delegates to a Spring Security authorization manager, but again, that typically checks the SecurityContext which would have been populated based on information available at the transport level.
InfoQ: You discussed the testing techniques in the book on how to test various components in messaging applications. Can you talk about some of these techniques and how Spring and Spring Integration frameworks make it easier to test the messaging components?
Iwein: Chapter 18 is rather clear about this. The most important eye-opener I had when working on SI test support and that chapter was that using a debugger is a really bad idea when you're trying to figure out how a multithreaded application works. After figuring that out I stopped using debuggers and went on to follow my own advice of fixing the tests and the logging instead. It's mind boggling how much time is wasted learning things about code in a debugger and then throwing that knowledge away as soon as it is discovered.The other thing worth mentioning is that most concurrency issues _can_ be provoked from a carefully crafted test case. Mark and I have had some late evenings where we couldn't reproduce each others issues, but if you are willing to walk the walk, there is always a way to a deterministic test case. You should be prepared to spend more time on this than on a general functional test, because these are the tests that will save your life when it counts.
InfoQ: What are the limitations of EIP patterns or what should the integration architects and developers keep in mind when they use these patterns in their applications?
Iwein: The problem with writing things in a book is that you're not forced to check if they are non-ambiguous enough to be parsed and compiled into real code. We've struggled with that several times in our book, but we've also struggled with the EIP book when implementing SI. When designing architecture, the architect should remember that the code _will_ look different, and the developer should remember that the emerging architecture needs to be as clean as the low level code. EIP are more of a guideline to get started, working software trumps that, even if the patterns look different in code than in the book.
Mark: Furthermore, when it comes to design patterns, it's often the language that is most important. That's what allows developers to make connections between what they've learned and how it's manifested in a given API. Therefore, whenever struggling to hit upon the right name for a property or a configuration attribute, we consult the EIP book and hope to be inspired by words used there. Consistency is also important; so whenever faced with a challenge of mapping a particular EIP into our API, we try to follow similar techniques and use similar terminology.
InfoQ: Can you talk about the emerging trends in enterprise application integration space?
Iwein: The major trend is (still) REST. Http is only beginning to be used to its full potential now that JavaScript frameworks and browsers have caught up with the spec. Another interesting trend I saw a few times now is to optimize for efficiency by moving far away from the established way of doing things. This can be useful for mobile devices, flaky networks; but also for unlikely devices like home energy measurement equipment for example. Spring Integration's TCP/UDP adapters might come in handy in such cases.
Mark: In addition to REST APIs, web-based messaging via Web Sockets and higher level abstractions such as SockJS are also increasingly popular. Those techniques open up a whole new world of opportunities for application integration where the client/server relationship is changing dramatically. This is another area of focus for us on the Spring Integration 3.0 roadmap.
Authors also talked about application integration in general and best practices.
Iwein: I'm not in favor of complicating applications, even if it is done by reusing my code. Many developers think that they will need asynchronous processing, so they use SI, Camel, or Akka. In the end it often turns out that no serious measurements were done to see if a single thread solution would have cut it. Don't be that fool because you can use a cool technology. Don't get me wrong, SI is awesome, but if you can get away without concurrency, you're always better off.
Mark: That said, when the message-driven model *does* fit your application's needs, Spring Integration hides most of the complexity. For optimal performance, you still need to understand how to configure your thread pools and triggers, but you do not need to write the code that handles task scheduling and asynchronous invocations. Instead, as with any Spring application, you focus on the POJOs of your own business domain. That also means your code is decoupled from infrastructure in a way that facilitates testing, and that's one 'best practice' upon which we can probably all agree.
About the Books Authors
Mark Fisher is the founder of the Spring Integration project. Currently at VMware, he continues to lead the development of Spring Integration while exploring the intersection of big data and messaging. He has been a committer on a number of Spring projects, including the Spring Framework itself and Spring AMQP, which he cofounded. Mark speaks regularly at conferences and user groups about messaging, data, integration, and cloud computing.
Iwein Fuld is an independent consultant, focusing on on high-quality development and coaching teams. He’s a jack-of-all-trades, but keeps coming back to server integration problems and algorithms. Iwein has been a committer on the Spring Integration project since early 2008. Apart from being an expert on TDD, concurrency, and messaging, Iwein especially enjoys building agile teams and lean startups.
