Rossen Stoyanchev on Reactive Programming with Spring 5 and Spring WebFlux

• 00m:35s Rossen is a senior staff engineer with Pivotal, and a core contributor to the Spring framework.
• 00m:45s His focus over the last few years has been primarily with the web tier with Spring, most recently integrations with reactor and Web Flux in Spring 5.
• 00m:55s Rossen discusses blocking and non-blocking architectures, upcoming changes with Spring 5 including reactor integrations, the new reactive non-blocking web tier Web Flux, that leverages an event style loop paradigm; discusses the differences between Reactor and RxJava.

What’s the main things you’ve been working on recently?

• 01m:30s We’ve been focussing on Spring Framework 5 for over a year; the actual preparations started even earlier. We are getting close to the release candidate stage. [Should be out by the time this podcast goes live - Ed.]
• 01m:55s The GA [General Availability] for Spring Framework 5 is scheduled for June 25.
• 03m:00s We’ve been working on a set of assumptions for the last 20 years of what a Web framework should look like, such as SpringMVC.
• 03m:20s We’re looking at that sort of offering but based on a whole new set of assumptions about what the architecture should look like.

What’s the difference between asynchronous processing and multithreading?

• 04m:00s The key differentiation is in the execution model. 
• 04m:15s The basic assumption of web frameworks [in Java] is that you have a Servlet which calls back with a blocking thread, served from a set of threads in a thread pool.
• 04m:25s The servlet container handles the concurrency and the individual Servlet doesn’t need to worry bout.
• 04m:40s This leads to a number of limitations and restrictions in what it can’t do, which may not be necessary any more in the world of distributed systems.
• 05m:20s When you base your design on a single thread processing a request, it makes it very easy to do. But if the thread needs to do anything asynchronous then it needs to block the synchronous part.
• 05m:30s Even if you’re calling a database, you have a synchronous block waiting for the return result to come back.
• 05m:45s Now it’s very common to call other REST services and orchestrate processing which means you’re dealing with asynchronous services.
• 06m:05s There are libraries to do asynchronous HTTP calls - as far as making that a first class citizen in the web application programming model we’re some way off.
• 06m:20s The Servlet API model has evolved - for example, Servlet 3 we saw the ability to do asynchronous request processing, where we aren’t tied to the thread initially given to the servlet
• 06m:50s If you have to call three other services, you can return to the Servlet container and then subsequently complete the response.
• 07m:00s There’s another change in Servlet 3.1 where non-blocking IO was added but that is a lot more difficult to add into existing applications.
• 07m:15s Non-blocking IO goes really deep, and to be able to take advantage of it you end up rewriting most of your application in any case.
• 07m:30s For example, something as innocent as getting the Servlet parameters from the API is a blocking call, because it may involve parsing the request.
• 07m:45s There’s a good talk by Greg Wilkins, which says that you can’t mix the two modes of execution.
• 08m:00s If you look at the traditional Servlet processing model, you have a set of Filters and then a Servlet at the end.
• 08m:05s When you get to the end, the actual part of processing the request, if the client is not able to process the bits you’re writing, you have to be able to pause and come back later. That’s a deep change which cannot be done without forgetting many of the existing assumptions that we’ve had for some time.

What are some use-cases of blocking multithreading approach versus non-blocking?

• 08m:55sIt’s never easy to separate easily; it can be mixed. But there is no denying it is becoming an important trend.
• 09m:15s I think the situations in where it becomes important is where you deal with increased latency in processing requests.
• 09m:30s When your application is calling other services and they take some time to respond, it’s useful if that can be non-blocking.
• 09m:45s It becomes more important if you’re orchestrating, because you can declare what calls you want to make (and in which order) so that you absorb the latencies.
• 10m:10s The other part is when you’re writing back to the clients, there is latency outside of your control.
• 10m:25s If the clients are on a slow clients on a network, then you can’t fine-tune the number of threads in a thread pool that can handle that.
• 10m:55s If you think of thread pools as the solution, you need to have thread pools at different levels.
• 11m:15s The more you find yourself in the situation of managing a bunch of thread pools which aren’t easy to predict, which means you can’t be resilient, scale, degrade or fail gracefully.

What did Node.JS bring to the game?

• 12m:00s Node.JS brought to the forefront the whole notion of what you could do when you didn’t have a thread pool to manage.
• 12m:15s In Node.JS’ case, it was a restriction of dealing with limitations - you didn’t have multiple threads. So you have to do the most with what you have.
• 12m:25s It shows if you must use a few threads, you can’t make synchronous blocking calls. That event loop style processing leads naturally to asynchronous processing.
• 12m:55s You try to have one process per CPU core, and then not use any additional threads but deal with asynchronous threads.

What are some of the gotchas with using asynchronous APIs?

• 13m:15s It brings significant changes to the way that you program and the impact on the language.
• 13m:35s Consider an API which gives you back an asynchronous result, like a Future, CompleteableFuture or Promise, where it gives you back a result or an error.
• 13m:50s If you think of this as a kind of event, then you can imagine if you have this many parts of your applications, then it becomes much more event driven.
• 14m:10s Spring Integration introduced this kind of channel-based and event-driven oriented processing, where you’d have a channel for errors and a channel for results.
• 14m:20s It allows you to deal with asynchronousity differently, because you can introduce it later because it lends itself to asynchronous processing.
• 14m:35s In many ways we’re talking about doing this as the core of the programming model, because RxJava and Reactor are all pointing the way to do this.
• 14m:45s We’ve also seen this with Java 8, with lambda-style programming that enables a functional style programming approach, making it much easier to work with these style of APIs.
• 15m:20s This kind of declarative way of breaking down your logic (taking outside the realm of just collections) can be the foundation on which you generally deal with asynchronous and non-blocking logic.
• 15m:35s This means you’re taking an algorithm, splitting it up into pieces, because each one of those pieces may need to pause before triggering the next part.
• 15m:45s It’s the same as using a Stream to process the contents of a Collection, it becomes easier to be parallelise or distributing processing across threads.  

How do Reactive Streams and RxJava compare to Spring 5?

• 16m:50s We’re talking about a small and focussed specification that can become the foundation for asynchronous libraries and component providers, so they have an API and a protocol for how to deal with asynchronousity, and especially with an event driven system with overflow of events, to ensure that a producer of data cannot overwhelm a consumer of that data.
• 17m:15s If you have a Java 8 Collection, you have a finite amount of data. But if you have a reactive stream, you have a potentially infinite number of items in a producer.
• 17m:40s So for example, in a full stack web application, you have a data provider, a web application and an HTTP client. 
• 18m:00s You want to ensure that there is a single chain of events from one end to another; and if the downstream cannot continue (because it is blocking), you want to send the signal upstream to indicate that they need to slow down.
• 18m:45sThat is the main problem that reactive streams tries to solve, which has been embraced in the Spring framework in a major way, and a lot of different Spring projects are working together with the reactive project giving reactive streams a very prominent place.
• 19m:00s From the Spring Framework perspective, we are to some degree agnostic; the main thing we focus on is reactive streams as the main contract.
• 19m:10s What we found as an experience in building our own reactive non-blocking web stack - the parallel to SpringMVC, what we call Spring Web Flux - what we found is that ew cannot simply build on reactive streams (which are just four interfaces) - they solve a simple protocol problem between disparate provides. But each library needs to choose its own reactive facilities. For us it was the reactor project. We found that we were doing a lot of low level reactive things to be able to support reactive streams - and doing that without a library is just reinventing the wheel.
• 20m:10s We’re using reactor quite extensively as an adaptation to the different HTTP runtimes, including Servlet 3.1 containers, Netty and Undertow.
• 20m:20s We’re also using reactor to compose and declare our own logic, and then on top of that, as our own web framework like SpringMVC supports controllers to be written in different flexible styles, different method signatures, that allows us to seamlessly support RxJava as well.

What’s the distinction between RxJava and Reactor?

• 21m:30s Reactor 3 and RxJava 2 are in many ways closely aligned; there’s a lot of collaboration between the projects. They also both participate in the Reactive streams discussions
• 21m:55s RxJava comes from the background of RX, which is a set of patterns that originated at Microsoft by Erik Meijer and have been adopted by a number of different languages.
• 22m:10s A lot of people come across this with the UI angle, or perhaps with the JavaScript version with RxJS, so they naturally follow into RxJava.
• 22m:20s The relationship between RxJava and Reactive Streams is a little different because it comes from a different angle.
• 22m:35s Its more oriented to server-side processing, with back-pressure. I think Erik Meijer thought the idea of back-pressure wasn’t doing it right; however, in a user interface environment, provided that you respond to the UI quickly enough you don’t need back-pressure.
• 23m:10s In server-side applications, the concept of back-pressure fits more naturally; if you’re dealing with data and databases, you are already familiar with talking to databases and IO.
• 23m:30s Reactor is in a really good position, because it is philosophically rooted in reactive streams first and RxJava second; but it borrows all the patterns from RxJava so you will find they are very similar. 
• 23m:50s If you go to the alphabetical list of names in RxJava, you’ll find that other languages have different names for the same things.
• 24m:00s In that sense, Reactor has more flexibility; it doesn’t have that legacy.
• 24m:15s If you look at the types you have Flux and Mono, which correspond to multiple elements or a single element (like Java’s Optional). It’s a very simple API story, especially for data APIs which can return a Flux or a Mono. However, RxJava has separate interfaces for Flowable and Observable.

What is the relationship between SpringMVC and Web Flux?

• 25m:00s Spring MVC is not a separate framework; it’s part of the Spring Framework, and it’s been that way from the beginning.
• 25m:30s MVC is just a convenient name for the Spring Framework servlet framework.
• 25m:55s We couldn’t use the core contracts of Spring MVC because they follow the same assumptions of the Servlet API, along with the (blocking) InputStream and OutputStream.
• 26m:10s We knew that we could reuse the algorithms and concepts from SpringMVC, so we created Spring Web Flux to do this.
• 26m:30s It used to be called Spring Web Reactive, which is a more friendly name when you first hear it, but it isn’t easy to remember or pronounce - which is how we came up with Spring Web Flux.

So WebFlux doesn’t replace SpringMVC?

• 28m:20s They are both modules that sit side by side; they share many mechanisms, and there is a lot of code in the Spring Web module, such as HTTP message conversion.
• 28m:40s Obviously SpringMVC isn’t depending on Web Flux directly, but they benefit both sides when they’re used.
• 29m:15s If you look at the Web Flux module, it’s not only about the server-side programming model; it contains server-side endpoints and controllers that support Flux and Observable; but it also has a functional server-side endpoint programming model, which is going to look and feel different to what most programmers are used to.
• 29m:45s We also have a functional web client, which is going to be a very interesting feature for those who are currently using SpringMVC and aren’t warming up to Web Flux, but who need an http client; and being able to orchestrate calls to other http services with ease and functional programming aspects is exciting to SpringMVC users as well.
• 30m:30s Even if you’re not going to use Web Flux in your Spring MVC controllers, you can still use the web client and use Flux that way.
• 30m:50s The other thing we’ve done is we support RxJava and Reactor first class where we can within SpringMVC as a return value, so if you happen to be making a call through the web client which returns a Flux or an Observable, when you return that value and we will decouple the thread from your servlet container and take the Flux output as it comes, and do server-send events from the client.
• 31m:20s We think that’s the way a lot of people are going to experience it initially, and as their comfort level grows, for some it’s going to be a different experience - there’s a certain audience for whom going all out reactive is an important topic.
• 31m:35s We want to be there on both sides, and I think that’s the unique proposition - we’re offering ways that you can take this at different levels of commitment. 

What is the upcoming QCon talk about?

• 32m:00s It’s very common in a web application to make calls to other services, so if you’re using the web client you’re orchestrating, so that’s one common scenario.
• 32m:20s We’ll be looking at what the different processing model is for using Web Flux and a Servlet-stack based Spring MVC application, and how does it affect the use of the Servlet thread pool in the container.
• 32m:30s What is Web Flux going to buy you in comparison to using the Spring MVC environment - I think that’s an angle that’s going to be interesting to most people.
• 32m:50s Another use case is the use of Spring Data, or other data provider like Mongo, Cassandra or Couchbase, then you can use their async drivers and Spring Data is gearing up support for doing that and essentially you write an interface and you gain Flux or Observable for your data.
• 33m:20s There are also other scenarios with streaming data between client and server, and what is the impact of that, so that’s some of what we’ll be talking about.

Resources

If you want to find out more about Reactive programming with Java we've curated a series of articles to help developers hit the ground running with a comprehensive introduction to the fundamental reactive concepts, followed by a case study/strategy for migrating your project to reactive, some tips and tools for testing reactive, and practical applications using Akka actors.

Spring Framework 5
Spring WebFlux
Reactor
rxJava