We are here at the SpringOne Platform Conference in San Francisco, and we are talking to Pieter Humphrey and Simon Basle, both of Pivotal.
InfoQ: Thanks for joining me. Can you tell us a bit about what you do?
Pieter Humphrey: I do the product marketing for Spring team. I’m basically the marketing guy for the Spring engineers; I organize the content for the YouTube channel, and that sort of stuff.
Simon Basle: I am a software engineer, working on the Reactor project.
InfoQ: This is a pretty impressive conference; you have a lot of great content, you have a lot of appeal, you’re growing dramatically; how do you locate the presenters, curate the content, and how do you get the word out to the potential audience?
Humphrey: The challenge for us has been marrying vendor topics with open source content, and bringing a lot of infrastructure development topics, along with application development topics. Historically we haven’t seen these things all at the same conference before. So to many people, as we started doing this it was a little odd, they are used to this having a dev conference or an infrastructure conference, but trying to bring them together, you have a lot of people with a lot of different interests and sometimes they don't transfer over very well, so you get people with very polarized reactions to the content, saying "I don't care about this", and "why is there not more of that", so it's hard to please everyone. But I think the trend is starting to work in our favor, where people are realizing that infrastructure is code, and that is something they need to pay attention to. So it's becoming less difficult to put together ten tracks of things like app dev infrastructure, agile, case studies, all together in one conference. That's been the biggest challenge to try to keep the attendees engaged.
InfoQ: Well it is becoming more of a DevOpsy world, and I noticed that there was a track devoted to DevOps. I assume that’s thanks to the new DevOps CI/CD capabilities of CloudFoundry?
Humphrey: Yes, but not just CloudFoundry; also Kubernetes is very container-management centric, CI/CD is a very relevant topic, in fact, as we start talking about these platforms you sort of assume that people have good CI/CD practices, and I think our products in some ways depend on having good CI/CD practices. That’s a big challenge.
InfoQ - Can you describe how CloudFoundry helps with the DevOps pipeline?
Humphrey: We got some early success with Concourse because Jenkins servers were stateful, and you could not check the configuration into version control. That's one of the nice things the developers tell us about Concourse; that you can take your pipelines and check them in and the servers themselves are stateless, so you can recreate a concourse worker node from the definition at any point. That's been pretty popular. Pipelines as code, infrastructure as code, everything is code now.
InfoQ: But in the keynote we were hearing a common theme about how the Pivotal CloudFoundry umbrella handles a lot the DevOps pipeline. Can you describe how PCF plays in to those?
Humphrey: What we have seen happening in the market is that Kubernetes' traction has been pretty substantial. We had a container management system embedded inside of CloudFoundry that predated Kubernetes, by many years actually, but when something comes along like this that gets a lot of market traction, it's hard to ignore. And in many ways I think, what we're now calling Pivotal's application service, that was a great abstraction for apps. The PKS (Kubernetes service that we're working on) is going to be a really good abstraction for containers, where you need a little more control. So that's the balancing act; you're going to have Bosh as this back plane, almost as the platform, and upon that are going to rest a few different swim-lanes for different workloads. If you're a developer and you just want the nice kind of cf push automated experience, you have stateless applications, you're working with a service tier perhaps; great, PAS is going to be awesome. But once you start adding databases, or you want to run Elastic Search, or you need to control the TCP traffic at a different level, or define custom ports; well Kubernetes is great for that. And then, you want to run functions, and whatever the next generation workloads are going to be, we're trying to create different swim-lanes for all of those on top of Bosh. It's an evolution; instead of trying to make Diego and our Elastic Runtime fit all use-cases, we realized that one approach is great for one set of use cases, and we're going to have to come up with some different way of tackling these other new ones.
InfoQ: Whenever I look at the Reactor API's it seems like they're growing quickly, covering much more ground, and becoming much more robust and mature. What goes into building that, and how do you come up with new features, find out if they are useful, reject them if they're not, and how do you code them?
Basle: (Laughs) Coding them is the hard part! As we have more and more users, people will always come to us with new use cases, like "this would be great if we could have that sort of operator." We welcome the community feedback. But really what we want to do as a first step, is determine whether there is any combination of existing operators that can fit your needs. That's the real power of the library, is that you have these fundamental features, and you can combine them to produce complex, asynchronous pipelines. But sometimes somebody says "this is hard to debug, so maybe you can offer some solution in terms of a new operator, and those suggestions and requests make good sense. And of course, there are always bug fixes, improvements, and optimizations to the libraries.
There are also committers that are not Pivotal employees that give us feedback. Like David Karnok, who is in charge of RxJava, but is also involved in Reactor. He was initially very heavily involved, actually in the original collaboration project that was used at the beginning, to design all of the next generation components of the reactive libraries, so a lot of work went into that. And he is still very active and is one of our champions, and you will find him commenting on pull requests with improvements and techniques.
But the good thing with Reactor was that we didn't have all the baggage of RxJava so we could move away from some of their historical choices and design choices, but there is value in both libraries, and it is definitely a good sign that David still contributes to Reactor.
InfoQ: Why does the world need two reactive projects? If I am a Java developer using Spring, why would I use RxJava?
Simon: They are not the same; RxJava targets a different audience. The fact that RxJava still supports Java 6 is one of the main reasons it was adopted so eagerly by Android developers, whereas Android is not really the Reactor target. We are on the same page with David, if you are doing front-end Android development, by all means, use RxJava; whereas if you're doing back-end Spring development, then Reactor is a perfect fit. But the fact that both are tied to the reactive stream specification, the choice is yours. Inside Spring itself, all of the plumbing is done with Reactor, but if you want to go the RxJava route and use Flowables, it's no problem.
InfoQ: I often find myself needing a certain Flux operator, and the whole beauty of Java is that it is extensible, so if I need a function I can write it. Is there a similar way to create bespoke components using Reactor to produce new Fluxes?
Basle: Again our goal is to provide a sufficient core of operators to compose what you need. But if it is not there, it's hard. If you look at the Reactor source code, you're in uncharted territory. The patterns that we use are pretty advanced, and that's the whole value proposal from Reactor and RxJava, that you don't have to write this operator yourself, because you really have to think of a lot of things before you can implement a new publisher; we offer an API on top of reactive streams, and we build the operators for you. But if there is no combination of operators, that is when things get harder and David has a Wiki page for RxJava2, where he explains his patterns, but that is hard and should be a last resort. If you can't find an existing operator for what you need, then that might suggest that you should be thinking about the problem in a different way. Maybe a combination of operators, or an imperative processor as a third option. But if you're really stuck then you might need a new operator, and you should add those to our issue tracker.
For example, during a presentation today, somebody asked for a kind of zip operator that did not terminate when the first stream terminated, kind of a left-outer zip. There is not an obvious way to do this, so perhaps that would be a candidate for a new operator.
InfoQ: I guess you'll know you're finished when you can compose the old ones to produce any new ones you need.
Basle: Yes, it's a matter of finding the right operators and the right approach, but beware, your habits from imperative programming might get in the way of your seeing the correct solution.
InfoQ: During the keynote it seemed there was an emphasis on the interoperability of Spring Web MVC and Spring WebFlux, and I thought that was a great way to bridge the steep learning curve, allowing me to modify things incrementally, so I could write a Web MVC controller and turn it into a WebFlux controller by just changing the return type to a Flux. Was that a technical decision or a marketing decision?
Basle: We were trying to provide good developer experience for people who were not familiar with the reactive programming model. It was really an evolution; if you have an MVC controller that returns a Flux, the framework knows how to deal with that. So even though you are still using Web MVC, if you have parts where you use the web client, for instance, and you return a Flux, Web MVC will be able to subscribe to that Flux, and do the right thing. These are done in the service of making the developer experience easier, and we want to make that as incremental as possible. You can start small, go from Web MVC, change the return type of one of your end points to a Flux instead of a List, and start playing with the web client. If it works for you then start working with the APIs; if your needs grow beyond that then you can switch to the WebFlux stack, and a whole new world awaits you.
Humphrey: Right, so it was more of a pragmatic decision than a marketing one or an engineering one. We were trying to do the best we could to meet the users where they are currently at.
InfoQ: I think it was a great decision; one of the hardest parts of using reactive is overcoming the learning hump, it's a new language truly. So I think anything you can do to smooth that thump is greatly appreciated. One day you're dealing with Objects, and the next you're dealing with streams, so it is a huge cognitive leap.
Humphrey: Yes. And on the other hand, we are careful not to say that everything must be reactive. There is a tendency in some circles to fall into that trap. You have a choice of doing either, and you should not look at the world as being all one way or the other.
InfoQ: I noticed the conference featured a lot of talks on process; there were talks on agile, Vaughn Vernon did something on domain driven design, and there was a whole track devoted to DevOps.
Humphrey: We are trying to get more architecture methodology talks. We're Pivotal Labs, so we do talk a lot about agile, test driven development, CI/CD and those kinds of concepts. I would love to see more of this kind of content in the program, talking about the design, the architecture; a lot of times the focus is on the new tools, and a lot of times the developers are hungry to see more, so that is something we will keep a sharp eye for.
InfoQ: What is the demographic of attendees?
Humphrey: The majority are from the United States, 60% developers, 40% everything else; Ops, IT management, executives, etc. We have content for that group as well, e.g. case studies, technical stories, things like that.
InfoQ: What is the future of the SpringOne Platform conference; do you have a location planned for 2018?
Humphrey: We're going to be in Washington DC next year, September 24-27. We want to try to move it around, East Coast, West Coast, Central East Coast, and timing will generally vary according to venue availability, aiming for the last quarter of the year. This year we sold out to 3000 people, we sold out our sponsorships, we're in Moscone center; this is literally double the size of last year, so it's really growing, and that’s great, management is behind it more than ever, the community seems to be behind it more than ever, so as you see the growth of the infrastructure-as-code movement, you're also going to see a segment of the audience that isn't just coming for the app-dev stuff, and they're coming more for the infrastructure-dev.
InfoQ: Spring is as important to my Java development work as Java itself, and I can't imagine going a day without using Spring pretty actively; most Java developers feel that way, and so it's a great service Pivotal is doing for the development world, taking custody of Spring, and they're obviously doing a fantastic job. But what's in it for Pivotal?
Humphrey: Right, we have to eat lunch too, and people might look at us and say you're crazy, giving away all this stuff. But we're very much about teaching our clients how to agile, and how to TDD, how to develop for the platform. The main commercial motion for us comes from people looking for the CloudFoundry and Bosh platforms; once you get out of the app-dev, and you get to build all the things, run all things, that's where we have commercial motion. So that's our strategy; offer enough value to developers that they want to stay in our programming paradigm, then make it really easy to run those apps on our platforms. For two guys in a garage, they may not need all we can provide, but for larger enterprises, there is some real value here.