2. Can you tell us a little bit about your talk?
Yes, so the talk was all about building micro services using an event driven approach that involves both event sourcing and CQRS, and ultimately then I talked about how the services get deployed using Docker.
Yes, I guess it first started a few years ago now, where I was sort of playing around and exploring NoSQL databases. I think that was kind of the original motivation was, if you think about in NoSQL database like Cassandra, it’s got some great attributes, but one of the sort of limitation is that it only supports primary key lookup effectively, and so if you want to do queries on top of that you end up having to create a denormalized view of the data, so now you’ve got two, you’ve got your system of sort of system of record and you have this denormalized copy and you have to keep them in synch, and sort of seems like events were a good way of driving the synchronization. And then if you look to another scenario where say you’ve got a SQL database as the system of record and you are denormalizing it into say Redis for some highly performant queries, you have the same problem. So this even happens like even if you’ve read about eBay Architecture, they have multiple SQL databases that they need to be kept in synch, and they use an event driven approach, and so sort of events just sort of seems to be everywhere.
So there is that, and then over more recent times I’ve been digging into micro services and some of the same kinds of problems crop up because with micro services you are functionally decomposing the application and you also functionally decomposing the database, and now you’ve got multiple databases and yet you might have say business transactions that need to update multiple databases and given the two phase commits is not something desirable or fashionable, you end up, you need another mechanism and so having an event based approach where events get published by one micro service when the state changes, consumed by another one which will then update its state, seemed to be an answer there. So there is sort of like events, events all over the place but then it’s a question, how we reliable generate events if you dig into it, Event Sourcing seems to be the best way I’ve discovered of reliably and atomically updating state so to speak and publishing events simultaneously. So a long answer but it sort of solves kind of this recurrent theme and modern application architectures.
4. And I suppose CQRS gives you that granular approach to making those changes?
Oh yes, but then the problem you have is if you use Event Sourcing, you only have basically primary key lookup of an entities events, and so in reality if you, you know for a screen you need to execute a query that gathers data to display on a screen, that’s likely need to do a join across multiple entities, and then sort of support that you end up using CQRS where you have consumers of the events which update views that support the queries that your system needs. So it’s kind of like if you use Events Sourcing then that creates a problem which is then solved by using CQRS.
Harry: So the logical extension many times with CQRS and Event Sourcing models is the granularity lends itself to domain driven design.
Yes, but there is another interesting sort of parallel like so the book “Implementing Domain Driven Design”. If you look at that sort of the model is you’ve chop up your domain model into aggregates and sort of transaction applies to one aggregate, but then aggregates kind sort of send to other aggregates and trigger updates there in that kind of like mirrors exactly what kind of imagine happening inside the micro service architectures as well, so there is a very strong parallel there of these individual pieces, publishing events and consuming events and which drives state changes.
Harry: Adrian Cockcroft’s view of micro services which is, a micro service is a representation of a small bounded context.
Yes, exactly, that’s throw you into more dDD terminology.
That’s a really interesting question, like I gave my talk and it was a whole ton of Twitter traffic and someone commented that you should only use this for complex domains and what’s interesting is it depends what’s complex, because for instance imagine you have an app where your system of record is a relational database and yet you need to use text search and so all of the sudden you’ve got to kind of keep elastic search or cloud search in synch with your system of record and you can view that as quite a challenging problem to keep two things in synch, and so I could argue that while an Event Source approach and using CQRS kind of is a good way to do that, even though the problem is on the surface simple. So it’s kind of hard to say and maybe I’m overly optimistic but it’s almost like maybe very few things we do today are actually simple.
Harry: And we have the ability to introduce complexity at any stage, so perhaps there is something simple but there is this requirements change or deployment strategies change.
Yes, or just you know using and throwing in text search sounds simple but it’s not exactly, right?
6. Could you tell us a little bit how Docker swings into this?
Yes, so I mean I’ve been a huge fan of Cloud Computing for years. Interestingly it started off where back in 2006, I’ve run the Java Users Group in Oakland, and so we had someone from Google come talk about their APIs and it was like Google check out APIs I think, and then we invited someone from amazon come talk about their APIs and it was like: “We though like you are going to talk about APIs for buying books” but instead they talked about EC2, this is October 2006, way, way in the distance past and I just totally blew my mind, I got totally sold on Cloud Computing since then. And so I’ve sort of been a big fan of EC2 and ever since, but if you think about like one approach for packaging your application, packaging your service is to build an AMI. On one hand that’s a great kind of way of containerizing your application because doesn’t matter what’s in your AMI, it’s got now a standard interface and you can provision at regardless of the same way, regardless whether it’s a Ruby App or Java App.
But on the other hand AMIs are incredibly heavy weight, like to build them takes minutes and to boot one up requires booting up an entire OS and as well as copying across the network out of storage and then booting across the network, but and there is Docker just as kind of unbelievably lightweight virtualization technology, I mean it’s funny when I first heard it it’s like lightweight virtual machines, big deal right? But then I tried it and it’s so, I mean it’s orders, many orders of magnitude faster than sort of a VM based approach like for instance in the app that I working on, I can build a Docker image and I think it’s 5 seconds, because all it’s doing is basically copying the WAR file 30-40 Mb into a Docker image, super lightweight and then uploading it to S3, where the registry in S3 is taking 30 seconds and so you can imagine tests complete 5 seconds plus 30 seconds and then another 30 seconds to pull it down and then I can started up in virtually instantaneously, so it’s sort of like was that 70 seconds from being tested to being running in production, unbelievably fast and lightweight. So instead of going this service per AMI approach it’s like multiple Docker containers running on top of some large EC2 instances and I think that’s a very Agile and very cost effective approach to deployment. And then of course it got all the other benefit that you can actually do it on like if you actually got your own service instead of sticking a virtualization layer on them just stick Docker on and you’ve got this uber lightweight sort of virtualization technology. I think it’s really exciting, I mean there are some complexities there, but quite enthusiastic about it.
7. And so what’s next for you?
What’s next for me, so I left the, I after gone through the Cloud Foundry Acquisition and Spring Source and then VMWare and then Pivotal, I actually left there a little over a year ago and sort of took a break, and since then I’ve been digging into couple of different startup ideas, and so one of which is sort of, one of my passions is actually food and cooking, and so I’ve been building an app around that. But along the way I build it rather than building a simple MVP, I actually ended up building this micro service based event driven architecture.
8. Because what can be more complex than cooking?
Yes, and you know I’m actually kind of looking at maybe doing some kind of startup in that area instead.
Harry : Sounds awesome.
Yes, I’m kind of excited, there are not enough hours in a day, that’s sort of the down side right now.
Harry: It’s a great feeling to have. Thanks a lot for your time Chris!
Well thank you, it was fun!