I am a Technical Development Manager for Red Hat's SOA platform, which basically means I am involved in all our SOA strategy. I have various groups reporting to me like workflow, transactions, ESB. I am also a Director of Standards, so I am also responsible for participation in W3C groups, OASIS and JCP.
So "Transaction" is a term that is misused probably a hundred and one different ways in our industry. It's probably better to be a bit more explicit and say it's atomic transactions. An atomic transaction has its history back in the '60s, and it's hard, it's like it's a fault tolerance mechanism. It's based on work that was done around that time in the '60s on spheres of control. Basically an atomic transaction is a sphere of control, it's an activity that guarantees that work done within the scope of that transaction is either all done, or it's not done at all, you get no partial failures.
A typical example would be a bank account system. Suppose you are transferring money from a current account to a high interest account, and one way of doing that without using transactions would be to remove money from the current account, so you are holding the money and then deposit it into the savings account. If you have a crash of the system at doing this, then depending on where the crash happens, you may lose your money. If you take the money out of the current account but it hasn't quite been put into the high interest account yet, where is it? Obviously the bank hopefully would have information about where it is, and they can do some kind of money resolution, but that could take days to actually sort out and meanwhile you haven't got your money, you can't pay your bills. If you are to do that transfer within an atomic transaction, then the atomic transaction would guarantee that if there was a crash the money goes back into the current account, the transaction system would do that, or depending on where the crash is, it would guarantee that it would eventually turn up in the high interest account, hopefully within a matter of seconds or minutes at most. And it would do that itself automatically, there will be no requirement for manual intervention.
Web services transactions development has been going on for almost as long as web services has been developed. So I started doing work around that in 1999, which is pretty close after SOAP was first released. What we were looking at then and what we have continued to look at over the intervening eight or nine years is actually a number of different approaches for transactions in the web. So, traditional atomic transactions that I described earlier have some in-built assumptions about how they will work and the environment in which they will work, so pretty much they assume they will work in a closely coupled environment that can be trusted and they last for seconds, milliseconds hopefully, but seconds, maybe at the utmost minutes.
On the web those kinds of interactions typically don't happen, you know you might be booking a night out, or buying a book from Amazon and you might be doing that over the course of hours or days. And to do all of that within the scope at the top level of an atomic transaction, just doesn't work. So we were starting with how do we do transactions that are specific for web services or for these long duration interactions. And there was a lot of work that was done back in the '80s and early '90s on what is known as extended transactions.
There is a range of extended transactions. Basically the principle about extended transactions is to relax the very properties that are inherent within an atomic transaction, so if you go and look at the literature then you'll find that another acronym that is put around atomic transactions is also known as ACID transactions. That is ACID - A for atomic, everything happens or nothing happens, C for consistent, the state of the system moves from one consistent state to another, I for isolation, so you can't see dirty data and D for durable, so that if the work happens it is made persistent even if there is a crash, you'll eventually get the same state. Extended transactions relax those properties, so you might relax atomicity, so when an extended transaction or a certain type of extended transaction terminates, you may say "I want to commit but I don't want to commit two out of three of these participants, I want to commit that one, but the other two I actually want to undo".
Another extended transaction model might relax isolation. And the reason for relaxing the different properties is to cater for the type of use cases that you want, and that's why there is a lot of different extended transactions models. There is no one model that actually fits everything you could ever want to do.
So that's what we have been doing over the last eight years, we have been looking at extended transaction work that has been done and trying to come up with a way of allowing people to develop extended transaction models that are good for their particular use case, rather than try as a transaction industry has done twenty years prior to this, shoehorn the ACID transaction into absolutely everything, let's have targeted models, targeted implementations, and we have got there. So it has taken eight or nine years to get there but finally in OASIS there's the WSTX technical committee, which has defined a framework, WS-Coordination, which allows you to plug in different intelligences, so this would be the different types of extended transaction models.
Out of the box, the standard provides two extended transaction models, because of the use cases that we currently have that we need to adopt. One is Business Activity, which is for these long running units of work, the other is Atomic Transaction, so despite what I said earlier about atomic transactions not being good for web services, if you can recall that back when web services where first starting and even through to today, people are using them for interoperability, as much, if not actually more than for Internet scale computing.
So, atomic transaction in the WSTX spec is really there for interoperability between heterogonous systems running on closely coupled networks. You could use it across the Internet, there's absolutely nothing to prevent you from doing that, but there are really good reasons why you shouldn't.
The Atomic Transactions spec in WSTX has given us transaction industry interoperability between obviously Red Hat, IBM, Microsoft, and a couple of other companies. All heterogeneous transaction protocols within about a year and a half of the spec's being finalized, probably less actually, whereas if you are looking when we last tried to do this, which was in the OMG within the Object Transaction Service work, that really took us about ten years. So there were definitely benefits for doing it in web services.
4. So you have some practice now, so maybe it's no wonder it took you less than ten years to start.
Yes, you are right, we did learn from our previous mistakes.
Yes, I would, but some people still want to do it. So you can make recommendations but ultimately if they want to do it then nothing in WS-Atomic Transaction will prevent them.
I think some of this comes back to what I said at the start about the use of the word "transaction", a lot of people when they see transaction they immediately assume ACID transactions, two phase commit, database transactions, however they've run into transactions before. And in that case they are right, I would not recommend to customers to use atomic transactions across the Internet or within the corporate firewall if what they are trying to achieve is a service oriented architecture based system.
But if you look at extended transactions, like I said about the relaxation of different properties, you are going to actually see that there are certain ones that are actually good for SOA based applications, they provide you the guarantees that you might want but they don't provide the restrictions that ACID transactions require. So if you actually look at some of the work that we did in WS-CAF, the web services composite application framework, which predated WS-TX, there's one of the transaction models there, that unfortunately we didn't adopt into WS-TX, which is actually much more relevant to SOA based implementations.
It's the WS Business Process Model, and I think we actually started that back in 2003, and since then companies like Microsoft and obviously Red Hat, and other companies, are talking about very similar things where you no longer have this notion of global consistency, there's no notion of "everybody has the same state", because in a large scale system you can't guarantee that. Well, you can guarantee it, but it might take you until the heat death of the universe to make sure that it is the case. And that is exactly what WS-BP did, assumed that there were these domains of consistency and in between them there might be domains of inconsistency, up to a certain level or even fuzzier than that. So for people looking to use transactions in a SOA based environment I would suggest that they don't come into it with the preconceived notion that transaction equals ACID transactions or atomic transaction. "Transaction" is too over-used, there are extended transaction models out there that can be of benefit to your application.
So, the work actually on extended transaction being standardized, started back in 1997, in the OMG with Arjuna, the company that was with us at that point, and IBM and a few other companies working on something called the Additional Structuring Mechanisms for the OTS, rolls of the tongue, which is short hand to the CORBA Activity Service. That was developing a framework essentially a pluggable coordinator, where you could add the intelligence for your specific transaction models. If you map that to what we have in WSTX you'll see that there is almost a one to one: the pluggable framework and the OMG spec is essentially WS-Coordination. And the intelligences were the different protocols. So for WS-Transactions there is an equivalent mapping in the OMG spec.
When we actually started to do work on the web services transactions back in the 1999 we were working with IBM on essentially taking this model and adapting it to web services. But you are right, at that point the web services wars between Microsoft and IBM fighting Sun, Oracle and pretty much everybody else, they kicked off, and IBM and Microsoft went their own way, building on this work, and we went our own way again building on the same kind of framework, and what came out of it was WS-T from IBM and Microsoft in 2001, and WS-CAF which came out in 2002. Eventually everybody kind of kissed and made up, and we had the OASIS WSTX TC that formed, but the principle input to that was still WS-T and WS-CAF, despite the fact that I am a co-author on both of those original specs and obviously the standards as well, WS-CAF was still better I think. It was much more SOA based and much less "CORBA with angle brackets", if you like. And it's died a death now, but yes the TX one is the one that we are stuck with.
8. Sounds like a VHS/BetaMax story.
It is and the BetaMax was the better one. Don't even mention Blue Ray!
So we did add transactions to REST when I was working at HP in 2000, we actually did some work on trying to standardize a transaction-based, REST-based protocol. And we did it and looking back I am not sure if it was a hundred percent REST, I actually think it's probably ninety five percent REST, but anyway. The reason we did that is because we actually had customers who were coming to us and say "This web services stuff is a little bit too new for us at the moment. We are not too sure"; it was Axis 1.0 days, Axis didn't perform very well, I'm not even sure if it does these days.
"We would like to do something about coordinating multiple updates to web servers, we are using HTTP, sorry multiple web servers, we are using HTTP, can you do something for us?" And we did it, we did it for these customers, but it never got progressed and when I left HP it pretty much folded, went on the shelf, because web services were big or at least were getting big, and it hasn't really been any push at transactions back into vanilla web if you want, or REST or HTTP. I think that's because people are diverted towards web services; it's not because I don't think the requirement is no longer there, I have come across a few companies over the last five or six years who have asked similar questions, but they've gone eventually either for not using transactions at all and chancing to luck, or they have gone with web services. So I think the need is there, I don't think it's a huge need, but then there isn't a huge need for transactions anywhere.
I think that it would be nice if there was a standard, I think this is one of the problems with REST over HTTP at the moment, in that although it's a standard there is no generally agreed upon standardization of protocols that might sit on top, like transactions, like group communications, that sort of things. And hopefully if we can put these REST and web services wars behind us and kiss and make up then maybe we can actually take some of the benefits of both systems and standardize things in REST over HTTP. That's some customers are actually crying out for.
I think it's going way too long and I think it's become very polarized in some sectors when it shouldn't have. There are certainly good reasons for using REST over HTTP, so obviously there is a distinction between REST and what I would like to call REST over HTTP, which is one way of doing REST.
There are also good reasons why you might want to use web services. And I think for web services it really is the interoperability and the fact that everybody has got together and we have standardize these high level application protocols that sit on top. I don't think it's an either/or situation, it might be in certain cases, it might be that it really does make sense to use REST everywhere in a particular deployment, but I think in general and if you look back over like forty-odd years of distributive systems development, there has never been a global panacea for distributed systems. One thing does not work well for absolutely all distributed system or component within distributed system like you might come up with.
RPC has worked very, very well since the'70s, and we are revisiting that over the last ten years or so, but RPC isn't dead, people are saying RPC is dead, it is not, if you actually look around RPC is running a lot of backend infrastructural systems that are always going to be RPC-based. So when people come and say "You need to change this to message oriented" that is not going to buy those companies anything, they are happy with what they have got. And I think the REST and web services wars should pretty much just end; let's agree that there are good things and bad things about both, and let's try and use them together if we can. I think they can actually be used well together, the work to merge them together and make them work efficiently together hasn't been done, I think it could be done, and let's just get on with it, let's just stop too much fighting.
Yeah, there isn't a huge need for transactions, but there is a need for transactions. Security, high availability, WSRX for instance, you can build on that to do high availability services. From a REST perspective, the uniform interface does make a lot of sense in many cases. I think one of the problems that we have with web services is WSDL, to be perfectly honest. Certainly when I started doing work around web services back in 1999/2000 WSDL was still in its infancy and when we were developing specs and actually doing implementation in HP, we were developing on SOAP over HTTP and back porting the WSDL afterwards because the WSDL really got in the way, and I think it still does today. So getting away from WSDL and looking at the benefits that a uniform interface can provide rather than a specific interface, and also try not to abuse transports, so saying that SOAP over HTTP is the same as SOAP over TCP/IP because HTTP is most obviously a transport, it's most obviously not a transport.
I think that the web services guys, and I kind of include myself in this group, need to realize that certainly interactions across the Internet that are based on HTTP are more likely to be REST based than not. And bridging between web services and HTTP or REST over HTTP I should say, should be addressed in a more efficient manner. We shouldn't try to bastardize HTTP anymore as a transport, we should try and work with it rather than against it, and I don't think we are really doing that at the moment. Web services uses HTTP for a really good reason, and it's so that you can tunnel through firewalls.
And that's the real reason. I was at the first OMG meeting where SOAP was brought to life, and it was one of the debates about why it was there. I think we kind of progressed that we are doing it for the same reasons over the last seven or eight years. But I think that as a web services community we can do more to embrace REST than we are currently doing. I think we should. I don't think there is anything technically that would prevent us from doing that. If you actually look at the way the web works, there is nothing that should prevent us from being able to use true web protocols with nothing else laid on top of them, to actually talk between web servers across different continents.
Like I said before about the extended transactions stuff that's the WS-BP spec that I mentioned which has this notion of loosely coupled domains of consistency with inconsistencies between them, that's kind of very similar in that what happens within the corporate firewall might well be a combination of CORBA, REST, web services, DCOM, Java RMI ... whatever you want. But between the corporate firewalls, I think it should more likely try to be not REST we should actually work with that rather than try and fight it.
15. Could it be that they were just right?
To speak personally I did transactions over REST back in 2000, it's not like I certainly had a epiphany moment six months ago. I think a few other people have been kind of coming around to this, and yes whether you want to say that they suddenly realized that what other people were saying was right or whether they always knew it was right, and they just had different masters at that time who wouldn't let them say what was really going through their mind, I obviously can't speak for everybody.
I think there is. I am not necessarily sure that it's SOAP over REST. I think that again going back to what we were doing with transactions on REST back in HP, we were actually working to bridge web services transactions to RESTful transactions. And you could do it. I like to think that if we been allowed to finish it maybe it would have been one of those little gems that would have grown and maybe unified people around that fact that web services and REST can be used together. I do believe that we can do it, I'm not suggesting that it is easy, but if you look back at the amount of time and effort that has been wasted in these fights that we've had from individuals to big corporations, I would like to think that if we'd actually spent that time actually talking and trying to get these things resolved in a reasonable manner we could have been there by now.
Red Hat acquired JBoss in July 2006. It definitely wasn't a smooth transition but I certainly didn't expect a smooth transition. But that wasn't because it was JBoss and it was Red Hat, I have been involved in more acquisitions in my career that I care to remember, and none of them have been particularly smooth. So it didn't come as a big surprise to me, I think one of the big problems though which is probably specific to the JBoss/Red Hat acquisition was the culture within JBoss because of its history of having to fight against the man, who was IBM or who was Oracle, it was much more combative than Red Hat. Being assimilated into a company like Red Hat, that had a different kind of culture like that, did cause friction.
I won't go into specifics but things like being a very private company as well, various mailing lists that people would talk within JBoss to each other throughout the whole company and use very colorful language, for a start you probably wouldn't want to do that in a public company anyway, but also it can offend ... the larger the size of the company the more chance it is it would offend somebody. There were those kind of things you have to be careful about what you say, your level of freedom has gone down a bit as a result, but obviously it's a bigger company, it has more money, has a bigger reach, so there are tradeoffs.
I think overall it has been a good thing, I think. Some people have left for one reason or another, sometimes because they didn't like the culture change, others because they wanted to stay with the startup mentality and not go to a big company, so they gone to other startups. There are still the odd culture clash within Red Hat and I think quite a few people, old time Red Hat people see JBoss guys as upstarts and brash and trouble makers, and treat them a bit like that as well and that doesn't go down well with some individuals who are brash and upstarts. They push back. There is a lot of to and throw but I think the analogy is probably teenage son versus forty years old father, JBoss is going through puberty and the father is the one who has been there and done it before.