I wasn’t around at the time, but I can speculate. The JCP has been around for a little over 10 years (actually closer to 12 now). It started in about 1998 and I would guess that the answer was, in the very early days, "It’s way too premature to think about international standardization". There were some early attempts, but I think there was a constant tension between Sun’s desire for collaboration and cooperative development, and obviously for the seal of approval that you would get with some kind of standardization process, but also the desire (let’s be frank) to retain a reasonable amount of control for themselves and also to ensure compatibility. Obviously, this was an early platform just beginning, it was going to evolve fairly rapidly and [they wanted] to make sure that the various implementations would be compatible with each other.
So, while there was some talk in the very early days about going to ISO and I think to ECMA, that fairly quickly fell by the wayside and what they started out with was basically a consultative... it was not a truly - in any way - a democratic or level playing field. It was very clear that Sun was the boss. They were inviting others to help, to provide comments and some suggestions, but in the early version, there was no Executive Committee, there was no voting. Sun was the boss, it was very clear, and everybody else was there to advise.
2. Do you think that has improved over the years?
Clearly. The changes since then have been fairly steady; the increase in the influence that people other than Sun and now Oracle have and a corresponding decline in Sun’s influence. As I say, at the very beginning it was clear: Sun was the boss, they made all the decisions and that was probably fair at that time; Sun was the only implementer. In the very early days Sun had invented this technology, they were creating specifications, they were doing an implementation and to their credit they said, "We don’t want to do this in a vacuum, we want to consult with others". But at the time, in the very beginning, there was no real idea that others would begin to implement. Over the years, the changes were gradually to make it more possible for others to implement, to become a spec lead. In the beginning, the IP vested all in Sun, then it would vest jointly in Sun and the spec lead. Then, after that, it went to the spec lead rather than to Sun.
Later changes were around concessions towards the open-source movement, prompted primarily by Apache who took a look at the initial legal agreement, the constitution and the way that the organization was set up and said, "This doesn’t work for us. The IP grants don’t work. Your assumption is that people are going to license from the spec lead, not only the right to implement the spec (the intellectual property), but also the source code itself. We don’t want your source code; we want to do an independent implementation". That was a major change that came in after the first few years.
3. How has the JCP changed since Oracle completed their acquisition of Sun?
I mentioned Apache and the concessions to open source, so the consequence of that was mixed. In Java EE things worked pretty well, we have multiple open-source implementations of various EE technologies; not so much in ME, the business model there doesn’t quite fit. In SE it’s no surprise and I’m sure it’s on your list of questions to ask, maybe later rather than now, what happened there and the promise or the expectation of Apache that they could do an implementation didn’t come to pass. You asked me originally about what happened since Oracle took over. The point I’m trying to make here is that Oracle inherited a situation from Sun: the dispute with Apache had been brewing for several years.
In practice, I took over Java’s chair in mid 1997, little knowing what I was getting myself into. By late that year this thing had begun to become critical. We engaged in a series of discussions and negotiations, if you like, between Sun and Apache over a period of a year or so. Then we got into this kind of stalemate where Sun was not willing to make concessions, Apache said, "We want changes, we want our TCK license. We don’t like the one you are offering to us," and we got into this situation of stalemate. How things have changed since Oracle took over is we are no longer in a situation of stalemate.
I should also make this point, I guess: the licenses that Oracle is offering, the business practices, everything that they are doing is basically identical to what Sun did. What they did do was basically (let’s put it this way) to have the courage to say, "This is the way it is. I’m sorry Apache, if you and if other people in the open source community don’t like the decision that we’ve made, but we have made this decision and we want you guys on the Executive Committee to make your commitment; to let us know whether you are willing to vote our JSRs for Java SE forward or not." The answer came back from most people, "Well ok, yes.We’re willing to do so." Primarily, how things have changed is that we’ve got ourselves out of that kind of stalemate situation and now things are beginning to move again. We have a bunch of JSRs for Java SE which were approved late last year. We have a bunch of JSRs for Java EE which have just been submitted over the last few weeks basically.
So EE 7 is moving ahead. ME is coming a little later, they are lagging but they will come too. In addition to that, I’m about to submit a couple of JSRs to reform the process itself. After a period of relative stagnation, we’re moving again and that’s good.
It’s a really complicated matter and a decision was made early on in the creation of the JCP that, unlike some standards organizations where basically the specs are created and are given away for free, the decision was made that the right to license the technologies would be granted to the spec lead. I think that was partly because unlike other standards organizations where the only or the primary output is a specification, in the JCP we also require, in addition to the spec, a reference implementation which is required to be a full implementation of the spec. There is great value in that because you can prove that the spec is implementable. If we didn’t have that requirement, I could create a spec for an anti-gravity machine, give it the seal of approval - that would be great.
5. That’s quite unusual isn’t it, in the context of standards organizations generally?
They usually require proof of implementation, and sometimes they will require two proofs of implementation before they will finalize a spec. But there is no requirement that those implementations be complete. So, one implementer could take half the spec and implement that, the other could take the other half and implement that. It may be that, if you try to implement the entire spec, it would be impossible because they are incompatible with each other, they’re contradictions, but with many standards bodies, you won’t find that out until later. There is also a third deliverable that we require, which is the TCK or the conformance test suite - the set of tests that must be executed to prove that implementations conform to the spec.
You’ve got to deliver all three together, which is tricky; I know because I’ve been involved in the process. You are trying to go to test suite for something that doesn’t exist. You are trying to build an implementation for a spec that is still evolving and they all depend on each other. So, it’s a moving target, but it’s a powerful model because each of them feeds the other. Building the reference implementation helps you to find inconsistencies or ambiguities in the spec. Building the test suite and then executing it against that implementation will find bugs in the spec, or bugs in the test suite itself, or bugs in the implementation. It’s difficult but it’s powerful, but it’s also expensive, very expensive, to build these things and to deliver them. I suspect that was partly why the decision was made that the spec lead had the right to license these technologies... had the obligation to license them: you can’t prevent someone from licensing your technology, but you are allowed to charge for it under fair, reasonable terms.
I still think it’s a long-term goal. There are clearly problems when an organization calls itself a standards setting organization and yet there are people who say it’s not, or it is not a level playing field - one partner, one entity, has more influence than others. So, clearly, life would be easier from one perspective if we were independent. On the other hand Oracle, since they have come over to the position of being the steward of Java, being the owners of the technology and being responsible for moving it forward, clearly came to the same conclusion that Sun had come to, which was that this is premature; that at the moment we want to be the stewards, we feel we have a responsibility to guide the direction of Java, to guard compatibility.
And let’s be honest, Oracle is doing most of the work. We did some stats last year about JSRs that were actively working their way through the process. At the end of 2010 there were 46 of them; 26 of those being led by Oracle. I went back and I did a historical analysis of all the JSRs that we completed since the JCP was formed; two thirds of them were spec led by Oracle, or companies that Oracle has since acquired. So, in other words, two thirds of all the work that has gone into developing Java has been done by Oracle or its subsidiaries. Under these circumstances, I think Oracle feel they are entitled to return on their investment into this role of steward. But I can’t deny that it causes tensions. In fact, you asked me earlier about the historical evolution of the JCP. To a great extent, that evolution has been precisely about this relationship between the steward (initially Sun and now Oracle) and the other members of the organization and where power should lie.
Over the years, power has clearly shifted from Sun, and now Oracle, to the other members of the organization. People sometimes say what they said in the old days, "Sun has veto power, Sun makes all the decisions." Now they say the same thing about Oracle. It’s not true! The by-rules of the organization give Oracle a veto power in a couple of very specific areas over changes to the language, changes to the VM and the creation of new platforms; in other words if we were to create a fourth platform. In each of those cases Oracle would have to vote "Yes".
7. Has the veto ever been used? Did Sun ever use it?
No, I don’t believe so. The real veto lies with the Executive Committee because they have to approve everything. A JSR cannot start, cannot progress, cannot complete unless they agree. As we saw in the dispute with Apache, if they choose not to approve or if they give the impression that they will not approve, they can effectively put the brakes on things. So it is a difficult and sometimes delicate balance and I guess people will say, "Oh, if we were an independent organization, then all these problems would go away." - No, I don’t think they would. That’s like saying there would be no politics in that organization. Anybody who has ever worked in a university or any other large organization knows that of course there are politics.
Typically, those who spend the money are the guys who end up with the most control. It’s conceivable that it might be possible to spin off the JCP into an independent organization and, so long as Oracle was doing two thirds of the work, maybe nothing would change in practice. I don’t know and I should say I guess (maybe I should have said this at the beginning) when I do events like this... I’m here at QCon and when I do interviews like this, typically I like to say I’m speaking not for Oracle, I’m speaking for the JCP. Even though I am an Oracle employee, I do my very best to be as independent as I can and to try to express the interests of the community as a whole and the organization in particular.
8. How significant do you think it is that Sun and now Oracle are developing JavaFX outside of the JCP?
Superficially you may think, "This suggests that they think that it’s not necessary to standardize or would rather not standardize". I can’t speak for them and for what their thinking is, but I guess I would say there are times when it’s appropriate to standardize and times when it’s not. Sometimes people say the JCP is too slow, it’s stodgy, it takes a long time for something to work its way through. I say, typically that’s good. You shouldn’t be innovating in a standards developing organization. Standards are supposed to define things concretely. Once you’ve defined them they are supposed to remain solid and stay unchanging for a considerable period of time. If you are going to be defining these kind of interfaces you should do it in a deliberative manner.
I sometimes say, "Next time I get on the plane, go back to California, I want to know that the interface between the wings and the fuselage has been really well thought out. That it wasn’t just some idea that some guys came up with in the back room in a few weeks or whatever." So, in practice JavaFX originally was a brand new language. Now they’ve changed it slightly and they are talking about incorporating the essence of it in the form of Java APIs, but it’s a very new way of doing things. I think it’s probably too soon to standardize, just as it was too soon when Gosling and his buddies way back when were inventing Java in the very early days. You have to retain a certain amount of freedom to experiment, to change with things, to try them out.
Once you try them out in the real world, and you show you got things right and you are willing to commit to it, then that’s the time to standardize.
9. The general point would be that a JSR is not the place to innovate, you do that work somewhere else.
Right. Then, I think, if you do that, you can also address the question of agility and how quickly you can do stuff, and inclusiveness. I personally think the right model is to do your experimentation through some kind of collaborative, preferably open-source, development process that allows you to be agile, to be very inclusive, to try out new ideas, to go down blind alleys. Do that for a year or two or however long it takes, get some real world implementations out there, get people using them and when you are sure that you have the right approach then you can standardize.
I just came out of a panel with Stephen Colebourne and various others. Stephen is an individual who’s creating the JSR 310, he is spec lead in JSR 310 - date and time. Someone asked him, "Why are you doing this? You had a previous instantiation that’s called Joda-Time - why didn’t you just go with that?" Stephen said, "Well, I didn’t get it right the first time". You almost never do; that’s the way it is in the software world. You are probably going to throw one away, or maybe two. That’s ok if what you are throwing away is your first implementation in an open-source project, but if you nailed it down and said ,"This is a standard" and then discovered you had to throw it away, that would be a problem.
If you are doing a JSR that really touches a bunch of others, then probably the best way to handle that is to work through the relevant platform expert group, because that’s the group that has the responsibility of making sure that different JSRs don’t contradict each other, that they play well with each other, and so on. I think that JSR was followed up by one (I think it was 330, that Bob Lee lead - dependency injection) which was precisely this: it was an attempt to reconcile two different dependency injection models; one that was being used in SE and potentially one that was being used in EE. It was only possible to do this by collaborating closely with both of the expert groups, with the SE and the EE expert groups.
I think the short answer is "Yes"; to the extent that your JSR touches a bunch of other stuff, the more that is the case, the more likely it is that that JSR should be done as part of an umbrella JSR and in collaboration with the relevant platform expert group. The converse of that is that if you want to do a JSR in a relatively brief period of time and you want to be successful, and you don’t want to spend years of your life dealing with politics and liaison and so on, make it self-contained, make it very well-defined and constrain it as much as possible. To the extent that you do that, you are more likely to be successful.
We’re still figuring it out. The short answer is that the intention is that OpenJDK should be the venue through which the implementation of the specifications is performed. We have a similar model in Java EE where the GlassFish project, which is more mature and been around for longer than OpenJDK, is the way in which first Sun and now Oracle implements, creates the reference implementation for the various EE specifications. So, the short answer is OpenJDK is the equivalent and, in theory, you have an expert group that submits the JSR and works it through the process and ensures that it is standardized. Remember what I said earlier: you don’t submit and complete the JSR first and then do the implementation, you have to do the implementation and the TCK in parallel.
The reference implementation for Java SE will be done through the OpenJDK project. It’s unlikely that TCKs will be done in that manner because, to be frank, TCKs are expensive and difficult to produce and they are not the sexiest things in the world, and in practice you are not going to be able to get much of an open-source community around building TCKs. So it’s likely that Oracle will continue to do the TCKs in a more traditional manner, but the reference implementation for the platform specification and all of its components will be done through OpenJDK. I think the important point to note is this means they are all being done out in the open, in a transparent manner, where people can participate freely, and we hope that they do.
That’s actually the normal way to do things. Typically there are various stages that a JSR goes through and the very first stage is the JSR proposal or submission. That’s all that happened back in December. There were two platform JSRs submitted: one for [Java] 7, one for [Java] 8. In fact, it’s the [Java] 7 that was unusual rather than [Java] 8, because, as I said, the normal process is you come up with an idea; you say, "I’m willing to become the spec lead and to drive this through." In the case of platforms Oracle is the spec lead, so it’s time to do a new revision. You go and lobby and get some support and you pull together (at this stage it’s not even an expert group) a bunch of members who will support you. Then you write a proposal and you say, "I would like to be the spec lead and these are the people who are supporting me and here is what we propose to do."
That gets voted on (Yay or Nay) by the Executive Committee and, assuming that they vote "Yes", then you begin the work of creating the spec. So, the normal process is: JSR submission first, spec creation and then of course, at the end of the process, you have the final vote on the JSR, and maybe that’s what you were thinking of. So, there is a defined process: submit the JSR, work on the spec, finalize it and then implement great commercial products. SE 7 was more interesting because, due to the difficulties over the Apache situation and Sun’s reluctance to submit the JSR, work actually started, and things were really quite far advanced, before the JSR was submitted.
That is a more unusual situation and in fact, I think, it’s not a bad situation, but I’ve heard some concerns by some members of constituent expert groups who say, "Hey! There’s a year’s worth of history here. We’ve only just formed the expert group and now we’ve got a year’s worth of catching up to do because you were off working this thing through an OpenJDK project. Do you expect me to rubber-stamp everything that’s done, or can we open up all the questions that you spent the last year discussing and resolving so I can get my input?" So that’s the more unusual situation in fact, not necessarily a desirable situation. I think it was unusual and a one-off and hopefully we’ll be back to business as usual from here on.
It’s theoretically possible. It’s the job of the spec lead to orchestrate this kind of stuff and in practice I don’t think it is going to happen. But, yes, it’s at least theoretically possible.
14. I’d like to get your thoughts on the Hologic seat controversy; the ratified seat controversy.
Just so that our listeners understand the situation: as a consequence partly of the annual election and partly of the resignation of Doug Lea and Apache and Tim Peierls, various seats opened up on the Executive Committee. We have two kinds of seats on the Executive Committee: some are what we call "ratified seats" where Oracle gets to choose the candidate and presents the candidate to the community and then members of the community vote Yay or Nay. Until the Hologic situation, the members of the community had never failed to ratify anyone that Sun had nominated, so that was an interesting case. What happened here was that Oracle said that they wanted to try and change the balance of both Executive Committees (because there’s one for ME and one for SE and EE combined) [because they] are pretty heavily dominated by implementers of the technologies.
On the ME we have the Motorolas and Nokias and so on. On SE/EE we had Oracle (when they were independent), IBM, HP, Fujitsu, Intel and so on. What we were lacking was consumers of these technologies - the people that were actually going to be taking the specs and actually implementing them and then using them; not building the platforms and shipping them, but taking the platform and then actually building applications on top of it. Oracle thought (I remember thinking this myself and trying to push for this a few years ago) it would be nice if we could get some more end-users on the Executive Committee to give their perspective; that was the intent with Hologic. For whatever reason the logic behind this wasn’t explained as well as it could be and, even though I believe Hologic was and is a large user of Java technology, it wasn’t presented in that way.
The broader community took a look and said, "We don’t know who these people are. Why are they there? We think Oracle is just trying to put their friends on the committee and we don’t approve." That was an interesting situation because it had never happened before; it was an interesting surprise for Oracle. They took it to heart. We still haven’t held that election and the reason is that people inside Oracle, senior executives, are working very hard. They still want, and I still think it’s a good idea, to find a strong corporate consumer, a user of Java technologies. But they’ve learned the lesson: they are working hard to find someone who has a higher profile, someone who, if you go to Google and google the name of this company and Java, you will find out that they actually are big users. Hopefully the person chosen to represent is someone who has something of a reputation in the Java community. That’s what happened there and hopefully we’ll soon settle on a candidate as a replacement for Hologic.
It’s interesting to know we actually have two ratified seats open. A decision has been made on the second and that was an interesting choice, because Oracle chose to nominate a Brazilian Java user group, SouJava. I used to say that they are the biggest in the world with 20 some thousand members, but in fact there is another Java user group in Brazil with 46,000 members, who nobody has ever usually heard of. I don’t remember their name, I’m ashamed to say, but they are Java developers who work for Brazilian government institutions. Anyway, be that as it may, Oracle is nominating SouJava to a seat on the EC with Bruno Souza, a well-known open-source and Java promoter, as their representative and I think this will be a really good addition to the Executive Committee. It will bring a different perspective.
So what they are looking to do is to have this group as a community nominee and then a strong end-user corporate member as an end-user representative.
I’m not a lawyer and I’m not going to render an opinion here. All I can say is the JSPA is a complicated document. It is a legal document. I’ve been reading it not daily, but quite regularly, for several years now and to be honest, there are parts of it still that make my head spin. The language is ambiguous enough that Sun’s corporate lawyers felt that they were able to say that what they were doing (and let’s be honest about this: this whole business started with Sun, Oracle was just unfortunate enough to inherit it.) But Sun’s corporate lawyers said, "We believe we have the right to do this," and other corporate lawyers said, "We don’t believe you have the right to do it." When you have two groups of highly experienced, highly paid, lawyers disagreeing with each other far be it from me to get into the middle of this.
I might say almost parenthetically that - I want to make this point - that the license that was offered to Apache was identical to the license that was offered to everybody else who wanted to implement Java SE. The only difference was others had to pay and for Apache it was free. Apache’s concern was they said that, "Field-of-use restrictions are incompatible with our way of doing business". Sun’s position on that was, "Well, I’m sorry, we’re not necessarily obliged to offer terms that any possible taker would find amenable or satisfactory. We believe we’re being fair because we’re offering you the same terms as everyone else."
As I said though, I am not going to render opinion (I wouldn’t dare) as to who is right in this situation. I will say that if one of the founding documents that the organization is based on contains language that is ambiguous enough that two groups of lawyers can disagree about what it means, then it probably behooves us to go back and look at that language and try and clarify it one way or the other. I’m hoping that we can do that in the future.
I hope they do. I mean they are a really important player in the Java ecosystem and it was great to have them in and it was a shame to see them leave. Whether we can come up with a clarification or a modification of the processes that would make them happy, I’m not sure, and of course I can’t speak for them. All I can say is it’s unfortunate they left and I’m glad that, at least at the moment, the position is that while they are not willing to actively participate in the future development of Java specifications, they do intend to continue to actively implement them, so that’s a plus. Analogously, you talked earlier about Doug Lea and him going to work on OpenJDK - it’s worth quoting he’d always been involved in OpenJDK, so even though in a sense we lost both of them, in another sense they both (Apache and Doug Lea) continue to be involved in different capacities and that’s better than them being gone altogether.