Debate has once again arisen in the community around JSR 277, which is a proposed dynamic module system for Java 7. The flashpoint of the debate this time around is the version numbering system that is planned for JSR 277 Java Modules (JAMs). InfoQ examined the discussions and arguments to understand more about the current state of JSR 277 and it's acceptance by the community.
Questions around the JSR 277 versioning system were first raised by Alex Blewitt, who was concerned about the inconsistent versioning strategy used by Sun in the past. Blewitt also said that the JSR 277 strategy of having four numeric version numbers and a string qualifier seemed too verbose, and pointed out that with Java itself having a version number of e.g. 1.5.0_13, the 1 has not changed in the lifetime of the Java platform, the 0 is a rarely-used digit, and the 13 is related to the build number for the given version of Java (5). Finally, Blewitt pointed out that the way that range specification is done within JSR 277 seemed unnecessarily complex, and questioned why JSR 277 felt the need to invent an entirely new versioning schema when existing versioning formats such as OSGis are well understood and widely used, as well as simpler.
Shortly after this, JSR 277 spec lead Stanley Ho published a response to Blewitt in the form of a two-part article, one part which dealt with the reason for choosing a four-digit versioning scheme and the other part which discussed versioning ranges. In supporting a four-digit versioning system, Ho said:
The fact that some programs cope with three or even two numbers is not a prima facie reason against four. The OSGi policy of three-number versions where the third is for bug fixes (OSGi R4.1 3.6.2) leaves only two numbers for the mainstream product version. This simply isn't enough for some programs, whose OSGi metadata then has to use all three numbers for the mainstream version and the qualifier for bug fixes. Unfortunately, this requires care because qualifiers are compared lexicographically (OSGi R4.1 3.5.3). People who try to simulate a fourth number with the qualifier - 1.2.3.1, 1.2.3.2 - are confused when OSGi deems 1.2.3.2 higher than - and preferred to - 1.2.3.10. So using a qualifier, rather than a fourth number, for bug fixes doesn't really fly. (If OSGi had adopted natural order string comparison, then qualifiers with numeric content would be much more user-friendly, and would be a reasonable alternative to a fourth version number.)
Ho also discussed qualifier semantics in JSR 277:
The Java Module System is sticking with the policy that "an empty string for qualifier is deemed greater than any other value". That is, we view a qualifier as somewhat subservient to "real" versions; a version without a qualifier is higher than the same version with a qualifier. 1.2.3 is higher than 1.2.3-alpha and 1.2.3-beta. This contrasts with OSGi, where 1.2.3 is less than 1.2.3.alpha and 1.2.3.beta. (OSGi R4.1 3.5.3)
Neither scheme is provably more "intuitive" than the other, but the Java Module System follows the traditional versioning policy of the JDK. It is worth examining that policy. (We mean the policy for real developer versions - 1.1.x thru 1.6.x - not the "product" versions - "Java 5.0", "Java SE 6" - used for marketing. Yes, we know the developer/product split is controversial.)
Consider a product which uses the qualifier to distinguish pre-FCS (pre-First Customer Ship) and FCS versions, e.g. 1.2.3-beta and 1.2.3-fcs. A programmer who wants to import the FCS version would have to know its exact qualifier, which is inconvenient considering each vendor will likely have their own FCS qualifier. An easier scheme, which is JDK policy, is to standardize on the unqualified version as the default "acceptable" version, and use qualifiers to denote pre-FCS versions.
If the FCS version is unqualified, how should post-FCS changes be described? The JDK policy is that once a product is given an FCS version, that version's code never changes again. Any code change raises a new version number. (Maybe with a qualifier, maybe not.) This makes support more manageable than if qualifiers express both pre-FCS and post-FCS information on the same version number.
This pair of policies let programmers easily distinguish between pre-FCS and FCS versions and between FCS and post-FCS versions.
Finally, Ho explained how version ranges work in JSR 277:
In the Java Module System, modules describe their dependencies on other modules, and on particular versions of those other modules. Versions and version ranges, like all syntactic elements, are fundamentally matters of taste, but we have tried in the Java Module System to make common versioning tasks easy and advanced versioning tasks possible.
[...]
Where the Java Module System differs from OSGi is in the interpretation of a single version. OSGi interprets a single version like 1.2.3 as "1.2.3 or greater" (OSGi R4.1 3.2.5). To get precisely version 1.2.3, you must write [1.2.3,1.2.3]. Adopting an implicit range is certainly concise, and use of exact versions is sufficiently rare that their [...,...] overhead is not too bad. But millions of programmers will first encounter versions via the Java Module System and it is best to adopt the simplest possible interpretation of any term. There is no doubt that someone seeing 1.2.3 for the first time thinks "exactly 1.2.3", so that is how the Java Module System interprets it. Ranges are denoted with explicit symbols.
This article generated a number of responses, including one from Josh Reed, who said:
I also think that the decision to make a dependency of "1.2.3.4" to mean version 1.2.3.4 and only version 1.2.3.4 is the wrong move. I agree with you that the first time a developer saw that dependency listed they would assume exactly version 1.2.3.4. However, in reality, this is very rarely what you want when deploying an application that consists of modules from multiple sources/developers. I'm a lazy programmer, so if I'm working with Library X version 1.2.3.4, then I'll likely specify that as my dependency. If I incorporate a module from someone else and they also depend on Library X but they were working with version 1.2.3.5. All of a sudden the application has to keep two copies of the same library around when in 99% of the cases, version 1.2.3.5 would satisfy both sets of dependencies. I guess it is an ideological difference, but I think the common case is 1.2.3.4 or greater rather than exactly a particular version. If you are working with a library so unstable that you need an exact version, then in my mind you should have to go to the extra work to say [1.2.3.4, 1.2.3.4]. Make the common situation (of 1.2.3.4 or greater) easy for us poor, lazy programmers; make it easy for us to do the right thing.
So I guess my take away from these two blog posts is that you've worked really hard to come up with a versioning scheme and dependency syntax that is close but different enough from OSGi that you can call it new, unique, different, and right. You won. However, it is a pyrrhic victory as you've spent (wasted?) all this time inventing something that really didn't need to be invented while we're still no closer to seeing how it's all going to work together. To me, as a developer, it's all about where the rubber meets the road and you've done nothing to show me how it's going to work, nor really even convinced me that we need a new approach.
A user named soronthar also commented:
People, please stop bashing SUN as they are not the only ones using a "more than 3 digits" versioning scheme: IBM (yes, the same IBM that created Eclipse and pushed OSGi) uses 4 digits for Websphere and DB2, Oracle uses 5 digits for the database, the Linux kernel uses 4, and I could go on.
The point is: bashing the proposal and say that it is trying to bash OSGi because they use a numbering schema over another is pure... vapour, as other known and successful products have demonstrated the need of a "more than 3 digits version schema" for a lot longer than the OSGi platform have existed (who choose the 3 digit versioning schema anyway? which are the reasons?)
This bashing is even more futile if the spec lead states that the new versioning and the current OSGi versioning will be compatible....
Paulus Benedictus expressed frustration around the JSR 277 process itself:
This JSR is turning into a philosophy class!! ... a debate of minutia. So much thought is going on to reinventing the wheel (i.e., versioning), and all the development is going on in private. Stanley, I don't think the public will give you a break until it appears the work is less secretive and less esoteric. And please, stop comparing against OSGi to show how this JSR uses so-called better conventions. Replicate what works.
Patrik Beno proposed an alternative method of versioning:
With proposed solution (version compatibility based on version history), module system would not have to do any potentially incorrect assumptions, it would not enforce the-one-versioning-policy-to-rule-them-all ... simply, it would be more flexible
Point is:
(a)library client MAY NOT know what future versions he will be compatible with, but
(b)library provider DOES know if he breaks anything from the past or not…
You could have versioning sequence like this (trying to make my point, not to be factually exact):
- kestrel
- merlin (backward-compatible=false)
- hopper
- mantis
- tiger (backward-compatible=false)
- 6.0 (backward-compatible=false)
As did James Ervin, who looked at the reasons for versioning from a higher level:
I like the wikipedia definition of a version in this case. It is pretty generic, but it goes something like this, "Software versioning is the process of assigning either unique version names or unique version numbers to unique states of computer software." The definition tries to cover all cases, so the point to focus in on is the unique identifier of the unique state of the software. So what kind of state are we trying uniquely identify? Well changes to the software you as a developer have made of course, but I think that they fall into two categories at least at the component level. One category are those changes that affect dependency management because the external API on which clients rely has been altered. The second category, I will term bookkeeping for this blog entry. I term it bookkeeping, because it denotes internal changes that should, notice I say should, not affect clients using the external API. "Bookkeeping" comes into play because Software developers are human, and even though a change should not impact clients adversely, they still do. "Bookkeeping" lets a system integrator/admin note that a version of the component X worked, but X+1 sure did not, which is exactly the information someone providing support wants to hear. It is really difficult to say one category matters more than the other. Changes in the first category will prevent a system from coming up correctly, whereas changes in the second category could help explain why the system later goes haywire.
Peter Kriens also responded to the growing controversy, saying:
There are always reason to improve on existing schemes, but any improvement should be balanced against the interests of the existing and future audiences. I am not stating that the OSGi version is the mother of all version schemes, we are as fallible as all of us. Maybe we were too rationalistic; we looked at a lot of schemes and saw how people were also looking for more room at the low-end of the version scheme and hardly ever incremented the first numbers. If you standardize there is always a tension between allowing as much freedom as possible but also minimizing the complexity of the implementations that are depending on the variations you offer. We chose simplicity.
Is the OSGi scheme usable? Well, we have no outstanding requirements or bugs in this area, nor were any proposed in JSR 291, while at the same time it is heavily used. Jason van Zyl, Maven, told me they were thinking of adopting the scheme as well. SpringSource converted almost 400 open source projects to bundles and they did not complain. Seems there is a lot of practical usage out there.
Wouldn't it be a lot more productive for all of us if Sun would just adopt the OSGi scheme? There is lots of work to do on the module API, why not reuse existing specifications where you can? And if the OSGi scheme has burning issues, why not report that as a bug or change request, after all Sun is a distinguished OSGi member.
Ian Skerrett had a different take on the root cause of the controversy:
Hal Hildebrand, the Oracle OSGi guru, provides the best insight into what is wrong with JSR 277: politics. For all Sun’s executive-speak about being a hip open company, they continue to behave like an old fashion hardware vendor. The non-existent JSR 277 expert group is a mockery of the JCP process and by anyone’s definition of openness. Instead of trying to create a community and build bridges between JSR277 and OSGi, Sun is using backroom tactics to find some type of ‘compromise’ between OSGi and JSR277.
Sun stop the bilateral discussions, re-boot the JSR 277 expert group with a real spec leader and start participating in the OSGi organization. In short stop the politics and start a real open discussion to ensuring OSGi and JSR 277 are compatible. Anything else is just going to be bad news for Java.
Alex Blewitt looked at the difference between product and library versioning:
The point is that the versioning system of the underlying components can be completely different from the version number of a product-as-a-whole, which is (invariably) a collection of components underneath. One may have a user-friendly name (Leopard), a numerical version (10.5.3), a build tag (9D34) or indeed other ways of representing the install (/System/Library/Frameworks/System.framework/Versions/B). So arguing for a particular versioning scheme based on the versioning that may be used for marketing (or other) purposes is somewhat futile; both can co-exist and mean different things to different people. Eclipse learned the lesson in the 3.1 days, and since then, Eclipse 3.4 contains plugins likeorg.osgi.util_3.1.300which are entirely distinct from their product name, since they haven't changed other than bugfixes (i.e. no API or incompatible changes). Eclipse's notes on Version numbering are recommended reading for anyone maintaining large systems. Indeed, Eclipse has moved away from version numbers for the product to use code-names like Europa and Ganymede.
Blewitt also pointed out that OSGi adoption is increasing:
[A recent OSGi survey done by the Burton Group] concludes that if the application server supported OSGi bundles directly, then almost 9/10 of respondents would consider developing bundles directly to run in such a server, and tools such as Infiniflow and SpringSource Application Platform are well set for the future.
The question is, where does this leave JSR277? Is it already too late for this me-too implementation?
In an SD Times article, Danny Coward, chief architect for client software at Sun, pointed out that JSR 277 is still just a proposal in the early stages of development. Coward also said:
When an application is deployed in JDK 7 that expresses a dependency on a certain OSGi bundle, when the Java Module System with the OSGi support looks around for the OSGi bundle to resolve that, it will make the conversion step between the versioning numbering scheme in the Java Module System and the one in the OSGi bundle
[...]
There will be a little bit of code in JDK 7 that will manage that conversion and bridge that gap between the two schemes.
Peter Kriens commented on dependency management in JSR 277, explaining why fine-grained Import-Package dependencies are better than coarse-grained Require-Bundle dependencies:
Two very practical use cases that illustrate these problems. Eclipse, who heavily relies on Require-Bundle, needed to support SWT (the graphics library) on the embedded platform and on the SE platform. The embedded platform needed to be a subset of the SE platform. Unfortunately, all users of SWT had been using Require-Bundle. This made the simple solution, refactoring the SWT bundle in two bundles, impossible because it would break each and every Eclipse plugin. Bundles that used Import-Package would have been oblivious of this change.
The other use case is described in a recent blog: Catch-22 Logging with OSGI Frameworks . It is a rather long story but it boils down to that he could not combine two libraries due to unnecessary constraints caused by Require-Bundle with respect to logging. If you have the masochistic desire for the same sensation as having chewing gum in your hair, I recommend to read this blog.
Mandy Chung, a senior staff engineer at Sun Microsystems, also recently announced a prototype of an OSGi Repository for JSR 277:
The draft specification for supporting OSGi bundles in the Java Module System is currently under discussion in the JSR 277 Expert Group.
To enable the EG and the community to provide feedback, we have included an OSGi repository prototype in the OpenJDK Modules project. The prototype currently supports only Apache Felix OSGi implementation for proof-of-concept. There are open issues to be resolved. The out-of-the-box support is yet to be designed and implemented. Supporting other OSGi implementations is to be investigated. We hope that this prototype enables the community to begin participating and provide feedback for OSGi interoperability support.
We are working on posting a JDK binary built from the modules-dev project to make it ready for you to try out and experiment the JAM modules development as well as the OSGi support. In the meantime, you can clone the source from the Modules mercurial repository and build the JDK.
What do you think?