Last week Peter Kriens, one of the main drivers behind OSGi, announced on his blog his return to the OSGi Alliance, where he served as director for 11 years until early 2012. InfoQ caught up with Peter to discuss his return and latest project jpm4j. (The opinions expressed are those of Peter Kriens and do not necessarily reflect the opinions of the OSGi Alliance.)
InfoQ: Welcome back to the OSGi alliance, Peter. What have you been doing since you left, and what will you be doing going forward?
Peter: My primary reason for leaving was that I needed to be back in the trenches. My computing career started in the late seventies and due to enormous luck I was able to develop pretty advanced distributed systems at a very young age. Then later Ericsson gave me a fantastic opportunity to lead some really large projects and they topped it off by paying me handsomely to play in their Application Research lab in the late nineties. These were incredibly practical formative years until OSGi became the place to send invoices in 2001.
However, after more than a decade of spec writing I felt like I was too often talking theory not based on deep practice. Though I think this is pervasive in our industry, I become very unhappy when a technology is not in 'my fingers'. This nagging became louder because of the increasing importance of the OSGi Enterprise Expert Group. My experience was more from the (semi) embedded world so many of the 'best' practices puzzled me.
The second reason to leave was all about repositories. I think there are a number of problems with the way we handle binary artifacts today. Don't get me wrong; Maven Central is an incredible asset for our industry and not just for Maven users. However, its underlying file/directory based model is too simplistic in the long run. Applications are growing in complexity all the time and I think we need more functionality for repositories, that more intelligently organizes the artifacts. I tried to push such an OSGi repository in the Alliance but unfortunately never got any traction. Combining this subject with my desire to be in the trenches again sounded like a wonderful idea for a sabbatical, and man did I enjoy myself!
However, in the beginning of this year it became clear that doing jpm4j right would require more people. I'd start to talk to different companies to see if they were interested but I could not find a company that was willing to sufficiently invest in this project. Now, business development is not my strength and I did achieve my primary goal of learning so overall I am actually pretty happy, and happy to be back.
InfoQ: There is so much work going into OSGi, it seems disproportionate to the consumer base. What is being done to spur traction?
Peter: Well, I am hired to increase this adoption. To understand our strategy, let me explain some of my experiences in the last year.
For jpm4j I had embraced OSGi and was developing with Bndtools, Neil Bartlett's Eclipse plugin. Having to actually build a real life-sized application that leverages OSGi was pretty hard because most JEE libraries (and thus open source libs) look so clumsy and awkward in OSGi. Since I did not want to compromise (I was on a learning exercise after all) I had to develop a number of solutions that were 'native' OSGi. I developed bundles for subjects like security, queuing, configuration, batch jobs, mongo, browser integration, etc. These components turned out to be so much simpler than their JEE or open source counterparts. Not that I am so brilliant that I can replace these sometimes sizable libraries with a few weeks of work, unfortunately not. The reason I could get away with it is because OSGi provides a set of primitives that most libraries replicate in a proprietary way. By standardizing life cycle, dependency, and configuration primitives the components tend to have synergy out of the box. If you embrace OSGi you find that you often have to write very, very little code since the environment handles a lot of the mundane, error prone, details. In the non-OSGi world, since there are no such standards for service discovery, life cycle, and configuration, you must often find yourself repeating yourself as well as writing glue code.
Now combine this with Bndtools. I admit not trying out all IDEs in the market but I am awed that my OSGi framework keeps running while the bundles are updated hundreds of times per day. It is so nice to hit Ctrl-S, switch to the browser, refresh it, and then see the results. It is as short as a Ruby on Rails edit-debug cycle but now with all the power of Java, Eclipse, and OSGi under the belt. Now you can have your cake (quick edit-debug cycle) and eat it too (refactoring, quick help, type safety, navigation, strong modularity, µservices, etc).
So I actually fell head over heels in love with OSGi ... again ... However, this time with a much better insight into its strengths in the enterprise as well as its shortcomings. I always said that OSGi is a fantastic concrete foundation, ready for the tallest skyscraper, but that there is just too little support to build even a nice little townhouse. The result is that many people have to integrate the pieces themselves. What was needed was an OSGi application skeleton geared towards the common needs of the largest group of developers, the web application developers. I became convinced that we could build something like this for OSGi without too much effort. Virtually all the parts were already there; what was missing was a way to put all these parts together into a complete application.
So I proposed doing such a skeleton to the OSGi Alliance. IBM's Dan Bandera, the OSGi Alliance president, made a proposal that was discussed in the board. I was pleasantly surprised when I was told that they wanted me to start immediately! Since I still want to do some work with jpm4j we decided that I would split my time between the Alliance and this jpm4j work.
InfoQ: Can you tell us about jpm4j?
Peter: A couple of years ago I realized that over time, public repositories will grow forever because you can never remove any revisions. Removing a revision might break some builds out there! What is much less understood is that adding a revision to a repository can also break builds when you use version ranges or the powerful OSGi Capability/Requirements model. A build with version ranges can work today and fail tomorrow because the repository had a newer revision available. That is, the content of the repository is a first class citizen in your build dependencies. This usually not-so-well understood aspect of repositories is the main reason why Sonatype advises people not to use Maven version ranges.
The fact that repositories need to be versioned like, and in conjunction with, source control was already clear to me many years ago; inside the OSGi Alliance we always managed the binary dependencies with our source control management system for this reason. I figured that there should be an opportunity for a cloud based repository service, that is, a bit like Github for binaries.
However, to be able to version repositories you first need to have a catalog of all possible artifacts. After a weeklong discussion with Karl Pauls and Richard S. Hall (OSGi committers and co-authors of Manning's OSGi in Action) I came to the conclusion I needed a public community site with a catalog of all available artifacts. This catalog could then be the base of my versioned repositories. However, having such a catalog also allowed me to address a huge frustration, what I call the 'last mile' of Java. Virtually all dynamic languages have some way to install their applications on different operating systems. NPM, Gem, CPAN, etc. While Java is arguably the most portable language ever, and has the enormous advantage of portable binaries, developers must jump through too many hoops to install an application. So jpm4j has the jpm command that makes it trivial to install apps locally on the command line or as services. Jpm knows how apps are handled on Windows, Linux, and MacOS and installs these apps at the appropriate place.
This feature, although it has taken somewhat of a back seat over time, is what gave jpm4j its name: "Just another Package Manager for Java". That said, it is not finished and there is still a lot of work left. The total project if done right is quite sizable. As said earlier, I am talking to a company that is willing to provide a good home for it and fund further development.
InfoQ: Sounds interesting on paper, but isn't there a chicken and egg problem, bootstrapping the repository with a large base of useful bundles?
Peter: Well, my primary goal at the time was not OSGi. Yes I know that sounds odd. Jpm4j is a Maven repository since this is by far the most popular repository model. So I wanted to provide these versioned repositories to the Maven world. That said, since my heart was still very much with OSGi (more than I realized at the time) I made sure OSGi was a first class citizen in this world.
It was straightforward to start with all artifacts in Maven central. The jpm4j core is mostly a catalog of JARs, so I wrote a scanner for Maven central and automatically imported all their artifacts. Jpm4j scans all JARs and when it detects an OSGi bundle it catalogs it as such. Jpm4j can pickup Maven metadata and OSGi metadata, and has a plugin model for other metadata. There are currently over 400.000 revisions in jpm4j.
However, though being Maven centric, jpm4j is agnostic of any repository formats. I also made it very easy to create binary repositories on Github, you only need to provide a post hook on Github and then jpm4j will scan the repo and updates its catalog. A Jenkins build could do similar hooks. I planned to also scan all Eclipse bundles and have a number of company repositories included.
So I tried to make it very easy for people to open up their bundles to the rest of the world. I think I succeeded there; the threshold is actually quite low.
InfoQ: What about the consumers, do they need to migrate to OSGi before they can consume a bundle? Or is this intended for projects that are just kicking off?
Peter: Well, neither I think. OSGi is a significant investment since it is not like some libraries you can just slap on the classpath and get some fantastic new functionality. OSGi is so small because most of the work is done by you, the user. Parnas's seminal modularity paper is all about how to decompose a problem into modules so that coupling between modules is minimized and cohesion is maximized. The simple lessons we were taught in the seventies got trampled on by the Object Oriented community when we tried to evangelize the OO paradigm. OSGi is only the fabric that provides the primitives to execute such modules.
OSGi's huge benefits are realized by being able to do more with less code that is also much easier to maintain. However that affects almost any part in the development process when done correctly. Though there are clear migration paths, I would not try OSGi without a willingness to change. It is very similar to loosing weight, temporary diets rarely work long-term; you need to change your lifestyle if you want to get slim and stay slim. So if you want to keep the cost of your bloated code base in hand, there was only one quick fix: wait until computers got faster. Now that Moore's law is running out for classic sequential programs, that quick fix is becoming harder.
That said, the threshold to start with OSGi is way too high. Too many developers suffer a long learning curve due to this steep threshold. Obviously I am aware of the application server frameworks out there but that is not what I mean. These frameworks are not leveraging the real strength of OSGi. In their quest for JEE compatibility they tend to become a combination of the drawbacks of both environments. The coolest feature of OSGi is the service model but none of these environments embraces it. I hope to show how this should be done with my new OSGi effort.
InfoQ: OSGi requires a great degree of discipline, and perhaps that discipline is exactly the missing link that makes software engineering so much fuzzier than other engineering disciplines. But how can we expect teams who have been doing it one way forever to change course, acquire new quantum gains in discipline, and start thinking in terms of modules rather than the ad-hoc architectures so prevalent in our industry?
Peter: Spot on, as I said earlier, OSGi is a life style, not just a diet.
Now the interesting point you raise is how we are getting away (what is it, a 500 billion dollar industry?) with our way of working? There are many reasons but the root cause in my humble opinion is that we are not held liable. If Amazon violates its SLA they only refund you. If employees are not paid because the payroll program crashed, well, tough. If quants blow up the financial system, well, the society bails them out. Every enterprise developer over 30 has worked on failed projects but I do not know anybody that lost his job for that reason. In contrast build a bridge and when it collapses try just shrugging it away; that would never fly.
In the embedded world you see that software is made much more carefully since they usually remain liable for their software. For example car companies know that any failure on their part results in very expensive law suits. However, in the enterprise software world there is virtually no such feedback loop. The result is that anything goes. I hope we are the only engineering discipline with rock-stars, trends, and fashions. This lack of liability creates all kinds of false incentives in our industry.
It never ceases to amaze me how developers will spend copious amounts of time but utterly refuse to spend even a dime of their money. Liability would deeply involve non-developers in the development process. For non-developers the trade off between time and money is much more transparent. I am sure this would quickly drive the case for OSGi.
Until that time, the most important thing for OSGi is to drive adoption by grass roots efforts. This is clearly one of the goals of my work for the OSGi Alliance. By making it easier to get started with OSGi for the most common cases we hope to increase knowledge of OSGi and show the benefits of the service model. A lot of developers love the architecture of OSGi, it is the getting started part that throws them off I think.
InfoQ: Will you be managing any other roles back at the OSGi alliance?
Peter: Though I will not be writing the specifications this time, I will participate in some of the technical expert groups and I will of course visit the conference circuits again. I plan to push for a number of specifications that would make web applications easier to build. I also participate in board meetings. For my other 50% I intend to push jpm4j further since it is clearly not yet finished.
InfoQ: Is there anything the community can do?
Peter: I love to get feedback on the ideas around the OSGi effort to lower threshold. And I am also interested in any people that want to discuss jpm4j. I intend to have the community as much as possible in both efforts.
InfoQ: A personal question, I have seen you present many times at various conferences, and your presentations are always as breathtaking as the technology you are presenting. How do you do it, do you have tips for presenters? How do you locate or produce such spot-on graphics, tie them to a pithy blurb, and pump out a meaningful presentation in your inimitable style?
Peter: Wow, thanks. I actually always feel kind of inadequate in this area every time I step on the stage so it is hard to answer this rather unexpected question.
The only thing I can say for myself is that I am very visually oriented. The primary reason is that I can stare for hours at an XML document and not get it. But show me a good diagram and it seems obvious. Just look at the OSGi specifications; few specifications have so many very carefully made illustrations. The main reason was to make sure I actually 'got it'. I have many friends that do not look at the diagrams and are able to get the same information from an XML document, and I am always utterly impressed with them. My brain needs graphics and I guess that rubs off on my presentations.
About the Author
Peter Kriens is an independent consultant since 1990.He currently works for the OSGi Alliance and jpm4j. During the eighties he developed advanced distributed systems for newspapers based on microcomputers based on, at the time very novel, object oriented technologies. For this experience in Objects he was hired by a number of international companies, including Adobe, Intel, Ericsson, IBM, and many others. During his work at Ericsson Research in 1998 he got involved with the OSGi specification; Later he became the primary editor for these specifications. In 2005 he was awarded the OSGi Fellows title. After taking a sabbatical in 2012 to develop jpm4j he returned to the OSGi Alliance to help increasing adoption. He is Dutch but decided to live in France. Follow Peter on Twitter @pkriens.