1. My name is Ryan Slobojan. I'm here with Neil Bartlett. Neil, what is OSGi?
OSGi is a module system for Java. It is pretty much the standard runtime module system for Java. Widely accepted, defines the de facto and de jure standard for separating applications into well-defined modules with explicit dependencies between them. It's also a dynamic module system, so it does have the ability to dynamically install, update and remove bundles on the fly, without taking down the JVM. So it has a number of very powerful results from that.
2. Why did OSGi become successful as a modularization technology?
I think just if you look at the history, it's been around for almost as long as Java itself has been around. It started with JSR 8 back in 1997, with Sun, Ericsson and several of the other big vendors. It's been going for all that time and it's got a group of really deep experts in modularization working on it and refining it over that period of time. We see it as something of a recent phenomenon in terms of it's popularity and we see it in a lot of for example application servers in ESBs and IDEs and so on, but it really is quite old technology and quite rooted in good engineering practices.
I think actually its heritage of working in mobile phones and other very small constrained environments was very beneficial and it's never had the opportunity to bloat; it's always been focused on a very tight API, a very core set of requirements with additional features added on top as optional capabilities, but a very tight core. It actually turns out that was very powerful in the Enterprise space - I guess it was somewhat unexpected. The ability to very cleanly modularize an application is something which is very important when you're in a constrained space, because you need to eliminate everything that's not absolutely essential. But it turns out to be extremely important in Enterprise as well because application complexity is just accelerating and modularization is really the only approach to managing the complexity of large applications.
There are 2 problems, really. 90-95% of libraries work just fine and all you really need to do, if you have a standard JAR file, is to add the metadata - the metadata that describes the dependencies that this library has on potentially other libraries. That in itself can lead to problems, because there are an awful lot of libraries out there that depend on the entire world of other libraries, such as the entire world of Apache Commons or something like that. It's very hard to say "OK, I just want to bring in this one library" because you can run some kind of tools and discover that you have to bring in 10 libraries and you have to provide that metadata for those 10 libraries.
The first problem is providing that metadata for the vast majority of libraries and, fortunately, that situation is being addressed with repositories - there is a great repository, which is often by the SpringSource guys where you can download pretty much any open source library fully bundelized. As you said, there is a second problem, which is as soon as a library starts to do clever things with the class loading. A good example is Hibernate. Hibernate doesn't know in advance what classes it's going to load because it will need some classes from your domain model to map it to the database. There are various approaches to make those classes available to that library.
Another good example is something like Spring Framework. I did some work with Spring Framework before they started using OSGi and it was actually already very easy, because they simply provided the ability to pass a class loader. Spring loads classes dynamically by name, but they allowed you to pass a class loader and that was before they ever started building OSGi support. That worked just fine - we could pass in a bundle class loader. Yes, there are approaches to fixing the problem. There are certain libraries that try to do extremely clever things.
The famous example would be Apache Commons Logging, which attempts to scan your class path for an implementation of a log, such as, if it tries to find a log for J or some other log library. That causes the problems in OSGi because there is no application class path. It was doing the wrong thing. By the way, it was doing the wrong thing in other environments like J2EE as well, but OSGi did raise these problems. There is a replacement for Apache Commons Logging, which offers the same interface, but just with static binding to particular log implementation, for example. But this really is the minority of libraries and there is not problems to solve and there is a lot of people working on solving these and just go and get one of these libraries from one of the repositories. A lot of these problems are being solved and fixed all the time.
5. How does adopting OSGi make one change their development process?
I think it opens some very interesting possibilities that were just not there with standard Java. This is a quote from John Wells - who, at the time was a senior architect at BEA - and this is after they decided to do most of their development in OSGi. He said "We thought we were doing modular development until we started using OSGi". The discipline that a true modular Runtime system provides really exposes all those places where you thought your design was clean and modularly separated, that actually was not.
You cannot get away with a little secret back doors between your modules, you have to do it the right way. Of course, you end up with very good quality code at the end of it. Also, it enables you to really component-ize your application. You can more easily now, I believe, separate a large complex application into independent components that can be worked on by different development teams and tested in isolation.
There has been some research done into that. I have to prefix anything I say about .NET. I'm just not a .NET expert, I haven't done development in .NET. So, some research was done by Richard Hall, C. Escoffier, D. Donsez and R.S. Hall. they wrote a paper called Developing An OSGi-like service platform for .NET and they looked at the options. It turns out that .NET's module system has always been better than the standard Java module system or the standard lack of a Java module system, really, that we get without OSGi.
They have the concept of assemblies in application domains and so on, which is really missing from Java. But the key feature that Java has is class loaders that can be unloaded. That enables OSGi bundles to be dynamically installed and uninstalled and updated on the fly, without taking down the JVM.
Whereas, if you are something like an assembly within .NET, as I understand it from reading that paper, that cannot be dynamically unloaded, so you cannot upgrade on the fly. You can map to this separate concept, which is the application domain and that is your kind of bundle, but then it turns out you have to use .NET remoting for all calls between the application domains. That would enable the dynamics, but actually all calls within OSGi are the highest possible performance direct method calls, rather than any kind of remoting with the expensive marshalling and unmarshalling. I think there are some challenges; you can't get a direct mapping of OSGi onto .NET. If we have a ranking in terms of the power of module systems, OSGi would be above .NET, which would be above standard Java.
JSR 277 is obviously going on for a while. By most estimates, JSR 277 is probably dead. It's officially on hold until after Java 7 and we haven't seen anything from the expert group or the spec leads since October 2006 when they last released the EDR - the Early Draft Review - for JSR 277. I don't think that's going anywhere. Project Jigsaw was obviously the big new project announced by Sun at Devoxx last year. There is not that much detail that is public at the moment and I'm not privy to their private communications either within Sun or between them - the OSGi Alliance .They have promised to work with OSGi; there appears to be open communication and OSGi people are certainly open to that, as well. What we have to bear in mind about Project Jigsaw is it's not a JSR, so it's not going to be standard Java. It is an internal implementation detail of one Java VM, which is the open JDK. I know that's the one that most people use, but it is just going to be available on that one implementation of the Java VM.
It's not really going to be something that application developers should use. It's going to be used to internally modularize the JRE libraries, which is a great thing and they can use some OSGi techniques to do that. If you look at Apache Harmony, which is another implementation of the Java Virtual Machine, they have used OSGi metadata to modularize the JRE libraries. I think there could be some input from OSGi, but it's really up to Sun whether they go with it. It's going to be interesting to see where that goes, but there isn't very much public information about that at the moment.
On the other hand, there is a project, a separate JSR. This is a JSR which is 294, which used to be called the "Superpackages JSR", but now it's better known as "modules within the Java language". This is adding direct support modularity within the Java language. OSGi being something that was built on top of the Java Specification and the Java Language Specification has never had the opportunity to change the Java language. We describe all of our metadata, both bundles or modules within the manifest Java JAR files, so there is an opportunity within JSR 294 to add keywords and possibly annotations to the Java language to support moving the metadata to the actual Java source. That is something that people within the OSGi community are quite positive about because we can work with that. It's not necessarily a competing module system to OSGi, it's a Java language feature. If it interoperates with OSGi, then that's great.
It's quite a close race. They are all very close fits to the specification. They are all very compliant to the specification, so a lot of the time - I mean it should be 100% of the time - you can write bundles and they will work on all 3 of those platforms. Equinox being the implementation that is part of Eclipse, is bigger so it comes in as a larger JAR file and it perhaps uses a little bit more memory but it is optimized for the large use cases of many bundles. It's really optimized for Eclipse and certain uses of Eclipse. That doesn't mean it can't be used for non-Eclipse uses - far from it! It can.
It's very heavily optimized, but it's a little bit bigger, so perhaps you wouldn't want to use that on a very small device, you'd look at one of the other ones. Felix and Knopflerfish are smaller. Knopflerfish is probably the most mature because it's been around for a very long time. As far as I'm concerned, there is not a big difference between them, except perhaps in licensing, whether the Apache license or BSD or EPL is your preferred license. There is a fourth one that you could mention actually called Concierge.
Concierge is not an OSGi release 4.0 implementation, it's an OSGi release 3.0 implementation, so it's one behind, but it is extremely small and lightweight and you see it being used in a few embedded applications such as some modular hardware coming from a company called Bug Labs - they use Concierge. That comes in as an 80k JAR file, so it's really compact. As for comparisons, do testing on several of them, do your performance testing. SAP did some performance testing. They did find that Equinox was faster in most of the use cases and the metrics that they tried, not by very much and not is all the use cases, but that's what the result was.
9. What do you think will come with OSGi release 5.0 or revision 5?
That's bit far than the line. I don't really know. I guess what we will see is trying to provide more support for Enterprise use cases. We do have going on the Enterprise expert group, within the OSGi Alliance and that's trying to define support for Enterprise use cases. This isn't necessarily trying to replace or compete with something like J2EE. It's more, if you are using OSGi and you are in an Enterprise context, and you need to use existing Enterprise services like JMS, JNDI, all of these Enterprise features, what is the best way of using those within OSGi, as an optional thing on top of what remains a very lightweight core? I think we'll see that developing more. Other features of R 5 I just can't say - that's going to be several years away. What we've got coming out soon, I think this year is release 4.2 and that's mostly some fairly nice, but mostly incremental improvements to the Framework.
I guess OSGi at the moment is used in a lot of what I call infrastructures. We do see it in pretty much all of the application servers that are either already using it or moving to use it. The same story with the ESBs, the same with Eclipse IDE and I believe it's a matter of time before NetBeans starts using OSGi for its module system as well, but that's just my prediction. Where it needs to grow at the moment, is with application development. The guys who are writing banking applications or telecoms or invoicing applications are really starting to push the modularity story to people who are not building infrastructural software products, but really business applications and that's a big push, at the moment.
That's what I talked here at the QCon was why it's beneficial to application developers and not just infrastructure product developers. A lot what you see with the application servers - as I said, they are almost all using OSGi, but - it's kind of hidden as an implementation detail. WebSphere runs on OSGi, but you can't simply deploy an OSGi bundle into WebSphere, you have to use the J2EE standard deployment descriptors and so on; similarly, at the moment, with Glassfish V3, but that's a very rapidly evolving product, similarly with what appears to be happening with WebLogic and JBoss.
On the other hand, if you look at something like SpringSource's DM Server there attempting to open up. The OSGi APIs and the OSGi deployment methods directly to application developers so that they can write OSGi bundles using services that are consumed from other bundles directly within their application server platform. I think that approach is got to become more popular across several of those application server platforms, as people notice the benefits to their application development life cycle.
There is a question of whether you want to run just Equinox or Felix or one of those as your main Java Runtime and have everything deployed to that as bundles and essentially, you are not going to reuse a lot of the existing infrastructure of something like WebSphere or WebLogic, if that's what you have. I think that's one valid approach, but of course, there are people who want to say "We've got the WebSphere product running, we've got all of our support processes around that, so, while we would like to use OSGi for development of our applications, in reality, deployment has to be to something more of a legacy application server."
There are techniques to take an OSGi Framework and embed it within a larger application, whether that's an application server or something else. For example, you can embed Equinox as a Servlet within any standard J2E container. I've done this myself with WebSphere. You would have the OSGi Framework running and HTTP request being rooted to a particular service within that OSGi Framework, so that's a good approach, if you want to develop a modular web application within that existing infrastructure. I guess I would like to see more of those legacy things just die off and be replaced with more of a pure OSGi Framework, but it's a matter of going from here to there, really.