Bndtools is an open-source project and it actually started when Neil Bartlett was doing some independent consulting and training with people and effectively what he wanted was to use Bnd, which he thought this was this great tool, and he wanted to use Eclipse. But actually he found a complete nightmare. So he started to put together this plug-in for Eclipse to basically make it easier to do his training courses. And that slowly evolved into what was the first release of Bndtools, which was then picked up; and so Peter Kriens, who is the author of Bnd and one of the originating creators of OSGi, noticed it and thought “wow, this is really cool!” and so they got together with B. J. Hargrave who is the [OSGi] Alliance CTO and a number of other people who joined the project over the last few years and worked on it a lot and that's what's become the Bndtools that we know and love today.
Okay. So, the past couple of releases have been quite big ones for us. So we had 2.2, which was the one where we introduced baselining. And then 2.3 added some fixes for baselining but also a lot of detailed work went into the repository resolving, which there had been some problems with, so there were some fairly significant fixes there. And also our repository integration with things like JPM. 2.3 also added Gradle plugin support for building, so you can now build with Gradle as well as with Ant. And the first milestone we thought was important to get out because we actually wanted something that worked really well with the first release of the Maven Bnd plugin which shouldn't be confused with the Maven Bundle plugin, because effectively what we've got now, are two plugins that can use Bnd either starting from a kind of Maven-centric view of the world, with the Maven Bundle plugin, or a Bnd-centric view of the world over here, with the Bnd Maven plugin.
It's about how you choose to actually build and package. So if you're really looking to use Maven all the way through your stack then the Maven Bundle plugin is probably a better choice. But if what you want is to be able to say release into Nexus using Maven, or integrate with other tools that use Maven build as their start point, but actually you'd prefer to drive things through the Bnd file and to get therefore some of the slightly better integration with some of the existing Eclipse tooling, then you can go that way with the Bnd Maven plugin.
All of the version information is automatically pulled through from your dependencies or from package-info files or the OSGi version annotation, which is one of the reasonably recent spec editions. So there are lots of ways of getting the version information in, but what there hasn't been is an easy way of making sure that when you make a change that version is still correct. So what baselining is doing is looking at all of your exported API and it takes your exported API and it actually performs a diff against the previous release in your baseline repository, which is typically going to be the repository that you'd previously released to, or it might be Maven Central, or wherever your artifacts live. And it will find the previous version and it will say “okay, since then you've added this method, or you've deleted this method, and then based on whether the interface is one that's for providers or whether it's one for consumers, you'll end up effectively getting a nice red cross and a broken build until such time as you fix the baseline error. And there's actually quick fixes for that in Bndtools so you go there you hit Ctr1+1 enter and it will automatically fill in the correct version for you.
Yes. And it does that very nicely. It also actually picks up that if you've made changes to the bundle, so the binary content has changed and you haven't updated your bundle version, it will also flag that to you, so it will push you into making the correct version updates at bundle level as well.
Alex: Automated semantic versioning.
Automated semantic versioning.
6. How does Bndtools work with repositories like Maven Central?
Bndtools has a plugin model for its repositories. It comes with a standard few out of the box, so you've got local indexed repositories, or fixed indexed repositories, for something that you've got. So local indexed will be on your file system, a folder somewhere that you're going to drop bundles into and it will produce an index and that you can use that to resolve. Fixed indexed repos, are a bit more like you've got a nexus repository somewhere and you're using the OBR plugin for it to generate a repository index, but you can't push to it, you know, it's on a http URL and you can't just push things in, but you can consume from it. There is also in one of the other things that came in 2.4M1 was some improved support for the JPM repository, so you can pull thing in from JPM very easily. Actually you can do a search, it will bring up a web-browser window in Eclipse and you can just drag-and-drop from the web into your build-path, and it will automatically get pushed into your repository and it will get added, and you get the option to add transitive dependencies to your build-path as well.
7. What is JPM, for those who don't know about it?
JPM4J is effectively a project that Peter Kriens went to work on for a while. He had been working with the OSGi Alliance for a very long time, I think it was over a decade, he'd been writing all the specs. He decided, you know, really he wanted to get his hands dirty again, because it has been a long time since he had really had projects where he was writing code and producing a big system. So he had this idea, which was what we have in a lot of languages, things like ruby gems, or going way back to Perl, you can just pull in things and install and get them running, but Java doesn't really have that. I mean you've got Maven Central, which is a source of lots of stuff, but it's not always that easy to search, and you can't just kind of pull stuff down and run it. So JPM has a client and a large repository and there's a lot of cool things you can do with it.
But one of the things that it does automatically is it scrapes and indexes information from Maven Central. So I think it's now over 600,000 JARs that are in there and some huge number of actual programs that that relates to. So what it gives you is the ability to plug into it and pull stuff down and if you're using something like OSGi, where you've got the requirements and capabilities metadata that you need, you can actually search for all the things that you could possibly use and you can pick or choose what you want from the repository and build your own essentially local view of Maven Central with just the things that you want in it.
So in some senses it doesn't. You can still require things by identity in OSGi. And actually when you are building up a build-path that can be quite a sensible thing to do. You want to know “these are the things that I built against”. But actually at runtime and certainly when you're putting together tests, you don't necessarily want to tie yourself completely to something by identity. So actually if what I need is the Servlet API, and then in my tests I need to run in a Servlet container, do I really want to say “okay, that means I've got a hard compile-time dependency on Jetty or Tomcat. What if I want to switch between them? What if I want to use them both?”. So what OSGi gives you is the ability to take those things, and you can depend on things maybe by contract. So Java contracts are something that have recently been brought in to cope with some of the more complex versioning, that they're doing with Java EE specs. But you can also just depend on things like packages, or capabilities provided by something. So the OSGi extender model has its own capability namespace and you can say “I need a Declarative Services runtime”, “I need an HttpService” and you can just pull those things in towards you. And you get them effectively from the repository for free.
If the implementation advertises that in its metadata, then you can ask JPM and it will provide it. So again, you don't get everything for free, you have to make sure that you actually advertise the capabilities of your bundles using the standard headers that are out there, but a lot of declarative services implementations are now doing it, it's the latest rev of the spec, actually defines the extender namespace for the declarative services and says if you're an implementation of the latest DS spec you've got to do this. So it's there, people are doing it, and you get a lot of flexibility from it. I mean if you're just after some of the more traditional requirements then it's great. You can search JPM4J with P: and then a package name and you can find everybody who provides that package. It's quite interesting to see how many different people provide org.apache.commons logging.
11. How does Bndtools actually assemble its classpath when it's building bundles?
What Bndtools does is actually takes the Bnd file and it will look through your build-path there and it will assemble that into an Eclipse classpath container, so if you look, what you'll see is you've got your standard kind of “here's my JDK” and then underneath that you'll see another container that has the same kind of icon saying “this is the Bnd classpath container”. And now if you untwist that you'll notice there is this list and it's all the things that you've listed in your build-path. And what that does it that actually sets up Eclipse so that the ECJ compiler is running with all the same stuff and is producing the same artifacts that you would get in an offline build. So you get that same incremental and rapid on-save compile and rebuild that the Eclipse build-path and Eclipse container gives you, but you're getting all of the advantages of actually the packaging happening using Bnd.
No, it creates the JAR file for you. It's packaged up using all of the packaging instructions that are in your Bnd file. And what's even better, if you're currently running a test framework and that bundle is deployed there, the framework will get hot updated with the new code, so if you're debugging and you see that there is a problem, you're not relying on hot code replace from the debugger, which as we all know can be a bit iffy at times. You're actually deploying the new code there and then your bundle gets restarted and it's there, you know, it's the fastest development cycle I've ever been involved with.
Alex: You can save the change, switch over to the browser, hit refresh and you see the changes immediately.
Yes, absolutely.
So if you want to depend on other projects within your build workspace that's fine. The Eclipse workspace is actually treated as another kind of repository. So what you'll have there is you'll have all of your build projects and the output of those is considered to be something that you can theoretically resolve against or use as a build dependency. What do you need to do is specify that you want the latest version, because otherwise the build workspace is considered quite volatile and a lot of the time you don't want to build against exactly what's in your workspace, you want to build against the previous release. But if you say that you want the latest you can build against your workspace and it will just be there.
So there were at a couple of points some issues with the older Ant build, where if you were doing some of the build using Ant, it could take a while to do the rebuild. Actually the incremental build in Eclipse is very fast as long as you set it up correctly. That's certainly been my experience. I've worked on workspaces that have up to a couple of hundred build projects, but actually with probably about 800 or 900 outputs from those. And even then we're talking about seconds for changes, even in fairly deep guts of it. So, you know, obviously if you have a very large Eclipse workspace then you'll have to pay some level of cost for that. But it's not as big as you might think.
15. How do you testing within Bndtools?
Bndtools testing. If you're using things like Declarative Services annotations then your objects are POJOs and unit testing is really easy. It's exactly the way unit testing should be. But what Bnd also gives you is the ability to run in-framework tests. So you have these things called Bnd runfiles which you can use to launch and play around with the framework inside your Eclipse, but you can also use them to define your test definitions, or you can create a separate, we typically call them, integration test. You have an integration test project where you define your test classes again using JUnit, but it will get packaged up and run as an OSGi bundle. So you can actually do things like register services and interact with the bundles you've created and see how they're working. And it's nicer.
There are a couple of things that PDE does better if you're writing an Eclipse plugin because it's designed to write Eclipse plugins, not just OSGi bundles. But everyone I've ever spoken to, who's used both, has said that Bndtools is their choice everyday of the week and twice on Sundays. It's something where actually we get regular requests to say can you look at whether you would be able to actually replace PDE with Bndtools because people just find it so much quicker and so much easier. And part of that is that they just have a fundamentally different philosophy about how you put the bundle together. So PDE is what's called a manifest-first solution. So you're responsible for writing your OSGi manifest. You know, you author that file by hand, you're responsible for saying Export-Package, Import-Package, all the rest of it. And that is fine, except for the fact it's really difficult to reason about what all your imports are. A lot of people I know think they know everything that their bundle depends on.
And then they put it into Bnd, which is a code-first solution, which generates the manifest for you and analyzes your bytecode, so it knows exactly what dependencies you have. And you often find that people are depending on things they never knew they were depending on. And this is one of the fundamental differences between PDE and Bnd. It's about how you write your OSGi bundles. Humans aren't very good at working out dependency graphs. Humans aren't very good at working out uses constraints. So it's a lot safer to have the tool do that for you and then, you know, cope with the results you get and then maybe change your code to try and factor out a dependency that you don't think you should have. But at least that way you know you've got the right answer and then you can change your bundle to try and fix stuff that you think shouldn't be there, rather than saying this is what I think the answer is and stuff randomly blowing up later.
It's interesting. One of the things that we're really quite excited about is the fact that you can just export things to executable JAR files. So you can create a project output and get an executable JAR file that you can run with java -jar and it will run your application. You don't have to use all of the OSGi functions unless you want to. I mean we would wholeheartedly recommend that you do; OSGi services are incredibly powerful and the modularity you get is incredibly useful too, but if you don't want that you can still use this tooling to help generate and maybe understand the dependencies that you have. But then just run it as a JAR file anyway.
18. And does that JAR file essentially contain all of the classes that you depend on?
The JAR file will aggregate up everything that you need. You'll launch it and away you go. It looks exactly like when you run it within an OSGi framework in Eclipse, it's just you've not have to go through the hassle of managing your own OSGi framework. It's all just done from the command line for you.
Bndtools is obviously an Eclipse plugin.
Yes.
So it was started as an Eclipse plugin and an awful lot of it is tied quite heavily to the Eclipse API, because that's how the Eclipse classpath container works and it's how Eclipse incremental build works. We've had a lot of calls asking if we can port it to IntelliJ and if we had the time and the money backing us to do it, then we'd love to do it. But equally if the guys over writing IntelliJ want to produce their own OSGi tooling, based on Bnd or not, and they blow us out of the water, we'd be happy too, because what we want is we want it to be easier to write OSGi stuff. So if someone can provide that in IntelliJ, that would be great. If they want to help us to do it, than that's great too.
20. In terms of the Bndtools project, where can we find more information or get involved?
Bndtools.org has a lot of information about how you can install and how you can use Bndtools for the first time. There are also a number of tutorials and there was a great Bndtools 101 talk, that was on at DevCon which I think the slides will be up on InfoQ over the next couple of months. So that's one way you can look at getting started if you're wanting to develop with it. If you're wanting to help improve it then the Bndtools project on GitHub is where we do all of our issue tracking and our feature planning and our release planning. So if you want to get involved then raise bugs, help, submit patches and you know, help us. We'll gratefully accept anything that you're willing to do.
Alex: Tim Ward, thank you very much.