1. Hi Brian, would you like to introduce yourself?
My name is Brian Warner, I’ve been working at Mozilla Labs for about a year now. Before that I was at startup doing a distributed storage systems cryptographic security kind of thing and I’m working specifically on the Jetpack Project, which is our effort to make a new platform for building add-ons for Firefox, Thunderbird, Fennec which is Firefox mobile, that whole suite of applications.
I should warn you I’m not the best authority on this, I haven’t started working with the Mozilla platform until I started working there about a year ago, but as I’ve seen it, add-ons have been written as a collection of XUL, the API language that it uses and JavaScript which is kind of patched into the internals of the browser. Most of the add-ons were written a lot of people had to fend for themselves to find useful interfaces to connect to. As a result some of those add-ons tend to be very fragile. It’s very common for add-ons to break when the browser is upgraded. A lot of people don’t want to upgrade their browsers for fear of breaking their favorite add-ons, which drives the security team nuts because they want to have people on the latest code.
It’s been challenging to reuse code. There was not a lot of sharing, there was not much of a module system on add-ons before the Jetpack effort.
The important thing to be aware of is that Jetpack is an SDK, it is a set of tools and a set of libraries that you use to combine code that you’ve written with modules that are provided in libraries that come with the SDK and modules that other people have written that you’ve downloaded from other sites and combine them all together into a traditional XPI. The output of this process is the same kind of XPI as the old style of add-on. We’ve not really changed very much about the browser itself to accept these things. There are a few tweaks there: there is the ability to install add-ons without restarting the browser. But we haven’t done any large scale changes to what’s inside the browser for the purpose of Jetpack.
A Jetpack add-on is a collection of CommonJS modules, if you are familiar with that. CommonJS is an emerging standard for writing JavaScript code that has well-defined entry points, well-defined exit points. So you can tell what functions you’re providing to the outside world. You can clearly declare what sort of functionality you are getting from the outside world when you import those modules. A Jetpack add-on is a CommonJS top level module that uses the required statement to access functionality coming from other modules, either from third parties or things that are inside the SDK. Then, those are combined together along with the bootstrap function, along with the module loader into the same standard XPI (really a ZIP file) that the browser knows how to load.
4. All the modules you mentioned at the beginning - you are talking about JavaScript code?
Yes, and the SDK that we’re shipping right now, the 0.9 release we made about a week ago contains APIs for about a dozen of the most common functionality that we figured people would want to start with. There is the ability to inject content in pages - the page module; there is the tabs module which lets you find out what tabs are open or open up new tabs. There is the selection API which lets you find out which text is currently selected or to modify it; storage - there is an HTTP request module - the basic set of stuff. The APIs that are in there right now don’t really begin to cover the full range of things that the browser can do as a whole, but anybody can write these modules - it’s just JavaScript.
If you are familiar with traditional XUL-based add-on development, then you are in a really good position to write one of these modules that will expose a certain piece of functionality to other people. Folks that are in that position can help people that don’t know that layer to get more functionality out of their add-ons.
As I was describing in my talk, the browser is really becoming the interface with which people interact with the entire internet and the most important way is the browser is the user agent that is on the user side. It’s negotiating with lots of parties, but the browser is the thing that is rooting for the user whereas all the other sides have their own interests. In some ways I think it’s important that the browser retain that role and add-ons that are installed there can be very powerful, they can do interesting good things, they can do interesting bad things. I think there is a danger of having add-ons. If a side is relying upon an add-on to give the functionality in the user’s browser that the browser wasn’t going to provide them natively, then there is a bit of a question of whether that site is really following standards.
In some way the site might want to just ship a full scale application and they are shipping an add-on as an easier path of getting part way there. So I think that add-ons in that case can be very powerful, they can do anything they want, but I worry about it a little bit because I think it starts to confuse the power interests within this agent. That an end-user who is relying upon their browser to do certain things for them may find that it’s now doing things for somebody else instead. That said, I think that the add-on platform gives users and websites like that a very powerful platform to do interesting things. It’s how that gets used and how much control and awareness the user has that I think it’s the determining factor of what people pay attention to.
I’m not involved in the process myself, but I’ve met a lot of the reviewers that we have and I’ve looked at some of the public documents they have explaining what their process is. AMO add-ons that mozill.org is the host for most of the add-ons that people get. There is a gallery there, there are a lot of search tools, there are a lot of user ratings and there is a team of people. I think it’s on the order on a dozen people or so that had the responsibility of looking at the new code add-ons and make a decision of whether this is safe and whether it does what it says it’s going to do and whether that’s something that’s appropriate. If it passes through that process, then it goes up in the gallery and people start rating it, they download it.
As I understand it, they have some tools to help them. There are some automated code review tools. They can look for some well-known unsafe patterns that are in there. They can look for certain functions that are being called, things that are known to be troublesome. They can also look for differences between versions. If they’ve reviewed version 1 and now they are presented with version 2, the tool shows them differences between those two so they concentrate the review effort. But it’s a manual process and part of what I’m hoping to accomplish with Jetpack in the corners of it and the pieces of it that I’m working on, are to improve some of those tools and to create add-ons that are structured in such a way that we can get more information in front of those reviewers.
So they can know "I’ve already reviewed that entire module. There is this small piece that’s the only thing that’s new. I can focus my attention on this", particularly to give them more information about how much power, how much authority each of these different modules is getting, so that they may choose to not review certain pieces because they can tell at a glance that it doesn’t do very much interesting.
I’m a big fan of the object capabilities model, which is some ways it’s the logical extension of object oriented programming. You isolate functionality into separate modules, into separate objects. You limit the interaction they can have so that each of these components can protect its internal state and it can provide limited access to the outside world, they can maintain their invariants. With Jetpack when we’re building add-ons out of CommonJS modules, we have a natural structure for isolation there. We provide a bunch of modules with the SDK itself for doing low level things like modifying pages, making HTTP requests, putting UI elements on the screen. Those modules are then included or referenced by other code, but they don’t necessarily get all of the authority of those other modules.
The example I was using in my talk was there is a module called "Request" which is doing HTTP requests; it’s a nicer wrap around the usual XMLHttpRequest object. You could make a module that just gets the weather. It takes the name of the city and it internally uses the request object and constructs URL that goes to some weather service, does a specific kind of query, returns the data in certain format. If then a higher level module references the weather module, instead of referencing the underlying request module, then it has much less power, it has much less authority that the reference that it has to the weather module allows it to make one kind of request with one kind of data, but doesn’t give it the more powerful access that would let it talk to any server on the internet.
That’s confinement or that’s attenuation that that weather module is taking a large power and reducing it, attenuating it, providing a very small subset of it to a caller. That means that the caller can do as much. You are giving them the power they need to do [it]. If you have an add-on which is supposed to put a temperature on the weather indicator in your status bar that top level code doesn’t need general purpose HTTP request, it only needs this weather thing. So by layering it this way and providing isolation between these layers, you can reduce the amount of power given to this top level code. That has two benefits: one of them is that if something goes wrong in that top level code, if there is a bug, if it gets compromised somewhere or if the person that wrote it is malicious and wants to somehow take advantage of you, they have fewer tools to work with, they have less power.
The other is that somebody who is reviewing that module doesn’t need to pay as much attention to it. They can spend their time looking at something else instead because by construction, they can tell "The worst this can do is query a weather server for information". They decide that, if they do badly, that’s not that big of deal. That kind of technique can turn a lot of compromises from catastrophic failures into just harmless nuisances. If the top level add-on has the ability to query for weather and control the image in your panel, the worst case, if that’s written by somebody completely evil or compromised, the worst you can do is flood this server with lots of requests for the weather and draw rude messages down your status bar.
That would be a far cry from the current situation where compromised or badly written code could do lots of damage. You could violate lots of privacy, share lots of secrets, erase lots of data.
8. This isolation you talked about happens you choose with functions to export.
The isolation would be when you are writing one of these modules, as a programmer, you are making a decision about what sort of API to export to your caller. You provide a certain set of functions, but you don’t provide some other ones. There are some functions that you reserve for yourself or you reserve for different callers. And internally you have access to modules that you’ve imported that have more power than you are revealing to your caller. In an object oriented programming style, you have public methods and you have private methods and internal data structures. We are applying that same kind of technique here where the functions that you export in your CommonJS module are the ones that are public and everything else is private, just for your internal use.
For the full details I can show you the slides that I just presented. Basically there is nothing directly preventing that. Any module can use a "require" statement and import any other module. The way that I’m planning to handle this in Jetpack (this is still a work in progress) is that each of these modules has some "require" statements that indicate which other modules they want to require. Those require statements have two purposes: at runtime it’s the actual function call they use to get access to this other module (it’s their interface to the module loader), but at compile time we have a linking time step.
When we take all of the add-on code, things you’ve written, things you are taking from the SDK and recombine them together into an XPI, there is a tool that runs at that time that scans all these modules. It literally just greps through each line of the regular expression looking for the "require" statements and it writes those down into a manifest, then that serves two purposes: the manifest is something that reviewers can look at to tell whether module X is trying to import Y or not. And it’s available to the runtime module loader which will enforce the manifest. If the module tries to sneakily make a "require" statement, say it assembles a string dynamically, that will be prevented by the runtime module loader because it’s not on this manifest.
What that does is it forces authors of these modules to commit to the sorts of authorities that they want, the sort of modules they want to import upfront. That’s something that a reviewer can see; they get metadata that says "Module X is importing module Y and Z" and based on how much power that represents, they can decide whether that’s reasonable, whether that’s appropriate. So it’s really about making these programs, making these components easier for a reviewer to understand. That gives them a better chance of making good decisions and recommending good decisions to the user at the end of the day.
It’s possible. I think this is not a simple issue. It’s very much not a binary thing. Part of what I’m studying now, what I’m trying to implement is the idea that there is reviewer involved, there is a user who is deciding whether to install something or not. And we’re trying to give them more information to make a good decision. What you do want out of an isolation scheme is you want one piece of code to be able to define its relationship with the code that’s using it. You want to be able to protect its internal state, you want it to be able to defend itself against callers that are trying to interfere with that - "maintaining invariants" would be the technical term for it.
From what I’ve seen in CommonJS, it’s providing the right tools. The fact that there are exactly two objects that are provided to any module is a big deal, is the right thing to do. I’ve been thinking about whether the "require" statement should have a second argument to it that would be a parameterization of this module. The way that I might want to use that would be to factor out some of the attenuations that I think Jetpack add-ons could use. For example, we have a request module that does HTTP requests. You can imagine a limited request module that is configured with a single site or a single URL or regular expression and that would accept any requests that match that or reject it if it didn’t.
You wouldn’t want to have to go and make a new one of those, write the code for that new each time, so you might want a module called "request limiter" and it imports the request module internally, but you construct it with a second argument that is this list of patterns. I’ve not yet worked out how to express that in this module graph manifest that I have in mind, but that might be a possible addition that could help with this.
I think that most galleries of add-ons have the right pieces. They have user reviews, they have rating, they have information about the author. Some of them have information about what sorts of authorities the add-on is asking for. I’d like to see more visibility into that. When you install a Chrome add-on, you get a dialogue box that says "This add-on is going to have access to data on this site, it’s going to be able to store data locally." But as far as I’ve seen it’s not visible on the Chrome add-on gallery. You have to push the install button before you learn about that. I’d like to see that information promoted a little bit more. I’d also like to see information about what people think about those sorts of authorities.
I’ve heard people talk about the Android gallery and being asked about this list of permissions when you install an application on an Android phone and not really knowing what those are and not really knowing whether they are appropriate or not. It would be nice if you could learn from other people about their thoughts on it; learn if you could drill in and say "Why do you need that particular power? Why do you need to know my location? Why does this application need to have internet access?" Then, if you could disable individual features like that, that would be a lot of power in the hands of the user. So I’d like to see a little more visibility and communication between different users on the gallery sides.
Absolutely. Part of what I’ve been looking at is trying to find ways of making this a bit more end-to-end. I think there is a really valuable role for gallery in promoting and reviewing these things, but I’d like that to be an attribute of a given add-on, not a requirement, not the only distribution channel. I’d like it to be possible to take an add-on that you find lying on the street and learn enough about it to make a reasonable security decision as to whether the functionality that provides is worth the risk that it might represent. For that, I’d like to have better tools inside the browser to look at some metadata attached to this add-on, look at the modules that it provides and whether they’ve been approved by certain parties, whether they’ve been reviewed by certain parties and to present that information to the user end in some kind of meaningful way.
One of the big challenges in this space is figuring out how to provide information to the users to help them make good decisions, because there is a lot of information there and most people don’t care about that. For most people they are comfortable delegating that decision to somebody else. I think that’s fine, I think we really need to respect the way that people use their time, but I’d like that to be a more explicit feature of the platform rather than implicitly delegating that decision off to somebody else. I’d like there would be a configuration menu that says "I trust the following people to tell me whether add-ons are good for me or bad for me." So this means the users are going to add different people to that; they could add their friends, they could remove parties from that - things like that.
I would say the manifest is not directly sufficient for that. There are systems in which the APIs that are available to add-ons are much more constrained. The way that Chrome, the way that Safari from what I’ve seen work - there is a fairly limited set of functionality that the add-on can get to. It’s maybe five or six different items and it includes storage, it includes modifying pages, making calls to certain pages. Where we’re going with Jetpack is to have something that’s more flexible than that. Modules can be written by lots of people; there is a big community of add-on developers and I want to make sure that they can write interesting powerful flexible modules. What that means is that manifest that identifies the name of a module that you are importing is insufficient to describe what that module actually does.
If you say that you are importing the request module, requesting what? I don’t know just by looking at that one word that we’re talking about HTTP requests and we’re also talking about POSTs and we’re talking about PUTs and DELETEs. In addition, there is the naming question there. Is the name "request" somehow reserved? What if somebody else writes a different request module? Some of the tools, some of the designs I’m trying to put together are involved finding strong names for that. Basically it involves hashing the documentation for a given module. Using that as an identifier, as a proxy for what that module can do, using the hash of the module identifier use the hash module interface as the description of what it can do and then referencing that from everything else that tries to touch that.
What it kind of implies is building up a database of module identifiers that maps to descriptions of what they can do, descriptions of who else is using this, collections of decisions that other people have made regarding that module, finding out that module called "request" with contents X, Y, Z has been used by 5,000 add-ons. However, this other module called "request", with a different content A, B, C has only been used by two. That might make you wonder "That’s not the same thing. I need to pay attention to what that thing can really do. Maybe I should put more effort into reviewing it, I should require more people to have reviewed it before I’m willing to rely upon it, things like that.
But I think that kind of like folksonomies, I think we can build a system which people, if you are going to learn all about the other names that people are using, if there is a community where you hear about these things on a regular basis, then people can converge. People that want to make a new module to provide access to cookie functionality that is not already there can talk about it and talk about the API that they want and agree upon a good set of functionality and then write a module that everybody will use. Very similar to the way that Python standard library has acquired functionality over the years, all the big languages that have large libraries like that. There is a community as well and I think that’s going to be a big part of it.
The APIs available in Firefox are very powerful, probably because they aren’t really APIs. Traditional Firefox extensions are just digging into the guts of the browser and anything you can find there you can use. So the add-on is as powerful as everything else in the browser. That’s mixed blessing - it can be very powerful, it can be very dangerous. It’s unfortunately easy to make some mistakes that result in an add-on that is vulnerable to attacks by outside servers and it would leave the user vulnerable to that. The new Firefox system grew organically and more recent browsers have had the benefit of thinking about the problem and doing something much more deliberately and consciously. The APIs that are provided by Chrome and Safari are very deliberate and they are less powerful.
That also means that it’s harder to be vulnerable to the same kind of things that a Firefox add-on could be vulnerable to. Part of what we’re trying to do with Jetpack is to find a middle ground and to have a system in which if you stick to the modules that have already been written and have been reviewed that are used by a larger community, then you probably won’t be vulnerable to those sorts of things. But it’s still possible for you to go and write the low-level modules that will give you more powerful access or access to things that aren’t already exposed. In those ways, there is a limit to what you can do with the Safari plug-in, but it’s kind of a cleaner design overall. And part of what we’re trying to do is to find a balancing point between those.
Yes, certainly in the early days, looking at the source code was the best way to find out what was available. In Jetpack every module has a documentation file that comes along with it and when you download the SDK or when you go to add-on builder, the web-based IDE to write these things. Both of these tools have built-in documentation browser. So when look at the request module and you look at the panel module or the simple storage module, there is file there that shows you exactly what the API is and what you can use it for and has a bunch of examples down the bottom. I think that the documentation situation is much better now. We have a clear place for it, part of the checking requirements for adding a new module is that you include documentation for it. so it’s a part of the development process as well.
I can try. Mozilla Lab is the group that I’m in and it has been working on that. Chromeless is a browser without any of the decoration; it’s kind of "build your own browser" system. It’s a toolkit that lets you make an area in which you can render HTML and build your own interface pieces around it. I’m not directly involved in the project, but the motivation is that I’m seeing for it is letting people experiment with how they feel a browser ought to work, not being constrained by the UI or the UX, the how your user should interact with the browser. Decisions that have been made on Firefox have been giving people a change of experimenting it, trying out new things.
From what I’ve seen of the code, they’ve implemented some pieces as a Jetpack extension. Jetpack is not just add-ons for Firefox, it’s actually for all of the different Mozilla applications built on the same platform: Firefox, Thunderbird Email Client, Fennec the name for Firefox mobile running on Android. There is also a tool called XUL Runner which is part of the Mozilla central suite, which is for running an application that is written in XUL like the browser and all these other things, but is not actually a web browser, it has access to that. I believe that what they are doing is using Jetpack to create XUL Runner application, but has an area where you can render HTML and do things like that, but also link in your own module to provide the UI for it.
I’m not the best person to ask about that. Mike Hanson who sits right next to me has been working on that. They were calling it The Appetiser Project. I think that’s on GitHub as well. A couple of months ago, Google announced a Web App Store. There has been a trend of web pages being treated as applications. If you think of Gmail it’s not fair to call it a web page. It’s a collection of pages with lots and lots of code running around - it’s a big service.
18. Actually it’s much more complex than some of the simple email clients out there.
There are large scale software systems that are working together to provide a common set of functionality and services to users, which is what an application is, and they happen to be delivered mostly over the web. I think there has been this trend of calling things "applications" to distinguish them from smaller, less coordinated collections of pages and programs and scripts. The Open Web App Project is about defining what that framework is, making sure that there is a standard for that that lots of people can be involved in. At Mozilla we’re really keen on community, really keen on giving users control over things, giving power to the hands of the users and we like to see healthy market places with lots of people involved.
We’d rather there be an open standard for how this application ought to work so there can be lots of web stores, lots of places where you can buy or download or use these applications and lots of browsers where they can be used on. We don’t want something that only runs on Firefox, we don’t want something that only Mozilla or only Google or only Apple is providing, we want something that’s more diverse and more healthy than that. That’s why this project is focusing (from what I can tell not actually being one of the developers on it) on creating a standard that lots of people can implement that enables the kinds of patterns that these app stores are going for, but in a way that won’t lock a user or an application provider into any particular channel.
20. For example if everyone uses CommonJS modules it might be easier?
It depends on the level. Greasemonkey they have forms of it for lots of different browsers and if you’ve written a Greasemonkey plug-in, then anywhere that there is the next higher level of plug-in it will work there. I think there are certainly subsets that you can write that should work in a bunch of different areas. With a strong module system that gives you a better place for an abstraction boundary to let you put in emulation layers, so that a Jetpack add-on that only uses certain APIs that we know that we can emulate in Chrome, in Safari, given the limited APIs they provide, we could probably write a shim of some sort that would take your Jetpack add-on, rearrange it a little bit and then turn it into a form that would work there.
Or I think somebody could write an add-on format. I think that we could probably write some stuff in Jetpack that would take a Safari add-on or a Chrome add-on, provides the necessary APIs that they are expecting and then run it within the Jetpack environment. I think that’s probably technically feasible and I’m sure that as people get more comfortable with these frameworks, folks will build these things. The fact that Greasemonkey became so popular tells me that people are really interested in doing that and so somebody will do it. We’re just trying to provide enough tools to make it possible, knowing that other people come along and take care of it and it will do that for them.