Bug (issue) tracking systems have become a standard tool for any organization that develops software and have evolved greatly in the last years. InfoQ has conducted a virtual panel with people from JIRA, FogBugz, Basecamp and MantisBT about this evolution and the future developments in this field.
The participants in this virtual panel where:
- Jon Silvers & Ken Olofsen from Atlassian
- Daniel Wilson from Fog Creek Software
- David Heinemeier Hansson from 37signals
- Victor Boctor from MantisBT.ogr
InfoQ: Could you describe the evolution of your product the last 3 years? What have been the most important milestones?
Jon & Ken: Over the years, bug trackers have moved from mainly tracking simple tasks to much more sophisticated software that can do things like managing the project lifecycle, creating custom workflows and offering greater reporting to improve future planning work. And by improving project management features, bug trackers have extended beyond the domain of just programmers.
Over the past three years especially, JIRA has evolved considerably to become a centerpiece for high-performance development teams. JIRA began as a simple but powerful system for tracking software defects. Today, it can be used to track and manage a wide variety of issues - from bugs to pre-release development tasks to recruitment activities. JIRA supports all the Scrum functions, from planning boards to burn-down charts.
Key milestones in JIRA's evolution include its plugin architecture (which allowed it to be easily extended and customized), its connectivity to Integrated Development Environments (which allowed development teams to interface to it directly from their coding workbench), its integration with key Atlassian collaborative development tools (from a continuous integration server to a peer code review to source code browsing, everything a dev needs in a single suite of tools).
And JIRA 4.0 sees some of the most ambitious changes, including a refreshed and dynamic user interface, dashboards and gadgets based on OpenSocial (JIRA is an OpenSocial container and publisher) and social features like ActivityStreams (which assembles user activities into easy-to-follow feeds, like Facebook). In other words, JIRA and bug trackers in general, have really evolved into full-featured, multifaceted applications.
Daniel: Wow, three years in Internet time! Looking back at FogBugz circa August 2006 (version 5.0) feels a bit like examining the fossil record from a distant geological era. For starters, FogBugz 7.0 sure is nicer to look at!
But the real news is that FogBugz has since evolved into a complete project management system, of which issue tracking is the essential element. If you think about it, it's sort of crazy to use one tool to manage your project plan, then another to track the bugs that arise in development. You're really doing the same thing in both cases: documenting tasks, gathering work estimates, and tracking progress with an eye on the schedule.
So we designed FogBugz 7.0 to track your entire project, from planning to release. Our Subcases feature makes it a snap to outline tasks in detail, and Agile/Scrum teams can manage a product backlog right from the case list. You'll ship on time because our Evidence-Based Scheduling (EBS) algorithm actually considers a developer's estimation history. And forget about emailing around the various versions of your specification, because FogBugz also includes a full-featured wiki for authoring and reviewing project documentation.
And we've built all of this around our core issue/bug tracking functionality, which gets slicker all the time. Our AJAX-enhanced case list makes entering bugs as easy as jotting them down in a text editor. On the customer support side, a Bayesian Filtering algorithm zaps spam and turns email into trackable cases routed to the correct department.
Finally, we've just introduced a Plugin architecture that allows you to customize and extend nearly every piece of FogBugz functionality. The first batch of Fog Creek Plugins lets you modify your case workflow to match your team's working style, add custom case fields and more. And the platform is open to outside developers, who are already cranking out powerful Plugins that integrate deeply with FogBugz.
David: The most important thing we've done in the past three years is not to change too much. It's incredibly easy and tempting to over-evolve a popular product by adding feature upon feature until the product becomes increasingly complex and hard to use.
That's not to say we haven't been working on the product, quite the contrary. For example, we recently got upload progress indication and multiple file selection via a Flash widget. We added exporting to HTML, so you can get your entire project as a standalone local site.
But perhaps most importantly, we've kept refining the UI around all the things that people do every day. Like having quick-view milestones on the dashboard so you don't have to click through and fancy zooming of attached images.
Victor: A lot has happened in the last 3 years, but here are some highlights: plugins support, simpler/richer Source Control integration for CVS, SVN and GIT (incl remote servers), Twitter integration, tagging, improved LDAP/AD support, full customization of issue fields, xml import/export, soap api out of the box, updated docbook manual, localization in 46 languages, etc.
Also one of the developments that happened over the last 3 years was our move to use git as our source control tool. This has improved the developers productivity and provided the community with a better framework to manage their own customized branches and push relevant changes to us to integrate into the core product.
InfoQ: What do you think about the integration of bug trackers with the mainstream IDEs? Where do you think this is going to lead?
Jon & Ken: IDEs have improved significantly over the last few years, and developers are increasingly demanding to be able to use products from within the IDE. We now support working with issues, creating code reviews, reviewing build failures and navigating remote source repositories through our free IDE connectors for JetBrains IntelliJIDEA and Eclipse. In a similar way that you can use Google Reader, or Gmail from an iPhone, but not change settings, we believe developers will increasingly interact using their IDEs on a daily basis, returning to the web interface to set configurations and perform more complicated functions.
Daniel: Most of our users prefer to just keep the FogBugz case list open in a browser while they work, but some developers may prefer to manage cases right in the IDE. So we offer a Visual Studio (2005 and 2008) Plugin that lets you manage your FogBugz cases in a snap-in panel, and FogLyn (offered by a third-party) is a similarly slick Plugin for Eclipse. I'm not going to predict that these IDE Plugins will someday be as popular as our web interface, but I think they'll remain an indispensable part of the issue tracking toolbox.
David: Basecamp doesn't have any native integration with IDEs, but we have a rich API that people can use to hook into it. I don't really know how big of a deal this is going to be for most people though. It's not something we've personally been missing and we use Basecamp to track our programming all the time.
Victor: MantisBT provides integration with Eclipse IDE through MantisConnect. In our opinion, the bug tracker should reach out to each user in the tools they use. Program managers may prefer some Excel integration, developers use IDE integration, customer may prefer to use the web interface, etc. Having an API is an integral part in this strategy to enable the community to build such connectors.
InfoQ: When do you think it is more suitable to use a bug trackers as a service and when should an organization install the software on its own infrastructure? Do you think that cloud computing favors one of the above approaches?
Jon & Ken: While the majority of our customers are behind-the-firewall installations, we see increasing popularity of our on-demand products - JIRA Hosted and JIRA Studio - amongst smaller teams, in part, because there are no infrastructure requirements to get started. The primary reasons customers seem to prefer to use their own infrastructure are a) when they have an existing on-prem enterprise stack with which they want to integrate, and b) when they have exceedingly high sensitivity to IP security and are not ready to accept the risk, however small.
That said, we think the future of almost all software development and deployment is in the cloud. Knowing that an application is being run on the net allows for the use of external services (for example, using Google Visualizations for a richer charting experience), as well as providing a much lower TCO.
Daniel: If your organization doesn't have unique needs that require control of the entire system, and the bug tracking vendor really knows how to host your data securely, then it's almost always a great choice to use bug trackers as a service.
FogBugz on Demand is growing fast in popularity because it's extremely secure (128-bit encryption, world-class server infrastructure) and takes the burden of management off of the customer. The installed version of FogBugz remains popular with companies who want to manage things themselves, and we're committed to offering them the best possible support.
As to which approach is favored by cloud computing, that really depends on the specific needs of the customer. I mean, FogBugz On Demand is a cloud computing service by definition. But because we're so serious about security and bandwidth availability, we generally don't approve Plugins for On Demand that make custom requests to outside services in the cloud.
But if you maintain your own FogBugz instance, you can install Plugins that communicate with just about any service in the cloud (or inside your firewall, for that matter).
David: It's just so much easier to get going with software as a service. There are very few times where it's worth the hassle of setting up and maintaining something like this inhouse in my opinion. But maybe if you're working on top secret CIA projects, that could be one cause.
Victor: There is no one size fits all here. Some organization may prefer to host their own bug tracker, implement tighter integration and customization. Other organizations may prefer a hands off out-of-box software as a service approach. It really depends on the company strategy and size. The key point is that both options should be made available to the customer, and MantisBT offers both.
InfoQ: How good do you think bug tracker software adapts to the internal processes that an organization might have? How suitable do you think it is with respect to software development process framework like Scrum and Lean?
Jon & Ken: Generally, bug trackers have evolved to provide the ability to easily customize workflows for each team. Since every company practices their own style of development (waterfall or agile), the application has to be flexible to meet the needs of different teams.
In the case of agile, we offer an extension for JIRA, GreenHopper, that directly supports agile practices by representing issues as virtual index cards for ease of iteration planning and task execution. But we also let our customers connect JIRA to the other agile project management tools out there. You can't have a successful bug tracker if it only works in a "closed" environment, so we've opened JIRA up as much as possible.
Daniel: We originally built FogBugz to reflect our own working style at Fog Creek. We like a simple case workflow that automatically assigns a resolved case back to the opener for verification, and keep the number of case fields down to a bare minimum.
But the cool thing about Plugins is that they can make FogBugz a perfect fit for just about any organization. Our Custom Workflow Plugin lets customers create custom case categories and statuses, as well as tweak the rules for case assignment. The Custom Fields Plugin allows customers to easily add those one or two extra case fields they can't live without.
And because FogBugz is a complete project management system, it provides excellent support for a variety of software process frameworks. For example, we fully support Agile / Scrum methodologies in FogBugz 7.0, which introduces burn-down charts, a backlog management Plugin, and a simpler means of managing short-term project milestones.
David: In Basecamp we try to keep everything very simple so that we don't enforce any particular methodology. Our main way of tracking bugs in Basecamp is just to post them to prioritized todo lists and link those todo items up to a message if more explanation is needed.
I think it's easy to over-specialize your software to a particular style and then quickly become seen as rigid.
Victor: MantisBT allows users to customize the project states, workflow, native issue fields, custom fields, reports, etc. In fact the community has already developed a Scrum plugin for MantisBT. We also know of people who use MantisBT to manage issues that are not related to the software field.
InfoQ: The usability of bug tracking software has increased substantially over the last years. Do you think that there are many things that can be improved still? How do you think technologies like HTML 5 will affect this?
Jon & Ken: Atlassian has led the way in bringing bug tracking software into the mainstream. Seven years ago we pioneered a new, richer web-based interface, at a time when most commercial products offered only windows clients. In the last three years we've seen a lot of innovation around Rich Internet Applications, and we will continue to see more.
We're beginning to see web-based applications (Gmail/Google Reader) that are better than their desktop equivalents. We will continue to improve the usability and productivity in our apps. Will HTML 5 make a huge difference? Obviously, the offline support, and support for more control widgets, will make it easier to develop those features, and the general availability of tools will increase vendor productivity, but most of the parts of the spec can already be implemented now in various forms with browser plugins, Flash for SVG. HTML 5 will make it more standard.
Daniel: Of course we can still improve usability! We've led the charge in transforming the issue tracker from "Web 1.0 Database Front-End" to "AJAX-Rich Web App," and we'll continue to improve by listening to customers and applying top development talent to hard usability problems. (By the way, to anyone reading this: Do you have suggestions for improvements to FogBugz? Send answers to support@fogcreek.com.)
We'll definitely have a good look at HTML 5 and see which features might further enhance the user experience. Just to cite two examples, the ability to run "Web Worker" background threads could improve performance in some instances, and the "Canvas" element appears to be an attractive alternative to Flash for displaying graphs.
David: Good usability rarely depends on fancy technology. One thing most software developers would do well to spend more time on is the copywriting in their applications. Writing is designing and most applications have terrible writing.
HTML5 does have some exciting prospects, though. Like drag'n'drop from one browser window to another. I'm sure we'll see an uptake of that.
Victor: A very important aspect of bug trackers is to reduce the friction involved in reporting a bug. Hence, new technologies will be always be adopted to achieve that. For example, providing offline support, richer user experience, etc. However, it is important to still support users who are using older browsers and continue to provide them with the decent always-works / access from anywhere experience that they get today.
InfoQ: How do you see bug trackers evolving in the short term future? What new features would you like to see?
Jon & Ken: Two areas Atlassian will continue to innovate include:
- IDE and interface integration: as discussed earlier, blending a rich experience into the client interfaces preferred by different types of users - be it an IDE, a slick web application, or components (like Gadgets) that can be added to other web applications (like Gmail) - provide greater freedom of choice and offer a more frictionless experience, and less context-switching to get work done. OpenSocial is a key ingredient of this focus.
- Improved social capabilities and user connectedness: we believe we're leading the way here, but social capabilities should be a birthright of any tool that involves collaborative work between participants. Expanding existing social capabilities like ActivityStreams, networks and advanced user profiling is a key area of focus.
Daniel: As software developers, our favorite new features are those that make issue/bug tracking simpler and more painless, so that we can concentrate on making great software. You can expect future versions of FogBugz to maintain a focus on making the end user happy, because issue tracking only really works when everyone wants to participate.
David: We continue to keep it simple. We've been evolving all our products by using our products for 5+ years. I don't think bug tracking is inherently that special from all other types of task tracking. As we like to say, bugs are not special. They should for the most part just be prioritized next to feature enhancements and other improvements.
Victor: These are some of my thoughts on how bug trackers my evolve in the short term:
Use by smaller teams / even by 1-person teams - It is now cheaper than ever to buy, setup and host your own bug tracker. You can do this for free today on SourceForge and other hosters even provide 1 click installs.
Mobile access - As more people are using their mobile phone to get their work done and to access the internet, providing a rich mobile experience will become essential.
Offline access - As HTML 5 capable browsers gain market share, providing offline access will gain popularity in a lot of productivity web applications including bug trackers.
Interop - More work is going to be done for bug trackers interop with each others to manage relations to upstream / downstream bugs. The ideal goal here would be to achieve a standard way for such interop to allow different bug trackers to talk to each other.