The Agile Australia conference was held in Melbourne, Australia on 30-31 May 2012. The conference attracted over 800 attendees from all around Australia and New Zealand, and over 70 speakers from around the world. Among these speakers was Dr Rebecca Parsons – the Chief Technology Officer for Thoughtworks globally. Shane Hastie from InfoQ interviewed Rebecca about her workshop at the conference, her work on the Agile Alliance board and about Thoughtworks’ vision of continuous design linked with continuous delivery.
InfoQ: Good afternoon. This is Shane Hastie. I’m sitting here in Melbourne, Australia with Rebecca Parsons and we’re talking about the events surrounding the Agile Australia Conference, Rebecca’s involvement with the Agile Alliance and her role as the Chief Technology Officer of Thoughtworks.
Rebecca, welcome and thank you very much for taking the time to come and talk to InfoQ today.
Rebecca: Thank you, Shane.
InfoQ You and I know each other fairly well through our involvement on the Agile Alliance but probably most of our audience don’t. Wouldn’t you mind briefly introducing yourself to us?
Rebecca: I’ve been in the software development in business for longer than I care to admit. I always hate it whenever I do the Math; but I actually started programming when I was about 13 years old and I often tell people that I’ve been in enough different aspects of our industry that I can talk about whether or not the grass is green anywhere. I’ve been at Thoughtworks since December 1999, so I’m one of the few more senior technology people who actually got to celebrate December 31st 1999. I actually got to celebrate December 31st 1999 because I was not "mission critical" on anything.
In my current role at Thoughtworks, I’m the chief technology officer which involves both trying to herd the technologist within Thoughtworks as well as be more broadly involved with the technology community as a whole which has me doing lots of speaking as well as participating in things like the Agile Alliance Board.
InfoQ: Let’s start and talk a little bit about the Agile Alliance board. Where (both as an Agile Alliance board member and in your Thoughtworks role) is Agile heading? What’s happening in the Agile world today ?
Rebecca: Well I think one of the interesting things is that Agile is now really seen as mainstream; it’s no longer something that when we walk into a potential client as Thoughtworks that, "Okay, you can talk about the principles on such but don’t use the ‘a’ word;" it really has become much more accepted. Of course, anytime you have a movement like this, there is always the issues around what happens as it becomes more broadly recognized and more and more people want to be able to say, “I’m doing Agile” and then you start to get the inevitable failures that result when people might say they’re doing it but they actually are not really practicing Agile approaches or applying the Agile principles.
One of the things that we’re dealing with at the moment is how do we help people understand what it really takes to be successful using Agile, so that they understand and can more readily assess if there has been a failure in an organization and what was really the cause of that.
In terms of the Agile Alliance, I think one of the interesting things that’s happening as a result of this more mainstream status of Agile, the Alliance is in a position where it now has to make a decision.
In the early days, advancing the adoption of Agile was both about educating people about Agile processes as well as helping to advance what it meant to be Agile and looking at various aspects of the software development life cycle and seeing what changes would need to be made or tools that would need to be developed or other thinking that would go into what did it mean to be an Agile ‘x’ whether that was a development or a QA or a DBA, an architect, etc.
Now that we’ve got to this mainstream point, I feel like the Alliance has a decision to make in terms of its identity. Pushing adoption of Agile at the moment could be viewed as far more about the breadth of adoption: are we educating more and more people about what it means to be Agile?
But there’s a second aspect of this which is further developing our understanding of Agile; Agile processes, Agile tools and techniques, as well as expanding Agile into more and more areas. So there is a gap of thought around what constitutes Agile and I think the Alliance has a choice to make – does it want to take the breadth approach? Does it want to focus on the depth? Or is there a way that a single organization can in fact serve both masters?
I think that’s actually an important discussion for the alliance to be engaging with the broader Agile community about what the appropriate path this.
InfoQ: One of the things that I’ve certainly seen is the Alliance had been viewed as really just running a conference; and now there’s a lot more starting to happen with initiatives such as the Advancing Agile programs, and others. It will be interesting to see where that organization goes.
Rebecca: The example that you used of the Advancing Agile program is a good one; the Alliance does have the mechanism for its membership to get involved in various types of programs and activities and some of those could quite easily be promoting either further adoption breadth in markets that haven’t yet been addressed or some of the more detailed exploration of newer areas; and so I would encourage people who are members of the Alliance to think about where they see gaps in the coverage of the Alliance at the moment and look to see what kinds of program that they might propose to fill some of those gaps.
InfoQ: You’re in Australia talking at the Agile Australia conference and catching up with the ThoughtWorkers here locally. You gave a talk yesterday at the workshop part of the conference. Can you tell us a little bit about that?
Rebecca: Yes, I gave a talk on evolving NOSQL databases and this talk was really inspired by an internal talk that was given by one of our U.S. based ThoughtWorkers about some work they were doing for a retail client and how helpful it was to them as they were evolving their data representation that they were using a NOSQL database and how that technology had in fact simplified their process of being able to continually evolve the representation as they were going through the Agile development.
This talk was about taking the basic principles for refactoring databases that Pramod Sadalage of Thoughtworks and Scott Ambler from IBM have laid out in their book; those apply to relational databases and I was looking at if we extracted that basic approach and started to look at the various classes of NOSQL databases what might we learn about how Agile and NOSQL databases can help support each other.
InfoQ: And some conclusions coming out of that?
Rebecca: One issue, of course, is that the whole term NOSQL database is not a very good name because in part, it’s clumping a lot of things together that aren’t necessarily the same. There are four broad classes of NOSQL databases: the key-value stores, the column stores, the document database; graph database and although there is a form and similarity amongst the column store, the key value store and document databases, the graph databases are a very different beast.
How I approached it in the talk was to look at MongoDB, which is an example of the document database, style as well Neo4J which is an example of the graph database style and tried to look at how evolution of your data representation in those two different categories of NOSQL database can be applied.
Each one of them gives quite a different perspective on what kinds of things are easy or more difficult to evolve but I think at least the general conclusion so far is that technology itself certainly does lend itself to the continual evolution of the data model which is a key part of what an Agile software development life cycle is about.
InfoQ: So some interesting stuff coming up there in the future. What’s happening at Thoughtworks?
Rebecca: One of the things that we’re talking about lately is obviously Continuous Delivery on the back of the book that Jez Humble wrote with Dave Farley who is actually an ex-ThoughtWorker. Continuous Delivery is really getting a lot of traction but what we’ve been looking at is what happens after Continuous Delivery and why might you want to deliver very quickly.
One of the things that we’re trying to look at right now is: if you view the role of delivery as actually the testing of a hypothesis the business might have about how it might improve its business; then you can set this up as a cycle where I come up with a hypothesis and I very quickly cast that hypothesis in software, along with the measures that I will be using to determine whether or not the hypothesis is in fact true. Then we do the continuous deployment and the continuous delivery which keeps out very quickly so we can quickly test that hypothesis.
And then what we’re looking at now is we’re going to basically "square the circle" and bring us back around. We start to think about how you would meld notions of continuous design and how you will evolve your product, and how you will evolve your customer experience, and the customer value proposition that you’re attempting to achieve through this piece of software.
Now you have a cycle that starts from the design hypothesis as executed through code; then you have a testing phase where you’re actually looking at the actual results of the delivery to see ‘well, was my hypothesis true or false?’ and based on that actual data of what real users are using your product, you can then start that circle.
So we’re looking at this combined method of continuous design and continuous delivery and how we hook those things up together and use that cycle. A key part of that, actually, is thinking about what are the analytics that you need to support the verification or disproving the hypothesis and how can we incorporate these business analytics into this whole cycle in a natural fashion.
InfoQ: And tooling for that, because this is not going to happen easily?
Rebecca: Right, the tooling for continuous delivery is actually more advanced, I think, than the tooling around some of the other things; obviously you need a strong release management product and we’ve put a lot of effort into thinking about Thoughtworks Studios product GO in terms of how it supports the continuous delivery aspects. There's all sorts of things around the infrastructure automation - tools like Chef and Puppet which are crucial for properly maintaining environments so that it is easier to test, easier to deploy. The various test automation tools are also quite critical for the whole continuous delivery.
So I think we’ve got actually a fairly rich set of tools for the continuous delivery part of that.
The whole Agile user experience and some of the tools around that, there’s still lots of work, it's still evolving and what is the right way to join these things up? There is probably a bit more thought or a bit more intellectual work in the area of Agile analytics. There’s a book by Ken Collier where he talks about applying Agile processes both to the more traditional data warehousing and business intelligence as well as how you might think about analytics differently once you start approaching it from the Agile perspective instead of the traditional data warehousing perspective; but there’s still, I think, a lot of evolution and our thinking about what constitutes Agile analytics and how would we actually implement that.
We feel there’s quite a bit of strength in the relationship between the NOSQL databases, which give you much more flexibility in how you think about data and how you process that data, and I think there’s a lot to be leveraged there. Much as the traditional data warehouse vendors realized the straight relational model is not necessarily the right way to look at the historical data as they were processing in the data warehouse.
I think we find the same kinds of things as we’re looking at what sorts of data do we have to collect to help us square that circle and understand what is the impact of the changes that we’d introduced and what might we have to do to continue to evolve.
InfoQ: And the skill set for people involved doing this, what’s going to change there?
Rebecca: I think it’s continued emphasis on the collaboration aspects. One of the strengths, I believe, of Agile is the extent to which it’s a focus on roles rather than a person doing a particular job and we were introducing new roles. You have the Agile user experience designer who is now working in a different way in talking to the analytics people about how this change might manifest itself in some kind of measurable behavior.
We have different relationships I think as well with the businesses as they’re thinking about through the help of the user experience folks as well as the business analysts; "how might I cast my ideas in this more hypothesis format?". So I think a lot of it is collaborating between the existing roles in different ways than we have in the past; and obviously the continuous delivery aspect of this is putting a lot of emphasis on the relationship between the development team and the operations team; so the whole Dev-Ops movement plays into this as well.
One of the strengths of discussions around continuous delivery is the extent to which it’s highlighting for development teams the issues that the operations folks have always been dealing with and while you don’t have to do continuous delivery to have Dev-Ops, the whole notions around Dev-Ops in breaking down the barriers between the development organization and the operations organization is an enormous enabler for this whole idea.
InfoQ: Interesting stuff and, changing tack slightly, where’s Thoughtworks heading?
Rebecca; Well geographically: we’re heading into Africa in February; we opened an office in Johannesburg; and a couple of weeks ago, we started as a small center in Kampala, Uganda; and I actually came here to Melbourne from Accra, Ghana where we are exploring the possibility of opening a center there that would serve all of west Africa including the quite large country of Nigeria; one in five people in the African content, I believe, are Nigerians, so that’s not a market you want to ignore.
In addition, we’re putting a lot of thinking around what it will really take to get this cycle going; what does it mean to tie up continuous design and continuous delivery and how that impacts the way that we do our work.
Agile analytics is something that’s also quite a big focus for us in terms of what is the real impact and how can we start to address some of the critical business intelligence problems that an enterprise is facing using more Agile approach.
So I think those are the major things that we’re focusing on; we’re looking at continuous design in Agile user experience as well as the continuous delivery separately but we’re really excited at the potential for putting this into this loop that will allow us to have those things feed off of each other.
InfoQ: Wow, very interesting times. Rebecca, thank you very much for talking to InfoQ today, and for taking time out from the conference.
Rebecca: Thank you so much, Shane and it’s a pleasure.
About the Interviewee
Dr Rebecca Parsons is Thoughtworks' Chief Technology Officer. She has more than 20 years application development experience, in industries ranging from telecommunications to emergent internet services. Rebecca has published in both language and artificial intelligence publications, served on numerous program committees, and reviews for several journals. She has extensive experience leading in the creation of large-scale distributed object applications and the integration of disparate systems.
Before coming to Thoughtworks she worked as an assistant professor of computer science at the University of Central Florida where she taught courses in compilers, program optimisation, distributed computation, programming languages, theory of computation, machine learning and computational biology. She also worked as Director's Post Doctoral Fellow at the Los Alamos National Laboratory researching issues in parallel and distributed computation, genetic algorithms, computational biology and non-linear dynamical systems.
Rebecca received a Bachelor of Science degree in Computer Science and Economics from Bradley University, a Masters of Science in Computer Science from Rice University and her Ph.D. in Computer Science from Rice University. Rebecca blogs here.
