Thank you very much. I'm an Agile testing consultant, trainer and coach and I am completely and totally test obsessed. In fact, I brought a present for you. This is the green band of test obsession - it says “Test obsessed" on it. I have one for you and this is how you can mark yourself as somebody who is totally test obsessed.
2. Thank you! What does it mean to be test obsessed?
It means that I love software testing, I love being able to explore software and understand what it does and I love applying the principles of testing to just about anything. I have a tendency to test everything. In fact, when I hired myself an assistant recently I devised a test to find the right assistant.
Yes, I am. The meeting was fabulous. The other person who was co-organizing it with me, Janita Andrea is the program chair for the AAFTT program. It's a program of the Agile Alliance and what that assembly of letters that's very odd means is The Agile Alliance Functional Testing Tools Group or Program. Jeff Patton was our facilitator and we had, I think, 30 something people in the room, all of whom are interested in the next generation of functional testing tools. We had a great meeting that I think is going to turn out to be a catalyst to get to the next step for the next generation of functional testing tools.
4. You seem really excited about the AAFTT workshops. Tell me why.
You're right! I am so excited because the way I see it, we are building community and one of the things that happened historically, is that each toolsmith working in their own company would have built their own proprietary combination of whatever - open source tools, plus commercial tools, plus homegrown stuff, plus whatever amalgamation of code it was necessary to automate just their stuff. We weren't really as a community, sharing information well enough.
There was some amount of it, but what the AAFTT is doing, the purpose of the program, is to help support getting to that next generation of test automation tools and doing it by building community and providing opportunities to share information, knowledge, experiences, insights, code and all of the things that are necessary to really get critical mass on a community so that good things can come out of that. I'm so excited because the community is growing tremendously.
Yes, absolutely. Most of what came out of the first workshop was sharing of information and knowledge and then we started seeing that captured on the web. We have the AAFTT mail list on Yahoo Groups and a lot of information started getting posted there to the yahoo group. Then, we also had a bunch of people who were writing blog posts and I actually got an email from one of the participants and he was so inspired by what happened at that meeting that he uprooted. His family moved some hundred miles north to a whole different city to pursue some of the things that he got excited about in the workshop. I see it as really having a transformational effect, potentially, and the community is just growing.
6. What happened this week at the Agile 2008 AAFTT Workshop?
It is a mouthful. I'm getting teased for that. What happened was another example of the community growing. This time we had over 30 participants and actually got a couple of surprise participants who wandered by and said “I heard about this and I'd like to join. Would that be possible?" and they were able to join there. It was growing even as we were in the room. What happened was we did it in an open space to scale the meeting. We had a more open format and people were able to pursue the topics that they had a lot of energy around and all the notes from those discussions are actually already on the mail list. What happened in the room is already starting to have an effect - people are following up on action items from that and more information is coming out. It's great!
7. If people want to join that community, what' the mailing list they can look at?
AA-FTT. It's a Yahoo Group. If you go to Yahoo groups and you look for AA-FTT, then you'll find it.
8. What kind of things are happening within that community?
We've got a lot of knowledge sharing about what the tools are that are out there right now and also a lot of knowledge and opinion sharing about where the tools need to be going. One of the things that's becoming interesting to me is noticing we, as a community, have a lot of alignment and then noticing where we have a little bit of controversy. I've been kind of surprised about where we have controversy.
I was even more surprised about where we had alignment. It turns out that we do have general alignment, at least amongst the people in the room, on Monday, about looking at architectures for test automation in terms of drivers and frameworks, which may sound a little bit weird, but let me explain what I mean by that. The frameworks' responsibility is to establish the format in which we express the tests to manage the execution of those tests and to report the results of those tests. The drivers' responsibility is to be able to drive the interface, not necessarily a GUI.
It could be a SOAP API, it could be a programmatic interface of some kind. Whatever the interface is, that's the driver's responsibility. In FIT, for example, a fixture would be an example of a driver, but if you are doing web development, a driver would be Watir, or Selenium RC, something that can drive that web interface. I was really surprised and happy to discover that we do have general alignment amongst the community, at least in that room, to say “All right. Here are frameworks. Frameworks talk to drivers. Drivers drive applications. So, we can think about separating out the responsibilities."
9. You've got alignment on some things. Where is the controversy?
I was really interested to discover that we have a fair amount of controversy around vocabulary - things like tests vs. examples. It's more than just vocabulary: there are some deep concepts there. There is some controversy around that; there is also some controversy around whether or not to be going through the GUI. It was very interesting for me to discover that there seemed to be some people who felt very strongly that a majority of the acceptance level or systems level test should be going all the way end-to-end through the GUI.
Then there are other people who feel very strongly in Mike Cohn's pyramid, where it says basically that “Yes, you do have some of your tests go through the GUI - they are end to end", but that's lake the fats and oils in the food pyramid. You only have a few of those and then you have a lot more that are going through other, like interfaces going under the covers, but not going all the way through a GUI interface. I was interested to discover that there is actually a controversy around that. I need to explore that a little bit more and understand more what that's about.
10. Is that the kind of thing that you explore on the list?
Absolutely. That is the kind of thing that we would have topics, discussions about on the list and it will be interesting to follow in the coming weeks and figure out what's going on there.
11. What else has come out of the meeting this week, at Agile 2008?
One of things that's come out of the meeting is that there is a now an effort to establish a sort of compendium of all the tools that are out there, a database of it. I don't have the database address on me right now, but I think it's on the Agile 2008 wiki maybe, so I'm going to have figure out where that is. I'm really excited that that effort is on the way as well.
I think it's a little bit too early to tell exactly what changes we are seeing, but one of the things that I do know for sure is that people who create tools are paying attention at what's going on in the AAFTT community. A lot of them are part of the community, but even people who more on the peripheral, they are paying attention to mail list and I think that some of the ideas that are being discussed are certainly the people who create tools, including commercial vendors, are starting to pay attention to this. That's exactly what we want to have come out of the program - this information getting out to the people who are building tools so that we get the tools that we need to support Agile development teams.
Absolutely. I think that there is hope. Before I explain the frustrations I want to say how much I love and appreciate that these tools are out there, because they did give us such a fabulous first step and everything I'm talking about is just to build on what they've already done. The next frustration for a lot of people has to do with, in particular, two things: one is the editing interface. If you are using HTML, the editing interface is whatever HTML editor you are going to use and the HTML editors, unless you go all the way up to DreamWeaver and its overkill to express tests using DreamWeaver, are kind of wacky.
The other frustration that people often have is the maintenance because the things in the tables have significance to the things in the code. If you update the column title in one of the tables, it breaks the connection with the code and now your tests are red again, but it's not because the software actually broke. Looking at what I see coming down the pike, I do see some hope on the horizon, but I also see some opportunities for tool vendors. I don't even mind paying for tools. This isn't about open source. This is about getting tools that support Agile teams.
HTML probably at least for now, is the best format for expressing tests and I would love to see a test editor that is designed to work really effectively with HTML and HTML tables without trying to give me all the things that a web designers needs with CSS and publishing to the web and all that. I just want something that works really well with HTML and tables and lets me express my tests. That's not on the horizon. In terms of maintenance, one of the things that I'm really excited about is looking at Robot Framework, which was recently open sourced. That's only been out for 6 weeks, but it was under development for the last 3 years and it's been in use for the last 3 years in proprietarly inside the company that created it. It uses HTML, so it does have the challenge of having to find a good HTML editor.
However, the way that the tests hook into the software under test uses keywords and, although it doesn't completely fix the maintenance problem, the connection between the tests and the code, it does at least keep that to one column in one place where it's pretty clear to anybody editing the test, that there is significance to the names in those columns. They know what's dangerous to touch and that was for me one of the things with FIT. I would forget which pieces hooked into the software under test, so I would keep forgetting which things I was allowed to change and which things I wasn't allowed to change.
14. What's next for the AAFTT?
We are not 100% sure yet. What's next is more discussions on the mail lists. AA-FTT at the Yahoo Lists and what' probably next is another workshop and now we get to self organize and that's the other thing we are going to do. So, we are looking for people who are passionate around this and who want to participate in helping to organize these meetings because it can't always be just Janita and me organizing these. Get involved! We are really excited!
15. People who want to help out with this, should get on the list and talk to you?
Yes, get on the list and talk to us.