Lisa Crispin: Great.
Janet Gregory: Really good, actually. Thank you.
Craig: It is great to have you both in the same room. You guys are probably best known to those in the community - and if they have not read it, they need to - from your awesome book “Agile Testing”. I always tell my classes: “If you need a book, you want to write this down: the best book on Agile testing is… “Agile Testing”.
Lisa Crispin: Thank you. We appreciate that.
Janet Gregory: Good question.
Lisa Crispin: I co-wrote a book called “Testing Extreme Programming” back in 2000. It was probably 2001 with Tip House. At the time, it did not sell very widely, because people still didn’t understand the need for testing.
Janet Gregory: Especially in XP.
Lisa Crispin: In XP projects and just a series of circumstances with the publisher getting bought at that time and stuff like that. So, in 2007 maybe, my editor contacted me and said “We would really like to do that book again, but call it Agile Testing and make it more broad”. And I said that it would be good to have a do-over, although the first book I still think is helpful, but my first co-author, Tip, wasn’t interested in doing another book. So, I coerced Janet into it.
Janet Gregory: She did.
Lisa Crispin: And we had been working together a lot on presentations and articles and things like that. So, we have complementary experience and it worked out real well.
Janet Gregory: I was part of that, given the opportunity, because I never think myself as a writer, even though we have two books now. It just worked out really, really well, I think.
Lisa Crispin: As Mike Cohn – I think he is the person who said this to me and I think it is true – the best reason to write a book is because you have something to say. And we did feel like if we share our experiences and those of all the contributors to our book, because we have a lot of people contributing their stories, that helps people, that helps them from maybe having to go down the wrong path, just like we did and they can start from a better place with their experimenting.
Janet Gregory: My favorite is: “Oh, we have to have that independent test team because otherwise something nasty will happen” and I go “You know what? You cannot get infected by sitting next to a programmer. It doesn’t work, right?” If you have an independent mindset, I think that’s the more important thing, thinking of the application from a customer’s point of view and I think that is what testers bring a lot to it is a different perspective, a different point of view and I think it is very helpful to have that on teams.
Lisa Crispin: We do want everybody on the team to do testing because we have so much testing to do, from before we start coding to after we release, but still testers add value, they still have, as you say, a unique skill set, unique experience, unique perspective. On most teams, not all teams maybe need a tester, but on most teams, that does contribute value.
Janet Gregory: I think the teams that have a lot of business domain, especially some that have business rules coming out of here and impacting that, it is really hard for programmers to know all of that stuff. So having a tester who can look at those big pictures, who can help ask the questions, it is a good thing. Asking the “what if” questions, if nothing else.
Craig: I was going to say that. There are some tricks and tips I think, even ways you can tell a good tester just by the way that they communicate with the team and the questions they ask.
Janet Gregory: Yes.
Janet Gregory: Actually, the models and you mentioned the quadrants, I think it did not surprise me as much as it made me really happy to see people would use it because we wanted it to be a thinking tool, for people to say “Look what we can do with it, what canwe do with our testing” What surprises me is when people take it and put it and say “Thou shall do it this way. This is the law”. That is a little frustrating almost, because that is not how we meant it. We wanted it to be used as a model so when people like Gojko come and said “Woah, let’s do it this way” that’s great because there are a lot of people who need that model. So, in our second book, we have adapted them. We have got Gojko’s change, Elisabeth’s, just to show that people can adapt. Not everybody is the same box. Even the pyramid – we have got several different ways of looking at that because there is no one context.
Lisa Crispin: It is all new material, it is all different. Of course, it is based on the first book. We didn’t want people to have to go read the first book, so we do have to have some of the same concepts in there. But in a lot of cases, we give an overview and say ”Go see the first book for more”. But we wanted to get into a little more depth with things like test planning, which we covered in the first book, but we felt like there was more we can say about it. Exploratory testing – we didn’t have a lot of room in the first book for that. Just to get deeper into some of the subjects and also branch out into the areas that people are still struggling with like working at a big enterprise organization, working in a regulated business, testing embedded software, mobile software. So, in the last few years, because with technology changing so fast, there are so many challenges people deal with and of course, we are not experts at every kind of testing, but we know a lot of people.
Janet Gregory: We were very fortunate.
Lisa Crispin: We recruited 40 people who contributed - 70 sidebars I think - to the book with how they did automation in a big enterprise company or how they approached testing embedded medical software. Things like that. So, people can read the chapters that apply to them and if something is not in their context, they do not have to worry about it. It is still a large book, but you do not have to read it all.
Janet Gregory: Yes. We also put in a chunk about learning some of the skills and things like that, because that was important too. We have a lot more people asking us about what kind of skills do we need, what do we want to do, how do we get them. So we have a section in there on thst as well.
Craig's full question: That’s something that always concerns me, particularly in my work, with testers and as a testers myself, that quite often people don’t know where to go and learn. Books like yours have certainly bridged that gap, but because it is not something you typically come through and study, it is more a field that we often fall into or are passionate about. So what is some advice that you can give to people who might be watching about where themay might be able to start building thei skills set or learning?
Janet Gregory: Good question. I think what I would ask people is to recognize where their strengths are. So, you are working on a team – what are you contributing? I think if people would stop and look and be aware of what they have to offer and then start saying to themselves “What do I want to do?” and be open, because there is podcasts out there, there is slide decks, there is courses now online, there is a little bit of everything. But I think if you focus a little bit and say “I really want to learn how to read code” for example. Not necessarily code, but learn how to read code. Then really try to find out how to do that, sometimes just going on a forum, like the Agile testing one and say “Has anybody got any ideas about where or what book might be the best book or where I could do this?” Just reach out to the community and I think that is probably the best thing. But examine yourself to find out where you are and what you want to do.
Lisa Crispin: Also, from a team perspective, if the team is having struggles get together and talk about maybe there is a skill that you are lacking and that maybe we need to make time for some groups of people to go learn that skill. There are so many great courses provided online now, some of them cost money, some of them don’t. But I think it is also really important I still hear from the person from a training course who say “Wow, the testers on my team are still really struggling with the Agile transition and they are having difficulty getting involved at the beginning and testing things like paper prototypes. It is all new to them”. I think that from a management perspective, we need to give those people support, we need to give them time to learn, make sure we build that time in. We talk a lot in the book about nurturing a learning culture, because without time to learn, innovation does not happen. So, it is really important.
Janet Gregory: And there are people out there that will do mentoring. If somebody is struggling, sometimes you just have to reach out and say “Hey!” and there will be somebody that picks up and says “Hey! I can help you with that!”
Lisa Crispin: There are lots of testing communities out there.
Janet Gregory: Exactly.
Craig: I guess one of the things that it would be remiss of me to not mention at this point is we have been talking about testers a little bit, but in both your books, they are actually called Agile Testing because everybody essentially is a tester and everybody owns quality, which is something you guys have been talking about at the conference here. So tell us a little bit about that.
Lisa Crispin: It is something we started in the first book – the whole team approach – which really comes from Extreme Programming. That is what drew me to Extreme Programming is. everybody on our team does care about quality and we are going to take responsibility to make sure we build it into our product. There is the old saying “You cannot test quality in”. So we know that is true. We have to bake it in from the beginning and that is what our books are trying to teach people.
Janet Gregory: Testers have that, one of their strengths is their curiosity. That is what I find in a good tester. They are curious and they want to know. In traditional projects, they test the software. So one of the things that I have been trying to change in my own vocabulary is not to say “software tester”, but we are “testers”, if you want to call yourself a tester. Part of that whole team is how do change our behavior from testing the software, we still have to do that, but also to testing ideas, testing early, being able to abstract a little bit. I think that is where that whole team comes in. How does the whole team work together? Testers have that innate curiosity and that critical thinking kind of thing to ask those questions. Let’s just bring it earlier so that we prevent defects and building the wrong thing. We can ask those questions a lot earlier.
Lisa Crispin: Yes. I think we have a lot to contribute to helping customers identify the right value, to build and things like Gojko Adzic’s Impact Mapping and Jeff Patton’s Story Mapping. These are all practices that we can do to help zero in. People waste so much time building something that in the end nobody really wanted to use and I think testers are in a good place to help facilitate those conversations. Obviously, there are a lot of other people. So, Analysts can facilitate those conversations, Product Managers, lots of people can do that, but it is a focus more on competencies, rather than individual roles. So we talk a lot about the T-shape skills. We all have a broad range of skills and then some special expertise that we bring to the party, where we really add the most value and it is important for the teams to understand all the advantages they have, all the benefits they have from having all these diverse people, to take advantage of that.
Janet Gregory: I think sometimes, that is where the courage comes in, take that Agile idea of just trying it and going in and saying “I really want to be part of this. Can I be in here and just start?” and to say “I am going to try it”. Sometimes it takes courage to be able to say “Hey, manager! I am going to.”
Lisa Crispin: The one good, safe place to bring it up is often in a retrospective. I think it would help the team if we could, I, as a tester, could work with the designer on the prototypes. Can we give this a try? Can we experiment with this? Don’t make it sound like the thing you are going to do forever. But slet’s just try it for this next feature. And that helps. Sometimes I think just asking forgiveness is better than asking permission. In my case, we hired a new designer and he immediately came to us, testers, and said “I really want you to help me. I want you to look at these prototypes with me” and conversely they were quite willing to come over and help us when we were testing the actual delivered software, to look and to see if all the pixels are there. So people want to collaborate, they understand the value of it. And the managers aren’t always caring about every little thing you do if you are providing value or being effective. They are happy! But it is about, sometimes the other tester on my team and I, we’re really keen to try something. Like I really want to try Example Mapping that I just learned about from Matt Wynne at this conference. It’s a small thing, it is something that the other tester and I could just ask a few developers “Are you interested in this? Let’s spend an hour and let’s try doing example mapping on this story.” So it is something that we can try. If it does not work, it does not work, but there is no harm in trying.
Janet Gregory: Yes, exactly. Fabulous tool. Let’s try.
Craig: Yes, absolutely. Speaking about UX and things, you were running a workshop here at the conference, about hands-on manual UI testing. Tell us a little bit about that workshop.
Lisa Crispin: It was a lot of fun. I did it with Emma Armstrong. She is really the expert at it, but I learned a lot doing it with her. We can do UI testing with paper prototypes, we do not have to wait till something is coded. I was in another prototyping workshop this week with Melissa Perri I think was her name and she was explaining how if you use a tool to mock up something that it looks like it is already coded and it is on the screen, people are reticent to change it because it is like killing the baby, somebody put all that work into it and you do not want to criticize it. Paper prototype where sometimes in our team we just draw on the whiteboard. It is easy to change it. So, there is an amazing amount of information you can tell just by looking at a prototype. Does it look like it will drive the right behaviors? You can take it to a user and say “What would you think would happen if you click that button?” Or “Can you see how to change quantities in this shopping cart?” Or we can just ask them open-ended questions and learn about your design. So, I am really excited to have more people trying to do this. It saves so much time up front because you are visualizing, but you are doing it with pen and paper and it is really easy.
Janet Gregory: People are way more willing to say “No, I do not like that!” if it is on a paper.
Lisa Crispin: And you can apply testing heuristics to it, the same way that you would move your hands on the keyboard.
Craig: In fact all the things: exploratory testing, maybe automated is a bit too far.
Lisa Crispin: A stretch.
Craig: Janet, you were talking about “Agile Testing the Enterprise”, which is something I think lots of people struggle with. If you work for a small company, you kind of have to get your hands dirty, but enterprises are still that almost final frontier in some respects.
Janet Gregory: Yes. I think that when you are looking at the enterprise, you have the same problems, just multiplied times 2, 4, 8, however many teams you have working. So, you have to think of the different levels. We talk about levels of detail when you are thinking about testing – whether you are at the task level or the story level or the feature level – and I think in the enterprise, you have to bring it up and really start considering and be much more aware of the dependencies between teams, thinking about what does this mean to the impact on all of these different and it raises it to a whole other level. I think a lot of teams, a lot of the books, a lot of the coaches we start talking about individual teams and then we get team silos versus the functional silos. So, there’s interaction between them – how do we and who is going to test the workflow through the whole system. So, it is the same problems, but elevated. So, I think a lot of it is just can we work with this and being aware. So, when you are planning it, it is not necessarily doing your 37 page test plan, but it is talking about it. Is there something really visible and simple that you can do to think about testing on that scale.
Lisa Crispin: The company I work for has really huge product teams now with 200 people or more on it and they are planning to having to add roles of people devoted to making dependencies visible between the teams and using some simple visualization tools. We know the project progress on each team, how can we raise that up and see the progress of all the teams together and how that relates. So it takes some imagination and creativity and maybe adding some roles we did not need before, but we experiment with that, we can see how that helps, without being something heavy weight.
Janet Gregory: Right. What I also find is things like test environments. On those enterprise systems and those large organizations, the test system is huge because chances are that they are connecting to a whole lot of third party kinds of things. You cannot have those kinds of tests environments so what gets tested in the team and where do you start to take economies of scale into place and start testing at a more depth kind of thing. There are no easy answers, no magic, but those are questions that teams need to ask.
Lisa Crispin: And taking advantage of technology. Our team is thinking about we do not have enough test environments and we are a small team, relatively. So we are thinking about how can we spin test environments up in the cloud. Virtual machines were great, but not quite good enough, so what is the next step we can take with that.
Janet Gregory: So it is a constant learning. What is the next step? What is it?
Craig: And that is one of the things as you were answering that I was pondering, is how are some of our new approaches now starting to affect testing because it used to be just be we would write the software, we test it and we go out the door and then we said let’s bring them closer together, but now we have got DevOps and we have got containerization and Cloud. That is obviously having a major effect on testing because it widens our scope.
Janet Gregory: Absolutely. It becomes more important to have that whole team thinking about testing.
Lisa Crispin: And to bring those people into the whole team. I have always worked with operations people, I have always worked with system administrators and database experts. Those people have always been there, it’s just in the last few years we started calling them DevOps and it is important. Too many companies keep them in separate teams which really causes a problem. So, if the company is not willing to change the structure, how can we liaise with those teams and maybe we need to add another role of somebody to just work with the operations team or work with the CI team or whatever. So it requires a lot of creativity for sure.
Janet Gregory: It does.
Lisa Crispin: The Example Mapping. That was something that was new to me. Today I saw a very interesting session and of course not limited to Agile testing, but we like to think of Agile as just doing a good job of delivering software frequently. It is visual testing and how your brain affects how you see things and any biases that you have. It is something that we really need to be aware of when we are testing and again, sometimes we can use technology. My team just started experimenting with a visual diffing tool. Sometimes we have small changes in our pages in a particular browser. It’s a small thing and we might not notice it, but it really impacts the user. So, we can put a screen shot for production, a screenshot for test and compare them automatically with this tool. We found things we would never have noticed by just ourselves looking at the screen. So, there is a lot of new technology. One of the things I am struck with coming to Agile 2015 is - think back to Agile 2005 – we did not have any of these test tools. There was almost nothing. We had a few, Fit, Fitnesse, but the tooling that has come along, the frameworks and libraries that have come along and the fact that they include everybody on the team, they are friendly and useful to programmers and customers and Product Owners. We just had to take advantage of these.
Janet Gregory: One other thing I wanted to mention and I know you have tried it yourself, is going to a workshop on mob testing. That was really kind of interesting taking the mob programming idea to getting the team involved in testing and getting different perspectives. I found that really quite interesting because I had not experienced it before.
Lisa Crispin: Yes, we tried it once so far and it works. I am keen to do some more, because it is a good way to involve other people and other roles on the team, the programmers, the designers, the Product Owners. It’s a really great way to get them involved with it.
Janet Gregory: You might not do it every time, but definitely everybody has interests that you would want to do at specific times.
Janet Gregory: I don’t know. The same question was asked to Elisabeth Hendrickson yesterday in her stalwart session and she had an interesting answer. It was quite a bit longer, but what I remember is that she said testers have so many skills, they can do anything they want. And I thought that was a really kind of very brilliant, succinct kind of thing because it’s true. Where do you want to be? What value do you think you can add to your team? I think that is where we need to think about it. When we start to say “testers will be here in the next five years because there has been a discussion on our user group”. And I am going you cannot predict that because things change, but I do think that people and testers and programmers and everybody have to be willing to adapt and keep up on their skills, what is the skill.
Lisa Crispin: Keep learning and keep paying attention to what new technologies are out there, keep paying attention to things like what new security exploits are out there. Those two are getting more creative. You really need to have your whole team pay attention and looking out for things like that or paying attention and looking out for new tools and I think our team, there is a chance to do a little Hackathon a few days every quarter and that is an opportunity for us to look out and see “Hmm, that looks like an interesting tool. Maybe it can help us with things like spotting browser incompatibilities in their software” and things like that. So, just being aware and keeping up and reading online blogsand magazines and video courses and things like that.
Janet Gregory: I am not on Twitter near as much as you are, but my morning starts, you get your ittle ritual, I go on Twitter and I kind of just scroll through and I look through and I go “That is an interesting looking blog, that is an interesting looking one” I just go click on them and I go back and read them so every day, I at least read three or four new articles or blogs, just to see what is going on. I think that is really important, just to keep up to date.
Craig: I think James Bachsaid something once which kind of made me think that we, in the testing community, we need more testers to be out there, talking about it, rather than just slaving away and thumping at the keyboard.
Lisa Crispin: Exactly. We need to come to conferences like this and share our experiences. I know so many people who say “I don’t have anything really to talk about”, but everybody actually has some kind of speciality or something unique about them and their experience that other people really would be fascinated to learn and it would help other people. So, we do encourage them to be brave and come out.
Janet Gregory: We have a joint web site – agiletester.ca or .com. My website is janetgregory.ca - “ca” for Canada.
Lisa Crispin: And mine is lisacrispin.com and you can find out my information there. I am very simple. @lisacrispin for Twitter, lisa@lisacrispin.com for email or you can get us at our agiletester.ca email addresses and we would love to hear from you.
Janet Gregory: Absolutely do.
Craig: Great to see you both in one location and thanks for your time today.
Lisa Crispin: Thank you!
Janet Gregory: Thank you!