A solid test framework with automated tests can increase the confidence to release. Cross-team pairing on a framework made it possible for a team to build quality in from the start; it also brought the teams together and upskilled the testers in test automation. Gwen Diagram, tester and co-organizer of the Leeds Testing Atelier, shared her experiences from creating a multi-team test automation solution in a large company at Agile Summit Greece 2018. InfoQ is covering this event with summaries, articles, and Q&As.
Diagram spoke about her experience with how several new teams in Leeds had inherited a large set of applications from several other teams based in London, which were quite abstracted from each other in an unpleasant way. To build a test framework that everyone would use meant that people had to buy into the idea instead of being forced into it, said Diagram. They decided to build bonds within the newly formed teams, using lessons from Dale Carnegie’s, "How To Win Friends and Influence People".
They needed an automation framework that gave them the confidence to release. They were still learning a lot about the product and the tests were not of much help, some being constantly failing, said Diagram. Most products were missing large chunks of automation layers and for the testers to be able to move between teams, they needed a solid framework that everyone could use.
With cross team pairing they created a product with quality built in from the start and brought the teams together. The framework has also upskilled the testers on the teams a lot, said Diagram; the teams now can write fantastic tests in Java.
InfoQ spoke with Diagram after her talk.
InfoQ: What were the problems with automated testing that needed to be solved?
Gwen Diagram: There were many different test frameworks that were being used. One product in particular had no unit tests, just a large bunch of functional tests. If the definition of a functional test isn’t set, the application of the tests can get very messy. These functional tests stretched from unit to end-to-end and were written in a language that a lot of the development teams weren’t very keen on (as was the application which we began to solve as well).
InfoQ: What was needed to implement the automation solution?
Diagram: Pairing! So much pairing. As the products were on the JVM, we voted to build the framework in Java - however, out of the testers, there was only one tester who knew any Java. I’m from a C# background and I started the framework… and wrote it in C# style. Capitals in the wrong places, brackets in the wrong places - and it all built but you should have seen my developers bleeding eyes, ha!
As testers, we can code in any language. However, code beautifully in any language? Maybe not. The developers really made the test framework shine.
InfoQ: How did you present the solution to team members, and what did you do to get senior management on board?
Diagram: The presentation of the solution is quite an interesting story - where I failed actually! The testers wrote a proof of concept in the language and test runner of their choice, so a mix of java, python, cucumber and gauge. Unfortunately, I didn’t think of the presentation part ahead of time and when we got the testers to present it, some were a lot more comfortable than others. I learnt a lot from that - it wasn’t a fair race for people to have their frameworks chosen.
Getting senior management onboard consisted of finding out what management wanted and working to that. We were lucky in that senior management had sent principals and so we could ensure that our vision matched theirs.
InfoQ: Which benefits has the test automation brought?
Diagram: The framework gives us confidence to release. The first few times we realised that the framework was really bringing benefit to us was when we had failing tests - and we thought it must have been our test framework that was the problem at first. After much debugging, we realised it was the application code that was the problem although it took us a while to trust the test framework.
We are also bringing on testers without automation skills to train them in automation which we couldn’t do without a solid framework.
InfoQ: What did you learn during the implementation?
Diagram: Don’t make technical choices by yourself. Some of the libraries I chose myself and as the framework spreads a few different applications, others chose different libraries. It made it very difficult to converge at the end. I didn’t comprehend at the time how attached people get to different json deserialiser libraries, and how difficult it would be to have everyone agree once something was already in place. In situations like this however, fighting about what technology is better is a waste of time; sometimes it’s just best to start anew with everyone’s buy in from the start.
