Convinced that “whatever can be unit tested should be unit tested”, Mona El Mahdy, a Software Engineer in Test at Google, has written a blog post proposing several approaches to perform unit tests on the user interface of Android applications. Mahdy recommends Robolectric and the Android Studio Gradle plug-in for general unit testing, and Espresso or UI Automator for creating and running UI tests.
End-to-End (E2E) Tests. The first approach to UI testing mentioned by Mahdy is E2E. Such a test should be able to fire up the Android application and all its related back-end systems, enabling the UI testing in real life scenarios. Repeatedly performing such tests is difficult because “debugging and stabilizing E2E tests” is hard due to “many variables like network flakiness, authentication against real servers, size of your system, etc.” To simplify things a bit, Mahdy proposes another strategy for testing.
Testing using Hermetic Servers. Hermetic servers are servers that are isolated from the outside world, running locally on a single machine without network connections. Connections to other servers are provided at runtime through dependency injection, any static files that might be requested are bundled with the server, and, if necessary, responses to data store requests are faked with data from local files or an in-memory store. Another option is for a hermetic server to provide a sequence of responses recorded when returned by a real back-end server.
For testing purposes, hermetic servers run locally on the same machine as the System Under Test (SUT) where the Android emulator runs. While this approach speeds up test runs and eliminates flakiness sometimes associated with network connections, it requires separate integration testing to make sure the client app is in sync with the back-end. For this purpose, the Google+ team uses a pair of “golden” request/response files that are known to contain a correct sequence of transmissions between a client app and a back-end server. The golden request file is played and a file is created with all the responses coming from the server, being later compared with the golden response file.
Mahdy argues that using hermetic servers is better for testing than E2E tests, but debugging is still not easy and the hermetic server may cause some flakiness in communication. So, she proposes another improvement.
Using Dependency Injection (DI). The mobile application can be designed for using DI having some modules replaced with faked ones during testing. Instead of talking to a hermetic server through a network module, the app will communicate with a fake network module which provides answers to all requests. This makes UI tests even faster and reliable. For DI, Mahdy suggests using Dagger.
Multiple Libraries. When an Android application is larger Mahdy suggest splitting it up into smaller components each with its own module and view packaged in a separate library. Then, each component can be developed and tested using separate DI, fake modules and tests. Integration tests are necessary to make sure the entire app works. This approach speeds up testing even further.
At the end, Mahdy concludes: “Componentized UI tests have proven to be much faster than E2E and 99%+ stable. Fast and stable tests have proven to drastically improve developer productivity.”