Test Butler is an open-source testing tool for Android that aims to allow developers to reliably run UI tests, writes LinkedIn engineer Drew Hannay and creator of Test Butler, by allowing developers to programmatically control a number of testing environment settings.
Hannay explains that Test Butler was inspired by the 2014 Google talk Going Green: Cleaning up the Toxic Mobile Environment, which described how to solve recurrent issues affecting mobile testing, such as lack of reliability, reproducibility, slowness, etc. Tests are unreliable when their outcome may be impaired by environmental causes, in particular when they occur at the device/OS level, says Hannay. Examples include the Android emulator CPU suddenly going to sleep, rogue device orientation changes, random triggers of the the lock screen, system dialogs unexpectedly popping up, and so on. All of those events will cause Espresso UI tests to fail.
To address these issues, Test Butler is meant to run on the Android emulator as a background service providing a few features meant to make the test environment more stable, such as:
- Disabling animations, as it is required by Espresso to work correctly.
- Disabling system dialogs by installing a custom IActivityController.
- Monitoring the emulator’s lock screen, Wi-Fi, and CPU so they do not go to sleep.
The app under test can communicate with the Test Butler service to and control a number of test environment variables such as:
- Enabling/disabling Wi-Fi.
- Changing device orientation programmatically during test execution.
- Simulating different location services modes, e.g., battery-saving, or high accuracy mode.
- Defining the locale for the application to be run in.
One key aspect of Test Butler is its being signed with the system keystore for the stock Android emulator. This means it will automatically be granted any signature-level permissions it requests without going through ADB or tampering with the manifest file.
This is how you can set your app to work with Test Butler:
public class ExampleTestRunner extends AndroidJUnitRunner {
@Override
public void onStart() {
TestButler.setup(InstrumentationRegistry.getTargetContext());
super.onStart();
}
@Override
public void finish(int resultCode, Bundle results) {
TestButler.teardown(InstrumentationRegistry.getTargetContext());
super.finish(resultCode, results);
}
}
When the connection between your app and Test Butler has been established, you can use a number of methods provided by the TestButler class to control the test environment, such as changing location services mode, setting the Wi-Fi state, triggering a device rotation, etc.
Test Butler is currently used at LinkedIn to run over one million tests each day, says Hannay. It is open source and available on GitHub.