LinkedIn’s Bluepill allows developers to run iOS UI tests using multiple simulators on a single machine, writes LinkedIn software engineer Keqiu Hu, and aims to provide tool stability and test scalability.
Bluepill is an integral part of LinkedIn 3x3 philosophy –setting a goal of three releases a day “with no more than three hours between when code is committed and when that code is available to members”– to which it brings the two highly desirable properties:
- Testing platform stability, mostly related to the “flakiness” of Xcode iOS Simulator, which may cause your tests to fail due to problems in the simulator itself rather than in your code. According to LinkedIn statistics, this is the case with 2% of the tests they run.
- Test scalability, which fundamentally entails the possibility of running your tests in parallel to cover a multiplicity of simulated environments.
Bluepill addresses both platform stability and scalability by relying on Apple’s private framework CoreSimulator, which allows to create device simulators of different types, choose runtime versions, handle notifications from the simulator, etc. Using CoreSimulator insulates Bluepill from Xcode simulators, which are a black box that keep evolving with every Xcode update, Hu says, and makes it possible to run tests in parallel using multiple simulators.
In its simplest use case, Bluepill can be run like in the following example:
./bluepill -a ./Sample.app -s ./SampleAppTestScheme.xcscheme -o ./output/
The above command will start four simulators, but many options are available to customize the testing environment, including how many simulators are used, the runtime, how many retries are allowed for a failing test, a waiting time for tests that might get stuck, etc.
Bluepill is available on GitHub under a BSD–2 license. Binaries are also published for developers not wanting to build the tools themselves.