Lastminute.com has adopted contract tests to mitigate difficulties resulting from using system-level integration tests and to improve the feedback cycle and development process while eBay is using contract testing to help safely evolve their internal APIs and support client teams' requirements.
In distributed systems, such as microservice architectures, application services interact using RPC-style (remote procedure call) requests or asynchronous messages. A common approach to testing such systems is with system tests (end-to-end integration tests), that usually require the entire system to be deployed in a test environment.
Ivan Dell'oro, software engineer at lastminute.com, highlighted the challenges with using integration/system tests:
When we used to implement integration tests to verify the message exchange between two microservices, they failed for multiple reasons. This weakness leads us to ignore those tests to avoid blocking the development process. The result was that they were left ignored for months, and when something changed on the other side, both the CI pipelines were green: usually, we realized there was an error in the contract when something in production failed.
The eBay team remarked:
Another challenge for our Notification Platform team at eBay is that our APIs are consumed by many domain teams. Maintaining compatibility with all consumers while evolving our service APIs is a fundamental principle for us.
Both teams have been looking for ways to make tests less brittle and faster. The goal was to improve developer/tester experience, shorten feedback cycles and increase the pace of value delivery, while supporting the evolution of internal contacts, such as API specifications and message schemas.
In the end, after some research and experimentation, they adopted contract testing as the main way of validating the correctness of inter-service interactions. As a result, lastminute.com observed a positive impact on their microservice architecture and the delivery process, with test-execution times massively shortened, compared to standard system-level tests. eBay uses contract testing to verify integration points in their platform, supporting collaborative efforts to ensure internal APIs can evolve without incompatibility issues.
Lastminute.com had already adopted contract testing for RPC interactions between their microservices, using Pact, a consumer-driven contract-testing tool, and subsequently expanded its use to asynchronous interactions where services exchange messages over RabbitMQ broker.
Source: https://technology.lastminute.com/contract-testing-asynchronous-messaging-pact-junit-mockk/
The team at eBay explored semantic versioning of API definitions, based on OpenAPI specification but concluded that versioning alone was not enough to address problems with fragile system tests. They looked at BDD (behavior-driven development) as a way to describe API consumer's requirements, with producer and consumer teams working collaboratively to codify all requirements and make them executable. Such an approach turned out to rely too heavily on the API provider capturing and updating customer's requirements as they changed, which has proven problematic.
Lastly, the team discovered contract testing, which allows producer and consumer teams to maintain independent test suites, using mocks (or stubs) in their test cases.
They evaluated Spring Cloud Contract and Pact, settling on the latter due to more-straightforward usage patterns and better cross-team interaction support. They worked to provide seamless integration between Pactflow (a commercial Pact product) and internal CI/CD tools, and created a dedicated developer portal for configuring new contract tests.
Source: Dell'oro highlighted that contract testing alone is not a full substitute for system-level integration tests. Contract testing is meant to verify correct data exchanges between services, but service-level integration tests must exercise both business logic and error handling to ensure the overall process/data flows are correct and resilient.