Introduction

The integration of several modules of an application through APIs is a complex task, not for the purpose of defining the API, but throughout the life of this API.

CDC tests are a type of test that enable verification of the integration between two or more services. The validity of the solution for the evolution of the APIs can be verified.

Instead of conducting an integration test, CDC divides the test into two parts that can be carried out at different times:

1. A test is run in the consumer that causes a response from the producer. The producer is replaced by a mock service that emulates the producer’s behavior. One or several contracts are generated from these interactions with the producer.
2. In the producer, another type of test is run where a verifier is initiated that runs all contracts registered for the producer. This verifier will replicate consumer requests defined in the contracts and verify that the producer’s response complies with the contract.

Therefore, the consumer of a service will have one or more contracts for each consumed service, and each implemented service must comply with each and every one of the contracts in force with each of the system’s consumers. This way, if these tests are run in each of producer’s versions, it ensures that each delivery does not leave a consumer with mistakes in the communication with the service.

To learn more, we recommend a post by Pierre Vincent: Why you should use Consumer-Driven Contracts for Microservice integration tests, or ThoughtWorks article written by Ian Robinson: Consumer-Driven Contracts: A service evolution pattern.

Tools for CDC Testing

The microservice testing transparencies show three tools to conduct CDC tests:

• Janus is a tool that makes it possible to specify the contracts among services in Clojure, facilitating the creation of tests in the producer. However, Janus does not allow the consumer to validate contracts using mocks of the producer.
• Pacto is a tool in ThoughtWorks implemented in Ruby. This tool has currently been abandoned and they recommend using Pact.
• Pact is a tool that enables the creation of contracts (called pacts) among the services: from the consumer, it allows for specification of the producer’s behaviors and registration of all interactions during the integration tests though a mock producer; from the producer, it facilitates the assessment of pacts, emulating the same requests registered in the pacts. In addition, it has several deployments in different languages: NodeJS/JavaScript, JVM (Java and Scala), Go, Swift/Objective C, Ruby, .NET.

We have chosen Pact for the assessment, because it provides support to both the consumer and the producer and because there is support for NodeJS and JVM, which are the languages in which we have decided to conduct the tests.

We've chosen Pact for the assessment, because it provides support to both the consumer and the producer and because there is support for NodeJS and JVM.

Testing Pact

Pact is a tool that allows for CDC testing, whose communication channel is HTTP and messages are in JSON format.

Pact consists of three parts:

1. Pact for the consumer of the service.
2. Pact for the producer of the service.
3. A server to manage the pacts.

Pact for the consumer of the service

In the consumer section, it facilitates the creation of pacts (contracts, according to the CDC terminology) using a mock server where the correspondence between the consumer and producer are specified.

This mock server registers a collection of interactions for a mock, so that when the tests are run and there is an interaction that fits the request, the mock server will return a definite response in the executed interaction. The server records all requests and responses generated from the mock in order to later create the pacts.

This mock server has the following functions:

• Create a new mock in the port.
• Add interactions to a mock.
• Verify that a stored interaction has been executed.
• Erase stored interactions, but without erasing the identified pacts.
• Stop a mock and generate the pacts identified up to that point.

An interaction consists of:

• A description of the status of the producer prior to the test. This is the Given in the GivenWhenThen in the integration tests. This will be useful when validating pacts in the producer.
• Establishing which one is the consumer and which one is the producer.
• A description of the test.
• Describe the request and response in the interaction.
  • In the description of the request, the HTTP route, query, headers and body of the request can be specified. Regular expressions can be used to describe the request in a more generic manner.
  • The response describes the headers and response in JSON, and can also indicate extra validations in the JSON message like indicating that it fulfill a regular expression, which could be a data type or other restrictions.

As we did not find how to use the mock server in tests in the documentation and since we have also found some examples of improper use, below we explain the normal cycle during tests to generate pacts:

1. Create the mock server.
2. Create as many mock services as are involved in the suite of tests, one service per port.
3. Add interactions involved in the test that will be run.
4. Execute the consumer part that leads to executing the HTTP requests to producers.
5. Verify that a programed interaction has been executed and verify that the test is valid.
6. Clean the interactions to be able to execute the next test.
7. Go to step 3 until all possible interactions with other services have been executed.
8. Finalize the mocks, which provokes the creation of pacts.
9. Finish the mock server.

Pact for the producer

Pact facilitates the implementation of pact verification tests. These tests start the service, execute all requests registered in the pacts and finally, verify that the responses coincide what what is specified in the pacts.

There are languages that do not have complete support to run tests using the test frameworks of the language. What Pact provides is an application implemented in Ruby that reads the contracts from a URI and then communicates with the producer’s tests through HTTP requests. The latter occurs with NodeJS and we will show how to use this tool to run the pact validation tests later.

Pact Broker

When storing and managing the created pacts, Pact offers a pact service, Pact Broker.

Pact gives support in several languages to be able to download the latest pacts from the producer. But if a language or test framework is not supported, the Pact Broker has a REST API to be able to navigate through all the consumers and producers detected in all the pacts, including their versions, in order to be able to download the pacts.

Experimenting with Pact

We've developed and published a project designed to assess Pact in our public GitHub repo.
This sample project includes two consumers (consumer1 and consumer2) and two services (users and sales):

• The users service provides a list of customer information from a purchase system. Information like email and address are given on each customer. This service is implemented in Scala.
• The sales service provides a list of sales made per customer in the system. Each item shows the purchases made by a user, which includes the price of each article and type of article purchased. This service is implemented in Scala.
• Consumer1 is a process that executes a report on the money spent by each customer and where the customer lives, so a report on the amount spent by urban area can be created. This service is implemented in Scala.
• Consumer2 is a process that executes a report on the type of products that interest each customer and their email in order to send them emails with promotions for similar products.This services is implemented in NodeJS.

Scala was chosen to test the support provided in this language. As an alternative it is always possible to use the support provided in Java.

Limitations found

In consumers:

• Pact supports Node, Scala and Java, but with some limitations. We found that if a consumer entails calling two or more producers, Pact can only be used for JUnit if implementing on JVM. In other words, Scala support only works if the test depends on a producer.

In producers:

• The Scala verifier does not connect to Pact Broker. Therefore, an extra test needs to be run to use the Pact Broker in order to look for pacts and then download them.
• The Node verifier downloads a project in Ruby (Pact Provider Verifier). It downloads the pacts from the Broker or through a URL and then communicates with the verification test through HTTP to request that it execute the Givens from each pact. This makes the test fairly cumbersome as an HTTP server must be raised. It has also been verified that the engine that runs the validations does not implement all matchers, which means that the validations cannot always be correctly executed.

Results from the Pact experiment

We can conclude that Pact supports contract testing and thanks to Pact Broker, facilitates a mechanisms through which each producer can select and filter which pacts to validate.

On the other hand, the current version only completely supports Java, using JUnit as a test framework for both the consumer and producer. In Node, it only supports the consumer part.

Conclusions

With the experiment using Pact, it was possible to check that two services are integrated without having to establish a specific integration test. It also enables the possibility of not only checking the compatibility of the latest versions of each service, but of a range of them.

However, there is a limited variety of tools to do contract testing and those that do exist do not completely support many languages.

These results leave the following unresolved issues:

1. Due to the lack of support in different languages, we suggest using a commonly accepted language to specify tests (contracts), so that consumers can specify what they expect from producers and then these producers can take these tests and implement them in their deployment line. For now, the candidate to specify these tests would be Gherkin, as it supports the most commonly used languages.
2. Once how to generate and verify contracts can been established, it’s important to see the different strategies to integrate them in the continual deployment lines, avoiding the possible interdependence of deployment lines as much as possible (avoid a greater fan-in).

We encourage you to send us your feedback to BBVA-Labs@bbva.com and, if you are a BBVAer, please join us sending your proposals to be published.