3. Bronze - get a single test working manually
Bronze diagram
If you have not used Pact before, consider trying it out on a spike application first.
The simplest first steps you can take
Tasks
- Write a Pact test in your consumer project (consult the documentation for your chosen language) and generate the pact.
- Ship the pact file to the provider team - it still doesn’t matter how yet, we’ll get on to that in the next step.
- Verify the pact against the provider (again, consult the documentation for your chosen language)
- Talk about it.
- Decide whether or not Pact is the right tool for you, taking into account that there are situations for which Pact is a good solution, and situations for which Pact is not a good solution.
Write a consumer test with no provider state
Pact is a consumer-driven contract testing framework. That means that the contract is established by the consumer,
based on their understanding of the interface they're consuming from the provider and their specific needs as a consumer.
This is actually important and powerful. Too often a provider guesses at what their consumer needs and comes up with an
API based on that. When you let the consumers drive the contract, it is much better aligned with their needs.
So you write the consumer test first. You can read more about consumer tests here.
If you are on the provider team, you don't own this test, although you can work with your consumer team to help them write it. But ultimately the consumer owns these tests and drives the understanding of the contract.
Note that if at all possible this first test should not introduce the complexity of provider states. You can add that as a later step.
Step 01 - Write your first pact
See folder ./Step_01_WriteYourFirstPact
An example scenario: Order API
Here we have an example describing Pact tests between a consumer (Order Web), and its provider (the Order API).
In the Consumer project, we're going to need:
- A model (the
Orderclass) to represent the data returned from the Order API - A client (the
OrderApiClient) which will be responsible for making the HTTP calls to the Order API and returning an internal representation of an Order.
Note that to create a pact, you do need to write the code that executes the HTTP requests to your service (in your client class), but you don't need to write the full stack of consumer code (eg. the UI).
Testing the Order Web (consumer) project
Scope of a Consumer Pact Test
Ideally, the Pact tests should be "unit tests" for your client class, and they should just focus on ensuring that the request creation and response handling are correct. If you use pact for your UI tests, you'll end up with an explosion of redundant interactions that will make the verification process tedious. Remember that pact is for testing the contract used for communication, and not for testing particular UI behaviour or business logic.
Usually, your application will be broken down into a number of sub-components, depending on what type of application your consumer is (e.g. a Web application or another API). This is how you might visualise the coverage of a consumer Pact test:
Here, a Collaborator is a component whose job is to communicate with another system. In our case, this is the OrderApiClientcommunicating with the external Order Api system. This is what we want our consumer test to inspect.
1. Start with your model
Imagine a simple model class that looks something like this (order.js).
loading...
The attributes for an Order live on a remote server, and will need to be retrieved by an HTTP call to the Order API.
loading...
2. Create an Order API client
Here we have our external collaborator client. Its job is to both make the external request to the Order API and convert the response into the internal Order model as per above:
loading...
3. Configure the mock Order API
The following code will create a mock service on localhost:1234 which will respond to your application's queries over HTTP as if it were the real Order API. It also creates a mock provider object which you will use to set up your expectations.
loading...
4. Write a test
loading...
5. Run the test
We will run the npm run test:consumer command
loading...
Hopefully this should be green!
Running the passing Order API spec will generate a pact file in the configured pact dir (./pacts by default). Logs will be output to the configured log dir (./log by default) that can be useful when diagnosing problems.
You now have a pact file that can be used to verify your expectations of the Order API provider project.
Now, in real life you would rinse and repeat for other likely status codes that may be returned. For example, consider how you want your client to respond to a:
- 404 (return null, or raise an error?)
- 400 (how should validation errors be handled, what will the body look like when there is one?)
- 500 (specifying that the response body should contain an error message, and ensuring that your client logs that error message will make your life much easier when things go wrong. Note that it may be hard to force your provider to generate a 500 error on demand. You may need to collaborate with your provider team to create a known provider state that will artificially return a 500 error, or you may just wish to use a standard unit test without a pact to test this.)
- 401/403 if there is authorisation.
Run the consumer Tests!
OK enough talk - let's run the consumer test. If you like, click around the project to see the files from above in context. The most interesting file is the consumer test in ./consumer/consumer.spec.js .
You should have a file that looks like this - this is our pact file (lowercase p) - Pact is our framework, that produces pact files.
It contains the details captured in your consumer test.
loading...
Manually run the provider verification test
When you run the consumer test and it runs successfully, it is going to generate a pact file (for more information you can read this overview). This file records a series of interactions where the consumer sends a request and the producer returns an expected response.
Now you need to see if the provider behaves as the consumer expects. Ultimately this will happen automatically as part of running your provider test suite both locally and in CI/CD. But you can make sure it works correctly before setting up all that automation. What you can do is get the pact file that was generated by the consumer, and copy it into a folder that is accessible to the provider. Then write your provider verification test and configure it to point to that pact file. Then run the test and see if it passes. Usually it won't at the first attempt unless you're pretty lucky. So now you can iterate with the consumer and the provider until the tests pass.
Sharing the contracts with the provider team
Now that you have created and run your consumer tests, produced a contract (the pact file) as an artefact, we need to share it with the team responsible for managing the Order API, so that they can confirm they meet all of the expectations set in it.
There are multiple ways to share pacts, but the recommended approach is to use a Pact Broker as it enables powerful automation workflows.
At this point, we just want to get our first test verified on the provider side, so we can build confidence.
We will introduce out Pact Broker at a later stage.
Testing the Order API (provider) project
Scope of a Provider Pact Test
On the Provider side, Pact needs to replay all of the interactions (usually HTTP requests) against your service. There are a number of choices that can be made here, but usually these are the choices:
- Invoke just the controller layer (in an MVC app, or the "Adapter" in our diagram) and stub out layers beneath
- Choosing a real vs mocked out database
- Choosing to hit mock HTTP servers or mocks for external services
Generally speaking, we test the entire service and mock out external services such as downstream APIs (which would need their own set of Pact tests) and databases. This gives you some of the benefits of an integration test without the high costs of maintenance.
This is how you might visualise the coverage of a provider Pact test:
1. Create the Order API
Below we have created a simple API using Express JS.
loading...
The attributes for an Order in our provider API. This is a simple in-memory database. The principles are the same if you are using any type of data-store
loading...
2. Setup your provider verification tests
We now need to perform the "provider verification" task, which involves the following:
- Telling Pact where to find the contract files, and where the Order API will be running (lines 3-13)
- Starting the API (line 16-18)
- Running the Provider verification task (line 22)
We mentioned earlier that we are sharing this file directly with the provider, we will set the provider up so it knows where this file is
loading...
This is our verification task for a provider, it is designed to run every time a consumer pact file contents change.
We will build out a seperate verification task that runs every time the provider codebase changes later, the important thing to note is that we can load a pact file into our broker from a file path, or from a remote location.
loading...
3. Run the Provider tests
We will run the npm run test:provider command
loading...
Run the Provider tests
Let's run the provider test. If you like, click around the project to see the files from above in context. The most interesting file is the consumer test in ./provider/provider.spec.js .
We will run the npm run test:provider command
loading...
Success! Our pact is correctly verified and we know that our consumer and provider are compatible.
You can play around with the codebases now
- try removing a property from the returned order and run the provider verification task
- try changing the endpoint the consumer users
both of these should fail!
Additional Notes
We also have features such as:
- provider states
- matchers
However before diving in, folllow this guide to make sure you understand how the pact generation and verification steps work, these will form part of our Honours course.