Mocking Dependencies on Component Testing
Mark Cohn test pyramid
When Mark Cohn in his book “Succeeding with Agile” came up with the concept of the “test automation pyramid” in 2009, developers around the world began to understand the importance of separating tests by level of complexity and by cost. In addition, the idea that Cohn actually brought to the IT community was to test system components in isolation as much as possible, as this is the cheapest and fastest way to go.
As software architectures have evolved a lot since 2009, we can consider the first version of the test pyramid as obsolete nowadays. There are actually many versions of the same diagram in the literature, and most of them have more than three layers and shades of those suggested by Cohn.
The core concept is still there though: test components in isolation as much as possible (Unit Testing). Then proceed with integrating external modules one at a time (Integration Tests) until the entire system is put under test (end-to-end testing). Thus, the general principles of the Cohn version of the Testing Pyramid can be applied to any software component (class, library, package, etc.) and even system architectures.
For example, when we test a Microservices architecture, we start by checking the behavior of each internal component, layer, or class (Unit testing). then we move on to integration with external dependencies (Integration testing), and finally we test the system as a whole (E2E testing).
Component testing
Let’s focus for a second on the Microservices Architecture scenario since it is the main topic of this post. When we need to perform integration testing we can follow two main strategies:
–We can test the integration of external components one at a time (Integration testing)
–But we can also test the whole service in isolation, providing all the dependencies it needs (Component testing).
We can consider Component Testing as black-box testing that has the objective to check the public interface of our service with these two characteristics:
- the service must be executed in an isolated environment
- the service configuration must be as close as possible to the production environment configuration.
Component testing is often referred to as “Service Testing” or “Service Level Testing”. Since it is beyond the scope of this post to delve into this topic, I suggest you read this short presentation by Martin Fowler on this subject: https://martinfowler.com/articles/microservice-testing/#testing-component-introduction
Mocks and stubs
Before we can proceed with planning the testing of our component, we need to be aware of one issue: external dependencies are not always available and we need to find a way to provide them to our service. This is where stubs and mocks come in.
What exactly are mocks and stubs? Mock and stub are so-called Test Doubles. Basically, they are instances that can replace other instances during a test (think of the stuntman/double in a movie). A mock is an entity that replicates the behavior of a specific software component (class, object, API, system). A stub, on the other hand, is a static object that is used to match the expectations of a test but has no behavior.
You can find more on test doubles on this page from the Martin Fowler website: https://www.martinfowler.com/bliki/TestDouble.html
A practical example: the Weather Service
Suppose we are building a Spring Boot microservice called Weather Service and we want to perform some Component Testing on it.
The Weather Service just provides weather information and forecasts through a REST interface. To do so, it retrieves meteorological data from an external provider (https://openweathermap.org) by simply calling its REST APIs.
Here’s the simple implementation of the provider client in Java. As you can see, we’re using OpenFeign from the SpringCloud framework in order to make things a little bit easier.
@FeignClient(name="WeatherServiceClient", url = "http://api.openweathermap.org/data/2.5/")
public interface WeatherServiceClient {
/**
* call to api.openweathermap.org/data/2.5/weather?lat={lat}&lon={lon}&appid={API key}
*
* @param lat
* @param lon
* @return
*/
@GetMapping(value = "/weather")
WeatherData currentWeather(@RequestParam("lat") String lat, @RequestParam("lon") String lon, @RequestParam("appid") String apiKey);
/**
* call to api.openweathermap.org/data/2.5/onecall?lat={lat}&lon={lon}&appid={API key}
*
* @param lat
* @param lon
* @return
*/
@GetMapping(value = "/onecall")
ForecastData forecast(@RequestParam("lat") String lat, @RequestParam("lon") String lon, @RequestParam("appid") String apiKey);
}
Now, let’s suppose we want to test the whole Weather Service in isolation by running it into a Docker container. The only thing we have to do is create a container that hosts our application and invoke all the endpoints we expose one by one.
Here’s the Dockerfile
FROM amazoncorretto:17-alpine-jdk WORKDIR /weatherApp COPY . . RUN ./mvnw -DskipTests=true package WORKDIR /weatherApp/target EXPOSE 8080 ENTRYPOINT ["java","-jar","openweathermap-mock-demo-0.0.1-SNAPSHOT.jar"]
If we build and run the container image above, we can reach our service by hitting http://localhost:8080/current and http://localhost:8080/forecast.
Now that we know how to have the Weather Service up and running inside a container, we can test it by simply invoking its endpoints with, for example, a Postman suite and making assertions about the format or payload response.
There is one problem though, we of course don’t want to use the same API key we use in production when performing tests. One simple solution could be to use a free subscription but, since free plans have a limited amount of calls, it is clear that this is not the way to go. A valid approach would be to somehow simulate the entire weather data provider by using a live http mock.
There are a number of tools and libraries that can accomplish this task, however for this example we will use MockServer (https://www.mock-server.com).
MockServer
We could have chosen another option, but we picked MockServer because:
- It is easy to use
- It offers endless customization options
- It provides a docker image on official Docker Hub
What MockServer can do for us is, given a specific configuration, simulate an API. To configure Mockserver we simply provide it with a list of Expectations. An Expectation is a rule that is used to match a request and return a specific (static) response.
Here’s an example of an Expectation in JSON format:
{
"httpRequest": {
"method": "POST",
"path": "/login",
"body": {
"username": "foo",
"password": "bar"
}
},
"httpResponse": {
"statusCode": 200,
"body": {
"sessionId": "2By8LOhBmaW5nZXJwcmludCIlMDAzMW"
}
}
}
You can read it as follow:
Whenever MockServer receives a POST request which body contains “foo” as username and “bar” as password, then return an HTTP 200 response having “2By8LOhBmaW5nZXJwcmludCIlMDAzMW” as sessionId.
Other similar HTTP mocking tools
Other similar tools are:
- WireMock (http://wiremock.org)
- Cornell (VCRpy + Flask) https://github.com/hiredscorelabs/cornell
- mockttp (https://github.com/httptoolkit/mockttp)
Putting it all together
Now that we know how an Expectation looks like, in order to set up our mock server we need to:
- Pull the latest MockServer image via “docker pull mockserver/mockserver”
- Configure the MockServer instance by providing an Expectation for each weather provider API route.
- Create a docker-compose.yaml file to make the Weather Service container and the mocked weather data provider speak to each other.
Here an example of the expectation that intercepts all the calls to http://api.openweathermap.org/data/2.5/weather?lat={lat}&lon={lon}&appid={API key} and return a response stub.
{
"httpRequest":{
"method":"GET",
"path":"/data/2.5/weather",
"queryStringParameters":{
"lon":["[0-9\\.]+"],
"lat":["[0-9\\.]+"],
"appid":["[A-Za-z0-9\\.]+"]
}
},
"httpResponse":{
"statusCode":200,
"headers":{
"Content-Type":[
"application/json; charset=utf-8"
{
"httpRequest": {
"method": "GET",
"path": "/data/2.5/weather",
"queryStringParameters": {
"lon": ["[0-9\\.]+"],
"lat": ["[0-9\\.]+"],
"appid": ["[A-Za-z0-9\\.]+"]
}
},
"httpResponse": {
"statusCode": 200,
"headers": {
"Content-Type": [
"application/json; charset=utf-8"
]
},
"body": {
"coord": {
"lon": 12.5113,
"lat": 41.8919
},
"weather": [
{
"id": 802,
"main": "Mocked-Clouds",
"description": "Mocked-scattered clouds",
"icon": "03d"
}
],
"base": "stations",
"main": {
"temp": 288.02,
"feelsLike": null,
"tempMin": null,
"tempMax": null,
"pressure": 1016,
"humidity": 67
},
"visibility": 10000,
"wind": {
"speed": 1.54,
"deg": 190,
"gust": null
},
"clouds": {
"all": 40
},
"dt": 1644065908,
"sys": {
"type": 2,
"id": 2037790,
"country": "IT",
"sunrise": 1644041914,
"sunset": 1644078560
},
"timezone": 3600,
"id": 6545158,
"name": "Trevi",
"cod": 200
}
}
}
Here’s the docker-compose.yml
version: '3'
services:
weather-service:
container_name: weather-service
image: weather-service:latest
ports:
- 8080:8080
networks:
weather-service-network:
aliases:
- weather-service
depends_on:
- mock-weather-apis
mock-weather-apis:
image: mockserver/mockserver
command: -serverPort 443,80
environment:
MOCKSERVER_INITIALIZATION_JSON_PATH: /config/initializerJson.json
volumes:
- ./mock-configuration:/config
ports:
- "80:1080"
networks:
weather-service-network:
aliases:
- api.openweathermap.org
networks:
weather-service-network:
By running docker compose up we spin up our Weather Service and also the weather provider mock.
Every call our service makes to the weather data provider gets intercepted by the mock server which returns a stub response we defined in the json above.
Final thoughts
We have achieved so far:
- We build up an environment that allows us to perform blackbox testing on our Weather Service.
- By using mocks, we were able to replace dependencies seamlessly and without the need to change the service configuration. The service was actually running in a near-production environment.
- We also saved expensive resources by avoiding making costly calls to the weather data provider.
If you are interested in running the code above by yourself you can find it here https://github.com/cloudacademy/openweathermap-mock-demo/
If you want to learn more about Docker, Testing or Microservices, consider looking at our courses at Cloudacademy.com
