It should come as no surprise to learn that testing is at the heart of our engineers’ daily activities. Testing is intrinsic to our development process, both in practical terms and in our thinking. Our engineers work with complex systems that are made up of complex components. Individual components may have many external dependencies.
When testing, the scope of what is to be tested is important – it can be system wide, focused on a particular feature or down deep into the methods and classes of the code. To be able to focus our testing, we want to be able to mimic or ‘mock’ the behavior of external dependencies using a BDD testing tool.
The purpose of this post is to walk through a couple of simple code examples and provide an overview of and explain the need for Behavior Driven Development (BDD) testing.
BDD, Acceptance Tests, and Automation
At Rapid7 we apply the BDD methodology which is an extension of Test Driven Development (TDD). BDD and TDD both advocate that tests should be written first, which for BDD this means acceptance tests (ATs), followed by unit tests driven by the ATs. For now, let’s say that at the outset of any task, BDD focus is on capturing the required behavior in User Stories, and from these acceptance tests (ATs) are written.
Over time a large number of ATs are generated. Therefore not only is the methodology important but also the supporting tools to automate and manage our work. For some background on this, another colleague, Vincent Riou has described theautomated testing, continuous integration and code-quality control toolsthat we use.
BDD Testing Example: Ubiquitous Language and AT Scenarios
To borrow from Vincent’s post, “The idea with acceptance testing is to write tests (or behavioral specifications) that describe the behavior of your software in a language which is not code but is more precise than standard English.”
Doing this allows people who are not software engineers, but have knowledge of the requirements, such as Product Management or Marketing, to write the scenarios that make up our ATs. This is a powerful thing when it comes to capturing the required behavior. People in the BDD community sometimes refer to this as a ‘Ubiquitous Language’.
Again borrowing from what Vincent states “Additionally, those tests can be run using a parser which will allow you to easily match your language to functions in the programming language of your choice.”
Here is an example AT scenario – in this case following a template and language constructs used by the Cucumber / Gherkin parser.
Given the customer has logged into their current account And the balance is shown to be 100 euros When the customer transfers 75 euros to their savings account Then the new current account balance should be 25 euros
Example step definition in Python
The following is an example of mapping a step definition to a Python function. In this case, the final step Then is shown. This is where an ‘assert’ is used to verify if the AT will pass or fail, depending on the final account balance.
@Then('^the new current account balance should be "([^"]*)" euros$')
def the_new_current_account_balance_should_be(self, expected_bal):
expected_bal = int(expected_bal)
assert expected_bal >= 0, "Balance cannot be negative"
new_bal = get_balance(account_id)
assert int(new_bal) == expected_bal, "Expected to get %d euros. Instead got %d euros" % (new_bal, expected_bal)Since we are writing our tests before the actual implementation of the behavior, the AT will fail – so it’s important that the error message thrown by the ‘assert’ is meaningful. Remember also that an AT may fail at a future date if some behavior of the ‘system under test’ (SUT) is modified, intentionally or not – this is part of the value of having a body of automated ATs.
Mocking Behavior of External Dependencies
The components and sub-systems that we work with have many external dependencies that can be complex. When running an AT against a particular component, it may be necessary to mock the external dependencies of that component. This is different from using a framework as described below in unit testing. Instead this is about trying to mimic the behavior of a second black-box so we can test the behavior of the first black-box.
In our work we encounter this all the time, especially where a SUT has a dependency on the behavior of an external server. One approach for example is to build a simple mock server in Python using the Bottle module, that gives us a basic server to build on. We mock the behavior that is required to meet the needs of the SUT. Note that this is not building a duplicate of an existing component – we are trying to mimic the behavior as seen by the SUT to complete our testing.
BDD Testing Example: Unit Testing
After completing the acceptance tests come the unit tests. These are more closely coupled with the code of the final implementation, although at this stage we still do not start our implementation until the required unit tests are in place. This approach of acceptance tests and unit tests are applicable to GUIs.
Unit Testing Example: Mocking with some JSON
The following example is a combination of using the Junit framework with the Mockito library to create mock objects. In this example we want to show in a simple way a technique to mock a response that contains data in JSON format from a GET request on some external server. The test data, in JSON format, can be an actual sample captured in a live production scenario. We are also going to use a Google library to help with handling the JSON file.
In this simple example we are testing a method ‘getCountOfStudents’, found in a data access class, that is used by our imaginary application to get the number of students on a course using that course ID. The actual details for that course is held on some database externally – for the purposes of testing we don’t care about this database.
What we are interested in, however, is that the method ‘getCountOfStudents’ will have a dependency on another piece of code – it will call ‘jsonGetCourseDetails’ which is found in an object called ‘HttpClient’ – as the name implies this object is responsible for handling HTTP traffic to some external server – and it is from this server our application gets course data. For our test to work we therefore need to mimic the response from the server – which returns the data in JSON format – which means we want to mock the response of the ‘jsonGetCourseDetails’.
The following code snippets come from a Junit Test Class, that is testing the various methods found in the class that defines our data access object. Note the required imports for the Mockito and Google libraries are added.
import static org.mockito.Matchers.any;
import static org.mockito.Matchers.eq;
import static org.mockito.Mockito.*;
import com.google.common.io.Resources;
import com.google.common.base.Charsets;Prior to running the test a mock object of the HttpClient is created using the test class ‘setup()’ method, and tidied up afterwards with ‘teardown()’.
private HttpClient httpClient;
@Before
public void setup() {
apiClient = mock(HttpClient.class);
...
...
}
@After
public void teardown() {
reset(httpClient);
...
...
}For the test method itself, we use the Mockito when, so when the ‘jsonGetCourseDetails’ on the mock ‘httpClient’ object is called with the ‘course_id’, it then returns a mock response. We create the mock response using some test data, in JSON, we have in a file ‘course_details.json’.
@Test
public void testGetCountOfStudentsWithCourseID() throws IOException {
private String course_id = "CS101";
when(httpClient.jsonGetCourseDetails(eq(course_id))
.thenReturn(getMockResponse("./tests/course/course_details.json"));
Integer count = dao.getCountOfStudents(course_id);
Assert.assertEquals(10, count);
}To create the mock response there is a utility method we have written that uses the Google library ‘Resources’ class. For this example the method simply returns a mock response as the String from the ‘Resources.toString’.
private String getMockResponse(String jsonResource){
String MockResponse = null;
try {
MockResource = Resources.toString(Resources.getResource(jsonResource), Charsets.UTF_8);
}catch(IOException ex){
ex.printStackTrace();
}
return MockResponse;
}At this stage we have a unit test with a mock object and we can use data in JSON format. Of course, also at this stage the test method will fail. Why? Because the implementation of the ‘dao.getCountOfStudents(course_id)’ has not yet been done! We are writing our tests first, mocking the external dependencies (behavior) our code is reliant on.
When writing the code for the implementation, we will know we are finished when all the tests are passing.