In recent years, there have been more software teams increasingly implementing the Agile software methodology in their development process to adapt to this fast-changing market. This trend challenges testing teams to manage test cases and test scripts which have to be maintained according to changing requirements. Finding an appropriate testing method right from the beginning is one of the key elements for the successful Agile software project.
WHAT IS CUCUMBER?
In fact, many Agile teams have successfully implemented the Behavior-Driven Development (BDD) method to their testing process using Cucumber tool. So, what is Cucumber? And why is it the suitable strategy suggested for Agile projects, along with BDD?
Cucumber is a tool used to run automated acceptance tests created in a BDD format. One of its most outstanding features of the tool is the ability to carry out plain-text functional descriptions (written in the language called Gherkin) as automated tests. Let’s take a look at the below example:
Feature: Update password Scenario: Admin user can update the user password Given I am in the HR system with an Admin account When I update password of another user Then I receive a message for updating password successfully And user password is updated to the new password
This incredible feature of Behavior-Driven Development (BDD) approach with the advantages as below:
- Writing BDD tests in an omnipresent language, a language structured around the domain model and widely used by all team members comprising of developers, testers, BAs, and customers.
- Connecting technical with nontechnical members of a software team.
- Allowing direct interaction with the developer’s code, but BDD tests are written in a language which can also be made out by business stakeholders.
- Last but not least, acceptance tests can be executed automatically, while it is performed manually by business stakeholders.
CUCUMBER HELPS IMPROVE COMMUNICATION
Cucumber helps improve communication between technical and non-technical members in the same project. Let’s have a look at the below requirement and its automation tests:
As an Admin User,
I would like to change the password of other user's accounts.
Feature: Update password
Scenario: Admin user can update the user password
Given I am in the HR system with an Admin account
When I update password of another user
Then I receive a message for updating password successfully
And user's password is updated to the new password
With TestNG, the above test scenario can be implemented as below:
@test public void testAdminUserCanUpdateUserAccountPassword() { // create users User userAdmin = new User(UserRole.ADMIN, username, password); User user = new User(UserRole.VIEWER, user_username, user_password); // use Admin user to update another user password String message = userAdmin.updatePassword(user, user_new_password); // verify password changed Assert.assertEquals(message, "Password changed successfully"); Assert.assertEquals(user.getPassword(), user_new_password); }
The same test case can be written using Cucumber:
Feature: Update password
Scenario: Admin user can update the user password
Given I am in the HR system with an Admin account
When I update password of another user
Then I receive a message for updating password successfully
And user's password is updated to the new password
Both automation test scripts above perform well to complete the test automatically. But do all testers of your team make out these tests? Can other business analysts and other stakeholders use these tests again at the acceptance testing (AT) stage?
The automation test with TestNG may be difficult for most manual testers and BAs to catch up with. Moreover, it is impossible to use this test again for AT. As a result, based on these flaws mentioned before, this can not be considered as a suitable method.
In contrast, the automation test using Cucumber is created in a business domain language or in natural language, which can be easily made out by all members of the software project team. Communication is crucial for any development team, especially in the Agile team. There are usually many continuous chats, discussions, or even arguments happening among developers and testers in order to figure out what the correct behavior of a feature is. By using Cucumber, the same feature specification is now used for developing by developers, for testing by testers. It is considered to be a powerful tool because it can help lower the risk for misunderstanding as well as the communication breakdown.
CUCUMBER IS AN AUTOMATED ACCEPTANCE TESTING TOOL
The acceptance test typically is carried out by BAs/customers to make sure that the development team has built exact features. Typical activity in this testing stage is verifying the system against the original requirements with specific, real data from production. Cucumber testing not only follows the requirements as its test scenarios but also helps BAs or Product Manager to easily adjust test data. Here is a demonstration with a little adjustment:
As an Admin User,
I would like to change the password of other user's accounts.
Feature: Update password
Scenario: Admin user can update the user password
Given I am in the HR system with an Admin account
When I update password of another user
Then I receive a message for updating password successfully
And user's password is updated to the new password
The automation test written in Cucumber framework:
Scenario Outline: Verify Updating user password feature Given I am in the HR system with "<account_type>" account And there is another user with "<old_password>" password When I update password of the user to "<new_password>" Then I got the message "<message>" And the user password should be "<final_password>" Examples: |account_type |old_password |new_password |message |final_password | |Admin |$Test123 |@Test123 |Password changed.. |@Test123 | |Viewer |$Test123 |@Test123 |Invalid right access.. |$Test123 |
ALL TESTERS CAN TAKE PART IN AUTOMATION TEST WITH CUCUMBER
In addition to improving communication among members of the same testing team, Cucumber also helps leverage tester’s skills efficiently. Expertise gap always exists in every organization. In other words, there are some testers who have high technical expertise in programming utilizing automated testing, while others performing manual testing with limited programming skills in the same team. Thanks to Cucumber, all testers, no matter what their skill levels are, can participate in the process of performing automation tests.
Let’s take a look at the above example:
- Any tester who is aware of the business logic and workflow can write feature files, add more scenarios, and test datasets.
- Any tester who has a basic knowledge of programming and know how to create objects, access properties, call methods, can generate step definitions.
- Any tester with higher programming skill level can take part in the process of making a framework, define data source connection and so on.
There are still a few potential issues when implementing Cucumber:
- Cucumber helps run test scenarios specified in a plain text file using business domain knowledge. Thus, the usage of languages and the perception of the one who creates the test might directly influence the test scenarios, leading to the risk for misunderstanding. Test scenarios should be presented clearly, and their implementation should perform accurately for each step. For instance, when you want to verify the Search feature on Google, the test should be:
Scenario: performing a search on google Given I am on "www.google.com" site When I search for "Cucumber and BDD" Then ...
These steps may be incorporated to have the following test:
Scenario: performing a search on google When I search for "Cucumber and BDD" Then ...
The stages of the Cucumber tool are performed in an ordinary language. They can be used again in various test scenarios. This helps reduce the effort to create tests. However, maintaining the test to be both readable and reusable is really a big challenge. If the test is written at a very high level for any stakeholders to make out; few steps (bold) can be reused:Both the above scripts are correct; however, the second one is not apparent because it does too much more than expected: opening Google’s website and searching with the specified text. Imagine if you want to extend the test to search more texts, you may repeat the above step, and the Google site is consequently opened twice. If you do not strictly follow the requirement, the Cucumber testing tool will cause misunderstanding sooner or later and be so difficult to maintain when being extended.
Feature: Update password
Scenario: Admin user can update the user password
Given I am in the HR system with an Admin account
When I update password of another user
Then I receive a message for updating password successfully
And user's password is updated to the new password
Scenario: Viewer user cannot update the user password
Given I am in the HR system with a Viewer account
When I update password of another user
Then I receive a message for not able to update the user password
And user's password remains the same
In contrast, if the test is generic and can be reused, i.e., verifying updating user’s Last Name, non-technical stakeholders will have difficulty in catching up with and performing Acceptance Tests:
Scenario: Admin user can update user password: Given I am in the "$System.HR_Page" with "admin@test.com" username and "$Test123" password And there is another user in "$System.HR_Page" with "user@test.com" username and "$Test123" password When I update "$UserTemplate.Password" of "user@test.com" user to"@Test123" And I save the response message as "response_message" Then "$response_message" should be "Password changed successfully" And the "user@test.com" user's "$UserTemplate.Password" should be"@Test123"
During the testing process, you have to adjust test scenarios regularly until they completely reach an acceptable balance where all members can understand and reuse.
Scenario: Verify Updating user password feature Given I am in the HR system with "Admin" account And there is another user with "$Test123" password When I update password of the user to "@Test123" Then I got the message "Password changed successfully." And the user password should be "@Test123"
Or with some more test data:
Scenario Outline: Verify Updating user password feature
Given I am in the HR system with "<account_type>" account
And there is another user with "<old_password>" password
When I update password of the user to "<new_password>"
Then I got the message "<message>"
And the user password should be "<final_password>" Examples: |account_type |old_password |new_password |message |final_password | |Admin |$Test123 |@Test123 |Password changed.. |@Test123 | |Viewer |$Test123 |@Test123 |Invalid right access.. |$Test123 |
IMPORTANT NOTES FOR THE TESTING TEAM WHO WANTS TO GET STARTED WITH CUCUMBER
- Consider automation tests as essential as a real project. The code should follow coding practice, convention, etc.
- An appropriate editor tool should be considered. This editor should help debug and edit feature files in standard text format. Aptana (free editor), RubyMine (commercial editor) and Katalon Studio are suitable options which completely support BDD-based Cucumber.
- Last but not least, make feature files an actual “communication” layer where you can store received test data and format test data. Domain business logic is not contained.
To conclude, by offering the real communication layer on top of a robust testing framework, Cucumber not only can execute automation test on a wide-ranging testing demand from backend to frontend but also create deep connections among members of the testing teams. This feature is hardly found in other testing frameworks. With many years of expertise in the field of building and applying automation tests, I suggest that Cucumber for Web UI and Web services testing should be applied for Agile software projects. Using Cucumber-based BDD with Katalon Studio as an editor is a right choice which can significantly help reduce the effort to execute regression tests.