Software development often involves using a lot of code written and maintained by others—and testing is no different. If your team is interested in implementing behavior-driven development (BDD), you’ve probably come across many different frameworks promising to simplify the process, but it can be difficult to decide on the best option.
In this article, we will take a look at BDD framework options for each major programming language, as well as some tips for deciding on the right framework for your organization.
There’s no shortage of BDD frameworks for each major programming language—how do you decide which one to use?
What is Behavior-Driven Development?
Behavior-driven development, or BDD, is a set of practices designed to reduce wasteful behavior in software development. At its core, BDD involves thinking through desired outcomes and API structure before writing any code. Automated tests are a byproduct of the process that drives implementation.
BDD can be divided into two parts:
-
Deliberate Discovery: A conversation between product owners, business analysts, domain experts, users, programmers, testers, and others that culminate in concrete examples that can be converted into executable specifications.
- Test-Driven Development: Natural language tests, often written in Gherkin, that help programmers and non-programmers spell out the desired outcomes of concrete examples without any ambiguity.
BDD often occurs at the very beginning of the software development cycle after a user story is written. The deliberate discovery meeting produces high automated tests that describe the outcomes of a user story. Developers use this as a starting point for lower level test-driven development, or TDD.
BDD Frameworks & How to Choose One
BDD test automation frameworks read executable specifications written in plain text (e.g., Gherkin or other domain-specific languages, or DSLs) and validate that the software does what those specifications promise. The framework generates a report that indicates a success or failure for each specification.Download our free list of must-have BDD development tools to support your implementation.Download Now
There is a BDD automation framework for every major programming language. These frameworks are generally derivatives of Cucumber or JBehave, but some are non-Gherkin spec runners that have BDD-like features. They all strive to achieve the same goal but use different methodologies.
Let’s take a look at some of the most popular BDD frameworks for each major programming language.
.NET & C#
SpecFlow is the most popular BDD framework for Microsoft .NET languages. While there is a free and open source version, it requires a separate unit test runner (e.g., Unit or MSTest) in order to run scenarios. The premium version, known as SpecFlow+, provides a BDD-friendly runner, Microsoft Excel integrations, living documentation and premium support.
Alternative BDD frameworks for Microsoft .NET languages include:
- xBehave.net: xBehave.net seamlessly integrates with xUnit.net and supports a variety of different testing styles, such as BDD, TDD and ATDD.
- LightBDD: LightBDD provides native support for refactoring, code analysis, test running, and other features like Visual Studio offers during development.
- NSpec: NSpec is designed to be similar to Mocha and RSpec, which makes it a natural choice for developers already familiar with those frameworks.
- Concordion: Concordion is best known for its attractive and flexible living documentation and robust tooling.
Java & JVM
Cucumber-JVM and JBehave are the two most popular BDD frameworks for Java and popular JVM languages (e.g., Scala and Closure). JBehave is one of the original BDD frameworks developed by BDD Godfather Dan North, but it lacks some common Gherkin features, whereas Cucumber-JVM is fully Gherkin-compliant solution.
Alternative BDD frameworks for Java and JVM languages include:
- JGiven: JGiven uses plain Java with a fluent, domain-specific API that generates reports that are readable by other domain experts.
- Spock: Spock has its own highly expressive language, which sets it apart from other BDD frameworks, and is compatible with most IDEs, build tools and CI servers.
- Concordion: Concordion is best known for its attractive and flexible living documentation and robust tooling.
JavaScript
Jasime and Mocha are the two most widely used TDD frameworks for JavaScript, but they lack a full set of BDD features. Cucumber.js and Yadda have become the most popular BDD frameworks that add Gherkin or Gherkin-like syntax. Cucumber.js stays the closest to plain language step definitions, whereas Yadda is designed to be less invasive and flexible.
Alternative BDD frameworks for JavaScript include:
- Vows: Vows is designed to apply BDD testing to asynchronous code with an emphasis on execution, clarity and the user experience.
PHP
Behat and Codeception are the two most popular BDD frameworks for PHP. While Behat is the Cucumber-endorsed framework focused exclusively on BDD, Codeception may be more flexible as a programmer-focused testing framework that can handle many different types of tests, including API testing, unit testing and integration testing.
Python
Behave and Lettuce are the two most popular BDD frameworks for Python, although they are hardly the only options available. Behave is Gherkin-compliant and similar to Cucumber, whereas Lettuce lacks some Gherkin features. Pytest-BDD has also become increasingly popular given its integration with Pytest, which is a popular testing framework in the wider Python community.
Alternative BDD frameworks for Python include:
- Radish: Radish is designed to be a Gherkin-compatible framework with additional syntax and a more Pythonic experience.
Ruby
Cucumber is by far the most popular BDD framework in the Ruby community. Since many Ruby developers are already familiar with Cucumber, most Ruby projects tend to stick with the framework to eliminate the need for developers to learn a new tool.
Alternative BDD frameworks for Ruby include
- Spinach: Spinach focuses on mapping features to its own classes and steps to methods within classes to introduce more structure.
Choosing the Right BDD Framework
The most popular BDD frameworks have become popular for a reason: They are actively supported by enthusiastic maintainers; many developers are already familiar with how to use them; and, they tend to set the standard for newer BDD frameworks (or they can cherry pick features from newer frameworks to build in if needed).
Don’t forget to download our free list of must-have BDD development tools to support your implementation.Download Now
That said, the best BDD framework depends on your organization’s specific requirements. Some BDD frameworks take different approaches that may be better suited to your specific application or the way that your team operates.
Some important points to consider include:
- Is the framework actively supported?
- Will you have to train new developers?
- What kind of testing are you doing?
- What are the framework’s limitations?
- Are the tests separate from the code?
- Is the framework opinionated?
The Bottom Line
Behavior-driven development can help dramatically improve the quality of your software projects while helping you deliver on time and on budget. While there are many different BDD frameworks out there, most of them follow the same core principles and skill sets tend to be transferable.
In addition to choosing the right framework, it’s important to ensure that both business and technical teams remain on the same page throughout development.
Hiptest provides a common web interface for both business and technical teams. In addition, Hiptest Publisher enables developers to quickly translate scenarios created in the web-based interface into step definitions that are compatible with most of the BDD frameworks discussed in this article.