When it comes to testing tools, one of the fine balancing acts that tools developers need to make is how to make their tool easy to use for non-coders, while still providing a high level of customization and modularity to make it robust enough for most applications. And Cucumber - a Ruby-developed tool - is something that has truly mastered the former aspect of software test automation with its strong support for Behaviour Driven Development (BDD). So, before I discuss the tool further, I will quickly diverge to explain what BDD is and why it makes so much sense to the non-technical tester with a strong business understanding of the application.
What is BDD?
Behaviour-driven Development (BDD) is a software development technique that has evolved from TDD (Test Driven Development), which is an approach or programming practice where the developers write new code only when the automated test case fails.
Behaviour-driven development approach involves the usage of shared language that enhances communication between various tech and non-tech teams. Tests are more user-focused and based on the system’s behavior. In BDD, “Given-When-Then” (often referred to as Gherkin syntax) is the proposed approach for writing test cases. This makes it easy for anyone with an understanding of the application to be able to contribute towards the test automation effort.
Consider the below example for a better understanding:
Given the user has entered invalid credentials
When the user clicks submit button
Then display the proper validation message
Benefits of BDD in Cucumber Framework
Focuses on defining ‘behaviour’ rather than defining ‘tests’
Enhances communication among the members of a cross-functional product team
Helps reach a wider audience through the usage of non-technical language
Enables you to understand how the system should perform from the developer’s and customer’s perspective
The improvement in the quality of code results in reduced costs of maintenance and also minimizes the project’s associated risks.
This BDD approach is essentially what separates Cucumber from other test automation tools. In fact, while it is classified as a tool, it is perhaps more a framework approach with the tool simply providing a foundation for building this sort of framework into your testing effort. In terms of actual application integration or assertion management, Cucumber works with other platforms like Selenium to help achieve these things, limiting its “tool” functionality to acting as this key interface layer between user and tool.
It has gained a lot of popularity though for the ease with which it allows people to write automated tests and how it ties the core automation effort into the needs of a business, also making it easy for the BAs and Product Owners to make sense of and often even contribute to.
However, as with all tools that try to simplify the automation process, that means that it comes with a high level of abstraction - requiring step functions that form the foundation of its BDD processes that need to be written and maintained by someone who is a stronger coder. If this is done effectively though, then there is plenty of opportunity for reuse, and allows automation scripts to be written quickly. If it is done poorly, it can leave teams with a maintenance nightmare as step functions need to be constantly maintained to deal with the changing parameters of the software development. And this latter statement is perhaps also why Cucumber works best with more established front-end systems with clear business logic rather than modern web frontends which may be too dynamic in operation for the tool.
Below is a brief history of the Cucumber testing tool:
Inception (2008-2009): Cucumber was initially created by a software developer named Aslak Hellesøy in 2008. Aslak was working on a project using Ruby on Rails, and he wanted a better way to bridge the communication gap between developers, testers, and business stakeholders. He developed the first version of Cucumber as a Ruby library.
Early Development (2009-2011): After its initial release, Cucumber gained popularity in the Ruby community. Aslak Hellesøy joined forces with other developers, including Matt Wynne and Joseph Wilk, to further develop and improve the tool. They established the Cucumber core team and began refining the Cucumber framework.
Expansion and Language Support (2011-2012): Recognizing the value of Cucumber beyond the Ruby ecosystem, efforts were made to port Cucumber to other programming languages. The core team developed Cucumber-JVM, which allowed Cucumber to be used with Java and other JVM-based languages. This expansion broadened the adoption of Cucumber across different technology stacks.
Open Source Community (2012-present): Cucumber became an open-source project, enabling developers worldwide to contribute to its development and enhancement. The Cucumber community grew rapidly, with contributors adding new features, improving documentation, and extending language support for additional programming languages.
Cucumber Pro (2013-present): Cucumber Pro, a commercial product, was introduced to provide additional collaboration and reporting features on top of the open-source Cucumber framework. It offered a web-based platform for teams to manage and collaborate on Cucumber scenarios, making it easier for distributed teams to work together.
Continuous Development and Adoption (2013-present): Cucumber has continued to evolve, with regular releases and updates to incorporate new features, bug fixes, and enhancements. It has gained widespread adoption in the software development industry, becoming a popular choice for teams practicing BDD and behavior-driven testing across various programming languages and frameworks.
Benefits of Cucumber framework
Readability: Cucumber allows the creation of test cases in a human-readable format called Gherkin. This syntax is easy to understand for both technical and non-technical stakeholders, fostering collaboration and communication.
Business-Readable Tests: Cucumber promotes the creation of executable specifications written in a business-readable language. This enables stakeholders, such as product owners or business analysts, to participate in defining and validating requirements.
Collaboration: Cucumber facilitates collaboration between developers, testers, and business stakeholders. It encourages discussions around feature behavior, requirements, and acceptance criteria, promoting a shared understanding of the software being developed.
Test Coverage: Cucumber encourages a focus on behavior-driven testing, ensuring that the implemented features cover the desired behaviorus. Aligning tests with business requirements helps identify gaps in test coverage and reduces the risk of missing critical scenarios.
Reusability: Cucumber scenarios can be modular and reusable. By defining steps that can be shared across multiple scenarios, it promotes the creation of maintainable and DRY (Don't Repeat Yourself) test suites, reducing duplication and improving maintainability.
Cons of using cucumber:
Overhead: Setting up and maintaining the infrastructure required for cucumber can be complex. It involves configuring the test environment, integrating with other tools or frameworks, and maintaining the test suite as the application evolves. This overhead may require additional time and effort.
Learning Curve: Cucumber introduces new syntax and structure (Gherkin) that may require a learning curve for team members who are unfamiliar with it. This can lead to a slower ramp-up time and potential challenges in adoption, especially for teams new to BDD.
Performance: Cucumber tests can be slower compared to unit tests or other testing frameworks. This is because Cucumber tests are typically end-to-end or integration tests that interact with the application through its user interface or API. Executing a large number of these tests can result in longer test execution times.
Maintenance: As the test suite grows, maintaining and updating Cucumber tests can become challenging. Changes to the application's behavior or UI can require updates to multiple scenarios, leading to additional effort and potential maintenance overhead.
Test Fragility: Cucumber tests can be sensitive to changes in the application's UI or underlying implementation details. Small UI changes or modifications to the application's behavior can potentially break multiple scenarios, requiring updates to the tests even when the intended functionality remains the same.
When to use Cucumber framework:
If you as a business have very clear business requirements that you would like to automate
You have a mixed test team with a strong contingent of non-coders
You are comfortable with setting time aside in sprints for framework maintenance (side-note – all teams should do this anyway, but with Cucumber it may become more prevalent)
You have a large need for automated end-to-end tests that you don’t want to run inside a CI/C pipeline.
Cucumber is a great tool that provides non-coders with an easy way to get into test automation and ties in well with a BDD approach to testing design. There are some limits to what the tool can do and it is dependent on its integration with other testing tools and more established coders in the testing team to maintain the underlying code. However, if you as a company can manage this, along with not needing these tests to run in a pipeline, then it could be a very effective way for you to get the entire testing team involved in automation.