Most of the testing tools I’ve been looking at so far deal with the aspect of testing at or post the development stage. However, today I want to look at a testing tool plug-in for Jira that looks to address an important gap in the market that can have a big impact on software team delivery. Test design itself.
A Gap in the Test-Driven Design Market
Test-driven design is a common concept in the software development world that requires developers and teams to consider the testing approach, expectations, and permutations before they begin their development effort. The biggest constraint for many teams trying to follow this is a lack of understanding of the software they are trying to build to have the right information to design their testing and have an understanding of the different cases and permutations that need to be covered to achieve the desired result. It sounds simple enough, but with any software system featuring quite complicated requirements, it can be easy to miss cases and understand the coverage required to satisfactorily test the system.
Queue Yest for Jira. A relatively new tool from Smartesting that is designed to address the issues in this space. One of the key things it does in trying to help teams shift left is visualise the test design process to make it easier to understand the different dependencies and permutations required. This may sound quite laborious to spend a lot of effort trying to design the test cases but the tool makes this process easier as it can identify the test scenarios based on your needs and in understanding the requirements can script the test for you – both at a manual and automated level.
Visual Requirement and Test Design
While the focus of the tool is largely around designing tests better, a huge benefit is that for teams to have the right amount of information to build the tests correctly teams need to spend more time at the analysis phase figuring out what it is they need to build. It requires teas to analyse what needs to be built, understand clearly how everything should work together, and – importantly – what is required from a testing perspective Once that heavy-listing is done it can then create the tests for you. This includes an understanding of the required datasets, parameters, and permutations, along with identifying the lowest number of tests that will help achieve the maximum coverage in your testing effort.
As with many JIRA tools, there is both a free and a paid-for option. And credit to Smartesting, Yest actually offers fairly decent features in its free component, though this is limited to purely its web interface built into JIRA and doesn’t provide full usage of its integration features and does have some limitations on the automation side, It can generate useful test scenarios though which can still add a lot of value. If you want to get the full benefit out of the tool though, you will want to make use of the separate integrated Yest app which you will pay for. It’s not cheap but considering not every member of a team necessarily needs this tool, it does provide a level of flexibility in a team everyone can use the tool, but only those who need full access to its feature set need to pay for the license fee.
Test Design and Automation
The tool claims that it can improve testing effort by 40%, though in my analysis, I have found it is perhaps not that good. At least not initially. While it does create tests quickly, there is still some effort required to verify and update tests where it may have missed areas. Not so much in understanding the different permutations, but rather understanding the underlying data required to complete specific test scenarios which is a significantly more complex thing to achieve. It’s great and efficient for straightforward testing scenarios, but not as effective when dealing with a lot of complexity. However, I have found this true for almost all tools that claim any form of self-generation or AI-built testing engines. Works well for most UI and data scenarios, but when complex data sets are required, more specificity and expertise are needed and that draws on the expertise of the team that tools will struggle to replicate.
This is perhaps not its fault directly and more just a consequence of the design process, which does become increasingly more difficult as the complexity increases. However, the benefit gained from actually spending time with proper analysis and how this leads to both better testing and traceability in the whole process – is a massive asset and even if it’s not saving significant effort in all spaces, it does lead to better test driven development and design, which is a massive win. Also, even if you consider the complexity angle, if you cannot get clear coverage of testing because either your design is too ambiguous or features datasets that are far too large – it does open up the design conversation for a simpler design. This might not always be possible, but a tool like this does drive those conversations in the right direction.
Automation Integration
Smarttesting has also spent time ensuring it can integrate with many popular automation tools like Selenium, Cypress, Playwright, Robot Framework, Tosca, and UFT. All tools I’ve previously written about here and should help teams speed up their automation scripting once they have a clear understanding of what they are building. It is also able to work directly with Git to ensure the scripts are stored in the correct repo as well. It doesn’t take away the idea of automation scripting completely though, as a proper framework still needs to be designed to meet proper robustness, speed, and functionality, but it does at least lay a good blueprint on which automation teams can build.
Another important feature that is very useful with Yest is its ability to update as changes occur. This allows people to work collaboratively on the same design and make modifications together but then also updates the underlying test and data scenarios based on the changes made. This is especially useful with future maintenance in mind where any changes to an existing workflow can have tests easily defined and converted into tests that are tangible and useful for teams.
Requirements traceability for JIRA
As a JIRA plug-in though, its biggest benefit comes in the way that it integrates with JIRA user stories and the entire agile process. One of the biggest criticisms I’ve had of JIRA and most agile user-story-based tools is that they don’t provide enough focus on requirements. User stories are about development and what is required to achieve specific development goals. However when it comes to software maintenance and linking to long-term documentation of your requirements, this is often lost in the process and teams are required to keep these in some form of document storage that cannot easily be traced back to the initial works, testing, or defects. By making use of Yest, you now have only a repository for requirements that are traced back through the process, but every aspect of its usage and flow is mapped out and linked to a test.
This means that when issues arise, it’s easy to see which requirements are directly impacted and often identify potential design errors. And then as the design and requirements are modified, you can easily identify further impact on any changes made. Something that can help teams better understand the impact their changes have on the overall design process.
Below is a list of the pros and cons of using Yest
Pros:
Better requirements analysis: Building the workflow in Yest in a visual manner leads to better design decisions and an understanding of how the software will operate.
Faster Test Design: Yest can automatically generate both manual and automated tests from the design, with the required data scenarios, and save a lot of time in the design process.
High test coverage: While there is no such thing as perfect coverage, Yest does help to maximize the coverage you can achieve in as few tests as possible and in most basic UI or API interactions, can easily identify full coverage.
Requirements Traceability: Its ability to trace requirements and design elements through the design process and throughout the software lifecycle can lead to interesting insights.
Integration Support: Yest supports most automation tools and can help generate automated test steps across a variety of different languages as well.
Automatic Updating: As design requirements change, tests are automatically updated to reflect the change.
CI/CD Integration: Yest does provide integration into the popular Ci/CD tooling that is already supported by JIRA.
Collaboration: It’s easy for different members of the team to work on aspects of the test and analysis at the same time and identify the impact these will have on each other.
Great features in its free version: The list of features available to teams for free is quite extensive and while you will still need a licensed version of the application to use it effectively, it can save on license fees.
Cons:
Not great for complex datasets: Its generation of different test scenarios and data sets for testing is fantastic, but when dealing with extremely complex data scenarios spanning different data sources, it’s not effective at identifying or creating tests effectively.
Integration to JIRA: If you are not using JIRA in your development toolset, you won’t be able to make use of the tool.
Test Efficiency: Yest identifies how to get a high coverage in as few tests as possible, but when looking at the execution of those tests at an automated level, they are not very performant.
Test Automation Script Generation is limited: While Yest can produce working automation scripts, you still would want to build a proper framework and modify its scripts to best fit your chosen framework and approach
Licenses still required for proper usage: It offers a good set of features for free, but you will want to use a license to use it properly. Its license requirements are not excessive though and reasonable in comparison to many other testing tools.
Unit Test Integration: While Yest supports test-driven design, it isn’t able to actually script unit tests in advance of developers writing the code and focuses more on automation of integration and end-to-end testing tools. So, developers will still need to write unit tests based on the scenarios provided.
Limited EXternal Support: While Smartesting does provide support for the tool’s usage, it's perhaps not as good as more established companies, and because the tool is still quite new, it's difficult to find lots of information outside of the official documentation.
When to consider choosing Yest
You are already using JIRA and are looking for a tool that can assist with requirements design and management
You want to follow proper test-driven design but aren’t sure how to identify scenarios from the design process
You are willing to invest time in designing your requirements and needs before rushing into development
Smartesting’s Yest might be a new tool on the market but it is a well-thought-out one that provides coverage for a gap, not enough tools cover. Its visual approach to workflow design may take a lot of time in the design process but can lead to a lot of benefits in designing better software while also identifying a set of very clear testing scenarios that can save teams a lot of time in identifying the required coverage. It might not meet every need, but as a new tool, it still has a lot to develop and can grow its feature set over time.