So, last time I discussed the anatomy of a test automation system. So, what’s the difference between a system and a framework. Well essentially, while a system incorporates everything from the system under test and the processes you use, a framework is more related to the scripting aspect of your automation and how you go about converting your tests (or business requirements) into usable pieces of executable code.
At its very simplest definition, a test automation framework is an integrated system that sets the rules of automation for your application under test. It will essentially serve as an integration point for function libraries, test data sources, object details and various reusable modules. While some record and playback tools may combine these systems, you will want to keep them separate to make them reusable and allow only for the relevant components to be updated rather than an entire suite.
Above: Example of an automation framework using Eclipse and Selenium
1) Interface Environment
At the core of your framework is essentially what is called an interface engine. This forms the foundation of all your scripting and is code that dictates how test cases are translated into actual executable code and interact with the system/s under test. This will obviously dictate the programming language that forms the foundation of your framework, though the environment and engine (below) can also be designed in such a way as to work with different programming languages to cater for a wide variety of tools, applications and skillsets.
The environment should also include the following important aspects of your automation system:
- Function libraries
It should contain a set of shared functions which are available to all people that they can use for a variety of things related directly to their interaction with both the system under test and the tests themselves.
- Test data sources
One of the most important aspects of testing is data. We cannot get outputs from our software without the different permutations of data providing input and how we manage this data should form a large part of our framework. For small unit tests, a lot of this can be easily mocked but the bigger and more complex the scenarios, the more likely you will need to rely on large quantities of data that need to be customer representative, but also not be actual customer data for security reasons. Your framework should contain both rules and methods for dealing with this, preferably with an actual integration to test databases where data is regularly created/updated with the right amount of data for the scenarios required.
- Expectation Management
More than simply just being code that serves up data and checks the output, a test framework should be clear in how expectations or, more specifically, assertions are handled. The guidelines around where and how to assert different types of objects and mapping these back to the business requirements are important and your environment should provide the technical management of this.
- Reporting
The interface environment needs to make it clear exactly how test results are stored and displayed. This will often depend on the engine and how it works, but ideally, you want some form of consistent visualisation here to report back on the effectiveness of the test executions and provide a trend on runs and common issues where possible.
2) Interface Engine
Interface engines are built on top of Interface Environment and perform the actual execution of the tests. This is often performed by the relevant testing tools that have been chosen but can also be scripted independently All frameworks will require some form of engine though to perform the actual. Interface engine consists of a parser and a test runner. The parser is present to parse the object files coming from the object repository into the test specific scripting language. The test runner executes the test scripts using a test harness. I will uncover how these work in more detail in my next article, as these traditionally form part of the test automation tools themselves.
3) Object Repository
Any test framework will need to be interaction with objects of the system under tests, whether it be actual API endpoints or UI elements on a screen. These details should all be storied independently of actual tests so that should the underlying objects get changed or updated, the tests shouldn’t need to change themselves.
So, these might be the big three aspects that make up your standard automation scripting framework, but they are far from the only things to think about. Your framework also needs to consider the different type of scripting technique you are planning on following as well, ideally focusing on the one that will allow for the fastest and best maintainability scripting in your organization.
Various framework/scripting techniques are generally used:
Linear (procedural code, possibly generated by tools like those that use record and playback)
Structured (uses control structures - typically ‘if-else’, ‘switch’, ‘for’, ‘while’ conditions/ statements)
Data-driven (data is persisted outside of tests in a database, spreadsheet, or other mechanisms)
Hybrid (two or more of the patterns above are used)
There is no set rule for which framework works best and most often you would want your test automation scripting framework to be a hybrid, as you will need some aspects of your framework to be structured or linear (unit and component tests) and others to be data-driven or keyword driven (integration and end-to-end tests). This also depends on the skill level of the testers in your team, with the data and keyword driven approaches also offering ways for testers to create tests without needing extensive coding knowledge.
So, I will leave the decision-making up to you, but hopefully in understanding the different aspects of your test automation system and scripting framework will help you to make the right decisions and perhaps also prepare you for the final part of the discussion, which is the anatomy of a test automation tool
Comentarios