Anatomy of a Test Automation System
Automation remains a key buzzword and focus for many companies as it remains a core strategy to enable companies to deliver new features to market reliably and quickly in today’s current market. This obviously makes investment in a variety of automation tools and frameworks a vital part of any software development company’s strategy.
Do we really understand what the words test automation framework and tools mean though? In the next few articles, I plan to unpack a little around the anatomy of a successful system, framework and automation tool that might help you understand these a little better.
First, I will begin with unpacking all the different elements that make up a Test Automation Framework. In simple terms, a test automation framework can be defined as a set of processes, standards and interactions between the components in which scripts are designed and executed. However, a framework is more than this and includes many other components which I will outline below.
Also, rather than give an opinion on how I feel these should be applied, I will rather just define the different elements and allow you to then formulate a solution that best suits your organization. The purpose is to allow you to identify what is important in building a strong automation framework that suits your products and then setting standards in place to allow you as an organization to make use of them correctly.
1) System Under Test
The first component of any automation system lies not with the automation itself, but the system that you want automated. After all, before you should even begin to look at how to automate your system, you need to consider the different components of your system and understand how automation can impact them in the first place.
The reality is that the success of many a company’s automation efforts starts with the system itself. The automation strategy that needs to be applied is largely dependent on the design of your system. If you have the privilege of designing your system from scratch, then there are best standards by designing smaller modular API heavy systems which are more conducive to good automation. However, this is not always the case and your automation framework should ideally be built around the automation needs of the system under test. Trying to approach it from the framework first is going to simply lead to frustration.
2) Automation Scripting Framework
This is the time when many companies might choose to start shopping around for tools, but what is needed first is an actual design on how your automation is intended to look, including standards that should be adhered to when building the necessary scripting framework.
Many times, when automation fails in organizations, it not necessarily due to a lack of skills or poor tool section, but often because of a lack of adherence to set standards and coding best practices. There is no short-cut to quality automation and so you need to ensure that proper standards are put in place to make this a reality. Which is why when talking about the actual scripts themselves, I emphasize standards so much.
There are many things to consider when designing your scripting framework, but the below things should all be considered:
- Handle scripts and data separately
Automated test scripts should be clearly separated from the input data store (e.g. XML, JSON files, Flat files or Databases), so that no modifications are required to the test scripts whenever data has to be changed for multiple input values.
- Coding Standards
Scripting standards should always be maintained across the test automation framework, which will discourage individual coding practices and help in maintaining code uniformity, which makes it easier for software testers and developers to interpret.
- Extensibility and Maintenance
An ideal test automation framework should steadily support all new enhancements to the software application and allow modification of existing features e.g. A reusable library can be created, which would help in enhancing application features with minimal effort.
- Script/Framework Versioning
Versions of framework/scripts should be maintained either in a local repository or versioning tool, which would help in easy monitoring of changes to the software code.
A library should contain all reusable components and external connections such as databases, generic functions, application functions etc. Software testers should be exposed only to the implemented libraries and tests should be performed by invoking these libraries.
Your automation framework should not just focus on a level of automation that testers are traditionally involved in, but apply to the entire software, meaning that unit, contract, component and integration tests, across both frontend and backend should all fall under your strategy with an approach tied in accordingly.
I will go into more details around this in my next article when I unpack the anatomy of a test scripting framework further.
It's only once you understand how you plan to build your framework that you should then only consider the appropriate automation tool. Too often companies tend to approach automation looking at the tool first and then build around that when in reality, a tool is simply just a means to enable interaction with an object or application (more on that in my later articles).
While choosing the right tool is important (especially when licensing fees are involved), it should not be your primary driver and its better to fit a tool into your framework and strategy rather than the other way around, otherwise you may find yourself locked into a tool that is perhaps not fit for purpose and increasingly irreplaceable.
I also want to add at this point that there is another reason why I choose to focus on my scripting before a tool is because true automation needs to be scripted. There may be many vendors that offer a low code/no code automation solution, but these tend to not work unless you only intended for your automation to be bare bones. If you want deep coverage and high-quality automation that is highly reusable, then you will need to apply a programmatic approach to your automation efforts.
Lastly comes your processes. Like every component in the automation framework, this is also important and if you are not going to get this right, automation will continue to frustrate you. First up, proper automation takes time and it needs to be provided within the development cycle to be built correctly. Much of this focus should be on building automation work across all areas of the stack in work planning with automation seen as a set definition of done before you can expect automation efforts to be taken seriously
Automation is also a culture that requires the whole team to contribute to this. The more unit tests developers right the fewer integration tests are likely required by the testers making their automation loads easier. It also requires getting clarity on functionality early and the team is willing to work around the extra overhead of automation if needed.
Automation is complex, which is why it remains such a problematic focus for many organizations. However, if you want to make a success out of your automation approach it's important that you get all the different components of it working together.
In saying all this, I’m not trying to mitigate some of the technical work involved to make it happen - which I will go through in more detail with my next articles when I look at the anatomy of a test automation framework and a test automation tool.