Know how to test where
Software testing is a complex science and more than simply just a verification process of checking if things work once the team are done with their development parts. For one, software itself is complicated and there are so many permutations and considerations to consider in any one system, that you can’t simply just verify that things work.
There are so many more technical details around the system, functionally, architecturally, performance or security-wise and the intended business use that is needed to ensure quality in this regard. However, at the same time, no piece of software is every truly done and with software development very much an iterative process where small enhancements are added on a daily basis, automation and efficiency because very important. And in order to ensure you maximise that efficiency without jeopardising the quality of the system across all of its layers, it vital that you know how to test the software in the most appropriate fashion.
So, in this new series, I will delve into the different aspects of each facet of testing, from unit to end to end testing, along with unpacking aspects of the UI or backend and how you need to approach testing these different. I will try and provide several best approaches to testing in each o these areas, though the final truth is likely that some specifics to your particular software architecture or team culture may warrant a different approach.
To start off, I will begin with providing a high level of places that you should be testing and then will write an article on some of how you can utilise each of these across the different forms of testing.
The different layers of testing:
The most important, but often least utilised, aspect of testing strategy. It’s essentially the code that tests the code. Simplistic in construction and design, but actually the one where most emphasis should be placed, because it’s the easiest to automate and places where you are going to likely achieve most of your test coverage in the shortest space of time. I have already written an extensive article on this in the past, but needless to say, you should be placing a lot of emphasis in developing these tests in your team.
And it’s not just because they’re fast, but because they are closer to the code, they are also easier to debug, identify the error and maintain as its easy to align actually code changes to specific unit tests meaning that it offers fantastic benefits to traceability of your coverage and test execution.
While they require a fair amount of coding knowledge to wire effectively, they are actually the earliest test to write, focusing simply on calling functions, inputting values and asserting on those outputs. At times there may be a fair amount of mocking to mimic the operation and interaction of other components but once these frameworks have been implemented, they remain relatively easy to maintain.
Similar to unit tests, but rather than focusing on small modules of code, rather grouping several functions of code together as part of a microsystem or microservice and making use of extensive mocking to provide a decent level of coverage.
Component tests can be quite difficult to script, require detailed knowledge of how the system works and are often fairly time-consuming to put together. Why they remain important though is because of the types of defects they catch. While unit tests ensure that your code works on a fundamental level how different modules of code operate together is often different entirely and this allows you to have a decent range of automated tests in this layer. They are the closest you will get to full integration tests without the need for an actual integrated environment and as such, make execution in an automated pipeline a lot easier.
Contract testing is a very lightweight test that is perhaps most relevant for APIs but can also be utilised by stored procedures, databases and frontend and is essentially just a test that looks for the relevant fields in which it needs to integrate with to ensure that the files it requires for interaction are still exactly as it requires. This is used to test for breaking changes when other teams might make a change on a system and not aware that it may affect another system.
This is typically where most of the testers will apply their focus, and where code is deployed into a shared environment and put through testing with other components to ensure they work correctly. The focus on this testing is more in using relevant data rather than mocked and testing a happy day scenario across the entirety of its journey. The idea here is not to test an extensive number of scenarios but is very important in ensuring that a system works as close as possible o the way a customer would use it. They are relatively time-consuming to script, slower in execution through a pipeline and require fairly detailed data but remain necessary and increase in importance the larger the system.
Acceptance / End-to-End testing
Very similar to integration testing but the larger the system the more it encompasses. While a small system would probably have their integration testing serve the purposes of a full end to end system, a large system of many multiple microservices and applications will probably have different levels of integration testing before one big end to end phase of testing where the entire system is put together. In big companies, this may be manual as opposed to automated just due to the complexity of all the moving parts, but if companies can build their systems right from the ground-up, even larger companies can ensure this entire process gets automated.
Verification / Smoke Testing
You’ve tested your system, but you still need to deploy it into Prod or as close to a Prod-like environment to ensure it works. The software has already been extensively tested, but you want to ensure that it remains operational, especially when a server or other software updates happen on the infrastructure. These scripts are lightweight but hot enough critical services to ensure that nothing major is broken. They don’t need to complete a full user journey if they are in Prod, but it should be able to ensure that all parts of the application can speak to the others and are operational.
Where is regression testing in this list?
I haven’t spoken about regression testing in this mix because it really is something that falls within all these areas. Additionally, with the move towards more automated testing, it shouldn’t just become a thing of what tests form part of your regression because they all should form part of your regression. So, while the act of regression is still necessary, these should be automated and typically form part of all of the above categories and the more you rely on unit, contract and automated integration testing, the faster and more regular you can execute this automation with little difficulty.
Now whether you are approaching testing from a front-end or backend perspective and regardless of the architecture of your system, you should ensure that these testing phases are considered. Just focusing on one aspect is likely to leave gaps in your testing approach and therefore the quality of your software.