When evaluating the context of the SUT and its environment, the factors that influence test automation must be identified in order to find an appropriate solution. These may include the following factors:
SUT interfaces
The automated test cases invoke actions on the SUT. To do this, the SUT must provide interfaces that can be used to control the SUT. This can be through UI controls, but also through lower-level software interfaces. In addition, some test cases may provide interfaces at the communication layer (e.g., via TCP/IP, USB, or proprietary messaging interfaces).
Decomposition of the SUT allows test automation to interact with the SUT at different test levels. Levels. It is possible to automate testing at a particular level (e.g., component level, system level), but only if the SUT adequately supports it. For example, at the component level, there may be no user interface that can be used for testing, so various, possibly customized, software interfaces (also called test hooks) must be available.
Third-party software
Often, the SUT consists not only of software written in-house, but may also include third-party software. In some contexts, this third-party software needs to be tested, and if test automation is warranted, a different test automation solution may be required, such as using an API.
Levels of Intrusion
Different test automation approaches (using different tools) have different levels of intrusion. The more changes need to be made to the SUT specifically for automated testing Testing, the higher the level of intrusion. Using dedicated software interfaces requires a high level of while using existing UI elements has a lower level of intervention. The use of hardware elements of the SUT (such as keyboards, hand switches, touch screens, communication interfaces) have an even higher degree of intrusion.
The problem with a higher level of intrusion is the risk of false alarms. The TAS may have failures that are due to the level of intrusion imposed by the tests, but these are not likely when the software system is deployed in a real environment. Testing with a high level of intrusion is usually an easier solution for the test automation approach.
Different SUT architectures
Different SUT architectures may require different test automation solutions. A different approach is required for a SUT written in C++ with COM technology than for a SUT written in Python. It may be possible to handle these different architectures with the same test automation strategy, but this requires a hybrid strategy that can support both architectures.
Size and complexity of the SUT
Consider the size and complexity of the current SUT and plans for future development. For a small and simple SUT, a complex and ultra-flexible test automation approach may not be warranted. A simple approach may be more appropriate. Conversely, it may not be advisable to implement a small and simple approach for a very large and complex SUT. However, sometimes it is appropriate to start small and simple, but this should be a temporary approach.
Some of the factors described here are known (e.g., size and complexity, available software interfaces) when the SUT is already available, but most of the time test automation development should begin before the SUT is available. In this case, several things need to be estimated or the TAE can specify the software interfaces that will be needed.
Even if the SUT does not yet exist, test automation planning can begin. Example:
- Once the requirements (functional or non-functional) are known, candidates for automation can be selected from those requirements, along with identifying the means to test them. Begin planning for these candidates, including identifying the requirements for automation and defining the test automation strategy.
- Once the architecture and technical design are developed, the design of software interfaces to support testing can be undertaken.
Source: ISTQB®: CTAL Test Automation Engineer Syllabus Version 1.0