Introduction to TAS development

The development of a TAS is similar to other software development projects. It can follow the same procedures and processes, including peer reviews by developers and testers. Specific to a TAS are its compatibility and synchronization with the SUT. These must be considered during TAA design and TAS development. The SUT is also affected by the test strategy, e.g., by the need to provide test interfaces to the TAS.

In this section, the development process of the TAS and the process-related aspects of compatibility and synchronization with the SUT are explained using the Software Development Lifecycle (SDLC). These aspects are equally important for any other development process chosen or in place for SUT and/or TAS development – they must be adapted accordingly.

Figure 2: Basic SDLC for TAS

The requirements for a TAS must be analyzed and collected (see Figure 2). The requirements form the basis for the design of the TAS as defined in the TAA. The design is translated into software using software engineering approaches. Note that a TAS may also use specialized test equipment hardware, which is not considered in this article. Like any other software, a TAS must be tested. This is typically done through basic capability testing for the TAS, followed by interaction between the TAS and the SUT. After a TAS has been implemented and used, further development of the TAS is often required to add more testing capabilities,
modify tests, or update the TAS to align with the changing SUT. TAS evolution requires a new round of TAS development in accordance with the SDLC.

Please also note that the SDLC does not map the backup, archiving, and demolition of a TAS. As with TAS development, these procedures should follow established methods in an organization.

Compatibility between the TAS and the SUT

Process Compatibility
Testing of a SUT should be synchronized with its development – and in the case of test automation, with TAS development. Therefore, it is beneficial to align the processes for SUT development, TAS development, and testing. A big gain can be achieved if SUT and TAS development are compatible in terms of process structure, process management and tool support.

Team compatibility
Team compatibility is another aspect of compatibility between TAS and SUT development. When TAS and SUT development are approached and managed with a compatible mindset, both teams benefit from reviewing each other’s requirements, designs and/or development artifacts, discussing issues and
finding compatible solutions. Team compatibility also helps teams communicate and interact with each other.

Technological compatibility
In addition, technological compatibility between the TAS and the SUT should be considered. It is beneficial to design and implement seamless interaction between the TAS and the SUT from the beginning. Even if this is not possible (e.g., because technical solutions are not available for either the TAS or the SUT), seamless interaction may be possible through the use of adapters, wrappers, or other forms of intermediaries.

Tool compatibility
Tool compatibility between TAS and SUT management, development, and quality assurance must be considered. For example, if the same tools are used for requirements management and/or issues management, information exchange and coordination of TAS and SUT development will be facilitated.

Synchronization between TAS and SUT

Synchronization of requirements
After requirements elicitation, both SUT and TAS requirements must be developed. TAS requirements can be divided into two main groups of requirements: (1) requirements that relate to the development of the TAS as a software-based system, such as requirements for the TAS functions for test design, test specification, test results analysis, etc., and (2) requirements that relate to testing the SUT using the TAS. These so-called test requirements correspond to the SUT requirements and reflect all those features and characteristics of the SUT that are to be tested by the TAS. Whenever the SUT or TAS requirements are updated, it is important to check the consistency between the two and ensure that all SUT requirements to be tested by the TAS have defined test requirements.

Synchronization of development phases
To ensure that the TAS is ready when it is needed to test the SUT, the development phases must be coordinated. It is most efficient when the requirements, designs, specifications, and implementations of the SUT and TAS are synchronized.

Synchronization of defect tracking
Errors can relate to the SUT, the TAS, or the requirements/design specifications. Because of the relationship between the two projects, correcting an error in one project may also affect the other. Defect tracking and confirmation testing must relate to both the TAS and the SUT.

Synchronize the development of the SUT and TAS
Both the SUT and the TAS may evolve to include or disable new features, to correct bugs, or to accommodate changes in their environment (including changes to the SUT or TAS, respectively, since one is an environmental component for the other). Any change made to one SUT or one TAS may impact the other, so management of these changes should affect both the SUT and the TAS.
Two approaches to synchronization between the SUT and TAS development processes are shown in Figure 3 and Figure 4.
Figure 3 shows an approach in which the two SDLC processes for the SUT and the TAS are synchronized mainly in two phases: (1) the TAS analysis is based on the SUT design, which in turn is based on the SUT analysis, and (2) the testing of the SUT uses the deployed TAS.

Figure 3: Synchronization example 1 of TAS and SUT development processes

Figure 4 shows a hybrid approach with both manual and automated tests. Whenever manual tests are used before automated tests or when manual and automated tests are used together, the TAS analysis should be based on both the SUT design and the manual tests. This way, the TAS is synchronized with both. The second important synchronization point for such an approach is as before: testing the SUT requires deployed tests, which in the case of manual tests could simply be the manual test procedures to be followed.

Figure 4: Synchronization example 2 of TAS and SUT development processes

Building Reuse into the TAS

Reuse of a TAS refers to the reuse of TAS artifacts (from any level of its architecture) across product lines, product frameworks, product domains, and/or project families. The requirements for reuse are derived from the relevance of the TAS artifacts to the other product variants, products, and/or projects. Reusable TAS artifacts
may include:

• (Parts of) test models of test objectives, test scenarios, test components, or test data.
• (Parts of) test cases, test data, test procedures or test libraries themselves.
• The test engine and/or the test reporting framework
• The adaptors to the SUT components and/or interfaces.

While the aspects of reuse are established when the TAA is defined, the TAS can help increase the ability to reuse by:

• Following the TAA or revising and updating it whenever necessary.
• Documenting the TAS artifacts so that they can be easily understood and incorporated into new contexts
• Ensuring the correctness of each TAS artifact so that its use in new contexts is supported by its high quality.

It is important to note that while designing for reuse is primarily a matter for the TAA, maintaining and improving reuse is a concern of the entire TAS life cycle. Continued thought and effort is needed to enable reuse, to measure and demonstrate the value added by reuse, and to encourage others to reuse existing TAS.

Support for a variety of target systems

TAS support for a variety of target systems refers to the ability of a TAS to test different configurations of a software product. Different configurations refer to any of the following characteristics:

• Number and interconnection of SUT components
• Environments (both software and hardware) on which the SUT components run
• Technologies, programming languages, or operating systems used to implement the SUT components
• Libraries and packages used by the SUT components
• Tools used to implement the SUT components.

While the first four aspects affect the TAS at each level of testing, the last aspect applies mainly to testing at the component and integration levels.

The ability of a TAS to test different software product configurations is determined when the TAA is defined. However, the TAS must implement the ability to handle technical variance, and it must include the
manage the TAS functions and components needed for different configurations of a software product.

The handling of TAS variance relative to the diversity of the software product can be handled differently:

• Version/configuration management for TAS and SUT can be used to provide the appropriate versions and configurations of TAS and SUT for each other.
• TAS parameterization can be used to match a TAS to a SUT configuration.

It is important to note that while designing for TAS variability is primarily a TAA concern, maintaining and improving variability is a concern throughout the TAS lifecycle. It requires continuous thought and effort to revise, add, or even remove options and forms of variability.

Source: ISTQB®: CTAL Test Automation Engineer Syllabus Version 1.0