Software Testing Life Cycle (STLC) defines the steps/ stages/ phases in testing of software. However, there is no fixed standard STLC in the world and it basically varies as per the following:

• Software Development Life Cycle
• Whims of the Management

Nevertheless, Software Testing Life Cycle, in general, comprises of the following phases:

STLC Phases

Phase	Activity	Deliverables	Necessity
Requirements/ Design Review	You review the software requirements/ design (Well, if they exist.)	‘Review Defect’ Reports	Curiosity
Test Planning	Once you have gathered a general idea of what needs to be tested, you ‘plan’ for the tests.	Test Plan Test Estimation Test Schedule	Farsightedness
Test Designing	You design/ detail your tests on the basis of detailed requirements/design of the software (sometimes, on the basis of your imagination).	Test Cases / Test Scripts /Test Data Requirements Traceability Matrix	Creativity
Test Environment Setup	You set up the test environment (server/ client/ network, etc) with the goal of replicating the end-users’ environment	Test Environment	Rich company
Test Execution	You execute your Test Cases/Scripts in the Test Environment to see whether they pass.	Test Results (Incremental) Defect Reports	Patience
Test Reporting	You prepare various reports for various stakeholders.	Test Results (Final) Test/ Defect Metrics Test Closure Report Who Worked Late & on Weekends (WWLW) Report [Depending on how fussy your Management is]	Diplomacy

Note that the STLC phases mentioned above do not necessarily have to be in the order listed; some phases can sometimes run in parallel (For instance, Test Designing and Test Execution). And, in extreme cases, the phases might also be reversed (For instance, when there is Cursing prior to Testing).

Interestingly, no matter how well-defined a Software Testing Life Cycle you have in your project or organization, there are chances that you will invariably witness the following widely-popular cycle:

• Testing
• Cursing

In this type of STLC, you skip phases like design review, test planning, etc – in the hope that the skipping will save you some time and/or cost [But, it never does].

Requirement Analysis

During this phase, the testing team gathers application knowledge & details about priorities and focus area. The testing team studies the requirements from a testing point of view to identify the testable requirements. The QA team may interact with various stakeholders (Client, Business Analyst, Technical Leads, System Architects etc) to understand the requirements in detail. Requirements could be either Functional (defining what the software must do) or Non-Functional (defining system performance /security availability). Automation feasibility for the given testing project is also done in this stage. Activities of this phase are to identify types of tests to be performed, gather details about testing priorities and focus, prepare Requirement Traceability Matrix (RTM) , identify test environment details where testing is supposed to be carried out, Automation feasibility analysis. Deliverables of this phase are the list of Testable requirements.

Test Planning

Test plan derives the complete process of the Software

Application. Activities of this phase are the preparation of test plan/strategy document for various types of testing, test tool selection, test effort estimation, resource planning and determining roles and responsibilities, training requirement. Deliverables of this phase are test plan /strategy document, effort estimation document.

Test Plan contains the following details:

Test plan identifiers

It gives a unique identifier with a version number. Can also be called as document number.

Introduction

It gives an overview of the software application and purpose of this document.

Test scope

It defines the extent of testing and defines the clear boundaries for the testing process.

Test objective

It defines Testing Goal and provides guidelines for Testing Process.

Assumption and Dependencies

It provides documents prerequisites, which if not met could have a negative impact on schedule, Quality of Budget of the testing project. Examples Readiness of Test Environment setup etc.

Risk Analysis

It provides document possible risks during testing activities and their estimated impact on the test effort and schedule.
Examples of risks: Build not available on time for testing, test environment-related issues, defect not getting on time

Strategy

It provides what methods of testing to be used Black box or white box or combination. It provides levels of testing in scope unit, integration etc It provides a number of test execution cycles to be planned It provides non-functional tests to be conducted performance, Security. It provides a process for Test Environment setup, Test Data creation.

Features to be tested

This provides a listing of the functionality mentioned in Software requirement specifications which are to be tested & features not be tested and features which are not in the current scope of testing.

Role and Responsibilities

It provides a testing team structure and responsibilities.

Test Schedule

It provides major test activities, their sequence and their interdependencies, time estimation for each activity and test resource plan i.e. planning of people, tools, infrastructure, and facilities.

Entry and Exit criteria

It describes when to start testing and when to stop testing. For example, testing can be started when build, test environment and test data are ready and for exit criteria when all test cases are executed and all the defects are closed. Suspension criteria: It denotes when to pause or suspend Testing activity. For example when major functional defects are found blocking the further testing process

Resumption Criteria

When all the suspension issues are resolved to continue with the testing process.

Test environment

It provides hardware /software configuration, Environment requirements for each stage and type of testing, Procedures for Configuration management and release/version control, location for individual test artifacts, defect tracking mechanism.

Communication plan

It provides what to communicate in terms of reports, status, documents for information etc

It provides whom to communicate in terms of stakeholders, when, how in terms of emails, formal and informal meetings, working sessions.

It provides which communication tools to be used such as electronic bulletin boards, emails, intranet sites etc.

It provides Escalation procedures viz communication protocol, a hierarchy to be followed.

It provides miscellaneous items such as project contact lists, meeting audiences.

Testing tools to be used

Test management tool, configuration management tool, defect tracking tool, special tests testing tools, test script development tool, test utilities i.e record playback or test data creation etc. Document Control It includes reviewer of the test plan, approval of test plan and test plan distribution list.

Test Design

This phase of STLC covers the process of getting inputs to start test design, the creation of test scenarios, constructing test cases, creating test data, using techniques like equivalence class partitioning, boundary value analysis, and requirement traceability matrix. This phase aims at documenting „how‟ to do testing. This phase involves creation, verification, and rework of test cases & test scripts. Test data, is identified/created and is reviewed and then reworked as well. Activities of this phase include to create test cases, create test data, automation scripts, review, and baseline test cases and scripts Deliverables of this phase are Test cases/scripts and Test data.

Once test planning is completed, the test plan is available to the testing team members. It gives clarity about the testing scope and entire the testing process that needs to be followed. The first key activity of the testing team is to design a test case. Test case design consists of the creation of test scenarios, test cases, test data, and requirement traceability matrix.

Test Scenario

Test Scenario can be defined as a top view of functionalities under test or what is needed to test at a high level. Test scenario identification ensures coverage of all features of the application in testing. If test scenarios are not identified, it may result in a particular functionality not getting tested in detail or not getting tested at all. Test scenario further helps in developing End to End or combination scenarios which actually simulate the complete user interaction with the application. Though test scenarios are important to create, they need not be formal deliverable always.

Identification of Test Scenarios

From use case- Use case describes how a user (Actor) will interact with (or Use) the application functionalities (processes). Here each process is the Test Scenario.

For example, in the user registration use case, registration can be considered as one process. By functionality breakdown – Functionality breakdown includes identifying various features or services provided by the application based on the requirement specification documents. A number of test scenarios will be equal to the number of broader level functionalities of the application. Using state transition diagram technique- In transition based applications, the status of an entity can change based on the user transactions. The
process that changes the state of an entity can be identified as one test
scenario. This is called a state transition technique. For example, the cancel
order process can change the state of order from in process to cancel.
Hence cancel order is identified as one test scenario.

Test case

Test case provides a detailed procedure that helps to test a particular aspect or feature of an application in detail. It specifies how to test the particular functionality and describes steps to be performed with input data and output expectations based on the user requirements. Test cases are used while test execution to check the actual behavior of the application.

The process of Test case Creation

Test case creation includes identification of test conditions and then documenting the detailed process to test this test condition. Test conditions can be identified in the following two ways: From Test Scenario breakdown- For every scenario, identify various paths – a normal or happy path, alternate path, and error flow. Each path will lead to a unique test condition. From Use Case or functionality- Check normal flow, alternative flow and error flow documented for use case or functionality. Each of them leads to a unique test condition.

Test Case Review

Once test cases are created by the tester they need to undergo a review before they are actually used in validation or dynamic testing. Reviewers can peer i.e. test team members, Superior i.e. Test lead or application expert. Sometimes, customers can also review test cases.

Why review Test Cases?

Test cases are the most important deliverable of the Test design phase and if needed, are shared with customers. To check correctness and completeness of documented Test cases.

Test Case Storage

Test cases should be stored in Test Management Tool. Quality center and Testlink are Test Management Tools.

Test Data

The test case includes test data to be used during test case execution. One test case can be executed multiple times with different sets of test data to check the application behavior under different conditions.

Requirements Traceability Matrix (RTM)

Requirement Traceability Matrix is a tool to ensure that the project`s scope, requirements, and deliverables are in line.

Use of RTM

• To check that all requirements are implemented
• Helps to do impact analysis when any particular requirement changes –Tester can find out the affected. Test scenarios and test cases using RTM.

Test Implementation

This phase implements the test cases by using Automation tools by writing test scripts. Deliverables of this is Test scripts

Test Execution

On completion of the Test design phase, test cases which document how to do testing are ready. The next logical phase is “Test Execution”. In this phase test cases are executed and the application’s behavior is observed to evaluate whether it is as expected. Test execution is defined as the processing of a test case by the software under test to produce an outcome. During this phase, the testing team will carry out the testing based on the test plans and the test cases prepared. Bugs will be reported back to the development team for correction and retesting will be performed.

Activities are

• Document test results, and log defects for failed cases
• Map defects to test cases in RTM
• Retest the defect fixes
• Track the defects to closure

Deliverables

• Completed RTM with the execution status
• Test cases updated with results
• Defect reports

When to start Test Execution?

Test execution entry criterion defines when the test execution should start. Test Environment readiness, availability of test cases and test data, completion of prior levels of testing and availability of build for testing are the minimal requirements for test execution.

Test Execution Cycle

Test Execution Steps

1. Prioritize test cases: Test case prioritization is needed when a tester has insufficient time in hand. Test cases which test the core functionality of the application or critical test cases are executed on high priority and rest of the test cases follow them based on the time availability.
2. Execute identified test cases.
3. Perform the steps mentioned in the test case and observe the application’s response.
4. Compare Actual result and Expected Result and accordingly update the status column as “pass” if both match or “fail”. Other status value can “on hold” if the test case is dependent on completion of a failed test case or” not executed” if the functionality is not in the scope of the current cycle.
5. Log defects for failed test cases.

When to Stop Testing?

Testing should be stopped when it meets the exit criteria put in the test plan. Test manager, client and respective stakeholders will consider all above and similar other factors to decide when to stop.

Test Closure

Test closure is the last phase of STLC. In this data is collected from completed test activities to consolidate experience, leanings, facts, and figures. Test closure activities include the following major tasks:

• Check which planned deliverables have been delivered
• Create a Test Summary Report.
• Handover to test ware to the maintenance team.

Test summary Report provides details of the testing outcome and advice on the release readiness of the application. The purpose is to summarize the results of the designated testing activities and to provide evaluations based on these result. Testing team will meet, discuss and analyze testing artifacts to identify strategies that have to be implemented in the future, taking lessons from the current test cycle. The idea is to remove the process bottlenecks for future test cycles and share best practices for any similar projects in the future.

Mapping of Software Test Process and Testing Types

There are four major phases in the software test life cycle which are earlier explained in this paper. Each phase requires some sort of testing to be performed. In test analysis and test preparation phase, only verification of required documents and other test documents is done. Verification involves all types of reviews, inspection, and walkthroughs. It is mainly done before validation. Once Test Preparation is done and all artifacts are reviewed and baselined, then Test execution begins, wherein actual validation is performed. In this phase, all types of testing are performed which are shown in Below Figure below at a high level.

What is Test Strategy?

A strategy plan for defining the testing approach, what you want to accomplish and how you are going to achieve it.

Test Strategy vs. Test Plan:

Over the years, I see a lot of confusion between these two documents. So let’s start with basic definitions. Generally, it doesn’t matter which comes first. The test planning document is a combination of strategy plugged with an overall project plan. According to IEEE Standard 829-2008, strategy plan is a sub-item of test plan.

Every organization has its own standards and processes to maintain these documents. Some organizations include strategy details in test plan itself (here is a good example of this). Some organizations list strategy as a subsection in the testing plan but details are separated out in the different test strategy document.

Project scope and test focus are defined in the test plan. Basically, it deals with test coverage, features to be tested, features not to be tested, estimation, and scheduling and resource management.

Whereas test strategy defines guidelines for test approach to be followed in order to achieve the test objectives and execution of test types defined in the testing plan. It deals with test objective, approach, test environment, automation strategy and tools, and risk analysis with a contingency plan.

To summarize the test plan is a vision of what you want to achieve and test strategy is an action plan designed to achieve this vision!

Common sections of test strategy document:

Step #1 – Scope and Overview:

Project overview along with information on who should use this document. Also, include details like who will review and approve this document. Define testing activities and phases to be carried out with timelines with respect to overall project timelines defined in the test plan.

Step #2 – Test Approach:

Define testing process, level of testing, roles, and responsibilities of every team member. For every test type defined in test plan (e.g. unit, integration, system, regression, installation/un-installation, usability, load, performance, and security testing) describe why it should be conducted along with details like when to start, test owner, responsibilities, testing approach and details of automation strategy and tool if applicable.

In test execution there are various activities like adding new defects, defect triage, defect assignments, re-testing, regression testing and finally test sign-off. You must define the exact steps to be followed for each activity. You can follow the same process which worked for you in your previous test cycles. A Visio presentation of all these activities including a number of testers and who will work on what activity is very helpful to quickly understand roles and responsibilities in the team.

E.g. defect management cycle – mention the process to log a new defect. Where to log in, how to log new defects, what should be the defect status, who should do defect triage, whom to assign defects after triage etc.

Also, define change management process. This includes defining change request submission, template to be used, and process to handle the request.

Step #3 – Test Environment:

Test environment setup should outline information about a number of environments and required setup for each environment. E.g. one test environment for functional test team and another for the UAT team. Define the number of users supported on each environment, access roles for each user, software and hardware requirements like operating system, memory, free disk space, number of systems etc.

Defining test data requirements is equally important. Provide clear instruction on how to create test data (either generate data or use production data by masking fields for privacy).

Define test data backup and restore strategy. Test environment database may run into problems due to unhandled conditions in the code. I remember the problems we faced on one of the projects when there was no database backup strategy defined and we lost whole data due to code issues. Backup and restore process should define who will take backups when to take a backup, what to include in backup when to restore the database, who will restore it and data masking steps to be followed if the database is restored.

Step #4 – Testing Tools:

Define test management and automation tools required for test execution. For performance, load and security testing describe the test approach and tools required. Mention whether it is open source or commercial tool and how many users are supported on it and plan accordingly.

Step #5 – Release Control:

As mentioned in our last UAT article, unplanned release cycle could result in different software versions on test and UAT environments. Release management plan with proper version history will ensure test execution of all modifications in that release.

E.g. set build management process which will answer – where new build should be made available, where it should be deployed, when to get the new build, from where to get the production build, who will give the go, the no-go signal for production release etc.

Step #6 – Risk Analysis:

List all risks that you envision. Provide a clear plan to mitigate these risks and also a contingency plan in case if you see these risks in reality.

Step #7 – Review and Approvals:

When all these activities are defined in test strategy plan it needs to be reviewed for sign-off by all entities involved like project management, business team, development team, and system administration (or environment management) team. Summary of review changes should be tracked at the beginning of the document along with approver name, date and comment. Also, it’s a living document meaning this should be continuously reviewed and updated with the testing process enhancements.

Conclusion:

Test strategy is not a piece of paper. It’s the reflection of whole QA activities in software testing life cycle. Refer this document time to time in test execution process and follow plan till the software release. When project nears the release date it’s fairly easy to cut on testing activities by ignoring what you have defined in the test strategy document.

But it is advisable to discuss with your team whether or not cutting down on any particular activity will help for release without any potential risk of major issues post-release.

Most of the agile teams cut down on writing strategy document as team focus is on test execution rather than documentation. But having a basic test strategy plan always helps to clearly plan and mitigate risks involved in the project. Agile teams can capture and document all high-level activities to complete test execution on time without any issues.