Business Case For Cloud Based Testing


Organizations invest millions of dollars in developing and maintaining software products / applications that are core to their businesses and help them to improve their competitive advantage through operational excellence. Continuous changes in business environment forces the organizations to innovate and optimize their business process, resulting in continuous changes in the software applications that support the business processes. As a consequence, delivering applications rapidly that are defect free, scalable, and reliable becomes challenging. Testing becomes a critical and vital step in the process – not only in terms of coverage but also in terms of performance, security and usability. Setting up test environments that closely mirror the production environment can be expensive – in terms of hardware, licenses, people.

'Testing as a Service' – a new paradigm of Cloud based 'On Demand' testing service can help organizations to address this challenge. This article focuses on how organizations can optimize their IT budget through a strategic initiative in the form of 'On Demand' testing.


The recent sharp downturn in the economy is forcing organizations to reconsider their approach towards IT investments. In a world, where companies are more focused towards improving efficiencies and return of capital employed, CIOs / CTOs need to re-consider how they can reduce their technology investments, or get higher return on the same or incremental investments. Testing is critical to enhance user satisfaction and reduce support cost. However, testing requires organizations to invest in people, tools and environments and can take up a significant percentage of the available budget. But quality can never be compromised. New ways of development and testing are enabling organizations to ensure higher quality but with significantly lower investments.

Requirements for a Test Lab

Testing is a vital phase in any software development and maintenance initiative. Frequently changing requirements coupled with a reduced development life cycle has increased the pressure on testing teams to do more with less. A dedicated test lab is one of the solutions to handle this challenge. Traditionally, to support a test lab initiative, an organization would need to put the following infrastructure / resources in place:

· Target testing environment, similar to production environment
· Multiple target software platform for compatibility testing
· Skilled professionals to design, develop, and execute test scripts; and analyze the results of the tests
· A good test automation software with multiple virtual user licenses
· Sufficient bandwidth for simulating real life scenarios
· Any other equipment required (firewall, switches, etc.) to simulate realistic environment

Challenges Faced

Setting up a dedicated, in-house test lab comes with its own set of challenges. Some of the major challenges faced are:

· Infrastructure: hardware & software resources, establishment of proper tools & processes, and other resources like bandwidth. This creates a strain on the overall budget.
· Scalability: To ensure that software works in a real life situation, it needs to be tested in a real life environment. It is not easy for an organization to create a scalable infrastructure that simulates the production environment.
· Cost: There is a major capital investment required to own a proper test lab. Since most of the cost is a fixed cost (hardware, software, tool licenses) it also creates a challenge to allocate budgets for this kind of investment and justify the ROI.
· Availability of skilled engineers: Skilled test engineers, especially automation engineers, are not readily available and are very expensive.

Cost, along with other challenges of such an initiative causes decision-makers to compromise on critical aspects such as regression coverage, and release software without completely validating load, performance, and scalability testing.

Cloud Testing – New Paradigm

Cloud Computing, one of the most highly publicized IT technology trends, is a new approach to deploy / test applications "over the Internet". Cloud Testing utilizes the same computing concept to extend current testing paradigms using shared, scalable, 'on-demand' testing infrastructure that is allocated on a "pay as you go" basis. This model provides an unparallel flexibility of ramping up and teasing down a testing environment in short notice. A new test harness can be launched in the cloud with all the necessary configuration work completed, including operating system, software, etc. in almost no time. Organizations need not procure any server, tools or licenses-they need to hook up, deploy the software, test, and start paying for just the resource usage. Same efficiency applies to shutting off a cloud environment – just cancel what you do not need. The Cloud Testing model's flexibility reduces much of the capital cost, risk and effort associated with establishing an appropriate testing environment for the enterprise. More importantly, organizations can focus on their core capabilities.

Using Cloud for Test Environment

As software applications become more critical for business changes, the software development process is becoming more agile, distributed and non cohesive. This, along with the emergence of a global delivery model, has resolved in smaller distributed teams operating independently for development, testing, and integration. To support this kind of development processes and the need for continuous testing, organizations spend a major part of their budgets in setting up test environments and automated tool licenses.

Cloud Computing reduces the cost by providing a test environment and easier provisioning. Scaling up and teasing down a test environment is possible within a very short time, sometimes within minutes. This, along with 'on demand' testing services by testing service providers, also helps organizations to reduce capex as well as opex.

Suggested Approach

To achieve maximum out of their cloud testing initiative, organizations need to have a cloud based test strategy before initiating the process. Organizations need to come out with a clear plan stating the testing goals, types of tests to be conducted, benchmarks, infrastructure requirements, and last but not the least, a gap analysis of required and available resources.

Define Test Strategy: For a successful cloud based testing initiative, a test strategy should be defined. It should include the objective of this initiative (cost savings, scalability of the infrastructure, ROI), types of testing to be done on cloud (System testing, load testing, UAT), Insource / outsource among other normal planning items like timeline, dependencies , and risks.

Identify Infrastructure Requirements: Based on the test strategy, organizations need to identify resources required. These resources should include Hardware, Software requirements, Test Automation Tools & number of virtual users, Bandwidth, processing power, storage capacity, etc. Organizations also should identify the usage requirements.

Identify Service Providers: It is absolutely critical to identify a service provider with an established reputation to ensure quality and reliability of services. It is suggested that organizations identify service providers for providing end to end services starting from infrastructure, tool licenses, and provisioning. One important criterion to be evaluated is the time required for provisioning and tear down the required infrastructure.

Test Execution: Once a service provider is identified, organizations can start testing as per their plan. Organization should plan to optimize the usage of the infrastructure to minimize their cost.

Monitoring & Measurement: It is important that the results of the cloud testing initiative are validated continuously on a scheduled or event-driven basis. This will enable organizations to intercept and resolve issues, find cloud usage patterns, and measure success against stated objective. This monitoring & measurement activities are key to demonstrate the ROI of cloud testing.

Testing as a Service (TaaS)

Although having a cloud infrastructure solves most of the challenges related to test lab, organizations still find it difficult to find skilled testers who can prepare test strategies and perform test design and test automation. 'On Demand' testing or 'Testing as a Service' offered by leading testing service providers can help organizations in overcoming these challenges. In this model, the service providers take over the complete Cloud testing initiative. This helps to maximize the ROI as internal resources gets free and can be deployed for other core initiatives. This also helps to convert the fixed cost to variable cost. The flexible scale offered by the service providers also helps to shorten the test cycle, resulting in a faster time to market.


Organizations can leakage cloud based testing for higher flexibility and lower cost. 'Testing as a Service' provides organizations an option to set up a virtual test lab without any upfront investment in Lab infrastructure, Automation tools licenses, and skilled resources. The Pay-per-use pricing models are great in controlling IT budget and maximizing ROI.

Source by Somenath Nag