There are certain testing procedures that can be followed on any development project (agile or not) to produce quality products. These include writing tests in advance to express the right behavior, focusing on early avoidance, detection, and elimination of defects, and ensuring that the right types of tests are executed at the right time and as part of the right level of testing. Agile practitioners strive to implement these practices early. Testers in agile projects play a key role in guiding the use of these testing practices throughout the lifecycle.

Test-driven development, acceptance test-driven development, and behavior-driven development are three complementary techniques used in agile teams to perform testing at the various levels of testing. Each technique exemplifies a fundamental principle of testing, the benefits of early testing, and QA activities as tests are defined before code is written.

Test-Driven Development
In test-driven development (TDD), code is developed using automated test cases. The process of test-driven development is as follows:

• Adding a test that captures the programmer’s idea of how a small piece of code should work
• Run the test, which should fail because the code does not exist
• Write the code and run the test in a tight loop until the test passes.
• Refactor the code after the test passes and run the test again to make sure it passes with the refactored code.
• Repeat this process for the next small piece of code, running the previous tests as well as the added tests.

Tests are primarily written at the unit level and are focused on the code, although tests can also be written at the integration or system level. Test-driven development gained its popularity through Extreme Programming but is also used in other agile methodologies and sometimes in sequential lifecycles. It helps developers focus on well-defined expected outcomes. Testing is automated and used in continuous integration.

Acceptance Test-Driven Development
In acceptance test-driven development, the acceptance criteria and tests are defined during the creation of the user stories. Acceptance test-driven development is a collaborative approach that enables all stakeholders to understand how the software component needs to behave and what the developers, testers, and business representatives need to ensure that behavior.

In acceptance test-driven development, reusable tests are created for regression testing. Special tools support the creation and execution of such tests, often as part of the continuous integration process. These tools can connect to the data and service layers of the application so that the tests can be executed at the system or acceptance level. Acceptance-test-driven development enables rapid bug fixing and validation of functional behavior. This helps determine whether acceptance criteria for the function have been met.

Behavior-driven development allows developers to focus on testing the code based on the expected behavior of the software. Because the tests are based on the demonstrated behavior of the software, they are generally easier for other team members and stakeholders to understand.

Specific frameworks for behavior-driven development can be used to define acceptance criteria based on the given/if/then format:

Given initial context,
When an event occurs,
Then ensure some outcome.

From these requirements, the behavior-driven development framework generates code that developers can use to create test cases. Behavior-driven development helps the developer collaborate with other stakeholders, including testers, to define accurate unit tests that focus on business requirements.

The Test Pyramid

A software system can be tested at different levels. Typical test levels are, from the base of the pyramid to the top, unit, integration, system, and acceptance. In the test pyramid, the emphasis is on a large number of tests at the lower levels (base of the pyramid), while the number of tests decreases as development progresses at the upper levels (top of the pyramid). Typically, tests at the unit and integration levels are automated and created using API-based tools. At the system and acceptance level, automated tests are created using GUI-based tools. The concept of the test pyramid is based on the testing principle of early quality assurance and testing (i.e., eliminating defects as early as possible in the lifecycle).

Testing Quadrants, Test Levels, and Testing Types

The test quadrants defined by Brian Marick align the test levels with the corresponding test types in the agile methodology. The test quadrant model and its variants help ensure that all major test types and test levels are included in the development lifecycle. This model also provides a way to differentiate and describe test types to all stakeholders, including developers, testers, and business representatives.

In the test quadrants, tests can be business (user) or technology (developer) focused. Some tests support the work of the agile team and confirm the behavior of the software. Other tests can verify the product. Testing can be fully manual, fully automated, a combination of manual and automated, or manual but supported by tools. The four quadrants are as follows:

• Quadrant Q1 is unit-level, technology-oriented, and supports developers. This quadrant contains unit-level tests. These tests should be automated and included in the continuous integration process.
• Quadrant Q2 is system-level, business-focused, and validates product behavior. This quadrant contains functional tests, examples, story tests, user experience prototypes, and simulations. These tests check acceptance criteria and can be manual or automated. They are often created during user story development to improve story quality. They are useful in creating automated regression test suites.
• Quadrant Q3 is a system or user acceptance level focused on the business and includes tests that critically examine the product against realistic scenarios and data. This quadrant includes exploratory testing, scenarios, process flows, usability testing, user acceptance testing, alpha testing, and beta testing. These tests are often performed manually and are user-focused.
• Quadrant Q4 is a system or operational acceptance level that is technology-oriented and includes tests that critique the product. This quadrant includes performance, load, stress, and scalability testing, security testing, maintainability, storage management, compatibility and interoperability, data migration, infrastructure, and recovery testing. These tests are often automated.

During a given iteration, tests may be required from one or all quadrants. The test quadrants relate to dynamic testing rather than static testing.

The Role of a Tester

Throughout this syllabus, general reference has been made to Agile methods and techniques, and the role of a tester within various Agile lifecycles. This subsection looks specifically at the role of a tester in a project following a Scrum lifecycle.

Teamwork
Teamwork is a fundamental principle of agile development. Agile emphasizes the whole team approach, consisting of developers, testers, and business representatives working together. The following describes the best organizational and behavioral practices in Scrum teams:

• Cross-functional: Each team member brings different skills to the team. The team collaborates on test strategy, test planning, test specification, test execution, test evaluation, and reporting of test results.
• Self-organizing: the team may consist of developers only, but ideally one or more testers should also be included.
• Co-located: the testers sit together with the developers and the product owner.
• Collaborative: Testers work with their team members, other teams, stakeholders, the Product Owner, and the Scrum Master.
• Empowered: Technical decisions related to design and testing are made by the entire team (developers, testers, and Scrum Master), in collaboration with the Product Owner and other teams as appropriate.
• Engaged: The tester is committed to questioning and evaluating the behavior and characteristics of the product in light of customer and user expectations and needs.
• Transparent: the development and testing progress are visible on the Agiletask board.
• Credible: The tester must ensure the credibility of the test strategy, its implementation, and execution, otherwise stakeholders will not trust the test results. This is often achieved by keeping stakeholders informed about the testing process.
• Open to feedback: feedback is an important aspect of the success of any project, especially agile projects.
• Retrospectives allow teams to learn from successes and failures.
• Resilient: Like all other activities in agile projects, testing must be able to respond to change.

These best practices increase the likelihood of successful testing in Scrum projects.

Sprint Zero
Sprint Zero is the first iteration of the project where many preparation activities take place. During this iteration, the tester works with the team on the following activities:

• Identifying the scope of the project (i.e., the product backlog).
• Creating an initial system architecture and high-level prototypes
• Planning, acquiring, and installing the necessary tools (e.g., for test management, defect management, test automation, and continuous integration)
• Creating an initial test strategy for all test levels that consider (among other things) test scope, technical risks, test types, and coverage objectives
• Performing an initial quality risk analysis
• Defining test metrics to measure the test process, test progress in the project, and product quality
• Establish the definition of “done”.
• Create the task board.
• Define when to continue or stop testing before the system is delivered to the customer.

Sprint Zero provides direction on what testing should accomplish and how testing should accomplish this during the sprints.

Integration
In agile projects, the goal is to continuously deliver value to the customer (preferably in every sprint). To enable this, the integration strategy should consider both design and testing. To enable a continuous testing strategy for the delivered functions and features, it is important to identify all dependencies between the underlying functions and features.

Test Planning
Since testing is fully integrated into the agile team, test planning should begin during the release planning session and be updated during each sprint. Test planning for the release and each sprint should consider the items discussed in the Release and Iteration Planning section.

Sprint planning leads to a series of tasks that are entered on the task board, with each task taking a day or two to complete. In addition, all test issues should be tracked to ensure a steady flow of testing.

Agile Testing Practices
Many practices can be useful for testers on a Scrum team, some of which include:

• Pairing: two team members (e.g., a tester and a developer, two testers, or a tester and a product owner) sit together at a workstation to perform a test or other sprint task.
• Incremental test design: test cases and charters are built incrementally from user stories and other test fundamentals, starting with simple tests and moving to more complex tests.
• Mindmapping: mind mapping is a useful tool in testing. For example, testers can use mind mapping to figure out which test sessions to run, point out test strategies, and describe test data.

These practices are in addition to other practices.

Source: ISTQB®: Certified Tester Foundation Level Agile Tester Syllabus Version 1.0