There are a number of Agile approaches in use by organizations. Common practices across most Agile organizations include collaborative user story creation, retrospectives, continuous integration, and planning for each iteration as well as for overall release.This subsection describes some of the agile approaches.
Agile Software Development Approaches
There are several Agile approaches, each of which implements the values and principles of the Agile Manifesto in different ways. In this syllabus, three representatives of Agile approaches are considered: Extreme Programming(XP), Scrum, and Kanban.
Extreme Programming (XP), originally introduced by Kent Beck [Beck04], is an Agile approach to software development described by certain values, principles, and development practices.
XP embraces five values to guide development: communication, simplicity, feedback, courage, and respect.
XP describes a set of principles as additional guidelines: humanity, economics, mutual benefit, self-similarity, improvement, diversity, reflection, flow, opportunity, redundancy, failure, quality, baby steps, and accepted responsibility.
XP describes thirteen primary practices: sit together, whole team, informative workspace, energized work, pair programming, stories, weekly cycle, quarterly cycle, slack, ten-minute build, continuous integration, test-first programming, and incremental design.
Many of the Agile software development approaches in use today are influenced by XP and its values and principles. For example, Agile teams following Scrumoften incorporate XP practices.
Scrum is an Agile management framework which contains the following constituent instruments and practices [Schwaber01]:
- Sprint: Scrum divides a project into iterations (called sprints) of fixed length(usually two to four weeks).
- Product Increment: Each sprint results in a potentially releasable/shippable product (called an increment).
- Product Backlog: The product owner manages a prioritized list of planned product items(called the product backlog). The product backlog evolves from sprint to sprint (called backlog refinement).
- SprintBacklog: At the start of each sprint, the Scrum team selects a set of highest priority items (called the sprint backlog) from the product backlog. Since the Scrum team, not the product owner, selects the items to be realized within the sprint, the selection is referred to as being on the pull principle rather than the push principle.
- Definition of Done: To make sure that there is a potentially releasable product at each sprint’ send, the Scrum team discusses and defines appropriate criteria for sprint completion. The discussion deepens the team’s understanding of the backlog items and product requirements.
- Timeboxing: Only those tasks, requirements, or features that the team expects to finish within the sprint are part of the sprint backlog. If the development team cannot finish a task within a sprint, the associated product features are removed from the sprint and the task is moved back into the product backlog. Timeboxingapplies not only to tasks but in other situations(e.g., enforcing meeting start and end times).
- Transparency: The development team reports and updates sprint status on a daily basis at a meeting called the daily scrum. This makes the content and progress of the current sprint, including test results, visible to the team, management, and all interested parties. For example, the development team can show sprint status on a whiteboard.
Scrum defines three roles:
- ScrumMaster: ensures that Scrum practices and rules are implemented and followed, and resolves any violations, resource issues, or other impediments that could prevent the team from following the practices and rules. This person is not the team lead, but a coach.
- Product Owner: represents the customer, and generates, maintains, and prioritizes the product backlog. This person is not the team lead.
- Development Team: develop and test the product. The team is self-organized: There is no team lead, so the team makes the decisions. The team is also cross-functional.
Scrum (as opposed to XP) does not dictate specific software development techniques (e.g.,test-first programming). In addition, Scrum does not provide guidance on how testing has to be done in a Scrum project.
Kanban [Anderson13] is a management approach that is sometimes used in Agile projects. The general objective is to visualize and optimize the flow of work within a value-added chain. Kanban utilizes three instruments [Linz14]:
- KanbanBoard: The value chain to be managed is visualized by a Kanban board. Each column shows a station, which is a set of related activities,e.g., development or testing. The items to be produced or tasks to be processed are symbolized by tickets moving from left to right across the board through the stations.
- Work-in-Progress Limit: The amount of parallel active tasks is strictly limited. This is controlled by the maximum number of tickets allowed for a station and/or globally for the board. Whenever a station has free capacity, the worker pulls a ticket from the predecessor station.
- LeadTime: Kanban is used to optimize the continuous flow of tasks by minimizing the (average) lead timefor the complete value stream.
Kanban features some similarities to Scrum. In both frameworks, visualizing the active tasks (e.g., on a public whiteboard) provides transparency of content and progress of tasks. Tasks not yet scheduled are waiting in a backlog and moved onto the Kanban board as soon as there is new space (production capacity) available.
Iterations or sprints are optional in Kanban. The Kanban process allows releasing its deliverables item by item, rather than as part of a release. Timeboxingas a synchronizing mechanism, therefore, is optional, unlike in Scrum, which synchronizes all tasks within a sprint.
Collaborative User Story Creation
Poor specifications are often a major reason for project failure. Specification problems can result from the users’ lack of insight into their true needs, absence of a global vision for the system, redundant or contradictory features, and other miscommunications. In Agile development, user stories are written to capture requirements from the perspectives of developers, testers, and business representatives. In sequential development, this shared vision of a feature is accomplished through formal reviews after requirements are written; in Agile development, this shared vision is accomplished through frequent informal reviews while the requirements are being written.
The user stories must address both functional and non-functional characteristics. Each story includes acceptance criteria for these characteristics. These criteria should be defined in collaboration between business representatives, developers, and testers. They provide developers and testers with an extended vision of the feature that business representatives will validate. An Agile team considers a task finished when a set of acceptance criteria have been satisfied.
Typically, the tester’s unique perspective will improve the user story by identifying missing details or non-functional requirements. A tester can contribute by asking business representatives open-ended questions about the user story, proposing ways to test the user story and confirming the acceptance criteria.
The collaborative authorship of the user story can use techniques such as brainstorming and mind mapping. The tester may use the INVESTtechnique [INVEST]:
According to the 3C concept[Jeffries00], a user story is the conjunction of three elements:
- Card: The card is the physical media describing a user story. It identifies the requirement, its criticality, expected development and test duration, and the acceptance criteria for that story. The description has to be accurate, as it will be used in the product backlog.
- Conversation: The conversation explains how the software will be used. The conversation can be documented or verbal. Testers, having a different pointof view than developers and business representatives[ISTQB_FL_SYL], bring valuable input to the exchange of thoughts, opinions, and experiences. Conversation begins during the release-planningphase and continues when the story is scheduled.
- Confirmation: The acceptance criteria, discussed in the conversation, are used to confirm that the story is done. These acceptance criteria may span multiple user stories. Both positive and negative tests should be used to cover the criteria. During confirmation, various participants play the role of a tester. These can include developers, as well as specialists, focused on performance, security, interoperability, and other quality characteristics. To confirm a story as done, the defined acceptance criteria should be tested and shown to be satisfied.
Agile teams vary in terms of how they document user stories. Regardless of the approach taken to document user stories, documentation should be concise, sufficient, and necessary.
In Agile development, a retrospective is a meeting held at the end of each iteration to discuss what was successful, what could be improved, and how to incorporate the improvements and retain the successes in future iterations. Retrospectivescover topics such as the process, people, organizations, relationships, and tools. Regularly conducted retrospective meetings, when appropriate follow up activities occur, are critical to self-organization and continual improvement of development and testing.
Retrospectives can result in test-related improvement decisions focused on test effectiveness, test productivity, test case quality, and team satisfaction. They may also address the testability of the applications, user stories, features, or system interfaces. Root cause analysis of defects can drive testing and development improvements. In general, teams should implement only a few improvements per iteration. This allows for continuous improvement at a sustained pace.
The timing and organization of the retrospective depends on the particular Agilemethod followed. Business representatives and the team attend each retrospective as participants while the facilitator organizes and runs the meeting. In some cases, the teams may invite other participants to the meeting.
Testers should play an important role in the retrospectives. Testers are part of the team and bring their unique perspective [ISTQB_FL_SYL], Section 1.5. Testing occurs in each sprint and vitally contributes to success. All team members, testers and non-testers, can provide input on both testing and non-testing activities.
Retrospectives must occur within a professional environment characterized by mutual trust. The attributes of a successful retrospective are the same as those for any other review.
Delivery of a product increment requires reliable, working, integrated software at the end of every sprint. Continuous integration addresses this challenge by merging all changes made to the software and integrating all changed components regularly, at least once a day. Configuration management, compilation, software build, deployment, and testing are wrapped into a single, automated, repeatable process. Since developers integrate their work constantly, build constantly, and test constantly, defects in code are detected more quickly.
Following the developers’ coding, debugging, and check-in of code into a shared source code repository, a continuous integration process consists of the following automated activities:
- Static code analysis: executing static code analysis and reporting results
- Compile: compiling and linking the code, generating the executable files
- Unit test: executing the unit tests, checking code coverage and reporting test results
- Deploy: installing the build into a test environment
- Integration test: executing the integration tests and reporting results
- Report (dashboard): posting the status of all these activities to a publicly visible location or e-mailing status to the team
An automated build and test process takes place on daily basis and detects integration errors early and quickly. Continuous integrationallowsAgiletesters to run automated tests regularly, in some cases as part of the continuous integration process itself, and send quick feedback to the team on the quality of the code. These test results are visible to all team members, especially when automated reports are integrated into the process. Automated regression testing can be continuous throughout the iteration. Good automated regression tests cover as much functionality as possible, including user stories delivered in the previous iterations. Good coverage in the automated regression tests helps support building (and testing) large integrated systems. When the regression testing is automated, the Agiletestersare freed to concentrate their manual testing on new features, implemented changes, and confirmation testing of defect fixes.
In addition to automated tests, organizations using continuous integration typically use build tools to implement continuous quality control. In addition to running unit and integration tests, such tools can run additional static and dynamic tests, measure and profile performance, extract and format documentation from the source code, and facilitate manual quality assurance processes. This continuous application of quality control aims to improve the quality of the product as well as reduce the time taken to deliver it by replacing the traditional practice of applying quality control after completing all development.
Build tools can be linked to automatic deployment tools, which can fetch the appropriate build from the continuous integration or build server and deploy it into one or more development, test, staging, or even production environments. This reduces the errors and delays associated with relying on specialized staff or programmers to install releases in these environments.
Continuous integration can provide the following benefits:
- Allows earlier detection and easier root cause analysis of integration problems and conflicting changes
- Gives the development team regular feedback on whether the code is working
- Keeps the version of the software being tested within a day of the version being developed
- Reducesregression risk associated with developer code refactoring due to rapid re-testing of the codebase after each small set of changes
- Provides confidence that each day’s development work is based on a solid foundation
- Makes progress toward the completion of the product increment visible, encouraging developers and testers
- Eliminates the schedule risks associated with big-bang integration
- Provides constant availability of executable software throughout the sprint for testing, demonstration, or education purposes
- Reduces repetitive manual testing activities
- Provides quick feedback on decisions made to improve quality and tests
However, continuous integration is not without its risks and challenges:
- Continuous integration tools have to be introduced and maintained
- The continuous integration process must be defined and established
- Test automation requires additional resources and can be complex to establish
- Thorough test coverage is essential to achieve automated testing advantages
- Teams sometimes over-rely on unit tests and perform too little system and acceptance testing
Continuous integration requires the use of tools, including tools for testing, tools for automating the build process, and tools for version control.
Release and Iteration Planning
As mentioned in the Foundation Level syllabus[ISTQB_FL_SYL], planning is an on-going activity, and this is the case in Agilelifecycles as well. For Agilelifecycles, two kinds of planning occur, release planning and iteration planning.
Release planning looks ahead to the release of a product, often a few months ahead of the start of a project. Release planning defines and re-defines the product backlog, and may involve refining larger user stories into a collection of smaller stories. Release planning provides the basis for a test approach and test plan spanning all iterations. Release plans are high-level.
In release planning, business representatives establish and prioritize the user stories for the release, in collaboration with the team. Based on these user stories, project and quality risks are identified and high-level effort estimation is performed.
Testers are involved in release planning and especially add value in the following activities:
- Defining testable user stories, including acceptance criteria
- Participating in project and quality risk analyses
- Estimating testing effort associated with the user stories
- Defining the necessary test levels
- Planning the testing for the release
After release planning is done, iteration planning for the first iteration starts. Iteration planning looks ahead to the end of a single iteration and is concerned with the iteration backlog.
In iteration planning, the team selects user stories from the prioritized release backlog, elaborates the user stories, performs risk analysis for the user stories, and estimates the work needed for each user story. If a user story is too vague and attempts to clarify it have failed, the team can refuse to accept it and use the next user story based on priority. The business representatives must answer the team’s questions about each story so the team can understand what they should implement and how to test each story.
The number of stories selected is based on established team velocity and the estimated size of the selected user stories. After the contents of the iteration are finalized, the user stories are broken into tasks, which will be carried out by the appropriate team members.
Testers are involved in iteration planning and especially add value in the following activities:
- Participating in the detailed risk analysis of user stories
- Determining the testability of the user stories
- Creating acceptance tests for the user stories
- Breaking down user stories into tasks (particularly testing tasks)
- Estimating testing effort for all testing tasks
- Identifying functional and non-functional aspects of the system to be tested
- Supporting and participating in test automation at multiple levels of testing
Release plans may change as the project proceeds, including changes to individual user stories in the product backlog. These changes may be triggered by internal or external factors. Internal factors include delivery capabilities, velocity, and technical issues. External factors include the discovery of new markets and opportunities, new competitors, or business threats that may change release objectives and/or target dates. In addition, iteration plans may change during an iteration. For example, a particular user story that was considered relatively simple during estimation might prove more complex than expected.
These changes can be challenging for testers. Testers must understand the big picture of the release for test planning purposes, and they must have an adequate test basis and test oracle in each iteration for test development purposes as discussed in the Foundation Level syllabus [ISTQB_FL_SYL], Section 1.4. The required information must be available to the tester early, and yet change must be embraced according to Agile principles. This dilemma requires careful decisions about test strategies and test documentation. For more on Agile testing challenges, see [Black09], Chapter 12.
Release and iteration planning should address test planning as well as planning for development activities. Particular test-related issues to address include:
- The scope of testing, the extent of testing for those areas in scope, the test goals, and the reasons for these decisions.
- The team members who will carry out the test activities.
- The test environment and test data needed, when they are needed, and whether any additions or changes to the test environment and/or data will occur prior to or during the project.
- The timing, sequencing, dependencies, and prerequisites for the functional and non-functional test activities (e.g., how frequently to run regression tests, which features depend on other features or test data, etc.), including how the test activities relate to and depend on development activities.
- The project and quality risks to be addressed.
In addition, the larger team estimation effort should include consideration of the time and effort needed to complete the required testing activities.