The process of testing an integrated hardware and software system to verify that the system meets its specified requirements.
Verification:
Confirmation by examination and provisions of objective evidence that specified requirements have been fulfilled.
Approach:
It is performed when integration testing is completed.
It is mainly a black box type testing. This testing evaluates working of the system from a user point of view, with the help of specification document. It does not require any internal knowledge of system like the design or structure of the code.
It contains functional and non-functional areas of application/product.
Focus criteria:
It mainly focuses on the following:
- External interfaces
- Multiprogramming and complex functionalities
- Security
- Recovery
- Performance
- Operator and user’s smooth interaction with the system
- Installability
- Documentation
- Usability
- Load / Stress
Why System Testing?
- It is very important to complete a full test cycle and ST is the stage where it is done.
- ST is performed in an environment which is similar to the production environment and hence stakeholders can get a good idea of the user’s reaction.
- It helps to minimize after-deployment troubleshooting and support calls.
- In this STLC stage Application Architecture and Business requirements, both are tested.
How to perform a System test?
It is basically a part of software testing and test plan should always contain specific space for this testing.
To test the system as a whole, requirements and expectations should be clear and the tester needs to understand the real-time usage of application too.
Also, most used third-party tools, version of OSes, flavors and architecture of OSes can affect the system’s functionality, performance, security, recoverability or Installability.
Therefore, while testing system a clear picture of how the application is going to be used and what kind of issues it can face in real time can be helpful. In addition to that, a requirements document is as important as understanding the application.
Clear and updated requirements document can save tester from a number of misunderstandings, assumptions, and questions.
In short, a pointed and crisp requirement document with the latest updates along with an understanding of real-time application usage can make ST more fruitful.
Example Test Scenarios
System testing sample test scenarios for an eCommerce Site:
- If the site launches properly with all the relevant pages, features, and logo
- If the user can register/login to the site
- If the user can see products available, can add products to his cart can do payment and can get confirmation via e-mail or SMS or call.
- If the major functionality like searching, filtering, sorting, adding, changing, wishlist etc work as expected
- If a number of users (defined as in requirement document) can access the site simultaneously
- If the site launches properly in all major browsers and their latest versions
- If the transactions are being done on the site via a specific user are secure enough
- If the site launches properly on all the supported platforms like Windows, Linux, Mobile etc.
- If the user manual/guide return policy, privacy policy and terms of using the site are available as a separate document and useful to any newbie or first time user.
- If the content of pages is properly aligned, well managed and without spelling mistakes.
- If session timeout is implemented and working as expected
- If a user is satisfied after using the site or in other words user does not find it difficult to use the site.
Types of System Test:
ST is called a superset of all types of testing as all the major types of testing are covered in it. Although a focus on types of testing may vary on the basis of product, organization processes timeline and requirements.
The overall it can be defined as below:
Installation Testing:
To make sure that product/software can be installed on specific or support defined system can be configured and can be brought into an operational mode.
Functionality Testing:
To make sure that functionality of a product is working as per the requirements defined, within the capabilities of the system,
Recoverability Testing:
To make sure how well the system recovers from various input errors and other failure situations.
Interoperability Testing:
To make sure whether the system can operate well with third-party products or not.
Performance Testing:
To make sure the system’s performance under the various condition, in terms of performance characteristics.
Scalability Testing:
To make sure the system’s scaling abilities in various terms like user scaling, geographic scaling, and resource scaling.
Reliability Testing:
To make sure the system can be operated for a longer duration without developing failures.
Regression Testing:
To make sure the system’s stability as it passes through the integration of different subsystems and maintenance tasks.
Documentation Testing:
To make sure that the system’s user guide and other help topics documents are correct and usable.
Security Testing:
To make sure that the system does not allow unauthorized access to data and resources.
Usability Testing:
To make sure that the system is easy to use, learn and operate.
Tips for fruitful System testing:
- Replicate real-time scenarios rather than doing ideal testing as the system is going to be used by an end user and not by the trained tester.
- Verify the system’s response in various terms as the human does not like to wait or to see wrong data.
- Install and configure the system as per the documentation because that is what end user is going to do.
- Involving people from different areas like business analysts, developers, testers, and customers can end in a better system.
- Regular testing is the only way to make sure that littlest change in the code to fix the bug has not inserted another critical bug into the system.
Entry Criteria:
- Software development -> completed
- All modules are integrated fully and integration is tested
- Unit testing -> completed
- Production like test environment is ready
What is System Integration Testing?
It is a type of testing in which system’s ability to maintain data integrity and operation in coordination with other systems in the same environment, is being checked.
Example of System Integration Testing:
Let’s take the example of a well-known online ticket booking site – http://irctc.co.in.
This is a ticket booking facility; an online shopping facility interacts with PayPal. Overall you can consider it as A*B*C=R.
Now on the system level, online ticket booking facility, online shopping facility, and online payment option facility can be system tested independently, followed by check perform Integration tests for each of them. And then the whole system needs to be system tested.
So where does System Integration testing come into the picture?
- The web portal http://Irctc.co.in is a combination of systems. You may perform tests at the same level (single system, the system of systems), but at each level, you may want to focus on different risks (integration problems, independent functionality).
- While testing the Online Ticket booking facility, you may verify if you are able to book tickets online. You may also consider integration problems e.g. Ticket booking facility integrates back-end with front-end (UI). For example, how front-end behaves when the database server is slow to respond?
- Testing of Online ticket booking facility with online shopping facility. You may verify that the online shopping facility is available for the users logged into the system to book tickets online. You may also consider verification of integration in the online shopping facility. For example, if the user is able to select and buy a product without a hassle.
- Testing of online ticket booking facility’s integration with PayPal. You may verify whether, after booking tickets, money was transferred from your PayPal account to Online Ticket Booking account. You may also consider verification of integration in PayPal. For example, what if the system puts two entries in a database after debiting money for once only?
Difference between System Testing and System Integration Testing:
The main difference is –
- System Testing looks after a single system’s integrity with relevant environment
- System Integration Testing looks after multiple systems’ integrity with each other, being in the same environment.
Thus, ST is the beginning of real testing where you test a product as a whole and not a module/feature.