Application or system performance varies significantly depending on the architecture, application, and host environment. While it is not possible to provide a complete list of performance risks for all systems, the following list includes some typical types of risks associated with specific architectures:
Single computer systems.
These are systems or applications that run solely on a non-virtualized computer. Performance can degrade due to
- excessive resource consumption, including memory leaks, background activities such as security software, slow storage subsystems (e.g., low-speed external devices or disk fragmentation), and operating system mismanagement.
- inefficient implementation of algorithms that do not use available resources (e.g., main memory) and therefore execute slower than required.
These are systems that run on multiple servers, each of which performs a specific set of tasks, e.g., database server, application server, and presentation server. Each server is, of course, a computer and is subject to the risks already mentioned. In addition, performance can degrade due to poor or non-scalable database design, network bottlenecks, and insufficient bandwidth or capacity on a single server.
These are systems of systems, similar to a multi-tier architecture, where the different servers may change dynamically, such as an e-commerce system that accesses different inventory databases depending on the geographic location of the customer. In addition to the risks associated with multi-tier architectures, this architecture may experience performance problems due to critical workflows or data flows to, from, or through unreliable or unpredictable remote servers, especially if such servers experience periodic connectivity problems or intermittent heavy load.
These are systems in which the physical hardware hosts multiple virtual computers. These virtual machines can host single computer systems and applications as well as servers that are part of a multi-tier or distributed architecture. Performance risks specific to virtualization include excessive loading of hardware by all virtual machines or improper configuration of the host virtual machine resulting in insufficient resources.
These are systems that can scale as needed and increase in capacity as the load increases. These systems are typically distributed and virtualized multi-tier systems, albeit with self-scaling capabilities, specifically designed to mitigate some of the performance risks associated with these architectures. However, there is a risk that these features may not be properly configured during initial setup or subsequent upgrades.
These are systems that run on a client and communicate through a user interface with a single server, a multi-tier server, or a distributed server. Since code is running on the client, the risks of a single computer apply to that code, while the problems mentioned above also exist on the server side. In addition, performance risks exist due to connection speed and reliability issues, network congestion at the client connection point (e.g., public Wi-Fi), and potential issues due to firewalls, packet inspection, and server load balancing.
These are applications that run on a smartphone, tablet, or other mobile device. Such applications are subject to the risks mentioned for client-server and browser-based (web apps) applications. In addition, performance issues may arise due to limited and variable resources and connectivity on the mobile device (which may be affected by location, battery life, charge level, available memory on the device, and temperature). For applications that use device sensors or radios such as accelerometers or Bluetooth, slow data flow from these sources can cause problems. Finally, mobile applications often interact heavily with other local mobile applications and remote web services, each of which can become a bottleneck to performance efficiency.
Embedded real-time systems
These are systems that operate or even control everyday things such as cars (e.g., entertainment systems and smart braking systems), elevators, traffic signals, heating, ventilation, and air conditioning (HVAC) systems, etc. These systems often have many of the risks of mobile devices, including (increasingly) connectivity-related issues as these devices connect to the Internet. However, the reduced performance of a mobile video game does not typically pose a safety risk to the user, whereas such slowdowns in a vehicle braking system can have catastrophic consequences.
These are applications – in many cases decades-old applications – that often support mission-critical functions in a data center, sometimes via batch processing. Most are quite predictable and fast when used as they were originally designed, but many of them are now accessible via APIs, Web services, or through their database, which can lead to unexpected loads that affect the throughput of established applications.
Note that a given application or system may contain two or more of the architectures listed above, which means that all relevant risks apply to that application or system. In fact, given the Internet of Things and the explosion of mobile applications – two areas where extreme levels of interaction and connectivity are the norm – it is possible that all architectures may be present in some form in an application and thus all risks may apply.
While architecture is clearly an important technical decision that has a strong impact on performance risks, other technical decisions also influence and create risks. For example, memory leaks are more common in languages that allow direct management of heap memory, such as C and C++, and performance issues are different for relational and non-relational databases. Such decisions extend down to the design of individual functions or methods (e.g., choosing a recursive as opposed to an iterative algorithm). A tester’s ability to learn about or even influence such decisions depends on the roles and responsibilities of testers within the organization and the software development lifecycle.