If you look at traditional web server load testing tools, such as Load Runner or Jmeter, you will notice a basic design pattern for recreating large volumes of web traffic. With these tools a multi threaded program opens TCP sockets and places (via API calls) HTTP requests directly on those sockets. In this way the required volume of HTTP traffic can be simulated without the overhead of running web browsers. Typically around 500 so-called ‘virtual users’ can be simulated from a single Pentium 4 PC with around 1GB of RAM. If ‘real browsers’ were to be executed in parallel on the same equipment only around 6 browsers could be simulated. For these reasons the approach of recreating HTTP traffic in multi-threaded processes, calling APIs, has remained popular for the last 15 years or so.
Although the HTTP driven approach remains popular today there has emerged another viable alternative approach to simulating large volumes of web traffic, and this approach involves running multiple ‘headless’ browsers in parallel to recreate the required web browser traffic for load testing the web server.
The term ‘headless’ browser refers to a fully functional web browser, in terms of the required APIs to communicate with the web server, but without the user interface (GUI) portion. There are a number of headless browsers freely available (e.g. HTMLUNIT and PhantomJS) and these can all be programmatically driven to interact with the web server in the same way a real browser would. Whilst these browsers do not have a GUI they are extremely useful for sending requests to a Web Server in order to parse out the results for screen capture, network monitoring or general page automation.
Selenium WebDriver is an automated ‘functional’ testing tool that can simulate user actions on a web browser such as Firefox or Internet Explorer. Selenium can also simulate actions against HTMLUNIT. Given the resource footprint (CPU and Memory) of HTMLUNIT is far less than a traditional browser multiple instances (executed as threads) of Selenium driven HTMLUNIT browsers can be executed on a single computer to drive web traffic for load testing a web server. Typically around 25 headless browsers can be driven from a Pentium 4 PC with 1GB ram. In this way around 500 ‘virtual users’ can be simulated with 25 PC’s (or virtual machines) making this technique of driving web server traffic viable for many load testing projects.
In order to run multiple instances of Selenium WebDriver using the headless technique a multi threaded Java framework is required. For this purpose either Jmeter or The Grinder open source load testing frameworks will suffice.
Although not suited for simulating extremely large volumes of web traffic the headless browser approach should be considered when it would otherwise be difficult to recreate the required traffic using HTTP drivers. Situations where the headless browser load driving approach is advantageous includes web pages that use url re-writing for session handling or where accurate traffic simulation is required. The headless browser approach will recreate web traffic that is closer to real browser traffic, than that generated by the traditional HTTP driver frameworks.