Fork me on GitHub

performance-pipeline-image

As we get closer to production our testing will get more involved. The three main areas of testing is automated test suites during CI, traditional performance load/stress testing in system test and production-like testing during staging. Let’s look at each of these to understand how we can performance test at each stage.

Continuous Integration testing.

There are a couple of approaches in which you can include performance at this stage:

System Testing & Staging

During staging, more robust performance testing can be performed as you’re now in a cloned production environment, or as close as you can be. You will want to start generating some synthesized load on your application as the resulting metrics will be closer to real-world results. First, you should establish baselines which you will then set as your target. There are a number of different ways to synthesize load on your applications to test application response time, system response time, load bearing capacities and sustained runtime behavior.

Load Testing

Load testing helps you establish your benchmark. Configure a predetermined amount of load that should be expected with the “normal” amount of traffic. Stable runtime conditions should be reached here, so be well-informed and realistic in determining what the baseline standard will be.

Stress Testing

Stress testing follows closely behind load testing. Run a higher than expected load against the system to identify how much the system can take before it fails. You’ll want to make sure there is a considerable margin here, but less obviously, the goal isn’t necessarily to reach a scalability nirvana as errors are expected. More importantly, identify the failure point to increase the predictability of your releases and establish a point of reference for when a higher amount of traffic is expected, such as during special campaigns.

Spike Testing

Spike testing lets you see the system’s response to a sharp increase in traffic. This is a good chance to see how quickly a node can spin up in your high availability cluster, or how quickly your failover cluster responds. You can test to make sure your application recovers from periods of heavy load and to see how long your application is affected after a short burst in traffic.

Soak Testing

Soak testing helps you measure the endurance of your application under load over a significant period of time. This helps to reveal issues like memory leaks and backed up queues. The goal is to make sure the throughput and response times do not decline.

To pair with the synthetic workloads, now is a good time to examine metrics that span a wider scope outside of your application, such as, heap, garbage collector activity, CPU hotspots and threads. Monitor your application to not only reveal holes in the ecosystem or application that need optimizing; when thinking in a performance-centric manner, you’ll use these metrics to track and prevent any form of performance degradation, even if minor. Keep in mind that a few extra milliseconds in response times can have a business impact.

Staging

There are many aspects to ensuring that your staging environment is equivalent to your production environment, including mirroring hardware and software. Consider replaying production load on your staging environments, if you find that synthesized load is not providing you with reliable results. Also, try to make your data as production like as you can make it, if you cannot mirror your production data exactly.

Take aways

Next Stage: Production