Performance Testing, Part 4: Integration Cycle Performance Testing

This is the final post in a four-part series that will walk you through everything you need to think about when performance testing a new site.

At the end of the development cycle, we want to start testing in the integration or staging environment. This will involve escalating the code to an environment that roughly approximates (or for that matter is) the production environment.

For this set of performance tests we will want to switch to rendering performance tests. These tests will not only stress test the integration environment, but will also fully render the pages in actual browsers. The point of these tests is to determine if there are any bottlenecks in the production environment as well as whether there are any performance issues with pulling in the content necessary for completing the page.

It is very possible to have a system that performs well in a set of server response tests fail page rendering tests because the collateral necessary to render the page is not cached correctly or there are other problems in the production cluster configuration.

At this point we will also want to test access to the production cluster from clients in geographically disparate locations.

In addition to the node configuration being tested in the server response tests, we will want to add additional elements to the configuration to get an accurate view as to the actual concurrent load the system can sustain. Primary to these elements would be the load balancer, to distribute load to all the publisher nodes in the cluster, and an external cache such as Akamai or Limelight to forward post assets and rendered pages.

At this point it would also be a good idea to create a process to “warm up” the caches, prior to the actual tests. This will lessen the load on the servers are startup.

We recommend Apica for this phase of testing. Apica is a SAAS-based performance testing tool, which has testing nodes located in geographically disperse locations. It also tests pages in a full set of browsers, giving accurate measures of load times.

Apica uses a pay-as-you-go model, which allows the user to pay per test. There is also an unlimited test option.

The test bed infrastructure developed in the server response test is still useful at this point, however if there is a set of production level data this should be layered in over the course of the testing.

New scripts will have to be written to simulate usage patterns, as the scripts developed for the server response tests will not translate. However the same patterns of usage can be used.

Keep in mind that as with the server response tests, there will be cycles of tests as the feedback from the tests are internalized and the server code and configuration are changed.


Check out the rest of the series: