performance testing – Performance modelling (the mBrace way)
With mainstream load and stress testing, you throw load at an application and increase it through the number of users until you can see response times decline, or until the application collapses. This is a sound method that has its merits.
Performance modelling provides an alternative method of performance testing. It has many advantages including speed, much application intelligence, a simple process and reliability.
The well-known phrase: Garbage in, garbage out stresses that the quality of the data collection protocol is crucial for feeding accurate data into the model. mBrace performance modelling includes five data collection protocols with tooling that serve various purposes. Two of them require scripting tools, the other three can be done manually. One of them is now fully automated for applying Agile performance testing during Construction, for which I will soon post another article. You collect data about the transactions of the application under test while executing these transactions in a single user fashion. These data are parsed into transaction profiles that are fed into the model. The model, specific for the application you are testing, is completely configured and populated from the data you collected. In the model you can conduct many tests in a multiuser way.
What does it feel like to do such a test in the model?
You can do various scenarios in which you alternatively “load test” the Aplication for e.g. 10,000 users. In the volume section you change the number of users to 10,000 and click the ‘recalc’ button. A few seconds later the updated model dashboard shows up and with the blink of an eye you assess the changed profiles and the new resource profiles. If 10,000 users is too much, the dashboard shows which resources are overcharged and which response times are taking too much time. Further the transaction profiles unveil the queuing times that extend the response times. From the % utilisations that the resources are charged (or seek if > 100%) you can quickly see how you have to increase capacities of the overcharged resources. You may change the number of servers in a cluster, hit the recalc button etc.
If you like to get more of the look and feel, I published a series of four articles about this phenomenon that provide the full flavour a while ago. You can download them from our site https://www.mbrace.nl/en/documentation/articles/or from CMG/MeasureIT, or the Practical Performance Analyst.
Supported scenarios include: determining the required capacities, stress testing (what is the maximum load?), analysing the impact of: distance, parallelism, software resources, changing load in a marketing campaign, other applications on a shared infrastructure, fall back scenarios, corporate mergers and analysing the stability of the performance for disturbances. While it takes hours or days with load and stress testing, it takes only minutes to do such scenarios with model based performance testing.
Load and stress testing has its merits. Performance modelling provides an alternative method with great benefits.