Scripting
Scripting is what makes load and stress testing complicated and expensive. It is done through record and playback. The transactions are executed single user while being recorded. For load testing they cannot immediately be replayed in multiuser fashion. They first have to be adapted. The recorded transactions contain variables specific for the single user, which have to be replaced by proper variables. This step in the process is called correlating. It is a labour intensive process.
By contrast the mBrace Test Tooling collects measurement data while executing the transactions in single user mode. Subsequently load and stress testing is mimicked in the model by simulation. Do we need to replay the transactions? Yes indeed and this is done by replaying the transactions again in single user mode. Do we need a script for that? Yes, but it is only a single user script that does not need correlating.
In the mBrace Test Methodology you replay transactions for regression testing in CI/CD settings. The scripts can be automatically replayed in the commonly used nightly run by Jenkins if so desired. mBrace provides a very effective and elegant regression test method. This shows not only if a transaction fails the test, but provides all insights needed on the anomalies that caused it.
The mBrace Method makes use of multiuser scripting in Stage 3 through a conventional LST based endurance test. The efforts for scripting this endurance test are reduced a great deal. This is made possible by the fact that the endurance test is not needed for performance evaluation but purely for defects finding.
- Hence this leaves all the room for skipping the endurance test when the risk of defects is low enough. E.g. adding new features to an Implementation that runs already stable for months, bears hardly any risk.
- Further defects finding does not require all of the application to be scripted. A selection of its transactions is sufficient.
Our friends frequently challenge the average figure of 1.5 hours effort per scripted transaction we use. Allegidly this should be much too high, though I haven't yet found a party willing to contract this fixed price. What turns out again and again is that our friends mix up end user transactions and web requests (and they are not the only ones, this concept seems extremely difficult). Again: an end user transaction consists of one or more web requests. Hence if the transactions of an application for instance have 6 web requests per transaction on average, than your scripting poductivity will be 15 minutes per web request, not per transaction!