Agile Performance Testing the mBrace way (1) addition
The mBrace method for performance testing is unconventional entailing quite a paradigm shift. So before I go on with the second and third parts on Agile Performance Testing I would like to further clarify a couple of aspects / stress a few items that were addressed in feedback both on the article and in presentations I have given.
Is this real testing or just calculation?
In fact with mBrace we collect a lot of measurement data from the Application Under Test (AUT) while running it in a real environment. The model assists in analysing all these data to unveil the Performance DNA of the AUT. This approach has many advantages. Our customers particularly appreciate new insights from a multitude of application intelligence and have no problem to consider this testing.
Performance testing in three stages
Like any performance test method, this method aims at reducing the risk of poor performance, while dividing that risk in three areas:
- Risk 1. Insufficient performance potential of the application code;
- Risk 2. Insufficient capacities, hardware and software;
- Risk 3. Defects in the production environment consisting of hardware, software and parameter settings (I call this the Implementation).
In the mBrace method, Agile or Waterfall, the three risk areas are tackled step by step, each at their appropriate time. The ability to do this in three steps has considerable advantages. Each step can be done at the proper stage in the lifecycle of the AUT. Depending on the risk profile one can decide to do Step 3 or skip it if unnecessary. The first step testing and optimizing the performance potential of the code can be done fully integrated in the software development process. The delivered code has guaranteed performance. In this step only the performance potential of the code is optimized. Nothing is done to capacities and defects in hardware or software yet. That is left for the next stages. Of course in this step you could take a look and analyse the capacity need of one or a couple of transactions. This makes however more sense when the full package of functionality to be delivered is ready.
A real threat is the possibility of an application redesign. In my opinion this makes Risk 1 the most important. With this method the necessity of a redesign is discovered and solved at its earliest stage. Hence there will be no further code built on a wrong foundation. When the application is ready there will be no redesign surprise.
The second step, optimizing capacities, can be done immediately when all functionality is ready for deployment. I will treat this one in the next article.
The third step if necessary, eliminating defects in the implementation, can be done as soon as the production (like) environment is ready.
Roles of software developers and performance testers
In Stage 1 the tool provides the option to have developers do this part of the performance testing themselves, which is very efficient. No management overhead, no or little overhead for registering and handing over findings, they are solved on the spot. Of course the tool can be used for Agile or waterfall testing by performance testers too.
Scripting
This method reduces the efforts for scripting a great deal, which reduces cost a great deal too. This is an intriguing subject that I will address at a later stage.