Software developers' productivity
The software developers' productivity is a major concern. Can they do performance testing without loss of productivity? The mBrace Agile Application Test Tool enables them to do that while increasing their productivity at the same time. How?
Performance testing one transaction takes less than 3 minutes. If the transaction passes the test they can deliver. If the transaction fails the test, it takes 3 minutes more to drill down for PIOs (Performance Improvement Opportunities). PIOs help them to optimize their Code and secure its performance potential. On average 10% to 30% of newly built transactions fail the performance test. Developers doing their performance tests themselves rapidly develop more feeling for optimizing their Code a priori. As a result their average success rate increases. When the developer has to repeat the test cycle three times while optimizing the Code, he / she has spent 18 minutes for testing until the transaction is released. This occurs only in between 10% and 30%, let's say 20% of cases. Overall 80% x 3 + 20% x 18 = 6 minutes. At the same time also scripts are recorded, which are and remain single user scripts that take much less time to handle than multiuser load test scripts. The resulting scripts can be handed over to a performance tester who combines them in a logical way into scenarios. The scenarios can be run for performance regression testing in CI / CD settings.
Due to the short feedback cycle, all the developers have done is still fresh in their minds and they will need very little effort to optimize their Code for performance. The feedback cycle of the mBrace tool is so short that it even allows developers to experiment with multiple options while developing the Code.
By contrast a mainstream load and stress test takes on average 1.5 hours scripting and more than an hour per transaction for testing. Any attempt to integrate that into the software development process would disrupt the software developers' productivity.
The Shift Left Performance Test Method has software developers create scripts for load and stress testing. The load tests are done overnight. The scripts can be rerun for performance regression testing each time new features are delivered and implemented. Creating such a script for multiuser purposes takes 1.5 hours (90 minutes) on average. What do partial Load & Stress Tests deliver? Apart from the not-very-agile feedback loop and low efficiency, what is left of the risk reduction capability? The effectiveness of the Shift Left Performance Testing leaves much room for debate.