Risk based Performance Engineering
Performance test effort estimation can be tricky. Especially, in integration projects, when a high number of applications need to be tested and the time line is very limited, a better approach is required.
In one of my past projects, I led a non-functional testing stream and I had to plan performance-testing efforts for more than 70 affected applications. My testing team consisted of 8 experienced performance test experts. However, the go-live date was already in 6th month. Initially, I thought we would never meet the target timeline and we would need a very efficient performance engineering approach.
Due to the high number of affected applications and the limited testing time it was not possible to plan, implement and execute performance tests for all 70 applications. Our limited chance was a risk based performance engineering approach.
In the first place, we created a non-fnunctional-requirement questionnaire, which captured performance-requirements related topics such as response time, expected transaction volume and growth estimates. In addition, this questionnaire contained a formula, which allowed us to calculate the performance risk. We used performance risk category from 1-4, specified performance test activities for each of the categories and decided that only performance tests for applications with performance risk category 1 and 2 would be conducted.
Secondly, we sent this non-functional-requirement questionnaire to our product owner of all affected applications, and set a deadline as to when this had to be returned and also presented the idea behind this questionnaire.
Finally, we received the response from all product managers. Surprisingly, just 10 per cent of our 70 applications had a performance risk category of 1 or 2. We prepared and implemented the required performance tests for these 10 applications and were easily able to meet the tough timeline.
Risk based performance engineering has helped us to focus our efforts on performance critical applications and brought transparency to our performance engineering activities.