Speech about Fast and Reliable Software at MF Summit on 20-21.June 2017 in Düssledorf

Come and visit the Microfocus Summit in Düsseldorf on 20.-21.June 2017. I will talk about fast and reliable software on 21.June 11:00 am. Registration, agenda and more details under this link.







Why Companies Invest in Load and Performance Tests

Over the past decade companies, large and small have started with the integration of load and performance testing into their development process. There are many good reasons for this evolution. In this post will outline why testing of performance requirements has become so popular.

Naturally, load & performance testing reduces the risk of severe outages or system failures dramatically. Even if hundreds of your software testers conduct thousands of manual tests on a new application, this is no guarantee that the system can handle the expected load pattern on production. A typical application works fine under normal load or data volume conditions. Depending on the design and implementation decisions the end-to-end response times often rise when the new system needs to handle a certain number of concurrent user requests.

Our professional user community expects high-speed applications, and with the growing service portfolio, your user sits more and more on the driver seat. The response time of a business application determines whether you hold your customers or lose commercial revenue. Optimized high-speed applications will make customers happy, and satisfied users will often return. Therefore, the spending in performance testing is an excellent investment.

Companies invest plenty money in search engine optimization and neglect an even more important aspect, the actual speed of their application. Maybe this is still not well known, but search engines rate applications with low speed or pure application design down. Obviously, it is better to improve the response time prior making investments in search engine optimizations.

Hardware sizing is often guesswork with a sad end. Oversized infrastructure is a waste of money, and a shortage of system resources can have a severe impact on user experience. Load and performance tests eliminate this try and error exercises. Simply simulate the average load on the new system and verify if system resource utilization is within the recommended boundaries is enough for an appropriate investment in IT infrastructure.

Our online business relies on image and reputation. Major players such as Amazon or Apple understood that slow loading or not responding applications lead to a negative user impression and damages their reputation as a leading actor. Continuous performance tests help to detect hotspots proactively and protect your users from this nasty touch of not reliable applications.

Organizations working in the firefighting mode understood that this endless war room sessions are frustrating for customers and also for IT staff. You can’t tune and eliminate performance hotspots on production because this kills your trustability. Users will forget failures in your product, but they will always remember that this application is slow.

Stop wasting your money and integrate performance testing into your development process.

Mastering Load and Performance Tests in 7 Steps

The best way to avoid bad user experience is to integrate load and performance testing in your development process. In this post, I will shine a light on the necessary steps involved which you can use right away in your next performance test project.

Step 1 – Performance Requirements

Business analysts specify their performance expectations for the new application. They outline actual and future growth load patterns including transaction rates, user volumes and response time expectations. Make sure that you specify speed expectations with 90th percentiles instead of average values because they lie.

Step 2 – Test Plan

Performance Engineers review nonfunctional requirements and develop an appropriate performance test plan which contains performance requirements, the scope of this test, responsibilities, test-data, environments, load patterns, relevant use cases, scheduled test runs, and success criteria. Agree on status reporting and defect tracking with test and development teams involved. Make sure that the application owner approves this test approach prior you start with the implementation.

Step 3 – Test Environment

Test teams agree about activities on the used environment. Load and performance tests will be executed in exclusive slots to avoid the negative impact of background activity. The sizing of your performance test environment can be a critical factor. Therefore, make sure that you execute your tests in a production-like sized setting.

Step 4 – Implement Test

Engineers decide on the simulation approach and develop the necessary test scripts. Depending on the technology of your application under test and the goal of your performance test you will implement protocol-level, headless-browser of real browser based scripts. Some load testing solutions provide a script recorder which allows convenient test script creation without any programming skills. The tester just executes the use case manually while the recorder captures the interactions and creates the test script. With open source tools, the script implementation often requires programming skills.

Step 5 – Configure and Execute Test

The application owner enables system resource monitoring while test staff compiles their scenarios. They setup component level tests to run on a daily basis and end-to-end test scenarios to run on the fully integrated production like environment. Don’t forget to inform dedicated teams about your test execution slot’s to avoid unexpected downtimes during your test.

Step 6 – Result Analysis

After the test execution, test staff collects all results including response times, log files and system resource metrics. Start from the end users perspective and verify if response times are within the expected boundaries. Continue with horizontal and vertical analysis to identify the performance hotspots. Based on my experience, in more than 90 % of the executed tests bottlenecks will be determined, defects need to be raised, and the root-cause analysis starts.

Step 7- Tuning

If the performance is not acceptable corrective actions must be conducted. Engineers lay out their test approach, discuss actual results with developers, infrastructure, database, network and architect teams. This virtual team decides on tuning measures, implements corrective actions and repeats the test until performance is acceptable.

That’s it. The earlier you start testing your performance requirements, the easier you can fix identified issues. Be proactive and eliminate critical hotspots before they affect your business users. Good performance – happy customers – growing revenue!