The Best Ways to determine Load Patterns

During my career as a performance engineer, I worked in several performance testing exercises.  You can imagine that I experienced all kinds of challenges regarding estimation and simulation of realistic load patterns. There are some pitfalls which you should avoid.

Initially, you have to clarify whether you’re going to test a brand new system or if it’s already in production. The correct pattern for the latter can easily be derived, but the former is, of course, more challenging.

A brand new application

Let’s assume you are going to prepare a workload for a new business application. You have no reference system or production environment where real users are doing day-to-day activities. All that your customer told you is how many users are going to use this business application in the future. How many concurrent users will you simulate?

I use little’s law in such cases because this allows you a realistic calculation.

Little’s law: Latency = Load / Throughput

Latency: Session time of your use case or business transaction

Load: Concurrent user

Throughput: Expected number of business transactions per interval

Latency and Throughput are often part of non-functional requirements.Let’s calculate the load or concurrent user for this example:

Number of users: 1000 user will work with the new application
Latency: 30 seconds for a typical business transaction
Throughput: 600 transactions per hour
Load: 30 * (3600/600) = 180 concurrent user

Given the sample above you can schedule a performance test with 180 concurrent users. There is absolutely no need to run a test with 1000 concurrent user because those will most likely never use this application at the same time.

An already in production application

Well, you can take the real user monitoring statistics and derive the required figures such as concurrent users per interval, the number of business transactions or realistic session times. Don’t forget to account growth rates and schedule current regular, current peak and future growth test runs.

If you follow my advice, I promise that you will calculate realistic workloads in the future. I am looking forward regarding your experience with load patterns.

Advertisements

The 3 User Experience Antipatterns

The digital transformation is ongoing, and user experience will become more important than ever before. Obviously, there are many benefits of this development, but there are also flipsides. This post will outline typical user experience antipatterns.

The importance of user experience
Nowadays there are still areas in our day-to-day activities where digital services are very limited or not available. Many businesses are closing their digitalization gaps and will come up with new or improved services. The risen portfolio will lead to picky users who chooses only those with excellent user experience.

According to a study from DoubleClick, user experience has already impact on revenue. Those firms with average web page response times below 5 seconds generate 2x more commercial revenue compared to their competitors.

Naturally, websites with low latency will lead to satisfied users. Here are tips you can use right away. You’ll make your online customers happy if you avoid those three antipatterns.

1. No design for user experience
Application design is often extremely functional oriented. Design specialists create a detailed description of the new features and focus more on the visual representation than non-functional aspects. Developers bring their preferred frameworks and create a rich browser-based application. Often nobody cares about design best practices.

2. No testing for user experience
At least one test case for each requirement will be documented and executed. As there are no non-functional requirements, nobody will verify performance, scalability, usability or throughput. In best cases, a penetration test will be performed. Once all tests have been executed, and the identified defects have been solved the new App will be installed at production.

3. No monitoring of user experience
Application support teams are manually watching the log files, or in the best case, there is an automated error pattern detection in place. A system resource monitoring is available, but not used at all. Issues identified by the business user cannot be reproduced and often the ticket will be closed without any corrective action.

Keep doing the good work and kick out the mentioned antipatterns from your software development chain. How do you deal with user experience?

Mobile performance excellence

Companies are more and more investing in mobile applications because smartphones and tablets have become a fundamental element in our daily life. Those of us who use for instance apps to buy tickets agree that this is both, convenient and time preserving.

But, what should we consider during development and operation of reliable Apps?

Traditional development processes mainly focused on functional aspects. Non-functional elements were often completely ignored which resulted in unsatisfied users and miserable end to end response times. In worst cases, users did not use the slow performing and not reliable application.

Whether you are developing a mobile App from scratch or not, consider performance and usability first in your application lifecycle.

Here are tips you can use right away. You’ll make your mobile customers happy if you consider the steps below in your development chain.

  1. Simplify your mobile use cases
  2. Specify meaningful non-functional requirements (NFR)
  3. Consider performance best practices
  4. Setup performance tests to verify your NFR
  5. Check your App on different devices, connection speeds and WAN locations
  6. Setup real user experience monitoring and performance  management on test and production stages

Therefore, don’t waste money and consider NFR testing from day-1 in your mobile App development pipeline.

Lifecycle Performance Engineering

Mobile or web based applications are playing a vital role in our daily life. Checking emails, sending a message to our loved ones or purchasing something online, without reliable applications some of us would be lost nowadays.

However, responsive applications are not for free and those who tried to improve reliability by upgrading their infrastructure often just frustrated their user community. Business users, developers, and testers are focusing on the functional aspects of the new or changed application. How the new system handles those requests is often neglected.

Wise companies integrated performance engineering already to their software development process because they understood that early performance evaluation saves money and leads to satisfied business users. After their application is deployed in production, they continue with performance monitoring and capacity management activities.

Here are tips you can use right away. A typical lifecycle performance engineering approach includes the following:

  • Nonfunctional requirements
  • Performance analysis
  • Module or service based performance test
  • Component-based performance test
  • End-to-End performance test
  • Network impact performance analysis
  • Application performance monitoring
  • User experience monitoring

Be proactive; improve your performance-engineering majority to provide high reliable and responsive applications to your user community.

Application Security Antipatterns

Cybercrime is on the rise and in 3 of 4 security breaches, hackers target vulnerable applications instead of the backend infrastructures. However, this trend is surprising because the risk reduction is quite simple.

The Attack
A common application layer security breach is not too complicated. Cyber criminals often use a standard computer and operating system, and with basic IT knowledge, they can conduct highly effective cyber thefts. All over the internet or locally available applications can become a victim of an application layer attack.

Since several years open web application security project (OWASP) provides statistics on most frequently used vulnerabilities. About 2 of 5 attacks are SQL injection or cross-side scripting (XSS).  The SQL injection vulnerability, for instance, allows cyber thefts eventually to bypass the login procedure by using ‘OR 1=1–‘ instead of a valid password.

The Risk Mitigation
Application layer firewalls are essential, but they will not provide sufficient protection against application tier attacks. Even hardening of your infrastructure does not reduce the risks related to becoming a victim of such an attack. The good news is; the risk mitigation is manageable.

Firstly you should educate your developers how to implement applications which are robust against top vulnerabilities.

Secondly, you should eliminate security issues from the root. Use Enterprise Security API to filter user input and never access your database without prepared statements.

Finally, you should regularly execute secure code reviews according to security standards.

Besides this, it’s better to use standard libraries for critical areas such as encryption instead of re-inventing the wheel.

 

NFR Testing should not be an afterthought

Our world is moving quickly and therefore, short time to market is more important than ever before. Everyone who is working in the software testing business is constantly challenged with reduction of testing time and shorter release sprints.

There are some who tend to cut quality assurance to the absolute minimum, and in some cases, the customers are assessing their products. I understand this trend to some extends but we need to keep in mind that our end user cannot test non-functional requirements.

While your end user community can conduct some functional tests, non-functional requirements must be verified in your test labor as early as possible in the lifecycle. Late detected performance or security faults would be expensive, result in frustrated user, or in worst cases, in critical outages or data loss.

However, make testing of non-functional requirements, namely, security and performance testing, part of your development pipeline. Also, NFR testing will reduce risks and gives you the opportunity proactively solve issues before they affect your user community.

 

 

Myths and Reality behind Automation

New services are popping up overnight, and those who launch their products too late or in a poor quality often lose substantial market shares.

Automation is a fundamental element in our fast construction cycles. Amazon, for instance, deploys every 11.6 seconds a new software version to production. They have obviously a highly automated development chain that allows them to test, implement and launch new applications without any human interaction.

Some people argue that automation will remove jobs very soon. Based on my experience the opposite is the case. Imagine that you have automated repetitive tasks in your development process and for some reason, the build failed. Obviously, in such situations is human interaction required to analyze and solve such issue.

I believe that automation is a productivity driver because it eliminates time-consuming, repetitive tasks and brings us more time for challenging activities such as innovation or simplification. Finally, the quality of our products will increase, and we can launch more features than ever before.

If we look back to the Machine Age – Josh Seiden’s Blog – when factories replaced steam power with electric engines, the economic benefit was not the electrification. The expected productivity increase was realized after they re-oriented the machines around the flow of materials. Eventually, the same principal applies for automation.

All things considered, there is no indication that automation will eliminate jobs in the future because it won’t come overnight. However, the ball is already rolling, and soon it will be a great productivity driver.

 

 

 

A Performance Engineer’s Toolbox

According to a performance research from DoubleClick, slow loading sites often results in reduced revenue. Therefore, successful companies integrated performance engineering in their development chain. They verify non-functional requirements early in the application lifecycle and monitor performance also at production. This post will outline tools and skills required for the former and the latter.

Skills

A performance specialist is technology-savvy and has hands-on experience in a broad range of load injection, tracing and supporting tools. Based on my knowledge, the most valuable skills of non-functional testing specialists are software development, a thorough understanding of modern technologies, real collaboration, and excellent problem analysis.

Toolset

Besides, they should have hands-on experience in at least one load injection and tracing tool. The table below summarizes a standard toolset of a performance engineer.

Type Tool Details
Load injection SilkPerformer Many technologies supported
LoadRunner Many technologies supported
JMeter Limited technology support
Tracing and Monitoring dynaTrace AppMon User experience, application performance, infrastructure monitoring
AppDynamics User experience, application performance, infrastructure monitoring
Introscope Application performance, infrastructure monitoring
Fiddler Web debugging proxy
Wireshark Network protocol analyser
Silk Performance Manager Simulation of synthetic transactions
Utils PageSpeed Web design analysis
YSlow Web design analysis
Ultraedit Text Editor
Wanbridge Network simulation, delay, latency, package loss
DBVisualizer DB queries
V-Studio .Net development
Eclipse Java development
Office Reporting
Putty Access to Unix environments

With the rise of new technology, their toolset will possibly increase in the future. I recommend rethinking your performance engineering approach, methods, and toolset regluarly because innovation is the key to success and survival.