Performance Engineering and Application Security Knowledge Sharing

Testing and Monitoring platforms provide a broad range of user simulation methods such as protocol, headless, and real browser-based. In this post, I will outline the main aspects of those followed by a comparison matrix which you could use for choosing an appropriate simulation approach.

Protocol-level Simulation

Protocol level or HTTP-based testing was very popular in the early days of our digital age. With the rise of rich web client technology the proved simulation approach has been more and more outdated.

A typical HTTP-based test driver executes service requests and parses responses. Modern web 2.0 applications consist of many client-side scripts, which are totally ignored and not measured, in this type of test execution. In worst-case complex use cases cannot be simulated on protocol level due to a shortage of client side generated ids.

Complex protocol based use cases can be difficult to implement. A performance engineer needs to deal with cookies, session ids, and other dynamic parameters. Depending on the used technology of your system under test some web form names often change once a new version have been deployed which will cause the HTTP-based script to fail.

// Sample SilkPerformer protocol level script
transaction TMain
var
hContext: number;
begin
WebPageUrl(“http://lab3/st/”, “Greetings”);
WebPageStoreContext(hContext);
WebPageLink(“Join the experience!”, ” – New Visitor”);
WebPageSubmit(“Continue”, CONTINUE001, “Main menu”);
WebPageLink(“Products”, “ShopIt – Products”);
WebPageLink(NULL, “ShopIt – Product”, 3);
WebPageSubmit(“Search”, SEARCH001, ” – Search”, 0, NULL, hContext);
end TMain;

dclform
CONTINUE001:
“name”            := “jack”,
“New-Name-Button” := “Continue”;
SEARCH001:
“search”          := “boot”;

After all, protocol level scripts are perfect for service based tests in continuous integration environments, uptime monitoring or also for stress tests.

Headless Browser Simulation

With the rise of web 2.0 technologies the testing and monitoring business was faced with serious challenges. Rich browser applications could no longer be tested or simulated on the protocol level due to the missing client side logic during script replay.  Therefore, several headless browsers have been introduced such as HtmlUnit, PhantomJS or SlimerJS. They are often built on top of WebKit, the engine behind Chrome and Safari.

Headless Browsers have all the advantages of real Browsers, and they run faster without the heavy GUI. Many test automation, monitoring, and performance testing platforms are using headless browsers as they allow realistic user simulation.

Some tool providers have built their headless browser engines and ran in maintenance pitfalls. They have to keep the pace with new browser versions. It’s highly recommended using free available headless browser kit because there is a broad community which works on improvements.

// Sample phantomjs script
“use strict”;
var page = require(‘webpage’).create(),
server = ‘http://posttestserver.com/post.php?dump’,
data = ‘universe=expanding&answer=42’;

page.open(server, ‘post’, data, function (status) {
if (status !== ‘success’) {
console.log(‘Unable to post!’);
} else {
console.log(page.content);
}
phantom.exit();
});

Finally, headless browsers are ideal for test automation, performance testing, and SLA monitoring.

Real Browser Simulation

Web2.0 applications are full with JavaScript, Flash, Ajax and CSS. Without a full browser, it’s not possible to measure the actual end-to-end response times of the whole web page. Real Browser Monitoring allows you to verify the site’s functionality and performance as perceived by the end-user.

A typical real browser monitoring solution collects loading times of images, javascript, CSS and more. Often they provide waterfall charts, which visualize the load time of those components.

One of the biggest disadvantages is their footprint. A single real browser script requires much more CPU and memory resources on the load injection or monitoring machine. Therefore real user based simulation is not recommended for stress test scenarios.

// sample SilkPerformer real browser-based script
transaction TMain
begin
BrowserStart(BROWSER_MODE_DEFAULT, 800, 600);

// navigate to the login site
BrowserNavigate(“http://demo.com/TestSite/LoginForm.html”);
// set the authentication for the secure site
BrowserSetText(“//INPUT[@name=’user’]”, “BasicAuthUser”);
BrowserSetPassword(“//INPUT[@name=’pwd’]”,     Decrypt3DES(“Ax7/X9sk1kIfHlbAZ434Pq4=”));
// submit the form
BrowserClick(“//INPUT[@value=’Submit Query’]”, BUTTON_Left);
end TMain;

After all, real browser simulation is useful for acceptance testing or SLA monitoring. Don’t use it for performance or stress testing because the resource footprint is too high.

Comparison Matrix

Obviously, there are good reasons for protocol, headless or real browser-based user simulation. The table below provides some guidance to choose the appropriate approach.

Criteria	HTTP	Headless Browser	Real Browser
Realistic User simulation	No	Yes, but with some limitations	Yes
Easy script creation	No, depends on application complexity	Yes	Yes
Robust script replay	Yes	Yes, sometimes tricky tweaks required	Yes
Easy Script maintainability	No	Yes, debugging could be tricky	Yes
Multi-Browser Support	Yes	Yes, but with some limitations	Yes
Low footprint on load injection machine	Yes	Yes	No
Good for continuous integration	Yes	No	No
Good for performance tests	Yes	Yes	No
Good for stress tests	Yes	No	No
Good for uptime monitoring	Yes	Yes	No
Good for SLA monitoring	No	Yes	Yes

Keep doing the good work and share your experience with me.