Testing and Monitoring platforms provide a broad range of user simulation methods such as protocol, headless, and real browser-based. In this post, I will outline the main aspects of those followed by a comparison matrix which you could use for choosing an appropriate simulation approach.

Protocol-level Simulation

Protocol level or HTTP-based testing was very popular in the early days of our digital age. With the rise of rich web client technology the proved simulation approach has been more and more outdated.

A typical HTTP-based test driver executes service requests and parses responses. Modern web 2.0 applications consist of many client-side scripts, which are totally ignored and not measured, in this type of test execution. In worst-case complex use cases cannot be simulated on protocol level due to a shortage of client side generated ids.

Complex protocol based use cases can be difficult to implement. A performance engineer needs to deal with cookies, session ids, and other dynamic parameters. Depending on the used technology of your system under test some web form names often change once a new version have been deployed which will cause the HTTP-based script to fail.

// Sample SilkPerformer protocol level script
transaction TMain
var
hContext: number;
begin
WebPageUrl(“http://lab3/st/”, “Greetings”);
WebPageStoreContext(hContext);
WebPageLink(“Join the experience!”, ” – New Visitor”);
WebPageSubmit(“Continue”, CONTINUE001, “Main menu”);
WebPageLink(“Products”, “ShopIt – Products”);
WebPageLink(NULL, “ShopIt – Product”, 3);
WebPageSubmit(“Search”, SEARCH001, ” – Search”, 0, NULL, hContext);
end TMain;

dclform
CONTINUE001:
“name”            := “jack”,
“New-Name-Button” := “Continue”;
SEARCH001:
“search”          := “boot”;

After all, protocol level scripts are perfect for service based tests in continuous integration environments, uptime monitoring or also for stress tests.

Headless Browser Simulation

With the rise of web 2.0 technologies the testing and monitoring business was faced with serious challenges. Rich browser applications could no longer be tested or simulated on the protocol level due to the missing client side logic during script replay.  Therefore, several headless browsers have been introduced such as HtmlUnit, PhantomJS or SlimerJS. They are often built on top of WebKit, the engine behind Chrome and Safari.

Headless Browsers have all the advantages of real Browsers, and they run faster without the heavy GUI. Many test automation, monitoring, and performance testing platforms are using headless browsers as they allow realistic user simulation.

Some tool providers have built their headless browser engines and ran in maintenance pitfalls. They have to keep the pace with new browser versions. It’s highly recommended using free available headless browser kit because there is a broad community which works on improvements.

// Sample phantomjs script
“use strict”;
var page = require(‘webpage’).create(),
server = ‘http://posttestserver.com/post.php?dump’,
data = ‘universe=expanding&answer=42’;

page.open(server, ‘post’, data, function (status) {
if (status !== ‘success’) {
console.log(‘Unable to post!’);
} else {
console.log(page.content);
}
phantom.exit();
});

Finally, headless browsers are ideal for test automation, performance testing, and SLA monitoring.

Real Browser Simulation

Web2.0 applications are full with JavaScript, Flash, Ajax and CSS. Without a full browser, it’s not possible to measure the actual end-to-end response times of the whole web page. Real Browser Monitoring allows you to verify the site’s functionality and performance as perceived by the end-user.

A typical real browser monitoring solution collects loading times of images, javascript, CSS and more. Often they provide waterfall charts, which visualize the load time of those components.

One of the biggest disadvantages is their footprint. A single real browser script requires much more CPU and memory resources on the load injection or monitoring machine. Therefore real user based simulation is not recommended for stress test scenarios.

// sample SilkPerformer real browser-based script
transaction TMain
begin
BrowserStart(BROWSER_MODE_DEFAULT, 800, 600);

// navigate to the login site
BrowserNavigate(“http://demo.com/TestSite/LoginForm.html”);
// set the authentication for the secure site
BrowserSetText(“//INPUT[@name=’user’]”, “BasicAuthUser”);
BrowserSetPassword(“//INPUT[@name=’pwd’]”,     Decrypt3DES(“Ax7/X9sk1kIfHlbAZ434Pq4=”));
// submit the form
BrowserClick(“//INPUT[@value=’Submit Query’]”, BUTTON_Left);
end TMain;

After all, real browser simulation is useful for acceptance testing or SLA monitoring. Don’t use it for performance or stress testing because the resource footprint is too high.

Comparison Matrix

Obviously, there are good reasons for protocol, headless or real browser-based user simulation. The table below provides some guidance to choose the appropriate approach.

Criteria HTTP Headless Browser Real Browser
Realistic User simulation

 

No Yes, but with some limitations Yes
Easy script creation

 

 

No, depends on application complexity Yes Yes
Robust script replay Yes Yes, sometimes tricky tweaks required Yes
Easy Script maintainability

 

No Yes, debugging could be tricky Yes
Multi-Browser Support

 

Yes Yes, but with some limitations Yes
Low footprint on load injection machine Yes Yes No
Good for continuous integration

 

Yes No No
Good for performance tests

 

Yes Yes No
Good for

stress tests

 

Yes No No
Good for uptime monitoring

 

Yes Yes No
Good for SLA monitoring

 

No Yes Yes

Keep doing the good work and share your experience with me.

Advertisements

Posted by JM

Resourceful, solution-focused and intuitive reliability engineer with over 15 years of demonstrated success in architecting, developing and maintaining effective testing and monitoring solutions. Offers a wealth of knowledge and experience surrounding modern application architecture and development of best practices.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s