JMeter Tips and Tricks

JMeter Tips and Tricks - Correlating with Regular Expressions

Anotherpost in this current theme of JMeter tips and tricks… You will hearLoadRunner consultants preaching to the converted about the requirementto correlate data accurately. What they’re referring to, is making surethat dynamic data received by the client from the server (typically ina response body, header or url), which changes from request to request,is accurately correlated. If you don’t, you will probably ‘break’ theapplication logic with most obvious signs during replay being thingslike HTTP Server 500 errors.

In practice, correlation requirements relate to things like sessionids, viewstate, cookies, date time stamps and other transactionalinformation. So more often than not you will see this data changingfollowing a post back or response from the server. With theintroduction of AJAX calls, correlation requirements probably increase(per page) as the number of transactional requests probably increasesas well.

In LoadRunner, you typically correlate data manually using theweb_reg_save_param function. This in effect is a prepared statement,placed before the request that will return the data in the response youare looking for. For example, your function will specify a left andright boundary of the data that it is looking for before you make therequest. Then the request is made and if a match is found, then thedata is saved in a parameter which you can later call as a variable insubsequent requests. LoadRunner also has a feature called CorrelationStudio which takes affect during recording of your script, using asimilar approach (left and right boundaries) and then dynamicallyswapping corresponding matches with the variabilzed data. All this isquite painful though, and whenever I’m specifying primitive left andright boundaries or using the flaky Correlation Studio (I can explainif you like), I’m left wishing I could just use regular expressions …

The trick to correlating data easily with JMeter is to first knowwhat data it is that needs correlating! (duh…) Experience will makethis easier as a lot of common web apps have common dynamic datarequirements as already mentioned. Another trick is to always record the script twice (and diff to see differences between unique user sessions), then more often than notrecord 2+ iterations of an action within a single script (and diff tosee differences between user transactions/actions). If you do thisbefore anything else, and diff the results, you will save bucketloadsof heartache in getting your script harnesses running, no matter whichtool you choose.

But back to JMeter, once you do know what data you need tocorrelate, simply create a Post Processor -> Regular ExpressionExtractor to ‘variablize’ that data. Huh? Read on for a more detailedexplanation of this.

In my following example, I have 4 dummy pages that return unique jsessionid’s for each page as follows:

I’ve done this to simulate the changing session informationtypically found in any jsp based web app. To look for a jsessionid Isimply add a new Post Processor called the Regular Expression Extractor. The following screenshot shows an example:

In the example provided, the reference name is what I can later callupon as a variable within the script, for example, ${jsessionid_trans}.The regular expression is the pattern I am looking for. Note in using aparenthesis I’m specifying a subexpression which I can get to with thematch result number of n(1). If you’re new to regex, my example isequivalent to saying look for a left boundary of jsessionid= then matchany character until you hit an ampersand. Even if you know nothingabout regex, this will probably get you out of hot water 90% of thetime. The real power in regex though is the ability to manipulate andnarrow in on search results, or search for varying patterns (liketelephone numbers, alphanumeric sequences and so on). I still find thePerl regular expression tutorials the best:
http://perldoc.perl.org/perlretut.html

So that’s it, you’ve found the jsessionid and to use it later on inyour script, you simply call on it as a variable i.e.~koops/test_jmeter/page2.html?value=${jsessionid}
I often find it is easier to open up your test plan (.jmx) in yourfavourite text editor and then use the same regular expression tosearch and replace all instances of that pattern with your newly setvariable.

However there is a twist to all this, which I refer to as scope. Youreally need to understand the scope of your regular expressionextractors and how they are applied. By scope I’m loosely referring tovariable scope in a normal programming language (which JMeter is not).I think the best way to understand the scope of your extractors, is toidentify which ‘container’ they are placed in. For a JMeter test plan,there are multiple containers you might place these extractors in:

If you’re familiar with programming languages and scoping, thenyou’ll probably identify the pecking order of the scope I’ve justmentioned, from global right down to transaction level (I’ve made up afew names there). Global variables will override local variables and soon down the chain. So you need to be careful where you place yourregular expression extractors and what reference (variable) names youuse, as these can interfere with each other. Sometimes it is useful tolimit your extractor in scope to a particular transaction, thread groupor test plan.

In my example listed above, I’ve written a test plan to explore thedifferent scopes identified. Using the debug sampler, you can see thatthe trans scope is limited to single request sampler, because it is added as a child element. This means the variable jsessionid_trans gets set only once. The localscope is limited to a transaction controller. I placed it before thesecond request sampler but not as a child to the first, to highlighthow it sets the variable jsessionid_local twice (once for each request contained in that transaction controller). The thread scope is limited to the entire thread group, so you will see the jsessionid_thread variable set three times as each page is requested. The same applies to the global scope in which case the jsessionid_globalvariable is set on each occurrence of the pattern. I’ve also added asecond thread group with its own debug sampler to highlight how onlyglobal variables are accessible by this thread group. You can run this test plan (regex_test.jmx) yourself and check out the results as shown below:

Finally, here are some example regular expressions which have worked well for me for different types of web apps:
jsessionid=(.+?)&
jsessionid=(.+?)'
id="__VIEWSTATE" value="(.+?)"
jsd=(\d+)
strEntry=(.+?)'
CdzSession' value='(.+?)'
WebiSession' value='(.+?)'
DOCTOKEN VALUE='(.+?)'
etc ...