Performance Testing using Blazemeter for NYRR

Performance testing generally checks how the system performs and behaves. Performance testing examines reliability, scalability, responsiveness, stability, resource usage and speed of your software and infrastructure. Different kinds of performance tests give you different information. Before performance testing, it’s significant to determine your system’s business objectives, so you can understand if your system behaves perfectly or not according to the user’s needs.

After carrying out performance testing, you can examine different KPIs, such as errors per second, the number of virtual users, hits per second, latency, response time, and bytes per second (throughput), as well as the connection between them. With the help of such reports, you can recognize bugs, flaws, errors, and bottlenecks, and decide what needs to be done.
When should you use Performance Testing? The moment when you want to test your app and website performances plus networks, servers, databases, etc.

Performance Testing for NYRR (New York Road Runners)

New York Road Runners
New York Road Runners

ImpactQA did performance testing for New York Road Runner (NYRR), a marathon website using Blazemeter. Here is how we carried out the testing:

  • PROBLEM REPORT

We wanted to stress test a .NET application and were not sure if we wanted to test using VSTS or JMeter. VSTS has its special advantages while JMeter could easily be configured in BlazeMeter. The .NET application was to be tested for a load of 10K users and the challenge was to test it remotely. The web server was hosted in the US and the tests were being conducted from a remote location in India.

To simulate a real-time load environment we needed the client machine (load generating machine) in the US (the servers were already hosted in the US). We needed 10 machines of 8 GB RAM and 3.0 GHz processor). Acquiring such machines in a short time frame in the US and that too only 2 weeks was not only an expensive affair but also time-consuming and not worth the price.

That’s where BlazeMeter bring into play. With the help of BlazeMeter, we could easily simulate the load using the BlazeMeter as the client machine. The beauty of BlazeMeter is that it allows you to hit a server as if you are based out of that country – the screenshot below.

Blazemeter Features
Blazemeter Features

There is also yet another user-friendly feature of BlazeMeter that allowed the user to set the parameters by dragging the slider. See below.

JMeter Engines
JMeter Engines

We selected 290 threads (number of concurrent users) to load test. The duration of Load Test was 1 hour..

We also want to share our experience of scripting a .NET application using JMeter where we were faced with a unique problem while scripting – the problem with ViewState. Although this had nothing to do with us using BlazeMeter we thought it would be a good idea to put it in this blog.
The problem is if we do not correlate the ViewState variable the JMeter script may run 1st time but in subsequent run the script will create problems. The reason for this is that in the first run, the application may accept your recorded ViewState value but when you run the script next time, it will fail as it will no longer accept the previous view state value.

  • SOLUTION

Correlate the ViewState in the script if they exist in the application. Execute the following steps:

1- Find the ViewState parameter

Blazemeter Performance Testing Step 1
Blazemeter Performance Testing Step 1

2- Now see the HTTP Request Name (in our case its login)

Blazemeter Performance Testing Step 2
Blazemeter Performance Testing Step 2

3- Search for the ViewState (Search in View Result tree) in the response of the login that is just above the login in which you will find the ViewState parameter.

Blazemeter Performance Testing Step 3
Blazemeter Performance Testing Step 3

4- Extract it by using “Regular Expression Extractor” in the recorded “Login” sampler

Blazemeter Performance Testing Step 4
Blazemeter Performance Testing Step 4

5- Now use the “Reference Name” (in our case it is ViewState_Login) as a variable name.

Blazemeter Performance Testing Step 5
Blazemeter Performance Testing Step 5

Conclusion:

BlazeMeter is a great platform and delivers complete shift left testing. It has been trusted by the big giants and SMEs to deliver shift-left continuous testing at scale. It also saves time, improves coverage, accuracy and reduces complexity.

Through BlazeMeter, we have successfully run performance testing for our client NYRR and fixed all loopholes and glitches.

JMeter Distributed Testing: Step by Step

Pre-Requisites for Distributed Testing

For distributed testing using JMeter, we need to make some specific configurations both on the Master as well as the Slave machines.

Basic Requirements that both Master and the Slaves machines should have in common:

  • Same Java version
  • Same JMeter version (It’s better to copy the JMeter from one machine to the rest of all)
  • All machines should be on the same subnet

This doc will give you a brief idea about doing Distributed Remote testing using a Windows machine as the master and Linux machines as slaves.

Slave Configuration

After having a similar copy of JMeter in the slaves, follow the below steps for all the Slave machines:

1) Open jmeter-server file present in the bin directory of jmeter and uncomment the below line –

RMI_HOST_DEF=-Djava.rmi.server.hostname=xxx.xxx.xxx.xxx
Also instead of xxx.xxx.xxx.xxx, give the ip of the linux machine you have opened.

2) Give a specific rmi port no. in the below line of the same jmeter-server file –

${DIRNAME}/jmeter ${RMI_HOST_DEF} -Dserver_port=${SERVER_PORT:-2010} -s -j jmeter-server.log “$@”

This port will be further used in the master configuration, hence give different ports for all the different slave machines.

3) After making the above changes, execute JMeter-server on all the slaves.

Master Configuration 

The following changes need to be done for the master setup:

1) Open jmeter.properties present in the bin directory on the Master machine and mention all the slave ips in as remote hosts with the ports assigned in the above slave configuration –
remote_hosts=localhost: 1099, 104.239.234.143:2010
Localhost need not be mentioned if you do not want to use the Master machine to be a part of remote testing to put the load on the application.

2) Set the server port in the same jmeter.properties file 
-server_port=1099

3) After making the above changes, execute jmeter-server.bat on your Windows master machine which will be present in the bin directory.

4) Now execute jmeter.bat on Master machine for your JMeter GUI where the execution will be done and viewed.

Please note that the number of Virtual Users that will be assigned in the thread group in your JMeter GUI, an equal number of users will be generated from each of the machines you are using. For e.g.: let’s say we have one Windows as the master and two Linux machines as the slaves and the VUsers assigned is 10, then a total of 30 users will be making hits on the application under test.

Execution Methods

After doing the slave and master setup, open the scenario to be executed and execute in any of the 2 below methods:

1) Go to Run->Remote Start All. You can also Go to Run->Remote Start and select any one of the machines you wish to generate load.

2) On the top panel, we have a button for the remote start (After the test has completed, c lick on the button beside in red to remote stop all)

Exceptions and Workarounds

“Connection refused” is a common error that we encounter during remote distributed testing. For this, one should try the below checks:

1) Make sure the remote host entries and the ports assigned both on the master and slave are correct. Also, jmeter-server should be executed first followed by the jmeter-server.bat and jmeter.bat on the Master machine.

2) Verify the firewall settings on Windows and Linux machines. Disable if required. On Linux, that can be achieved by the below command: