Performance Testing using Blazemeter for NYRR

Performance testing generally checks how the system performs and behaves. Performance testing examines reliability, scalability, responsiveness, stability, resource usage and speed of your software and infrastructure. Different kinds of performance tests give you different information. Before performance testing, it’s significant to determine your system’s business objectives, so you can understand if your system behaves perfectly or not according to the user’s needs.

After carrying out performance testing, you can examine different KPIs, such as errors per second, the number of virtual users, hits per second, latency, response time, and bytes per second (throughput), as well as the connection between them. With the help of such reports, you can recognize bugs, flaws, errors, and bottlenecks, and decide what needs to be done.
When should you use Performance Testing? The moment when you want to test your app and website performances plus networks, servers, databases, etc.

Performance Testing for NYRR (New York Road Runners)

New York Road Runners
New York Road Runners

ImpactQA did performance testing for New York Road Runner (NYRR), a marathon website using Blazemeter. Here is how we carried out the testing:

  • PROBLEM REPORT

We wanted to stress test a .NET application and were not sure if we wanted to test using VSTS or JMeter. VSTS has its special advantages while JMeter could easily be configured in BlazeMeter. The .NET application was to be tested for a load of 10K users and the challenge was to test it remotely. The web server was hosted in the US and the tests were being conducted from a remote location in India.

To simulate a real-time load environment we needed the client machine (load generating machine) in the US (the servers were already hosted in the US). We needed 10 machines of 8 GB RAM and 3.0 GHz processor). Acquiring such machines in a short time frame in the US and that too only 2 weeks was not only an expensive affair but also time-consuming and not worth the price.

That’s where BlazeMeter bring into play. With the help of BlazeMeter, we could easily simulate the load using the BlazeMeter as the client machine. The beauty of BlazeMeter is that it allows you to hit a server as if you are based out of that country – the screenshot below.

Blazemeter Features
Blazemeter Features

There is also yet another user-friendly feature of BlazeMeter that allowed the user to set the parameters by dragging the slider. See below.

JMeter Engines
JMeter Engines

We selected 290 threads (number of concurrent users) to load test. The duration of Load Test was 1 hour..

We also want to share our experience of scripting a .NET application using JMeter where we were faced with a unique problem while scripting – the problem with ViewState. Although this had nothing to do with us using BlazeMeter we thought it would be a good idea to put it in this blog.
The problem is if we do not correlate the ViewState variable the JMeter script may run 1st time but in subsequent run the script will create problems. The reason for this is that in the first run, the application may accept your recorded ViewState value but when you run the script next time, it will fail as it will no longer accept the previous view state value.

  • SOLUTION

Correlate the ViewState in the script if they exist in the application. Execute the following steps:

1- Find the ViewState parameter

Blazemeter Performance Testing Step 1
Blazemeter Performance Testing Step 1

2- Now see the HTTP Request Name (in our case its login)

Blazemeter Performance Testing Step 2
Blazemeter Performance Testing Step 2

3- Search for the ViewState (Search in View Result tree) in the response of the login that is just above the login in which you will find the ViewState parameter.

Blazemeter Performance Testing Step 3
Blazemeter Performance Testing Step 3

4- Extract it by using “Regular Expression Extractor” in the recorded “Login” sampler

Blazemeter Performance Testing Step 4
Blazemeter Performance Testing Step 4

5- Now use the “Reference Name” (in our case it is ViewState_Login) as a variable name.

Blazemeter Performance Testing Step 5
Blazemeter Performance Testing Step 5

Conclusion:

BlazeMeter is a great platform and delivers complete shift left testing. It has been trusted by the big giants and SMEs to deliver shift-left continuous testing at scale. It also saves time, improves coverage, accuracy and reduces complexity.

Through BlazeMeter, we have successfully run performance testing for our client NYRR and fixed all loopholes and glitches.

5 Core Challenges Associated with Performance Testing

With the growth and demand of Mobile applications, there is growing competition in the marketplace and performance has a great impact on the app’s reliability and dependability. Therefore, there is a high requirement for performance and load tests with suitable blueprint and execution. In general, performance testing validates the scalability, stability, and speed of the applications. This is significant for providing an ‘Amazing’ and ‘Ultimate’ experience to the users.

Key Purpose of Performance Testing:

To ensure the stability, speed, and accuracy of the software.

The Performance test is critical to authenticate the scalability, reliability, and responsiveness features of the software under test.

In software testing, the rapidity and response time of the specific application is one of the vital attributes. The performance testing uncovers the performance hiccups and bottlenecks to uphold the attention and interest of the customer. Such type of Testing falls into several categories like Stress Testing, Load Testing, and Spike Testing to mention a few.

The most common performance hiccups and glitches observed in any web/ mobile app are as follows:

• Poor response time

• Memory utilization

• Disk usage

• Poor scalability

• CPU utilization

• Loading Bottleneck

• Extended Loading time

• Software configuration problems for the Web server,

• Operating System limitations

• Load balancers, databases, etc.

• Poor network configuration

5 Key Challenges of Performance Testing:

A software application should be thoroughly tested with accurate testing processes before releasing to the customers. Like any other testing procedures, this type of testing also comes with a number of Challenges.

1- Testing Tools and Situation

Performance testing requires you to test the application under reasonable scenarios. However, lots of times the enterprises do not have the budget or environment for executing the tests. One of the most efficient ways to conduct performance testing is to generate a replica of the production environment. Yet, the challenge lies in getting the proper outcome with limited resource and uneven scenarios. For example, creating a scenario where multiple users log into the system at the same time is hard.

There is countless consideration before moving further on with such type of testing. They comprise:

• Technology

• Protocols

• Browser

• Tool

• Platform

• Budget

• Hardware Requirements

2- Test Coverage

It necessitates vast knowledge and talent to cover all the functionalities when doing performance testing. But, the scenarios must be exhibitive of a range of parameters. One can try automating core functionalities by gathering various scenarios. User data must be predicted properly as there would be lots of users using the system in their own context.

3- Performance Test Analysis

Evaluating and examining the performance test outcomes is the most challenging task in Performance and load of testing. It requires you to have comprehensive knowledge and better decision to examine tools and reports. In addition, you need to by and large upgrade the tests based on the definite situation.

4- Non-Functional Requirements

Both functional and non-functional requirements are vital for such kind of test. Functional needs are more precise and include input data types, algorithms, and functionality to be covered. The challenge lies in discovering less specific non-functional requirements. They include capacity, usability, stability, responsiveness, and interoperability.

5- Analyzing the Performance Test Outcomes

This is indeed one of the significant and trickiest of the challenges for Performance testing. A good amount of judgment and thorough knowledge of the system is necessary to examine the several performance result reports besides the tools used for the actual tests. A skilled tester should be able to judge the scenarios and if the test is done or not. The tester should be able to continuously refine the tests and keep adding tests consistent with the situations estimated during the course of time.