Performance Testing using Blazemeter for NYRR

Performance testing generally checks how the system performs and behaves. Performance testing examines reliability, scalability, responsiveness, stability, resource usage and speed of your software and infrastructure. Different kinds of performance tests give you different information. Before performance testing, it’s significant to determine your system’s business objectives, so you can understand if your system behaves perfectly or not according to the user’s needs.

After carrying out performance testing, you can examine different KPIs, such as errors per second, the number of virtual users, hits per second, latency, response time, and bytes per second (throughput), as well as the connection between them. With the help of such reports, you can recognize bugs, flaws, errors, and bottlenecks, and decide what needs to be done.
When should you use Performance Testing? The moment when you want to test your app and website performances plus networks, servers, databases, etc.

Performance Testing for NYRR (New York Road Runners)

New York Road Runners
New York Road Runners

ImpactQA did performance testing for New York Road Runner (NYRR), a marathon website using Blazemeter. Here is how we carried out the testing:

  • PROBLEM REPORT

We wanted to stress test a .NET application and were not sure if we wanted to test using VSTS or JMeter. VSTS has its special advantages while JMeter could easily be configured in BlazeMeter. The .NET application was to be tested for a load of 10K users and the challenge was to test it remotely. The web server was hosted in the US and the tests were being conducted from a remote location in India.

To simulate a real-time load environment we needed the client machine (load generating machine) in the US (the servers were already hosted in the US). We needed 10 machines of 8 GB RAM and 3.0 GHz processor). Acquiring such machines in a short time frame in the US and that too only 2 weeks was not only an expensive affair but also time-consuming and not worth the price.

That’s where BlazeMeter bring into play. With the help of BlazeMeter, we could easily simulate the load using the BlazeMeter as the client machine. The beauty of BlazeMeter is that it allows you to hit a server as if you are based out of that country – the screenshot below.

Blazemeter Features
Blazemeter Features

There is also yet another user-friendly feature of BlazeMeter that allowed the user to set the parameters by dragging the slider. See below.

JMeter Engines
JMeter Engines

We selected 290 threads (number of concurrent users) to load test. The duration of Load Test was 1 hour..

We also want to share our experience of scripting a .NET application using JMeter where we were faced with a unique problem while scripting – the problem with ViewState. Although this had nothing to do with us using BlazeMeter we thought it would be a good idea to put it in this blog.
The problem is if we do not correlate the ViewState variable the JMeter script may run 1st time but in subsequent run the script will create problems. The reason for this is that in the first run, the application may accept your recorded ViewState value but when you run the script next time, it will fail as it will no longer accept the previous view state value.

  • SOLUTION

Correlate the ViewState in the script if they exist in the application. Execute the following steps:

1- Find the ViewState parameter

Blazemeter Performance Testing Step 1
Blazemeter Performance Testing Step 1

2- Now see the HTTP Request Name (in our case its login)

Blazemeter Performance Testing Step 2
Blazemeter Performance Testing Step 2

3- Search for the ViewState (Search in View Result tree) in the response of the login that is just above the login in which you will find the ViewState parameter.

Blazemeter Performance Testing Step 3
Blazemeter Performance Testing Step 3

4- Extract it by using “Regular Expression Extractor” in the recorded “Login” sampler

Blazemeter Performance Testing Step 4
Blazemeter Performance Testing Step 4

5- Now use the “Reference Name” (in our case it is ViewState_Login) as a variable name.

Blazemeter Performance Testing Step 5
Blazemeter Performance Testing Step 5

Conclusion:

BlazeMeter is a great platform and delivers complete shift left testing. It has been trusted by the big giants and SMEs to deliver shift-left continuous testing at scale. It also saves time, improves coverage, accuracy and reduces complexity.

Through BlazeMeter, we have successfully run performance testing for our client NYRR and fixed all loopholes and glitches.

Increasing Importance of Quality Engineering in Software Testing

How can a company win? One of the key criteria is to ensure good quality of its products and services. But the traditional testing and QC paradigm is not enough in the context of emerging technologies. It has proven to be inefficient: if some shortcomings are revealed, the product may have to be redesigned, requiring additional expenses and extra time. That is why something new is being executed in business — quality engineering solutions. Quality Engineering (QE) is the series of procedures by which software quality is analyzed and improved throughout the application or software development lifecycle. It differs from traditional Quality Assurance in that it prevents defects as well as discovers them.

The QE approach implies that every single stage of the product/ software development cycle is under a scrupulous test of quality engineers. Furthermore, the quality maintenance is offered long after the product is delivered. The execution o such strategy in manufacturing or software development procedures guarantees the sufficiency of the output from the very start reduces imperfections, flaws, and reduces potential losses. In other words, quality engineering is the analysis, development, management, and maintenance of diverse systems compliant with high standards.

What are the rewards of Quality Engineering?

With Quality Engineering, the core benefit for your application development cycle is that you are actually making all the proposed advantages of DevOps and Agile more real. Also the teamwork between developers and testers is more real, more in line with the agile ethos. It is also integrated with Test Management solutions so that the outcomes appear on the dashboard instantly, without a human trigger. With shortened release cycles, time to ensure Quality also reduces considerably. Testers have to be involved at the start of the cycle as they will be setting up the testing environment and framework which will be relied upon for all future sprints. Done right, Quality Engineering offers a great deal more speed in testing. It mainly relies more on Test Automation than manual testing. It is hard to imagine a Quality Engineering function that doesn’t have Test Automation at its center. Yet again, done right, it creates more flexibility and speed for the whole development cycle. It is not considered just functional and non-functional testing, but every single layer and integration that can and should be tested.

In current Digital era, a Quality Engineer should have experience in programming and be supposed to be able to write software as the situations demands. While the Software Development team focuses on constantly upgrading the application, the Quality Engineering team main responsibilities are:

  • Setting up new parameters and standards
  • Optimization of test cases, & improving automation efficiency
  • Identification of drawbacks
  • Generating a plan for improvement
  • Plan execution using different tools and methods
  • Assessing & implementing new technologies and tools
  • Following up to make sure that issues have been solved
  • Creating tailor automation solutions to address application specific use cases
  • Create frameworks & accelerators that help scale QE across manifold channels, Enterprise wide.

Quality engineering is driven by emerging technologies like AI (artificial intelligence), Big Data analytics and IoT. Automation is the driving force behind turning the traditional testing into an effectual quality support model.

Bottom line

Performance of the application/ software is of paramount importance. Every outage, crash, drawbacks, and even slowing down of the app or processing/ working on a client request has the potential to impact revenue directly. It is the responsibility of QE team to not only identify such issues, but also work on identifying/removing the root cause of such problems. This demands a sound understanding of app architecture, monitoring tools, several enterprise sub systems that are catering to the app etc. Overall, Quality Engineering team provides substantial insights about the root cause or issue and solved it in the fastest possible manner.

Growing Demand from Performance Testing to Engineering

In the modern era, companies have been investing enormously in building next-gen products and platforms using cutting-edge technologies. Yet, there are the plethora of companies leave the assurance for scalability for quite late in the development cycle or do not follow the testing path enough. This in future lead to disruption in services and later can profoundly impact the customer loyalty, brand image, and ROI (or revenue). If the specific website is undergoing slow or poor performance, customers will have no choice but to explore alternating channels. As a better website experience is like a worthy brand, once gone astray, it becomes tricky to retain the customers. Considering the significance of speed-to-market for the success of the project, businesses need the appropriate processes, tools, and skills for an agile delivery. This is where most advantageous performance engineering and testing comes into the picture and assist your enterprise to think out of the box and stay ahead of the curve.

Why Implementing a robust strategy crucial?

Performance engineering, though, is a wide set of processes, and it is also an art based on years of scrutiny and observation that have led to proven practices. Putting into practice a robust strategy has been a cognizant and strategic decision for all types of enterprises operating across varied businesses. Today, validating and ensuring the responsiveness, speed, and stability of an application is absolutely business critical. The range of facets of Performance Testing has been adopted for the same, confirming that the app does not vacillate under unexpected conditions. Though, there has been a shift in conceptual thinking, where the awareness has shifted more from Performance Testing towards Performance Engineering. It critically refers to the techniques that are being applied in the application development lifecycle, which make sure the non-functional necessities. Some of the key necessities are ensuring usage, throughput, and latency of the memory. These features are required to confirm and to judge that the systems are secure, precise, user-friendly, and scalable over even in the long run. As recommended, it is a significantly demanding practice for Agile and DevOps teams to validate the presentation and efficiency of the applications. The careful application of the ideology of performance engineering makes it possible for businesses to support employees, please customers, and boost returns, all at the same time.

The growing recognition is because of the growing complications with new-age apps and the emerging technologies that are creating these manifold layers. Most of the apps these days engage with multiple third-party vendors and parties to drive innovation as well as growth for the customers. For this reason, it is no more a single action driven growth, it just getting increasingly complicated.

Some of the supplementary benefits are given below:

  • Reduced system & hardware expenses
  • Early detection of bugs and application defects
  • Guaranteed customer satisfaction
  • Enhanced revenue & profits with higher conversions
  • Lesser cost of change related to performance tuning
  • Improved experience & quality from a user’s perspective

Within the Agile process, the methodologies for Performance Engineering can be effectively aligned with the ‘shift-left’ approach. It helps to determine the issues way ahead in the product development process. In this fashion, bottlenecks can be identified and reasons can also be uncovered too. Furthermore, the overall system performance can be optimized in this procedure. The well-thought-out Engineering & Test strategy can enable teams to deal with all sorts of the critical challenges that are being posed by inefficiently performing applications. It aids in benchmarking the app performance and eventually assessing against business critical scenarios for efficient test. The digital sphere has countless aspects that keep challenging the app’s flawless functioning and predictable parameters.

ImpactQA’s comprehensive Engineering Solutions is consistent in mounting business revenues and dropping costs. The experts help in benchmarking the app presentation and helps you recognize each and every business-critical scenarios for tests.