UI / UX – Performing A/B (Split) Testing

Websites and Web applications have become progressively more complex as our industry’s technologies and methodologies advance. What used to be a one-way static medium has evolved into a very rich and interactive experience. Additionally, users have been accessing websites in an increasing number of ways: mobile devices, a vast landscape of browsers, different types of Internet connections, carriers, locations etc.
But regardless of how much has changed in the application development process, a website’s success still hinges on just one thing: how users perceive it. “Does this website give me value? Is it easy and pleasant to use? ” These are some of the questions that run through the minds of visitors as they interact with your products, and they form the basis of their decisions on whether to become regular users of your website or to exit and never come back.
They say a newspaper’s most important story of the day goes always goes above the fold on page 1, on the right-most column and that layout has stayed the same for centuries because of the simple fact that readers’ eyes would go there first, but the rules for laying out a webpage are not as well-established as the rules for laying out a newspaper. Customers’ eyes can be directed away from your content with a less-than-optimal layout. Web visitors also have shorter attention spans – for instance, nearly half of web users expect pages to load in two seconds or less. So you have precious few seconds to make a lasting impression.
While your designers try their best to provide a great UI/UX to their customers, it is imperative that as a product manager you bring in the UX Testing into play right from the very beginning of your products / web application development.
So how does a QA person look at the UI/UX of a website? Some of the common questions they can look out for are:

  • What is the first area a visitor/user focuses on as he/she enters/opens the website?
  • Most commonly looked out or searched feature?
  • Most accessed and time spent feature?
  • How is the user trying to obtain results/objectives, pattern categories of navigation?
  • Satisfied user versus a foreclosure of the site (reasons for a user to abandon the activity on the site prior to finishing it)?
  • How is help accessed and at what level of navigation?
  • Distraction points on the site/app?
  • Check for Accessibility of site/app i.e. site load time should be reasonable, adequate text-to-background contrast, images have appropriate Alt tags etc.
  • Check for the Identity. e.g. logo is placed prominently, clear path to contact page
  • Proper Navigation. i.e. logo linked to home page, number of buttons/links is reasonable; navigation labels are clear and concise.
  • Focus on Content. For e.g. Major heads are clear and descriptive, style and color is consistent.

Another common option for testers is to conduct the A/B testing. A/B testing (a.k.a. split testing) compares different versions of a page, and it can be conducted with any of several programs. Basically, A/B testing software splits a website’s traffic into two equal segments. One group sees version A, and the other group sees version B. Statistics such as conversion rate and bounce rate are tracked for each version. Split testing determines which version is better based on these statistics. One of the most popular applications for A/B testing is Google’s Website Optimizer.
The image below shows a snapshot of  A/B testing performed on a site.


In May 2012 ImpactQA performed the functional testing of RoyalK9 and the analytics overview can be seen below. Contact us for a complete usability testing report.


A/B Testing is a widely used approach for UI/UX testing. No wonder Bill Gates was able to appreciate the use of A/B testing much before the User Experience became a priority for software development firms.


There are other tools (open-source and paid) available that can help track the traffic flow and bounce rate.

The author of this article, Jyoti Prasad Bhatt is Founder and CEO of ImpactQA