Read time: 6 minutes.

Beginner’s Guide to A/B Testing –  How You Can Get Started


A/B testing is one of those digital marketing buzzwords you’ve probably heard if you’ve looked into marketing your business in a more serious way. It sounds a little intimidating, doesn’t it? Unless you’re a very data-driven person who loves a good spreadsheet, the idea of running something like an A/B test can be daunting.

It’s a step beyond just setting up campaigns and hoping for the best, offering concrete evidence on what works and what doesn’t. This makes it an extremely powerful tool but one you might think you’re not equipped to use effectively.

However, various inexpensive, even free tools can help just about anyone to set up and run A/B tests that help you optimise your website so that it offers the best possible experience to your website visitors and the people interacting with your ads and other marketing materials. In today’s blog post, we’ll go over what exactly A/B testing is and recommend some of the best tools out there for designing and running them.


What is A/B testing?

The number of visitors to a webpage is the number of opportunities a business has to convert those visitors into customers. In order to achieve it, you want users to take specific actions (also known as conversions) on your website. The percentage of your website visitors who complete this specific action  is known as the “conversion rate”. Conversion rate could be almost anything, like requesting a call with you or signing up to your newsletter.

The more optimised a webpage is, the higher chance a user will convert into a customer. One of the best ways to optimise a webpage is through A/B testing.

A/B testing is a technique used to compare two versions of a webpage against each other to determine which of them performs better – aka has a better conversion rate. When you perform an A/B test, two variations of a single webpage are shown to your website visitors at random. Then, statistical analysis is used to establish which variation performs better for your conversion goal.

Executing an A/B test that compares a new variation of your webpage – one with new copy, images or calls to action for example, against what’s currently shown to all visitors will help you to ask focused questions about what makes people convert. A/B testing aims to take guessing out of the website optimisation framework, and enables you to make data-informed decisions. We should all shift from saying “we think” to “we know” – and A/B testing allows us to do just that.



How A/B testing works

A/B testing works by taking an original asset – a webpage or something like a marketing email or a PPC ad –  and modifying it to create a second version of that asset. The change made can be as simple as a single headline, button, or image. After a new variation of the original asset has been made, the user traffic going into that webpage, email or ad is split evenly between the two assets. Half of the traffic is shown the original asset (known as the control) and half are shown the modified version (the variation).

As users are served either the control or variation, you can collect, measure and analyse user engagement and experience in Google Analytics. After the test has been running for the desired period (you can find a free duration calculator here), you’ll be able to determine if the changes you made to the original asset had a positive, negative, or no effect on user behaviour. Here’s a free tool to help you find out if your test results are statistically significant.


Why you should A/B test

A/B testing allows your business to make careful changes to user experiences based on facts, not feelings. This allows you to learn how certain changes have an effect on user behaviour. Additionally, A/B testing can highlight why your opinion about the best experience for a given conversion could be wrong. More than just answering a simple either-or question, A/B testing can be used to continually develop your user experience, aiming to improve conversion rate over time.


Example of an A/B test

A B2B company wants to increase the amount of quality leads from their PPC campaign landing page. The company wants to A/B test changes to multiple elements on their landing page: the headline, imagery and CTA (call to action).

The company creates a variable of each of these elements, but only tests one of them against the existing element at a time. This helps them identify which change they made had an effect on user behaviour, and which of the changes had no effect.

If they tested all changes at the same time, it would be impossible to tell which element caused the change in conversion rate – if there was one. After testing all the elements individually, the company combines the changes that improved their conversion rate to demonstrate the improvements of the new landing page over the old one. This way, they’re able to present a snapshot of how all the improvements put together can change their conversion rate.


Top A/B testing tools

To get better business performance from your digital assets, you must adopt a “test and learn” approach to pave your way to success. Because sometimes, industry-recognised “best practices” and the latest trends simply don’t work for your website. This is where A/B testing can help you, and you’re going to need a few things to make this happen: the correct KPIs, access to the necessary data and the tools to run successful tests. Here are some of the best tools to give you a headstart to A/B testing.


Google Analytics and Google Optimize – Powerful, free analytics

The free A/B testing starter pack that both of these tools offer is one of the best data collecting A/B test solutions around. With it, you can get just about everything you need, for free!

In terms of features, these tools offer you the ability to use extremely powerful tracking to understand the effects test changes make to your user behaviour. However, the key downside to both of these tools is that you really need to know the platform to get the best out of it. This might be more technical than you’re interested in.


Optimizely Web – Affordable, simple and powerful set of A/B tools

Optimizely is one of the digital marketing industry’s leading CRO tools, offering a range of products, and Optimizely Web is the name for its A/B testing tools. The A/B testing suite provides features for optimising web pages and has the ability to personalise different messages to different audience segments.

Your A/B tools are all pre-built with Optimizely, which means they’re very easy to implement without any of the technical skills needed to do the same via Google Analytics and Google Optimize.


VWO – Similar to Optimizely, but with some important differences

VWO is Optimizely’s largest competitor, and these two platforms are very similar. In terms of A/B testing, there is not much difference between the two platforms. However, there are some feature differences. For example, Optimizely has a dedicated mobile testing tool, while VWO allows you to get written feedback from users.

VWO has a simple pricing model and its platform is very straightforward. If simplicity is your personal preference for A/B testing and conversion optimisation, it ‘ll be hard to beat VWO.


How a digital marketing agency can help you reach your A/B testing goals

To be successful with A/B testing, you need more than just tools. You need a team of experts to run your testing with a strategy behind it. But is it easier to build a team in-house or outsource to a digital marketing agency?

Outsourcing all of your digital testing efforts to an agency that specialises in digital marketing means they’ll handle everything for you, from creating test strategies and ideas to designing tests, implementing them and analysing the results.

This is a great option if you’re new to A/B testing and don’t have the internal resources needed to run effective testing. Usually, this is a much quicker way of getting better results as the agency provides all of the required expertise to run effective tests. Agencies can help break down internal barriers and you don’t need to learn how to use any of the tools or best practices required if you don’t have time.

Here at Bundle Digital, we are experts in delivering successful website testing strategies to our clients. If you’re interested in finding out more about our past experiences and how we can help you increase your goal conversions, get in touch today.