A/B Testing: Why and How We Do It


If you’ve worked with any form of digital marketing, the phrase “A/B Testing” is bound to have come up at least once during your experience. In short, A/B testing is a tool used to figure out the best digital marketing strategies for your business through testing different variables. A/B testing is frequently used with many different channels including email, social, website, and display. Variables can be related to copy, imagery or even user experience functionality. For instance, a popular A/B test includes testing out different subject lines for the same email to see which drives more engagement. A/B testing allows you to single out what the most compelling aspects of your digital marketing are, which can help drive strategy and successful returns.



The first step to a successful A/B test is to understand two major things: your business goals and the goals of your buyers. If you don’t know why you’re A/B testing in the first place, it’s difficult to pin down the right variables to test. Understanding who your buyers are and what they want helps you understand where tension may be occurring in your messaging. Is your offer in line with what your customers want? Is your copy consistent with your landing page or imagery? Are customers understanding what you’re trying to say? These are a few common pain points in digital marketing that can be used to not only understand what you should test, but what metrics and hypotheses are important to your test.

Hypotheses are used to provide a sound foundation for your A/B test. Kevin Ho at Wishpond provided a simple framework for developing a quality A/B testing hypothesis:


Using this framework can help develop strong A/B testing hypotheses in-line with your goals and objectives. Use these hypotheses to guide your metrics and test development to always bring it back to the bottom line.


Once you understand these important pieces of information, the next is to develop your content. The name “A/B Test” is an allusion to the fact that you are creating two different versions of one piece of content (Version A/Version B) and only making changes to a single variable. Common variables are email subject lines, landing pages, a call-to-action button, or even the image associated with your content. You will then distribute these two versions to your sample.


In order to keep your data clean and informative, you should only run one test per campaign. This is to make sure that any increases or decreases can be directly attributable to the single variable that you are testing. For instance, if you decide to A/B test your copy and your imagery, how do you know which one impacted your engagement? The example below illustrates the testing of one variable for a hypothetical Facebook ad, the image. Notice the copy, link and CTA remain the same.



A/B tests are not done overnight, you want to make sure you are generating enough traffic and engagement to justify any decisions you are going to make regarding future marketing initiatives. Therefore, it is important to let A/B tests run anywhere from a few days to a few weeks in order to make sure that results are substantial enough to take confident action. However, it is also important to make sure that you aren’t letting your A/B tests run too long because this can also negatively affect your data. Ultimately, it’s up to you to decide what the appropriate amount of traffic or length of time is, but be sure to make smart decisions about when enough data is truly enough.


The goal of A/B testing is to obtain actionable data about your marketing tactics. What this data looks like and whether or not you consider it actionable is up to you and the goals of your business, but typically you eliminate the lower performing variation or use the data to inform a new A/B test. Your data will either prove or disprove your hypotheses or be found inconclusive. Either way, this data should be used to optimize your digital marketing efforts through reducing inefficiencies by obtaining a thorough and clear understanding of what your customers are looking for.


The goal of any business is to increase its bottom line and A/B testing allows you to do this by optimizing your marketing strategies. It’s important to understand why and how A/B tests are used in digital marketing in order to remain informed on what messaging and experiences your customers are looking for. While there are endless amounts of variables to test, following the above advice can help you craft an effective A/B test and increase your digital marketing returns.