A/B Testing Part 2

Marawan Mansour
6 min readJul 26, 2021
Source https://www.invisionapp.com/inside-design/let-go-a-b-testing/

A/B Testing (A/B Testing with Custom Landing Pages) has become very popular over the past few years. In this fast-moving world, sales and leads can happen much faster than they were just a few short years ago. Having a lead or sale on your website really makes your business stand out and makes you look professional. With the right page in place, you can get a high click-through rate and a lot of interested parties signing up to your mailing list. All you need to do is follow these tips to ensure that your web page really stands out and converts like crazy!

How to A/B Test: There are three elements involved when it comes to A/B testing. The Control: The control basically is the previous page you’d like to replace. The Challenger: This is a completely different version of your control with the exact changes you’d like to test. The Measurement: Once you’ve identified a problem with one of your pages, you want to look at what you can change about that page and how you can make that page more effective at getting a higher open rate. The third element is measuring the performance and tracking it constantly to identify areas for improvement.

Testing A/B Testing with Custom Landing Pages When conducting A/B testing with custom landing pages, you really want to consider two things when conducting your testing. For one thing, you want to make sure that your landing page contains only relevant information and copy. Your conversion rates will suffer if the landing page contains unrelated keywords, offers unrelated sales pitches, and includes non-relevant distractions. The second thing to consider is that you need to get your user behavior in two formats — A and B. Without getting your user behavior in both A and B, you won’t be able to get your results, which means you won’t have a good measure of effectiveness.

A/B Testing with Custom landing pages For most A/B testing experiments, you’ll either want to do A and B together, or start with A followed by B. With A and B combined, you’ll get the best conversion possible on your test page. But when you do A and B separately, you’ll have to eliminate other options that might be better. That means you won’t necessarily know whether you’re using the most effective copy or you’re just wasting your time.

A/B Testing with a hypothesis will help you test for generalize results, as well as specific results. In general, A and B provide similar conversion rates and click-through rates. However, there will sometimes be differences between the percentage of visitors who are converted and the percentage of visitors who are deterred from clicking through. For this reason, you’ll want to run a hypothesis. A hypothesis is a statement that describes the general result you’re aiming for with your A/B testing.

Your hypothesis should contain words like “conversions,” “converted,” “deterred,” or “clicks against.” “Conversion” is the term used to describe the action that took place when a visitor clicked through. “Deterred” is the word used to describe those who were deterred from clicking through. “ CTRs” (click-through-rate) are the numbers you’re tracking. “A” will be the number of unique visitors and “B” will be the number of visitors who clicked through. These numbers will be used to compare your A/B pages to the final pages in your funnel.

A/B Testing with Custom landing pages For A/B testing with custom landing pages, you need to know the conversion rate on each individual page before you even begin testing. These pages will be the exact same as the final landing page, just optimized for A/B testing. By having this information, you can begin to eliminate the pages that don’t convert. For instance, if you’re optimizing a personal page for A/B testing and you discover that it doesn’t convert, then it is likely that there aren’t any sales conversions on this page. You don’t want to spend time and money on testing if you have no idea what the conversion rates are on individual pages.

A/B Testing with Google Analytics Once you have identified the conversion rates, you can begin to focus on user behavior. This involves observing how users navigate your site. User behavior is easily the most influential factor in determining the success of an online business. Google’s analytics software will allow you to observe where your visitors go once they reach your home page or the main navigation. Understanding user behavior allows you to customize your a/b testing so that it gives users the experience that brings them back to your site over again.

Monitoring your A/B Testing

Monitoring your A/B test results is a critical part of good clinical practice. Whether you are a health care team leader overseeing the quality and safety of a hospital or an administrator managing a study group, you have a responsibility to ensure that the data collection is accurate. Otherwise, you could be opening the door to numerous mistakes in the interpretation of results and treatment plans. That is why it is important to monitor your test results. When you choose your testing method(s), here are some things to consider when designing your plan.

In addition to the data collected through your a/b testing, you need to know which groups are performing well and which ones are performing poorly. Data collection on demographics helps you determine the health care needs of each patient in your study. Statistics can also help you design better treatments and better patient care. Without this knowledge, it is easy to make the wrong choices when developing the design of your experiment.

A/B testing requires you to provide a post hoc analysis when the results of your initial test set you on a course for treating a particular patient with a specific medication. While this may seem like a tedious task, it can be one of the most important steps in determining whether your study’s conclusions are accurate. Careful identification of all the variables involved in the study will provide you with a precise account of the results and allow you to discard the unnecessary variable in your hypothesis. However, identifying the right variables and their corresponding effect on your hypothesis can be difficult. For example, if you observe a significant finding that the primary outcome is significantly different from the secondary outcome, but fail to control for many of the important characteristics of the variable, you may be committing a costly mistake.

Your A/B test dashboard should allow you to select specific outcomes from your experiment and track their relative frequencies. The dashboard should also provide a means to select and run new users’ reports and to check your experiment’s statistical significance. Most analytics programs allow you to create new users and send them metrics reports based on their data sets and settings. New users are typically limited to a limited number of metrics reports, so keeping the data set simple will help you maximize performance.

The final component of an effective a/b testing strategy is proper data cleansing. This process removes the duplicated variables from your a/b testing profile. In general, the fewer dimensions you need in your a/b testing profile the better. This ensures that the data sets collected during your investigation yield only the most relevant results and that the variables you remove do not significantly alter the main results.

Monitoring your A/B test involves more than just collecting metrics reports on your trial. It requires you to take the steps needed to keep your a/b testing profitable. Monitor your experiment with integrity and make sure it is running with as few bugs as possible. Clean and bug-free samples will help you get started with your analytics experiments quickly and help you save money on expensive retesting efforts.

If you would like to read more, please check CXL Institute and learn from the world's top marketing leaders.

--

--