A/B and Multi-Variate Testing (MVT) are essential conversion optimization activities and they deserve a separate pillar within a CRO Maturity Model. In a nutshell A/B and MVT testing allow you to test two or more different experiences (and it can be pages, images, copies.. you name it) head to head and find out what is working best for you.

There can be defined several maturity stages of your online testing program. The more mature you do online testing the higher your chances are to get conversion lifts. If testing belongs to your responsibilities read it carefully and work on the progress to shift your organization up on the online testing maturity ladder.

Stage 1: No Testing

Indeed, that’s an initial stage where a lot of businesses still belong today. They have various reasons why they don’t do A/B or Multi-Variate Testing, below are four of the most common.

a) Lack of knowledge

If you read Online Behavior you’re most likely to have some knowledge of online testing, so probably it is not your case. But there are many folks who have just started with website optimization. The tools got affordable and easy to use, so optimization became more popular. For those guys I suggest reading this Ultimate Guide to A/B Testing on SmashingMagazine. It is a good and comprehensive overview.

b) Lack of traffic/conversions

In my experience this is one of the most common reasons why a company doesn’t take advantage of A/B testing. When you run just a small e-commerce site or your business is rather offline, you probably don’t have hundreds of online orders per week. You might have around 200 orders per month, which is good but not enough to do A/B testing effectively. However you can (and should) still do website optimization and measure the performance before and after the changes to track the results.

c) Technical issues

Although testing tools got a lot easier to implement recently, you might still experience some technical issues. I would suggest talking to your IT staff and/or your (potential) testing tool vendor. I am sure together you can remove the barriers and enter the holy land of testing.

d) Organization issues

I know few cases where the organization just doesn’t appreciate the advantages of online testing. E.g. the CEO is an authoritative person who believes he knows best how the website should look like. In this case I would suggest start operating below the radar. Do a pilot and then show the results to the CEO, tell the customers prefer a different experience and speak in the terms of $ – how much money an alternative experience can make the company.

Stage 2: Ad-hoc Testing

The next stage is so called Ad-hoc Testing. You occasionally do A/B testing, but your biggest problem is that you do it… just occasionally. Typically you play with it only when you have time or when you read an article about superbly performing big orange buttons. It doesn’t belong to your regular conversion optimization activities as you mostly implement the changes without testing.

As the quantity of your tests is the first issue, the second one is usually the quality. It goes hand in hand. When you test just a little, you usually execute poor tests. Poor tests have poor results, and poor results don’t persuade you enough to do A/B testing more often. It is a vicious circle. Step out!

First of all, drive better insights. Dedicate a couple of hours analyzing performance, do a research, identify weak spots and come up with solid hypotheses how to improve your conversion. Then prioritize them – start with high impact/low effort ideas and end with low impact/high effort ones (well, don’t even execute these ones).

And finally start rolling out the tests, one by one. Prepare a test, execute it, analyze results and take actions! And again… and again. Mainly the quality of your research phase determines whether you succeed or not.

Stage 3: Regular & Planned Testing

You made a progress and moved from Ad-hoc Testing to more advanced level. At this stage you do A/B testing on a regular basis. You start executing more sophisticated tests, including Multi-Variate ones, tests within just specific segments or behavioral targeting tests.

The typical outcome of the test won’t be only a performance uplift but you start caring about getting learnings about customers’ behaviour. You realize that testing is a great way to better know your customers and you start thinking about using those learnings for follow-up tests.

You also pay more attention to test planning and their execution. You’re trying to maximize the number of executed tests within the limited resources you have. Well-thought planning comes to the scene. You create a testing roadmap which you share with key stakeholders including your manager, developers, designers, QA and other operational units. You set start and end dates to all tests. One test follows the other. When a test is launched, other is being prepared. With good planning and execution you can double the quantity of tests you run.

Stage 4: Iterative Testing

Iterative testing improves mainly the quality of your testing program. You start to optimize in iterations. When you find out that your key landing page stinks and needs to be improved, you run a concept test first – A/B test few different concepts/layouts of the page (either long or short page, either emotional or rational design etc.).

Then when you find the right one, you focus on the single element, in iterations. Find the right headline, call to action, image, price etc. You find iteratively the best performing experience. It requires high traffic and enough resources – to be able to run tests fast and find the best experience relatively quickly – but it is a great way to find out what exactly is working in your case.

Find inspiration in a structured approach to conversion rate optimization. Focus on testing the right thing, always get ideas from analysis, measure different metrics within each tests and evaluate it considering the impact on all of them. Learn from every test.

Even a test with 20% decrease in conversion rate can give you valuable insights. Use them to build follow-up tests. Share the learnings within the whole organization. Evangelize online testing and spread its impact beyond the online part of business. Many tests can have a real impact in offline marketing, communication or a product itself.

Stage 5: Disciplined Testing

The top notch of the testing maturity. When you’re at this stage CRO is one of the core assets of your business. You’re an optimization company, always testing and improving by failing and experimenting.

There is not a single opportunity for a test you don’t use. A high-traffic landing page a week without any test running on it? Unthinkable! Your company have a full department dedicated people on conversion rate optimization – analysts, designers, project managers, developers etc. You have a large repository of tests to run, same as large repository of initiatives and visuals to test.

Your goal is to display every single customer a tailored best performing experience – just for him.

There are not many companies which can position themselves as Online Testing Stage 5. Frankly I have not been working with any of them yet. However I believe the companies which experiment a lot, never stop improving, test bold ideas, really react to customers’ needs (as a company, not just their website!) have a bright future in today’s very dynamic world.