CRO – Where should we put failed tests?

Ideas for the next generation of conversion optimisation

Conversion Optimisation has become a regular fixture in a marketers toolkit. The ability to gain efficiencies based on data driven facts is a great opportunity for many businesses. As testing becomes commonplace we are approaching the next innovation in testing which is rarely discussed. CRO Testing currents involves running variances against a base. You monitor performance on single or multiple KPIs and a winner is found. The winner then get the majority/all of the traffic and we move onto the next test. We look for statistically significant results in the data to show success in performance and live and die by the numbers.

What happens when a user prefers the version of the test that “fails”. If 90% of people preferred the new version - you still have 10 out of every 100 having a preference of your base. We talk about segmentation and targeting when setting up the test yet this is rarely used post test.

The opportunity for marketers is to get to the position where testing is not about success and failure - but its about making educated decisions on the design, user flow and content engagement. Testing should take the business to a point where they are able to understand more about their users - supporting a future targeting strategy.

For example, consider this case:

Geoff runs a successful online e-commerce company. He is currently seeing that PPC traffic converts with a higher revenue per basket than any other channel. Especially the branded PPC traffic. Geoff runs a test where the landing page highlights products based on a higher revenue bracket to help improve product consumption and navigation. The test is a success and conversion improves, bounce rate improves and revenue per basket increases further. These KPIs are valid for 8 out of 10 people and Geoff uses the new page for all visitors. Geoff is right and has improved a number of his KPIs - this is good… but not great!

We should look to innovate and segment the user base to target people based on the following methodology

It is possible with the targeting options available to the modern digital marketer to create a digital ecosystem where testing will no longer be about right or wrong - but what is right for that individual. Going back to the example - if we know common characteristics between the 20% of people who did not like the new PPC page we could target them with a second round of “negative testing” and look to improve their KPI results again. Meanwhile we would segment the 80% that saw positive results to look at other opportunities to increase the propensity for success. In turn both of these 2nd round tests will further segment the websites users enabling the testing to become a function of pseudo personalisation as well as a way to increase the opportunity to convert!

Taking this to the next step we can look to the future and where we are going with attribution and multi-channel analytics and overlay testing. A lot of modern multi-touch management profiles use statistical analysis to create profiles and therefore a calculation of weighting based on multiple touch points. Testing will take us to a position where this algorithm has to be live and flexible where tests can impact a users digital journey and interaction map in realtime.

We need to stop looking at the KPIs within testing as the opportunity to fail but more the opportunity to learn more about the audience. This will mean that no test is useless but a journey to better customer understanding!

Do you think this is the future of CRO and if so when will this turn into reality?