This split test increased ecommerce profitability by reducing return rates

AWA digital helped Xero Shoes increase conversions and reduce return rates by split-testing sizing guide copy

Minimalist footwear company Xero Shoes came to conversion rate optimisation (CRO) agency AWA digital wanting to increase the conversion rate from their e-commerce website, which was already performing well. Using a data-driven CRO methodology that has evolved over 9 years (see figure below), AWA analysed over a million data points, cross-referencing findings to form test ideas.

Through surveys, live usability testing, and quantitative data analysis, it became clear that one of the main concerns for users was finding the correct shoe size.

While sizing is a common concern with apparel and shoe companies, Xero Shoes presented a unique opportunity as their sizing process had to accommodate the innovative minimalist footwear designs that have garnered a loyal, fast-growing fanbase.

Our research showed that users were confused about how their normal shoe size translated to their size in Xero Shoes. In fact, many customers would accidentally buy the wrong size, inevitably leading to returns which were a significant expense for the company.

If we could reduce user anxiety related to sizing, there would be an opportunity to reduce the rate of returns in addition to increasing the number of purchases. This is what excites us about CRO - making a multifaceted, meaningful impact on the entire sales funnel.

Triangulating quantitative and qualitative data to form test hypotheses

In order to develop a better picture of the users on the Xero Shoes website, AWA employed a variety of research methods and analyzed qualitative data such as survey results, interview transcripts as well as quantitative data from Google Analytics data, heatmaps, and clickmaps.

Sizing quickly surfaced as a major concern for many buyers; 27% of survey respondents cited size confusion or concern as a key part of the reason they chose not to buy.

Heatmaps and clickmaps showed that size guides, included on every product page, received a lot of attention. Naturally, this became a focus area for the team at AWA. These size guides were inconsistent across products and often included verbose and/or language that didn’t resonate with the purchase decision. Combined with the common sizing anxieties users faced, these factors led us to hypothesise that rewriting the size guides for improved clarity and intuitiveness would improve conversion rates and reduce return rates, as it would reduce user anxiety as related to sizing.

As AWA’s resident copywriter Mel Henson, puts it:

Copy is usually used in a CRO process when, as in this case, research identifies that the current messaging is misunderstood by customers or fails to allay the concerns of potential buyers. CRO copywriting uses traditional direct marketing methods as well as modern techniques of persuasion developed in the last few years by leading neuroscientists such as Daniel Kahneman, Roger Dooley and Robert Cialdini. 

 Test hypothesis

Because we saw high engagement with size guide content on product details pages, but also high reported levels of anxiety and confusion with sizing, we expected that rewriting size guides for clarity and intuitiveness would reduce customer anxiety and confusion related to sizing, which would not only increase conversion rates, but also reduce return rates.

Control vs. variation

Control:

The original size guides were reasonably good already, though not always consistent across shoe types, and ultimately caused unnecessary anxiety in the purchase decision-making process.

Additionally, the original size guides were very long and contained little white space - so there was an opportunity to improve the flow of information.

Variation:

For the test variation, we rewrote the size guides for all Xero Shoes products. In our methodology, AWA digital treat every webpage as a sales conversation. From that perspective, it’s important to present the right information at the right time, framed in the right way.

Copywriter Mel Henson explains here how we approached rewriting the size guides:

We gave people clear instructions on how to order the correct size, including an embedded command (Buy your usual US size). This simplified the decision making for the customer, which made it easier for them to buy. The instructional copy was immediately followed by risk-reduction copy around easy returns. This reassurance helped overcome residual barriers to purchase.

Additionally, we prioritized creating more white space, larger font sizes, and bolded headings in the size guide format in order to make it feel more intuitive for the reader. We ultimately wanted the customer to feel confident in their understanding of the copy and what their next step should be.

Testing technology

The testing platform was Visual Website Optimizer. AWA is the only Premium Partner of VWO in the UK and South Africa. By using a combination of VWO to monitor test results and long-term output, we were able to track key metrics such as conversion rate, revenue per visitor, and average order value. It is standard practice to also integrate the testing platform with Google Analytics for deeper and segmented post-test analysis.

In addition, we worked closely with Xero Shoes to track return rates on each individual product before, during, and after the testing period.

VWO’s SmartStats engine was used to determine the outcome. This model uses Bayesian statistical confidence algorithms, which can arrive at a statistically robust result in a reasonable time frame.

Outcome

We bundled all the replacement size guides into one test, as this would allow for faster results and the changes made to size guides followed a consistent pattern (meaning that we would not have to worry about conflating different changes with changes resulting from the test). Through tight GA-integration, we are able to measure the impact of a global test on segments and individual categories.

At a high level, the results from VWO were looking great:

• Control: 6.53% CR
• Variation: 6.98% CR
• Conversion Uplift: 6.9%
• Revenue Uplift During Test Period: 2.4%
• Probability: 95%

AWA’s philosophy is to understand the full impact of a test on a business, beyond what you’re able to see in VWO and even GA. Because of the insight that size confusion led to returns, the return rate was an important metric to consider in the final results. Tracking return rates across all products before and after the test allowed us to determine that return rates decreased by an average of 38% per shoe - leading to significant savings in costs and logistics for Xero Shoes.

Long-term Impact

Upon conclusion, traffic balance was temporarily set to 100% on the winning variation until the changes could be implemented by the Xero Shoes development team. Today you can see the new size guides hard-coded on every Xero product page.

This win, along with a number of other winning tests, is resulting in a long-term increase in revenue and conversion rate, along with a significant decrease in the costs and logistical issues associated with returns, for Xero Shoes.

Conclusion

While there are many techniques that can improve a website’s revenue and conversion rates, it’s easy to overlook the power of copywriting to reframe the user’s perceptions of the product, and particularly in this case, reduce common anxieties.

When conceiving and executing A/B tests, consider how the test can impact touchpoints all along the funnel instead of just focusing on those KPIs surfaced in the testing platform. Little changes can affect the customer experience in a multitude of ways simultaneously, leading to the discovery of new opportunities with a multiplying effect across the business.

Conversion Rate Optimisation – or CRO – may seem like a dark art if you’ve never experienced it first-hand. Ecommerce companies who embrace CRO claim to get stellar results and ever increasing online sales. If you’d like to replicate their achievements, then the first step is to understand the CRO process as a whole. To help with that, we’ve created this infographic which gives an overview of all the key elements involved in successful CRO, designed like the well-known periodic table for chemical elements.