Conversion.com’s Stephen Pavlovich (founder and chief executive) and Kyle Hearnshaw (head of conversion strategy) joined InternetRetailing for a recent webinar, How to create your retail A/B testing framework in 30 minutes. Here’s a bulletpoint overview of what they covered.
Stephen Pavlovich, chief executive of conversion.com, opened the webinar with an introduction to the company, and by explaining how having a framework for testing means companies are more likely to be testing strategically and not tactically. “When you do so the value of your testing increases over time,” he said. “The more experiments you run the more insights you get and the more impact on your programme.”
Four ways to find out if you’re testing tactically or strategically
- What do you focus on? Where does each experiment start? Start with the solution? Probably thinking tactically. Start by thinking about the problem? Eg how do we increase our returns rate? Thinking strategically.
- Where do experiment ideas come from? Tactical: best practice, competitor research , heuristics. Strategic: data and insight approach to testing.
- How do you plan your experiments? Short term or long term?
- How do other teams manage experimentation? In a tactical world, all teams have own approach. Not much data being shared. Often convoluted. Strategic approach: common framework or approach – even if areas focused on are different.
How to create a retail A/B framework in 30 mins or less
- Define goals and KPIs: important to set clear experimentation goals.
- Sketch KPI tree to show what matters: what we can influence. Two or three is plenty. Tips include: focus on binomial metrics (that can be expressed as a %) and have a primary metric and a guardrail metric.
- Audiences and areas: Kyle Hearnshaw, head of conversion strategy, then went on to address how the make up of audience and the areas we cover affect conversion rates. It’s important to know who you’re trying to influence. Take five minutes to map audiences: ask questions to find which audience should look at. eg. Which are converting the least, have best/worse conversion rates? These might include segments such as devices and product interest. For each, identify high opportunity areas. Look across the whole customer lifecycle. Mapping the experience, working out how to affect KPIs.
- Levers: previous research/data can suggest what conversion levers should be: barriers and motivations. Looking for evidence of levers to then go on and validate. List potential levers – and refine it. Validate quickly through a minimum viable experiment. Iterative approach to testing rapidly redefines our set of levers: “we started off with a list of 8 to 10 and want to get down to the ones that show the most value.”
- This completes the core framework (downloadable printable available through the webinar). To do more: run new research to gather more evidence for levers; run ideation sessions to refine minimum viable experiments.
The webinar finished with a lively Q&A session.
Click here to find out more about our upcoming and previous webinars.
