CRO AB Testing
Published: 15 August 2019

Let’s start with a bold statement – you’re wrong a lot of the time. Your ideas are wrong, your solutions are wrong and the problems you are trying to solve are the wrong problems.

And when I say you, I mean all of us.That’s step one in your CRO journey. The site you tend to, develop for, lavish content on and drive traffic to will be far better if you accept that.

What is CRO?

There are definitions all over Google – Moz provides one that is often quoted, but they tend to be fairly similar, suggesting:

‘Conversion Rate Optimisation (CRO) is the relentless drive to help users complete a set action within your customer journey’

Essentially, a CRO programme looks to do certain things:

  • Find issues on a site/journey
  • Understand which issues can be turned into opportunities
  • Analyse the likeliness that resolving an issue will increase a measurable metric, and how beneficial that could be for the user and business
  • Create hypotheses and tests that allow a brand to show real users an altered experience, and measure the impact this has vs users seeing the original experienceAnalyse the test results, understanding how this new experience impacts not just key metrics, but every part of the customer interaction
  • Repeat. Constantly

How does this differ from the traditional approach?

Consider this famous testing example by Maxis and EA for Sim City 5

CRO Testing SIM City 

The Maxis team believed that moving a promotional banner to a prime position above the fold would deliver an increase in sales because it incentivised users to pre-order and offered them something, over the game itself, in exchange.

There’s nothing controversial about the idea. Incentives, by their very nature, can incentivise.

The results?
The original page drove 43% higher sales than the version with a promo code. Had conventional wisdom been followed, and the idea been deployed without testing, the business would have lost serious money. And not known that it was happening.

Why did people reject the incentive?
Did people just want to buy the game, and not be tied into having to buy another one? Did they think ‘maybe I can find this cheaper somewhere else if they are discounting’? Or, did they think ‘what’s wrong with the game that they need to incentivise it like this?’

Ultimately, users were given something to think about. When they just wanted to buy a game. There’s a famous book entitled ‘Don’t make me think’, which covers usability and it holds true.

It doesn’t mean the original idea was bad though, but testing helped to clarify that it wasn’t right in this instance. And that is the power of testing. Even sensible ideas can be damaging, but testing and iteration allow you to quickly move past those to find what works.

CRO in Practice

The core tenet of CRO is to test your assumptions, no matter how sound the insight they are based on. Once upon a time, a ship was built that was unsinkable. That’s certainty right there. We’ve built a vessel to move people from a to b, in opulent luxury and absolute safety.

Except, out in the real world, there was something the experts hadn’t considered. And don’t get me started on what they did to Leo…

Apply the same logic to your website, development process or creative process. How often have you thought ‘this is a great idea – we all think it, we’re experts’ and then, when released to live, results have stagnated or fallen off a cliff?

That same scenario, where an active CRO programme is in place, looks like this.

‘That idea is great – we all think so, we’re experts’
‘What does the data say?’
‘We’ve polled users on-site, seen session recordings where the error is happening and analytics shows us people are missing the action we’d like them to complete’
‘How does the idea address the issue? How will you measure the impact? How can we test this?’

Immediately, the position changes from an assumption that an idea will create an impact to an insight-led hypothesis, where change will be measured and, if not successful, another approach will be taken.

This thinking is common in large organisations (some, but not as many as you think) but still, in the main, the approach of iterative testing is under-utilised.

The Benefits for Brands

Depending on the vertical, and size of the business, it’s a fair assumption that not all of your competitors are building and creating iteratively. And whilst some might be, the speed at which they are iterating will differ vastly.

This gives a brand looking to embrace CRO a competitive advantage – test as much as you can to deliver the experience users want and need, and you are a step ahead. Approaching experience changes in year-long cycles, without iteration during the process, risks a negative experience and a consumer who can vote with their feet and investigate a competitor’s offering.

That’s a long paragraph – the short version:

  • testing gets you to the experience your users deserve more quickly, and with less risk.
  • it encourages your team to think creatively and know that they can iterate and consider two or three variations of an experience and measure the impact of each.
  • it provides security for brands that no experience change that is negative will be seen by all users, ensuring only positive experiences reach full deployment.

Key Takeaways

It's OK to be wrong.

If you aren’t comfortable with being wrong, you (whether brand or agency) will only chase middle-of-the-road options and deliver best practice. And that doesn’t drive growth.

Embrace being wrong - learn from it, draw insight, and use the information you gather from users rejecting your idea to ensure that the next iteration is closer to friction-free.

The CRO space is still in its infancy, and it’s a mindset change from the rigid roadmaps of old, where activity was planned out in advance and didn’t alter until new budgets were released.

We’d love to help you embrace that change and deliver measurable experience improvements for your customers and prospects or to just chat about the space and discuss your challenges, either way, if you’d like to speak to an expert, drop us a call on 0208 070 7820 or email us at hello@sagittarius.agency

kier-humphreys

Kier Humphreys

Head of Customer Experience

READ MORE FROM KIER HUMPHREYS
Drum Recommends 2020   Wirehive_Partner     rar   Sitecore Partner / Powered by Sitecore  Bima 2012GooglePartnerFooter

Page Name: {% PageName %}

Page Template: {% PageTemplate %}

CampaignID: {% AgentReferrer.ID %}

CampaignName: {% AgentReferrer.Name %}

CampaignPhone: {% AgentReferrer.Phone %}

Item Location: {% PageLocation %}

Search Session Exists: False