Upcoming LunaMetrics Seminars
New York City, Aug 4-8 San Francisco, Aug 11-15 Los Angeles - Anaheim, Sep 8-12 Washington DC, Sep 22-26

Ignore Your Gut – Increasing Conversion With User Testing

Part 2 in my unofficial “You’re Probably Doing It Wrong” series. Read the first part about using Google Analytics to gain Insights.

The Cruel Tutelage of Sayf Sharif

You are wrong. This is the first rule. You think you know what the right thing to do on your website is? You are wrong. You think you know what to do to properly improve your digital presence? You are wrong. You think you know what your users want? You are wrong. Before you can truly begin to optimize your website and your online digital presence and marketing you must come to this realization. You must be unmade before you can build yourself back up. You must know that everything you think you know to be good, and right, is wrong.

I emphasize this with a beard swish.

The only way to know whether something is going to work or not, is to test it on your users.

And then you need to test them again.

And again.

Your gut hunch might be right, or it might be wrong. Your users might want something, but if they get it, they might be more upset. Sometimes what you think people want, really isn’t what they want. The only way you’re going to get a clear picture is by actually testing users, asking them questions, and listening to and even watching their responses.

“Run the Ball!”

Anyone who has been to an American Football game (college or professional) or even has watched a game in a vibrant sports bar is probably familiar with someone yelling “Run the Ball!” There’s always some point where the users (in this case the viewers of the game) know what they want. They want the team to run the ball. So they scream it. At a television in some cases.

Putting aside the futility of that action, it’s interesting how much users actually affect what teams do, specifically decisions made by coaches.  There have been studies about whether NFL coaches should “go for it” on fourth down.  And in many cases they don’t, when statistically they’d be better off doing so. If you’ve ever been at a game where a team is Fourth and inches, and the team chooses to punt, you’ve heard the groans from the stands. “GO FOR IT!” come the screams. The coach punts. Well, even if it’s early in the game, and the team is at midfield, it makes SENSE for them to go for it when it’s Fourth and under 3 yards to go. The users WANT the coach to go for it. The numbers say he’s better off going for it. So why not? Because if they go for it and fail, it’s the coaches fault, and the users (the viewers and fans) will blame the coach.

It’s a situation where mathematically it makes sense to do one thing, the users want it intrinsically, but a decision is made against doing it because the coach wants to defer the risk. The “gut feelings” of the coach, and the users, defy expectations, and are ultimately selfish and irrational.

Meet your users.

Oh wait, did you think they would all be rational people, full of common sense, with great vision both literally and metaphorically?

It’s OK for your Website to have an Ugly Girlfriend

Moneyball is another sports example of the problem.  It was a situation where math and statistics indicated something different from decades of experience of scouts. The scouts were convinced THEY knew what the right answer was. To hell with the math and statistics. Instead Billy Beane showed how you could use that to essentially increase his conversion (number of games won) by focusing on the metrics that mattered, rather than going with ‘gut feelings’. Scouts ignored players because they had weird throwing styles, or ugly girlfriends. The math simply didn’t care about what a players girlfriend looked like, it cared about their onfield performance.

it’s the same with your website, your email marketing, your paid ads. It doesn’t matter if they have an ugly girlfriend. What matters is if they convert. Who cares if the subject title line doesn’t make any sense if it hits home runs? There has been a lot of talk about Obama’s use of analysis in his 2012 win. Politics aside, the proof of their testing scheme and money raised is pretty incredible. And do you know what the top subject line for their emails was overall? Of all their emails which raised tens of millions of dollars?

“hey” 

I’m not kidding their number one email subject line was supposedly “hey”.  But here’s some other examples.  What was interesting was how they’d segment their users, send out a variety of emails, and see what worked, and what didn’t, and then what worked in the smaller cohorts, they’d then send out to the main lists. hat might have generated $400,000 instead generated $2.5 million. Those kind of differences. They’d split up demographic groups into various cohorts and test within each and send different emails to each. One person might get an email from Joe Biden saying “we’re in trouble” and his wife might get an email  at the same time from Michelle Obama saying “help me help barrack”. It was all targeted, tested, and optimized.

Nobody said “hey this one is obviously the best one”. They put together a bunch of ideas. Tested on small groups, and only after that did they push live.

It’s Not Rocket Surgery: Test Your Users

Anyone can do it, and everyone should. 

For your website, try always having at least one Content Experiment running in Google Analytics. Test a home page marketing header. Test the color of a button. Test which side you put your product image on. Test the placement of social media buttons. If you have an idea to make something better, then instead of just changing it, test it. Don’t ever just change something. Test it first.  If you’re redoing your entire website, test the wireframes, test the designs, test a beta version of the site. Test before you launch it at all.

For email marketing it can be as simple as just sending one email to half your users, and a second email to the other half. Every time you send out an email, send two and see which one gets more opens, which one converts more on your site, etc. Then keep optimizing. Or send a bunch of emails out with different messages to different groups. if you have demographic breakdowns, use those to create separate sub lists, and test away. Never be sending out just one email to everyone on your list. Even if you have only 100 people on your list, you can always just send out two different emails.

Adwords allows A/B testing of your ads built into the system. Use the functionality provided to you.

Heck even with your content marketing, watch what sorts of content gets responses from people, and what doesn’t, and then test your content. Test when you post things on social media.

Test. Test. Test.

TL;DR

  • You think you know what people want, but you’re wrong, you don’t.
  • You should user test everything, all your websites, emails, ads, you name it.
  • If you’re not testing you are almost definitely leaving money on the table, and it could be a significant amount.
  • If you don’t know how to organize it all and get started then contact us here at LunaMetrics, we’d be more than happy to help you.

 

Sayf Sharif

About Sayf Sharif

Sayf Sharif is a Web Analyst, and expert in Usability and UX, who has worked with businesses large and small to maximize their online presence since the beginning of the Web, winning numerous awards along the way. Sayf has studied human tool use from the stone age (he went to graduate school for Archaeology) to the information age (he started programing on his father’s TRS-80), and is always interested in what goals people wish to accomplish using their tools, and how successful that experience was.

http://www.lunametrics.com/blog/2012/12/04/ignore-gut-increasing-conversion-user-testing/

2 Responses to “Ignore Your Gut – Increasing Conversion With User Testing”

Phil Sharp says:

Well said Sayf. I think this one line of yours does an awesome job of summing the whole thing up: “You think you know what people want, but you’re wrong, you don’t.”

I’m a pretty stubborn guy so I often say stupid things like, “Trust me, this version will win” or, “Trust me, designing it like this is way better.” The one thing people learn is…not to trust me :). I’ve finally learned my lesson and now I’m sure to always test things.

Sayf Sharif Sayf Sharif says:

Thanks Phil. If there’s one thing that using usertesting.com has taught me, is just how wrong I can be about what I expect users to see, and what I expect users to do.

No matter what brilliant idea we have to make users use a website feature more, it’s humbling to put it into a micro test of 5 users, and have not a single one of them even see the feature we want them to test.

One think people often say to me is “what if we test something and it doesn’t work?” to which i say “Well if we hadn’t tested it, and you had just put your brilliant idea live, wouldn’t that be worse?”