Corporate Politics and Conversion

"Sure Boss, We'll Try Your Idea, Too"

It's great to read newsletters and books about great converting websites, best practices and all that other good stuff. But when your boss says, "Make those pictures smaller" or " I hate blue buttons," what's an employee to do?

Fortunately, there is an excellent and inexpensive answer to this problem: multivariate testing. Frequent readers of this newsletter already understand the concept, but you can go back and read the original article we wrote on MVT a couple of months ago. So when you boss says, "Make the button green," you can say, "That's a great idea. Let's make that one of the options we try with our multivariate testing." Then after you run the test (if you use Google's Website Optimizer, the software is free), you can see which version pulls the best. You don't have to fight words with words, you can fight them with numbers. "Oh yes, we did try the version you wanted, the one with small pictures and green buttons," you'll write in your email. "But larger pictures and red buttons resulted in twice as many conversions," you might add. The email part is key; that way, when he insists on having it his own way, you'll have the results and recommendation documented when your current boss loses his job and you get a new boss.

How significant is user testing?

Just because multivariate testing is awesome, cheap, fun and solves corporate politics is not enough. You may just not have enough traffic on your page to learn enough in a short enough period of time.

You could do a conversion best practices analysis. But a more interesting way to go would be with user testing. After all, if you give it to us, we'll tell you what we think is wrong. But if you let us show it to users, they'll tell you what they know is wrong.

Recently, a potential customer said to us, "I have a hard time believing that 5-8 users are going to teach me anything significant." After we published that statement, Dennis Mortensen, the COO of web analytics company IndexTools, did a whole piece on his blog about the statistical significance of user testing. He showed mathematically that you usually learn 67% of your problems with five users. By eight users, you usually learn about 89% of your problems. After ten users, it is no longer cost effective to do user testing (at least on that part of the site.)

We usually like to do user testing first -- that way, we have better ideas for the multivariate testing. But your boss may just want to see his idea tested....

Read last month's article about user testing.