Dear Jared:

Before I say how much I disagree with your recent comments on five second tests and homepages, I should point out that I love your new series of podcasts. Not the stuff you do with Josh or the seven other usability experts who pontificate about what Brown University should do. That other stuff was so boring that I deleted you from my iPod. (Do I win the award for the Queen of Criticism or what?) But as luck would have it, I bought a new iPod and “made” my daughter set it up for me (I am just too busy) and she set it up to get ALL my podcasts. So I was surprised and delighted to hear that you and Christine Perfetti are doing a “usable” series on usability — stuff that I can really use. I listened to home pages and store pages and gallery pages. And then I heard you talk about five second tests.

(Now, for the other thousand people who may eavesdropping: a five second test is when you show a user a page for five seconds, take it away and ask them, “What does this page do?” Reason being: that’s about what users give a page, five seconds, before they decide that they are in the right place or not. It’s fast and cheap and helpful.)

But Jared - I think you are wrong when you say that five second tests aren’t helpful with home pages. I use five second tests with users to test home pages all the time.

In your podcasts, you said something like, “Users already know what the site is about. People don’t need to know what the homepage of UPS is all about, they know what UPS does, they go there to track a package, for example.” Well sure, that’s true if the site is UPS, or JC Penney, or Fidelity Investments, or your favorite site, the Bureau of Labor Statistics. But do you really believe that all municipal sewer engineers are familiar with RedZone Robotics? Do you think that every realtor knows HomeFeedBack? These non-Fortune-500 sites may be found in the organic listing (by their home page) and the user will give the site — what, maybe five seconds? — before deciding to bail. Bloggers and linkers like me almost always link to a homepage, and if someone follows a link, they may end up on a homepage of a site they don’t know, giving it what — five seconds? — before deciding to leave. Or not.

I have yet to start a user test with a five second homepage test and hear the user say, “Oh them. I know them.”

You also pointed out that when someone goes to a site with a specific purpose — like finding instructions on their HP printer — they only care if they can find the link to their printer. They don’t care if the page “looks” like a company that sells printers and computers.

But the example is somewhat lacking. It’s really not any different from the UPS example - the user has a task to complete and knows what site he should complete it on. What about when you are searching, when you are following a link, when you are typing a link in that you heard on the radio?

Recently, we did some user testing on a site that sells do it yourself pieces to construct parts of your home. As part of that test, we did a five second home page test. “I would use this site if I wanted to have the company do the construction for me,” users said, almost uniformly. They were all really surprised that the site was a place to buy the parts so that they could do the construction themselves. In retrospect, it’s clear as the nose on my face, but my nose was way too close to that site. Now, we are going to test the pieces of the home page that gave the users the wrong impression.

All because we did five second home page tests.

Robbin

While I am waiting for everyone to submit entries to the “Criticize GA Documentation” contest, I have to tell you about this great conversion clinic I did on Thursday.

You know how the SEO events have clinics, where people submit their sites, and a few experts evaluate problems/opportunities in real time? Well, I did the same thing for conversion - an “expert” analysis. I cautioned the audience that the best experts would be users, but that this might be one place to start.

It must have been a success, because after the event, four or five people came up to me and typed in their sites and asked me to do those evaluations, too. (Not surprisingly, that’s when I got to see the really low-hanging fruit.)

Here are some of the things that I saw across many sites:

Incompatibility with Firefox. Not a few website owners were surprised at the way their sites rendered in Firefox, and commented that some text didn’t show up, or that links were broken there.

No scent. Scent comes in lots of different flavors (ooh, there’s a good one), but one of the ways I want to see scent is inline links that enable me to pursue my goal (links right in the copy.) I saw pages and pages of text with no links.

Hidden Forms. I saw lots of B-to-B Contact Us forms that were hidden behind links. If that’s your goal, why don’t you have a form on every page, or at least, start your form on every page?

Hidden Phone numbers.. If a phone call is one of your most important calls to action, why don’t you have it at the top of every page? This is the same issue as the last one — why do people hide their call to action?

Navigation. I saw those standard B-to-B non-descriptive navigation terms: Home, Services, Products, About Us, Contact Us and Resources. (Why can’t anybody say what they sell in their navigation? Well, one lady did that.)

On-site search. I found very few on-site search boxes. The only one that I remember was from a lady who had a Yahoo! store.

It’s All About Us. Companies that talked about themselves instead of their customers.

You sell what? I saw this a lot, companies that didn’t make it clear what they sold. More than once, I had to ask a website owner, “Now, what is it you guys do or sell?”

I didn’t see a lot of “my nephew created it” websites, and no one required a sign-in before they allowed you to spend money. (Although one guy made his “sign in here if you are already registered” so prominent that it obscured the other options.)

I love to speak in public, and this clinic was more fun than perhaps any other presentation I’ve done. I couldn’t have done the seminar if it weren’t for the help that LunaMetricians Taylor Pratt and Shareen Jordan gave me (during the two minutes that I studied each site, they got up and talked about best practices in conversion, or analytics.) Also helping in that space was Tim Sweet from Nauticom. (He absolutely lives up to his name.)

End notes: Many thanks to Scott Baldwin for debugging our blog and showing me why Safari was so intolerant of the unmatched tag. And to the two people who already made submissions to my “Criticize GA Documentation” contest. So come on, come on, let’s get some more entries.

Robbin

Yesterday, I was talking to a potential client. Like most, he wanted to improve his conversion rate, and like even the biggest companies, he had to live within a budget. We discussed the pros and cons of doing user testing with 5-8 users vs. my doing an “expert” conversion analysis. The costs weren’t different enough to be a consideration.

“Don’t take this the wrong way,” he said, “But I highly doubt that I am going to learn anything with 5-8 users.”

Now, at some level he was absolutely right. Five to eight users would not be statistically significant. Nor, for that matter, would fifty or eighty.

So would one conversion analyst be more statistically significant? No. I can say, “Studies show that making the button red tends to work better,” but it might not work better for your site. Plus, it is incredibly hard for a best practices person to discard the “professional blindness” that knowing websites brings with it. (This is the reason that I don’t allow web designers to sign up for our user tests - same problem.) Web analytics are statistically significant, but we only know that everyone bails out on a page, we don’t know why. Multivariate testing is the best, but you still need to know what to test.

This is why I love user testing. It’s not that expensive, especially if you do it yourself. Sure, you won’t learn everything you always wanted to know, but when you hear four people out of five tell you that the deal isn’t good enough for them to be interested, you sure do know what to test.

Speaking of which, I think too many people think user testing is just usability. Usability does go hand in hand with conversion, but you learn about pricing, about trust, about what people notice on the page. And a great idea (I owe this to a LunaMetrics customer) is to do user testing on your competitors’ sites. That’s when you learn where they have elements that are worth testing on your own site.

Robbin

I never give any money to the Harvard Business School (HBS). So finally, they sent me a survey, asking why not.

(For the record: I always think that giving to the Greater Pittsburgh Community Food Bank is so much more important than giving to a corporation with a $25 billion endowment.)

Now, I applaud them for actually asking why I never give. But their survey was really lousy. And unlike companies, who may be stuck with a pre-packaged web survey from an outfit like BizRate, this is a school that can afford to buy anything they want — especially when it comes to raising money.

Rarely does someone pay for your time when you fill out a survey. So if I am donating my time, why don’t the people who put together the surveys allow me to skip some of the questions? If I don’t like a question, I’m going to lie, anyway, right? And if you had all the money in the world (like HBS does) you could even create a survey that points out that you missed a question and still lets you skip it (in case you don’t want to answer.)

Well, the HBS survey was one of those incredibly hard to fill out surveys — specific numbers of answers required here and there, not much thought put into usability. So finally, I wrote Bill Sahlman, the senior dean for external relations, since he had sent the original email.

Dear Bill: I tried to take your survey but never succeeded, due to error messages. {Screen shot of error message was included.}

I am a conversion scientist and web analyst. When you require people to answer questions, you lose them. It is even worse when they WANT to answer the question and can’t find the answer.

I wasn’t important enough for Bill to answer personally — despite how important I tried to make myself sound –  so someone named Courtney Fanning wrote me back.

I am replying on behalf of Bill Sahlman. The error your received is likely due to the limit of options. For some questions, you can only check 2 options. IF you check more than 2, it will not allow you to continue. Thank you for taking the time to complete the survey.

Sorry Courtney. I never took the survey.

End notes: I promise to go back to custom filters for Google Analytics later this week. Many thanks to Stephan Hamel (again) who updated this blog to www.lunametrics.com/blog in three places (so I can thank him at least twice, right?), and to Eric Peterson, who updated us as well.

Do you ever start to leave question and then delete it?

(Technically speaking, Bryan Eisenberg posed this question to me in a bar two weeks ago. But he was really talking about user engagement, and I am talking about getting customer service sites to convert.)

So it is Saturday, and I am supposed to be doing financial stuff for my company and my family. Quicken and Quickbooks are not always my friends, though. For the first time, I decided to leave a comment about how unhelpful the Quicken help can be. I was absolutely shocked to click on the “Did this help you?” button and get this picture:

… with the words, “Hi, we’re responsible for your feedback.” And it really made me feel like someone was going to read and care about it. I especially love the way they did the photography, it was just a group photo of colleagues. Prettier and more professional isn’t always better, I think (would have loved to see the results of that test.)

Robbin
LunaMetrics

Shopping carts should make customers feel secure about more than just their credit cards. One of the jobs of a shopping cart is to make the customer feel that he is actually paying for the product he thinks he selected.

I continually have this same problem (i.e. website-induced doubt) with shopping carts — either they don’t tell me what I am buying (so I wonder, “Gee, did I click the right button?”) or they give me just enough extraneous information to put some doubt into my mind.

For example, about a month ago, I got the Quicken 2007 upgrade in the mail. It was time to upgrade, and so I started to. But when I got to the last screen, I got this screenshot:

I probably would have been okay with the “unlock” of Quicken 2007, if it weren’t for the free download they threw in of Quicken Willmaker. As soon as they called Willmaker a download, I knew that the other — the one I really cared about — wasn’t a download. In fact, only when I started writing about it that day did I figure out that I was only getting an “unlock” because I already had the software in my hands. I had to think too hard — they needed to call it an “unlock code for your software upgrade” or something more descriptive.

Here, this one is even worse. I was making my registration at the hotel for the Emetrics Summit. It looks like a gorgeous hotel on Nob Hill, so I really wanted to stay there. I just didn’t want to spent $900. So I’m going to share a room with someone, to cut costs. But, when I got to the last screen, I was told that I would pay $40 per extra person per night. Eventually, I made my reservation the old fashioned way, and the customer service rep told me that the extra person charge applied to rooms with three or more people. So either the hotel needed to remove that information, or make it clear that the information only applied to rooms with 3+ people.

Robbin Steif
LunaMetrics

My daughter complains that I never blog about her. Well, here goes:

I watched her try to buy a Smashing Pumpkins poster. Actually, I wasn’t really watching, I was just doing my work, all snowed in here in Pittsburgh, and listening to her yell at her computer.

First, she tried on Amazon (or more accurately, was led to Amazon at every turn.) “What happened?” she asked. “I just clicked on the button to go to my shopping cart and it says that I bought it, I don’t even know what I paid.” So I went over to her computer, only to see that she had accidentally used Amazon’s 1-click. This would not have been a problem until I pointed out that she had paid as much in freight as she paid for the product. She started to scream (my kids get big allowances every month but have to buy all their own stuff - clothes, gifts, entertainment, so they really care what happens to every dollar.) We went to the FAQs and I showed her how to cancel her order.

“I don’t want to buy it on eBay,” she claimed, “I don’t have money in my PayPal account.” Well, I explained, PayPal no longer requires that you wait 4 days to transfer money in. So she started down the eBay road.

First, she had to register for eBay. Then she clicked through to PayPal, but PayPal insisted that she sign up for their credit card (you have to have really good eyes or use it a lot to notice the “continue without getting a PayPal card” link. Then she couldn’t remember her PayPal password, so they emailed it to her. At that point, she either had no way to get back to her transaction or couldn’t find her transaction. But it didn’t really matter, she was so angry that she was ready to break the computer screen.

So what could everyone have done differently? Well, Amazon’s famous one click may work well for them, I wonder how many customers they alienate. (You can turn it off but you have to know to do so.)

eBay definitely needs to find a way to keep your transaction “live” or in a separate window while you process all their stuff. And PayPal had better be doing one heck of a job signing up credit cards that people are actually using, or they should just get rid of that screen and improve their user experience.

Anyone have a new Smashing Pumpkins poster they want to sell?

Robbin Steif

It is always hard to know how many options to give a user. Too many and you lose the conversion; too few and they can’t achieve their goal (and you lose the conversion.)

VistaPrint is a well-known Internet printing company, and is arguably best-known for their business cards. I clicked first on the product I wanted, “premium business cards” on the home page and landed here:

So this was the first moment of frustration. It didn’t matter whether I scrolled up or down, I couldn’t find any action buttons except the opportunity to upload my artwork or choose one of their templates. (And I really wanted them to tell me that they would walk me through a process.) But anyway, I uploaded my file (using the link that I have the red arrow pointing to) and was very frustrated to see that they had taken my vertical card, assumed that it was horizontal, and then yelled at me (electronically) because the aspect ratio of my image did not match the “chosen template.”

Eventually I realized that I had landed on tab 3 of a four tab wizard, as you can see below:

Finally, by making them back up (i.e. choosing Tab Two), I got to tell them that I wanted a vertical, not horizontal card:

While writing this post, I noticed that there *is* a business card wizard, right at the top of the home page. (I never saw it and it is a great example of ad blindness.) Even if I had, it is a “free business card” wizard, whereby you choose one of their templates and get their advertising. There truly is a way to do this right, just not a very intuitive way…

Robbin Steif
LunaMetrics

I just couldn’t resist showing this tiny piece of a non-profit website.

Wouldn’t a check box next to “Please don’t show my name in your printed thank you material?” do a better job? Now that I’ve looked at it 15 times, it makes perfect sense but when I first read it, I felt like I had to parse all the negatives.

Robbin Steif
LunaMetrics

Three times in the past week, I have tried to make a purchase from a (big important secure) website, and every time, the site makes me feel like they couldn’t understand my goals.

The first site was iTunes. I wanted to buy a gift certificate, and so went right into my own iTunes setup and choose Give an iTunes gift from the iTunes store. But in the shopping cart, they made me sign in. Hmm, I thought, do they think I am buying something for myself? Are they going to charge my personal credit card, when I need to use a different card? And shouldn’t my grandmother be able to make this purchase without owning her own copy of iTunes? So I went to the Apple site and succeeded in purchasing a cerfticate there — but all they needed to say was, please sign in and you will get a chance to choose a credit card.

The second site was NetFlix. I wanted to buy a gift certificate (again), but they took me all the way to the screen where I had to sign in (like iTunes) and even told me that when I hit enter, the sale was final. However, they still hadn’t given me a chance to tell them where to mail the DVDs to, so obviously they were going to send them to me, right? I put this one off for two days and finally went back to it. Eventually I realized that the individual who gets the gift chooses when to redeem it and where to receive the DVDs, so I got back on my computer (as if I ever leave it) and finally made the purchase. (I also submitted the suggestion to them — after all, they are offering $1M for a better algorithm, maybe they are paying conversion scientists??)

The third one was PR-Web. I used to do a lot of work with them and for whatever reason, haven’t lately. Maybe their interface has changed and maybe I just don’t remember it, but once again, they made me do things in the “wrong” order without giving me any reassurance. In this case, they insisted on taking all my credit card information without telling me that “On the the next screen, you will have a chance to upload your press release.” I finally just did it in their backward manner because they do tell the customer that s/he can give them funds to sit there unused (like a no-interest checking account) — I figured I’d have a way to use the money even if I didn’t get a screen for the press release.

Just think - all I wanted was a little reassurance.

Robbin Steif
LunaMetrics