eMetrics Summit: Eric Peterson vs. Matt Belkin

/

Here’s what Omniture‘s Matt Belkin said about the presentation given today at the eMetrics Summit by Visual Science’s Eric Peterson.

Eric’s presentation, which I missed in large part but caught the end of and the Q&A, was centered on WA methodology. It was called, The Business Process in Web Analytics. Here, you can read much of the presentation yourself on Web Metrics Guru.

After the presentation, it was break time and I wandered into the vendor area. I started talking to Matt Belkin, VP of Omniture’s Best Practices group. As part of our conversation, he showed me his computer, where he had typed, “Lack of methodology is responsible for the failure of web analytics.”

“That’s what Eric thinks,” Matt said, referring to what he had typed on his computer. “Well, what do you think?” I asked. Matt started to speak, so I grabbed a piece of paper and scrawled down his answer while Bill Bruno from Stratigent stood and laughed at us (Bill was the source of a post I wrote at the Summit last April, so he knew exactly why I grabbed a notepad.)

Matt opined that for many companies, looking at web analytics is like drinking from a firehose – they are drowning in data and can’t figure out what is actionable. He thought that analysts should find quick wins (“I saved us $30K on Google Adwords with the use of our web analytics last quarter,”) and that with those quick wins, the rest of the organization would quickly get on board the web analytics train. He also pointed to the question that Megan Burns of Forrester Research had asked at Eric’s presentation. Her question went something like this, “If organizations codify their web analytics process, won’t it be a big book that no one looks at and quickly becomes shelfware?” Matt really agreed with that shelfware problem and told me I could quote him on this all. (I hope I have captured this well, despite my lack of a tape recorder.)

Since I hadn’t attended Eric’s entire seminar, I didn’t think it would be fair to quote his words out of Matt Belkin’s mouth. So I went back to Eric and read the sentence to him. Eric changed it to be, “Lack of methodology has contributed to the failure of web analytics.” And he strongly disagreed with Matt’s thesis. He pointed out that his seminar was in an enormous room and was filled – if “easy wins” were so easy, why was everyone in his eMetrics session instead of the other sessions? (It’s true, the room he spoke in was enormous and full.) “Well,” I countered, “You’re a fabulous speaker. [He really is.] And you’re Eric Peterson.” Or as one of my friends pointed out a little later in the day, “How can you go to the eMetrics Summit and not listen to Eric speak?”

Late tonight at dinner, Jim Sterne listened to me point out the two sides. “They are both right for different companies at different times,” he said. “There are huge quick wins to be had for instant ROI. But eventually they peter out, and then you need a good methodology so that you can maintain continuous improvement for your site.”

Robbin Steif
LunaMetrics

Our owner and CEO, Robbin Steif, started LunaMetrics twelve years ago. She is a graduate of Harvard College and the Harvard Business School, and has served on the Board of Directors for the Digital Analytics Association. Robbin is a winner of a BusinessWomen First award, as well as a recent Diamond Award for business leadership. You should read her letter before you decide to work with us.

  • Daniel Waisberg

    Great post! Very interesting.

    How I see the way they think: both Eric and Matt speak for VERY high priced products. Those products (or the people who buy them) have to justify their price.

    Omniture sells for all kinds of companies and different levels of skilled users. So the way to justify the product fast is to first take the hanging fruits and then start planning.

    Visual Sciences sells only to high skilled users, companies which usually have a culture of Web Analytics. So re-organizing and planning is usually the most appropriate for the users.

    Well, this is the way I see/explain the difference in approaches. Not that it matters that much…

    Be well
    Daniel Waisberg

  • Andrew

    Robbin,

    This is a wonderful summary! I enjoyed your recap of the comments. I do not really understand Eric’s side; can you explain exactly what he means by methodology? Is it continuous improvement?

    Also, I’ve read both Eric and Matt’s blogs and there is clearly a difference in their opinions and style. It seems that Eric likes to talk more about the industry and generic themes. Matt is more focused on the customer viewpoint. This seems to mesh well with their backgrounds. So perhaps both are right!

    I will admit that I find myself relating more to Matt’s comments because I’m a customer of analytics not a research analyst that cares about industry stuff. But that’s me and I appreciate both Matt and Eric for their contribution.

    Thank you again for the great post, and I’d love to hear more about exactly what Eric meant.

    – Andrew

  • benry

    We’ll said Robin. I understand Matt’s position being someone who has struggled with the operationalization (if that’s a word) of Web Analytics and getting organizational buy-in. Personally I think a good process, that gets you a couple quick wins, but that can be later leveraged for ongoing improvement is where I’ve sen the best uptake.

  • http://www.lunametrics.com/blog LunaMetrics Blog

    Ah, I am so sorry. It is 10:55 pm, I have another post to get out and a presentation to get ready for. Eric wrote on his blog, http://www.webanalyticsdemystified.com/ that he will have a copy of his presentation up there soon. Please forgive my lack of time and heavy eyelids!! Robbin

  • Derek M.

    Jim’s right. If you have a well-designed site and you have implemented many best practices then you’ll have fewer quick wins. But if you’re at a relatively immature company with a mediocre website, then you should definitely go after the low-hanging fruit. But if you never put together a process then you’ll be reactive rather than proactive and as every analyst knows, that’s often the worst situation to be in.

    I’m not sure what Megan means in her reference to shelfware because creating a business process doesn’t mean that you have to create a lengthy document. You just need simple rules for who to talk to, what to talk about, and when to talk about it–like when you determine reporting requirements for new site features. You want to include analysts in the design process so that they can foresee any behavioral elements that will need to be tracked.

  • http://www.lunametrics.com/blog LunaMetrics Blog

    I talked to Eric today while I was writing my Visual Sciences post, and he promised to get on and answer Andrew’s question. Derek, I hope he speaks to your points too, and some of these other great comments as well. I did hear the in-person answer to Megan’s question, and I was disappointed that the answer wasn’t, “Make the process document one page long.” But then, it’s hard to answer questions on the spot exactly the way you want to.

  • Eric

    Daniel: I actually gave the presentation as the author of Web Analytics Demystified (not that it really matters) but it is my strong opinion that the amount of money you spend on analytics is IRRELEVANT if you’re not thinking about how to make you’ll actually USE the application inside your organization. It is just as important for Google Analytics users (or Analog, or ClickTracks, or …) to consider how tags are deployed, how reports are generated, how analysis is done and distributed and how changes are measured.

    Derek: Megan Burns referred to her experience with CMM and software engineering and her concern that the Web Analytics Business Process diagrams I’m advocating for would gather dust. Again, my response is that until we’re all so good at using web data inside our orgainzations to drive improvement that considering process is tremendously valuable. Your comment is SPOT ON about the difference between being “reactive” and “proactive” …

    Andrew (and everyone): I have pushed a slightly modified version of my PPT out at my web site that you’re welcome to download. Check it out at http://www.webanalyticsdemystified.com/wabp.asp and consider following my weblog at http://www.webanalyticsdemystified.com/weblog/

    Robbin: Thanks for your following this. I really do appreciate the nice compliment you paid me in the original post and I’m always happy to discuss these processes with you here in your blog or over in mine. 😉

  • June Li

    Robbin,
    Great post – glad you caught some of Eric’s session. Thanks for adding Jim’s comments as well. (And if my Internet connection worked in my room, I would have read your post and we could have discussed this in person!)

    Process is everywhere, regardless of what you name it. Eric is suggesting that the process be more consistent:
    – To select a Web analytics tool, one follows a process. It may not be the best process, but it was a process. It probably won’t be the same process the next time the next WA tool is selected, because you would have learned something from the first experience.
    – To successfully identify winning opportunities (quick or not so quick), there has to be a process, with a staff and structure to uncover them. Quick wins don’t just pop out of the air.

    More posted on my blog.

    It was great to see you again Robbin!

    ~June

  • Chris Deringer

    Robbin,

    It was fantastic to pick your brain and make your acquaintance yesterday.

    This is a great summary that has led to a great discussion. As a web analytics writer, I’m a neophyte, but as a practitioner I’ve got nearly 5 years of experience. I’m in agreement with Eric’s perspective on this. If your company/client has never used web analytics, its easy to create immediate gains. However, once the organization gets even a bit sophisticated with analytics, you’ll quickly exhaust those opportunities. After reaching this maturity point, the value that you can provide the organization is entirely dependent on the degree to which you can infuse your needs/processes/requirements within your organization’s. I don’t consider our company to be at the cutting edge of web analytics, but we no longer change any important flow without A/B testing the proposed changes–being wrong is too costly, and testing is too cheap and easy. To have reporting requirements and a methodology embedded in the company’s most basic processes is incredibly important for an analytics team to pro-actively add value in my opinion. While I don’t have a diagram that shows how our processes integrate with our organization’s, I do get to sign off on any project/release that touches our retail Storefront. We formally submit any reporting requirements we need to measure whatever changes are being made to our site just as finance or Engineering does. We even create formal test plans that we can sign off on with our Marketing team prior to a release, so that we have a pre-set framework for measuring the changes.

    I also agree with Matt (although my opinion here is primarily informed by hearing what others do, since I am only intimately familiar with our company’s analytics’ maturity) that it sounds like most organizations don’t get nearly enough out of their analytics tools. I think this says alot about the industry in general and little about companies that are doing analytics well. My assumption is that the people in the audience at Eric’s presentation (it was compelling) are in the last group and that the content was tailored to us.

  • René

    Robbin,
    It was great meeting you at last at eMetrics. Thanks a lot for sharing this with the world (even if the WA world is still very small ;-)). Thanks for your presentation also on Wednesday (I was the frustrated marketeer ;-)). Remember your CrazyEgg screenshot of your blog where you saw that people was clicking (unsuccesfully) to the post titles? I’ve just made exactly the same ‘mistake’! If I may share my experience: I was clicking to a post to have it in one page with the comments underneeth and to be honest I felt frustration when it didn’t work… Then I remembered your slide at eMetrics and smiled. I had to get down at the end of your post and search for the link comments (I’m still a bit jet laggued and it took me a few seconds to find it). Just to share with you you my personal usability test, and don’t worry I won’t be asking you for a 20$ Starbucks gift certificate 😉

    Keep up the good posts and please pay us a visit when you next come to Europe.

    René
    http://www.ox2.be

  • http://www.lunametrics.com/blog LunaMetrics Blog

    Hi Chris – we talked and talked and I never even understood that you had been at the Summit as well. (How did I miss that?) Most people are really coming down squarely in your camp, process is necessary. Speaking of people not getting enough out of their tools (my usual soapbox): On the one hand, I love working with Big Software, on the other hand, I have a hard time justifying it if the customer can get what they need out of Google Analytics.

    Rene, you are right, Blogger really sucks. It’s not your jetlag.

Contact Us.

LunaMetrics

24 S. 18th Street, Suite 100,
Pittsburgh, PA 15203

Follow Us

1.877.220.LUNA

1.412.381.5500

getinfo@lunametrics.com

Questions?
We'll get back to you
in ONE business day.