Ever since starting at LunaMetrics, I’ve become increasingly immersed in and enthralled by Google Analytics. I thought I knew a lot about it before, but every day I find out about a new way to mine and organize the data I need on a day to day basis. I also think it’s interesting that the more one knows about how to do things, the more one comes to rely on the resulting reports.

My recent find, thanks to my coworker Jim, is a unique use of the new secondary dimensions and pivot function.

Keyword Research and Google Analytics

There are a number of ways to do keyword research using Google Analytics, but until the addition of the secondary dimensions, there was no easy way of extracting actionable data from keyword reports generated by Google Analytics.

For instance, have you ever done a GA keyword report, found an unexpected keyword in the top ten and wondered where in heck the user found your site by searching for it? Well, now you don’t have to wrack your brains and search through every single search engine ever created.

Lets say you are doing keyword research. You want to find out the following:

• What keywords are bringing users to the site
• What landing pages are those keywords bringing users to
• What source is directing traffic through those keywords (Google, Yahoo, Bing, PPC Campaign etc.)
• How qualified is the traffic that is coming from those keywords

Generating the GA Keyword Pivot Chart

All of these things can be found in one easily generated report. Here’s now to create it:

1. Go to Traffic Sources in the Dashboard and choose Keywords
2. Refine your report by clicking “Non-Paid” so that you only see keywords used in organic search
3. Choose “Pivot” From the available views. (This is where it gets good!)
4. Keep the default Pivot selection “Source”
5. Keep the default metric “Visits”
6. Add “Bounce Rate” from the adjacent drop-down.
7. Keep the dimension “Keyword” then add “Landing Page” next to it.

Voila. You have a list of keywords, what landing page they go to, where the visitors came from and bounce rates or whichever metric you’re most interested in.  If you get a lovely list of branded search terms, use the advanced filters to exclude those keywords. It’s much easier now that you don’t have to type in all the regular expressions anymore. Here’s a screenshot of what the final product should look like:

There are many things this information can tell you among which are:

• Conversion Stages: To see what stage of the buying process users are at when they reach a certain page.  Long tail keywords usually denote a readiness to buy/convert whereas shorter phrases typically indicate a research phase.
• Traffic Quality: If you’ve done your homework right and you’re drawing traffic in through keywords that you’ve optimized for, but you’re lacking conversions or continue to have a high bounce rate, you could be going after the wrong keywords.
• Usability: Once you know what keywords visitors are using to get to your site,  you can tailor the pages to fit their goals.
• Keyword Potential: There could be keywords that you never though of in this list. Take them into account when setting up an PPC campaign, or deciding how to further optimize the page.

I hope you found this report as useful as I find it! I’d love to hear about other uses or variations. Feel free to comment!

There is one particular mistake I see over and over with regard to setting up filters in Google Analytics.  This has been talked about before, but it comes up so often that I don’t feel bad about addressing it again (and again, and again).

The Mistake

Multiple Include Filters, causing no data to get into the profile.

What it looks like

1. Include – Request URI – /page.html

2. Include – Request URI – /other.html

What is intended?

The person that adds these 2 filters to a profile wants to include both /page.html AND /other.html.

But why doesn’t that work?

The Technical:

When GA processes filters it does so, sequentially, one at a time.  Filters are like a series of gates, in-line, along a road. In order for data to get from point A to point B, it has to pass through each gate in turn. If there is any one gate that it can’t get past, then no point B and it doesn’t end

up in the profile.

Each Include filter can be thought of as an “Include Only” filter.  Let’s look at the 2 example Include filters above with that wording in mind, and see how far our data makes gets.

Data –> A) /page.html, B) /other.html, C) /index.html

1. Include – Request URI – /page.html
2. Include – Request URI – /other.html

The first thing that happens is our data is checked against Filter 1.

Filter 1 says “Include ONLY data that has a Request URI that matches /page.html”
Data A matches /page.html
Data B does not match /page.html
Data C does not match /page.html

Now we have
Data –> B) /other.html, C) /index.html   -> didn’t make it past the filter
1. Include – Request URI – /page.html
Data –> A) /page.html
2. Include – Request URI – /other.html

The only piece of data that made it past Filter 1 was Data A, /page.html

Now it is tested against Filter 2, which says “Include ONLY data that has a Request URI that matches /other.html”

/page.html, is tested against Filter 2, /other.html.  It does not match, and ends its trip short.

So, no data in our set can possibly make it past both filters, since there is no single page that can match BOTH /page.html AND /other.html

The Solution

The solution is to use a single filter that looks like this:

1. Include – Request URI – /page.html|/other.html

We put both pages that we want to include into a single filter and seperated them with a pipe.  The pipe is the vertical bar symbol usually located just above the Enter key.  This is a Regular Expression (RegEx) symbol that can be read as “Or”.  So the filter says Include ONLY pages that match either /page.html OR /other.html.

Google announced lots of new features last week, and one of the most exciting is Advanced Segments. There are already some great posts out there in the analytics blogosphere explaining what Advanced Segments are all about and how to use them (here’s one from Justin Cutroni, and one from Avinash Kaushik). But we wanted to take a few minutes to compare and contrast using Advanced Segments vs. using profiles and filters to get a different view of the same site.

Up until now, there were some reports in Google Analytics, like the Visitor Loyalty reports and the Funnel Visualization report, that you could only segment by creating a new profile for your site with filters on it to get just the traffic you want. For example, do you want to see the Visitor Loyalty report just for your paid search visitors? Create a profile that only includes your paid search traffic. We’ve done lots of posts about filters in the past detailing how they work to set up exactly these kinds of things.

With Advanced Segments, though, many of these reports can be segmented. Here’s a screenshot of a segmented Visitor Loyalty report. (The Funnel Visualization report, sadly, remains un-segmentable.)

And the real upshot of using Advanced Segmentation is, you can look back historically on the data! With profiles and filters, the changes you make only have an effect from the time you make the change going forward. With Advanced Segmentation, the report instantly shows you the segments for the historical data you have.

Advanced Segmentation has several other advantages as well. You can see multiple advanced segments simultaneously in the same report.

You can also create advanced segments that wouldn’t have been easy or even possible with filters, such as segments for visits with metrics in a certain range (more than 3 pageviews, more than 2:00 on site, and so on).

And finally, the interface for creating advanced segments is much more intuitive than creating filters — you don’t necessarily need to understand regular expression to create most advanced segments you can dream up.

So given all this, who even needs to use profiles and filters any more? Forget them! Well, not exactly. There are still several very good reasons for using profiles and filters. One is for kinds of traffic you almost always want to exclude from your reports, such as internal traffic from your organization to your website (you don’t want your employees counted as visitors in most cases — and IP address isn’t a dimension you can use for creating Advanced Segments anyway). You’ll also still want additional profiles to handle more than four goals. And lastly, you can manage access to profiles with the User Manager, so it’s easy to keep track of who can see which sets of data — you can assign different users to different profiles as either read-only users or administrators. (Advanced Segments, on the other hand, are tied to a particular user’s login, like the Dashboard or email preferences. Each user has their own set of Advanced Segments that are available on whatever accounts or profiles they have access to.)

So here’s a summary of the differences between the two approaches…

Advanced Segments:

1. Segment previously un-segmentable reports (except Funnel Visualization), including historic data.
2. See multiple advanced segments at once in the same report.
3. Types of segmentation that weren’t possible with filters: visits with conversion, visits with more than 3 pageviews, visits that spent more than 2:00 on the site, etc.
4. More intuitive to set up for non-technical users.
5. Tied to a user login.

Filters & Profiles:

1. Still useful for filtering certain kinds of traffic you almost always want to exclude, like your internal traffic.
2. Segment the Funnel Visualization report.
3. Use multiple profiles for more than four goals.
4. Tied to the Google Analytics account, manage access with the User Manager.

- Jonathan

On a post way back in April on Custom Advanced Filters, Idris left a comment asking about the required/not required selection (seconded by Paul):

Hey, great articles. I am trying to do some advanced filtering, but I’m confused by the “Field X Required” option. If I say “Yes” to the requirement, which of the following two things does that mean?

a. If the regex in this field does not match, do not include this hit in the profile at all.

b. If the regex in this field does not match, skip this filter, and move on to the next, but still include these hits in the profile.

These two are obviously very different. Which does Google Analytics do?

The confusion is about what exactly is “required”. We were pretty sure we knew, but we did an experiment to confirm. It’s basically b from what Idris suggested.

Here are the details:

1. If the field is required and the regex matches, the output is written to the field you select.
2. If the field is required and the regex does not match, the output is not written to the field you select.
3. If the field is not required, the output is written to the field you select regardless of whether the regex matches.

In no case are the pageviews excluded from the profile entirely (you need an exclude filter for that). The filter just doesn’t apply if the field is required and it doesn’t match.

I have been working on this series, Filters for Google Analytics, for almost six months now. This is the last part of the series I am going to write (at least in the foreseeable future.) You can get the full thread at the bottom of this post.

So once again, I will start with the question a lady asked me on Friday morning: how does one create exclude filters that function as if they were AND filters? How, she asked, can I create a profile and put on it two different exclude filters that work together: Exclude the visit if the visitor is a new visitor AND her language is German? We can’t do that with two separate exclude filters, because as soon as the first filter works, the visit gets thrown out. Exclude filters are, by nature, “or” filters.

(If you are a little lost, and don’t remember how multiple exclude filters work like OR filters and multiple include filters working like AND filters, and how you have to work to make them do the opposite — see this post that I wrote recently.)

Well, I had to think about this one, and then realized that it was just a great practice for this series on filters for GA. The reason why: It is both a rewrite (custom advanced filters) and an exclude (custom filter). Here is the overview: you rewrite the two fields into one field, and then you do an exclude on that one field, making sure that it matches both pieces of data.

If that was clear as mud, or you are more of a step-by-step “please show me” person (like I am) — then this is a good place to keep reading.

First, you use a custom advanced filter to rewrite the data so both pieces of info — the visitor language and visitor type, which is GA-Speak for “new” vs “returning” visitor — are in the same field. Like this:

Now we have both pieces of data in the same field. If we could look at them, they would look like this, concatenated together on the same line.

1. en-us/New Visitor
2. de/Returning Visitor
3. da/New Visitor

and so on. They would be the language code followed by a slash and the visitor type. (Right? That’s the way I set it up, $A1 captures everything in visitor language settings, then a slash, then $B1 captures everything in Visitor Type.) Above, I choose three languages for my examples: English-US (en-us), German (that’s de) and Danish (da).

Next, we create an exclude filter. The hypothetical example was to exclude a new visitor whose language is German – the two excludes that must work together. This is the heart of our problem, exclude two different kinds of variables. So here is our exclude:

So what does this say? It say, go to Custom Field 1, where we now have a list of concatenated languages and visitors types (en/Returning Visitor, and so forth). If one of those lines in the list matches de (which stands for German – Deutch, right?) and has a slash and then the word new, it’s a match, so please exclude it.

And that’s how you can exclude two different fields in Google Analytics at the same time.

If you would like to read all the other posts in this series:

• Part 1: Get ready with profiles and ReGex
• Part 2a: Filters for GA, What have we here?
• Part 2b: Why does anyone use filters?
• Part 3a: Easy Filter Implementation
• Part 3b: Custom Implementation
• Part 3c: How do you filter those utm links you create?
• Part 3d: When do you use them?
• Part 3e: Include vs Exclude filters, how do they work?
• Part 4a: Custom Advanced Filters
• Part 4b: More on Custom Advanced Filters
• Part 4c: Cascading Custom Advanced Filters

I spoke about implementation Google Analytics on Friday, and someone asked me, “How do I create Exclude AND filters?” This is a cool question, but it has to wait until I write the info below on how Google applies your filters (and how that differs on Include vs Exclude filters.) Note: you can read the GA documentation on this one, which is not terrible.

You might consider include multiple filters to be AND filters, when you think about the way that GA applies all of them to a profile. So if you create a profile where an include filter specifies Visitor City = Pittsburgh, and the another include filter specifies Campaign Source = organic, you will filter in only visits that came from people in Pittsburgh who used the organic search to find you. It’s not good enough that they came from Pittsburgh and used paid search, or followed a link. Both criteria must be met.

Exclude filters, on the other hand, could be looked at as OR filters. If the visit matches any of the exclude filters in a single profile, it is thrown away. So using the same example as above, Visitor City = Pittsburgh, and the Campaign Source = organic, you will exclude anyone who comes from Pittsburgh, no matter what their campaign source. And you will exclude everyone who comes on an organic search, no matter what their city.

There are ways around the AND and OR issues. For example, let’s say that I wanted to include everyone from Pittsburgh OR Raleigh. I cannot do it with two separate city filters — right? Include filters are AND filters, so two separate filters would say: include everyone who comes from both Pittsburgh AND Raleigh, which is a pretty hard act. But you certainly could do include visitor city = (pittsburgh)|(raleigh). By putting both cities in the same filter with an OR pipe, you achieve that goal.

Now that I showed you how to defeat the “ANDness” of Include filters, you will be wanting to know, how to do defeat the “ORnesss” of Exclude filters. Plus, you might want to know how to make include filters be OR filters for different kinds of fields — include if they came from Pittsburgh OR they came organically.

But that is a different topic and a new post. Maybe even tomorrow.

Before I start (again) – this is the last reminder that the Criticize GA’s Documentation Contest ends on Tuesday the 26th. Which is probably today already if you are in Australia.

I have been doing a series on filters for Google Analytics, and am almost done. I have two more posts coming up, and then it feels like I have finished.

However, I read the Google Groups for GA and I can see that the combinations of problems people have are always amazing. So if you have a filters question that I haven’t answered yet — and it is not about cross-domain work, since that will be a different series — please send it my way. You can comment or send email to my last name at lunametrics.com — whatever is easiest for you.

Robbin Steif
LunaMetrics
(see, there is all the info you need to send the email)

This is part of a series I am writing on filters for Google Analytics (and when I am all done, I promise to thread them all.) This current entry is a particularly beautiful capability of GA, but you might want to first read about Custom Advanced Filters if you aren’t terribly familiar with them (my most recent post in this series.) I will do just the tiniest review, else this will get too long.

Just the Tiniest Review: In a Custom Advanced Filter, you get a field A and a way to extract it into a Regular Expression, and if you like, a Field B and a way to extract that one into a RegEx, too. Then you can play with them — put them together, for example. That’s what output to constructor (the third area) is all about.

New: But what if you wanted to put together more than two fields? That’s a job for Superman Cascading Custom Advanced Filters. BTW, I made up that name. I haven’t found that one in the Google Analytics documentation, but I don’t know what to search under. (Now, if I could enter the contest, I would submit this entry. But the rules are, I can’t win.)

Back to business: so, let’s say that every day your boss wants a report that tells her what keywords people were searching on, what was the source, which ad they clicked on, how many pages they looked at and did they convert? None of this is hard in Google Analytics – the hard part is getting the information easily and quickly. What do you do when you have 50,000 keywords and you need the information for your boss every day? What do you do when your boss wants that report to also include geographic location and browser size?

(Note: Caleb Whitmore from POP, who taught me how to do cascading custom advanced filters and gets a ton of credit here, calls these Insane Filters, because, “who has time to look at 50,000 keywords every day?” And in fact, I have made my example here pretty simple compared to the one that I did for a customer last week, which included seven filters.)

So remember our fairly simple example: we are looking to get keyword – source – Ad Title – pages/visitor – conversion rate.

We start by creating a filter that will associate keyword (Campaign Term) with source, and we’ll dump in into Custom Field 1. If you are now a little lost , please read this post on how to create Custom Advanced Filters. Here is a screen shot:

Now comes the “cascading” part. In this second custom advanced filter, we pick up Custom Field 1 as our A, we pick up the next thing we need, Ad Title, also known as Campaign Content, as our B, and we associate them in the constructor, but this time we put it into its ultimate resting place.

So what do I mean by that? With the final filter, you decide which report you are going to plop it all into — that’s what goes in the “output to” field. The decision on a report is important – the report you choose will give you specifics you might need. For example, if you want conversion rate, don’t drop it into your Top Content report (because you don’t get conversion rate there.) I have been using Visitor > Languages for this (but even if you have a seemingly obscure report like languages, you should create a custom profile for this work.) Notice that I didn’t do any work on pages/visitor or conversion rate, even thought I need it — that info will come automatically with the report, if I choose it correctly.

When you get the report, the first column will look like this:

shoes,google,Buy Red Shoes for $5

Right? First comes the keyword, then a comma, then the source, then a comma, and finally the name of the ad. This is how we associate more than two fields. The reason that I used seven filters for a customer last week was because they needed us to associate eight fields (i.e. you can keep doing this, and only in the last filter do you choose where to put the data.)

The other information (pages/visit and conversion rate) will be the same as always, in columns on the right. Plus there may be data you don’t care about, depending on the report you chose. Either way, all you need to do is export to Excel and voila! there is your report.

Tips and tricks: It is nice to use a comma in the constructor (i.e. like this: $A1,$B1) . That way, your file already has commas and separates nicely. However, depending on your data, you might have other commas get in the way. For example, just think of the mess you would make if your ad title was Shoes, Sandles, More — each of those would go into a separate cell in Excel. So a better way to go might be with a separator that isn’t used as often, maybe a slash or a star * or …. well , you get it. There is probably a way to get Excel to recognize those, but I generally have to export to a notepad and tell the notepad the name of the delimiter.

Also in the tips and tricks area, there is no reason that you can’t dump the whole thing back into your final field with each filter. So if you report is going to end up in Languages, you could always have every constructor go to Languages. But my advice is to resist that temptation and use the custom field until you get to the last filter. It’s a pain to get all your work done and start to get data, only to realize that the report you chose doesn’t work. If you use Custom all along the way, you only have the change the last filter.

Don’t forget that you can change the order of filters. This might be very important — you might need a different layout of your excel spreadsheet.

And finally — if you learned something here, please submit an entry to the Criticize Google Analytics Documentation contest. You have until June 26.

Endnotes: Thank you to Michael Harrison for telling everyone about the contest. Hearty congratulations to OX2 on your GAAC status. Jeff and Nick and Brian and Helen, thank you for helping me with all my GA endeavors.

Robbin

Custom advanced filters can be very cool. You can use them to rewrite stuff. And much cooler, you can use them to associate stuff.

For example, you can use an advanced filter to associate a Request URI (a page someone asked for) with a Campaign Medium (the medium they used to get to your site) and then dump it all back into Request URI (so that you can see it in your Content > Top Content report.) Before I go into the technical details, you may be saying, why would I ever want to do that? Well, if you have 10,000 URIs and you want to look at them all in Excel, it would be nice to have a report that you can download into a csv file and then sort any way you like.

So here is a screenshot of what that filter would look like. (Ooh, many thanks to Steve and to our web designer, both of whom gave me the same advice on better screen shots. So now this one is legible:)

Notice the A and B stuff in the middle. Here is what I am telling Google:

My field A is Request URI. Get everything, by using .* and put it in a variable, by using parens, so get-everything-and-put-it-in-a-variable looks like (.*)

My field B is Campaign Medium. Once again, get everything and put it in a variable, hence (.*)

Now comes the magic. Output it all to Request URI. And format it so that I see the A variable (that’s why I used $A1) and then a colon, and then the B variable. That last field, constructor, is not a regular expression. You can write anything you want there — I could have written $A1/$B1, or I could have written, $A1 AND $B1, etc. Also note: in the A field, there is only one variable, (.*), so I used $A1. Same for B. But there might be two variables, and then I would use $A1, or $A2, etc.

Choosing where to output your new mashed up string is important. For example, if you want to see, how many people came through a banner ad medium, touched the home page, and then converted, you wouldn’t want to use the filter I just created — even though it creates the correct information, a mashup of the medium they came in on with the page they touched. That’s became this particular example gets output to content >top content, a report that doesn’t include conversion. A better place to output it to would be campaign name. That way, it will get dumped into your campaign report, which has conversion associated with it.

If you are doing this kind of mashup, it is vital to create a new profile first. After all, you probably need most of the reports “unmashed” for some things, and if you do this in a separate profile, you can mess around all you want, while leaving your production data untouched. Need to learn more about creating new profiles?

Endnotes: Many thanks to Caleb Whitmore from POP, who taught me so much about custom advanced filters when we were at Google training. Caleb actually wrote this particular filter a few months ago. And to Dylan Lewis, for getting our new address right, www.lunametrics.com/blog and for letting his web analyst, Joy Billings, be co-chair of the WAA marketing committee. I have really awesome pictures of both Caleb and Dylan.

Custom Advanced filters are so cool, and there is so little information available about them. It’s too bad that they have such an intimidating name.

I find myself using them in three ways:

1. To rewrite stuff in GA. Usually, this it to rewrite a URI. For example, we work with a site where the CMS insists on calling the same page three different URIs, and we are using custom advanced filters (among other things) to rewrite them, so that we always know what page we are looking at.
2. To associate data that aren’t already associated. For example, Benjamin in SF wrote me and asked how to associate a referring source with a transaction ID. This is a job for Superman Custom Advanced Filters.
3. To change all sorts of other things (but which are mostly about #1 and #2.)

So let’s look at a really easy Custom Advanced filter — rewriting all your URIs to be Title Tags. (True, you can already see them in the Content Performance > Content by Title report, but this will rewrite them for every report.)

I want to teach two things before I start.

Thing #1: When I wrote about Regular Expressions, I explained how a dot means, match any one character. And I wrote about how a star means, match zero or more of the previous items. So when you put them together, they mean, match everything.

Thing #2: When you use parenthesis, it create a variable in Regular Expressions. Most of the time, I don’t care. But it really matters in Custom Advanced filters.

Putting Thing #1 with Thing #2: When you write this: (.*) it means, get everything and put it in a variable.

OK, now we are ready to start. Check out the screen shot below

First, I gave it a friendly name (“Rewrite URI, etc.”) Then I chose Custom filters, then within custom, I chose Advanced. As soon as I chose Advanced, I got all the other options below it.

Today, we are ignoring the middle set of boxes, the ones that say “Field B” and are just dealing with Field A and output. So everytime I talk about the A stuff, I am referring to the boxes that say, Field A –> Extract A.

Now, let’s sit back for a moment and think about what we’re doing before we do it. Our goal is to get the page title and to rewrite it — to reconstruct it — so that it shows up everywhere that Request URI might. So instead of seeing URI’s (urls – you can all fight about the correct way to say that), we’ll see page titles.

To do this, we first choose Page Title as Field A (just like we chose filter fields in this post that I wrote last weekend. You have to decide, what are you working on?) Then we extract it — we create a Regular Expression(RegEx) that describes it. In this case, our RegEx is (.*), i.e. get everything and put it in a variable (like I described early in this post.)

Next, we decide where we are going to put it. We want to output it to the URI.

Now, here is the magic (or at least, that’s the way it felt to me as I went through life, trying to understand what $A1 or $B3 was.) The first variable (the first set of parenthesis) in the –> Extract A field is called $A1. We only have one variable in this screenshot, but if we had a second one, it would be $A2. $A3 for the third one (if we had one), and so on. So when we use $A1 as our constructor, it means, use the first variable (.*) in the extract A field to reconstruct our URI.

I know that was confusing, so let me say it another way. Here’s what we did. We took the title tag, and rewrote it as a Regular Expression in the A field. The expression we used was (.*), i.e. get everything and put it in a variable. (So that means, we put the whole title tag in a variable.) Then we told the constructor fields to take the Request URI and rewrite it to be the first A variable — which is now defined as the whole title tag. Consequently, all URIs get rewritten as their page’s title tag.

Please comment if you didn’t understand anything. (I’m serious. I got on someone else’s blog today and said, I just don’t understand.) Or send me email to my last name at my company name dot com.

Robbin