Upcoming LunaMetrics Events
San Francisco, Apr 28-30 Los Angeles, May 12-16 New York City, May 19-23 Chicago, Jun 16-18

How To Use More Than One Include Filter (without losing data)

aka “Catching the Purple Squirrels”

Google officially recommends not to use more than one include filter, because it can lead (rather unintuitively) to excluding all the data in an Analytics profile.

“Adding more than one Include filter to a profile can cause data to not appear in your reports. To allow data to populate your reports again, we recommend assigning a maximum of one Include filter to each of your profiles.”

But sometimes using more than one include filter is the only way to catch the purple squirrels. Let me explain.

When One Include Filter Isn’t Enough

A reader sent us this question:

“I’d like to create a GA profile and to apply one filter to include only the SE organic traffic coming to a specific part of my site…. For example, all the traffic coming from SE organic to www.example.com/mydir/”

Unfortunately, even though he’d thought outside the box and (very creatively!) tried to use a single include filter based on Campaign Target URL, he was not successful.

In this case, a single include filter will simply not suffice. He needs one include filter for the traffic medium (organic) and another include filter for the set of pages visited (the specific subdirectory).

Why Using More Than One Include Filter Can Exclude All Your Data

So what’s the problem with using more than one include filter? Why does Google recommend against it? Here are two helpful tips to remember:

  • When it comes to filters, “include” means include only (i.e., exclude everything else)
  • When applying multiple filters, imagine the word “and” between them

If you remember those two things, then applying filters like the following will obviously exclude all your data:

“Include only squirrels AND include only dogs AND include only cats…”

It all boils down to the types of data you’re trying to include. Ask yourself, can members of one type be members of another type, or are they mutually exclusive?

Why Using More Than One Include Filter Can Be Okay

Our reader is trying to include two types of data that are not mutually exclusive. He needs to include data from organic search traffic and data from pages in a specific subdirectory. Viewed another way:

  • Data from one subdirectory is our set of squirrels (a different subdirectory could be dogs, etc.)
  • Data from one medium is our set of purple things (a different medium could be yellow things, etc.)

And here we want data that belongs to both sets, i.e. the elusive purple squirrels.

In this case we absolutely can use two include filters. We can say “include only purple things and include only squirrels” because then we’ll end up with the purple squirrels.

How To Catch the Purple Squirrels

First, the standard warning. Don’t apply new filters to existing profiles! Always create a new profile or use a test profile to “test drive” new filters and make sure they get the data you want.

Now create your two include filters. Navigate to the new/test profile, then go to the Filters tab and click +New Filter.

To include only organic traffic, create a custom filter. Choose “Include” and Filter Field “Campaign Medium” and Filter Pattern “organic” (don’t include the quotes). Click Save.

Then create another filter to include only traffic to a subdirectory. You actually have two options here: One way is to create a predefined filter. Choose “include only” and “traffic to the subdirectories” and “that are equal to” and then enter the subdirectory, such as /mydir/.

Predefined vs. Custom Filters

The drawback to the predefined filter is that it does not recognize regular expressions. (Say what? Check out our free e-book for an easy intro to regular expressions.)

So the other way to include only traffic to a subdirectory, and the only way if you need to use regular expressions, is to create a custom filter. Choose “Include” and Filter Field “Request URI” and Filter Pattern ^/mydir/.

In a custom filter I can use special characters from regular expressions, like the caret (^). The caret means “starts with”. So I enter Filter Pattern ^/mydir/ if I want to match pages like /mydir/abc.html but not /sub/mydir/abc.html. To match /mydir/ at any level, just leave the caret off.

And voilà! With an include filter for organic traffic and an include filter for /mydir/, our reader has a new profile containing exactly the data he wants.

But one more thing before I go…

Catching Squirrels and Dogs

Remember how I said you can’t include only squirrels AND include only dogs? We can actually work around that, using custom filters.

What I’ll do is create a custom filter that says “include squirrels OR dogs” because I can use a special character from regular expressions called the pipe (|). The pipe, or vertical bar (found above the Enter key on most keyboards), simply means “OR”.

To include traffic to /mydir/ or /mydir2/ create a custom filter, choose “Include” and Filter Field “Request URI” and Filter Pattern /mydir/|/mydir2/.

To match pages that have to start with /mydir/ or /mydir2/, change the Filter Pattern to ^/mydir/|^/mydir2/.

What are some other issues you’ve had with filters? Have you discovered creative ways to get the data you need? Let me know in the comments.

Dorcas Alexander

About Dorcas Alexander

Dorcas Alexander is a Digital Analyst working with Google Analytics. Her path to LunaMetrics included stints in ad agency creative, math, computer science, language technology research, and corporate training. She loves to learn and teach what she’s learned. One of the top-rated tournament Scrabble players in Pennsylvania, Dorcas has an insatiable drive to compete and win. “Impossible” is not in her vocabulary.

http://www.lunametrics.com/blog/2011/06/24/more-than-one-include-filter/

44 Responses to “How To Use More Than One Include Filter (without losing data)”

Cristina Chetroi says:

Great article! It is also worth considering that there is a 255 character limit on a profile filter, when trying to catch squirrels AND dogs :)

Dorcas Alexander Dorcas Alexander says:

Thanks, Cristina! Good point about that character limit. I’ve run up against it quite recently, trying to include a large range of IP addresses. Did not solve it satisfactorily, even using GA’s IP address tool.

Mitch says:

Just had to explain this to someone… agreed, turning it into an english statement often makes things easier for everyone. I usually tell people they can literally “read” their filters as follows:

Exclude = “do not show me”
Include = “show me only”
A subsequent filter (the cascading logic) = “, and of that group,”

So just as you pointed out, if you read three include filters in a profile (first squirrels, then dogs, then cats), the english statement would be:

“Show me only squirrels, and of that group, show me only dogs, and of that group, show me only cats.”

Dorcas Alexander Dorcas Alexander says:

That is a perfect explanation. Love it. Thanks, Mitch!

Nick says:

OK, with the 255 character limit and include filters, there is a way around this. Suppose you wanted to include Cats, Dogs, Mice, Horses, Sheep, Iguanas, Koalas, Hyenas but no other animals. What you can do is set up search and replace filters to replace, e.g. “Cats” with “Include/Cats”, and so on for all the other animals. Then setup an include filter to include “include”. Obviously you don’t want your reports to say “include/cats” etc so you then set up another search and replace to replace “include/” with “/”.

Dorcas Alexander Dorcas Alexander says:

Great idea, Nick. And it will work for any field that is available in search and replace filters. I thought about trying something like this for my IP address problem, but it’s not available (probably related to the Terms of Service which prohibits storing personally-identifiable info like IP addresses in your GA data).

Thanks for Posting such Valuable information.Kindly Keep Posting in it.

Anthony says:

For all of the great tools Google provides, I often find that they fall down when it comes to explaining the finer details. This post is a great example of that.

Prior to reading this, the only way I knew of getting the effect of multiple include filters on a profile was to layer exclude filters on top of each other. If you exclude everything else, then you are tracking only what you want. Tedious.

Today I tried implementing the pipe method, but I’d like to verify one thing with you:

If I apply a custom filter of /squirrels/|/dogs/ to a profile named Animals I see in the Park, should I expect the number of visits in that profile to equal the number of visits I see in the Squirrel Profile (which is using an include filter of /squirrels/) + the number of visits I see in the Dogs Profile (which is using an include filter of /dogs/)?

Dorcas Alexander Dorcas Alexander says:

Hi Anthony, since your filters are including only certain pages, you can add the pageviews for Squirrels and the pageviews for Dogs. That total should equal the pageviews for the profile that combines Squirrels and Dogs (assuming the URLs are truly mutually exclusive and we don’t have a URL like /squirrels/dogs/will-they-ever-get-along.php).

But you can’t add the visits together, because if someone views a /squirrels/ page and a /dogs/ page in the same visit, then you’d have 1 visit in the combined profile, 1 visit in the Squirrels profile and 1 visit in the Dogs profile. Hope that helps!

Lam SEO says:

What’s the difference between “equal to” and “contain” and “begin with”. Ex, I want to track subdirectory /abc on domain.com/abc. Which option I should use?
Currently I think and use the “that contain” option in Predefined filter. Is that OK?

Dorcas Alexander Dorcas Alexander says:

In a Predefined filter, “equal to” means the pattern you enter must match exactly – nothing can come before or after it. And “begin with” means nothing can come before the pattern you enter, but anything can come after it. The “contain” option means anything can come before or after the pattern you enter. Entering /abc or /abc/ using the “contain” option may or may not work for you depending on the variety of URLs you track on your website. Check your Content reports and filter the table by entering /abc to see if any URLs appear that you do not want to include. I prefer to create a Custom Include filter for Request URI, where I can use regular expressions to describe all the URLs I want to include (and I always double check my pattern in the Content reports).

Dmitry says:

Hi! Thanks for a great post. And is there any solution for the puzzle of including a wide range of IPs or it isn’t yet solved? I have 63 different IPs of my client’s partners to track in a separate profile :)

Dorcas Alexander Dorcas Alexander says:

Hi Dmitry, Unfortunately there is no solution for including 63 different IPs in a single profile. The best you can do is try to consolidate ranges of IPs and see if you can fit them into the 255-character filter field limit, using GA’s IP address tool. If you have 63 IPs and they do not share enough of the same digits to consolidate well, then there is currently no way to get all the data from those IPs into one profile.

thank you! a great article

great! Thank you for sharing

anh says:

Today I tried implementing the pipe method, but I’d like to verify one thing with you:

Sebastian says:

Hi there,
I found this post as I’m looking for a solution to unify a certain amount of pages of a website (lets say 20) in one profile, so that I can analyse these pages as one Content Group, for instance “Visits in Content Group”, “Pages Views per Visit in Content Group” etc.
As I learned, it is not possible to set up 20 include filters with the URIs (I thought this could be the solution). Now, to work with OR is not the solution due to the max. filter length of 255 characters.
So my question is: is there a way to unify a certain amount of pages (filtering through regular expressions is not possible as the URIs are different) in a profile so that I can analyse the profiles’ traffic as a Content Group of the Website?
Thanks in advance for your help!

Dorcas Alexander Dorcas Alexander says:

The current workaround for this problem is to use a page-level custom variable for each Content Group. Right before the code that tracks the pageview, you add code to set a custom variable with the Name of the Content Group. You can’t filter these pages into a profile, but you can create a custom report to analyze the data. Beware of trying to apply session-level metrics to page-level data, however. Use unique pageviews as a proxy for visits, and then you can calculate things like pages/visit.

zane says:

thanks for the “Catching Squirrels and Dogs” one, I was close but got the final answer from you

zane says:

I have to withdraw – it doesnt work – the recipe you give is wrong..

Dorcas Alexander Dorcas Alexander says:

Hi Zane, One reason the you might need to edit the filter from what I have shown is if your URLs don’t start with a slash. Check your Content reports and see how your URLs appear there, and then you can write the appropriate regular expression in the filter.

Ellen says:

Hi Dorcas,

How would you create more than one include folder on two different subdomains?

For example, get a combined data view of:
squirrels.park.com and dogs.park.com
but not park.com

Dorcas Alexander Dorcas Alexander says:

Hi Ellen, Create a filter for the Hostname field, and match the pattern: (squirrels|dogs)\.park\.com. This pattern will not match park.com, but will match squirrels.park.com or dogs.park.com.

Stef says:

Sometimes I see ^/mydir$ does this mean that the filter only shows traffic on /mydir? So without ^/mydir$ I would see /mydir + /mydir/seconddir etc? Am I right?

Dorcas Alexander Dorcas Alexander says:

Hi Stef, Yes, you are right. Removing the caret (^) and dollar sign ($) also removes the restrictions so that the regular expression can match more patterns.

Stef says:

For some reason I do lose data but I can’t figure out why. Hope you can help me

I have universal analytics installed on my website. I have http://www.domain.com where people can select a country like US, RU, JP which refers to http://www.domain.com/us, http://www.domain.com/ru, etc.

I have one main profile where all the data got collected, without any filter. I have 5 profiles for each country with a custom filter that include traffic from “request-url” ^/US

But when I look up in the main profile and check the amount of visitors on landing pages (content -> site content -> landing pages) with a RegExp search ^/us I see 50 visitors. And when I check the country profile with ^/us filter on it I see 70 visitors. When I check amount of pageviews in on all pages the number of pageviews match, but there’s a big different in bouncerates, entrences

How is it possible that the numbers aren’t equal?

Robbin Steif Robbin Steif says:

You mean visits, right? Not visitors (right?)

robbin

Dorcas Alexander Dorcas Alexander says:

Hi Stef, The numbers in the landing pages report don’t match between profiles because in your USA profile, ALL visits “look like” they start on a /us page, even if they actually started on a different page. GA is filtering out all the other pages due to your USA filter, so it has to assign the “first page” of a visit to a /us page. For example, suppose you had 100 visits to your site, and 50 of them start on a /us page, and 20 of them start on a different page and then go to a /us page. In your main profile, the landing pages report will show 50 visits started on a /us page. In your USA profile, the landing pages report will show 70 visits started on a /us page, because GA can’t see any other pages in the USA profile. Does that help explain things?

Mark says:

Thank you for sharing this. Am doing some research for my GA IQ exam and I think your content will be really helpful towards that

Hi Dorcas,

I’m trying to capture data from 9 sub folders from within my site (which is built using a CMS) so for example: /1/ & /2/ & /3/

Is this achievable using these filters?

My custome filter currently looks like:

^/uk-en/|^/uk/|^/why-atpi/|^/about/|^/media/|^/csr/|^/network/|^/careers/|^/contact-us/

Dorcas Alexander Dorcas Alexander says:

Hi David, Yes, the filter you’ve written will include traffic to pages that start with /uk-en/ OR /uk/ OR /why-atpi/ etc. It should capture the data you want.

Thanks Dorcas! I was trying to filter out two directories using the hostname filter and couldn’t understand what I was doing wrong! (*embarrassed face*) I simply needed to use the Request URI instead! :) Problem solved!

Allison says:

Is there any way to write filters that will allow us to include traffic from 2 hostnames, and also include a subdirectory relevant to one of those?

For example:
I want to include traffic from squirrels.park.com and dogs.park.com. However, on dogs.park.com, I only want to include traffic from the subdirectory of /dogbones/

Dorcas Alexander Dorcas Alexander says:

Hi Allison, Yes, you can first use a custom filter that adds the hostname to the request URI, and then write a single filter to include only the pages you want. The first filter is commonly used whenever more than one hostname is sending data to a profile, to disambiguate between home pages and other pages that might have the same request URI. There is an excellent description of the process on Stack Overflow.

Doug says:

Did I miss something?

I see how you can include data from multiple directories using the “|”. But this example was for including two different filter types, organic traffic coming in to a specific directory. How does the “|” work when those are different custom filter types? Your example only shows how to include traffic from multiple directories.

My problem is that I need to include traffic that:

1. includes traffic where the page title matches a particular string of text

2. includes traffic to my e-commerce pages (for conversion tracking)

I can set each of these up individually, but as you noted, they stop all tracking. Since they are different custom filter types, I can’t use the “|”. Is the only way this works if it is the same custom filter type?

Doug says:

I think I answered my own question (above).

First, you can have two include filters if they aren’t mutually exclusive. You will be able to track the overlap.

Second, you can’t track two include filters if they are mutually exclusive, unless you use the pipe (“|”), which requires that you use the same custom field type.

My situation required two separate custom field types (request URI & Page Title). As a work around, I found a way to use just the Page Title filter and used the “|” to include both fields.

Elizabeth says:

Thank you so much for this. We use sites like Pinterest & Houzz which attract visitors from many countries we don’t sell to. Using the pipe let me create a profile to easily track just Canada & the US where we do sell.

Kate says:

Hey,
I have around 12 domains on my web-site: http://www.mysite.com, http://www.mysite.ru, http://www.mysite.dk ect.
I have a GA account with many profiles: one for all the traffic and 1 for each domain.
I use 2 filters for the domain profiles: hostname and target campaign URL.
Does is worth to use both of them? The idea was to track all the traffic to the site (with hostname) and traffic from AdWords (with URLs).
Is campaign target URL filter included to the hostname filter?

Thanks in advance

Dorcas Alexander Dorcas Alexander says:

If you want to track traffic from AdWords to each domain, then I suggest changing the Campaign Target URL filter to two filters: Campaign Medium = cpc and Campaign Source = google. Use those two filters along with the domain filter for each domain profile. With your current setup, you have something different. It’s not just AdWords traffic to each domain. Instead you have any traffic to the campaign target URL(s) in your filter. You also exclude views of any other pages that came along with that traffic, so you would not be able to track goals that occur on other pages.

Kate says:

Thanks for your fast reply but I want to track traffic from everywhere (not only Adwords).

One more thing, we have separate account for mobile traffic in GA and we link it to our AdWords account. For some reason the number of visits from mobile campaigns in AdWords doesn’t correspond the number of paid visits from mobile campaigns in Analytics. In AdWords is always apprx. 50% higher. I thought it is caused by the filters. Our filters there are:
1) Mobile? – Yes;
2) Include host name – (.*?\.mysite\.it)

Looking forward for your advise.

Thanks,
Kate

Dorcas Alexander Dorcas Alexander says:

Your current filters track all traffic, but ONLY for pageviews of the campaign target URL(s) in your filter. You are missing data for traffic that did not see the campaign target URL(s), and you are missing data for views of other pages, including goal pages. Data in AdWords is frequently higher because AdWords clicks are not the same as GA visits, for several reasons. For example, AdWords will count all the clicks but GA will not count multiple visits if clicks on the same ad occur within 30 minutes of each other. All those clicks will be part of the same visit. See the Google article on data discrepancies between AdWords and Analytics.

Kate says:

Thanks for the link and for clarifying the discrepancies between Adwords and GA.

Just a clarification about the campaign target URL(s).
Right now my filters are:

1) Mobile? – Yes;
2) Include host name – (.*?\.mysite\.it)

It’s not clear to me how can I miss data for traffic if I’m using this regular expression:
(.*?\.mysite\.it)?

With this regular expression shouldn’t I be able to track ALL traffic? Because I think that I’m tracking the whole site (with all the pages) and not only a specific page. And then with the filter for mobile set to “yes” I only include mobile traffic. What do you think?

Looking forward for your advise.

Many thanks,

Kate

Dorcas Alexander Dorcas Alexander says:

Sorry, I thought you meant you had a “campaign target URL” filter which implied that you were filtering to include a specific page. If you don’t have a campaign target URL filter and your only two filters are “mobile=yes” and “include hostname xyz” then you are not missing any traffic.

Dorcas Alexander Dorcas Alexander says:

Of course, I should have added that you are missing the non-mobile traffic.