A Better Alternative To Exclude Filters in Google Analytics

/

better-alternative-to-filter-ga

I originally wrote this post back in 2014, but the concept and themes have proven valuable over the years, even if the screenshots have slipped out of date. I’ve updated the screenshots and the title. Originally, this post was titled “Say Goodbye to Exclude Filters in Google Analytics!” – which understandably elicited mixed reactions on social media and accusations of click-baiting. Exclude Filters aren’t going away anytime soon, but after reading this post you may not WANT to use Exclude Filters as frequently.  There are still many valid reasons why you may need to set them up, but when possible – it might be time to eliminate them.

Just as a quick refresher, you can use the Exclude Filters in Google Analytics to block traffic data from certain sources from showing up in a particular view.  Sometimes these are used to partition data into one view or another, for example, think of creating separate views for Internal or External Traffic.  For these use cases, Filters work beautifully. You can filter based off of IP Address, Hostname, Service Provider, etc…

But then there are those occasions where you want to block out traffic completely.  Just as easily, you can set up an Exclude filter for each of your views, and poof! The data has disappeared!  Except, it hasn’t really.

The raw, unfiltered data is stored by Google Analytics. Anytime you request a more specific report, either by adding a secondary dimension or applying an Advanced Segment, GA will go back to that unfiltered data at the property level to rerun your report, then filtering the results for your specific view.  Why is this important?

Even filtered data will increase sampling in your reports.

In the free version of Google Analytics, if you try to run a non-standard report that includes more than 500K sessions, Google Analytics will use a sampling algorithm. So all of this raw data you think you’re filtering out is actually going to continue to stick around and cause sampling issues for you. Click here for more information about how sampling works with Google Analytics.

Google Analytics 360 customers have a few secret weapons against sampling, specifically – sampling is calculated at the View level, after filters. However, that doesn’t mean that the hit goes away. Again, it’s just filtered from a particular view. Those hits count towards your hit quota and can affect how much you’re charged or if you’re nearing a particular tier.

Filtered data will count against your hit quota.

In cases where you want to filter out traffic completely, my advice is to stop sending that data to Google Analytics. Consider the difference between source separated recycling or single stream recycling. In source separated, you manually split your recycling before you send it to the recycling plant. In single stream recycling, it all goes together and the recycling plant has a machine that helps sort it for you.

Modified from Better World Betty

The answer here is Google Tag Manager, which is going to help us sort our traffic before it goes to Google Analytics. If you’re not familiar with Google Tag Manager, we’ve got a few articles on our blog that will help get you up to speed. Long answer short, we love it here and it will change your (analytics) life.

Let’s go over how data gets from your website into Google Analytics. In a standard implementation, using Google Tag Manager and Google Analytics, the following order of steps happen.

  1. Visitor Reaches Your Page
  2. Google Tag Manager Loads
  3. GTM Executes a Google Analytics Tag
  4. Pageview Is Sent to Google Analytics
  5. Google Analytics Receives the Pageview
  6. GA Applies Any Filters for Each View against the Pageview
  7. Pageview Shows Up In Your Analytics View

Depending on the outcome of Step 6, the Pageviews and Events will or will not show up in your Google Analytics reports. Rather than waiting until the Pageview hits the filter to decide that it’s not what we want, I propose we recreate the Exclude Filters using custom Variables and Triggers in Google Tag Manager.  If we can determine that we don’t want a particular visitor to show up in our Analytics, let’s not even serve them the Google Analytics tag. We’ll use Google Tag Manager as a shield in this situation, to keep that data out of our analytics entirely.

Fancy GTM shield graphic circa 2014.
Get it?!

If there’s a reason you want to see this data in the Google Analytics interface, let’s say from a development site or from internal traffic, once your triggers are created you can always serve it a different Analytics tag or use a dynamic variable to change the UA number, so that it won’t count against your main property’s hit limits or sampling limits.

Note: It will still be helpful to have Exclude Filters on your account as a backup. This blog post will help keep traffic from reaching your account, but it won’t stop traffic from any sites that aren’t loading your Google Tag Manager.

Our new process looks like this:

  1. Visitor Reaches Your Page
  2. Google Tag Manager Loads
  3. GTM Looks at Value on Data Layer
  4. GTM Decides Not To Execute a Google Analytics Tag
  5. Pageview Is Sent to Google Analytics
  6. Google Analytics Receives the Pageview
  7. GA Applies Any Filters for Each View against the Pageview
  8. Pageview Shows Up In Your Analytics View

Benefits Of Using Google Tag Manager instead of Google Analytics

Setting up View Filters will always have its place, but I encourage complementing View Filters with GTM Triggers for most of my clients and training attendees, for a few reasons. I mentioned above that View Filters won’t help influence your sampling limits or your hit limits.

Additionally, most people aren’t JUST using Google Analytics. By using View Filters, you’re putting logic in place that only exists in one tool – Google Analytics. Often, the reasons we want to exclude traffic from our reporting can apply to the other tags that we have in Google Tag Manager. If we don’t want these people getting our GA Tag, why should they continue to get our Google AdWords tag, our Facebook conversion pixel, our CrazyEgg Tag, etc.

Consider the following:

  • Advertising and Remarketing – Surely – we don’t want our employees being targeted by our own advertising. Use these tactics to block the tags on your site that adds visitors to remarketing lists. Spend less and spend more intelligently.
  • Surveys/Pop-Ups –  If you’re using a Tag to launch 3rd party surveys or ads, chances are you’re receiving some sort of report about how well they’re performing. It would make sense in this case to block these tags from even firing for Internal Employees. This should help improve your response/clickthrough rate.
  • E-Commerce – With your new Variables and Triggers, you can effectively block any test transactions from Internal Traffic from ever getting to Google Analytics.

So let’s get down to business. Two of the more popular Include/Exclude Filters are based on Hostname or on IP Address.

Using Google Tag Manager to Exclude by Hostname

First up, we can create a Variable/Trigger in Tag Manager that we can use to block anything that isn’t our main site. For this example, let’s assume that our site has the following subdomains:

  • www.mysite.com or mysite.com
  • dev.mysite.com
  • admin.mysite.com

We only want the traffic to our main site to be sent into Google Analytics. We can do this through a simple Variable and a new Trigger.

1. Create a Page Hostname Variable

Google Tag Manager has a number of variables that are built-in and just need to be enabled! In this case, just make sure the Page Hostname variable is enabled.

2. Create the Trigger – “Traffic – Bad Hostname”

Google Tag Manager Hostname Trigger

In Google Analytics, we have the option of using Include or Exclude. We could do the same thing here, but I’ve found that it’s much easier to create this trigger as a standalone trigger and use it to Block tags from firing. If you wanted to, you could add it to all the triggers that control firing, but then you’re duplicating the same logic over and over. I’ve written about this concept here: The Art of the Double Negative – Using Trigger Exceptions in GTM.

I’m calling this Trigger “Traffic – Bad Hostname” as a catchall for any hostname that isn’t www.mysite.com or mysite.com.

  1. Head to the Triggers section and click New.
  2. Name this Trigger “Traffic – Bad Hostname”
  3. Choose Custom Event for the Trigger type, and enter “.*”, and check “Use regex matching.”
    Because this is a blocking trigger, we’re going to block everything from this hostname. More details
  4. Choose the “Page Hostname” variable from the drop down list.
  5. In the next box, choose the “does not match RegEx (ignore case)” option
  6. In the next box, enter the following regular expression. This accounts for the fact that sometimes my site can be loaded with or without the www.

Use that Trigger to as a Trigger Exception For Pageview Tags

Google Tag Manager Trigger Exception

Now that we know this traffic isn’t the general public coming to our main site, we can safely block certain tags from firing. In this case, that will be the basic Google Analytics tracking tag.

  1. Open the tag you want to block with this Trigger.
  2. Under Triggering, hover until you see the Pencil icon, and click. (I click on the word Triggering.)
  3. Click “Add Exception”
  4. Select the “Traffic – Bad Hostname” Trigger
  5. Save!

Using Google Tag Manager to Exclude by IP Address

Now that we’ve set up our Exclude by Hostname capabilities in Tag Manager, let’s look at IP Address. This one is a little harder to implement and will require some server-side logic. By default, IP Addresses aren’t available through JavaScript on a webpage. Depending on the platform your site is running on, this should be fairly easy to implement. You’ll need to look up the appropriate way to get the visitor’s IP address depending on how your site is configured. Once you get that IP address, then it’s time to write it to the dataLayer.

Google Tag Manager utilizes something called the dataLayer to pass information from the page into Tag Manager. Since 2014, we’ve written extensively about the data layer. For more information, check out either of these posts:

The idea here is that you collect any sort of information about the visitor, the session, etc… and write it on the dataLayer when the page loads. These are values that your server knows, but that your page does not. When Google Tag Manager loads, it can see this information and using Variables, use this information for Triggers and other Variables.

For the IP Address Exclude, I’ve created a more complicated Variable to allow you to identify multiple types of visitors by their IP addresses. This will let you use these types of visitors in Firing and Blocking Triggers.

Add IP Address to the dataLayer

The dataLayer needs to be loaded ABOVE the Google Tag Manager snippet, so that this data is available when Tag Manager loads. You can use something like the following code:

Create the Visitor IP Variable

Google Tag Manager Visitor IP

  1. Now, back in Google Tag Manager – head to the Variables section and create a new User-Defined Variable
  2. Name this Variable “DLV – Visitor IP”
  3. Select Data Layer Variable as the type of Variable.
  4. In the Data Layer Variable Name, enter whatever you called the IP Address. In my case, this would be visitorIP.

Create the Visitor Type Variable

Google Tag Manager Visitor Type

  1. Create a new User-Defined Variable
  2. Name this Variable “CU – Visitor Type”
  3. Select Custom JavaScript as the type of Variable.
  4. Paste in the following code.
  5. Alter the code to suit your needs.  Regular expressions will come in handy here, just like in Google Analytics Filters.
  6. Save!

Note: I used a Custom JavaScript variable here to enable regular expressions. There a few tools that might be helpful – check out my old post on creating Google Tag Manager Lookup Tables, which also includes a link to a tool created by Seer Interactive that makes this process easier.

Create the Trigger – “Traffic – Not External”

Google Tag Manager Not External Traffic

Again, I’ll create a trigger as catch-all for everything that is not our “external” visitors that we want to track. It might be helpful at some point to create special trigger for different visitor types, but in this case, I’ll lump them all together.

  1. Create a new Trigger
  2. Name this “Traffic – Not External”
  3. Choose the “CU – Visitor Type” variable from the drop down list.
  4. In the next box, choose the “does not equal”
  5. In the next box, enter the word “external”

Use that Trigger to Block Tags from Firing

Google Tag Manager IP Exclude

  1. Open the tag you want to block with this Trigger.
  2. Under Triggering, hover until you see the Pencil icon, and click. (I click on the word Triggering.)
  3. Click “Add Exception”
  4. Select the “Traffic – Not External” Trigger
  5. Save!

General Conclusions

So there you have it, two ways to use Google Tag Manager to shield your Analytics data from unwanted traffic. I’ve written extensively about blocking out internal traffic, but consider any other decisions you can make inside of Google Tag Manager to determine whether or not to fire tags.

Sometimes you want to see the data going into Google Analytics to make sure everything is working properly. Filter may have helped you create separate views, but Lookup Tables and trigger exceptions can help create a similar setup using a test property with a different UA number for all testing/QA. This way your test data isn’t contributing to increased sampling and you can safely monitor your test cases. You can use these Variables to block one tag from firing, and the trigger another tag to fire instead.

Last note of caution – if you have a high traffic site and you’re constantly hitting sampling, this fix may not do much to help your situation. Consider Google Analytics 360 which increases the amount of visits you can include in your sampled reports, as well as giving you the option to request unsampled custom reports.

About

Jon Meck is our Technical Marketing Manager, promoting our services and trainings to the world. He has a jack-of-all-trades background, working for companies large and small in social media, website design and maintenance, and analytics. He is an Excel enthusiast, he loves efficiency, and he is strong proponent of the “Work Smarter, Not Harder” mantra. Jon is also the author of two number puzzle books.

  • Rahul Parmar

    Hi John, I’m not a developer but wanted to exclude a range of IP addresses that cover internal users devices. I was confused by the Visitor Type variable, specifically the //Monitoring System section – what is the need to define an external user and what does the IP range represent that you have defined as (“^65.65.65.12[0-9]$”)?

    • Hi Rahul,

      Sorry this is a little outside the scope of a blog post. There’s a bit of an assumed knowledge in order to make use of these recommendations. I’m using the Visitor Type variable to compare the user’s IP Address against a list of known IP Addresses. In my example, this includes traffic that I’ve labeled as Internal, as well as traffic that I’ve determined as coming from a Monitoring tool. You might only have Internal Traffic, and that’s fine.

      A note of caution – remember when you’re dealing with internal users devices that you only care about their external IP Addresses, or how they appear to the websites they browse.

      Check out this post for more details on Internal Traffic: https://www.lunametrics.com/blog/2015/04/27/internal-traffic-google-analytics/

      For more information on regular expressions, check out this post: https://www.lunametrics.com/blog/2013/12/26/practical-guide-regular-expressions/

      Hope this helps.
      -Jon

  • Steve Ward

    Hi, is there a way to block my single, personal iPhone from analytics? I frequently test my own website and don’t want my visits to count against analytics data. Cheers, Steve

    • Hi Steve,

      Here’s a post I wrote on blocking Internal traffic from GA. https://www.lunametrics.com/blog/2015/04/27/internal-traffic-google-analytics/

      I’m not sure this blog post that you’re commenting on (Excluding Traffic via Google Tag Manager) necessarily applies to your situation.

      In your case, I’d probably suggest using the methods in the internal traffic blog describing a hidden page on your site that sets a User-Level Custom Dimension for any device that visits it. Then you just make sure you visit that page on all browsers/devices that you want to filter from your traffic.

      A last case resort would be to use a custom query parameter when you visit any page on your iPhone, even something like ?exclude=yes. GA and GTM can both recognize query parameters and you can set up filters/blocking triggers based on something like that. The challenge is then making sure you include the parameter on everything you’re testing.

      Hope this helps.

      -Jon

Contact Us.

LunaMetrics

24 S. 18th Street, Suite 100,
Pittsburgh, PA 15203

Follow Us

1.877.220.LUNA

1.412.381.5500

getinfo@lunametrics.com

Questions?
We'll get back to you
in ONE business day.