Upcoming LunaMetrics Seminars
Washington DC, Dec 1-5 Los Angeles - Anaheim, Dec 8-12 Pittsburgh, Jan 12-16 Boston, Jan 12-16

The Growth of (Not Provided)

With Firefox poised to make HTTPS encrypted Google search its default search engine, the black hole of keyword referral data lumped into (not provided) in Google Analytics appears likely to experience another major growth spurt. After this news soaked in, I felt compelled to examine the growth of not provided. Here’s a look at the history, current state, and predicted future of the growth of the keyword that accounts for the most visits and conversions in the Google Analytics accounts of most webmasters the world over – (not provided).

Firefox HTTPS Recap:

The news broke almost a week ago. If all goes according to Mozilla’s plans, the default search for the Firefox browser will soon be https://www.google.com rather than http://www.google.com. HTTPS stands for “hypertext transfer protocol secure”, and is normal http with a layer of encryption – in this case ssl (“secure sockets layer”) – added to prevent unwanted leakage of information to third parties.

As of now, Firefox hasn’t yet reached the Beta stage for rolling out the new secure search. Firefox reportedly stated pretty clear intentions to move default secure search forward in the next few months. They have Google’s “go-ahead” and are currently testing the change for technical issues, and will then release it to its Aurora and Beta channels soon.

What this means, of course, for the many marketers and webmasters that rely on Google Analytics, is that they will not be able to see which keywords were used when Firefox users reach their websites via Google search – instead, these search will be lumped into the ever-growing (not provided) keyword referral group.

How much has (not provided) grown so far?

That’s the burning question I had inside, so I decided to raid the data stores. I slapped together raw, unfiltered data for 30 websites on the quantity of visits from Google nonpaid (organic) search and the amount of these visits whose keyword was (not provided).

To see how many of your site’s organic Google search visits are lumped into the (not provided) category, you can use Advanced Segments to create two custom segments. For your Google organic search segment, you will include the Medium organic, and you will include the Source Google. For the (not provided) segment, you will do the same but also include the Keyword (not provided).

The period of time for the data set was 24 weeks, from November 9 to March 24, a little before (not provided) first starting showing up around November 18, when Google stopped passing on keyword referral data to Google Analytics from users signed into their Google accounts. The websites were somewhat randomly selected for a diverse data set. To facilitate speed and diversity, I made a few exceptions to random selection: all sites used had more than 1,000 and less than 100,000 monthly organic Google visits, were mature sites, and were unique root domains. In addition, no more than 3 sites from the same client were selected. Because we can do so without leaking private info, we’ve shared this data with you. (And we’re not even going to make you pay for it;)

Below, you can see the growth in the proportion of keywords referred from organic Google searches. The purple line is (not provided) over all 3 million+ organic Google visits from all 30 sites. The other lines are the median and the average of the sites’ (not provided) percent of Google organic traffic.Growth of keyword (not provided) over 24 weeksThe line that probably gives the best indication of proportion of (not provided) is the red median line. You can see an initial explosion and then growth actually trailed off for a while. Then, towards the middle (not provided) again continued its ascent; this is likely due to the roll-out of Search plus Your World, which is enticing users to sign in to Google more often. Then, in week 21 growth explodes: this is because Google implemented its privacy policy changes on March 1, which tied all Google accounts together. For example, if someone was signed in only on YouTube and performed a Google search, the keyword they used would not be (not provided) in February but it would be (not provided) if they performed that action in March.

In the graph below, I’ve plotted the percent of organic Google traffic lumped in (not provided) for each of the 30 sites for the week of March 18 to March 24.

Percent of (not provided) from March 18-24You can see that there are 2 “outliers” with over 50% of organic traffic being (not provided). Both of these sites are on the subject of web analytics/web marketing. One of these site is LunaMetrics.com – many of our readers (like you?) are signed into Google Adwords or Analytics as they read our blog for tips. Thus, the figures for the averages may be a bit inflated. The median figure gives the best indication of a websites “typical” % of (not provided), which appears to be 22-24%. It seems as though Matt Cutts’ prediction was a tad conservative.

How Much Will (Not Provided) Grow in the Future?

A lot. Probably. Don’t be surprised to see that figure of 22-24% climb a little, albeit at a slower rate, over the next few weeks as the current trend continues. Then, if Firefox moves forward with making encrypted search their default – and I really think they will – expect (not provided) to explode. We don’t know yet exactly when and how they roll this out, but we know that, if Firefox moves forward, eventually almost all the visits to your site from Google organic search performed on Firefox will be lumped into (not provided). It is estimated that Firefox controls nearly 25% of the browser market, so the increase will be dramatic.

I’ll go on record as predicting that, by the 2012 holiday season, the majority of websites will see 40-50% of their keywords from Google organic search in Google Analytics lumped into the (not provided) category.

 

Reid Bandremer

About Reid Bandremer

Reid Bandremer is a Senior Search Project Manager. His background before joining LunaMetrics in 2011 includes eCommerce marketing experience and a pair of business degrees. He is a rabid fan of data, music, and holistic ROI-driven search marketing strategy. Other strengths include SEO metrics, migrations, and searcher segmentation (keyword research 2.0). Contrary to popular theory, Reid is not homeless – he just often stays at the office late because he is obsessed with maximizing the value of clients’ search traffic.

http://www.lunametrics.com/blog/2012/03/27/growth-not-provided/

17 Responses to “The Growth of (Not Provided)”

Adrian says:

Thanks for this breakdown. I manage two sites (completely different industries, one with roughly 2000 visits a month, one with about 850,000 a month) – the smaller site is focussed on organic traffic from SERPs on to optimised landing pages, the larger site is heavily branded and barely relies on organic traffic (outside brand keywords) at all. I noticed a bigger jump in early march than November – for the larger site, it increased 9 times over one week (although it still makes up less than one percent). For the smaller site, it is now the largest keyword referrer, bringing in circa 25% of all keyword traffic. As all the landing pages are heavily optimised, I can get some idea of what people are searching for by reverse engineering their paths and seeing what landing pages they enter on. Not ideal, but it’s probably the best we can do for now. I’ve seen some suggestions to offer a discount to clients that indicate what search term they used to find your site (if they came via HTTPS). Not sure I want to go down this route just yet, will keep a very close eye on my goal conversion % for (not provided) compared to my site averages.

Reid Bandremer Reid Bandremer says:

You’re welcome Adrian! Sounds like you’ve been affected “normally.” It also sounds like you’re in the right place in terms of coping. I agree that in most cases we needn’t offer coupons solely to incentivize sharing referral paths (keywords in this case) just yet. One thing you can do is estimate what proportion of your (not provided) that is branded based on the proportion of un-(not provided) traffic is branded and then look at the difference in engagement and conversions between branded and unbranded traffic to get a better “feel” for what your (not provided) traffic is doing. Additionally, google webmaster tools has some data on keywords and landing pages, and you can integrate this data into Analytics. Good luck in your web endeavors!

Adrian says:

Yeah, breaking down the branded/non-branded proportions is a good idea, something I’ve already been looking at. Webmaster Tools helps too, in a bit more of a general sense – between these 3 or 4 different approaches, I feel about as prepared as possible for the recent and future rise of (not provided). If anyone has any additional suggestions I’d love to hear them!

I’m waiting for the other shoe to drop…when Explorer announces this change as well.

Thanks Lunametrics for confirming the grim picture we already all see. I guess the future of all our websites (At least for SEO) is going to have to be done with completely dynamic/generated pages as we ready for the horde of (Not Provided)visitors.

Or you can just pay for PPC!

Unfortunately most small clients don’t have the budgets or know how to invest in those kinds of powerful web solutions (Without completely rebuilding). SEO might not be dead, but what are practitioners suppose to resort to?

-Ranking Reports?
(Waste of time with Personalization & Search plus your world)
-Visits?
(Welcome back to 1995)
-Goals?
what about the 95% of non-converting traffic?. -Attribution (I defer to Lunametrics!)
-Time on site? (If you’re a blog great..if not?)
-Webmaster Tools Query Report(Not a replacement without corresponding landing page info)

Other than putting a big annoying banner up on every one of your pages screaming:

=====================

“Hello completely Anonymous logged in Google user. We apologize in advance that we are unable to service you better. Unfortunately Google thinks it can do a much better job than we can! Which is why you landed on this page! Before you hit the back button…Please RE-ENTER the query you’re looking for! So we can show you why Google Engineers don’t know everything about our business”

========================

Hey Reid want write the next great SEO article – try this “SEO without KEYWORDS”

Searchengineman

Reid Bandremer Reid Bandremer says:

Great comments…
I think Chrome is more likely to go https than IE. Dare I say it’s possible Microsoft might become a search marketers last ally? I think we have to move towards using a greater combination of metrics to paint the picture of what keywords are working in terms of traffic and relevance; there is no single replacement for the missing keyword referral data. Finding the ideal “metric mix” and balancing level insight with practicality/time constraints is something we’re all working feverishly on, and what works for one site may not be ideal for another. I think all the data/metrics you mentioned are actually worthy of considering resorting to at this point.

Btw, you’re a funny guy.

Chris H says:

Good article – always good stuff from Luna. Agreed it’s only going to get worse & SMBs/agencies/etc need to stay ahead of the curve. I just posted some more specific ways to analyze (not provided) data: http://www.seoverflow.com/analytics/not-provided-data-4-ways-to-dig-deeper/ Hope it helps y’all!

Reid Bandremer Reid Bandremer says:

Chris, thanks for an example of when it IS appropriate to post blog comments with links. This is a difficult professional issue deserving of collaborative effort. In the end, better knowing which referring keywords result in higher quality visits helps out the whole web.

Robbin Steif Robbin Steif says:

@stanley — Christina Keffer is speaking on this topic at SMX in Toronto. I think her title is close to that, something like, “Search in a world without keywords.” Great minds and all that.

Alex B. says:

So now my site is up to 1/3 “not provided”. Traffic is not really high enough to assume normal distribution on all but a few keywords.

Screwed.

How long will it be before Google starts charging for a look at “Not provided”?

Pavlicko says:

Gee, this is fantastic news.
Or rather…what I meant to say was – this sucks.

Sure, we can integrate GWT search query stats into Analytics but I find it quite useless, outside of general keyword discovery or determining if a keyword segment is tanking.

They prevent the ability to include any other relevant dimensions (like revenue, or landing page, etc.) or advanced segment filtering within this report.

Even reviewing this information IN webmaster tools is an experiment in frustration. My heart goes out to the smaller SEO companies trying to keep on top of this.

Savannah G says:

This is seriously frustrating. It seems my % of not provided traffic is climbing daily. I rely on this information a lot for my sites :(

Sean Mysel says:

This is about 80% of my search terms in analytics. Crazy part is the avg time of site is 22 minutes under not provided

AngryNick says:

SEO has only recently become a concern for me as I help my wife build her tutoring business. 4 months into production and I’m seeing the same 20-25% “not provided” values as everyone else. It’s now the top keyword result.

I find it interesting that Google will give me nearly anything I’d want to know about a paid click user, but is taking a “privacy” stance with organics that are logged in or using https. This inconsistency seems to favor me buying more terms, but I don’t know what those terms might be ;-).

What are the options if IE and Safari follow Firefox? Webmaster tools is all but worthless for a low-traffic site like mine.

Affected by “keyword not provided”? Please spend 3 seconds voting in our super sexy survey – http://www.browsermedia.co.uk/2012/09/21/grrrr-is-not-provided-doing-your-head-in/

Reid Bandremer Reid Bandremer says:

I think it is unlikely (although possible) that IE, Safari, and Firefox all make encrypted search default and we are unable to obtain keyword data in Google Analytics. But if that does happen SEOs will have to get resourceful and make educated guesses on keyword performance.

Tools that estimate keyword traffic like Google and Bing Webmaster Tools as well as Spyfu and SEMRush will become more important. Low traffic sites will indeed have a tougher time, since these estimation tools have much larger margins of error with smaller sample sizes. Volume estimaters like AdWords Keyword Tool, Bing Webmaster Tools Keyword Tool, wordtracker, etc. might become a bit more important.

We might also start relying more on rankings reports again. If you reference a landing page’s ranking against that page overall traffic against estimated volume you might have a good idea of what words are bringing in the traffic.

Of course, PPC data would seem the easiest way to obtain data on keyword quality in this scenario…

JJ says:

Doesnt this imply, if indeed this has been thought out and deliberate, that Google is supporting the higher end SEO/marketing sites over the quickie affiliate type low value quick buck sites? By making things harder, doesnt that always support the harder workers, even though it is more annoying for everyone to do a better job, let alone encourage the paid ads more? I am no expert, but I am just trying to get input to see if Google’s direction is actually for benign purposes because I know there have been tons of complaints about highly ranked quickie affiliate sites.

Reid Bandremer Reid Bandremer says:

I think the (not provided) move was very thought out and deliberate. I also personaly belief that the motives were probably more intelligently thought-out business (profit-oriented) tactics rather than altruism. IMHO, deterring spammy affiliates may have been something they thought about, but would have been near the bottom of the list of reasons to go (not provided). Certainly, the motives for (not provided) were not the same as those for the Panda and Penguin algorithm updates.