Upcoming LunaMetrics Seminars
Washington DC, Sep 22-26 Boston, Oct 6-10 Chicago, Oct 20-24 Seattle, Nov 3-7

Link Intersect from the Command Line

With Moz’s Link Intersect tool on the mend, I thought it might be interesting to utilize the power of the Mozscape API to build a simple command line tool with similar functionality; let’s call it Links in Common. While it’s hardly a robust competitive link research tool, I’ve found it great for generating quick “intersection” reports. In this post, I’ll walk through the setup of Links in Common (on OSX) and provide a couple of examples of its usage in the wild.

Initial Setup

Links in Common is comprised of one Ruby file, which you can grab here. Open it in your editor of choice and give everything a quick look-over. If you’ve played with the Mozscape API in the past, much of the code will look familiar (as it is featured in one of the Ruby samples provided by the Moz dev team). You’ll notice that we’re using environment variables for the Moz ACCESS_ID and SECRET_KEY variables. You should do the same if you plan to share your code publicly, as you don’t want the public to have access to your API credentials. If you have no intentions of sharing your code, you can simply paste in your ACCESS_ID and SECRET_KEY values where the environment variables are set.

Moz API Credentials

A bit below these assignments, you’ll see the Link Metrics request parameters. You can toggle these if you’d like to customize your response beyond the current implementation. (Learn more about each of the parameters here.) Right now, our request is configured to retrieve the top 150 external followed links to the subdomain of the target URL, sorted by Page Authority. This means that our response will contain information about individual pages, which is great for the purpose of pinpointing link intersections.

Link Metrics Parameters

At this juncture, the tool is ready to use. Once you save the file on your system, you should be able to navigate to its parent directory via Terminal and run the file using ruby links_in_common.rb. (This has been tested with Ruby version 2.0.0p0.) Before we get to the examples, though, we can take one last step to make executing this file a painless endeavor. First, we’ll retrieve the full path of the links_in_common.rb file using the pwd command (print working directory) in Terminal. Once we have this path, we can create an alias for the ruby links_in_common.rb command in our .bash_profile. This will permit us to execute the file using our own shorthand (e.g., lic). From this point forward, I’ll be using this lic alias to represent ruby links_in_common.rb.

Example Uses

Links in Common has two distinct uses at this point, both of which we will discuss now. The first use case assumes that we’re looking for link intersection points between two competitors’ link profiles that do not also coincide with our subdomain’s link profile. (This last part—where we subtract links we already have—isn’t 100% effective, as we’re merely comparing subsets of data. However, it can certainly be improved upon and it does help to clean things up.) Our lic command, in this instance, takes three arguments – the first being our subdomain (or our client’s subdomain), and the latter two being the subdomains of two competitors. Arguments are delimited by spaces in the command line. Putting it all together, we get a syntax that looks like this: lic www.ourdomain.com www.competitor1domain.com www.competitor2domain.com. By pressing enter, we can execute the command.

Links in Common Example

If common links are found, we get output not only in Terminal, but also in a .txt file created in our working directory. You’ll notice that this process takes over 30 seconds to complete; this is because we’re delaying subsequent requests in order to comply with Moz’s Rate Limit. Learn to be patient, and think about the next subdomains you want to check as the current requests are being made.

Not too complicated, right? The second use case is that of comparing three subdomains’ link profiles and retaining any URLs that appear in all three. We can enable this functionality by adding the -ca (compare all) option to the end of our command. For example: lic www.competitor1domain.com www.competitor2domain.com www.competitor3domain.com -ca.

-

This isn’t revolutionary (by any means), but it’s fun to toy around with the Mozscape API and build small tools that improve our workflow. It’s quite amazing how much time free API access can buy us. I encourage everyone to give this a try (or to build something similar). You might also check out the Mozscape API App Gallery for inspiration. Please leave any questions and/or feedback in the comments below.

http://www.lunametrics.com/blog/2013/07/29/link-intersect-command-line/

11 Responses to “Link Intersect from the Command Line”

GaB says:

Hi,

Thanks for that but I can’t make it work. Full disclosure I’m an absolute noob in Ruby.

I’m on mac and I had to add require ‘rubygems’ before require ‘json’

I get this error:

in `digest’: can’t convert nil into String (TypeError)
from links_in_common.rb:24

Any idea?

Thanks!

Travis Loncar says:

Hi GaB,

Did you replace the environment variables (ENV["MOZ_ACCESS_ID"] and ENV["MOZ_SECRET_KEY"]) with your Mozscape API credentials? You can access your unique access ID and secret key here: https://moz.com/products/api/keys. Each of these should be string values.

Let me know if this works. Thanks for stopping by.

GaB says:

Thanks for your answer.

Yes I did and it seems that if I remove a character in my ID or KEY it’s the same error.

Not sure what I did wrong.

Travis Loncar says:

Hm. I’m not sure. If you’d like, you can create a Gist with your code and pass the link along here. I’m happy to take a look. (Don’t forget to remove your credentials for the time being if you do this.)

Also, what version of Ruby are you running? (ruby -v from Terminal)

GaB says:

I updated Ruby to 2.0.0p247 and the error changed a little:

links_in_common.rb:24:in `digest’: no implicit conversion of nil into String (TypeError)
from links_in_common.rb:24:in `’

Travis Loncar says:

Thanks.

Just to get clarity: When you’re pasting in your Mozscape API credentials, are you doing so like a) or b) below?

a) ACCESS_ID = ENV["1234567890"]
b) ACCESS_ID = “1234567890″

My instructions aren’t especially clear, but option b) is the way to go. Option a) will only work locally if you pass in the name of an environment variable (stored on your system).

GaB says:

I had ACCESS_ID = ENV["member-abc1234567"]

By doing case b it’s working!

ACCESS_ID = “member-abc1234567″

Thank you for your time, great tool!

Travis Loncar says:

Awesome. No problem at all. Hope you find it useful.

Spook SEO says:

Hey Travis! Thanks for sharing. Can’t wait to try this later tonight. Keep ‘em coming! :)

Travis Loncar says:

No problem, Spook SEO. Hope you enjoy it. Thanks for stopping by.