AuthorCrawler

Find who is linking to you. A free open-source SEO tool for link building.

author_crawler

AuthorCrawler is a proof-of-concept tool that I built to highlight the ways in which the SEO community could use Google's authorship markup to harvest data about links.

The tool examines the backlinks to a URL and crawls them looking for authorship markup, which it will follow to download a wealth of information on the author of those links. It uses the SEOmoz API for the links, and then crawls them each.

You'll end up with a report allowing you to sort your backlinks by the social standing of the authors of those links. See the example reports to see how this might look.



  Download AuthorCrawler

Single URL Report

Report for Single URL (example report for Single URL)

Single URL reports analyse the backlinks to the specified URL/Domain. Find out who is linking to your site, and where else they write.

Multiple URL Report

Report for Multiple URLs (example report for Multiple URLs)

Mutliple URL reports analyse the backlinks to multiple domains and do an 'author intersect' to find authors that are linking to your competitors but not you.

Initial Setup

This tool is written in PHP, so that it could be modified to run via a web browser (for anyone who is up to the challenge), but is currently designed for the command line. I've tested it on Mac OS X and Windows, it should also run fine on Linux.

There are two versions of the tool, one for running a report for a single URL, to get the authored back links, and a second to look at multiple URLs and get the author intersect.

Both versions need API keys for the SEOmoz API (its free!) and a Google API key (its free!).

Google+ API key: https://code.google.com/apis/console/ (turn on Google+, then wait a couple of mins)

Moz API key: http://www.seomoz.org/api/keys

I recommend running this tool from a cloud service (within your T&Cs!) such as Amazon Web Services or Rackspace Cloud Hosting, or from a server machine you have. You could spin up a cloud instance just whenever you needed to run the tool, and it would cost you less than $0.25 to run it a few time probably.

Configuration

Firstly, put both your API keys in the appropriate spaces in the config.php file.

Running the tool (single URL version)

Running the single version of the tool from the command line, is easy:

php single.php www.tomanthony.co.uk

The tool should start running immediately, you should see output like:

Example output.

When the tool finishes you'll get a list of authors and their details, and a list of authored URLs. However the important part is:

Writing report report_2012-03-30-08-22-26.html

This gives you the name of the HTML report, which you can now open and play around with. :)

Running the tool (multiple URL version)

Its not that different from the single version, just now you put multiple links:

php multiple.php www.mysite.com www.competitor1.com www.competitor2.com

You will get similar output, and will also receive the name of an HTML report to view/share.

Installing PHP on Windows

Download PHP 5.3 (thread safe version) from here:

http://windows.php.net/download/

Choose the option for 'don't setup a server', and let it install.

Then you need to add this:

C:\Program Files\PHP\

to your Windows PATH variable (see here for instructions).

Then run it from the command line as per the instructions above.

Caveats

The tool isn't perfect, a few notes:

Version 0.5 (30th March)

Version 0.6 (13th April)

Download AuthorCrawler v0.6:


  Download AuthorCrawler

I'd love feedback, bug reports and feature suggestions.

However, please keep in mind this was more a project to show the potential - I fully expect the pro SEO tools out there to start doing this stuff in the near future. At the same time, the tool is open source so you are welcome to go away and extend it yourself! :)

In the meantime, you can find me on Google+ and Twitter:

Encircle Tom