It’s a good question - Google’s geocoder permits you to make 50,000 queries a day, which sounds like a lot. However, that is only 35 a minute, which if you sustain for more than a few minutes, kicks in the limit (apparently…). So you might be interested in caching your results.

If you aren’t fussed about UK geocoding, you can access the regular Google geocoder using HTTP, as documented in the Google Maps Documentation.

However, if you want to temporarily cache results collected by the client (as per my UK geocoding example linked to above), you need to send them back over an AJAX connection, and that is what this article discusses. Although aimed at using the UK geocoding technique, a lot of this tutorial will be applicable to results via the normal Google Javascript geocoder.

So this short tutorial is going to do is show you how once a postcode has been translated into longitude and latitude, the result can be sent back to your server to be temporarily stored in a database. Then we are going to look at how we can query our own database before we query Google’s, for each result.

Note: I’m told it used to be in Google’s Terms that you couldn’t store geocoding results, but it doesn’t appear to be that way now. However, we are only going to store them temporarily, so we don’t have to hit Google’s server repeatedly for the same query.

If you are eager just to see how this all will work, you can go straight to the demo page.

So, lets get going…

Getting started

To begin with, we need to create our Google map page, and get our UK geocoder working. If you’ve not done so yet, take a look at the UK Postcoder Geocoder tutorial and get that setup.

We are going to make only a couple of changes to the Javascript files, but firstly we need some PHP, and a MySQL database to store the results in. At this point, if you are confident with your server side scripting, go ahead and use ASP and MSSQL or anything else that takes your fancy; just remember to asjust the calsl we’ll add to the Javascript accordingly.

Make the database

We need to make a database in order to store our cached results. We will have 1 table, which will have 5 fields: id, postcode, latitude, longitude and data_added. id is our primary key, you can ignore it. The other fields should be fairly self evident.

You can use this SQL to create your table once you’ve setup a database:


CREATE TABLE `postcodes` (
  `id` int(11) NOT NULL auto_increment,
  `postcode` text NOT NULL,
  `latitude` float NOT NULL default '0',
  `longitude` float NOT NULL default '0',
  `date_added` datetime default NULL,
  PRIMARY KEY  (`id`)
)


We need the date_added field simply as we are only storing the results temporarily, in our case for 24 hours. After a result is 24 hours old, we ignore it; I’ll leave it as an exercise to the reader to write code to clean up their old results.

Write the PHP to write

Now we need to store results in our shiny new table; so we are going write cache.php which writes a retrieved result to the database. It is pretty short and sweet, here is the code:

<?php
 
  $db = @ mysql_connect('localhost', 'DBUSER', 'DBPASSWORD');
  
  if (!$db) {
    die("Sorry, database error!");
  }
  
  mysql_select_db('DBNAME',$db);
  
  $sql = "INSERT into `postcodes` (  `postcode` ,  `latitude` ,  `longitude` ,  `date_added` ) VALUES (UPPER('" . mysql_real_escape_string($_GET['postcode']) . "'),  '" . mysql_real_escape_string($_GET['latitude']) . "',  '" . mysql_real_escape_string($_GET['longitude']) . "', NOW( ))";
  
  if ($result = mysql_query($sql))
  {
    echo "Success";
  }else{
    echo "Problem";
  }
  
?>


It just connects to the database, and sends one SQL query. Note we use UPPER() to ensure all postcodes are written to the database in upper case, we don’t want to be storing cached copies for both upper and lower case.

Don’t forget to change DBUSER, DBPASSWORD and DBNAME to the correct values; your database user, password and your database name.

If you have problems, you can access this file directly, and it should say “Problem” if it has a problem!

Write the PHP to read

Ok, we now have the PHP code to write results to the database, we just need to be able to retrieve them. For this we need geocode.php, another small PHP script.

Lets dive straight in with the code:

<?php
  header("Content-Type: text/xml");
?>
<locations>
<?php
 
  $db = @ mysql_connect('localhost', 'DBUSER', 'DBPASSWORD');
 
  if (!$db) {
    die("Sorry, database error!");
  }
 
  mysql_select_db('DBNAME',$db);
 
  $sql = "SELECT * FROM  `postcodes` WHERE `postcode` = UPPER('" . $_GET['postcode'] . "') AND `date_added` >  DATE_SUB(NOW(), INTERVAL 1 DAY)";
 
  if ($result = mysql_query($sql))
  {
    if (mysql_num_rows($result) > 0)
    {
      $location = mysql_fetch_array($result);
      echo "\t<location postcode='" . $location['postcode'] . "' latitude='" . $location['latitude'] . "' longitude='" . $location['longitude'] . "' date_added='" . $location['date_added'] ."' />\n";
    }
  }
 
?>
</locations>


The <locations> tags are because we are outputting XML. Eeek! XML! Don’t be scared, it is very simple, the actualy output we send to the browser looks something like this:

<locations>
  <location postcode='W1B 2EL' latitude='51.5144' longitude='-0.141959' date_added='2007-03-21 19:04:25' />
</locations>


We then do a simple SQL query to grab matching results that were added in the last day, and output the first one (there should only be one) into our little XML document. That wasn’t so bad, right?

Adjusting the Javascript

Ok, we have PHP that reads and writes to a database with our results; which just means we need to adjust our Javascript. Currently our code queries Google and then uses the result, all we want to do is add a couple of steps to this process.

We want to query our own database first, and if we find a result, then great, we use that. If we don’t find a result, then we want to query Google, as before; we’ll also want to write the result from Google back to our cache, for use with future queries.

To achieve all of this, we are going to add 2 new functions, and we are going to make a couple of modifications to one of our current functions.

First we will add createRequest() which is a common function for AJAX applications. It just creates and AJAX request object, depending on your browser. This tutorial isn’t designed to be an AJAX tutorial, so if you need help understanding this function, you should find plenty of helpful tutorials on the web.

Here it is:

function createRequest() {
  
  // create an Ajax Request
  
  var ajaxRequest;
  
  try
  {
    ajaxRequest = new ActiveXObject("Msxml2.XMLHTTP");
  }    
    catch (e1)
    {
      try
      {
        ajaxRequest = new ActiveXObject("Microsoft.XMLHTTP");
      }
        catch (e2)
        {
          ajaxRequest = new XMLHttpRequest();
        }
    }
  
  return ajaxRequest;
}


The object this creates will simple enable us to send requests back to the web server, via Javascript, and receive the reply; perfect for querying our cache.

We already have a function usePointFromPostcode() that queries Google, we just need a middleman to try our cache first. This challenge is answered by the might and brave usePointFromPostcodeViaCache() function:

function usePointFromPostcodeViaCache(postcode, callbackFunction) {
  
  map.clearOverlays();
  var ajax_connection = createRequest();
  
  ajax_connection.open('get', "geocode.php?postcode=" + postcode);
  
  // setup the function to deal with the reply
  ajax_connection.onreadystatechange = function(){
    
    if (ajax_connection.readyState == 4) {
      var xmlDoc = ajax_connection.responseXML;
      var markers = xmlDoc.documentElement.getElementsByTagName("location");
      
      if (markers.length > 0)
      {
        var resultLat = markers[0].getAttribute('latitude');
        var resultLng = markers[0].getAttribute('longitude');
        var resultDate = markers[0].getAttribute('date_added');
        var point = new GLatLng(resultLat,resultLng);
        document.getElementById("result").innerHTML = "Result for " + postcode + " came from cache. It was last cached on " + resultDate;
        callbackFunction(point);
      }else{
        usePointFromPostcode(postcode, callbackFunction);
      }
    }
  }  
  
  ajax_connection.send(null);
}


We clear any current markers, just for the sake of this tutorial, so you an query the same postcode over and over.

Then we setup our AJAX request, and write our callback function (for AJAX newbies, this is just the function that is called when a result is received from the webserver). This function parses the XML, and checks to see if we have any results by checking markers.length.

If we have no results, we call usePointFromPostcode(), which queries Google, as before. If we do get a result, then we handle it exactly as we do in usePointFromPostcode() when we get a result from Google; we make a GLatLng object and pass it to our designated callback function.

Lastly, having set this up, we actually send the request. Phew! Almost there.

Finally, we need to add a couple of lines to usePointFromPostcode() to write retrieved results to our cache. Between the lines var point = new GLatLng(resultLat,resultLng); and callbackFunction(point); add this code:

document.getElementById("result").innerHTML = "Result for " + postcode + " came from Google.";
  var ajax_connection = createRequest();
  ajax_connection.open('get', "cache.php?postcode=" + postcode + "&latitude=" + resultLat + "&longitude=" + resultLng);
  ajax_connection.send(null);


This sends an AJAX request, just to send the result to the cache. We need no callback function for this AJAX request, as we don’t need to examine the response.

You’ll notice in this function and the previous one we’ve added a line writing a comment to a ‘result’ element on our page. This is just so you can see in your working example, or on the demo page where the result is coming from. So we need to add this element to the index.php file, which is 2 minutes work. Last hurdle I promise, lets go…

Update index.php

Wherever you like add to your HTML file this code:

<p id="result"></p>


The very last thing to do is, in your HTML file, change all instances of usePointFromPostcode() to usePointFromPostcodeViaCache() or we will continue to query Google without checking our cache first.

You can check this all works on the demo page. If you can’t think of a Postcode, try ‘SP4 7DE’ to see Stonehenge!

Final words

This tutorial has kinda blitzed through the AJAX stuff, I know; but I wanted to focus on the task at hand. If you go and read a couple of AJAX tutorials, you shouldn’t find what is being done here too complicated. Having said that, post any questions in the comments, and I’ll do my best to answer.

Also, to keep it simple I’ve cut some corners here. Some examples are:

1) This code caches results across different visitors. If you have visitors who maintain long sessions and redo the same queries, you’d probably also want to cache them in a Javascript array so no repeat remote queries were necessary.

2) There is no thought to security given here; it is possible for someone to fill your cache up with junk (though they’d need to do it every 24 hours).

3) This is aimed particularly towards UK results. If you aren’t bothered about UK results, you could query your database, as we are doing, and if the result is missing, then your server could use the HTTP geocoder to get the result directly from Google to pass back to you. However, this technique will working for you too (you just need to modify it a bit).

However, hopefully it will be enough to get the people who’ve to get going with their projects.

Have fun with it, and please do post comments if you have questions (or just to say hi!:).

Technorati Tags: google maps, api, Web 2.0, javascript, caching, Web-Services

Google Maps API provides a geocoding feature, for finding the latitude and longitude of places or addresses; but it does not work for UK postcodes. This is thanks to Royal Mail who have a copyright on the data, and are very restrictive with their (expensive) licenses for it.

There are various solutions out there for using 3rd party services and importing the data to be used with Google Maps, or for using community built databases for the info. However, I’ve had a few people ask me about doing it just though Google.

It is possible — Google AJAX Search API does provide geocoding for UK postcodes. We need to use the two APIs in harmony to achieve our result.

So here it is.

Step by step

I’ll assume you already know how to use Google Maps API, and you came here just looking how to add geocoding for the UK.

Step 1.

Grab a two API keys, if you already have your Google Maps API key, just grab an AJAX search key. You can get them here:

http://www.google.com/apis/maps/signup.html

http://code.google.com/apis/ajaxsearch/signup.html

Step 2.

Google will give you a sample page, you need to stick your two API keys at the top of the page, followed by a reference to your Javascript file:

<script src="http://maps.google.com/maps?file=api&v=2&key=*KEY*"
type="text/javascript"></script>
 
<script src="http://www.google.com/uds/api?file=uds.js&v=1.0&key=*KEY*"
type="text/javascript"></script>
 
<script src="gmap.js" type="text/javascript"></script>


Ensure the reference to your Javascript file comes after the two API keys.

Step 3.

In addition to the Google Maps API stuff, you need to stick a reference to Google local search at the top of your Javascript file:

var localSearch = new GlocalSearch();


You can grab my Javascript file right here, but remember you’ll need to change the API keys.

Step 4.

The key to this Geocoder is only a single function:

function usePointFromPostcode(postcode, callbackFunction) {
  
  localSearch.setSearchCompleteCallback(null,
    function() {
      
      if (localSearch.results[0]) {    
        var resultLat = localSearch.results[0].lat;
        var resultLng = localSearch.results[0].lng;
        var point = new GLatLng(resultLat,resultLng);
        callbackFunction(point);
      }else{
        alert("Postcode not found!");
      }
    });  
    
  localSearch.execute(postcode + ", UK");
}


It takes 2 arguments; postcode is the postcode you want to look for, and callbackFunction is the function you wish to run on the results.

Why is it necessary to do it this way? It is the way AJAX, and thus Google AJAX Search API, works - the request is sent, and a callback function is designated to handle the results returned, when they are ready.

In our case, the callback function can do whatever you want with the results, which will come in the format of a GLatLng (often just called a point); I’ve supplied 2 sample functions, placeMarkerAtPoint and setCenterToPoint which do pretty much what they sound like they do.

Step 5.

Putting aside accessibility and graceful degradation for the sake of simplicity in this tutorial, the last step we need is just to add some hooks into our Javascript:

<input type="text" id="postcode" size="10" />
<input type="submit" value="Place Marker" onclick="javascript:
usePointFromPostcode(document.getElementById('postcode').value, placeMarkerAtPoint)" />


We have a field for inputting a postcode, and I’ve added a button for placing a marker there. Where I have placeMarkerAtPoint you can put a reference to your own function, or you can even add a function right in there, like this:

  <input type="submit" value="Do whatever" onclick="javascript:
  usePointFromPostcode(document.getElementById('postcode').value,
    function (point) {
      alert('Latitude: ' + point.lat() + '\nLongitude: ' + point.lng());
      })" />


Demo

If you are coming in from an RSS reader, either visit this blog post on the site, or see the demo page.

Postcode:

Conclusion

Until Royal Mail sort get their act together, and relax the licensing agreement, hopefully this will help people who want a ‘pure’ Google solution and hadn’t come across this option. Please use the comments section to let me know if you are using this, or if you have any improvements or suggestions.

Technorati Tags: postcode, google maps, api, Web 2.0, Web-Services

Is 2006 the Year of the Mashup? I think not. Mashups are at the stage that DHTML was at before it matured into Web 2.0 - lots of bells and whistles, but little real meat. 2006 will see people acclimatise themselves to the principles and technology, and maybe the best thought out mashups will become established. But it will be 2007 that will truly be the Year of the Mashup.

Programmableweb.com currently tracks 979 mashups, and 269 APIs, and shows a rate of 2.7 new mashups per day, so mashups are obviously very popular, so what’s the problem?

Who uses them?

How many mashups do you use, on average, everyday? How many of these were to actually achieve something (entertainment counts) and not just exploring what was out there? Now, as the sort of person reading this blog, you’re answer is likely much higher than the general internet populace; their answer is very likely zero.

The fact is: most people don’t know know what a mashup is (not that they need to to use one), don’t ever use them, and don’t know to look for them.

The use of APIs has become very well established, and many people are interacting with Google Map, Amazon Affiliate Stores, and Flickr streams as they browse the web, going about their business. APIs are mature and being used in creative, and most importantly powerful & useful ways.

Grow up, Mashup!

Mashups are yet to achieve the same level of maturity; they are very cool, a lot of fun and some are useful (especially in niche areas). However, I believe that 2007 will see mashups become mainstream; they will travel the path that DHTML travelled to become Web 2.0. They will shed their glamour and glitz, stop trying to show off with party tricks, and they will start to become incredibly powerful tools.

The large, and ever growing, streams of data provided by APIs can only go one route: tying the data together to improve and refine the way we get things done. We are already seeing people playing with this concept, but the unwashed masses of the internet are yet to reaps the benefits. What’s going to change this?

What did you do today?

While there are some mashups out there now which will become more established, and whose use will become widespread, there still seems to be very few of any real use. I don’t use any mashups, because there is nothing I want to do where a mashup provides an easier way to do something.

Normally any ‘problem’ they ’solve’ (matching news to a location on a map) is something I can do myself without the overhead and complexity (read the news story).

Learning from the past.

DHTML used to make pigeons fly round the screen as they followed my cursor, would make page elements jump around, my window wiggle and lots of other silly things to show off, while it was also doing something useful (i.e showing me the shipping address area once I said I wanted to ship elsewhere) which largely went unnoticed.

Mashups are the same - they do solve a problem, but they have a lot of stuff I simply don’t need going on. What we need are mashlets, smaller version of the same things, that solve our problem while staying out of our way. Ideally they would be integrated into my workflow (webmail, CMS, blog, host website, or desktop applications) to be there when I needed data combined to assist me.

My webmail may be able to map an invitation I’ve received, or otherwise use a microformat attachment that has arrived. Lastminute.com might show me a map of geo-coded photos when I am looking at holiday locations. Maybe my address book will allow me to Interfax a map of directions from a contacts location to mine for a meeting? There is a veritable plethora of possibilities.

Where next?

Mashups are here to stay, whatever form or name they end up taking. Maybe my idea of mashlets will make it, maybe not. In all likelihood there will be some large scale mashups that become established and popular. But one things is sure, mashups need to grow up before they grow out.

Where do you see mashups going? What mashups do you think will be going strong this time next year?

Technorati Tags: mashups, mashup, Web 2.0, Web-Services

Lots of people have commented on the day as a whole, so I’ll save repeating what they have said. To read them check out Technorati or Aral Balkan has made a list.

Overall, I had an fantastic time down in Brighton, the presentations were both fun and informative and I had a great time at the after party, meeting some awesome people who I really hope to meet again.

An extra bonus of the day was I won a prize for the Morse Code message I left for the podcast. I won several books, provided by Apress, all of which will be very useful.

On a side note: The Mac to PC ratio was about 9:1, which was much higher than expected. I’ve always been an Apple fan, and worked for Ambrosia Software for a number of years before moving to the web industry, so it was great to see the glowing Apples everywhere I looked.

Technorati Tags: dconstruct06

Well, that one wasn’t, but Jeremy appealed for more Odeo messages, so I thought it would be fun to send him a real one.

You can hear it here. I completely cheated and used this program to create the sound file.

Jeremy was stumped, and appealed for assistance. Stuart Langridge was one of those to decode the morse, which he did in a very neat way with a custom Python script, after converting the sound file. Very cool!

The message wasn’t terribly exciting, it was more about the challenge of decoding it. But for those interested:

Hi. Tom Anthony here. On the podcast you sounded like you really wanted a Morse message, or any message for that matter. Well, here you go. BTW I am so winning the golf.

Technorati Tags: dconstruct06, dconstruct

I’ve seen various other posts about the conference. But they seem to be missing the most important aspect of the conference…

The geek golf tournament!

Yes, this is incredibly exciting - there is a geek golf tournament at the conference after party (sponsored by Snipperoo).

My schedule for tomorrow actually looks like this:

9am: Practice Mini Golf.
4pm: Drive to Brighton.

To prove I’m not kidding, tomorrow I’ll be uploading some photos of my practice.

Update: Here are a couple of photos of my golf practice!

Technorati Tags: dconstruct06, dconstruct

You maybe are not in charge of 10 sites, do not have a long list of clients whose websites you work on, but are simply trying a DIY approach to getting your site better ranked in the search engines. However, when you search for “How do I rank better in Google?”, you are likely overwhelmed by the amount of information; SEO companies, blog posts, articles, forums, news articles and more.

So I’ve compiled this list of articles in the hope that at least some of budding students of search will find it. This is by no means all you need; SEO is ever evolving and you need to keep up to date (there are plenty of good blogs to help with that, which is another list for another time), but this list of 10 resources that will get you off the ground.

Overview / Introduction:

• A description and history of SEO
  This wikipedia entry is great for people who need an introduction to what SEO is before they go about learning it.
• Beginner’s Guide to SEO
  A fantastic and thorough guide to all you need to get started.
• Basics of search engine optimisation
  A great, to the point, list of stuff you need to consider.

Link building:

• 10 Tips to Train a Link Developer
  An annotated list of ideas and resources for learning link building.
• Sample Link Building Campaign Letter
  A brilliant link building letter template.

Keyword research:

• Keyword Research Guide
  Wordtracker is far from the only tool you need for keyword research, but their guide is a good, if lengthy for a beginner, introduction to the topic.

Code:

• High Accessibility Is Effective SEO
  Accessibility is great practice for websites anyway, but there is also an SEO payoff.
• 3C Markup Validator
  Checking that your HTML and CSS is valid is a good idea whatever, but having better written websites can only help.

Ranking factors:

• Search Engine Ranking Factors
  A comprehensive list of what are believed to be the most important factors that are used in ranking sites.

Press releases:

• Lowdown on Press Release Optimization
  In the right circumstances Press Releases can be a big help in getting traffic to your site. This is a guide on writing them well.

Technorati Tags: SEO, Search, Google

This will be my first conference since entering the web industry (I enjoyed conferences in London, Paris, New York, LA and San Francisco whie working in the Mac software industry), and I am looking forward to meeting some of the best in the business.

The schedule looks like it will be both fun and very informative. If you aren’t going then I suggest you take in the podcast so you realise you are missing you and make sure to book a ticket next year.

Richard Rutter has some more details, if you’re interested.

Technorati Tags: dconstruct06