I tend to find interesting angles for things that I read and more and more, things I’m going to write about in my book (I’ll have a website/blog up for the social media analytics book shortly where readers can give me feedback on ideas I’m developing to write about).
With that in mind I read a post yesterday on How To: Allow Google to Crawl your AJAX Content in JaTin’s blog that perked my interest because Google had actually solved the problem for content developers but most people probably aren’t even aware of it. Not only that, what Google can’t pick up (aka Ajax and Flash) isn’t available to Social Media Monitoring either.
Not that most of the Flash movies would be anything that Radian6, Brandwatch, Synthesio, etc, etc, etc, could do anything with, but then again, many movies, many Ajax sites actually do have a lot of content, but it’s invisible or Ultraviolet and fits nicely into my Spectral Analytics approach – clearly there’s a way to enable Ajax tracking of a website and Google wants you to know about it and implement it if you really care about getting that content ranked in Google. Once Google can get at it, I’m pretty sure that will also be the case with Radian6, etc.
But there’s another issue that comes up which I need to ask vendors about – how much does Radian6 or Sysomos actually pull data from Google? I mean, Facebook and Twitter are selling their data to Google, but do monitoring platforms actually look at Google to get data because I haven’t seen any evidence they do? If they don’t why don’t they?
I don’t love Google but I noticed no monitoring platform even attempts do deal with site ranking – why not? The same thing goes for Google Webmaster Tools – fantastic collection of information about a subscribed site, much of it may not apply to monitoring but some of it could – why isn’t anyone pulling data from it?
Anyway, getting back to Ajax enablement for SEO, it’s not that different than Flash Enablement in that you have to make 2 versions of the content, one for your users and the other for the search engine (sounds like Cloaking, eh!?). Well, who says all Cloaking is bad, maybe there are approved reasons for it and this appears to be one of them.
Essentially, sites following this proposal are required to make two versions of their content available:
- Content for JS-enabled users, at an ‘AJAX style’ URL
- Content for the search engines, at a static ‘traditional’ URL – Google refers to this as an ‘HTML snapshot’
Historically, developers had made use of the ‘named anchor‘ part of URLs on AJAX-powered websites (this is the ‘hash’ symbol, #, and the text following it). For example, take a look at this demo – clicking menu items changes named anchor and loads the content into the page on the fly. It’s great for users, but search engine spiders can’t deal with it.
As soon as you use the hashbang in a URL, Google will spot that you’re following their protocol, and interpret your URLs in a special way – they’ll take everything after the hashbang, and pass it to the site as a URL parameter instead. The name they use for the parameter is: _escaped_fragment_
Google will then rewrite the URL, and request content from that static page. To show what the rewritten URLs look like, here are some examples:
- www.demo.com/#!seattle/hotels becomes www.demo.com/?_escaped_fragment=seattle/hotels
- www.demo.com/users#!name=rob becomes www.demo.com/users?_escaped_fragment_=name=rob
As long as you can get the static page (the URL on the right in these examples) to display the same content that a user would see (at the left-hand URL), then it works just as planned.
Turns out this is exactly what Twitter just did (which also uses Google Analytics to track everything they are doing, bet you didn’t know what). Here’s an example from the post that was actually written by Rob Ousbey.
Now we know why the ugly pound-dash URL rewrite that Twitter had do to – they are tracking Ajax and following Google’s guidelines. Question is – will we? I think the answer is yes, if you want to get your content ranked and… picked up by Radian6 and monitoring platforms because they can’t read Flash and Ajax, either.
Fortunately for us, there’s a great demonstration of this proposal already in place on a pretty big website: the new version of Twitter.
If you’re a Twitter user, logged-in, and have Javascript, you’ll be able to see my profile here:
However, Googlebot will recognize that as a URL in the new format, and will instead request this URL:
Sensibly, Twitter want to maintain backward compatibility (and not have their indexed URLs look like junk) so they 301 redirect that URL to:
