You know you have a bad day when you get a request to pull all the instances of a twitter # hashtag, and /or, all twitter traffic for an account, where the activity happened more than 10 days ago.

That’s been happening a lot to me lately, as I pull metrics on campaigns – trying to answer these kind of questions:

how many tweets about our campaign were generated (total) ?
how many tweets about our campaign were generated using the hashtag #xyz ?
how many people we (and others) reached via twitter (i.e., our total audience reach)?
how many total impressions?
how many responses were generated during the campaign event (including re-tweets, @replies, etc.)?

If you didn’t go into Twitter Search, create a RSS feed for all the activity you want to capture – then subscribe to it in Google Reader, or use some of the newer Twitter account backup programs – all along – your kinda out of luck – sorta.

Out of luck because Twitter doesn’t keep more than about 10 days worth of real time search data - it’s annoying (if you don’t believe me, go to Twitter Search and try searching on your own account – see how far back you can go).

I have no real argument with Twitter over not allowing users to search and access tweets over 10 days old, because Twitter, after all, is a “Real Time” Search Engine – and why should it be letting you pull data from the far distant past of even a few weeks – when Twitter has enough problems just staying up, and not crashing?

Twitter is keeping all the data, from day 1, in their databases – but you can’t do a search on it – because it would impact performance.   So what do you do?

I figured out a way around these limitations – using Radian6 (first choice), Alterian/Techrigy/SM2, could stand in, but isn’t as good for this – I test and found that out the hard way.

I just tested the process and got all the answers – it didn’t take me but a few hours to figure it all out:

1. Create a Topic Profile in Radian6 (or an alternative, if you have one)
2. Open up a River of News, segment by Micro Blogs (Twitter/Friendfeed)
3. Export the data into Excel and use the search within Excel to find # hasthag instances (which don’t work within Radian6 or Techrigy, etc), and any other terms related to your event/account.

Once you have cleaned up the data, filtered out any results you don’t want – Radian6 gives the number of Twitter followers which allows you to figure out your total audience and the number of impressions around your event.

It works – i proved it – and, anyone can do this  … but you need some kind of collection platform.   Now, it’s easy to say, “Just Use Google Reader” …. but that’s not the point – sure, I could have used Google Reader and an RSS feed, if I knew I needed to …. but lots of times, in life, you get requests for information you could not have anticipated, before hand.

The problem I highlighted, by not being able to get the data I needed directly from Twitter – suggests that users expect the Web contain every activity we’ve ever done – but the reality is – it doesn’t – that’s why I came up with the work around, I did.

And by the way – if Radian6 can tell me how to set up topic profiles for a “hashtag”, that would help, alot.