The first post in this series examined the volume of “buzz” around Social Media Week NYC that is taking place next week here in New York.   What is the buzz made up of?

Remember – I’m using exactly the same query in all the platforms for the same period of time – the only way to do this.

Radian6

Alterian/Techrigy/SM2

Note: comparing Radian6 and Techrigy we already see a problem - Radian6 counts 390 blogs while Techrigy counts 66 blogs – Radian6 gets 964 Tweets while Techrigy gets 450, but both have very little coverage of Facebook, which is still mostly a walled garden.

Sysomos has 307 (closer to Radian6)  blog posts and 165 News mentions (similar to Techrigy) while Tweets are a sampled and Sysomos says as much.

Brandwatch numbers are much lower – but we don’t know if that’s good or bad yet.  News had 30 items while 72 blog posts and 24 tweets were found.

Biz 360 had 677 tweets and 168 Blog posts

My take away is that most people will never get the opportunity to meaningfully compare monitoring platforms in the way I am doing … because they can not get access to the platforms and don’t have the curiosity to test them the way I have. But now you can see just how different they are in their abilities, today, to track the same query across social media on the web.

On another level – it probably doesn’t matter that much each platform is so far apart in results it collects, because people will tend to believe the version of the truth that suits them the best.   Still, without the knowledge of how different they really are – an informed decision can rarely or ever get made.

My next post will be about Sentiment Analysis of these 5 platforms  – whew – that is a hot topic!

This post is the first in a series of comprehensive comparisons of 5 social media monitoring platforms I have access to which will be part of my course at Monitoring Social Media Bootcamp taking place in London on March 31st, 2010 (two months from today) - Radian6, Alterian/Techrigy, Sysomos, Brandwatch,  Biz360 Community Insights.  I also have access to Crimson Hexagon Buzz Monitor but am not including it in this analysis – I’ll be happy to include Crimson if asked.

I’ll use the John Battelle approach and develop my content and thoughts online, and let the community of readers comment – and your thoughts will add to my course.  In return, I’ll share with you what I know.

Sound fair?    Let’s start with the query – “Social Media Week New York City“  which is going to be taking place next week and will give me three things  I need to test and compare these platforms on

1. An Event
2. A Location (test Geo-Location accuracy)
3. Sentiment
4. Influencer Lists of Authoritative bloggers
5. A Summary of Social Media Week NYC after it’s over using 5 platforms.  Which ones will end up being most useful?

A couple of thoughts before I start  – and this series of posts will take time to complete.

1. How these  platforms perform today may be different in 6 months or a year from now – I am analyzing them as they are today.  In the future, the same tests should be run again.
2. There is no “control group” – we don’t know what is absolutely true here or not – there is no ideal monitoring solution – but there are monitoring solutions and results that are more useful than not – and I suspect, the value of these platforms is the usefulness and “cleanness of the data” .
3. I choose “Social Media Week New York” because there is a known attendee list (if I’m willing to data-mine for it) and I can judge geo-location better if I know who’s attending and tweeting / posting in NYC about Social Media Week NYC.

For today – I’m going to compare the tracking of Social Media Week – which hasn’t happened yet – but there have been a lot of posts about it up to this point.  Plus, the same exact query has been entered into all 5 monitoring platforms.

1.1.  Volume of Online Messaging about Social Media Week NYC – Jan 1-30th, 2010

Radian6 – shows 1364 items or posts

Alterian/Techrigy/SM2 – shows 1551 items or posts

Sysomos – 628 posts or items

BrandWatch - 147 posts or items – there may be more later – may still be processing

ddd

Biz360 shows 845 items or posts

Just from the comparisons above it’s clear that each platform is counting differently -   Without knowing the back end filtering each platform is doing it’s hard to know, based on this, what is the right number of items or posts is or should be.

Who knows with absolute certainty how much “content” was created on the web about Social Media Week NYC 2010 in the last month, so far?    How can I prove that any of these numbers is absolutely right or wrong?   I can’t.

The main thing all platforms catch is the a spike of data on 1/29th

However, I can look at the items each platform collects to evaluate how useful the information could be to me.

My next post will examine the quality of the data each platform picked up.  Please bear with me in my approach – I actually think going step by step, comparing each platform to each other for each “dimension” (similar to Web Analytics Dimensions) is the best way to get started.   As this series goes on I will use each platform for a deep dive – testing it out.

What I learned so far is each platform counts differently for the same exact query – therefore – we may want to consider how each platform is collecting data off the web.  I invite vendors and interested readers to come up with suggestions – perhaps those who program these platforms have a better idea than I do of why the numbers are different.

I think the part that might scare people about the difference in volume of posts or items is that lack of agreement on how to crawl or count social media content in the first place.  Suppose there could also be differences in how my query is processed by each platform – some, like Sysomos and Biz360 allow you to specify distance between words (exact, tight, loose) while others like Radian6, Techrigy and Brandwatch don’t appear to specify it – but may still process it.

All the more reason there should be some set of standard ways of processing information – more about that in a future post.

I recently talked with Stephen Debruyn recently about his list of social media monitoring vendors – DEFINITIVE GUIDE TO SOCIAL MEDIA MONITORING/CORPORATE REPUTATION MEASUREMENT FIRMS and felt I might use the guide  as a jumping off point to how Social Media Monitoring grew in 2009 and I’m working on a PowerPoint Presentation, now.

Note: in the post following this one – I put out a theory that Google is the main driver of this increase – which makes sense to me since Google is the driver of so much Web 2.0 Traffic – see 400% Rise in Social Media Traffic due to Search Engines – Google for more information and the next piece of this puzzle.

Used Alterian/Techrigy/SM2 platform to monitor what was being said about each brand in Debruyn’s guide with additional queries for “social media monitoring” and “social media buzz”, etc.

I noticed a curious thing – since this summer the volume of conversations about Social Media (focusing on monitoring Social Media) increased 400% from what it was at the beginning of this year, and for all of 2008.

I questioned the accuracy of the data I collected from Alterian, as I this profile took up most of  my 200,000 searches limit and wondered if it didn’t actually complete – and that’s what I was seeing.

Used Google Insights for Search and found, more or less, a similar pattern of “Social Media” searches increasing by 400% this year – and compared it to SEO, SEM searches, which haven’t changed.

That gets me to the reason I am writing this post – not so much to find out about the 400% increase, as interesting as it is, as to see which of the social media monitoring firms, which make a living monitoring the web, are generating buzz around their own brands, and highlights of what what was being said about them.

More later, as I complete this study and try to figure out what it all means.