Note: There’s an update to this post based on feedback from Jim Reynolds from Alterian -see the end of the post for the details.   Also- I heard back from Melyssa at Crimson Hexagon and Mashable is going to be publishing an analysis of President Obama‘s Health Care Speech later today or Friday – keep an eye out for it.

Since last week, as  I spoke with Melyssa Plunkett Gomez, at Crimson Hexagon, suggesting  Crimson Hexagon should analyze (much as they  analyzed the recent Disney-Marvel acquisition) Health Care reform and the Public option, that I had access to Crimson Hexagon so I could do the analysis, myself.

Not yet having access to Crimson Hexagon’s Opinion Monitor – I decided to go ahead and see what I could do with their much less powerful “Buzz Monitor” and compare it with Alterian/Techrigy/SM2 – to try to answer the fundamental question of how people feel about Health Care Reform and the Public Option.

Being a Web Analyst – and fairly hands on at that – I’m sharing the steps to how I derived the answer (which will be at the bottom of this post – so read on, all the way, if you want to know what I found out).

First Step, Build Monitors in Crimson Hexagon and Alterian/Techrigy/SM2 –

I built my queries (the same exact ones that are used on both platforms) using Google Insights for Search as my source for keyword data – this is probably the best data I can get about keywords people are actually using while searching about Health Care Reform and/or the Public Option.

I used “health care reform” as the first query in Google Insights and “public option” as the second query (here’s an example of the first query – you get the idea)

Queries:

Health Care Reform: “health care reform” AND (“health bill” or “health reform bill” or “health care bill” or “obama health reform” or “obama care” or “healthcare reform” or “healthcare” or “health reform 2009″ or “health care myths” or “health care facts” or “health care debate” or “hr bill 3200″ or “health care jobs” or “town hall meetings” or “hr 3200″ or “health care issues”)

Public Option: “public option” AND (“health public option” OR “health care” OR “the public option” OR “obama public option” OR “public healthcare option” OR “public option insurance” OR “single payer option” OR “public option plan” OR “public option bill” OR “health care reform” OR “public option obama” OR “public option poll” OR “public option healthcare” OR “public health option” OR “public health” OR “obama health care” OR “public insurance option” OR “health care” OR “public option support”)

While I said yesterday that no keyword tool that currently exists is built for Social Media, I do believe we can use search tools to  seed social media platforms such as  Crimson Hexagon, Radian6 and Alterian with keyword phases currently being searched for with the idea people are having conversations around  topics (keyword phrases) they are searching for.

Here’s Crimson Hexagon’s Buzz Monitor results for Daily Volume on both queries over the last month.  The Public Option had the most conversations on August 18th, 2009 (don’t know what happened that day).

Alterian/Techrigy/SM2 monitor for Health Care Reform, while not complete, is similar to Crimson Hexagon’s results (the Public Option profile is still running).

But that’s where the similarities end between both platforms; Crimson Hexagon’s Buzz Monitor indicates a slightly negative sentiment about Health Care Reform and The Public Option:

Health Care Reform Sentiment:

Public Option Sentiment:

To be honest, the “Topic View” Crimson Hexagon Buzz Monitor produces isn’t particularly helpful to me, any more than the “word clouds” that Radian6 and SM2 produce, help me get any insight at all.  Maybe the only thing I get out of the Topic View is a possible Keyword List for Organic and/or Paid Search – but that’s just me.  I admit – often felt Computer Scientists create what they find useful and assume everyone else wants it, instead of just asking people what they want in the first place.

Alterian/Techrigy/SM2 paints an entirely different picture and says the overall sentiment about Health Care Reform is Positive by a factor of 2:1.  The same is true for the Public Option (again, running the same exact query in Crimson Hexagon and SM2):

But when I looked at what SM2 considers a negative opinion, I got a hodgepodge of results, some which were negative, but others that were actually positive, but worded a different way (for example, if a blogger is angry that someone is denied medical care, the algorithm considers that negative, even though the blogger was really for the Health Care bill and the Public Option).

The Sentiment results from Crimson Hexagon appeared to be more relevant, when I read a few, even though I would not neccessary call them accurate.

Based on using both Crimson Hexagon and Alterian/Techrigy/SM2 I can say, neither one really answered the question, with any degree of accuracy and confidence, how people feel about Health Care or the Public Option.

I’m awaiting using Crimson Hexagon’s Opinion Monitor, which will give me the type of results you see, below – this kind of stuff is far more useful than what I could pull out of the Buzz Monitor or SM2:

… But .. you can only build these charts, well, using the exclusive Crimson Hexagon opinion monitor.

At the end of the day – aren’t much closer to knowing what people really think about Obama’s Health plan and the Public Option, than before; I’m disappointed – I had hoped one of these tools might have answered my basic question, but neither did, with any degree of confidence.

However,  Jim Reynolds from Alterian got back to me today and said I should have used a different report for sentiment than the one I did – here’s Jim’s words to me:

I was just reading your latest blog post and I wasn’t sure if anyone on my team every clarified the differences between Brand References & Content Tone.

Brand References will traditionally focus on a clear negative or positive conversation focused on a brand.  This is why you may have seen false positives.  We will traditionally walk our customers through tweaking the sentiment dictionaries to adapt the engine to the topic we are monitoring. For example if we did a campaign for the “angry whopper” we would add positive phrase to our dictionary so it doesn’t automatically rate the post as negative.

For a apples to apples comparison with the other providers I’d suggest using content tone.  This looks at the overall tone in the conversation vs references directly to a brand. From scanning the data this appears to be more accurate than the data that you’re seeing.

I’d love to discuss this in depth so I can share the details with the team on how to improve this piece of the application.

Well, the only thing I see, to be honest, is there’s almost all neutral or positive conversations – and that doesn’t match up with Crimson Hexagon’s Buzz Monitor – so while the information is more accurate, perhaps, it’s clear both platforms are counting sentiment differently – much differently, in fact.

I’ll be speaking directly with Jim Reynolds from Alterian, shortly.