My post last night on Forrester Wave Web Analytics – Then and Now ended with an open question, if a Forrester Wave report existed for Social Media Platforms; as luck would have it,  John Lovett, an analyst at Forrester responded (liking my Web Analytics Forrester Wave Comparison) and said  The Forrester Wave™: Listening Platforms, Q1 2009 (still recent enough to be relevant now – and I found a full copy online you can download and read) was the report I wanted to see.

As John Lovett (who I’ve met a few times at Emetrics Summit and XChange Conferences) explains, Forrester has pretty strict requirements to even consider a platform a contender – and many of the Social Media platforms that ought be here, are too small to be considered at this point – especially in Social Media – since it’s still an emerging space.  As John mentioned earlier:

… Additionally, vendors included met two specific selection criteria: 1) Corporate revenues in excess of $10 million; and 2) a substantial base of enterprise sized clients. According to that definition, several vendors that could have potentially been reviewed were excluded.

YES – there is a Wave for Social Media Monitoring: we call them Listening Platforms. Here’s a link.

Of course, the link John gave me would be the copy you have to pay for – and the link I gave me readers is one that Nielsen provides with the full report that is free for my readers.   A Forrester Wave for Social Media is probably going to miss a lot of the more exciting possibilities one can choose from, though it does include Radian6, which I work with.

The report starts out by citing related documents, but I’m not going to attempt to compare them since the Social Media space is changing much faster than the Web Analytics space – which it’s beginning to merge with.

1. “The Listening Platform Landscape” January 22, 2009
2. “New Uses For Brand Monitoring” April 16, 2008
3. “The Forrester Wave: Brand Monitoring, Q3 2006” September 13, 2006

Here’s what strikes me about the Forrester Wave report on Social Media:

” ….While brand monitoring platforms faithfully report on brand conversations and competitive news, they do not go far enough to turn this information into marketing insights.” – page 2.

Has anyone actually “formally” defined what a conversation is?  What’s the  protocol?  When systems talk to each other they have a set way to request for communications begin, continue and end (ie: TCP/IP communication protocol are defined).

“…. listening plat forms are in their infancy This is our first Wave evaluating listening as opposed to brand monitoring platforms. Not surprisingly, it revealed that this category is generally less developed than other interactive and direct marketing categories like enterprise marketing software, Web analytics, search, or email. While vendors aspire to deliver strategic insights to support the marketing organizations, they are often stuck in the world of tracking, monitoring, and delivering dashboards centered on operational metrics like mentions, reach, and discussion volume..”   Page 3.

So Radian6 qualified just because it had more than 125 customers, but Techrigy SM2 didn’t even though, now, or in 6 months, it would (see Two Interesting Meetings and the insights from them).

“…Too many solutions use the ‘neutral’ rating too liberally. I find ‘neutral’ ratings hard to believe. Consumers who are unaffected by our brand don’t participate. It is the happiest or unhappiest customers who willingly share information and insight with others.” Similarly, the platforms must correctly identify influencers and assign higher value to their feedback. Failure to do so is costly: Marketers will lose faith in the insights and abandon underperforming platforms”.

I don’t trust sentiment analysis that’s automated – it might work OK for some things like “chocolate” (tried that with SM2) but not for more complex analysis such as the “Iran Election” – the emotions are too complex, for machine analysis – SM2 tries, but Radian6 stays away from this kind of analysis because, as is pointed out – it’s too unreliable (see PDF2009 Workshop on “Social Media and Iran” with Katrin Verclas, Davar Iran Ardalan, John Kelly, Olivia Ma, and Nancy Scola and Combining Topic Aggregation from BlogPulse with Radian6 and SM2/Techrigy – going one step further).

My reasoning why Sentiment Analysis might work with a something like a “food item” or a consumable, is the choices are easier, I think, to pick out – and maybe automated sentiment analysis can do a decent job of it – but anytime you get more going on – you get a lot of false positives and a lot of neutral readings – like I did with “Firefox Problems” (see I’m fed up with Firefox, moving to Google Chrome).

“…Today listening platforms are self-contained silos of data that are focused on tracking a brand and its competitors’ presence in a wide variety of media. But marketers don’t operate in siloed channels.“  Page 6

The Forrester Wave ends with a warning to Radian6, which is currently the most popular platform for Social Media Monitoring, and one that I like, as well:

“…. Radian6, the new kid on the block, has managed to rapidly add a growing group of clients and partners. The biggest reasons for rapid adoption? A quick, easy-to-use tool that allows the PR professional to set up and define topics is an asset when combined with the ability to easily define custom influence metrics and reports. But to deliver full-fledged listening capabilities, Radian6 must improve in several areas like sentiment analysis, NLP, insight generation, and integration and consulting services.”

Let’s face it, most vendors for Social Networking don’t do much in the way of consulting unless your paying big bucks – I noticed this two years ago with KickApps.con Social Network platform that we brought on for the WAA Social Network my committee built at the WAA – called WAASOCIAL (waasocial.webanalyticsassociation.org) – I saw that Kickapps had great tools, still does, but for the most part, as with Ning – your on your own to figure out how to build your community – no one seems to be able to help much with that, or wants to.   There’s probably a whole business model around consulting that hasn’t even been tapped in this sector.

Anyway, any thoughts about The Forrester Wave™: Listening Platforms, Q1 2009?  Feel free to share your opinion by commenting.