Encouraged by various comments on Twitter I’m  trying out the Altimeter/Web Analytics Demystified  Social Marketing Analytics Framework for myself.  Even though there are 12 KPI’s spread across 4 business objectives – I doubt most businesses will use all of them – plus  I can only test 9 (have no access to “Facilitate Support” platforms or the use cases around them).

This post will only look at Share of Voice - and it will be a long.   Also, there are several ways to frame SOV for a business and the choice of how it’s done will be arbitrary.   It is great to have a Framework to fit into – but when you try to apply it to real problem – a real business – the results may be surprising.

As my use case – I’m working with a local restaurant chain in New York City as one of my clients and part of my agreement with Havana Central is that I can use them for Case Studies – still, I will be selective on what information is sharable .

Also – I do my work on the fly – much as I paint (when I do paint) much of the preparation happens in my mind before starting – been thinking about writing this post all day- letting my subconscious work on it before starting.

Earlier today – made an observation about the 12 KPI’s Altimeter and WAD came up with do not appear to address direct Return on Investment and wonder if  retail business establishments have  much to dig into here.    Right now, I think these KPI’s are designed  more for  large corporations than brick and mortar businesses.  Still, I will work have to work with the KPI’s in front of me.

Fostering conversations is the first Business Objective – are we communicating with customers and encouraging them to spread the word about Havana Central?  The first KPI is Share of Voice and is fairly easy to calculate.  But here’s where I run into the first red flag.

Who are the competitors?

Havana Central is a local NYC Cuban cuisine restaurant chain – comparing Havana Central them to larger national chains doesn’t  work – it would be  unfair to compare  a local chain to  The Olive Garden, Pizza Uno, Ollie’s and Houlihans – even as these restaurants are technically competitors – the share of voice metric needs to be modified in context of a business that is only competing, at this time, in Manhattan (a 4th branch in WestChester is in the works).

But, as we have one restaurant in Times Square, another in Union Square and yet another on the upper West Side – coming up with a list of competitors is problematic and open to interpretation.

So the first KPI must be applied somewhat subjectively- to even apply – at all.  Would the share of voice then be Havana Central against all local NYC Latin restaurant chains or would it be Havana Central against all local Times Square, Union Square and West End eating establishments – this is too big a problem.   It might be tempting to go back to the client and ask them (who they think their competitors are - and what if it’s Olive Garden and Pizza Uno plus some local establishments? – since they don’t know how to do it – that’s why they come to us – we’re the specialists) we may get mislead.  On the other hand, if we decide to come up with a search that’s too broad, or too specific – we’ll get results that are widely different and neither helps the client.

As an Analyst I will do this for Share of Voice (since there are no real guidelines – this is truly the Wild West side!).

Even going after local restaurants around each location of Havana Central is a problem – as there are so many possibilities – literally a jungle of restaurants to choose from - so you can’t apply this metrics without considerable interpretation – meaning I will get entirely different results that another analyst faced with the same question.

On one hand – using “analytics” and KPI speak, suggests our results ought to be precise – but the reality is – they are anything but precise.  My earlier posts showed how different each Social Monitoring platform is in Volume, Classification, Geo-Location and Sentiment – plus the analyst will skew the data according to the way they understand the task at hand.  This goes directly against what most customers/stakeholders expect of these tools – and it’s an industry problem – where Social Media Monitoring is today.

I will look at the last year.

Share of Voice>

“Havana Central” AND restaurant

——————————————

“New York” AND restaurant AND (“Times Square” OR “Union Square” OR “West End”)

Even here – should the competition be just Cuban or Latin Restaurants or all restaurants – clearly – that will largely be up to the client (in this case Havana Central – but since they don’t know how to measure Social Media – we must be sure, as mentioned above – to point out what is the best approach), since I’m acting in behalf of the client – it’s my decision – and for all I know, Eric T. Petersen will shoot me down and say …. no … do it this way or do it that way – except – he or John Lovett can’t -  as no one knows how to apply this metric - it’s just a general guideline - without the analyst and good insights – it’s worthless, perhaps even a detractor.

Using Sysomos – this is what I got – Havana Central got 2% of the online chatter of restaurants in the area – if that is to be believed (Twitter has a lot to do with it – if we discount Twitter – we get .5% or 4 times less.

The Share of Voice of Havana Central appears to have increased this year while the overall chatter of local area restaurants hasn’t – suggesting that having a Social Media community manager is, in fact, making a difference.  Does it translate into more mojitos being sold? … We don’t know – and this KPI will not tell us.

What if I geo-located and just considered queries that came from Manhattan – Sysomos can do handle that while Radian6 can’t – nor can SM2/Techrigy and I don’t think Scout Labs attempts to geo-located down to a city – so even here - what Altimeter/WAD came up with leaves a lot to the imagination of how your going to implement but since it’s a Framework – not a final solution - maybe that’s ok .

The local results for New York look  too good to be believed - of all the local chatter that actually came from “here” that Sysomos Map could pull up (they don’t use IP Address but rather – just the information they can find in the content they pick up – and it’s about 85% accurate – I’ve observed, based on other data pulls I’ve done.

Turns out Sysomos Map is picking up very little data that’s based just on NY so maybe the percentages aren’t that accurate – since we don’t really have much data.

Another problem is we many tweets and blog posts might use the name of the restaurant in the tweet, etc – but since we can’t know the totality of what we’re competing with – we must accept a  fuzzy query – fuzzy data – and the insights that come out of it or give up trying to measure Social Media, at all.

Based on the above – it looks like Havana Central is doing something right.

Looking at the guidelines that are published on page 13 of the Social Marketing Analytics Framework increases or decreases in your or your competitors online content is thought to be the main lever for Share of Voice.

After looking at the information I could get my hands or derive myself – decided that content wasn’t really the main driver – there’s been little change in Havana Central’s website for at least 6 months and whatever increases that were driven via Social Media to the site came mostly from Twitter.   In other words, Share of Voice has almost nothing  to do with content (if we were talking organic search traffic – that would be different ) – it has to do with people who are mentioning the brand Havana Central.

So maybe, what we need to look for in Share of Voice is the “Voice” part – what is being said more or less then before – or compared to competitors.

With that in mind I thought about local chatter of those who said, in one way or another, “I’m at Havana Central” but didn’t find anything that really showed there was that much more happening now than before.

At the end of the day – Share of Voice as a KPI might not be particularly relevant unless we have some business objectives around fostering dialog and specific areas where that is going to happen – otherwise we don’t have anything to put the changes we’re measuring online against.