Social Marketing Analytics Framework Review – Measuring Conversation Reach – Part 3 of 10

Was meaning to follow up with this sooner – the last of the three KPI’s that fosters dialog – the Altimeter Group and Web Analytics Dymystified formulated in Altimeter/Web Analytics Demystified Social Marketing Analytics Framework is called Conversation Reach.

I wrote about Share of Voice and Audience Engagement previously.

A couple of thoughts

  1. Total “people” participating can be derived by isolating unique twitter handles, unique Facebook identities or other unique account handles when they are published publicly.
  2. Total audience exposure is a little more complicated, according to the framework as you have to associate the “conversation” with a particular “scope” via a) a given set of keywords, b) marketing initiative or c) a particular topic.
  3. After you have done steps 1 & 2, then use your Social Media Monitoring platform(s) to determine where the conversation is occurring (ie: YouTube, Twitter, Facebook, MySpace, etc).
  4. Audience Exposure would be the “total mentions” for a set of keywords, marketing initiative or topic which you’ll use to divide into your own presence in those same keywords, initiative or topics.

To me, this does take a bit of “data analysis” and time to do “well”.

As always, I’m working with one of my clients, the Cuban restaurant Havana Central as the case study; they most care about “Cuban food”, “Cuban cuisine” and “Live Latin Music” – since it’s a NYC restaurant chain – I am focusing on NYC rather than “The World” conversations on those things.

According to Sysomos MAP There were some mentions but not as many as I expected but lets go with it anyway. This kinda suggests channels upon which we’d measure the conversational reach.

Sysomos is about to release public Facebook monitoring as a result of F8 last week – but it hasn’t happened yet – so I’m not expecting much Facebook mentions to show up yet. And, as we have focused on NYC, perhaps what we’re seeing in numbers is not unrealistic – since we’re not going after “the world” – but then again, I haven’t added any mis-spellings and we can go over and over what the correct query should be – and maybe hardly anyone spends enough time on this step. For example, I could have added “NYC” or “New York” or “NY” along with “New York City” and gotten way more results – and maybe I should have. But what I’m finding – there is no “right way” to do this today – no standard – the more time you spend on the query – the more confidence you’ll have in the results, all other things being equal.

Total Conversation:

Query denominator:

(Cuban OR Latin) AND (“New York City” OR NYC OR NY OR Manhattan) AND (food OR cuisine OR “live music”)

Query numerator:

(Cuban OR Latin) AND (“New York City” OR NYC OR NY OR Manhattan) AND (“Havana Central” OR “havanacentral.com” OR “@havanacentral” OR havanacentral) AND (food OR cuisine OR “live music”)

The way I’m reading the KPI – you’d take each one of these as the denominator and make whatever Havana Central’s share of the conversation, the numerator in each channel.

The result is disappointing and does not match my expectations – I think it’s due to how I wrote the queries – but it’s true, maybe, social media outreach is not yet calibrated to those conversations I’m attempting to measure here.

Havana Central’s share

I’m not going to bother doing the math because I know this KPI does not actually match what is going on – there is a lot of buzz going on about this restaurant chain – but then again – I was so specific in my query that I may have missed a lot of other possibilities I could have included in my query.

That means this KPI, Conversation Reach needs more than what Altimeter or WAD provided in order to be usable – although it is a start in the right direction. It’s almost as if we need a precise definition of how to write the query that’s missing – or how to break down the question to be measured in a consistant way so that anyone can come up with replication.

Another assumption is that every message was from a unique account – but we know that probably wasn’t true – there will be some duplicates – so this KPI definitely needs data analysis – de-duplication, at least, in the numerator, but probably in the denominator as well.

Until we get a standard on how to write the query – I question the usefulness of this KPI but if we had the “standard” in place, this would be a helpful metric measure over time.

Reblog this post [with Zemanta]

2 thoughts on “Social Marketing Analytics Framework Review – Measuring Conversation Reach – Part 3 of 10”

  1. Thanks Marshall for doing an analysis of this Analytics Framework from Altimeter. I have encountered clients who want agencies to make use of this framework and KPIs to measure the success of their campaigns. As you have shown with your examples, the relevance of these KPIs becomes less when there is incompleteness of data due to the difficulty of acquiring them–and showing the client how you arrived with your numbers becomes a difficult if not embarrassing task.

    I hope somebody can refine these formulas to make it simple but effective KPIs.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>