Forgot to mention that I had dinner a few nights ago with Gary Angel and Phil Kemelor of Semphonic; I invited my WAA Social Media Committee Co-chair,
I really enjoy meeting with Gary and we often have dinner when he comes into Manhattan, usually a couple of times a year.
One of things I wanted to bring up and ask Gary about was his amazing work with Analytics Reporting - probably the best I've ever seen - and I wanted to know how he did it, how Semphonic did it. The post were the chart I'm showing below came from was titled More on Analytic Reporting Models - it was published a little more than a month ago:
My basic understanding of how Gary Angel created this amazing chart was this set of steps (and feel free, Gary, to comment and correct me if I left something out):
- need to meet with stakeholders, look at their data, collecting over a year's worth - find out all the possible variables that might affect any single measurement.
- All the data for each variable is pulled into a spreadsheet and a series of Visual Basic Macros are developed to take the data and create this chart of probabilities, based on the data.
We know this chart, for example, is about visits to a site - visits were going down in August and September but trending up in October ... why?
Normally, an analyst would have to figure this out by looking at various reports - but with the chart above, the answers are clear, and are ranked by percentage of probability with "less" Direct Traffic causing the lions share of the traffic decline which was reversed in October by PPC Advertising buys.
Now..... the Analytics Modeling of data pretty much spelled out what someone would have had to spend hours and hours to have figured out - and may have missed details, or missed whole relationships of data.
My belief is this concept of mapping data by probabilities is where Web Analytics Reporting needs to go - and today, Semphonic is the only Web Analytics Company that can create the chart above - figured out a way to do it and the programming and insights behind it.
Now that Semphonic accomplished it - I'm sure a dozens of people could copy the concept - but I think Semphonic will always do this kind of reporting better than anyone else - as they're creative enough to have come up with it in the first place.
I've been at several Emetrics Summits where I heard mantras that Web Analysts should produce insight, not reporting. Well .... for all that talk, few really good examples of Web Analytics Insights were ever shown ..... and this chart, above, is the best example I've seen, of Analysis, and not reporting.
If I ran a Web Analytics Group, I'd want to know how to create charts like this one for as many measurements as I could, and honestly, a data warehouse could be very useful here. Also, I'd want to standardize, as much as possible, my reports to every part of the organization, to be prepared in the same way.
Anyway, to finish off the evening - which Gary Angel treated us all to dinner - we talked about the latest Eric Peterson - Avinash Kaushik wrestling match? and the next XChange conference (which I'll attend) sometime later this year.
Hi Marshall. Great post, and always nice to hear of the social side on analytics on the other side of the pond!
It still surprises me how few analytic vendors talk about probability or significant change. In most cases even referring to the differnt forms of average (mean, mode, etc) can cause confusion!
I completely agree with you that the industry needs to make better use of data, by offering reports that show causal rather than co-orelational effects. For example, a report that shows that a particular campaign actually had a significant impact on revenue, rather than simply saying that it was a bit better than another campaign, without any reference to other factors such as seasonal change. Additional pattern analysis will become increasing important to point out any event that has changed significantly during the reporting period, such as the propensity of a particular search term. These should be compared against a previous period, and against the average, so that true changes can be shown, rather than a simple ranking.
Much of the above currently requires manual analysis, resulting in the need for a dedicated team or out-sourcing to a consultacy firm such as Semphonic or Foviance - we have the Experience Management Programme, but we also use our own analytical platform - WebAbacus - to build reports showing these type of changes.
The above requires a flexible analytic platform, to ensure that as an organisations thirst for data based decision making increases, so too can their analytical ability.
I have been speaking with Phil recently, and the need for deeper and more flexible analysis is absolutely key. However this needs to be balanced, to ensure that analysts aren't swamped with data, taking Web Analytics back to the days of data paralysis.
Posted by: Sean Burton | January 11, 2008 3:05 AM | Permalink to Comment