Occasionally I read the Web Analytics Message Board that was created by Eric Peterson and now which the Web Analytics Association helps run and sponsor – I probably don’t read it enough (though I get the information by my RSS feeds and email); the post was about HitWise’s Search Intelligence tool, but I was more interested in what Michael Feiner had to say about Comscore and Nielsen and  I was struck by this Paradox:

“…..On their own admission Hitwise has a very small At Work panel, less than 1%. This, by the way, is a problem for the panel measurement companies as well (no matter what they say). The big corporates are very sensitive about being measured.”

All I can say is … ha, ha, ha ……  no one but the “big corporates” can afford ComScore or Nielsen – it’s so expensive – running into several hundred thousand dollars a year for a basic contract … and with new offerings by ComScore for Video, Segmentation, Audience Engagement, Advertising, etc – while all great info, dials up the price tag significantly.

And all the while – the basic accuracy for the “client” – the big corporations, is sabatoged by their own privacy policies.

You’d think the that there’d be some sort of “volentary” allowing of the Comscore and Nielsen clients on Corporate desktops, just so they information they feed back is “accurate” … but … that’s not to be.

So that makes me think the whole concept of using panel data, in the way it’s used, for whom it’s used for – while valuable (because there’s often no alternative) needs to be re-thought out.

For example, I know now that Comscore Video Metrix is using a combination method where a client can “tag” their own videos so that Comscore will more accurately count the traffic of the “client” …

That’s a good start, but won’t really scale well – since – …. hardly anyone else will do it AND ….. getting tags placed on pages is too much work, too much change control.

A better solution, but one that is totally unmentioned by any audience measurement service is to use sniffer technology such as Pion, made by Atomic Labs, which can be enginnered to work with Comscore and provide the data in a mannor that does not require tagging – and will be accurate.

So, instead of just charging Corporate Clients for all this money  – for the Panel data, suppliment it with Pion (give it away to the client and require it be installed as part of the contract) …… THEN …. the information coming to these “clients” will start to be more accurate.

It’s amazing to me that Comscore didn’t come up with this solution themselves – they put out all this expensive add ons – without really realizing that most clients are not even getting the full value out of the basic core functionality.

I just thought the whole situation was so full of paradox, I had to write a whole post on it – like the “blind leading the blind” kinda thing.

The Yahoo Message Board post on the Search Intelligence Tool also goes on to say that Comscore and Nielsen aren’t that bad, considering they have a limitation on how they collect and install the metering software:

“…The panel measurement companies have much greater control over their panel audience because they actually have significant information about these people (e.g. demographic data). Both comScore and Nielsen recruit their core panel prospects offline via telephone interviews (Random Digit Dial method or RDD). They supplement the RDD panel with cheaper online recruitment which is not as good “quality” panel but helps bolster panel sizes.

They will then use an enumeration survey to chart the US online population and will periodically weigh their panels to match the current US online population map.

These two constantly argue about the validity of each other’s recruitment methods but in general should have a smaller bias in terms of representing the US online population (vs Hitwise).

As I mentioned above as part of their recruitment process, comScore and Nielsen collect demographic data about their panel members. They then monitor these members’ online behavior on a 1-to-1 basis.”

Honestly, others have weighed in on which web audience measurement tool is the best and their problems, but I could not help laughing at the absurdity of the whole thing – your trying to sell “glasses” to the “blind” – if you can’t get accurate measurement of the Comscore or Nielsen client’s own traffic – how can you expect any of it to be that accurate?

Just a thought on this Sunday.