I’d like to think I’m a numbers guy. I may not be the best at financially modeling, but when I see numbers on the screen (or on paper) and can synthesize them, I generally get the big picture of what they are telling us.
Numbers are great because we usually assume they are correct. Here’s the typical issue that arises:
- Darren: We have 5 Apple products in our home
- Sherri: We have 4 Apple products in our home
Uh oh. Who is right? Darren or Sherri? What I’ve learned so far in marriage is that Sherri is correct. This is a significant issue however, as both of us are fairly intelligent and representing what we have in our home. You would believe either of us if we had told you how many Apple products were in our home.
The most logical way of figuring out how many Apple products in our home would be to go home and count one-by-one and validate them to each other. In the end, one of us is going to be right (or none, if we both can’t count).
What if we had $1,000,000 on the line here? What if this was Panasonic advertising on ESPN.com? The buy-side ad server reported X amount of impressions and the sell-side reported Y amount.
X /= Y
Folks, we have a dilemma here. I’m shocked that this has not been in the media as much as it has. Media Buyers stay late at ad agencies all night because of this. When we receive campaign reports for digital media advertising, the numbers are generally off a percentage. There are a few factors that we write off immediately, but there are some major discrepancies normally. Yikes.
What happens when we are planning to place media on sites that are supposed to have Z amount of monthly uniques and turns out, they have A?
Z /= A
Folks, we have another dilemma here. If we know that we must hit a certain amount of unique visitors to reach our advertising goals, and we expose a lesser amount, we failed. This is a problem.
There is a lot of commotion surrounding these topics this morning. Numbers are extremely strong when represented on paper (or the screen) but many people do not question them.
One of the largest debates around numbers and advertising is the methodology of generating a sample. How many people should be included in a representative sample? Good question and one that I’m not going to debate personally, but I’ll point you in the right direction to get that information.
I’m interested to see how these problems play-out as in digital media, most everything we do has an ad-serving component to them. If we can reduce the discrepancies, our life would be so much easier.
Another case in point: SiteMeter, Google Analytics, and QuantCast all report different traffic numbers for this blog; sometimes, by an order of magnitude.
If you enjoyed this post, make sure you subscribe to the RSS feed!
Digg
Del.icio.us
Stumble
Sphere It
Category: 
Tags: 
October 22nd, 2007 at 8:33 am
Acutally my answer would be 6: 2 ipods, 2 iMacs, 1 iphone and 1 apple tv.
October 22nd, 2007 at 9:09 am
In the past I’ve done a lot of analysis reports, both demographic and financial. I’ve done them for the companies that I’ve worked for as well as the clients, both pre-sales and post-campaign. The fact is that I could always tailor the reporting to the requirement. It was never lying or dishonest, but it’s easy to accentuate the positive and avoid the negative.
Here’s one for you… My Technorati rank has dropped from under 2,000 to 3,400+. Rather than tell people that, I can simply state that my Technorati ranks is still in the top 3,500. Sounds impressive, right?
Lies, damn lies, and statistics!
October 31st, 2007 at 8:01 am
[...] and have been reading up on companies like Compete and Alexa (amongst many others). I wrote a blog posting a few weeks ago talking about when web stats don’t match up on campaigns (buy/sell side [...]
April 4th, 2008 at 5:42 pm
[...] back a ways to a post by Darren Herman, Numbers Don’t Lie, Except When They Do, he touches on a very important point in the measurement of visitors, which set of data is [...]