The End of Theory: I heart data

In the current issue of Wired Magazine (Jul 2008), there is a fascinating article by Chris Anderson, the editor in chief of Wired entitled, The End of Theory. Chris points out that scientists have always relied on hypothesis and experimentation but now in the era of massive data, there’s a better way…. mining that data.

Over the past few months, if you’ve been following this blog, you probably have recognized that I’m falling in love with data and the applications for taking large data sets and applying them to problems. On the 16th of June, I posted a lengthy blog post about Advertising to Audiences. I believe this post sets the tone for advertising theory moving forward. The most important take-away from that post is the illustration below:


Back to the Wired article. Below is a quote that stands out:

At the petabyte scale, information is not a matter of simple three- and four-dimensional taxonomy and order but of dimensionally agnostic statistics. It calls for an entirely different approach, one that requires us to lose the tether of data as something that can be visualized in its totality. It forces us to view data mathematically first and establish a context for it later. For instance, Google conquered the advertising world with nothing more than applied mathematics. It didn’t pretend to know anything about the culture and conventions of advertising — it just assumed that better data, with better analytical tools, would win the day. And Google was right.

So, data is interesting and could be the holy grail (whatever that might be). I believe it has a shot. It takes someone with extreme analytical skills to mine the data for correlations and causations and to really be able to go beyond the “templated reports.”

I’m fascinated with how this is going to play out in the advertising world. There seems to be many startups in the data/advertising world and they are fascinating. Adding data onto a single impression will make many marketers happy. How does this play out in practicality?

For example, as a marketer, you can purchase someone who is “in market” to buy a Subaru WRX. You can purchase this data from a data exchange such as Bluekai, and then use that data to market/re-market through any ad network. This is only the beginning.

Why is all of a sudden, data becoming the holy grail? In my world (advertising), technology is proliferating and cost-effective data stores and computing power is available to crunch data sets. It used to only be that the large credit card, mortgage, and financial institutions used to mine extremely large data sets because of the infrastructure costs, but as the cloud is here due to the Petabyte Era, almost anyone has access to these resources.

I believe that the potential winner here is not the team who can store vast amounts of data, but the team who can analyze it best.

I think that I may follow up this post with how this is applied to advertising and media.