On data, planning and not planning (cc @APGLondon)

Last week I went to an APG talk about data and planning – a pretty good one, may I add.

Andy Nairn spoke about using data to produce things that actually help people be better in some way, for example, either by bettering themselves (Nike Fuelband), the environment (Fiat Ecodrive) or their lives (SFpark.org) – definitely the future of agencies simply because our clients are heading that way themselves, and rightly so.

Sue Unerman told us that data can help brands target the right people, primarily online by virtue of recalibrating strategies based on analytics, but even then we shouldn’t lose ourselves in it. She quoted T.S Eliot: “Where is the knowledge that we have lost in information?”

A few things that Rory Sutherland mentioned bear writing down:

Variable admission bias, which means that the variables you put into your model will affect the output. So depending on what you are trying to prove you can manipulate stuff such that it works for you. Which kind of makes the whole model a bit redundant. Interestingly, I couldn’t find *anything* about it on Google. It’s a Googlewhackable phrase, if you will.

The importance of cybernetic feedback, or feedback which feeds into your (circular) process to improve it. That basically means working in an agile way, and evoked lean principles to me. I don’t know whether that was an exhortation to traditional advertising to break out of it because that’s really old hat now, but if anyone reading this needs to be reminded – stop planning on a 9-month cycle, for heaven’s sake. Babies are born that way, not successful business ideas, in this day and age at least.

The danger of category error in our business: sometimes we try to apply data to concepts that aren’t data-oriented, and we fall into a hole when we do that. I buy this outright, in that as much as I like what data can do in certain circumstances, sometimes there are nuances that it cannot reveal. He mentioned a great quote by Hayek from his Nobel Prize lecture in 1974 which is worth reproducing here:

While in the physical sciences it is generally assumed, probably with good reason, that any important factor which determines the observed events will itself be directly observable and measurable, in the study of such complex phenomena as the market, which depend on the actions of many individuals, all the circumstances which will determine the outcome of a process … will hardly ever be fully known or measurable. And while in the physical sciences the investigator will be able to measure what, on the basis of a prima facie theory, he thinks important, in the social sciences often that is treated as important which happens to be accessible to measurement.

(Emphasis mine).

Ecological rationality: based on research by Gerd Gigerenzer [PDF] and the Max Planck Institute for Human Development. The phenomenon of ecological rationality says that when given two choices, you tend to recognize the one you know better as the right choice. (I imagine contestants of Who Wants to be a Millionaire use this principle by default). This plays to Byron Sharp’s work of course.

He also pointed us to Goodhart’s Law, which essentially says (and this sounded complex to me at first but isn’t, especially if you go on to read about the Lucas critique which is the precursor of Goodhart’s law and essentially the same thing but specifically in a macroeconomic setting), that once you use a specific social variable for the purpose of creating a policy, it loses the ability to impact the policy based on its content. As this post says:

The most famous examples of Goodhart’s law should be the Soviet factories which when given targets on the basis of numbers of nails, produced many tiny useless nails and when given targets on basis of weight produced a few giant nails. Numbers and weight both correlated well in a pre-central plan scenario. After they are made targets (in different times and periods), they lose that value.

And Crabtree’s Bludgeon: In relation to Occam’s Razor, which says, more or less, that the simplest solution is the best, Crabtree’s Bludgeon says that if two mutually inconsistent observations occur, then they need to have some explanation or they – in fact – do not really exist. Read this obituary of Professor R.V. Jones, credited with creating this fictional logic, for more (there’s a pun within a pun there – Crabtree doesn’t exist and was made up by Professor Jones to illustrate that people sometimes blindly believe things that are quoted without proof). I guess his point was that data can lead to some bizarre suggestions – and that we need to stop believing such possibilities are practical because it may not make sense for us to believe it, especially in terms of business strategy.

All things worth thinking about.

There’s a summary of the event by the APG here.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s