Pandemic Analytics or Lies, Lies and Damned Statistics

analytics text
Photo by Timur Saglambilek on Pexels.com

Leaving #digitalmarketing means I have a chance to call things out thst I previously would have put aside for fear of looking like I was going out of my way to have a pop at competitors.

But… we just need more budget…

Something I’ve noticed this week is a number of marketeers on social media asking for understanding and clemency because they can’t reproduce last years pandemic-powered results.

Well… of course they can’t. Lockdown was unprecedented (I’ve come to hate that word!) and it had an unprecedented but predictable impact on online performance.

Marketing agencies begging for understanding and clemency now weren’t honest with clients then – they were riding the crest of a wave  created by a captive audience with time and money on their hands. It couldn’t last.

Confirmation Bias in Website Analytics

To avoid being unkind, there is an alternative explanation – Confirmation Bias. Agencies working hard for customers, especially during an extreme event such as a pandemic, are likely to attribute positive changes in website performance to their own actions because that was what they were expecting to happen. A form of Selective Perception, the statistics are viewed through the lens of “We did X, and the website stats did Y” without taking into account that perhaps the website would have “done Y” even if you didn’t do X at all.

This is a trap it’s very easy to fall into and it’s why applying things like The Jurassic Park Test are very important.

It’s crucial not only to understand why you are making a change to your website, but to have controls in place so that you can understand what could happen if you didn’t do/stopped doing that activity.

How to judge if an agency really understood what was happening…

If you really want to get the measure of an agency and whether they understood what was happening during the pandemic, just scroll back through their social media posts and get a feel for whether or not their actions, and outcomes, match their words.

My advice? Stay away from anyone who was keen to share how well they were doing during the pandemic and crowing about their performance. Like a turtle on a post, they probably had no idea how they got there and no idea what to do next.

Be the first to comment

Leave a Reply