Way Too Much Data Can Make Decision-Making Worse

Despite popular perception, too much data is not a good thing. Nevertheless, we live in a world ruled by numbers, and there’s an expectation that we can solve our most challenging problems if only we had more data—and perhaps a better way of analyzing it.

The COVID-19 pandemic should be a case study on the limitation of data. We were exposed to never-ending data and numbers, and yet, despite the availability of metrics, our actions weren’t practical with many governments acting too late.

I worked with a company that was completely overwhelmed by all the data. They weren’t sure how to make sense of all the numbers and what they should do with the information. For them, data was a curse. They were better off looking at 1-2 metrics and then moving forward.

I’m not here to say that we should go back to a world without numbers. On the contrary, data has been essential in building our modern world. Instead, I’m arguing that data isn’t a panacea, and you need to get the balance right.

You need the right amount of data

Whenever I ask companies why they want data, they're taken aback. It sounds like such an obvious question that it doesn’t require an answer. I disagree. The moment something becomes obvious to the point of evading questioning, that’s when we should be skeptical. The only answer that fits is that data is meant to help the business achieve its outcomes, whether in the short-term, such as revenue, or long-term, such as innovation.

If your data isn’t helping your business move forward, why go through the trouble of collecting, storing, and analyzing it? Companies assume that simply by collecting data, it will get converted into actionable insights. Instead, they should be working backward by starting with the outcome, then the insights needed, and finally, the data points.

Let me give you an example. A marketing team is interested in optimizing their user acquisition campaigns. Their outcome is lower acquisition costs. To do that, they need insights on what campaigns and ads are performing better than average. To find these insights, they’ll need to collect data on marketing channels and their impact on conversions, such as sales. From here, they could then figure out the best tools for collecting and visualizing the data.

The example above is simple but helps you ignore countless irrelevant data points. For example, sentiment data might be exciting but perhaps not very helpful here. You also don’t need data on every page of your site, just the pages involved in the user acquisition. This is a powerful demonstration on the importance of getting the right amount of data. No more and no less as long as you can extract insights and make a decision.

Finding the sweet spot

Your next logical question will be on how you find the sweet spot. We could debate the importance of metrics like bounce rate in the above example. From a technical perspective, collecting a metric like bounce rate doesn't cost more, which is why companies are drowning in data. It’s too easy to track hundreds of KPIs that no one looks at.

Instead, you want to focus on what is needed to make a decision. I’ve worked with marketing teams on the example mentioned above, and watched many teams go around in circles. You know you’re ready to make a decision whenever you start rehashing arguments. If you can make the necessary optimization without a metric like bounce rate, then you don’t need it. The best approach is to start lean and add more metrics in the future rather than start with as many as possible and try to remove them.

You’ll never have 100% certainty. Look at air travel right now—predictions expected business travel to rebound by 2023 and perhaps 2024, yet the rebound might even happen before the year ends. Airlines are shocked by the consumer demand and are unable to handle it. You can imagine that many of these airlines have sophisticated data capture and analysis, but they couldn’t predict what would happen. Don’t worry about being certain. Worry about your ability to change course and adapt.

Don't let data hold you back

Data isn’t good or bad. It’s a tool that can help you make decisions. When you see data as a hammer in the search for nails, it’s bound to fail.

Retailers like Wal-Mart and Target have also been caught flat-footed and with too much inventory as consumer behavior changes. I’m sure their data backed up their inventory choices, but they were better off with common sense. If consumer demand increased at the start of the pandemic as people were locked down, wouldn’t consumer behavior reverse once the lockdowns disappeared?

Give data the appropriate place in your company’s strategy but don’t get caught up with only making decisions with data. Common sense and intuition may go further, especially in highly volatile situations. Perhaps a change in mindset around data will be the most enduring effect of the pandemic.