By Yongxing Deng, co-founder and CTO of Aloft, a real estate technology startup based in Seattle, WA.
getty
As a business leader, you are often expected to use data to make an informed decision, regardless of whether your job title contains the word “data.” Everything from how much budget to allocate to a marketing campaign, to how many headcounts to approve, to what the sales projection should be. However, making data-driven decisions is not just a slogan, it’s a tool that has best practices to follow. Here are three common mistakes business leaders make while using data to make decisions.
Skipping Data Validation
When presented with a tight timeline (as we often are) and a data set, it is tempting to immediately start analyzing the data set. However, your findings can only be as useful and informative as the quality of the underlying data, so it is crucial that you spend sufficient time and energy validating the accuracy of your data set.
When it comes to data validation, start with a skeptical eye toward the data. Put your detective hat on and try to find the flaws in the data. Use your existing business knowledge to complete the following sentence: If the data is accurate, then ______. Then, use SQL or Excel to validate those assumptions before proceeding with the actual analysis.
Underestimating The Impact Of Low-Probability Events
Events that are less likely to happen can sometimes have an outsized impact on the goals you are trying to achieve. As an example, while pandemics happen rarely, few businesses around the world haven’t been meaningfully impacted by Covid-19 in the past few years. As a business leader, it is impossible for you to foresee all the low-probability events that could happen, and yet you are often having to make a decision anyways. What do you do?
One approach is by explicitly asking yourself: Given the duration of the data available, what might the data not have “seen?” For example, if you have two years’ worth of sales data, then you can assume any rare events that happen once a year have probably been included in your data. As such, the events do not need special attention to be accounted for in your analysis. On the other hand, if you have only six months of sales data, then you should work with your team to think through situations that might only happen once a year (seasonality comes to mind) and use your business judgment to supplant your data findings. Presenting a list of low-probability, high-impact events alongside your analysis can often help your stakeholders make much better decisions.
Overlooking The Power-User Effect In Your Analysis
Let’s say you are a gym owner, and you are trying to estimate on average how often your members exercise at your gym. One “easy” way to do this: Stand at the front desk, ask the next 20 members who walk by how many times they have visited the gym in the past month, and take an average of those 20 answers. Beware—the average you derived this way will not represent your entire membership population. Why? Because a frequent gym goer is much more likely to be surveyed by you than a member who only visits the gym once a month.
When conducting an analysis on product usage, you must carefully examine whether the methodology you use results in findings biased toward your power users. This is not to say that you must disregard the results you find this way, but this does mean that you should proceed with caution.
It is not an exaggeration to say that many of our work lives now revolve around data. As business decision-makers, we must treat data analysis as a powerful tool that also has traps and mistakes and serious ways to cause damage. By combining data with our own intuition, and constantly challenging our own methodologies, we can maximize the utility of data analysis.