Is your data giving you problems? Are you having trouble finding the information you need or having the analytics at hand to key decisions?
Big data is often presented as THE solution for making better decisions and getting better business results. But that’s only true if you’re using good data (and if you know what to do with it). That means your data needs to be:
* Accurate
* Precise
* Valid
* Consistent
* Relevant
* Complete
* Accessible
* Reliable
But how can you be sure the data you collect and analyze meets those qualifications? You start by knowing what bad data looks like.
Seven Signs of Poor Quality Data
Poor quality data fails to exhibit the characteristics I just listed. But you can’t always measure by these factors just by looking at your data. So you need to watch for other signs. Here are some telltale signs that your data is lacking in the quality department:
1. You don’t have a strategy for collecting, processing, and handling data
Data governance is a hot topic these days, and with good reason. With so much data streaming into your company every day, you need a strategy for deciding how to handle it. Data governance establishes universal data definitions, protects data quality, and determines who will manage the data and how. Building out the specifics of a strong data governance strategy ensures that your data is accurate and useful.
2. You don’t have a documented data management plan
Data management is the implementation of data governance, including architectures, tools, and data policies. Managing data well means following standard procedures for how it will be collected and processed, where it will be stored, and what rules govern its use. As you can see in this illustration, all the various disciplines of data management center on your data governance strategy:
If you don’t have these first two elements in place, you’re going to notice a lot of problems cropping up when you try to do things with your data. Let’s turn our attention to some of the most common data hang-ups.
3. Your data results vary widely between reports
If you get vastly different results from one report to the next, red flags should start waving. That means one of two things: your data is bad, or your analysis is faulty. When you collect new data, you need controls to be sure it is accurate. You also need savvy employees who will raise questions when data results don’t make sense. If your reports show outrageous results, check to see whether the data is inaccurate or your data sets are at odds.
4. You collect more data than you have time to analyze
More isn’t always better, especially when you don’t have time for accurate or meaningful analysis. Excessive data collection opens the door for errors. To reduce your chances of drowning in data overload, look for ways to make better use of the data you already collect rather than piling on more and more data points. This is also a good time to take a look at your data collection methodologies. If most data gets entered manually, you’ll have a lot more opportunities for error than if you can automate some or all of your collection process.
5. You store key information in multiple, disconnected places
There are several problems with not having a centralized, integrated database:
You get duplicate records
Duplicate records create a ton of busywork, as you try to filter through the noise to get usable information. This annoyance is a sign that something is wrong in your data management plan.
You can’t easily analyze the data
Data silos make key data difficult to access, and will eventually impact your decision-making. If the goal of Big Data is to drive better business decisions, it doesn’t make sense thwart your efforts with fragmented data storage practices.
You can’t update data easily
What happens when a customer changes his address and you have that information stored in six different, unconnected places? You’re looking at six times the work to get it changed in your system. And if you miss one, the customer might not get his shipment on time. The same is true with any data point. If you have to update it in multiple places, then your overall strategy is lacking.
You risk violating data compliance standards
Some industries have strict data control standards like HIPAA or PCI that companies must follow to protect privacy. Inefficient data storage can put you at risk of penalties for compliance violations.
If you’re revisiting data storage solutions every year, it means your data infrastructure isn’t cutting it for you. As your data volume grows, your storage solution should be able to scale. If it doesn’t—or if it creates more problems than it solves—it’s time to find a new solution.
6. Company decision-makers don’t trust the data
Here’s another major red flag. A recent KPMG survey of global CEOs found that more than half don’t trust the integrity of the data they use to make decisions. They’re concerned about data quality, long-term effectiveness, and the accuracy of analytics. And that means they’re still making decisions based on instinct and experience alone. These decisions aren’t necessarily bad, but they can be faulty. Data—the right data used in the right way—should support instinct and experience with evidence.
7. You don’t know when to keep or purge data
When is data no longer useful? When does the cost of storing it outweigh the benefit it provides? When are you collecting more data than you can use to your advantage? These are important questions. If you can’t answer them, you need to take a hard look at your data policies.
Is Your Data Suspect? Here’s What to Do About It
If you recognize your organization in some of these scenarios, then you already know you need to take action. The question is: what should you do? Here’s a quick rundown of steps you can take to clean up your data and keep it viable:
Start with data governance and data management
Strategy and policy form the foundation for effective data analytics. You can’t collect data willy-nilly and expect to have useful information at the end of the day.
Build a consistent data storage solution
Next, upgrade your data storage to eliminate duplicate records and make your data easily accessible. Break down data silos so all parts of the system can talk to each other and share information.
Automate data collection
As much as possible, automate data collection to reduce data entry errors. Review all of the manual steps in your data management processes to and automate where possible to improve efficiency and accuracy.
Perform regular data audits
Even in the best systems with the most stringent precautions in place, bad data creeps in. Periodic data audits should assess the various methods and tools you use for collecting and storing data, clean out old, inaccurate or unused data, and consolidate any rogue data collection that hasn’t made it into the centralized database.
Conclusion
IBM estimates that bad data costs U.S. businesses over $3 trillion every single year. That’s a shocking figure, but this one makes it more personal: 83% of companies believe their revenue is affected negatively by poor quality data.
Bottom line: Bad data hurts. And it keeps you from taking advantages of exciting new technology like IoT, AI, and machine learning. Investing in good data architecture, governance, and management positions you to enjoy higher productivity and increased revenue. It’s time to take the next step.
