The State of Data Quality Today


The State of Data Quality Today

A New Research Report from The Information Difference – July 2009

The topic of business data quality has been with us for decades. Given the large number of vendors offerings dedicated to resolving data quality (DQ) issues, one might be forgiven for believing that the problems have all but been resolved. A glance through the current literature reveals, however, that the problem of poor data quality is still very much alive. Has the state of data quality in organizations improved over the past two decades? Or is data management, and data quality in particular, still in a ghastly state with organizations and senior management feeling that the problem is overwhelming? Despite the clear concerns from business and a plethora of software vendors, there is surprisingly little concrete information available regarding the state of data quality in business. In June 2009 we conducted a survey, sponsored by Pitney Bowes Business Insight and Silver Creek Systems, aimed at gaining deeper insight into the views of businesses regarding their current or planned data quality initiatives. Some 193 respondents completed the survey from all around the world, the majority from Europe (47%) and North America (44%). A high proportion (39%) of the respondents were from companies having annual revenues greater than US $ 1 billion; respondents represented a wide spectrum of industries. Some of the main findings from the survey are summarized below:

  • One third of respondents rate their data quality as poor at best and only 4% as excellent. Fully half considered their data quality as good, although this may be somewhat over-optimistic when set against other results from the survey. For example one respondent told us “Poor data quality and consistency has led to the orphaning of $32 million in stock just sitting in the warehouse that can’t be sold since it’s lost in the system.”
  • 63% have no idea what poor data quality may be costing them.
    Surprisingly, 17% have no plans at all to start a data quality initiative, compared with 37% who currently have some form of data quality initiative in place. The remainder plan to introduce data quality in the next one to three year period.
  • Some two-thirds plan for, or currently have, data quality spanning either the entire enterprise or one or more lines of business.
  • A remarkable 81% say that their data quality is focused wider than just “name and address” yet this latter is the area in which most (>90%) vendors currently have their base!
  • The top three data areas for DQ were ranked as: 1) product data; 2) financial data; 3) name and address data. It is interesting that financial data occupies second place but virtually no current DQ vendors specialize in this area. Product data is rated as a higher priority than customer name and address, yet only a few data quality vendors specialise in dealing with product data quality.
  • The top two barriers to adopting data quality were: Management does not see this as an imperative and It’s very difficult to present a business case. This is interesting given that the majority (63%) have not attempted to calculate the cost of data errors.

The report has 33 pages.