The Information Difference Company

The Data Quality Market - Q1 2010

The data quality market for the calendar year 2009 was worth around $803 million, of which software sales and maintenance accounted for around $637 million. This figure includes the professional services arms of data quality vendors, but excludes the (substantial) revenues of systems integrators and consultancies involved with data quality initiatives. This represents 9% growth over 2008, despite the difficult economic climate. The last year or so clearly saw dramatic economic events, especially in certain industries such as banking, and data quality companies saw fluctuating fortunes. Some experienced significant revenue declines, but others prospered, with Datanomic, for example, doubling its revenues.

Certain common themes ran through the industry. There is a clear emergence of software-as-a-service data quality offerings, with existing vendors seeing significant interest in their fledgling offerings, and some newer entrants such as Active Prime basing their whole business model on this approach. While some customers clearly have concerns about allowing their data outside the firewall, the advantages of avoiding potentially complex installation and upgrades on their own premises are beginning to outweigh these worries. Some vendors have been at this for longer than others – Uniserv has had such an offering for eight years now, and has seen it grow into a significant business line.

Data volumes continue to grow, and some vendors have responded by re-architecting their products for 64 bit architecture, and in some cases to allow massively parallel processing (MPP) approaches. Such approaches allow profiling, for example, to be carried out on large datasets rather than just relying on sampling. The data quality industry is hardly new: Innovative Systems has been providing data quality software for 32 years, yet there continue to be new uses and demands placed on the technology.

One theme is an increasing interest in identity resolution, beyond just name and address verification, as a reflection of the heightened concerns about both terrorism and on-line fraud. Infoglide built its business on this in the US (and Infoshare specialises in this area, mainly within the public sector in the UK), while DQ Global is another vendor who emphasise this capability.

As companies expand globally and Asia’s economies continue to develop, support for more exotic character sets becomes important: many vendors now support Unicode, and some have gone further. Omikron in particular provides software that tackles culture-specific name validation in various Asian character sets. Even established areas of data quality such as data profiling are seeing new entrants, in the form of X88.

A recurring theme of our client research is how customer name and address is not the only priority for customers – product data in particular is viewed as a major problem for enterprises, and is much more complex and less structured than name and address data. Companies such as Silver Creek (now bought by Oracle), Datactics and Inquera specialise in this area, while some other vendors are data domain-agnostic and have been adding functionality that is relevant to handling other datatypes.

Consolidation was a theme in 2009, seeing several significant acquisitions. Oracle purchased Silver Creek, matching specialist Netrics was bought by Tibco, and profiling vendor Exeros by IBM. In general, vendors continue to build out their offerings to encompass a full range of data quality functionality, sometimes through the OEM of specialist technologies. Some larger vendors (such as Informatica, IBM and SAP) now offer a broad suite of software encompassing data integration, master data management and data quality software. Microsoft is likely to follow this path in due course, given its prior acquisitions of Zoomix and Stratature.

The emerging area of data governance is beginning to have a distinct influence on the data quality industry. As the popularity of enterprise master data management continues, data governance initiatives have been set up, and the scope of these usually includes data quality. Hence data quality vendors are seeing demand for support for data governance initiatives, and many have added functionality in this area, such as support for the work done by data stewards. There are increasing connections between master data management and data quality, with many MDM vendors using OEMs of data quality vendors to round out their offerings, or in some cases (as with Tibco and Netrics) buying data quality vendors outright. One interesting development has been the other direction, with Ataccama for example starting with a data quality product and then releasing an MDM platform, a route that open-source vendor Talend has also followed; having started with data integration, Talend now has both data quality and MDM offerings.

Data quality remains a fragmented market. Since many vendors specialise in name and address verification, companies with deep local knowledge have built up offerings that track people and companies that move address in order to avoid badly targeted mailing and ensure optimal bulk rates for mass mailing (such as Satori Software). Significant enrichment of postal address data is possible via geocoding. This technique allows customers to not just check the postal code of an address, but to see it displayed on a map, and to display such things as its distance from nearby stores, the demographics of the area, its political constituency, or even whether the address lies within a flood plain. Pitney Bowes Business Insight (with its Mapinfo technology) is one vendor that majors in this area, with Capscan being another example. While most vendors support web services, some vendors specifically design their products to be embedded in other applications, such as Intelligent Search (Melissa Data and Netrics are others).

The diagram that follows shows the major data quality vendors, displayed on three dimensions. See later for definitions of these. The largest vendors of data quality in terms of revenue are Experian QAS, SAP, Informatica, IBM, Trillium and DataFlux.

It is important to understand that this is a high-level representation of the market, with vendors represented on the chart specialising in different areas and at very different price-points (HelpIT is an example of a vendor with a quite complete data quality toolset at a low price-point; Datras is another). If considering data quality software, it is important to tailor your selection process to the particular needs that you have rather than relying on high-level diagrams such as this. The Information Difference has various detailed models that can assist you in vendor selection and evaluation.

As part of the landscape process, each vendor was asked to provide at least eight reference customers (some provided over 20 references) which were surveyed to determine their satisfaction with the data quality software of the vendor (if insufficient references were provided then a neutral score was assigned). The happiest customers based on this survey were those of Melissa Data, followed by Active Prime, Datanomic, Pitney Bowes Business Insight, Satori Softare, DataFlux, Datactics and Infoshare.


Main Vendors
Below is a list of the main data quality vendors.

Other vendors of data quality software include:

• AMB DataMiners ( http://ambpdm.com/)
• Ciant ( www.ciant.com)
• Data Lever ( www.datalever.com)
• Data Mentors ( www.datamentors.com)
• Infosolve ( www.infosolvetech.com)
• Intervera ( www.intervera.com)
• Irion ( www.irion.com) [an Italian vendor]
• Ixsight ( www.ixsight.com)
• MSI ( www.msi.com.au)
• Rever ( www.rever.eu)
• Stalworth ( www.stalworth.com)
• TIQ Solutions ( www.tiq-solutions.com)
• Winpure ( www.winpure.com)
• Wizsoft ( www.wizsoft.com)

Research Methodology
The Information Difference Landscape diagram shows three dimensions of a vendor:

• Market strength
• Technology
• Customer base.

“Market strength” is made up of a weighted set of five factors: revenues, growth, financial strength, geographic scope and partner network. Each of these individual elements is scored, the total producing the “market strength” figure. Similarly “technology” is made up of four factors: “technology breadth” (the coverage of the vendors in various data quality areas as illustrated below), the longevity of the software in the market, analyst perception of the product via briefings, and customer feedback from reference customers (this has a high weighting), which we surveyed. In each case the scoring is on a scale of 0 (worst) to 6 (best).

Vendors were asked to submit answers to various questions via a questionnaire. Vendors were interviewed directly by an analyst and their software demonstrated and assessed. Reference customers were surveyed to give their experience of the software of each vendor. The technology functions which the vendors were asked about are as shown as follows. These are drawn from the Information Difference vendor functionality model; if you are interested in more detail on this then please contact The Information Difference.


Functional Areas