Data Integration is Crucial for Modernizing the Data Warehouse

The Business Application Research Center (BARC) publishes “The Data Management Survey 19”, a new study based on a recent worldwide survey of 782 data management software users. The survey results provide insights into current approaches to data warehouse modernization as well as a comparison of 11 leading data management products.

It is generally acknowledged that 70-80 percent of the effort involved in data warehousing is spent on data integration processes. It is therefore natural that users should try to modernize their data warehouse environments by means of ETL automation and data warehouse automation. According to The Data Management Survey 19, data integration technologies are still the main technology used for this purpose (by 42 percent of respondents), closely followed by data warehouse automation solutions (37 percent) and data modeling tools (31 percent).

Specialist data warehouse automation software rated better than data integration tools

The Data Management Survey 19 reveals that data warehouse automation is one of the top three application scenarios for data integration tools, alongside classic tasks such as loading data marts and enterprise data warehouse systems. A comparison between data warehousing automation specialists and data integration tools found that user ratings of specialist software were better overall for data warehouse automation, especially in terms of developer efficiency and time-to-market. However, data integration tools did not come off too badly in the comparison.

Leaders set a trend for using analytical databases

There is a variety of approaches to modernizing the data warehouse and addressing new analytical requirements. BARC‘s study shows that companies with a flexible data and process architecture or agile interoperable data services (termed as “leaders“ for the purposes of this research) favor a central approach – such as building an enterprise data warehouse – rather than using additional data marts. According to the survey findings, one in three leaders has already migrated to a central approach.

The implementation of an enterprise data warehouse requires powerful databases that are able to scale with large data volumes and a multitude of different workloads while remaining both manageable and highly performant. As a result, leaders use analytical databases much more frequently (41 percent) than average (27 percent). Other companies are more likely to rely on data marts for modernization.