Architecture and technology help balance centralized and decentralized data requirements

Centralized data architecture is not the silver bullet of the digital futureData warehouse, data lake, data lakehouse: which data architecture is right for the data-driven enterprise is the subject of ongoing debate. In principle, the approaches follow a similar rationale: they physically move data to one place to collect and compile it there. However, optimal data preparation is ultimately determined by the analytical problem to be solved. It is not possible to prepare data in advance for every conceivable application. This is demonstrated by the answers of our survey participants, and is independent of the type of data landscape. In all data landscapes, the majority of users tend to create their own data sets for analysis. Data is also often extracted and copied for analysis to prepare it individually. This tends to happen more often in companies with decentralized data landscapes, which shows that centralized data provisioning is helpful. Nevertheless, it will not be possible to avoid preparing data individually for new analytical questions. It is therefore not surprising that 63 percent agree that the data warehouse does not necessarily help to replace data silos.To what extent do you agree with the following statements in relation to the current state of your company’s data landscape? By maturity (n=311)Modern technology transforms centralized data silos into virtually connected data landscapeIn the future, it will become even more difficult to manage data in a centralized manner. The data landscape of the digital enterprise is highly distributed. In other words, we must live with data silos. Solution concepts must integrate data silos instead of fighting them.Which of these approaches is your company taking to deal with the challenges caused by data silos (top 5)? (n=318)This is reflected very well in the approaches that companies are taking to deal with the challenges caused by data silos. Not the data itself, but knowledge about the data should be provided centrally.Solution concepts must integrate data silos instead of fighting them.The most frequently chosen approach for this is creating transparency about existing knowledge of data and fostering collaboration (49 percent)This goes hand in hand with establishing a platform for collaboration and knowledge exchange, which 34 percent have chosen to do. Another 30 percent state that they are creating a central directory for finding and understanding data sources. These approaches are mostly implemented with the help of data catalog technologies.31 percent of companies are working to liberalize access to data and improve self-service support.This goes hand in hand with other measures to facilitate ad hoc access to data. For example, 36 percent plan to create a central platform for data shopping. Among best-in-class companies, this approach is even more common (45 percent). One building block for this is to make physically distributed data accessible without having to copy it. This plays a greater role especially for companies with distributed data provisioning (27 percent compared to 23 percent on average).Technology helps establish a business bridge across data silos to make them more usableMost companies are still very much focused solely on providing a central data directory. For best-in-class companies, good support for non-technical data users is a high priority. They have recognized that modern technologies play an important role in overcoming data silos efficiently and effectively.Which of these approaches is your company taking to deal with the challenges caused by data silos? By best-in-class (n=89)Best-in-class companies have recognized that modern technologies play an important role in overcoming data silos efficiently and effectively.Enabling interactive exploration of data (e.g., through data discovery) is one of the most popular approaches. 35 percent choose this approach, compared to 24 percent on average and 20 percent for laggards. Better support for data preparation is also an important goal for best-in-class companies. 33 percent (compared to 24 percent on average and 24 percent for laggards) would like to provide a central platform for ad hoc and automatable data acquisition and preparation from various sources (e.g., based on data fabric). 25 percent (compared to 17 percent on average and 8 percent for laggards) are planning to support smart, interactive data preparation by business analysts (e.g., with AI-augmented data preparation).