The existence of data silos is nothing new. Data-producing applications were once isolated systems. They were built to at least partially automate a specific subtask of a business process. The transactional data was stored in isolated data sets and initially served only one purpose, namely, to document the transaction that had taken place.
Over time, enterprises realized that data is worth more. Utilization of operational data for enterprise management helps to gain insights into the current state of the business and supports fact-based decision-making. This has been important for decades.
However, the operational data stored in data silos was not suitable for this task. Many companies therefore built a data warehouse to consolidate their operational data silos.
In the age of digitalization, more extensive data and analytics requirements have emerged for which the data warehouse was not sufficiently designed. Data-based insights are being used to automate decisions. The goal is to make business processes faster, more efficient and less vulnerable to risk.
Analytics-driven insights are also expected to drive business innovation. Thus, alternative data architecture concepts have emerged, such as the data lake and the data lakehouse. Which data architecture is right for the data-driven enterprise remains a subject of ongoing debate.
This study evaluates current thinking around data silos and addresses several questions:
Participants around the world were polled, with most responses coming from Europe. We examined the current approaches of companies of different sizes from various industries. For deeper insight, we also analyzed the answers according to data management maturity.
Participants were asked to rate the skills and competencies in the handling of data in their company compared to their main competitors. This allowed us to gain a better understanding of what “best-in-class” companies are doing to overcome their data silos in comparison to “laggards”.
Data warehouse, data lake, data lakehouse: the main paradigm for provisioning data for analytics is a centralized approach. However, business users often resort to extracting data and copying it for analysis to prepare it individually because centralized data sets do not meet their needs, they trust their own data sets more, or it enables them to get results faster.
Consequently, despite all attempts at centralized data provisioning, most companies are unable to break down the data silos they have. On the contrary, their proliferation continues to increase. As the number of silos continuously grows, finding and understanding the right data and preparing it for a specific purpose becomes even more complex and time-consuming.
Decentralized data preparation is both an opportunity and a risk. Create transparency about decentralized data preparation processes and where they negatively impact business efficiency and effectiveness. Identify how you can optimize central data provisioning processes to increase the usability of the data provided.
Having key personas tied up with elementary data problems instead of working on the digital future of the company is the biggest challenge caused by the state of current data landscapes. This is a vicious circle. Their capacity is tied up in maintaining existing data problems instead of solving them.
In turn, they do not have the time to work on data-driven evolutions, both for themselves and to support others. This limits business agility and velocity as well as the innovation of the data-driven enterprise.
Establish business responsibility for data and its strategic development. Make sharing knowledge about data and its use in the company culture mandatory and create corresponding processes. In the digital enterprise, distributed data sets are not only a matter of course, but a necessity. Therefore, logical, cross-functional accountability for data is indispensable.
Predominantly centralized data architecture is not the silver bullet of the digital future. Physical data silos will neither be avoided nor completely dissolved. Therefore, an exclusively centralized approach to data integration is doomed to fail. Instead, distributed data must be linkable as needed.
Architecture and technology play a significant role here. They help overcome the limitations of physically distributed data sets more efficiently by supporting intuitive and on-demand exploration, preparation and advanced analysis of data.
Plan a logical architecture that ensures combinability of physically distributed data. Populate this architecture with smart technologies such as data cataloging, data virtualization and AI-augmented data preparation that help overcome the boundaries of physical data sets.
Unified business terminology, not centralized data management systems, is the main success factor for the digital enterprise. Sharing data, analyses and data-based insights across business units only delivers value if they are understandable to others.
However, this requires a uniform (data) language. But because business units have been managed as silos for many years, these language barriers are primarily organizational. Data silos are often just a consequence of this. Accordingly, a lack of communication is the most encountered cultural challenge when it comes to improving the data landscape.
When it comes to ending the data silo dilemma, change requires bold actions. Shaping data strategy and goals and transforming the behavior of people in the company will not succeed without executive vision and leadership.
Secure the support of your company management for your data strategy and convince them that they must actively lead the digital transformation. Employees need guidance based on clear strategic goals and what becoming a data-driven business means in practice.
Interested in the full study results? You can download them here.
Annika conducts both product-based and topic-based BARC studies in the areas of BI & analytics, data management and corporate performance management.
As project manager, she is responsible for time and resource planning and oversees projects from questionnaire creation to publication.
Depending on the study, she also handles the implementation of project steps such as questionnaire design, field phase control, data preparation and analysis.
