Data Access

The ‘Right to Know’ Principle is More Beneficial Than ‘Need to Know’

Two basic principles can be observed in data access. We refer to the more restrictive approach as ‘need to know’. Here, users must ask for
authorization to access data. Data access is thus the exception to the rule of locking everything up. In contrast, data democracy propagates free data access for all employees, limited only by intentionally restricted data (e.g., secret, personal or similar data). We refer to this data access model as ‘right to know’.

‘Need to know’ has always been the predominant principle for data access, with 63 percent of participants confirming that this approach prevails in their organization. However, significantly more than half of our sample consider ‘right to know’ to be the most beneficial model.

We also support this point of view. The flexibility required today coupled with a stronger data orientation in general are leading to the call for a shift towards ‘right to know’. However, this shift goes hand in hand with a change to the organizational mindset and is therefore no small matter. Currently, only 37 percent of respondents live by the ‘right to know’ principle. For many respondents, however, there is still a significant gap between their wishes and reality.

‘Right to know‘ is practiced mainly by small companies. This is not surprising due to their mostly simple and flat organizational structures and straightforward communication channels. As the size of a company increases, so does its organizational complexity and the demands on data governance. The ‘need to know’ principle tends to prevail here. Regionally, we see a higher level of adoption of ‘right to know’ in North America (46 percent of respondents) and APAC (43 percent) compared to Europe (34 percent).

It should be emphasized that the majority of bestin-class companies (53 percent) rely on ‘right to know’ and see the benefits of greater freedom in the use of data. By contrast, only 24 percent of laggards concur.

This is also confirmed in our analysis of the benefits achieved. Companies that predominantly practise the ‘right to know’ principle believe that they generate greater benefits from data than companies adopting ‘need to know’. This applies to almost all the benefit categories surveyed. They report a much higher rate of achievement when it comes to gaining competitive advantage, preserving market position and growing revenue.

Companies adopting the ‘need to know’ principle only outperform their counterparts in three areas: ‘reduction in data silos’, ‘reduced risk and compliance concerns’ and ‘cost reduction through the use of data’.

Lack of Data Knowledge is the Biggest Challenge in Data Access

Data access is a cornerstone of data-driven companies. It is essential to be able to use data from complex and distributed data landscapes and generate value from it. The biggest challenges to achieving this lie in empowering business users. Often, they don’t even know what data exists (59 percent). Alternatively, they may not understand the data and its context (55 percent), or technical access to the data and options for data processing are too complicated (48 percent).

Transparency of data is essential here, and data knowledge is crucial too. Information about data is available in the form of metadata. Collecting, integrating, linking, enriching and evaluating this data is complex and time-consuming.

Automation must provide support here to create a comprehensible, trustworthy database. Automation is seen by 45 percent of respondents
as a major challenge, especially for the integration and preparation of metadata. This is partly due to the fact that there is no established standard for describing and exchanging metadata, so a reassessment of how metadata can be integrated and understood must be made for each software tool and provider in use.

AI/ML algorithms can also play a supporting role in integration and description. Self-learning is a laborious and tedious process to get up and running (training time, explainability) and is therefore rarely used. Instead, many rely on proven automation techniques such as partial automation of process control through scheduling and monitoring.

Nevertheless, data knowledge thrives on knowledge sharing through communication and collaboration. With reference to the importance
of data culture initiatives, it should be noted that those related to data communication, data literacy and data leadership take a back seat compared to some other topics. Nevertheless, they are essential in order to implement data access efficiently and correctly.

Data Access is Mainly Implemented in BI/DWH Tools and Excel

Data access requires technology support. About two thirds of the companies surveyed use traditional data warehousing and BI technologies

69 percent use Excel and 51 percent use selfservice analytics tools. If the aim is to solve challenges with existing tools for the time being, these figures are not surprising, but there are better options available.

It is worth mentioning that 32 percent use code to manage data access, which corresponds to our general market perception that languages such as Python are gaining a stronger foothold in the enterprise data landscape. This is happening primarily via exploratory use cases and concerns data engineering, data pipelining and the design of models.

In turn, the need for transparency to be able to find data, features and algorithms in an uncomplicated manner and to integrate them securely is also increasing, thus providing the breeding ground for software providers to offer new solutions that help to manage and monitor code in order to have a controlled and monitored process.

Best-In-Class Companies Rely on Modern Technologies and Concepts

There is a great deal of catching up to do in terms of technologies that help to increase transparency around data.

Fewer than 25 percent of the companies surveyed use data intelligence platforms or data catalogs. However, it is precisely these types of tools that help to compile knowledge about data outside of the BI context, across systems, and make it analyzable, thus addressing the main challenges to data access head on.

The importance of data knowledge has been recognized above all by best-in-class companies. 58 percent use data intelligence platforms, compared with only 19 percent of laggards.

Data virtualization and data cataloging technologies, the data mesh organizational concept and the data fabric architectural principle are also used much more frequently by best-in-class companies.

There is a Lack of Competence in the Use of New Technologies

Technology is only half the solution to data access problems. Many challenges have their origin in a lack of strategy or organization. The added value of technologies for increasing data access is limited.

Only just over half succeed in improving data access through BI and data warehouse technologies, and only one in three companies manage it with self-service analytics tools. This is surprising, since one of the main promises of self-service tools is to make access to data significantly easier for business users.

Data virtualization tools, data intelligence platforms and data catalogs play a special role in the technical support of data access. These tools can clearly add value, but there is probably a lack of knowledge and training to be able to use them extensively. 39 percent of respondents complain about a lack of skills as the second most common obstacle to data access.

Best-in-class companies and organizations that have adopted the ‘right to know’ principle report a significantly lower lack of data literacy than other companies in general. At the same time, we see a significantly higher level of benefits achieved through the use of ‘new’ technologies, concepts and principles.