BARC Score Data Intelligence Platforms 2024 – The Evaluation Phase

Our current project – BARC Score Data Intelligence Platforms – is intended to show potential end users of data catalogs, data marketplaces and data intelligence platforms the possibilities of these tools, shed light on the current market and explain the positioning of the software providers. To this end, we evaluate the scope and quality of each tool’s functionality and the market execution of the providers.

Our evaluation offers users a starting point when selecting software and provides additional information on the strengths and weaknesses of the products on the market. Our assessments are based on a detailed evaluation using information from questionnaires completed by the providers, briefings and our own research and project experience.

This blog post is part 2 in our “Behind the Scenes with an Analyst” series. In part 1, we explained how the market is fundamentally structured, how to progress from an overall market overview to a shortlist and what role the selection criteria play.

Understanding the data intelligence market with the BARC Score methodology

The market for data intelligence is diverse, including both standalone solutions and products with data intelligence as a function. This diversity leads to a huge variety in terms of the range of options available. Nevertheless, to compare the leading solutions, we have taken a close look at how we can best evaluate them.

Our approach is based on the proven BARC Score methodology, which looks at two axes: “Portfolio Capabilities” and “Market Execution”. These two aspects help us to determine whether a software vendor is dominating the market, is a leader, is challenging the leading vendors, focuses on a niche area or has just entered the market. We have been evaluating solutions in many other BARC Scores for years, for example, in the areas of business intelligence and financial planning. This year marks the second edition of BARC Score Data Intelligence Platforms.

We not only look at how the software vendors position themselves strategically and how important data intelligence is to their strategy, but also how well their tools address the most important use cases and support users. It should be emphasized that a software vendor is always a strategic partner for the user company and that the choice of vendor should be based not only on the number of features, but also on the vendor’s ability to support the company’s individual needs and goals. After all, data intelligence goes far beyond the subscription or purchase of a tool.

Market Execution: Focus on vendor performance

The “Market Execution” dimension evaluates the vendor’s strategy in general and specifically for the product. This is crucial to understand how much is being invested in the product, in which direction it is developing and how this could change in the future. It therefore serves as an indicator of investment security for buyers. The following weighted criteria are considered.

Criteria	Weighting
Product strategy	High
Vendor strategy	High
Customer enablement & support	High
Vendor stability	Medium
Partner strategy	Medium
Finance	Medium
Geographical strategy	Medium
Sales strategy	Low
Marketing strategy	Low

It is of great importance that the vendor maintains a solid and broad partner network to successfully carry out implementations and changes and to ensure connectivity to third-party tools if necessary. Partners should have experience in specific industries and ideally be able to accommodate requirements and support a company’s international structure if necessary.

The vendor’s sales, marketing and geographic strategy is of interest, as a strong presence close to the company offers advantages. A well-thought-out sales strategy forms the basis for a smooth proof of concept and timely implementation of the product. Broad marketing strategies are also relevant for end users, as they raise awareness of an entire category of tools and often generate valuable content. This allows users to better understand how data intelligence platforms work, how they can be used optimally and how other companies can benefit from them.

In the area of “Customer enablement & support” in particular, it becomes clear that a software manufacturer is more than just a supplier of a product. A best-in-class approach means that users are optimally supported during onboarding and beyond through various digital offerings, training and consulting services in order to get the most out of the product.

The stability and financial situation of the vendor are also important criteria to ensure a sustainable investment (i.e., by avoiding vendors with an unclear future).

Portfolio Capabilities: The versatility of modern data intelligence platforms

The “Portfolio Capabilities” axis reflects the functional scope of a product and is crucial in the evaluation of software, especially in terms of specific use cases. We consider the overall scope of the functions for data intelligence in our evaluation.

Our criteria are based on many years of project experience, exchanges with customers and providers as well as findings from the currently available resources on data intelligence, data cataloging, data marketplaces, data products and data mesh. The weighting of the criteria is based on our market assessment and in response to trending topics. For example, data marketplaces and self-learning AI are rated higher than purely administrative functions. More on this below.

The criteria were developed to best reflect the use cases of data intelligence in 2024. The following graphic illustrates the evaluation categories and shows the diversity of functionality available:

Functions that clearly increase the value of the platform (especially compared to metadata solutions from previous generations) are given a particularly high weighting in the evaluation, including the metadata repository & model, self-learning capabilities and connectivity to various sources. A medium weighting is given to user interactions, marked in green in the chart. Foundational functions have a lower weighting in the evaluation (e.g., metadata refinement and administration).

At the heart of the evaluation of modern data intelligence platforms is user interaction, represented by a variety of functions, all of which are designed to optimize the handling, management and use of data. The “Search & discovery” area, which enables users to effectively search, discover and understand data, is essential. In this category alone, the breadth of criteria ranges from simple to complex search queries, from hit list reduction to NLP support, and from browsing functions to reporting. Data lineage analysis functions are also evaluated here.

Governance features are indispensable for sustainable operations and include the monitoring of data quality, the management and monitoring of data catalogs (stewardship). This area also covers functions to support data governance processes such as rules & policy management, governance workflow management, security and privacy (PII identification and tagging) including support for associated regulatory requirements in the areas of data protection, ESG and more. Governance processes and associated features promote an inclusive and transparent data culture and ensure that data is reliable and trustworthy.

Collaboration plays a central role, as users can share their knowledge with other employees and expand their skills. This not only strengthens individual work with data, but also the collective intelligence within the organization. The scope of functions ranges from features for sharing feedback and triggering actions to notifications and ratings. Gamification functions to motivate users to actively participate are also nice to have.

Data products are becoming a game changer in the implementation of department-driven data strategies. Data intelligence platforms can support users of data products throughout the entire lifecycle (i.e., during development, provisioning and usage) and provide transparency about their value and quality.

Key functions include the support of data products in the catalog as well as information on data products (context, description, ratings, etc.), but also the provision of a central marketplace to bring producers and consumers of data products together in a regulated environment. Such a data marketplace supports data access workflows and data contracts. In addition, we evaluated whether solutions also offer direct access to data (e.g., in the sense of data virtualization).

Business enablement functions that focus on data valuation and user adoption are crucial for making catalog content accessible to business users in a curated manner, making the value of the data transparent and establishing a broad user base. These supplementary elements help to quantify the value of data and get users excited not only about the tool, but also about the associated work and its added value.

Workflow functions (as already mentioned with regard to data governance) support the enrichment of metadata and the curation of data products. These processes transform data into a valuable asset that can be used for analytics and decision-making because it is discoverable and trustworthy.

At BARC, we consider a tool as a data intelligence platform if all these functions are part of the solution. Here, we refer you again to the previous blog post, in which we differentiated between data inventories, data catalogs and data intelligence platforms.

The following table shows the criteria and weightings for the “Portfolio Capabilities” dimension:

Criteria	Weighting
Metadata repository & model	High
Self-learning capabilities	High
Connectivity	High
Data Governance	Medium
Search & discovery	Medium
Data collaboration	Medium
Data shopping	Medium
User enablement	Medium
Metadata refinement	Low
Administration	Low
Architecture	Low

Gen AI on its way to becoming a feature

Recent development in the area of automation and AI is particularly exciting. Providers have realized that there is no time to manually take care of metadata. Tools must be ready for automated integration, preparation and analysis of metadata right from the get-go. AI plays a crucial role in this by simplifying and even automating processes and taking the user experience to a new level. There are a number of approaches to this, ranging from supporting ML algorithms to early GenAI-based functions. These make it possible, for example, to automate the description of data assets or even translate complex SQL code into an understandable language. But the generative AI journey has only just begun, and many of the vendors’ roadmaps are full of interesting plans.

AI can make a difference. In the past, many metadata projects failed because nobody had the motivation to take care of the time-consuming maintenance work (e.g., inserting, describing and updating assets; creating lineages; classifying data). Today, many tools now offer solid support in this area. The integration of AI functions makes it possible to perform these tasks more efficiently and thus contributes to the successful implementation of metadata projects. This is a lever for achieving greater acceptance of data catalogs and maximizing the quality of insights.

Publication of BARC Score Data Intelligence Platforms

It’s almost time: BARC Score Data Intelligence Platforms will be published on March 12, 2024. It promises a comprehensive overview of the various providers on the market.

Want to make sure you don’t miss the publication? Then follow us on LinkedIn and subscribe to our newsletter.

5 reasons why you should read this BARC Score:

• The products were evaluated on the basis of detailed questionnaires about portfolio capabilities and aspects of market execution; briefings and product demos; experience from customer projects and web research. Interested in going into more detail? Get in touch with us [LINK].
• Our clear and understandable methodology has been extensively documented to ensure transparency.
• The profiles of the individual vendors provide information about their strengths and weaknesses.
• We look behind the marketing hype and check which providers have powerful AI under the hood.
• Based on our briefings with the providers, we can publish up-to-date information on their roadmaps.

Are you interested in getting started with data intelligence or launching a data catalog project? We can help you find the right provider in today’s diverse market. Our documented, structured methodology allows us to efficiently use the BARC Score results and generate customized shortlists to help you in the selection process. And that’s not all. We can share best practices, such as how to engage users and motivate them to actively engage. We are convinced that this is one of the biggest hurdles along the way to successful data cataloging.

You will benefit not only from the information we provide, but also from our extensive project experience and detailed knowledge of the tools, which we have gathered from direct discussions with customers and providers. We can tweak the weightings of the evaluation criteria to create a shortlist tailored to your needs.

What do you think of the evaluation criteria? Which important data intelligence use cases do you think we have not included? How important are automation and collaboration functions for you? We look forward to hearing your feedback.