Analytical Databases: What Are They and Why Are They Important?

A key question at the moment for many project managers is whether existing BI systems can be operated more flexibly, efficiently and cost-effectively using analytical databases.

This interest in analytical databases is also being fueled by well-established database providers, and new players alike, who are promoting these technologies heavily as they identify new sales opportunities in the seemingly saturated market for database technologies.

However, analytical databases are not a fundamentally new technology. They have been available for many years from providers such as Teradata, which, especially in the American market, has led to their wide distribution.

Nevertheless, organizations often hesitate to implement them because it is difficult to accurately assess the actual performance of analytical databases. It is also often a challenge positioning them correctly within an existing BI/data warehouse landscape.

This article is intended to provide guidance and support for BI professionals when assessing these technologies.

What Are Analytical Databases?

Analytical databases are specialized databases optimized for analytics, for example, through data storage (column-based), hardware usage (in-memory), integrated functions (mining), architecture concepts or delivery terms (appliances). Their main benefits are faster query performance, better maintenance, and scalability.

Analytical databases are available as software or as data warehouse appliances. The latter are preconfigured packages of hardware and software that promise a “balanced system” for easy operation.

Why Are Analytical Databases Becoming So Important?

The increased popularity of analytical databases in recent years is mainly due to the limitations of current OLTP relational and multidimensional databases as the technology foundation for implementing a data warehouse.

For a long time, the data warehouse primarily supported tactical decision support. Numbers of users and data volumes were kept within controllable limits, and the data was usually kept up-to-date during operation. Changes were rare.

In recent years, however, the demands of companies have steadily increased on both a tactical and operational level.

The desire for faster query performance for various kind of workloads, for more complex and advanced analysis methods as well as more frequent data updates (latency reduction) has put pressure on existing data warehouse architectures. They cannot support business departments with the flexible and efficient data management they now require.

Instead of static report queries, business departments now expect more agility in terms of data provision, data processing and data preparation from IT, and increased flexibility when it comes to analytics (i.e. self-service BI). They need to react as quickly as possible to changes and new requirements.

Since this has not been the case in most organizations to date, business departments are increasingly taking steps to develop and operate their own analytical systems locally, or at the very least they are calling for more commitment from their IT departments.

However, a rethinking of the existing data warehouse architecture has also become necessary to set out the prerequisites for smart and flexible data management driven mainly by new and emerging technical requirements.

Above all, the use of “big data” involves numerous technical, conceptual and organizational changes. A central requirement in this context is to be able to carry out explorative analyses from structured and polystructured data sources – beyond the quality-assured data space that IT provides.

And another factor has increased the pressure on IT and BI managers and brought analytical databases into the debate: the high system costs of existing data warehouse environments.

To address the problems and requirements of historically grown data warehouse environments described above, analytical databases appear to constitute a relevant building block in the solution: they provide functionality for both classic BI and big data demands, promise good maintainability and can offer high performance at a lower cost (maybe in conjunction with additional software components like Hadoop or Spark).

They are becoming more affordable due to falling hardware prices and, with their high processing speed, these technologies can make the complex data architectures of existing data warehouse databases leaner, thereby adding flexibility to data management overall.

How Important Are Analytical Databases?

Data from the BARC BI Trend Monitor 2017 confirms the importance of analytical databases.

Most of the respondents to our survey see analytical databases as an important trend.

Importance of Analytical Databases in 2017 (n=2,631)

However, there are a few noteworthy variations across our global sample:

• Consultants and users see the trend as more important than vendors.
• The DACH region (Germany, Austria and Switzerland) and France, as well as respondents from the telecommunications and manufacturing sectors, consider it less important.
• Analytical databases are more important to best-in-class companies

Challenges and What to Look Out for When Selecting an Analytical Database

Analytical databases alone do not guarantee successful data management or a lean, high-performance data architecture.

For instance, using an analytical database as a data mart to accelerate queries creates an additional level in the overall system. Analytical databases have enormous potential but need to be planned correctly in the overall analytics landscape.

They are not a substitute for a data warehouse that is based on a relational database.

Also, a complex data architecture with long data integration cycles is not improved by an analytical database, especially since data loading is one the weaknesses of many of these databases. In the worst case, an analytical database can even cause longer run times and possibly even greater complexity.

The implementation of an analytical database should not be undertaken without first conducting a careful functional evaluation as well as an architecture review. The systems available on the market today vary considerably – and analytical databases are not always the right solution.

Whether an analytical database is appropriate can only be verified with a concrete usage scenario. As part of the data warehouse platform strategy, important issues need to be clarified and planned such as:

• Data and user profiles;
• Data architecture;
• Data quality;
• Data modeling;
• Integration strategy;
• Tool portfolio;
• Organization;

It is only when BI managers systematically evaluate these new technologies that they can really achieve the benefits they hope for by combining the data warehouse environment with the analytical database.

The precise value of an analytical database can neither be proven by comparing it with existing data warehouse solutions nor by looking solely at its own system properties.

The usage scenario is critical, with the following criteria playing a role in determining value:

• Manageability of data volume, variety and velocity;
• Query capabilities for standard reporting queries;
• Mixed workloads;
• Scalability and query/load performance;
• Ad hoc queries and analysis queries;
• Data management capabilities;
• Technical support for self-service BI;
• Maximum number of users;
• Administration