April 27, 2023

Infographic: Data Warehouse and Data Vault Adoption Trends – Modeling, Modernization, and Automation

Modeling, Modernization and Automation

While most continue to struggle with data quality issues and cumbersome manual processes, best-in-class companies are making improvements with commercial automation tools.

The data vault has strong adherents among best-in-class companies, even though its usage lags the alternative approaches of third-normal-form and star schema. Compared with laggards, a higher portion of best-in-class companies adopt the data vault, embrace its standards, and intend to expand their use of it. They plan to expand their use of this modeling technique and methodology.

Eckerson Group wrote this report in collaboration with BARC by studying the results of a global survey of 238 data & analytics practitioners and leaders. BARC conducted the survey in December 2022 and January 2023, drawing respondents from companies of various sizes and across various industries. Lessons about data modeling, modernization, and automation include the following:

Focus on fundamentals
Companies place the highest priority on data quality, ease of use, analytics performance, and data governance.
Automate with commercial tools
Implement commercial automation tools rather than homegrown scripts because they help improve data quality and standardize and reuse tasks.
Get smarter about the data vault
Study how best-in-class adopters selected the data vault, trained their teams on the 2.0 solution, and plan to expand its footprint.

Overall takeaways

Architecture
Analytics environments include the data warehouse (79%), data lake (42%), and independent data marts (41%). The lakehouse, data fabric, and data mesh have 8-12% usage each. Most respondents (58%) also still analyze some operational data directly. Most companies (82%), especially larger companies, have multiple architectural types.
Priorities
Companies plan to “improve data quality” (58%), “automate manual steps” (55%), and “update business logic” (44%). They also intend to “improve performance and availability,” “migrate to the cloud,” and “extend architecture” (41% each).
Automation
Respondents say they automate most or all processes for “data integration” (69%), “platform monitoring” (58%), and “data quality monitoring” (38%).

Data vault takeaways

Standards
About one third (31%) of adopters say their “overall implementation” “fully” aligns with solution standards (for architecture, methodology, and modeling), and 60% say it “partially” aligns. A lack of training contributes to this gap: only 65% of data vault adopters say they have been trained on the data vault 2.0 solution.
Business drivers
Respondents cite “accelerated data delivery” (45%), “team skills/preference” (41%), and “advice of consultant” (40%) as their primary business reasons for adopting the data vault.
Technical drivers
Respondents cite “extensibility” (53%), “scalability (data volume and velocity)” (40%), “flexible architecture” (35%), “simpler data management” (33%), “unified data model” (32%), and “data quality” (27%) as their primary technical reasons for adopting the data vault.
Drawbacks
Half of data vault adopters (48%) cite “skills and training requirements” as a primary drawback, followed by “implementation complexity” (35%) and “query performance” (32%). Other responses include “design complexity” (29%) and “multiple versions of data” (29%).

Best-in-class companies vs. laggards

Automation tools
Nearly two thirds of best-in-class companies (62%) say their automation is “fully” or “mostly” based on commercial tools rather than homegrown scripts. This compares with 24% usage for laggards.
Future plans
A higher portion of laggards plan to improve in the areas of automation, data quality, performance, and availability when compared with best-in-class companies. They view data quality as one of their “biggest challenges”.
Data vault
Best-in-class companies adopt the data vault and take data vault 2.0 solution training in much higher numbers than the laggards. More of them (91%) also plan to increase the role of the data vault in their environments when compared with laggards (60%).