The Most Common Problems Companies Are Facing With Their Big Data Analytics

Insufficient Skills Are Curbing The Big Data Boom

Enterprises can derive substantial benefits from big data analysis. Nonetheless, there are a number of challenges to overcome too.

On the whole, big data appears to be a topic that brings many benefits, but many problems as well. Only six percent of all respondents said that they see no issues connected with using big data technologies.

Challenges related to the lack of necessary skills were cited particularly often. Approximately half of respondents reported having inadequate analytical or technical know-how for big data analytics.

The current labor market has a shortage of qualified personnel in this area. This problem is not regional as both North American and European companies reported similar grievances. The rates for inadequate analytical and technical know-how are around 50 percent in both regions.

Although companies may want to create jobs in this area, they might not be able to fill them due to the lack of suitable candidates on the labor market. This shifts the focus towards training existing staff.

Data privacy and security also rank high on the list of challenges for companies. The high value placed on data privacy is not surprising considering that many use cases revolve around customers.

As analyses center even more on customers, companies will have to focus even harder on anonymizing data to protect customer privacy. That means elaborate processes will need to take place before the actual analyses begin. The content, technical implementation and legal issues related to these processes all pose major challenges for companies today.

When it comes to big data analytics, data security is also a major issue.

Big data can contain business-critical knowledge. In addition, new problems can also arise in accessing new systems.

Manufacturers, for example, regard anything accessing their machines to capture machine data with suspicion. The reason is that these machines contain programs that are necessary for operations and must be protected as critical intellectual property.

Companies also harbor insecurities about saving and transferring data in cloud-based systems (e.g. social media). These uncertainties need to be addressed as well.

38 percent of companies still complain of a lack of compelling business cases.

This number is very high, especially given the amount of coverage big data receives in the IT and business media. It may be down to a lack of creativity in devising new ways to use or monetize data, or simply a reluctance to implement new methods and technologies for fear of failure.

Technical issues, in contrast, are not the main obstacle to deploying big data technologies. Only 26 percent of respondents view them as a problem.

No big differences between North America and Europe

Aside from inadequate know-how, our survey results from North America and Europe were similar in other areas as well.

Data privacy (50 percent in North America vs. 49 percent in Europe) and data security (56 percent in North America vs. 46 percent in Europe) stand out in particular. Contrary to popular belief, it does not appear that North American companies place less value on data privacy.

The only noticeable differences lie with technical problems, which are more widespread in North America (30 percent vs. 23 percent). This, however, is not surprising because North American companies are more likely to be forerunners with regard to big data analytics and adopt new technologies at an earlier stage.

A quick glance at the problems that companies face in the different stages of their big data initiatives reveals further insights.

Those that currently have no big data initiatives planned appear to face two main dilemmas.

While 56 percent of these companies have found no compelling business cases for big data processes, 50 percent stated that their business processes are not mature enough for big data. Companies that currently have no initiatives but are considering them in the future have large deficits in the requisite skillsets and are more likely than average to encounter problems financing big data initiatives (48 percent).

Surprisingly, companies that have already implemented big data analytics into their processes still reported high rates of inadequate know-how. Their most serious challenges, however, are data privacy and security, which they appear to have underestimated before.

Challenges with big data analytics vary by industry

While there are no major differences in the above problems by region, a closer look does expose a few interesting findings by industry.

The finance sector is more likely than average to cite a lack of compelling business cases (53 percent).

Manufacturing, however, faces more problems than average with inadequate know-how, both in analytical (63 percent) and technical (61 percent) respects. This is apparently a major obstacle that these companies will need to monitor on their road to “Industry 4.0”.

In the public sector, data privacy (68 percent), costs (54 percent) and inadequate business cases (51 percent) top the list of common issues.