Big Data: What are the challenges for organizations?

April 14, 2020

2 min read

The now-mainstream term big data describes the large volume of data, both structured and unstructured, resulting from the data explosion that the world has witnessed due to the dramatic increase in devices. The term refers to data that is so large, fast or even complex that requires a whole new generation of technologies and architectures to deal with it. 

The process of using large amounts of data for analytics has been around for a long time, but the concept gained momentum in the early 2000s when analyst Doug Laney articulated the definition of big data as the three V’s: Volume, Velocity, and Variety. In other words, organizations collect data from a variety of sources, data streams face an unprecedented speed and data comes in all types of formats.

More important than the amount of information that is available is knowing how organizations can extract value from it. Big data proves to be valuable only if it provides useful and reliable information that enables companies to make real-time business decisions. Dealing with data growth, as storing and analyzing, is still one of the major concerns. According to International Data Corporation (IDC, a global provider of market intelligence), the amount of information stored worldwide is doubling about every two years, which means that significant investments in technology and people are required.

Nevertheless, emerging big data projects have become a part of doing businesses and organizations from all industries have started to contemplate ways of how it can be used to improve productivity, to manage complex suppliers networks, to provide client insights and tools to conduct successful negotiations, to foresee and mitigate potential risks and ultimately to become more profitable.   

The customer is the most important asset any business depends on and big data allows organizations to profile customers in a far-reaching manner. The fact that a large volume of data can be analyzed in an integrated way makes easier to identify customers' priorities and preferences, which is crucial to attaining high customer retention. Understanding customer insights enables companies to match customers’ expectations, re-develop product lines and ensures that the marketing campaigns are powerful. By targeting potential clients with the right products, businesses can save money and ensure efficiency. 

Big data disrupting technologies and analytics are becoming available to more organizations than ever before empowering companies to position themselves actively in the markets, constantly looking for new growth opportunities and competitive advantages. However, risks associated with the retention and use of data should be taken into consideration, especially when personal data is involved. 

Organizations must comply with data protection laws and requirements of the General Data Protection Regulation. Breaches of data protection laws can be mitigated in several ways (for instance by imposing a limit for data retention or anonymizing personal data), but businesses should make sure that the appropriate practices are in place under penalty of heavy fines or even having reputational damages. Adequate security measures should also be in place and safeguards may need to be higher in the presence of sensitive data.

Big data is an important piece of the future for organizations. The ways how organizations employ big data to support their operations will continuously be molded by advancements in how we collect, store and analyze data. The real question is not whether or not companies should implement big data but if they are able to retain technology and talented people that enable them to extract valuable information for their business and especially to know their customers.

Andreia Pereira

Financial Analyst