Big data has become a popular buzzword used extensively in different organizations. So what exactly is big data, and why are so many businesses interested in it. Well, big data refers to the volumes of data that is collected on a daily basis by an organization. This data can be used by organizations to derive insights, using machine learning or predictive modelling, that can help them make sound business decisions – taking a business’s strategic decision making to the next level.
Let’s take a deep dive into what big data actually is. Big data refers to any data that cannot be processed in ways that we are familiar with, such as using a centralised database with fixed columns to store data, because the data is so complex, large and varying. This brings us to the three main and most well-known properties of big data, volume, variety and velocity.
Volume refers to the amount of data that an organization can collect from various sources, including data that is streamed from IoT (Internet of Things) devices like sensors, trackers or smart devices, data that is produced from client transactions for example, when a customer places an order or makes a purchase, or data that is produced on a daily basis from processes that take place within a business such as supply chain management, manufacturing or customer engagement.
Variety refers to the format and the type of the data when it comes in. Data can be structured or unstructured, consisting of various data types. Structured data is in a standard format that can be easily translated into a table where different relations between tables, rows and columns can be established. Whereas unstructured data does not conform to any format and cannot be stored in a traditional tabular format, as it does not possess any predefined schema.
Velocity refers to the speed at which data comes in, for streaming data in particular, data comes in near-real time whereas transactional data can come in at a slower rate. Each of these three properties of big data, can make it difficult to process traditionally using manual methods.
An important concept of big data is that it is not about how much data you can collect, but rather how you use your data. That’s where big data can help you create an immense amount of value in your company using your own data. Through analyzing big data, companies would be able to easily and effectively pick up on fraudulent behaviour, find the causes of failures and/or poor quality, manage their risk, and provide customers with personalised experiences. Ultimately big data can help organisations gain a competitive advantage, through well-informed and quick decision making. Some of the advantages of big data include better insight into your market and customers allowing for personalized recommendations and targeted marketing, improved operational efficiency and optimized supply chain management. Allowing your business to reach their full profit making potential.
Next we are going to look at typical big data pipelines, from collecting your data at its source to processing your data and creating value from your data using machine learning, predictive modelling and analytics. We are also going to look at some of the popular big data services on GCP, AWS and Azure.
References:
https://bigdataldn.com/intelligence/big-data-the-3-vs-explained/
https://www.sas.com/en_za/insights/big-data/what-is-big-data.html
https://www.talend.com/resources/structured-vs-unstructured-data/
https://www.mongodb.com/unstructured-data
https://www.upgrad.com/blog/benefits-and-advantages-of-big-data-analytics-in-business/
https://blog.adobe.com/en/publish/2013/11/19/6-ways-big-data-creates-value.html#gs.5ypatrthis
