Monday, August 11, 2014

Big Data

Big Data

Over the last few weeks i have been trying to wrap my hands and head around this this Buzz word. Although, i do see a lot of material - blogs, articles, seminars around this new "kid" in the town i still haven't been able to understand it completely.

What is Big Data?
Primarily Big data means "BIG" Data. Over the last few years, mostly with the advent of web2.0, we have seen a prolific growth of social interactions over the web. This has led to a huge growth of data that is being shared over the internet.
Here i would like to go back to one of the questions that i was asked when i was in college:
What is the difference between "Data" and "Information"?
Data is any piece of fact that i available to us. however meaningful data is information.
Herein lies the answer to the question at hand. Companies over a period of time have accumulated a lot of data in terms of web logs, click stream data, server logs, etc. Until recently this data was simply discarded or just backed up in tape drives only to be thrown away. However, this data that the companies had was like a hidden treasure in order to conduct their business effectively. Click Stream data, could help companies understand customer behavior, which could lead for eg. e-commerce companies to potentially sell their products to them when they actually are looking for it.

So, now let us try to answer the original question:

1. Big data is any data from which you can derive meaningful information.
2. Big data is data that is normally unstructured - weblogs, server logs, etc.

In this blog I will try to start with the basics around Big data and talk about various advances happening in this area.