Big Data is a hot topic these days. It’s been said that the amount of digital data doubles every three years. Today, less than 2% of all stored information is nondigital.
“The whole point of a big data strategy is to develop a system which moves data along from raw statistic to actionable insight,” says Bernard Marr. In his Big Data Guru blog, he explains the four basic layers you need to get a grip on Big Data. Here’s how he describes it.
1. Data sources layer
This is your raw data. It includes sales records, customer database, feedback, social media channels, marketing list, email archives and any data gleaned from monitoring or measuring aspects of your operations. This is where you decide what you have and measure it against what you need to answer.
2. Data storage layer
Your Big Data lives here once you gather it. A computer with a big hard disk might be all you need for smaller data sets, but when you start storing (and analyzing) truly Big Data, you need a sophisticated, distributed system. Besides a system for storing data that your computer system will understand (the file system), you also need a system for organizing and categorizing it in a way that people will understand—the database. Some possibilities: Apache Hadoop DFS (distributed file system) and Google File System.
3. Data processing/ analysis layer
When you want to use the data to find out something useful, you will need to process and analyze it. A common method is a MapReduce tool that selects the data elements that you want to analyze, and puts it into a format ready for you to glean insights. A large organization may have its own data analytics team. They will employ tools such as Apache PIG or HIVE to query the data and might use automated pattern recognition tools to determine trends, as well as drawing their conclusions from manual analysis.
4. Data output layer
This is where the analysis gets to the people who can take action on it. It can take the form of reports, charts, figures and key recommendations.
Results of Big Data
1. Cost reduction
2. Time reductions
3. New product development
4. Better decision-making