Data has been exploding for a while now. Mankind has generated data just in the last two years that is equal to all the data we had generated as a species in our entire history! Why are we generating so much data ?
The internet was the starting point. Everyone became an author on blogs
Mobile devices - we take so many photos now, videos are everywhere.
See this quote from Twitter Aditya Panda, a little controversial but he makes an interesting point on the amount of recording that is happening.
How will we, Individuals, Companies and Governments make sense of this vast amount of information? Traditional techniques have failed. Spreadsheets like Excel was a data cruncher, now we need so much more. SAS, R came along for a while and helped companies manage their data. But even that was not enough.
As data gained in three dimensions Big data was coined, born or came along. Those three things are:
- Volume
- Variety
- Velocity
As a result of all of this we need Big data analytics to even begin to understand this kind of data and manage it.
The volume, velocity and variety of data has grown many fold over the years and will continue to do so for the foreseeable future. Therefore, organisations are faced with the question of how they make sense out of it all.
Big Data was pioneered by the likes of Google and Yahoo! that had to develop ways to tackle the above problem in a cost effective and high performance manner, given the fact that they were being deluged by data of all sorts from across the globe - structured, unstructured and semi-structured.
The Big Data players are evolving and maturing rapidly with Hadoop distributions such as MapR, Cloudera and HortonWorks coupled with vendors/products that deal with pre-processing and loading data into Hadoop, as well as Analytics providers that are able to analyse massive amounts of data.
All this leads to a question I get asked quite a bit - Is this Big Data stuff real or just marketing hype? and Are companies actually using this effectively and benefiting? What are the adoption rates across industry verticals?
The answer to these questions is that it is and was a tough space for companies that are used to traditional relational structures and associated applications. However, early adopters that are nimble and not afraid to use new technology have jumped on the Big Data bandwagon, adapted to it and are benefiting tremendously.
A word of caution - this space is maturing and continues to evolve, hence it there will be changes and companies should expect this. The product ecosystem is large and fragmented, hence some consolidation will occur over time.
Big data analytics is the very important in big data. With the help of big data analytics people only. we need to find or explore new algorithm for big data maintenance. So big data analytics is very important
Comments