Big Data
CS4103 Distributed Systems Coursework Part 1: Big Data
Student ID: 080010830 March 16, 2012
Word Count: 3887
Abstract Big data is one of the most vibrant topics among multiple industries, thus in this paper we have covered examples as well as current research that is being conducted in the field. This was done based on real applications that have to deal with big data on a daily basis together with a clear focus on their achievements and challenges. The results are very convincing that big data is a critical subject that will continue to receive further study.
1
Introduction
Big data – in information technology – refers to the extremely large volume of data that needs to be captured [1], stored [2], searched [3, 1], shared [4, 1], analysed [2] and visualised [5, 1]. The exponential growth of these datasets can result in exabytes1 or even zettabytes2 of information. For example, telecommunications networks have seen their capacity to exchange information grow from 281 petabytes in 1986, 471 petabytes in 1993, 2.2 exabytes in 2000, 65 exabytes in 2007 and predictions say that it will reach 667 exabytes annually by 2013 [6]. Furthermore, to put these numbers into perspective, 5 exabytes of information is equal to “all words ever spoken by human beings” [7, 8, 9] and if we add all the combined capacity of all the computer hard drives that were available in the world in 2006 the total amount of free space would be approximately 160 exabytes [10]. However, this storage capacity is increasing at an astonishing rate and a proof of that is Seagate’s report that during the 2011 fiscal year alone, they have sold hard drives of a combined capacity of 330 exabytes [11]. These impressive statistics and the fact that more people than ever before interact directly with data [6] makes the analysis of big data very relevant, if not crucial.
1 2
1 EB = 1018 bytes = 1 000 000 000 gigabytes = 1 000 000 terabytes 1 ZB = 1021 bytes = 1 000 000 000 000 gigabytes = 1 000 000
Student ID: 080010830 March 16, 2012
Word Count: 3887
Abstract Big data is one of the most vibrant topics among multiple industries, thus in this paper we have covered examples as well as current research that is being conducted in the field. This was done based on real applications that have to deal with big data on a daily basis together with a clear focus on their achievements and challenges. The results are very convincing that big data is a critical subject that will continue to receive further study.
1
Introduction
Big data – in information technology – refers to the extremely large volume of data that needs to be captured [1], stored [2], searched [3, 1], shared [4, 1], analysed [2] and visualised [5, 1]. The exponential growth of these datasets can result in exabytes1 or even zettabytes2 of information. For example, telecommunications networks have seen their capacity to exchange information grow from 281 petabytes in 1986, 471 petabytes in 1993, 2.2 exabytes in 2000, 65 exabytes in 2007 and predictions say that it will reach 667 exabytes annually by 2013 [6]. Furthermore, to put these numbers into perspective, 5 exabytes of information is equal to “all words ever spoken by human beings” [7, 8, 9] and if we add all the combined capacity of all the computer hard drives that were available in the world in 2006 the total amount of free space would be approximately 160 exabytes [10]. However, this storage capacity is increasing at an astonishing rate and a proof of that is Seagate’s report that during the 2011 fiscal year alone, they have sold hard drives of a combined capacity of 330 exabytes [11]. These impressive statistics and the fact that more people than ever before interact directly with data [6] makes the analysis of big data very relevant, if not crucial.
1 2
1 EB = 1018 bytes = 1 000 000 000 gigabytes = 1 000 000 terabytes 1 ZB = 1021 bytes = 1 000 000 000 000 gigabytes = 1 000 000