Big Data

Trang Vuong
Big Data and Its Potentials
Data exists everywhere nowadays. It flows to every area of the economy and plays an important role in the decision-making process. Indeed, “businesses, industries, governments, universities, scientists, consumers, and nonprofits are generating data at unprecedented levels and at an incredible pace” to ensure the accuracy and reliability of their data-driven decisions (Gordon-Murnane 30). Especially when technology and economy are growing at an unbelievable speed, data volume is increasing at a faster rate than one would have expected. In particular, the widespread “accessibility, affordability, and availability of new digital devices that make access to the internet easy” produces the huge amount of data that traditional databases are not capable of storing (Gordon-Murnane 30). That urges the introduction of “Big Data” concept, referring to “datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze” (Bughin 11). Realizing the importance of big data in the current global economy, this paper introduces the characteristics, opportunities and challenges of big data, its business models as well as its applications in different fields of the economy.
Definitively, big data is data that “exceeds the processing capacity of conventional database systems” (Slocum Ch.2, location 31). Businesses turn raw data into useful and informative data that provides insights and helps with business strategies. When the data is too big and does not fit the structures of the company’s database architectures, companies need to find other ways to process and gain value from it. That is why public and private sectors are now taking advantage of big data analytics. Generally, big data is commonly characterized by three V’s: volume, velocity and variety (Slocum Ch.2, location 46). As mentioned earlier, the vast amount of data flows from consumers every day through “email, searching, browsing, blogging, tweeting, buying, sharing, and texting” (Gordon-Murnane 30). Even though the exponential growth of data volume is an important trait of big data, there is no exact definition of the minimum number of terabytes for data to be considered as big data because that number changes with the advance of technology.
Importantly, it is misleading to understand big data solely in terms of size. In fact, big data does not simply have bigger volume; it is more complex than that. The rate at which data flows into an organization has an important role in defining big data. For instance, online retailers “are able to compile large histories of customers’ every click and interaction: not just the final sales” (Slocum Ch.2, location 86). They utilize the data flowing back from consumers to understand what their customers want and thus make decisions on different parts of business, including new products, new marketing strategies, etc. (Gordon-Murnane 31). The velocity of incoming data is very important to organizations since the faster the data inflows, the faster they make decisions and thus gain competitive advantages over their competitors. Also, the speed at which input is transformed to decision is crucial to businesses for the same reason.
It is not only the increased amount of data that matters; the variety in types of data is also significant. Data can come from various and diverse sources. Indeed, with the development of digital world, including social networks, emails, and smart phones, data does not inflow in a particular order or structure. Instead, it can flow into organizations in forms of a line of text, an email, an image, or even a status update or a comment on social networking sites. “Different browsers send different data,” and different users might be using different software and tools to communicate (Slocum Ch.2, location 114), so the conventional way to store simple type of data is not capable of dealing with this complexity, creating the difficulty for collecting informative data. Therefore, big data processing is used to take various types of unstructured data and turn it into meaningful information. Also, big data analytics allows users to keep all in the information and not have to cut any part of it because there might be some useful details in the pieces that are thrown away. Indeed, this indicates an important principle of big data: “when you can, keep everything” (Slocum Ch.2, location 120).
This explosion of data is relatively new. Around year 2000, “only one-quarter of all the world 's stored information was digital,” and the remaining part was still on “paper, film and other analog media” (Cukier). Nowadays, “less than two percent of all stored information is nondigital” (Cukier). Big data was triggered to start and expand by big companies like Google, Yahoo, Amazon, Facebook, and Twitter; they produce tremendous amount of “clickstream data that is only valuable if it is collected and analyzed” (Lamont). This made the traditional Web analytics usages obsolete and insufficient, driving the expanding of big data analytics for handling the complex and massive data processes. Those companies are notable examples of big data used “as an enabler of new products and services.” For instance, Facebook has been able to create a personalized user interface and tailored advertising by combining a huge amount of signals from a user’s activities as well as those of their friends (Slocum Ch.2, location 46).
With the explosion of big data, Apache Hadoop was introduced to as a tool to handle processes that conventional relational databases could not cope with because ultimately, organizations would not want to throw away any data just because it does not fit; they do not know for sure if they will need these pieces of data in the future to support their decision making process. Hadoop is an open-source, “Java-based programming framework” that supports the distributed processing of large datasets across servers. Indeed, Hadoop introduces a cheap, new way to store and process huge amounts of data, regardless of how big, complex and unstructured the data is, or how many types of data there are. Unlike conventional distributed databases which require schemas for data and can only process structured data, Hadoop can deal with data of any format and any size, and that helps Hadoop drive the force behind the fast growth of big data nowadays (Slocum Ch.2, location 180).
The development of Apache Hadoop was originally derived from Google’s MapReduce and Google File Systems. MapReduce is Google’s patented programming framework and is used to reliably store a vast amount of data easily and inexpensively. It is able to take a query “over a dataset, divide it, and run it in parallel over multiple nodes – [computers]” (Slocum Ch.2, location 184). MapReduce consists of Map() and Reduce() procedures; the former performs the filtering and sorting functions, dividing input into smaller parts and distributing them to other nodes for storing and processing; the latter performs operations: worker computers solve the questions asked by users and return the answers back to the central computer, which then combines them and deliver the final result (Slocum Ch.2, location 185). By using these two procedures to distribute data and run processes in parallel sessions, MapReduce, and thus Hadoop, provides a great solution to the issue of data being too big to fit in a single machine (Slocum Ch.2, location 185). Because each of the servers needs data to execute the computations, the Hadoop Distributed File System (HDFS) ensures that they all have access to the data. HDFS is fundamentally similar to other distributed file system, but it offers more advantages over others. It mitigates the high risk of computational failures due to the fact that data collected and processed is very big, complex and unstructured. HDFS allows data to be backed up in case of failures, and the backups guarantee that no data will be lost when any of the working servers has errors (Slocum Ch.2, location 182). With HDFS, developers do not need to worry much about storing data, which allows them to focus more on analyzing data and getting meaningful answers from the data warehouse. Apache Hadoop is the powerhouse behind today’s data processing because it provides users with an easier and cheaper way to make sense of the big data amount they collect on the daily basis. It allows all data to be captured and used in decision-making process, and it can scale linearly (Slocum Ch.2, location 182), lessening the effort taken to store data for future use.
In order to explain why big data concept becomes so popular in global economy, it is essential to understand the value of big data to organizations and the benefits that it can yield. Statistically, businesses globally stored “more than 7 exabytes of new data on disk drives in 2010,” and one exabyte of data is equivalent to “more than 4,000 times the information stored in the US Library of Congress” (Bughin 13). Indeed, we have been generating so much data that it is nearly impossible to store it all. In fact, big data has now “reached every sector” in the economy; it becomes so crucial that no business can grow without it. It also plays an important role in the production process because without information, the process cannot take place.
Managing big data digitally allows stakeholders to have instant access to the data, reducing the searching time. Since organizations nowadays record and store transactional data in digital form, all information regarding the company, from customer orders, or inventories to employees’ performances, can be analyzed, giving meaningful answers to almost every question one might have about the company. Having the ability to store all data that can be pulled for future use will make people knowledgeable about their organizations. In addition, in a highly competitive market today, businesses will have a better chance to survive and grow by giving customers the most benefits such as providing products or services that meet specific needs of customers and creating highly personalized experience for those customers. Businesses can deploy big data to successfully make those marketing and production decisions because data collected from customers reflects customers’ preferences and will not be biased. Not stopping at that point, big data can also help businesses to brainstorm their new business models. Getting and analyzing all data from customers’ transactions can reveal the potentially attractive business model or the shortcomings of current business model. For example, Amazon.com started offering different new shopping sections on its website based on the sales forecast generated from customers’ reviews and transactions. Therefore, using big data has recently become the key for companies to gain competitive advantages and outperform their competitors because in this world, those who are not capable of making sense of data will be left behind (Bughin 16).
Technically, when information is collected, it is usually not in a ready state for analysis. In order to effectively use and analyze data, users need extracted and organized data, and doing this right is challenging. The faster and more accurate the process is, the better the decisions are. Furthermore, data analysis is not simply about storing, locating, and pulling data from a data warehouse; getting the right data for analysis in a timely manner has large impacts on business activities. With big data storage and analytics, users do not need to spend much time on trying to make the data fit in, or expressing data in a structured way to use in the analysis. Instead, they can use their limited time to find ultimate solutions and make more accurate decisions. Therefore, big data has a positive impact on business productivity.
Because big data is often “noisy, dynamic, [complex], heterogeneous, [and] inter-related,” methods for querying and analyzing big data are different from those used for simple, structured data. However, big data can become very relevant and valuable due to the correlation among variables existing in collected data. Indeed, small and structured samples, when being separately analyzed, might not give clear patterns or knowledge, whereas big data can do so because it does not exclude any details, which makes results more meaningful. For instance, for online retailers, observing correlations among customers’ shopping preferences and their transactions is a good way to create tailored marketing and products for their customers, and big data analytics can provide the answer to those correlation questions. More specifically, big data analytics has positive financial impacts on enterprises in different aspects such as product development, market development, operational efficiency, customer experience and loyalty as well as market demand predictions (Computing Community Consortium 2). Thus, big data gives more meaningful information to organizations when being examined carefully (Computing Community Consortium 8).
Although big data has provided organizations with new ways to succeed and outperform their competitors, there exist challenges with applying big data analytics. One of the biggest concern about big data is privacy. It is “the most sensitive issue, with conceptual, legal, and technological implications” (Letouzé 24). Privacy is considered as a fundamental human right, and is a controversial topic discussed every day. Indeed, when the digital world is growing at an unbelievable rate, people are more concerned about their privacy when they actively engage in various interactions in the digital world. With the rise of new technologies, it is possible that privacy may be compromised. Indeed, there is always great public fear regarding the inappropriate use of personal data, particularly “through linking of data from multiple sources” (Computing Community Consortium 11). Using the Internet and social networks, users are generating and sharing data such as their personal profile and information without knowing that they are actually doing so, and even when they are aware of that, they cannot surely know what that data is used for (Letouzé 24). For instance, Facebook and Tweeter users do not clearly know where the data exactly goes to when they post a status, make a tweet or like a photo, etc. As a part of privacy, access and sharing are main concerns for social network users. Although it is widely understood that much of the data is publicly available and has value for development, there are restrictions on how much data that private companies can share about their clients, users as well as their own operations (Letouzé 25). When practicing big data analytics, one should recognize the importance of sharing and privacy issues and how to handle the data so that privacy right is not violated. When businesses work with various other partners in different sectors, a great amount of data must be shared. Also, to make sense of big data effectively, having access and sharing private information may be necessary. These functions will create a controversy about whether the privacy of clients, customers, users or the company is protected or compromised (Letouzé 25). Together with the rise of big data’s advantages, legal issues also arise. Because data can be easily copied, making it harder to answer the questions about the intellectual property rights, or whether the data is being fairly used or not (Bughin 21). These legal issues need to be taken care in order to discover the full potential of big data.
System architecture is also a challenge to big data analytics. Most of businesses nowadays try to utilize business intelligence to gain competitive advantages in the marketplace. Typically, they have different systems to manage different tasks such as brand management analysis, social media analytics, risk assessment, etc. (Computing Community Consortium 13). Yet, with big data – large data sets, it is expensive to use separate systems. It costs the company for not only having to install multiple systems, but also the amount of time to load all data into these multiple systems (Computing Community Consortium 13). Hence, big data analytics drives the need to have one big infrastructure that can sufficiently handle all heterogeneous data, and it is challenging for companies to make sure that the architecture is flexible enough to handle and manage various tasks effectively and efficiently (Computing Community Consortium 14). Therefore, building an appropriate system architecture to handle big data analytics is a big challenge for organizations who want to utilize these business intelligence tools to outperform others.
Besides privacy, sharing and system infrastructure, talent is also a challenging aspect of big data analytics. It is true that with the use of big data, many new types of analysis can be carried out, producing various results that might have not seen before. Thus, to fully understand what those results tell us, it is important to have users who are capable of interpreting the results because decisions are made based on their interpretation (Letouzé 26). There is no point conducting an analysis if users cannot fully understand its result; in this case, big data is not helping the company but costing them more resources. Due to the complexity of big data, including many assumptions and possible errors, users cannot entirely rely on computer system to make the analysis and interpret them itself. Instead, they have to “understand, and verify, the results produced by the computer” (Computing Community Consortium 8). Also, for the right decisions to be made, one must go in-depth into the results and be able to explain or provide information about how the results were derived. Therefore, organizations need to have the right talent to derive the insights from big data, and it is challenging as most organizations do not have enough talent to do these tasks.
Nowadays, with its high potentials, big data can be effectively applied in different sectors of the global economy such as health care and retail industries.
Indeed, health care is a big and developing sector, and it especially becomes more important in the US economy. Clearly, the historic growth rate of the US health care expenditures is “unsustainable, and is a major contributor to the high national debt levels,” and it has been increasing rapidly (Bughin 49). Both providers and payors are looking for ways to lower the cost of health care (Bughin 50). A critical way to lower health care cost is to increase productivity in health care practices. With the rapid development of technology and especially big data, this sector can take advantages of these tools to start making changes and increasing treatment effectiveness and productivity. In fact, health care is a sector that is running behind other industries in adopting and utilizing technology for improvements (Bughin 49), so big data can be applied to transform this industry in many ways. For instance, it is possible to provide low-cost care by implementing best practices, and doing so might need large datasets to draw conclusions from (Bughin 49). Indeed, if health care providers collect enough information about medical treatments, analyze and use it effectively, they will be able to know more accurately which treatments will help which patient, and which do not (Slocum Ch.6, location 1452). The more data and analyses are done, the better understanding of relationship among patients, treatments, and outcomes the providers have, and that creates a huge improvement on medical practices. Moreover, there is so much data in the medical industry, but they are stored separately in different data pools; that makes some data hard to be retrieved and used effectively. Therefore, to explore and boost the potentials of health care industry, those data pools need to be integrated. Currently, primary data pools in this industry are claims and cost data, clinical data such as electronic medical records and electronic health records, pharmaceutical R&D data such as clinical trials data sets, as well as patient behavior data like patient behaviors and personal preferences, or exercise schedule (Bughin 52). Indeed, those data sources are available for accessing and retrieving, but they are not fully utilized. In case those data pools are integrated, many more analyses and statistics can be generated, and thus more effective medical practices can be done by providers. Because we have been already collecting data all the time, big data is a tool to integrate all data that different users collect and to fully utilize data collected. In fact, there are leaders in the health care industry that have utilized big data and had early sign of success. For instance, Kaiser Permanente – a California-based integrated managed-care consortium has fully implemented the HealthConnect system that allows information exchange “across all medical facilities and incorporate[s] electronic health records into clinical practice” (Bughin 126). With a crucial dataset, they were able to discover the adverse effects of a drug called Vioxx, which was subsequently withdrawn from the market (Bughin 51). Hence, health care system still has many rooms for improvement, and by realizing and making use of big data’s potentials, health care industry will be able to provide low-cost care and explore more major opportunities.
Beside health care, retail is one sector that has been using big data heavily to bring profitability to businesses. Digital data plays an increasingly important role since consumers “search, research, compare, buy, and obtain support online” (Bughin 74), generating a huge amount of digital data from every transaction. Even though retail is an important component of the economy, the profitability of this sector has been under “intense pressure” due to the competitiveness of the marketplace, and the convenience and flexibility that consumers have when obtaining immediate price and product comparisons (Bughin 74). Therefore, retail businesses seek new opportunities from big data in order to gain competitive advantages in the market. In fact, many US retailers have been leveraging technology and digital data in their decision making process. For instance, Walmart “pioneered the expansion of an electronic data interchange system to connect its supply chain electronically” (Bughin 76). Today, many leaders in the industry are using data mining of customer data to support their decisions about various aspects of the business such as pricing, marketing, supply chain management, merchandising, or new business strategies (Bughin 77). Successfully using big data analytics in the decision making process will help retailers reduce costs, gain an edge in the market and develop new strategies to grow their revenue as well as profit.
Big data is a key trend in the twenty-first century. It offers organizations with various opportunities that did not exist before. At present, it does not replace the approaches, tools, systems such as conventional databases, but the remarkable point about big data is that it has become a powerful tool to support various sectors in the economy, especially when digital data is increasingly produced. Therefore, by discovering its full potentials, we will be able to explore and bring new opportunities to the organizations we work in.

References
 Bughin, Jacques, et al. "Big Data: The Next Frontier For Innovation, Competition, And Productivity." Big Data: The Next Frontier For Innovation, Competition & Productivity (2011): 1-143. Business Source Complete. Web. 9 June 2013.
 Computing Community Consortium. “Challenges and Opportunities with Big Data.” Computing Research Association. 2012. Web. 25 May 2013.
 Cukier, Kenneth, and Viktor Mayer-Schoenberger. "The Rise Of Big Data." Foreign Affairs 92.3 (2013): 27-40. Academic Search Premier. Web. 20 May 2013.
 Gordon-Murnane, Laura. "Big Data: a Big Opportunity for Librarians." Online. 36.5 (2012): 30-34. Print.
 Lamont, Judith. "BIG DATA Interviews with the Experts." (n.d.): n. pag. 31 Dec. 2012. Web. 1 June 2013.
 Letouzé, Emmanuel. “Big Data for Development: Challenges & Opportunities.” UN Global Pulse. 2011. Web. 1 June 2013.
 Slocum, Mac, ed. Big Data Now Current Perspectives from O 'Reilly Media: 2012 Edition. Sebastopol, CA: O 'Reilly Media, 2012. 24 Oct. 2012. Web. 15 May 2013. Kindle Edition.

References:  Bughin, Jacques, et al. "Big Data: The Next Frontier For Innovation, Competition, And Productivity." Big Data: The Next Frontier For Innovation, Competition & Productivity (2011): 1-143. Business Source Complete. Web. 9 June 2013.  Computing Community Consortium. “Challenges and Opportunities with Big Data.” Computing Research Association. 2012. Web. 25 May 2013.  Cukier, Kenneth, and Viktor Mayer-Schoenberger. "The Rise Of Big Data." Foreign Affairs 92.3 (2013): 27-40. Academic Search Premier. Web. 20 May 2013.  Gordon-Murnane, Laura. "Big Data: a Big Opportunity for Librarians." Online. 36.5 (2012): 30-34. Print.  Lamont, Judith. "BIG DATA Interviews with the Experts." (n.d.): n. pag. 31 Dec. 2012. Web. 1 June 2013.  Letouzé, Emmanuel. “Big Data for Development: Challenges & Opportunities.” UN Global Pulse. 2011. Web. 1 June 2013.  Slocum, Mac, ed. Big Data Now Current Perspectives from O 'Reilly Media: 2012 Edition. Sebastopol, CA: O 'Reilly Media, 2012. 24 Oct. 2012. Web. 15 May 2013. Kindle Edition.