Guest Article: What is Big Data?
Rohit Rai I Director FSI, MongoDB
For organizations of all sizes, data management has shifted from an important competency to a critical differentiator. Fortune 1000 companies and government bodies are starting to benefit from the innovations of the web pioneers. These organizations are defining new initiatives and re-evaluating existing strategies to examine how they can transform their businesses using Big Data. In the process, they are learning that Big Data is not a single technology, technique or initiative. Rather, it is a trend across many areas of business and technology.
Today, new technologies make it possible to realize value from Big Data. For example, retailers can track user web clicks to identify behavioural trends that improve campaigns, pricing and stocking. Utilities can capture household energy usage levels to predict outages and to incent more efficient energy consumption. Governments and even Google can detect and track the emergence of disease outbreaks via social media signals. Oil and gas companies can take the output of sensors in their drilling equipment to make more efficient and safer drilling decisions. “Big Data” describes data sets so large and complex they are impractical to manage with traditional software tools.
Specifically, Big Data relates to data creation, storage, retrieval and analysis that is remarkable in terms of volume, velocity, and variety:
o Volume: Facebook ingests 500 terabytes of new data every day; a Boeing 737 will generate 240 terabytes of flight data during a single flight across the US; the proliferation of smart phones, the data they create and consume; sensors embedded into everyday objects will soon result in billions of new, constantly-updated data feeds containing environmental, location, and other information.
o Velocity: Clickstreams and ad impressions capture user behaviour at millions of events per second; high-frequency stock trading algorithms reflect market changes within microseconds; machine to machine processes exchange data between billions of devices; infrastructure and sensors generate massive log data in real-time. Each producing multiple inputs per second.
o Variety: Big Data isn’t just numbers, dates, and strings. It is also geospatial data, 3D data, audio and video, and unstructured text, including log files and social media. Traditional database systems were designed to address smaller volumes of structured data, fewer updates or a predictable, consistent data structure. Big Data databases, such as MongoDB, solve these problems and provide companies with the means to create tremendous business value.
BIG DATA FOR THE ENTERPRISE
With Big Data databases, enterprises can save money, grow revenue, and achieve many other business objectives, in any vertical.
o Build new applications: Big data might allow a company to collect billions of real-time data points on its products, resources, or customers – and then repackage it to optimize customer experience or resource utilization. For example, a major US city is using MongoDB to cut crime and improve municipal services by collecting and analyzing geospatial data in real-time from over 30 different departments.
o Improve the effectiveness and lower the cost of existing applications: Big data technologies can replace highly-customized, expensive legacy systems with a standard solution that runs on commodity hardware. And because many big data technologies are open source, they can be implemented far more cheaply than proprietary technologies. For ex: by migrating its reference data management application to MongoDB, a Tier 1 bank dramatically reduced the license and hardware costs associated with its previous proprietary relational database.
o Realize new sources of competitive advantage: Big data can help businesses act more nimbly, allowing them to adapt to changes faster than their competitors. For example, MongoDB allowed one of the largest Human Capital Management (HCM) solution providers to rapidly build mobile applications that integrated data from a wide variety of disparate sources.
o Increase customer loyalty: Increasing the amount of data shared within the organization – and the speed with which it is updated – allows businesses and other organizations to more rapidly and accurately respond to customer demand. For example, a top 5 global insurance provider, MetLife, used MongoDB to quickly consolidate customer information from over 70 different sources and provide it in a single, rapidly-updated view.
– ROHIT RAI I Director FSI, MongoDB
Other Articles in the newsletter:
The ‘Big’ Announcement – Sachin Dabir
Three Things to Unlearn about RDBMS before you Learn MongoDB – Sameer K
Internet of Things, Data explosion and Databases of the Future – Michael Santana