Big Data Basics: A Beginner's Tutorial

This tutorial explains the fundamentals of Big Data. We’ll cover the definition and explore the basics you need to know.

Definition of Big Data:

According to Gartner in 2012, “Big Data are high-volume, high-velocity, and/or high-variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization.”

In simpler terms, Big Data refers to the massive amounts of data being collected and stored from various sources like:

• Web Data, E-commerce
• Purchases at department or grocery stores
• Bank/Credit Card Transactions
• Social Networks

How Much Data Are We Talking About?

• Google processes 20 PB (Petabytes) a day (Statistics: year 2008)
• 1 PB = $10^{15}$ Bytes = 1 million gigabytes = 1 thousand terabytes
• Facebook had 2.5 PB of user data + 15 TB/day (Statistics: April 2009)
• eBay had 6.5 PB of user data + 50 TB/day (Statistics: May 2009)

Big Data Vectors

Big Data is often characterized by the “Three V’s”:

• High-Volume: The sheer amount of data.
• High-Velocity: The speed at which data is collected, acquired, generated, and processed.
• High-Variety: The different types of data, such as:
  • Text, audio, video, image data, XML
  • Relational data (e.g., tables, transactions, legacy systems)
  • Graph data (semantic web, social networks)
  • Streaming data (data that can only be scanned once)

What Can Be Done with Big Data?

• Aggregation and Statistics: Data warehousing and OLAP (Online Analytical Processing).
• Indexing, Searching, and Querying:
  • Keyword-based search
  • Pattern matching (XML/RDF)
• Knowledge Discovery:
  • Data Mining
  • Statistical Modeling

Hadoop was developed to handle the growing demands of Big Data.