Hadoop For Dummies

Let Hadoop For Dummies help harness the power of your data and rein in the information overload

Big data has become big business, and companies and organizations of all sizes are struggling to find ways to retrieve valuable information from their massive data sets with becoming overwhelmed. Enter Hadoop and this easy-to-understand For Dummies guide. Hadoop For Dummies helps readers understand the value of big data, make a business case for using Hadoop, navigate the Hadoop ecosystem, and build and manage Hadoop applications and clusters.

• Explains the origins of Hadoop, its economic benefits, and its functionality and practical applications
• Helps you find your way around the Hadoop ecosystem, program MapReduce, utilize design patterns, and get your Hadoop cluster up and running quickly and easily
• Details how to use Hadoop applications for data mining, web analytics and personalization, large-scale text processing, data science, and problem-solving
• Shows you how to improve the value of your Hadoop cluster, maximize your investment in Hadoop, and avoid common pitfalls when building your Hadoop cluster

From programmers challenged with building and maintaining affordable, scaleable data systems to administrators who must deal with huge volumes of information effectively and efficiently, this how-to has something to help you with Hadoop.

Introduction 1

Part I: Getting Started with Hadoop 7

Chapter 1: Introducing Hadoop and Seeing What It’s Good For 9

Chapter 2: Common Use Cases for Big Data in Hadoop 23

Chapter 3: Setting Up Your Hadoop Environment 41

Part II: How Hadoop Works 51

Chapter 4: Storing Data in Hadoop: The Hadoop Distributed File System 53

Chapter 5: Reading and Writing Data 69

Chapter 6: MapReduce Programming 83

Chapter 7: Frameworks for Processing Data in Hadoop: YARN and MapReduce 103

Chapter 8: Pig: Hadoop Programming Made Easier 117

Chapter 9: Statistical Analysis in Hadoop 129

Chapter 10: Developing and Scheduling Application Workflows with Oozie 139

Part III: Hadoop and Structured Data 155

Chapter 11: Hadoop and the Data Warehouse: Friends or Foes? 157

Chapter 12: Extremely Big Tables: Storing Data in HBase 179

Chapter 13: Applying Structure to Hadoop Data with Hive 227

Chapter 14: Integrating Hadoop with Relational Databases Using Sqoop 269

Chapter 15: The Holy Grail: Native SQL Access to Hadoop Data 303

Part IV: Administering and Configuring Hadoop 313

Chapter 16: Deploying Hadoop 315

Chapter 17: Administering Your Hadoop Cluster 335

Part V: The Part of Tens 359

Chapter 18: Ten Hadoop Resources Worthy of a Bookmark 361

Chapter 19: Ten Reasons to Adopt Hadoop 371

Index 379

Dirk deRoos is the technical sales lead for IBM’s InfoSphere BigInsights. Paul C. Zikopoulos is the vice president of big data in the IBM Information Management division. Roman B. Melnyk, PhD is a senior member of the DB2 Information Development team. Bruce Brown and Rafael Coss work with big data with IBM.

Learn to:

• Understand the value of big data and how Hadoop can help manage it
• Navigate the Hadoop 2 ecosystem and create clusters
• Use applications for data mining, problem-solving, analytics, and more

The easy-to-use, practical guide to using Hadoop for big data

With most of the world’s data created in only the past two years, Hadoop has emerged as the definitive computing paradigm to handle big data. This comprehensive guide from IBM big data experts provides a hands-on resource for those who want to dig into the details of HDFS and MapReduce to take data storage and processing to the next level.

• Get started with Hadoop — discover the origins of Hadoop, the realities of worldwide data growth, and practical use cases for this revolutionary platform
• Under the Hadoop hood — dig into Hadoop’s distributed framework, including HDFS and MapReduce and the best tools for working with data in Hadoop
• Hadoop and structured data — modernize data warehouses with Hadoop and discover data utilities like HBase, Hive, and Sqoop
• Hands on with Hadoop — get your hands dirty with details on configuring Hadoop clusters and an overview of day-to-day Hadoop administration
• Take your Hadoop knowledge to the next level — use additional Hadoop resources to understand the technology at a deeper level

Open the book and find:

• Coverage of the Hadoop 2 ecosystem and Yarn
• Real-world use cases to help you get started
• Details on Hadoop distributions and cluster setup
• How to use Oozie for scheduling workflows
• How to add structure with Hive and HBase
• Details on running native SQL queries on Hive
• On-premise and cloud deployment options for Hadoop
• The challenges faced by administrators