Hadoop + HBase + Cygwin + Windows 7 x64

In this post I will describe how to get a Hadoop environment with HBase running in Cygwin on Windows 7 x64.

Having spent the better part of a week reading through blog posts and documentation, I found that none of them covered the process in full detail, at least not for the software versions I intended to use.

This guide was written for Cygwin 1.7.7, Hadoop 0.21.0 and HBase 0.20.6.

UPDATE (Nov. 1, 2010): I’ve noticed some errors arising when using Hadoop 0.21.0 and HBase 0.20.6 and gone back to Hadoop 0.20.2 instead as this does not produce the same errors. If you intend to use HBase together with Hadoop I would recommend setting up Hadoop 0.20.2 instead, the installation is more or less identical.

You will additionally need ZooKeeper 3.3.1 in order to get HBase to run properly.

Throughout this guide I will assume that your Cygwin install path will be c:\cygwin and that Hadoop, ZooKeeper and HBase will be installed in c:\cygwin\etc\local (/etc/local/), this is however something you can choose yourself. If you choose to install Cygwin elsewhere, I would recommend to use folder names without whitespaces and other non-regular charaters.

The only prerequisite for this quite is that you have Java installed and added to your %PATH% variable (which is usually done automatically).

Step 1 – Download software
Step 2 – Install and configure Cygwin
Step 3 – Install and configure Hadoop
Step 4 – Install and configure ZooKeeper
Step 5 – Install and configure HBase
Step 6 – Start your cluster

Software

Download each software bundle and put it somewhere where you’ll easilly find it later.

• Cygwin: Cygwin 1.7.7,
• Hadoop: Hadoop 0.21.0
• HBase: HBase 0.20.6
• ZooKeeper: ZooKeeper 3.3.1

Cygwin
If you’ve never used Cygwin (or Linux/Unix/etc), you should perhaps get familiar with those environments first. If you still want to continue, read on.

Read more: alan said

Posted via email from .NET Info