Hadoop on Mac OS X
From NeoWiki
(Difference between revisions)
(New page: This guide is written to help Cornell students using Mac OS X 10.5 with setting up a development environment for working with [http://www.hadoop.org/ Hadoop] and running Hadoop jobs on the...) |
m |
||
| Line 1: | Line 1: | ||
| + | ''From http://www.infosci.cornell.edu/hadoop/mac.html'' | ||
| + | |||
This guide is written to help Cornell students using Mac OS X 10.5 with setting up a development environment for working with [http://www.hadoop.org/ Hadoop] and running Hadoop jobs on the [http://7thgen.info/images/intro_icon_blog.jpg Cornell Center for Advanced Computing (CAC)] Hadoop cluster. This guide will walk you through compiling and running a simple example Hadoop job. More information is available at the official [http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#Example%3A+WordCount+v1.0 Hadoop Map-Reduce Tutorial]. | This guide is written to help Cornell students using Mac OS X 10.5 with setting up a development environment for working with [http://www.hadoop.org/ Hadoop] and running Hadoop jobs on the [http://7thgen.info/images/intro_icon_blog.jpg Cornell Center for Advanced Computing (CAC)] Hadoop cluster. This guide will walk you through compiling and running a simple example Hadoop job. More information is available at the official [http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#Example%3A+WordCount+v1.0 Hadoop Map-Reduce Tutorial]. | ||
Revision as of 02:37, 17 November 2009
From http://www.infosci.cornell.edu/hadoop/mac.html
This guide is written to help Cornell students using Mac OS X 10.5 with setting up a development environment for working with Hadoop and running Hadoop jobs on the Cornell Center for Advanced Computing (CAC) Hadoop cluster. This guide will walk you through compiling and running a simple example Hadoop job. More information is available at the official Hadoop Map-Reduce Tutorial.
The overall process of developing a Hadoop job is as follows:
- Install Hadoop on your development machine (personal or lab computer)
- Compile the Hadoop job, create a JAR file
- Run the Hadoop job JAR file on your development machine, for testing and debugging
- Run the Hadoop job JAR file on the CAC Hadoop cluster, for production