Running Hadoop Example(TDG/CH02) on Cloudera Quick Start VM 1

prerequisite

1. VMWare Player (Oracle VM box didn’t work for me)

Download VMWare Player

2. Cloudera Quick Start VM

Download Cloudera Quick Start VM

3. Download CH02 example java code from here.

4. create a folder BigDataAssignment1/code(name it whatever you want) inside /home/cloudera/workspace/

5. Download hadoop-core-0.20.2.jar from here and place it inside code folder you created in the previous step

Steps

1. Compile Java Program

javac -cp hadoop-core-0.20.2.jar MaxTemperatureMapper.java

javac -cp hadoop-core-0.20.2.jar MaxTemperatureReducer.java

javac -cp hadoop-core-0.20.2.jar:. MaxTemperature.java

2. Create jar from classes

jar cf assignment1.jar *.class

3. Set environment HADOOP_CLASSPATH

export HADOOP_CLASSPATH=/home/cloudera/workspace/BigDataAssignment1/code/assignment1.jar

4. Copy input file HDFS folder /cloudera/data

hdfs dfs -put input.txt /user/cloudera/data

5. Execute using the following command

hadoop MaxTemperature ./data/input.txt output

6. Copy the content of HDFS output folder to local file system using the following command

hdfs dfs -get /user/cloudera/output/part-r-00000 /home/cloudera/workspace/BigDataAssignment1/output/part-r-00000

hdfs dfs -get /user/cloudera/output/part-r-00000 /home/cloudera/workspace/BigDataAssignment1/output/_SUCCESS

5/5 - (1 vote)

One comment on “Running Hadoop Example(TDG/CH02) on Cloudera Quick Start VM

  1. Reply Hadoop online Training Mar 30,2015 4:11 pm

    I was seeking this certain information for a long time.Hadoop Online Training .

Leave a Reply