Running Hadoop Example(TDG/CH02) on Cloudera Quick Start VM 1

prerequisite

1. VMWare Player (Oracle VM box didn’t work for me)

2. Cloudera Quick Start VM

3. Download CH02 example java code from here.

4. create a folder BigDataAssignment1/code(name it whatever you want) inside /home/cloudera/workspace/

5. Download hadoop-core-0.20.2.jar from here and place it inside code folder you created in the previous step

Steps

1. Compile Java Program

javac -cp hadoop-core-0.20.2.jar MaxTemperatureMapper.java

javac -cp hadoop-core-0.20.2.jar MaxTemperatureReducer.java

javac -cp hadoop-core-0.20.2.jar:. MaxTemperature.java

2. Create jar from classes

jar cf assignment1.jar *.class

3. Set environment HADOOP_CLASSPATH

export HADOOP_CLASSPATH=/home/cloudera/workspace/BigDataAssignment1/code/assignment1.jar

4. Copy input file HDFS folder /cloudera/data

hdfs dfs -put input.txt /user/cloudera/data

5. Execute using the following command

hadoop MaxTemperature ./data/input.txt output

6. Copy the content of HDFS output folder to local file system using the following command

hdfs dfs -get /user/cloudera/output/part-r-00000 /home/cloudera/workspace/BigDataAssignment1/output/part-r-00000

hdfs dfs -get /user/cloudera/output/part-r-00000 /home/cloudera/workspace/BigDataAssignment1/output/_SUCCESS

5/5 - (1 vote)

One comment on “Running Hadoop Example(TDG/CH02) on Cloudera Quick Start VM”