Quick Hadoop Map Reduce Reference Card

Map Reduce API

For Imports Classes&Interfaces
Configuration org.apache.hadoop.conf.* Configuration, Configured, Tool
MR Job,
Formats
org.apache.hadoop.mapreduce.* Mapper, Reducer, FileInputFormat,
FileOutputFormat,
TextInputFormat, TextOutputFormat,...
ata t!pes org.apache.hadoop.io.* Text, "ong#ritable, Float#ritable,
$!te#ritable,...
File s!stem org.apache.hadoop.fs.* File%!stem, F%ataInput%tream,
F%ataOutput%tream, &ath, File%tatus ...
'tilities org.apache.hadoop.utils.*
org.apache.hadoop.IO'tils.*
Cop!$!tes, ReadFull!
(ati)e Ja)a *a)a.io.IO+xception,
*a)a.util.Iterator, ...
MapReduce data types: Writables
#ritable -rapper classes for Ja)a primiti)es
Class i!e in
bytes
"escription ort Policy
in t#e sort p#ase
$oolean
#ritable
. #rapper for a standard $oolean )ariable False before and true after
$!te#rit
able
. #rapper for a single b!te /scending order
ouble
#ritable
0 #rapper for a ouble /scending order
Float#ri
table
1 #rapper for a Float /scending order
Int#rita
ble
1 #rapper for a Integer /scending order
"ong#ri
table
0 #rapper for a "ong /scending order
Text 23$ #rapper to store text using the unicode
'TF0 format
/lphabetic order
(ull#rit
able
&laceholder -hen the 4e! or )alue is not
needed
'ndefined
Your
Writable
Implement the #ritable Interface for a
)alue or #ritableComparable5T6 for a 4e!
7our sort polic!
MapReduce InputFormat
import org.apache.hadoop.mapreduce.lib.input.*
Input Format "escription
TextInputFormat +ach line in text file is a record.
key8 "ong#ritable, Offset of the line
$alue8 Text, content of the line
9e!:alueTextInputFormat +ach line is a record. First separator di)ides each line. %eparator
set b! key%$alue%separator%in%input%line propert!, default is tab
character ;<t=.
key8 Text, an!thing before the separator
$alue8 Text, e)er!thing after the separator
%e>uenceFileInputFormat59,:6 /n InputFormat for reading se>uence files. / se>uence file is a
?adoop@specific compressed binar! file format.
key8 9 ;user defined=
$alue8 : ;user defined=
("ineInputFormat "i4e TextInputFormat, but each split is guaranteed to ha)e exactl!
( lines. %et b! mapred%line%input%format%linespermap propert!
key8 "ong#ritable
$alue8 Text
MapReduce &utputFormat
import org.apache.hadoop.mapreduce.lib.output.*
&utput Format "escription
TextOutputFormat59,:6 #rites each record as a line of text. 9e!s and )alues are
-ritten as strings and separated b! a tab ;<t= character, -hich
can be changed in the mapred%te'toutputformat%separator
propert!.
%e>uenceFileOutputFormat59,:6 #rites the 4e!A)alue pairs in se>uence file format. #or4s in
con*unction -ith %e>uenceFileInputFormat.
(ullOutputFormat59,:6 Outputs nothing