Hi Vikalp,
finally I got to run Giraph successfully :-)
Following steps:
#Preparation
*
su
*
rm -rf /usr/local/giraph; mkdir /usr/local/giraph
*
chown hadoop:hadoop /usr/local/giraph
*
mkdir ~/setup; cd ~/setup
#Sourcecode (Version 1.1.0)
*
git clone https://git-wip-us.apache.org/repos/asf/giraph.git
*
mv giraph giraph_src; cd giraph_src
*
mvn clean
*
mvn -Phadoop_yarn -Dhadoop.version=2.4.0 -DskipTests package
#Installation (little ugly)
*
rm -rf myGiraph; mkdir -p myGiraph/lib; mkdir myGiraph/source
*
find . -name \*.jar -exec cp {} myGiraph/lib \;
*
cp -r conf myGiraph
*
cp -r bin myGiraph
*
cp CH* CO* RE* LI* NO* myGiraph
*
cd myGiraph
*
mv lib/*source*.jar source
*
mv lib/giraph*.jar .
*
rename 's/-SNAPSHOT-for//' *.jar
*
rm -rf ../myGiraph; cd ..; mv myGiraph ..
*
cd ../myGiraph; rm -rf /usr/local/giraph/*; cp -r * /usr/local/giraph
Now you have to put a variable into hadoop-env.sh (at the end is ok):
o
/MY_LIBS="$HADOOP_HOME/share/myLib/*.jar"/
o
/export HADOOP_TASKTRACKER_OPTS="-classpath ${MY_LIBS}
${HADOOP_TASKTRACKER_OPTS}"/
The folder myLib you have to create and put the Giraph-Jar:
* mkdir /usr/local/hadoop/share/myLib
*
cp
/usr/local/giraph/giraph-examples-1.1.0-hadoop-2.4.0-jar-with-dependencies.jar
/usr/local/hadoop/share/myLib
#Configuration
Put
<property>
<name>mapred.map.tasks</name>
<value>4</value>
</property>
into mapred-site.xml
*
echo export PATH=\$PATH:/usr/local/giraph/bin >> ~/.bashrc
*
echo export GIRAPH_HOME=/usr/local/giraph >> ~/.bashrc
*
source ~/.bashrc
Now you can test:
#creating testdata
*
rm -rf ~/giraph; mkdir ~/giraph; cd ~/giraph
* cat << EOF > tiny_graph.txt
* [0,0,[[1,1],[3,3]]]
[1,0,[[0,1],[2,2],[3,1]]]
[2,0,[[1,2],[4,4]]]
[3,0,[[0,3],[1,1],[4,4]]]
[4,0,[[3,4],[2,4]]]
* EOF
#Run example-Jar
*
jar=giraph-examples-1.1.0-hadoop-2.4.0-jar-with-dependencies.jar
*
runner=org.apache.giraph.GiraphRunner
*
computation=org.apache.giraph.examples.SimpleShortestPathsComputation
*
informat=org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
*
outformat=org.apache.giraph.io.formats.IdWithValueTextOutputFormat
*
hdfs dfs -rm -r shortestpaths
*
yarn jar $GIRAPH_HOME/$jar $runner -Dgiraph.yarn.task.heap.mb=3000
$computation -vif $informat -vip tiny_graph.txt -vof $outformat -op
shortestpaths -w 1 -yj $jar
*
hdfs dfs -cat shortestpaths/*
Finished!! Hope it functions on your site.
Best regards,
Alexander
On 12.08.2014 15:52, Vikalp Handa wrote:
Hi Alexander,
I looked into my gam-stderr.log file and found *Error: Could not find
or load main class org.apache.giraph.yarn.GiraphApplicationMaster*
As you have already mentioned in your reply about that for loop in
*hadoop-env.sh. *
So can you please tell me what is *mylib *and***share/myLib/giraph** ?
*Because I am having only doc and hadoop directories
inside $HADOOP_HOME/share/
Also how to get *giraph-1.1.0-hadoop-2.4.0.jar
*and*giraph-examples-1.1.0-hadoop-*2.4.0.jar (without the
dependencies) as I am having only
giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.0-jar-with-dependencies.jar
inside giraph-examples/target/munged/.
