Re: Run giraph on Hadoop yarn 2.6.0
Hi Flavian, do you try follow all this thread: http://mail-archives.apache.org/mod_mbox/giraph-user/201411.mbox/%3cca+bectmbb2x3mxsn_e0g_4iuwtwlhvwyyteu9hdvt+h7vqu...@mail.gmail.com%3e In any case I’ll test it as soon as I can against the new version. Cheers, Alessandro Il giorno 02/dic/2014, alle ore 11:09, Flavian Jacquot ha scritto: > Hello, I'm trying to use Giraph on hadoop 2.6.0 with yarn. > > I've managed to build it by removing STATIC_SASL_SYMBOL in in > the yarn profile. with the command : > sudo mvn -Phadoop_yarn -Dhadoop.version=2.6.0 -DskipTests package > > Then i've setup a single node cluster with hadoop 2.6.0 and I'm able to > perform the basic wordcount example. > I've copied the jar built into the directory share/hadoop/mapreduce to remove > the GiraphMaster not found execption. > > Now I'm trying to run the SimpleShortestPaths example with the command : > > $HADOOP_PREFIX/bin/hadoop jar > /usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.6.0-jar-with-dependencies.jar > org.apache.giraph.GiraphRunner > org.apache.giraph.examples.SimpleShortestPathsComputation -vif > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip > input_tiny2.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat > -op /user/hduser/output/shortestpaths445 -w 1 -ca > giraph.SplitMasterWorker=false -yj > giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.6.0-jar-with-dependencies.jar > > The data used is the tiny graph on the giraph quick start page and the job > should takes minutes to complete but it doesn't. > > The full log is at http://pastebin.com/CW1FnWz2 > > It's possible to make giraph work on hadoop 2.6.0 or do I have to use an > earlier version ? If so which one would you suggest ?
Run giraph on Hadoop yarn 2.6.0
Hello, I'm trying to use Giraph on hadoop 2.6.0 with yarn. I've managed to build it by removing STATIC_SASL_SYMBOL in in the yarn profile. with the command : sudo mvn -Phadoop_yarn -Dhadoop.version=2.6.0 -DskipTests package Then i've setup a single node cluster with hadoop 2.6.0 and I'm able to perform the basic wordcount example. I've copied the jar built into the directory share/hadoop/mapreduce to remove the GiraphMaster not found execption. Now I'm trying to run the SimpleShortestPaths example with the command : $HADOOP_PREFIX/bin/hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.6.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip input_tiny2.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/shortestpaths445 -w 1 -ca giraph.SplitMasterWorker=false -yj giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.6.0-jar-with-dependencies.jar The data used is the tiny graph on the giraph quick start page and the job should takes minutes to complete but it doesn't. The full log is at http://pastebin.com/CW1FnWz2 It's possible to make giraph work on hadoop 2.6.0 or do I have to use an earlier version ? If so which one would you suggest ?
Run giraph on Hadoop yarn 2.6.0
Hello, I'm trying to use Giraph on hadoop 2.6.0 with yarn. I've managed to build it by removing STATIC_SASL_SYMBOL in in the yarn profile. with the command : sudo mvn -Phadoop_yarn -Dhadoop.version=2.6.0 -DskipTests package Then i've setup a single node cluster with hadoop 2.6.0 and I'm able to perform the basic wordcount example. I've copied the jar built into the directory share/hadoop/mapreduce to remove the GiraphMaster not found execption. Now I'm trying to run the SimpleShortestPaths example with the command : $HADOOP_PREFIX/bin/hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.6.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip input_tiny2.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/shortestpaths445 -w 1 -ca giraph.SplitMasterWorker=false -yj giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.6.0-jar-with-dependencies.jar The data used is the tiny graph on the giraph quick start page and the job should takes minutes to complete but it doesn't. The full log is at http://pastebin.com/CW1FnWz2 It's possible to make giraph work on hadoop 2.6.0 or do I have to use an earlier version ? If so which one would you suggest ?
