Anyone please reply ..Is it portability problem??.. Does giraph has any issues with Hadoop 2.2.0??
Do I need to build Giraph on the new system ?? Thanks On Sat, Mar 1, 2014 at 2:28 PM, Jyoti Yadav <rao.jyoti26ya...@gmail.com>wrote: > Hi Sebastian.. > Thanks for the links given for big graphs.. > > Actually I want to tell you something about problem i am facing. > > Initially I was working with *hadoop 0.20.203* . I build Giraph there.. > it was running fine. > > Now to test very big graph related problem and to compare the performance > , I moved to new system which is of 64 cores and 512 GB memory and 3 TB > storage. Instead to building Giraph in the new system, I just copied > Giraph folder from my previous system to this new system. In this new > system *hadoop version 2.2..0 * . I tried to execute > SimpleSourceShortestPath algo on sample data set. It is throwing following > exception. > > I gave following command to execute the job. > > hadoop jar > /home/abcd2014/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar > org.apache.giraph.GiraphRunner -Dgiraph.SplitMasterWorker=true > org.apache.giraph.examples.SimpleShortestPathsComputation -vif > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat > -vip /user/abcd2014/giraph_input/tiny_graph.txt -vof > org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op > /user/abcd2014/output2/shortestpaths -w 1 > > > > 14/03/01 12:44:46 INFO utils.ConfigurationUtils: No edge input format > specified. Ensure your InputFormat does not require one. > 14/03/01 12:44:46 INFO utils.ConfigurationUtils: No edge output format > specified. Ensure your OutputFormat does not require one. > 14/03/01 12:44:46 INFO Configuration.deprecation: > mapreduce.job.counters.limit is deprecated. Instead, use > mapreduce.job.counters.max > 14/03/01 12:44:46 INFO Configuration.deprecation: mapred.job.map.memory.mb > is deprecated. Instead, use mapreduce.map.memory.mb > 14/03/01 12:44:46 INFO Configuration.deprecation: > mapred.job.reduce.memory.mb is deprecated. Instead, use > mapreduce.reduce.memory.mb > 14/03/01 12:44:46 INFO Configuration.deprecation: > mapred.map.tasks.speculative.execution is deprecated. Instead, use > mapreduce.map.speculative > 14/03/01 12:44:46 INFO Configuration.deprecation: > mapreduce.user.classpath.first is deprecated. Instead, use > mapreduce.job.user.classpath.first > 14/03/01 12:44:46 INFO Configuration.deprecation: mapred.map.max.attempts > is deprecated. Instead, use mapreduce.map.maxattempts > 14/03/01 12:44:46 INFO job.GiraphJob: run: Since checkpointing is disabled > (default), do not allow any task retries (setting mapred.map.max.attempts = > 0, old value = 4) > 14/03/01 12:44:46 INFO Configuration.deprecation: mapred.job.tracker is > deprecated. Instead, use mapreduce.jobtracker.address > > *Exception in thread "main" java.lang.IllegalArgumentException: > checkLocalJobRunnerConfiguration: When using LocalJobRunner, you cannot run > in split master / worker mode since there is only 1 task at a time! * > at > org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:165) > at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:233) > at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > > > > Would you suggest me something to fix this...If you need any details > further,please let me know... > > Thanks & Regards > > Jyoti > > > > > On Sat, Mar 1, 2014 at 1:35 PM, Sebastian Schelter <s...@apache.org> wrote: > >> Hi Jyoti, >> >> You can find a couple of very large graphs in KONECT [1] and on the >> website of the laboratory for web algorithmics from the University of Milan >> [2]. You will probably have to convert them to an appropriate format for >> Giraph. >> >> Best, >> Sebastian >> >> [1] http://konect.uni-koblenz.de/ >> [2] http://law.di.unimi.it/datasets.php >> >> >> On 03/01/2014 05:22 AM, Jyoti Yadav wrote: >> >>> Hi folks.. >>> >>> I got new system which is of 64 cores and 512 GB memory and 3 TB >>> storage.I want to test the performance of Giraph on this system. >>> Would anyone provide me the link for very large graph so that I can >>> execute Single Source Shortest Path Example. For this algo to run graph >>> should be weighted graph. and to feed it into the Giraph -input format >>> is >>> JsonLongDoubleFloatDouble >>> >>> Thanks in advance... >>> With Regards >>> >>> Jyoti >>> >>> >> >
