@Claudio Martella Your solution didnt work either. I basically tried copying the pageRankBenchmark to my own package, renamed the package. It compiles fine with giraph. But I couldnt run it even if I add those files to giraph jar before deployment. Help?
On Sun, Aug 25, 2013 at 6:33 PM, Claudio Martella < claudio.marte...@gmail.com> wrote: > you have this problem when you use two jars (one with giraph and one with > your classes) instead of a single fat-jar, correct? I tracked the same > problem a few weeks ago, basically zookeeper is run passing the wrong jar. > > > On Sat, Aug 24, 2013 at 4:51 PM, Vivek Sembium <vivek.semb...@gmail.com>wrote: > >> Thank you for your suggestion. It worked. Its not giving class not found >> exception. But its giving me a new error >> Its stopping at map 0% and reduce 0%. Upon inspection I found that its >> unable to connect to zookeeper service. >> >> java.lang.IllegalStateException: run: Caught an unrecoverable exception >> onlineZooKeeperServers: Failed to connect in 10 tries! >> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) >> at org.apache.hadoop.mapred.Child$4.run(Child.java:255) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:416) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) >> at org.apache.hadoop.mapred.Child.main(Child.java:249) >> Caused by: java.lang.IllegalStateException: onlineZooKeeperServers: >> Failed to connect in 10 tries! >> at >> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:727) >> at >> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:371) >> at >> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:204) >> at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59) >> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89) >> ... 7 more >> >> Immediately I ran page rank benchmark and it executed successfully both >> from giraph in lib directory and also from giraphs own directory. >> >> Can you give me a very simple java program(finding maximum in a graph or >> simple page rank program) in giraph along with its jar file and input files >> which I can place in my lib directory of hadoop and test if its working. >> And also the command to execute it. This should be added in the >> documentation as new comers can quickly setup giraph and concentrate on >> their project. >> >> >> On Sat, Aug 24, 2013 at 7:12 PM, Ahmet Emre Aladağ < >> emre.ala...@agmlab.com> wrote: >> >>> It isn't asking for edge input. It says make sure you don't need it. A >>> warning for the case you may have forgotten to give edge input when you >>> really needed. >>> >>> The cause of your error is what I'm wondering nowadays. I'm having a >>> similar problem. Currently I'm using a workaround: put all the jars >>> (giraph-core and my module giraph-nutch) in the lib folder of hadoop. Then >>> it works. But there should be a clean way of doing this. >>> >>> I should be able to say hadoop jar fat.jar ... >>> >>> Any help appreciated. >>> >>> >>> >>> >>> >>> ------------------------------ >>> *Kimden: *"Vivek Sembium" <vivek.semb...@gmail.com> >>> *Kime: *user@giraph.apache.org >>> *Gönderilenler: *24 Ağustos Cumartesi 2013 11:51:49 >>> *Konu: *Re: Help needed for Running my own java programs in Giraph >>> >>> >>> I tried with and without exporting hadoop classpath. I get the same >>> error. >>> >>> Here's the command that I tried >>> hadoop jar >>> /mnt/a1/sda4/hadoop/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar >>> org.apache.giraph.GiraphRunner -libjars >>> /mnt/a99/d0/vivek/workspace/Giraph/bin/SimplePageRankComputation.jar >>> practice.SimplePageRankComputation\$SimplePageRankMasterCompute -vif >>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>> -vip /user/vivek/tiny_graph.txt -w 3 >>> >>> >>> 13/08/24 14:21:00 INFO utils.ConfigurationUtils: No edge input format >>> specified. Ensure your InputFormat does not require one. >>> 13/08/24 14:21:00 INFO utils.ConfigurationUtils: No output format >>> specified. Ensure your OutputFormat does not require one. >>> Exception in thread "main" java.lang.NoClassDefFoundError: >>> org/apache/giraph/master/DefaultMasterCompute >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:634) >>> at >>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:277) >>> at java.net.URLClassLoader.access$000(URLClassLoader.java:73) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:212) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:321) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:314) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:266) >>> at java.lang.Class.forName0(Native Method) >>> at java.lang.Class.forName(Class.java:186) >>> at >>> org.apache.giraph.utils.ConfigurationUtils.handleComputationClass(ConfigurationUtils.java:425) >>> at >>> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:408) >>> at >>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:200) >>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:616) >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >>> Caused by: java.lang.ClassNotFoundException: >>> org.apache.giraph.master.DefaultMasterCompute >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:217) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:321) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:266) >>> ... 26 more >>> >>> >>> I gave the graph input as JSON file. Why does it ask for edge input file >>> again? >>> >>> >>> Why isnt this working? >>> >>> >>> On Sat, Aug 24, 2013 at 12:35 AM, Kyle Orlando <kyle.r.orla...@gmail.com >>> > wrote: >>> >>>> Hi Vivek, >>>> >>>> Sorry for the hasty response, I assumed the quick start guide covered >>>> that (I guess it doesn't). What you need to do first is put all of your >>>> compiled code into a jar file. Then, you'll be utilizing the hadoop option >>>> -libjars. You also need to make sure that you set the hadoop classpath to >>>> include the giraph jar you are using as well as your own jar. I actually >>>> asked a similar question not too long ago; prior to this, I was just >>>> combining everything into one jar. >>>> >>>> >>>> Your code should pretty much follow this format, which is copied and >>>> paster from a previous response by Claudio: >>>> >>>> export HADOOP_CLASSPATH="giraph.jar:your.jar" >>>> hadoop jar giraph.jar org.apache.giraph.GiraphRunner [ -D option ]* >>>> -libjars your.jar your.package.ComputationClass [ GiraphRunner option >>>> e.g. -vip -vif etc. ]* >>>> >>>> >>>> On Fri, Aug 23, 2013 at 2:51 PM, Vivek Sembium <vivek.semb...@gmail.com >>>> > wrote: >>>> >>>>> Thank You for your response. I could run all the example programs. I >>>>> want to place my development code in a different directory other than the >>>>> giraph directory. I copied the SimplePageRankComputation.java to a >>>>> different directory, compiled it(with a different package name). But I >>>>> couldnt replace the >>>>> "org.apache.giraph.examples.SimpleShortestPathsComputation" >>>>> with my class file. It gives me a ClassNotFoundException. Please Guide me >>>>> so that I can run any class file with all required methods (GIven the >>>>> location of that file). >>>>> >>>>> >>>>> On Sat, Aug 24, 2013 at 12:07 AM, Kyle Orlando < >>>>> kyle.r.orla...@gmail.com> wrote: >>>>> >>>>>> A "Quick Start" guide was recently added to the Giraph website. Now >>>>>> learning Giraph is easier than ever! Here: >>>>>> http://giraph.apache.org/quick_start.html >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Aug 23, 2013 at 2:34 PM, Vivek Sembium < >>>>>> vivek.semb...@gmail.com> wrote: >>>>>> >>>>>>> I am a new user in giraph. I could run all example programs in >>>>>>> Giraph. Can someone please guide me on how to run my own java program in >>>>>>> giraph (Assuming I have my input file in jason format in HDFS). >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Kyle Orlando >>>>>> Computer Engineering Major >>>>>> University of Maryland >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Kyle Orlando >>>> Computer Engineering Major >>>> University of Maryland >>>> >>> >>> >>> >> > > > -- > Claudio Martella > claudio.marte...@gmail.com >
