there must be a misunderstanding. i was referring to the zookeeper problem.
On Mon, Aug 26, 2013 at 11:14 AM, Vivek Sembium <vivek.semb...@gmail.com>wrote: > No. I added my files(it was just a copy of one of the example program to a > different package) to the jar files of giraph. But it was still giving me > classNotFoundException. Can you give me some simple example program with > instructions on how to deploy it. So I can start playing with giraph and > make changes to the program and learn, then start working on my project in > giraph. > I will be very thankful if you can help me with this. > > Thanking you > -Vivek Sembium > > > On Mon, Aug 26, 2013 at 2:37 PM, Claudio Martella < > claudio.marte...@gmail.com> wrote: > >> but you were still using an additional jar added through -libjars, right? >> >> >> On Mon, Aug 26, 2013 at 8:43 AM, Vivek Sembium >> <vivek.semb...@gmail.com>wrote: >> >>> @Claudio Martella Your solution didnt work either. I basically tried >>> copying the pageRankBenchmark to my own package, renamed the package. It >>> compiles fine with giraph. But I couldnt run it even if I add those files >>> to giraph jar before deployment. Help? >>> >>> >>> On Sun, Aug 25, 2013 at 6:33 PM, Claudio Martella < >>> claudio.marte...@gmail.com> wrote: >>> >>>> you have this problem when you use two jars (one with giraph and one >>>> with your classes) instead of a single fat-jar, correct? I tracked the same >>>> problem a few weeks ago, basically zookeeper is run passing the wrong jar. >>>> >>>> >>>> On Sat, Aug 24, 2013 at 4:51 PM, Vivek Sembium <vivek.semb...@gmail.com >>>> > wrote: >>>> >>>>> Thank you for your suggestion. It worked. Its not giving class not >>>>> found exception. But its giving me a new error >>>>> Its stopping at map 0% and reduce 0%. Upon inspection I found that its >>>>> unable to connect to zookeeper service. >>>>> >>>>> java.lang.IllegalStateException: run: Caught an unrecoverable >>>>> exception onlineZooKeeperServers: Failed to connect in 10 tries! >>>>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101) >>>>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) >>>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) >>>>> at org.apache.hadoop.mapred.Child$4.run(Child.java:255) >>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>> at javax.security.auth.Subject.doAs(Subject.java:416) >>>>> at >>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) >>>>> at org.apache.hadoop.mapred.Child.main(Child.java:249) >>>>> Caused by: java.lang.IllegalStateException: onlineZooKeeperServers: >>>>> Failed to connect in 10 tries! >>>>> at >>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:727) >>>>> at >>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:371) >>>>> at >>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:204) >>>>> at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59) >>>>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89) >>>>> ... 7 more >>>>> >>>>> Immediately I ran page rank benchmark and it executed successfully >>>>> both from giraph in lib directory and also from giraphs own directory. >>>>> >>>>> Can you give me a very simple java program(finding maximum in a graph >>>>> or simple page rank program) in giraph along with its jar file and input >>>>> files which I can place in my lib directory of hadoop and test if its >>>>> working. And also the command to execute it. This should be added in the >>>>> documentation as new comers can quickly setup giraph and concentrate on >>>>> their project. >>>>> >>>>> >>>>> On Sat, Aug 24, 2013 at 7:12 PM, Ahmet Emre Aladağ < >>>>> emre.ala...@agmlab.com> wrote: >>>>> >>>>>> It isn't asking for edge input. It says make sure you don't need it. >>>>>> A warning for the case you may have forgotten to give edge input when you >>>>>> really needed. >>>>>> >>>>>> The cause of your error is what I'm wondering nowadays. I'm having a >>>>>> similar problem. Currently I'm using a workaround: put all the jars >>>>>> (giraph-core and my module giraph-nutch) in the lib folder of hadoop. >>>>>> Then >>>>>> it works. But there should be a clean way of doing this. >>>>>> >>>>>> I should be able to say hadoop jar fat.jar ... >>>>>> >>>>>> Any help appreciated. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------ >>>>>> *Kimden: *"Vivek Sembium" <vivek.semb...@gmail.com> >>>>>> *Kime: *user@giraph.apache.org >>>>>> *Gönderilenler: *24 Ağustos Cumartesi 2013 11:51:49 >>>>>> *Konu: *Re: Help needed for Running my own java programs in Giraph >>>>>> >>>>>> >>>>>> I tried with and without exporting hadoop classpath. I get the same >>>>>> error. >>>>>> >>>>>> Here's the command that I tried >>>>>> hadoop jar >>>>>> /mnt/a1/sda4/hadoop/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar >>>>>> org.apache.giraph.GiraphRunner -libjars >>>>>> /mnt/a99/d0/vivek/workspace/Giraph/bin/SimplePageRankComputation.jar >>>>>> practice.SimplePageRankComputation\$SimplePageRankMasterCompute -vif >>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>> -vip /user/vivek/tiny_graph.txt -w 3 >>>>>> >>>>>> >>>>>> 13/08/24 14:21:00 INFO utils.ConfigurationUtils: No edge input format >>>>>> specified. Ensure your InputFormat does not require one. >>>>>> 13/08/24 14:21:00 INFO utils.ConfigurationUtils: No output format >>>>>> specified. Ensure your OutputFormat does not require one. >>>>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>>>> org/apache/giraph/master/DefaultMasterCompute >>>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:634) >>>>>> at >>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>>>>> at >>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:277) >>>>>> at java.net.URLClassLoader.access$000(URLClassLoader.java:73) >>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:212) >>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:321) >>>>>> at >>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:314) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:266) >>>>>> at java.lang.Class.forName0(Native Method) >>>>>> at java.lang.Class.forName(Class.java:186) >>>>>> at >>>>>> org.apache.giraph.utils.ConfigurationUtils.handleComputationClass(ConfigurationUtils.java:425) >>>>>> at >>>>>> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:408) >>>>>> at >>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:200) >>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74) >>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>> at >>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>>>> at >>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>> at java.lang.reflect.Method.invoke(Method.java:616) >>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >>>>>> Caused by: java.lang.ClassNotFoundException: >>>>>> org.apache.giraph.master.DefaultMasterCompute >>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:217) >>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:321) >>>>>> at >>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:266) >>>>>> ... 26 more >>>>>> >>>>>> >>>>>> I gave the graph input as JSON file. Why does it ask for edge input >>>>>> file again? >>>>>> >>>>>> >>>>>> Why isnt this working? >>>>>> >>>>>> >>>>>> On Sat, Aug 24, 2013 at 12:35 AM, Kyle Orlando < >>>>>> kyle.r.orla...@gmail.com> wrote: >>>>>> >>>>>>> Hi Vivek, >>>>>>> >>>>>>> Sorry for the hasty response, I assumed the quick start guide >>>>>>> covered that (I guess it doesn't). What you need to do first is put all >>>>>>> of >>>>>>> your compiled code into a jar file. Then, you'll be utilizing the >>>>>>> hadoop >>>>>>> option -libjars. You also need to make sure that you set the hadoop >>>>>>> classpath to include the giraph jar you are using as well as your own >>>>>>> jar. >>>>>>> I actually asked a similar question not too long ago; prior to this, I >>>>>>> was >>>>>>> just combining everything into one jar. >>>>>>> >>>>>>> >>>>>>> Your code should pretty much follow this format, which is copied and >>>>>>> paster from a previous response by Claudio: >>>>>>> >>>>>>> export HADOOP_CLASSPATH="giraph.jar:your.jar" >>>>>>> hadoop jar giraph.jar org.apache.giraph.GiraphRunner [ -D option ]* >>>>>>> -libjars your.jar your.package.ComputationClass [ GiraphRunner >>>>>>> option e.g. -vip -vif etc. ]* >>>>>>> >>>>>>> >>>>>>> On Fri, Aug 23, 2013 at 2:51 PM, Vivek Sembium < >>>>>>> vivek.semb...@gmail.com> wrote: >>>>>>> >>>>>>>> Thank You for your response. I could run all the example programs. >>>>>>>> I want to place my development code in a different directory other >>>>>>>> than the >>>>>>>> giraph directory. I copied the SimplePageRankComputation.java to a >>>>>>>> different directory, compiled it(with a different package name). But I >>>>>>>> couldnt replace the >>>>>>>> "org.apache.giraph.examples.SimpleShortestPathsComputation" >>>>>>>> with my class file. It gives me a ClassNotFoundException. Please Guide >>>>>>>> me >>>>>>>> so that I can run any class file with all required methods (GIven the >>>>>>>> location of that file). >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Aug 24, 2013 at 12:07 AM, Kyle Orlando < >>>>>>>> kyle.r.orla...@gmail.com> wrote: >>>>>>>> >>>>>>>>> A "Quick Start" guide was recently added to the Giraph website. >>>>>>>>> Now learning Giraph is easier than ever! Here: >>>>>>>>> http://giraph.apache.org/quick_start.html >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Aug 23, 2013 at 2:34 PM, Vivek Sembium < >>>>>>>>> vivek.semb...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> I am a new user in giraph. I could run all example programs in >>>>>>>>>> Giraph. Can someone please guide me on how to run my own java >>>>>>>>>> program in >>>>>>>>>> giraph (Assuming I have my input file in jason format in HDFS). >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Kyle Orlando >>>>>>>>> Computer Engineering Major >>>>>>>>> University of Maryland >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Kyle Orlando >>>>>>> Computer Engineering Major >>>>>>> University of Maryland >>>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Claudio Martella >>>> claudio.marte...@gmail.com >>>> >>> >>> >> >> >> -- >> Claudio Martella >> claudio.marte...@gmail.com >> > > -- Claudio Martella claudio.marte...@gmail.com
