hey francesco sorry im on my phone and have not been running giraph on yarn for some time but if i had to guess this looks like a version compatibility issue with the version of yarn the giraph is built against compared to what you ran on ec2 or perhaps the version of protobuf dep in the giraph build and the yarn cluster? either way id try an earlier version of yarn/hadoop2 for ec2 and your giraph build, as giraph hasnt been keeping up with all the yarn side changes.
alternate theory is i have seen startup argument and config errors show up with ambiguous traces similar to that. not able to review your conf file from phone but you might review configs and args against giraph and yarn code directly to ensure all those are good since docs might be outdated on some of those. Good luck! On Sat, Jul 28, 2018, 9:42 PM Francesco Sclano <sclano.france...@gmail.com> wrote: > Hi, > I'm using giraph-1.2 for my master thesis in computer science. I developed > in giraph the calculus of 4-profiles like > eelenberg.github.io/Elenberg4profileWWW16.pdf > I succesfully configured and runned giraph-1.2 and hadoop-2.5.1 in pseudo > distributed mode on my local pc with this configuration > <https://we.tl/dB0PmGhUPX>. > > In pseudo distributed mode I launch my giraph program with following > parameters: > giraph MY_JAR.jar SUPERSTEP0_CLASS -ca > giraph.master.observers=MY_OBSERVER_CLASS -mc MY_MASTER_CLASS -eif > MY_CUSTOM_INPUT_FORMAT -eip input.txt -vof > org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op output -w 1 > -ca giraph.SplitMasterWorker=false -ca io.edge.reverse.duplicator=true > > Then I tryed to use amazon ec2 with a simple cluster of 3 nodes with this > other > configuration <https://we.tl/Gbv9q0nQ7h> but I obtain the error below. I > also re-tryed pseudo-distributed mode in amazon ec2 with only 1 node and I > obtain the same error below, but on my local pc it works!!! > I'm using ubuntu 16.04 both in local and in ec2. > > Obviously I tried an hadoop example on my ec2 cluster and it works. > Please help me because I'm blocked with my master this for this error! I > red many forums about this error and I checked many times that I have only > protobuf-java-2.5.0.jar inside hadoop-2.5.2 and giraph-1.2 both in ec2 and > in my local pc. > > > > FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error > starting MRAppMaster > java.lang.VerifyError: class > org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final > method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet; > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:763) > at > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) > at java.net.URLClassLoader.access$100(URLClassLoader.java:73) > at java.net.URLClassLoader$1.run(URLClassLoader.java:368) > at java.net.URLClassLoader$1.run(URLClassLoader.java:362) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:361) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.getDeclaredConstructors0(Native Method) > at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) > at java.lang.Class.getConstructor0(Class.java:3075) > at java.lang.Class.getConstructor(Class.java:1825) > at > org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl.newRecordInstance(RecordFactoryPBImpl.java:62) > at org.apache.hadoop.yarn.util.Records.newRecord(Records.java:36) > at > org.apache.hadoop.yarn.api.records.ApplicationId.newInstance(ApplicationId.java:49) > at > org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) > at > org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1391) > > INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1 > > > Links of configurafion files works until 4 August 2018 but I can re-send > them. > > Many Thanks > > Best Regards > > -- > Francesco Sclano >
