Hi Eli, ok, thanks for you suggestion. I'll try it and let you know. In order to be sure, before try to use an earlier version of yarn/hadoop2, I'm using giraph-dist-1.2.0-hadoop2-bin.tar.gz. Does this giraph version officially support hadoop-2.5.1 with yarn? If not how I have to configure hadoop-2.5.1? Thanks
2018-07-29 9:47 GMT+02:00 Eli Reisman <apache.mail...@gmail.com>: > hey francesco sorry im on my phone and have not been running giraph on > yarn for some time but if i had to guess this looks like a version > compatibility issue with the version of yarn the giraph is built against > compared to what you ran on ec2 or perhaps the version of protobuf dep in > the giraph build and the yarn cluster? either way id try an earlier version > of yarn/hadoop2 for ec2 and your giraph build, as giraph hasnt been keeping > up with all the yarn side changes. > > alternate theory is i have seen startup argument and config errors show up > with ambiguous traces similar to that. not able to review your conf file > from phone but you might review configs and args against giraph and yarn > code directly to ensure all those are good since docs might be outdated on > some of those. > > Good luck! > > On Sat, Jul 28, 2018, 9:42 PM Francesco Sclano <sclano.france...@gmail.com> > wrote: > >> Hi, >> I'm using giraph-1.2 for my master thesis in computer science. I >> developed in giraph the calculus of 4-profiles like eelenberg.github.io/ >> Elenberg4profileWWW16.pdf >> I succesfully configured and runned giraph-1.2 and hadoop-2.5.1 in pseudo >> distributed mode on my local pc with this configuration >> <https://we.tl/dB0PmGhUPX>. >> >> In pseudo distributed mode I launch my giraph program with following >> parameters: >> giraph MY_JAR.jar SUPERSTEP0_CLASS -ca >> giraph.master.observers=MY_OBSERVER_CLASS >> -mc MY_MASTER_CLASS -eif MY_CUSTOM_INPUT_FORMAT -eip input.txt -vof >> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op output -w 1 >> -ca giraph.SplitMasterWorker=false -ca io.edge.reverse.duplicator=true >> >> Then I tryed to use amazon ec2 with a simple cluster of 3 nodes with this >> other >> configuration <https://we.tl/Gbv9q0nQ7h> but I obtain the error below. I >> also re-tryed pseudo-distributed mode in amazon ec2 with only 1 node and I >> obtain the same error below, but on my local pc it works!!! >> I'm using ubuntu 16.04 both in local and in ec2. >> >> Obviously I tried an hadoop example on my ec2 cluster and it works. >> Please help me because I'm blocked with my master this for this error! I >> red many forums about this error and I checked many times that I have only >> protobuf-java-2.5.0.jar inside hadoop-2.5.2 and giraph-1.2 both in ec2 and >> in my local pc. >> >> >> >> FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error >> starting MRAppMaster >> java.lang.VerifyError: class >> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto >> overrides final method getUnknownFields.()Lcom/google/protobuf/ >> UnknownFieldSet; >> at java.lang.ClassLoader.defineClass1(Native Method) >> at java.lang.ClassLoader.defineClass(ClassLoader.java:763) >> at java.security.SecureClassLoader.defineClass( >> SecureClassLoader.java:142) >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) >> at java.net.URLClassLoader.access$100(URLClassLoader.java:73) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:368) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:362) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:361) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> at java.lang.Class.getDeclaredConstructors0(Native Method) >> at java.lang.Class.privateGetDeclaredConstructors >> (Class.java:2671) >> at java.lang.Class.getConstructor0(Class.java:3075) >> at java.lang.Class.getConstructor(Class.java:1825) >> at org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl. >> newRecordInstance(RecordFactoryPBImpl.java:62) >> at org.apache.hadoop.yarn.util.Records.newRecord(Records.java:36) >> at org.apache.hadoop.yarn.api.records.ApplicationId. >> newInstance(ApplicationId.java:49) >> at org.apache.hadoop.yarn.util.ConverterUtils. >> toApplicationAttemptId(ConverterUtils.java:137) >> at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId( >> ConverterUtils.java:177) >> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main( >> MRAppMaster.java:1391) >> >> INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1 >> >> >> Links of configurafion files works until 4 August 2018 but I can re-send >> them. >> >> Many Thanks >> >> Best Regards >> >> -- >> Francesco Sclano >> > -- Francesco Sclano
