Hi again, Roberto! I used to build it with this line: mvn package -Phadoop_yarn -Dhadoop.version=2.4.0 -DskipTests.
I don't remember the exact solution that i took, but you have two options as far as i know: - Make that method public, by editing those clases manually - Remove the STATIC_SASL_SYMBOL munge symbol. To do so, change the following line 1197 of the pom.xml in the giraph parent directory from <munge.symbols>PURE_YARN,STATIC_SASL_SYMBOL </munge.symbols> Hope that some of those solutions are helpfull. Bye! Jose 2015-11-09 7:48 GMT-03:00 Roberto Gonzalez <[email protected]>: > The issue is that I use Hadoop 2.4.1, I don't think I can upgrade it > easily and the compilation with hadoop_2 fails: > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-compiler-plugin:3.0:compile > (default-compile) on project giraph-core: Compilation failure: > Compilation failure: > [ERROR] > /usr/local/giraph-1.1.0/giraph-core/src/main/java/org/apache/giraph/comm/netty/SaslNettyServer.java:[113,32] > getDefaultProperties() has protected access in > org.apache.hadoop.security.SaslPropertiesResolver > [ERROR] > /usr/local/giraph-1.1.0/giraph-core/src/main/java/org/apache/giraph/comm/netty/SaslNettyClient.java:[93,32] > getDefaultProperties() has protected access in > org.apache.hadoop.security.SaslPropertiesResolver > > > So, with mapreduce I cannot compile it and with yarn I cannot use it. > > Thanks, > Roberto > > > On Sat, 2015-11-07 at 15:49 -0800, Roman Shaposhnik wrote: > > Why do you want to run this as YARN app? Why not use > > good old MapReduce? > > > > Thanks, > > Roman. > > > > On Thu, Nov 5, 2015 at 5:54 AM, Roberto Gonzalez > > <[email protected]> wrote: > > > Hi all again, > > > > > > After compiling the version 1.1 I found the following bug: > > > > > > https://issues.apache.org/jira/browse/GIRAPH-859 > > > > > > I applied the patch and disable the permissions in the HDFS (I would > > > want not to do that... but I can accept it). > > > > > > but still executing the example as: > > > > > > > > > hadoop jar giraph-ex.jar org.apache.giraph.GiraphRunner > > > org.apache.giraph.examples.SimpleShortestPathsComputation -vif > > > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat > > > -vip tiny_graph.txt -vof > > > org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op > > > shortestpaths -yj giraph-ex.jar -w 1 > > > > > > > > > The program runs for about 10 minutes (the example graph has 5 nodes) > > > before failing. > > > > > > the gam-stderr.log file only contains info about SLF4J, and the > > > gam-stdout.log finish with: > > > > > > Container exited with a non-zero exit code 143 > > > > > > 2015-11-05 14:25:39,340 INFO [AMRM Callback Handler Thread] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:onContainersCompleted(605)) - After > completion of one conatiner. current status is: completedCount :1 > containersToLaunch :2 successfulCount :0 failedCount :1 > > > 2015-11-05 14:26:13,414 INFO [AMRM Callback Handler Thread] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:onContainersCompleted(580)) - Got response > from RM for container ask, completedCnt=1 > > > 2015-11-05 14:26:13,414 INFO [AMRM Callback Handler Thread] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:onContainersCompleted(583)) - Got container > status for containerID=container_1446634690791_0024_01_000003, > state=COMPLETE, exitStatus=2, diagnostics=Exception from container-launch: > org.apache.hadoop.util.Shell$ExitCodeException: > > > org.apache.hadoop.util.Shell$ExitCodeException: > > > at org.apache.hadoop.util.Shell.runCommand(Shell.java:505) > > > at org.apache.hadoop.util.Shell.run(Shell.java:418) > > > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650) > > > at > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) > > > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300) > > > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > > at java.lang.Thread.run(Thread.java:745) > > > > > > > > > Container exited with a non-zero exit code 2 > > > > > > 2015-11-05 14:26:13,415 INFO [AMRM Callback Handler Thread] > yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:onContainersCompleted(603)) - All container > compeleted. done = true > > > 2015-11-05 14:26:13,543 INFO [main] yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:run(195)) - Done true > > > 2015-11-05 14:26:13,543 INFO [main] yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:run(207)) - Forcefully terminating executors > with done =:true > > > 2015-11-05 14:26:13,543 INFO [main] yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:finish(221)) - Application completed. > Stopping running containers > > > 2015-11-05 14:26:13,578 INFO [main] > impl.ContainerManagementProtocolProxy > (ContainerManagementProtocolProxy.java:mayBeCloseProxy(145)) - Closing > proxy : computer62:59272 > > > 2015-11-05 14:26:13,579 INFO [main] > impl.ContainerManagementProtocolProxy > (ContainerManagementProtocolProxy.java:mayBeCloseProxy(145)) - Closing > proxy : computer66:45051 > > > 2015-11-05 14:26:13,579 INFO [main] yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:finish(226)) - Application completed. > Signalling finish to RM > > > 2015-11-05 14:26:13,586 INFO [main] impl.AMRMClientImpl > (AMRMClientImpl.java:unregisterApplicationMaster(321)) - Waiting for > application to be successfully unregistered. > > > 2015-11-05 14:26:13,688 INFO [main] yarn.GiraphApplicationMaster > (GiraphApplicationMaster.java:main(454)) - Giraph Application Master > failed. exiting > > > 2015-11-05 14:26:13,688 INFO [AMRM Callback Handler Thread] > impl.AMRMClientAsyncImpl (AMRMClientAsyncImpl.java:run(277)) - Interrupted > while waiting for queue > > > java.lang.InterruptedException > > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) > > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) > > > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > > > at > org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:275) > > > > > > > > > Moreover, even when the exception is 1 minute after the program starts, > > > it last more than 10 minutes to finish. > > > > > > Do you have any idea?? > > > > > > Thanks. > > > > > > > > > > > > > > > -- > > > Dr. Roberto Gonzalez > > > Research Scientist, Networked Systems and Data Analytics Group > > > NEC Europe Ltd. > > > NEC Laboratories Europe > > > Kurfürsten-Anlage 36 > > > > > > D-69115 Heidelberg > > > > > > phone +49 6221 4342 256 > > > fax +49 6221 4342 155 > > > e-mail: [email protected] > > > > > > NEC Europe Ltd | Registered Office: Athene, Odyssey Business Park, > West End Road, > > > London, HA4 6QE, GB | Registered in England 2832014 > > -- > Dr. Roberto Gonzalez > Research Scientist, Networked Systems and Data Analytics Group > NEC Europe Ltd. > NEC Laboratories Europe > Kurfürsten-Anlage 36 > > D-69115 Heidelberg > > phone +49 6221 4342 256 > fax +49 6221 4342 155 > e-mail: [email protected] > > NEC Europe Ltd | Registered Office: Athene, Odyssey Business Park, West > End Road, > London, HA4 6QE, GB | Registered in England 2832014 >
