I got the build working just by following above steps I mentioned. May be best option is to try giraph version which uses map-reduce API instead of pure YARN. Also there aren't any scripts which can use to submit jobs to YARN cluster (for pure YARN based implementation). So going with giraph version which is based on map-reduce API will be easy for you.
Thanks Milinda On Sun, Oct 6, 2013 at 10:04 PM, Matthew Laird <lai...@sfu.ca> wrote: > Hmm, is there a recommendation on a particular version which might be good > for building/testing against Hadoop 2.0.5? I realize I'm playing with the > bleeding edge here... > > I threw a try/catch around public static void printHaltInfo() just like > private static boolean waitForHaltInfo() has. However next I had issues > with the tests, I'm guessing because I'm running the hadoop head node on > this machine, it was a socket bind error, so I threw in a -DskipTests. > > Next I ran in to a org.apache.rat issue, I added an exclude for *.txt, > *.xml, and *.py. Unfortunately next was: > > [INFO] --- findbugs-maven-plugin:2.5.1:check (default) @ giraph-core --- > [INFO] BugInstance size is 2 > [INFO] Error size is 0 > [INFO] Total bugs: 2 > [INFO] > org.apache.giraph.yarn.GiraphYarnTask$OverrideExceptionHandler.uncaughtException(Thread, > Throwable) invokes System.exit(...), which shuts down the entire virtual > machine ["org.apache.giraph.yarn.GiraphYarnTask$OverrideExceptionHandler"] > At GiraphYarnTask.java:[lines 171-178] > [INFO] exceptional return value of java.io.File.delete() ignored in > org.apache.giraph.yarn.YarnUtils.exportGiraphConfiguration(GiraphConfiguration, > ApplicationId) ["org.apache.giraph.yarn.YarnUtils"] At YarnUtils.java:[lines > 47-240] > > I guess this is what I get for playing with the trunk. :) But, what version > was it that is mentioned in the previous mailing list threads that you can > apply the patch to and get it compiled? > > Thanks! > > > On 06/10/13 18:19, Matthew Laird wrote: >> >> Thanks for the reply, so I just grabbed that I assume is the latest >> version, the snapshot link from git: >> >> >> https://git-wip-us.apache.org/repos/asf?p=giraph.git;a=tree;h=refs/heads/trunk;hb=trunk >> >> And ran the command: >> >> mvn -Dhadoop.version=2.0.5-alpha -Phadoop_yarn clean install >> >> The error I receive in this situation is: >> >> [INFO] 2 errors >> [INFO] ------------------------------------------------------------- >> [INFO] >> ------------------------------------------------------------------------ >> [INFO] Reactor Summary: >> [INFO] >> [INFO] Apache Giraph Parent .............................. SUCCESS >> [1.593s] >> [INFO] Apache Giraph Core ................................ FAILURE >> [23.522s] >> [INFO] Apache Giraph Examples ............................ SKIPPED >> [INFO] >> ------------------------------------------------------------------------ >> [INFO] BUILD FAILURE >> [INFO] >> ------------------------------------------------------------------------ >> [INFO] Total time: 25.574s >> [INFO] Finished at: Sun Oct 06 18:17:21 PDT 2013 >> [INFO] Final Memory: 37M/692M >> [INFO] >> ------------------------------------------------------------------------ >> [ERROR] Failed to execute goal >> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) >> on project giraph-core: Compilation failure: Compilation failure: >> [ERROR] >> /opt/giraph/giraph-core/target/munged/main/org/apache/giraph/job/HaltApplicationUtils.java:[78,49] >> unreported exception java.lang.InterruptedException; must be caught or >> declared to be thrown >> [ERROR] >> /opt/giraph/giraph-core/target/munged/main/org/apache/giraph/job/HaltApplicationUtils.java:[81,50] >> unreported exception java.lang.InterruptedException; must be caught or >> declared to be thrown >> >> That looks like a bug in the code? >> >> Thanks. >> >> On 06/10/13 18:13, Milinda Pathirage wrote: >>> >>> AFAIK, you need to specify hadoop.version system property. So command >>> to build giraph with YARN (2.0.5) should be something like following. >>> >>> mvn -Phadoop_yarn -Dhadoop.version=2.0.5-alpha clean install >>> >>> Hope this helps. >>> >>> Thanks >>> Milinda >>> >>> >>> On Sun, Oct 6, 2013 at 9:04 PM, Matthew Laird <lai...@sfu.ca> wrote: >>>> >>>> Further, I let it go and after two hours I see the file: >>>> >>>> >>>> /giraph-core/target/munged/main/org/apache/giraph/yarn/GiraphYarnTask.java.rej >>>> >>>> is now 295GB in size. I'm not sure what it's stuck doing after applying >>>> this patch... >>>> >>>> Thanks. >>>> >>>> >>>> On 06/10/13 16:13, Matthew Laird wrote: >>>>> >>>>> Hello, >>>>> >>>>> I'm trying to build giraph against Hadoop 2.0.5, and I've been browsing >>>>> the mailing list archives but am still having an issue. I found the >>>>> previous >>>>> thread where Eli points to >>>>> >>>>> https://issues.apache.org/jira/browse/GIRAPH-688 >>>>> >>>>> I've grabbed the patch and applied it against a clone from git >>>>> (http://git-wip-us.apache.org/repos/asf/giraph.git, is that the correct >>>>> place to get the latest trunk?) Just like the other fellow asking a >>>>> few >>>>> months ago, patch complains the patch has been applied before. >>>>> >>>>> Once applied I run the suggested build command: >>>>> >>>>> mvn -U -Phadoop_yarn clean install >>>>> >>>>> It reaches >>>>> >>>>> [INFO] --- munge-maven-plugin:1.0:munge (munge) @ giraph-core --- >>>>> >>>>> And so far it's been sitting there about 20 minutes at this step, in >>>>> another window I can see it's working hard doing something, but I'm >>>>> assuming >>>>> it's stuck doing... I'm not sure what. >>>>> >>>>> Without the patch I receive the same error the other fellow was having >>>>> about dependancies for hadoop not being satisfied. >>>>> >>>>> Any advice on what might be going on, or is there a better place to get >>>>> a >>>>> snapshot that has the needed patches applied? (I see a lot of commits >>>>> since >>>>> the original thread) >>>>> >>>>> Thanks! >>>>> >>> >>> >> > -- Milinda Pathirage twitter: milindalakmal skype: milinda.pathirage blog: http://milinda.pathirage.org
