Giraph on YARN thus far doesn't break any compatibility with the MapReduce version. When I was working on it more actively, it had a slightly faster job startup but otherwise behaved similarly to the MapReduce version.
There are a number of things design wise that could make the YARN profile substantially better (in theory) but would require a fork or bigger design changes/agreements about the MapReduce profiles. This would include things like spawning the Master Giraph task in the Application Master itself, and many other things along those lines. There are also a number of smaller things that would probably make a difference like exposing YARN's per-task resource configuration features in a more flexible way. I haven't had much time to hack on Giraph this past year, and at some point last summer some folks like Muhammad Islam from LinkedIn did some great work to update the YARN profile to run on Hadoop 2.2.0 or newer versions but since then it hasn't gotten much love. I noticed there is still a note in the master POM from the original Giraph on YARN implementation that says its compatible only with Hadoop 2.0.3-alpha. I thought that was removed with Mohammad's Hadoop 2.2.0 patches but apparently it wasn't. We should remove that, it's no longer accurate and seems to be misleading people trying to build the YARN profile. On Fri, Oct 10, 2014 at 11:15 AM, Tripti Singh <tri...@yahoo-inc.com> wrote: > > Hi Matthew, > I would have been thrilled to give you numbers on this one but for me the > Application is not scaling without the out-of-core option( which isn't > working the way it was in previous version) > I'm still figuring it out and can get back once it's resolved. I have > patched a few things and will share them for people who might face similar > issue. If u have a fix for scalability, do let me know > > Thanks, > Tripti > > Sent from my iPhone > > > On 06-Oct-2014, at 9:22 pm, "Matthew Cornell" <m...@matthewcornell.org> > wrote: > > > > Hi Folks. I don't think I paid enough attention to YARN vs. MR1 when I > > built Giraph 1.0.0 for our system. How much better is Giraph on YARN? > > Thank you. > > > > -- > > Matthew Cornell | m...@matthewcornell.org >
