Re: YARN vs. MR1: is YARN a good idea ? Out-of-Core Graph ?
Hey Olivier, I got onto some other stuff and didn¹t get time to try out anything recently. I was able to briefly resolve the out-of-memory by increasing the frame decoder¹s frame length from 1GB to Integer.MAX_VALUE (search for LengthFieldBasedFrameDecoder in NettyServer.java and change the first parameter from 1024*1024*1024 to Integer.MAX_VALUE, in all the constructor calls). I did this becoz I was getting the following stack trace: exceptionCaught: Channel failed with remote address /10.216.185.26:52706 io.netty.handler.codec.TooLongFrameException: Adjusted frame length exceeds 1073741824: 1647797607 - discarded at io.netty.handler.codec.LengthFieldBasedFrameDecoder.fail(LengthFieldBasedFr ameDecoder.java:501) at io.netty.handler.codec.LengthFieldBasedFrameDecoder.failIfNecessary(LengthF ieldBasedFrameDecoder.java:477) at io.netty.handler.codec.LengthFieldBasedFrameDecoder.decode(LengthFieldBased FrameDecoder.java:403) at io.netty.handler.codec.LengthFieldBasedFrameDecoder.decode(LengthFieldBased FrameDecoder.java:343) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder .java:226) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecode r.java:139) at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChan nelHandlerContext.java:338) at io.netty.channel.DefaultChannelHandlerContext.access$700(DefaultChannelHand lerContext.java:29) at io.netty.channel.DefaultChannelHandlerContext$8.run(DefaultChannelHandlerCo ntext.java:329) at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java :36) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventE xecutor.java:101) at java.lang.Thread.run(Thread.java:722) exceptionCaught: Channel failed with remote address /10.216.198.32:37621 java.io.EOFException: fieldSize is too long! Length is 876099893, but maximum is 20404 at io.netty.buffer.ByteBufInputStream.checkAvailable(ByteBufInputStream.java:2 53) at io.netty.buffer.ByteBufInputStream.readFully(ByteBufInputStream.java:174) at io.netty.buffer.ByteBufInputStream.readFully(ByteBufInputStream.java:169) at org.apache.giraph.utils.WritableUtils.readExtendedDataOutput(WritableUtils. java:379) at org.apache.giraph.utils.ByteArrayVertexIdData.readFields(ByteArrayVertexIdD ata.java:52) at org.apache.giraph.utils.ByteArrayVertexIdMessages.readFields(ByteArrayVerte xIdMessages.java:130) at org.apache.giraph.comm.requests.SendWorkerDataRequest.readFieldsRequest(Sen dWorkerDataRequest.java:86) at org.apache.giraph.comm.requests.WritableRequest.readFields(WritableRequest. java:118) at org.apache.giraph.utils.RequestUtils.decodeWritableRequest(RequestUtils.jav a:52) at org.apache.giraph.comm.netty.handler.RequestDecoder.channelRead(RequestDeco der.java:89) at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChan nelHandlerContext.java:338) at io.netty.channel.DefaultChannelHandlerContext.access$700(DefaultChannelHand lerContext.java:29) at io.netty.channel.DefaultChannelHandlerContext$8.run(DefaultChannelHandlerCo ntext.java:329) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThread EventExecutor.java:354) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:353) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventE xecutor.java:101) at java.lang.Thread.run(Thread.java:722) exceptionCaught: Channel failed with remote address /10.216.196.21:48709 java.lang.OutOfMemoryError: Java heap space at org.apache.giraph.utils.WritableUtils.readExtendedDataOutput(WritableUtils. java:378) at org.apache.giraph.utils.ByteArrayVertexIdData.readFields(ByteArrayVertexIdD ata.java:52) at org.apache.giraph.utils.ByteArrayVertexIdMessages.readFields(ByteArrayVerte xIdMessages.java:130) at org.apache.giraph.comm.requests.SendWorkerDataRequest.readFieldsRequest(Sen dWorkerDataRequest.java:86) at org.apache.giraph.comm.requests.WritableRequest.readFields(WritableRequest. java:118) at org.apache.giraph.utils.RequestUtils.decodeWritableRequest(RequestUtils.jav a:52) at org.apache.giraph.comm.netty.handler.RequestDecoder.channelRead(RequestDeco der.java:89) at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChan nelHandlerContext.java:338) at io.netty.channel.DefaultChannelHandlerContext.access$700(DefaultChannelHand lerContext.java:29) at io.netty.channel.DefaultChannelHandlerContext$8.run(DefaultChannelHandlerCo ntext.java:329) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThread EventExecutor.java:354) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.ja
Re: YARN vs. MR1: is YARN a good idea ? Out-of-Core Graph ?
Following up, does anyone have thoughts re: MR1 vs YARN performance? Thank you. -- matt On Wed, Oct 22, 2014 at 8:34 AM, wrote: > Hello Tripti, > > I bumped into this mail, I am experiencing out-of-memory errors on my small > cluster, > and, as out-of-core graph does not seem to work on giraph 1.1-SNAPSHOT, I was > wondering if you had any jira / patched already posted to help solve this > issue ? > > > Thanks a lot > > regards > > > > Olivier Varène > Big Data Referent > Orange - DSI France/Digital Factory > olivier.var...@orange.com > +33 4 97 46 29 94 > > > > > > > > Le 10 oct. 2014 à 20:15, Tripti Singh a écrit : > >> Hi Matthew, >> I would have been thrilled to give you numbers on this one but for me the >> Application is not scaling without the out-of-core option( which isn't >> working the way it was in previous version) >> I'm still figuring it out and can get back once it's resolved. I have >> patched a few things and will share them for people who might face similar >> issue. If u have a fix for scalability, do let me know >> >> Thanks, >> Tripti >> >> Sent from my iPhone >> >>> On 06-Oct-2014, at 9:22 pm, "Matthew Cornell" >>> wrote: >>> >>> Hi Folks. I don't think I paid enough attention to YARN vs. MR1 when I >>> built Giraph 1.0.0 for our system. How much better is Giraph on YARN? >>> Thank you. >>> >>> -- >>> Matthew Cornell | m...@matthewcornell.org > > > _ > > Ce message et ses pieces jointes peuvent contenir des informations > confidentielles ou privilegiees et ne doivent donc > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu > ce message par erreur, veuillez le signaler > a l'expediteur et le detruire ainsi que les pieces jointes. Les messages > electroniques etant susceptibles d'alteration, > Orange decline toute responsabilite si ce message a ete altere, deforme ou > falsifie. Merci. > > This message and its attachments may contain confidential or privileged > information that may be protected by law; > they should not be distributed, used or copied without authorisation. > If you have received this email in error, please notify the sender and delete > this message and its attachments. > As emails may be altered, Orange is not liable for messages that have been > modified, changed or falsified. > Thank you. > -- Matthew Cornell | m...@matthewcornell.org | 413-626-3621 | 34 Dickinson Street, Amherst MA 01002 | matthewcornell.org
Re: YARN vs. MR1: is YARN a good idea ? Out-of-Core Graph ?
Hello Tripti, I bumped into this mail, I am experiencing out-of-memory errors on my small cluster, and, as out-of-core graph does not seem to work on giraph 1.1-SNAPSHOT, I was wondering if you had any jira / patched already posted to help solve this issue ? Thanks a lot regards Olivier Varène Big Data Referent Orange - DSI France/Digital Factory olivier.var...@orange.com +33 4 97 46 29 94 Le 10 oct. 2014 à 20:15, Tripti Singh a écrit : > Hi Matthew, > I would have been thrilled to give you numbers on this one but for me the > Application is not scaling without the out-of-core option( which isn't > working the way it was in previous version) > I'm still figuring it out and can get back once it's resolved. I have patched > a few things and will share them for people who might face similar issue. If > u have a fix for scalability, do let me know > > Thanks, > Tripti > > Sent from my iPhone > >> On 06-Oct-2014, at 9:22 pm, "Matthew Cornell" >> wrote: >> >> Hi Folks. I don't think I paid enough attention to YARN vs. MR1 when I >> built Giraph 1.0.0 for our system. How much better is Giraph on YARN? >> Thank you. >> >> -- >> Matthew Cornell | m...@matthewcornell.org _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
