Looking at the stack trace, it seems that the serializer has gone into the loop trying to serialize an ArrayList. Do you have a small sample application which could reproduce this issue. Or the stack trace at the start of the initial function, which will help to understand which data is being serialized.
-Tushar. On Mon, Mar 14, 2016 at 9:50 PM, Ganelin, Ilya <ilya.gane...@capitalone.com> wrote: > Bump on this - anyone have any idea why this would be happening? > > > > Sent with Good (www.good.com) > ________________________________ > From: Ganelin, Ilya <ilya.gane...@capitalone.com> > Sent: Friday, March 11, 2016 6:09:38 PM > To: dev@apex.incubator.apache.org > Subject: Re: Stack overflow errors when launching job > > Now with files: > https://gist.github.com/ilganeli/7f770374113b40ffa18a > > From: "Ganelin, Ilya" <ilya.gane...@capitalone.com<mailto: > ilya.gane...@capitalone.com>> > Reply-To: "dev@apex.incubator.apache.org<mailto: > dev@apex.incubator.apache.org>" <dev@apex.incubator.apache.org<mailto: > dev@apex.incubator.apache.org>> > Date: Friday, March 11, 2016 at 3:02 PM > To: "dev@apex.incubator.apache.org<mailto:dev@apex.incubator.apache.org>" > <dev@apex.incubator.apache.org<mailto:dev@apex.incubator.apache.org>> > Subject: Stack overflow errors when launching job > > Hi guys – I’m running into a very frustrating issue where certain DAG > configurations cause the following error log (attached). When this happens, > my application even fails to launch. This does not seem to be a YARN issue > since this occurs even with a relatively small number of partitions/memory. > > I’ve attached the input and output operators in question. > I can get this to occur predictable by > > 1. Increasing the partition count on my input operator (reads from > HDFS) - values above 20 cause this error > 2. Increase the partition count on my output operator (writes to HDFS) > - values above 20 cause this error > 3. Set stream locality from the default to either thread local, node > local, or container_local on the output operator > > This behavior is very frustrating as it’s preventing me from partitioning > my HDFS I/O appropriately, thus allowing me to scale to higher throughputs. > > Do you have any thoughts on what’s going wrong? I would love your feedback. > > > ________________________________ > > The information contained in this e-mail is confidential and/or > proprietary to Capital One and/or its affiliates and may only be used > solely in performance of work or services for Capital One. The information > transmitted herewith is intended only for use by the individual or entity > to which it is addressed. If the reader of this message is not the intended > recipient, you are hereby notified that any review, retransmission, > dissemination, distribution, copying or other use of, or taking of any > action in reliance upon this information is strictly prohibited. If you > have received this communication in error, please contact the sender and > delete the material from your computer. > ________________________________________________________ > > The information contained in this e-mail is confidential and/or > proprietary to Capital One and/or its affiliates and may only be used > solely in performance of work or services for Capital One. The information > transmitted herewith is intended only for use by the individual or entity > to which it is addressed. If the reader of this message is not the intended > recipient, you are hereby notified that any review, retransmission, > dissemination, distribution, copying or other use of, or taking of any > action in reliance upon this information is strictly prohibited. If you > have received this communication in error, please contact the sender and > delete the material from your computer. > ________________________________________________________ > > The information contained in this e-mail is confidential and/or > proprietary to Capital One and/or its affiliates and may only be used > solely in performance of work or services for Capital One. The information > transmitted herewith is intended only for use by the individual or entity > to which it is addressed. If the reader of this message is not the intended > recipient, you are hereby notified that any review, retransmission, > dissemination, distribution, copying or other use of, or taking of any > action in reliance upon this information is strictly prohibited. If you > have received this communication in error, please contact the sender and > delete the material from your computer. >
