You can try to run "jstack" a couple of times while the app is hung to look for patterns for where the app is hung. -- Ali
On Dec 3, 2015, at 8:27 AM, Richard Marscher <rmarsc...@localytics.com> wrote: > I should add that the pauses are not from GC and also in tracing the CPU call > tree in the JVM it seems like nothing is doing any work, just seems to be > idling or blocking. > > On Thu, Dec 3, 2015 at 11:24 AM, Richard Marscher <rmarsc...@localytics.com> > wrote: > Hi, > > I'm doing some testing of workloads using local mode on a server. I see weird > behavior where a job is submitted to the application and it just hangs for > several minutes doing nothing. The stages are submitted as pending and in the > application UI the stage view claims no tasks have been submitted. Suddenly > after a few minutes things suddenly start and run smoothly. > > I'm running against tiny data sets the size of 10s to low 100s of items in > the RDD. I've been attaching with JProfiler, doing thread and heap dumps but > nothing is really standing out as to why Spark seems to periodically pause > for such a long time. > > Has anyone else seen similar behavior or aware of some quirk of local mode > that could cause this kind of blocking? > > -- > Richard Marscher > Software Engineer > Localytics > Localytics.com | Our Blog | Twitter | Facebook | LinkedIn > > > > -- > Richard Marscher > Software Engineer > Localytics > Localytics.com | Our Blog | Twitter | Facebook | LinkedIn
