It may be caused by GC pause. Did you check the GC time in the Spark UI? -Xiangrui
On Sun, Feb 15, 2015 at 8:10 PM, Debasish Das <debasish.da...@gmail.com> wrote: > Hi, > > I am sometimes getting WARN from running Similarity calculation: > > 15/02/15 23:07:55 WARN BlockManagerMasterActor: Removing BlockManager > BlockManagerId(7, abc.com, 48419, 0) with no recent heart beats: 66435ms > exceeds 45000ms > > Do I need to increase the default 45 s to larger values for cases where we > are doing blocked operation or long compute in the mapPartitions ? > > Thanks. > Deb --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
