Thanks, I would go with log disabling. BTW, the master crashed while the application was still running.
------------------------------ *Mariano Semelman* P13N - IT Av. Corrientes Nº 746 - piso 13 - C.A.B.A. (C1043AAU) Teléfono (54) 11- *4894-3500* [image: Seguinos en Twitter!] <http://twitter.com/#!/despegarar> [image: Seguinos en Facebook!] <http://www.facebook.com/despegar> [image: Seguinos en YouTube!] <http://www.youtube.com/Despegar> *Despegar.com* El mejor precio para tu viaje. Este mensaje es confidencial y puede contener información amparada por el secreto profesional. Si usted ha recibido este e-mail por error, por favor comuníquenoslo inmediatamente respondiendo a este e-mail y luego eliminándolo de su sistema. El contenido de este mensaje no deberá ser copiado ni divulgado a ninguna persona. On 13 September 2016 at 12:52, Bryan Cutler <cutl...@gmail.com> wrote: > It looks like you have logging enabled and your application event log is > too large for the master to build a web UI from it. In spark 1.6.2 and > earlier, when an application completes, the master rebuilds a web UI to > view events after the fact. This functionality was removed in spark 2.0 > and the history server should be used instead. If you are unable to > upgrade could you try disabling logging? > > On Sep 13, 2016 7:18 AM, "Mariano Semelman" <mariano.semel...@despegar.com> > wrote: > >> Hello everybody, >> >> I am running a spark streaming app and I am planning to use it as a long >> running service. However while trying the app in a rc environment I got >> this exception in the master daemon after 1 hour of running: >> >> Exception in thread "master-rebuild-ui-thread" >> java.lang.OutOfMemoryError: GC overhead limit exceeded >> at java.util.regex.Pattern.compile(Pattern.java:1667) >> at java.util.regex.Pattern.<init>(Pattern.java:1351) >> at java.util.regex.Pattern.compile(Pattern.java:1054) >> at java.lang.String.replace(String.java:2239) >> at org.apache.spark.util.Utils$.getFormattedClassName(Utils.sca >> la:1632) >> at org.apache.spark.util.JsonProtocol$.sparkEventFromJson(JsonP >> rotocol.scala:486) >> at org.apache.spark.scheduler.ReplayListenerBus.replay(ReplayLi >> stenerBus.scala:58) >> at org.apache.spark.deploy.master.Master$$anonfun$17.apply( >> Master.scala:972) >> at org.apache.spark.deploy.master.Master$$anonfun$17.apply( >> Master.scala:952) >> at scala.concurrent.impl.Future$PromiseCompletingRunnable.lifte >> dTree1$1(Future.scala:24) >> at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(F >> uture.scala:24) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >> Executor.java:1142) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >> lExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> >> >> As a palliative measure I've increased the master memory to 1.5gb. >> My job is running with a batch interval of 5 seconds. >> I'm using spark version 1.6.2. >> >> I think it might be related to this issues: >> >> https://issues.apache.org/jira/browse/SPARK-6270 >> https://issues.apache.org/jira/browse/SPARK-12062 >> https://issues.apache.org/jira/browse/SPARK-12299 >> >> But I don't see a clear road to solve this apart from upgrading spark. >> What would you recommend? >> >> >> Thanks in advance >> Mariano >> >>
