Hi everyone,

Found a thrift server reliability issue on spark 1.3.1 that causes thrift to 
fail.

When thrift server has too little memory allocated to the driver to process the 
request, its Spark SQL session exits with OutOfMemory exception, causing thrift 
server to stop working.

Is this a known issue?

Thanks,
Judy

------------------
Full stacktrace of out of memory exception:
2015-07-08 03:30:18,011 ERROR actor.ActorSystemImpl 
(Slf4jLogger.scala:apply$mcV$sp(66)) - Uncaught fatal error from thread 
[sparkDriver-akka.remote.default-remote-dispatcher-6] shutting down ActorSystem 
[sparkDriver]
java.lang.OutOfMemoryError: Java heap space
                at 
org.spark_project.protobuf.ByteString.toByteArray(ByteString.java:515)
                at 
akka.remote.serialization.MessageContainerSerializer.fromBinary(MessageContainerSerializer.scala:64)
                at 
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)
                at scala.util.Try$.apply(Try.scala:161)
                at 
akka.serialization.Serialization.deserialize(Serialization.scala:98)
                at 
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
                at 
akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:58)
                at 
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:58)
                at 
akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:76)
                at 
akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:937)
                at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
                at akka.remote.EndpointActor.aroundReceive(Endpoint.scala:415)
                at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
                at akka.actor.ActorCell.invoke(ActorCell.scala:487)
                at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
                at akka.dispatch.Mailbox.run(Mailbox.scala:220)
                at 
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
                at 
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
                at 
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
                at 
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
                at 
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Reply via email to