[
https://issues.apache.org/jira/browse/LIVY-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16674935#comment-16674935
]
TaeYoung Kim commented on LIVY-532:
-----------------------------------
$ yarn logs --applicationId application_1541055457322_0145
i get below log.
===============================================================================
18/11/05 08:17:12 INFO yarn.YarnRMClient: Registering the ApplicationMaster
18/11/05 08:17:12 INFO util.Utils: Using initial executors = 1, max of
spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors
and spark.executor.instances
18/11/05 08:17:12 INFO yarn.YarnAllocator: Will request 1 executor
container(s), each with 1 core(s) and 1408 MB memory (including 384 MB of
overhead)
18/11/05 08:17:12 INFO yarn.YarnAllocator: Submitted 1 unlocalized container
requests.
18/11/05 08:17:12 INFO yarn.ApplicationMaster: Started progress reporter thread
with (heartbeat : 3000, initial allocation : 200) intervals
18/11/05 08:17:13 INFO yarn.YarnAllocator: Launching container
container_e09_1541055457322_0145_01_000002 on host master.dev.cloud.zoyi.co for
executor with ID 1
18/11/05 08:17:13 INFO yarn.YarnAllocator: Received 1 containers from YARN,
launching executors on 1 of them.
18/11/05 08:17:16 ERROR yarn.ApplicationMaster: RECEIVED SIGNAL TERM
18/11/05 08:17:16 INFO yarn.ApplicationMaster: Final app status: UNDEFINED,
exitCode: 16, (reason: Shutdown hook called before final status was reported.)
18/11/05 08:17:16 INFO yarn.ApplicationMaster: Unregistering ApplicationMaster
with UNDEFINED (diag message: Shutdown hook called before final status was
reported.)
18/11/05 08:17:16 INFO yarn.ApplicationMaster: Deleting staging directory
hdfs://master.dev.cloud.zoyi.co:8020/user/zoyi/.sparkStaging/application_1541055457322_0145
18/11/05 08:17:16 INFO util.ShutdownHookManager: Shutdown hook called
> Fail create session with spark,yarn-client mode
> -----------------------------------------------
>
> Key: LIVY-532
> URL: https://issues.apache.org/jira/browse/LIVY-532
> Project: Livy
> Issue Type: Bug
> Components: API, Interpreter, REPL, Server
> Affects Versions: 0.6.0
> Reporter: TaeYoung Kim
> Priority: Major
>
> I'm using
> * spark2.3
> * livy 0.6.0 snapshot (current master, last commit:
> 0442eb01aa293d540fdf23ddd86b86c4278b83b9)
> * yarn client mode
> I tried add jar in create session params.
> ```
> 18/11/05 08:17:16 WARN rsc.RSCClient: Client RPC channel closed unexpectedly.
> 18/11/05 08:17:16 WARN rsc.RSCClient: Error stopping RPC.
> io.netty.util.concurrent.BlockingOperationException:
> DefaultChannelPromise@638f9c7b(uncancellable)
> at
> io.netty.util.concurrent.DefaultPromise.checkDeadLock(DefaultPromise.java:394)
> at
> io.netty.channel.DefaultChannelPromise.checkDeadLock(DefaultChannelPromise.java:157)
> at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:230)
> at
> io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:129)
> at io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:28)
> at io.netty.util.concurrent.DefaultPromise.sync(DefaultPromise.java:336)
> at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:117)
> at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:28)
> at org.apache.livy.rsc.rpc.Rpc.close(Rpc.java:310)
> at org.apache.livy.rsc.RSCClient.stop(RSCClient.java:232)
> at org.apache.livy.rsc.RSCClient$2$1.onSuccess(RSCClient.java:129)
> at org.apache.livy.rsc.RSCClient$2$1.onSuccess(RSCClient.java:123)
> at org.apache.livy.rsc.Utils$2.operationComplete(Utils.java:108)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:518)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:492)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListenersWithStackOverFlowProtection(DefaultPromise.java:431)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
> at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:108)
> at
> io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> at
> io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:995)
> at
> io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:621)
> at
> io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:599)
> at
> io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:543)
> at
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.closeOnRead(AbstractNioByteChannel.java:71)
> at
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:158)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:564)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:505)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:419)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:391)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
> at java.lang.Thread.run(Thread.java:748)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: Exception in thread "main"
> scala.reflect.internal.FatalError: object Predef does not have a member
> classOf
> 18/11/05 08:17:16 INFO rsc.RSCClient: Failing pending job
> 3091ce65-e328-4c0f-9585-a861b90706d4 due to shutdown.
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.reflect.internal.Definitions$DefinitionsClass.scala$reflect$internal$Definitions$DefinitionsClass$$fatalMissingSymbol(Definitions.scala:1186)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.reflect.internal.Definitions$DefinitionsClass.getMember(Definitions.scala:1203)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.reflect.internal.Definitions$DefinitionsClass.getMemberMethod(Definitions.scala:1238)
> 18/11/05 08:17:16 INFO interactive.InteractiveSession: Stopping
> InteractiveSession 2...
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.Predef_classOf$lzycompute(Definitions.scala:1469)
> 18/11/05 08:17:16 INFO interactive.InteractiveSession: Failed to ping RSC
> driver for session 2. Killing application.
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.Predef_classOf(Definitions.scala:1469)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.isPredefClassOf(Definitions.scala:1459)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typedIdent$2(Typers.scala:4885)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typedIdentOrWildcard$1(Typers.scala:4908)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5340)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5360)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5396)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5423)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5370)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5374)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.interpreter.ReplGlobal$$anon$1$$anon$2.typed(ReplGlobal.scala:36)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5472)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5480)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:5012)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5312)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5359)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5396)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5423)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5370)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5374)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.interpreter.ReplGlobal$$anon$1$$anon$2.typed(ReplGlobal.scala:36)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5448)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.apply(Analyzer.scala:102)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.Global$GlobalPhase$$anonfun$applyPhase$1.apply$mcV$sp(Global.scala:440)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.Global$GlobalPhase.withCurrentUnit(Global.scala:431)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:440)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:94)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:93)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.collection.Iterator$class.foreach(Iterator.scala:893)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.run(Analyzer.scala:93)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1501)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.Global$Run.compileUnits(Global.scala:1486)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.Global$Run.compileSources(Global.scala:1481)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.interpreter.IMain.compileSourcesKeepingRun(IMain.scala:435)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compileAndSaveRun(IMain.scala:855)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compile(IMain.scala:813)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.interpreter.IMain.bind(IMain.scala:675)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> org.apache.livy.repl.SparkInterpreter$$anonfun$bind$1.apply(SparkInterpreter.scala:133)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> org.apache.livy.repl.SparkInterpreter$$anonfun$bind$1.apply(SparkInterpreter.scala:133)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> org.apache.livy.repl.SparkInterpreter.bind(SparkInterpreter.scala:132)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> org.apache.livy.repl.AbstractSparkInterpreter.postStart(AbstractSparkInterpreter.scala:72)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> org.apache.livy.repl.SparkInterpreter$$anonfun$start$1.apply$mcV$sp(SparkInterpreter.scala:89)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> org.apache.livy.repl.SparkInterpreter$$anonfun$start$1.apply(SparkInterpreter.scala:63)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> org.apache.livy.repl.SparkInterpreter$$anonfun$start$1.apply(SparkInterpreter.scala:63)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> org.apache.livy.repl.AbstractSparkInterpreter.restoreContextClassLoader(AbstractSparkInterpreter.scala:340)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> org.apache.livy.repl.SparkInterpreter.start(SparkInterpreter.scala:63)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> org.apache.livy.repl.Session$$anonfun$1.apply(Session.scala:128)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> org.apache.livy.repl.Session$$anonfun$1.apply(Session.scala:122)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 18/11/05 08:17:16 INFO utils.LineBufferedStream: at
> java.lang.Thread.run(Thread.java:748)
> ```
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
