[jira] [Created] (TOREE-390) Lazily start Spark sessions
Ryan Blue created TOREE-390: --- Summary: Lazily start Spark sessions Key: TOREE-390 URL: https://issues.apache.org/jira/browse/TOREE-390 Project: TOREE Issue Type: Improvement Reporter: Ryan Blue In our deployment, more than half of the startup time for a Toree notebook is taken by starting a Spark session and waiting for containers. Lazily starting Spark sessions helps the notebook environment feel faster, even if the user is waiting on Spark to start up because the time waiting for Spark to start is clearly Spark, not Toree, and is initiated by the user. Also, lazily starting a Spark session allows users to change settings that can't be changed in a Spark context. It also enables the same startup code that would be used in a spark-submit application: {code:lang=java} SparkSession.builder .config(...) .setAppName(...) .getOrCreate() {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (TOREE-389) Scala interpreter hangs if Spark is referenced before it is bound
Ryan Blue created TOREE-389: --- Summary: Scala interpreter hangs if Spark is referenced before it is bound Key: TOREE-389 URL: https://issues.apache.org/jira/browse/TOREE-389 Project: TOREE Issue Type: Bug Reporter: Ryan Blue Some users run cells in a notebook before the kernel is finished starting. The messages are queued and run after the KernelMessageRelay is enabled, but any messages that reference {{spark}} or bound variables cause IMain to throw an exception. That exception causes the interpreter to hang. It seems odd that a missing variable reference causes IMain to throw an exception, so there may be a threading issue, a race between binding {{spark}} and the cell that references it. The problem can be avoided by moving {{postInit}} tasks into init. -- This message was sent by Atlassian JIRA (v6.3.15#6346)