Ryan Blue created TOREE-390: ------------------------------- Summary: Lazily start Spark sessions Key: TOREE-390 URL: https://issues.apache.org/jira/browse/TOREE-390 Project: TOREE Issue Type: Improvement Reporter: Ryan Blue
In our deployment, more than half of the startup time for a Toree notebook is taken by starting a Spark session and waiting for containers. Lazily starting Spark sessions helps the notebook environment feel faster, even if the user is waiting on Spark to start up because the time waiting for Spark to start is clearly Spark, not Toree, and is initiated by the user. Also, lazily starting a Spark session allows users to change settings that can't be changed in a Spark context. It also enables the same startup code that would be used in a spark-submit application: {code:lang=java} SparkSession.builder .config(...) .setAppName(...) .getOrCreate() {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)