Hello, I'm looking for documentation to better understand pyspark/scala notebook execution in Spark. I typically see application runtimes that can be very long, is there always a spark "application" running for a notebook or zeppelin session? Those that are not actually being run in zeppelin typically have very low resource utilization. Are these applications in spark tied to the zeppelin user's session?
Also, how can I find out more about hive, pyspark and scala interpreter concurrency? How many users/notebooks/paragraphs can execute these interpreters concurrently and how is this tunable? Any insight you can provide would be appreciated. Thanks, Josh
