[ https://issues.apache.org/jira/browse/LIVY-325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928493#comment-16928493 ]
Naman Mishra commented on LIVY-325: ----------------------------------- [~jerryshao] Created a separate Jira for now for named interpreter group feature since the scope of this Jira is bigger: https://issues.apache.org/jira/browse/LIVY-666 > Refactoring Livy Session and Interpreter > ---------------------------------------- > > Key: LIVY-325 > URL: https://issues.apache.org/jira/browse/LIVY-325 > Project: Livy > Issue Type: New Feature > Components: REPL > Affects Versions: 0.4.0 > Reporter: Saisai Shao > Priority: Major > > Currently in Livy master code, Livy interpreter is bound with Livy session, > when we created a session, we should specify which interpreter we want, and > this interpreter will be created implicitly. This potentially has several > limitations: > 1. We cannot create a share session, when we choose one language, we have to > create a new session to use. But some notebooks like Zeppelin could use > python, scala, R to manipulate data under the same SparkContext. So in Livy > we should decouple interpreter with SC and support shared context between > different interpreters. > 2. Furthermore, we cannot create multiple same interpreters in one session. > For example in Zeppelin scope mode, it could create multiple scala > interpreters to share with one context, but unfortunately in current Livy we > could not support this. > So based on the problems we mentioned above, we mainly have three things: > 1. Decouple interpreters from Spark context, so that we could create multiple > interpreters under one context. > 2. Make sure multiple interpreters could be worked together. > 3. Change REST APIs to support multiple interpreters per session. -- This message was sent by Atlassian Jira (v8.3.2#803003)