Livy interpreter internally creates multiple sessions for each user, independently from 3 binding modes supported in Zeppelin. Therefore, 'shared' mode, Livy interpreter will create sessions per each user, 'scoped' or 'isolated' mode will result create sessions per notebook, per user.
Notebook is shared among users, they always use the same interpreter instance/process, for now. I think supporting per user interpreter instance/process would be future work. Thanks, moon On Wed, Jun 29, 2016 at 7:57 AM Chen Song <chen.song...@gmail.com> wrote: > Thanks for your explanation, Moon. > > Following up on this, I can see the difference in terms of single or > multiple interpreter processes. > > With respect to spark drivers, since each interpreter spawns a separate > Spark driver in regular Spark interpreter setting, it is clear to me the > different implications of the 3 binding modes. > > However, when it comes to Livy server with impersonation turned on, I am a > bit confused. Will Livy interpreter always create a new Spark driver (along > with a Spark Context instance) for each user session, regardless of the > binding mode of Livy interpreter? I am not very familiar with Livy, but > from what I could tell, I see no difference between different binding modes > for Livy on as far as how Spark drivers are concerned. > > Last question, when a notebook is shared among users, will they always use > the same interpreter instance/process already created? > > Thanks > Chen > > > > On Fri, Jun 24, 2016 at 11:51 AM moon soo Lee <m...@apache.org> wrote: > >> Hi, >> >> Thanks for asking question. It's not dumb question at all, Zeppelin docs >> does not explain very well. >> >> Spark Interpreter, >> >> 'shared' mode, a spark interpreter setting spawn a interpreter process to >> serve all notebooks which binded to this interpreter setting. >> 'scoped' mode, a spark interpreter setting spawn multiple interpreter >> processes per notebook which binded to this interpreter setting. >> >> Using Livy interpreter, >> >> Zeppelin propagate current user information to Livy interpreter. And Livy >> interpreter creates different session per user via Livy Server. >> >> >> Hope this helps. >> >> Thanks, >> moon >> >> >> On Tue, Jun 21, 2016 at 6:41 PM Chen Song <chen.song...@gmail.com> wrote: >> >>> Zeppelin provides 3 binding modes for each interpreter. With `scoped` or >>> `shared` Spark interpreter, every user share the same SparkContext. Sorry >>> for the dumb question, how does it differ from Spark via Ivy Server? >>> >>> >>> -- >>> Chen Song >>> >>>
