Re: Extending SparkInterpreter functionality

>>> Same spark versions but multiple clusters. So based on logged in user, we may want to route to different spark cluster or even let user choose the spark he wants to connect to. If you use standalone, you can set `master` in interpreter setting for different standalone cluster. If using yarn, th

Re: Extending SparkInterpreter functionality

Hi Jeff, #3 is not about different spark versions. Same spark versions but multiple clusters. So based on logged in user, we may want to route to different spark cluster or even let user choose the spark he wants to connect to. Will work with Jhon to create tickets on other #2. What is the tur

Re: Extending SparkInterpreter functionality

1) Spark UI which works differently on EMR than standalone, so that logic will be in an interpreter specific to emr. Could you create a ticket for that, and please add details of that ? I don't know exactly what the difference between EMR and standalone, we can expose api to allow customization

use R interpreter without spark

Hi folks, is it possible to use R interpreter without spark, like *python* interpreter and *pyspark co-exist ?* -- Thanks & Regards, Arun M.J Engineer R&D infra. Flytxt Mobile Solutions Pvt Ltd arunmj93 +91.9947734508 www.flytxt.com | Follow us | Connect on

Re: Extending SparkInterpreter functionality

This is exactly what we want Jeff! A hook to plug in our own interpreters. (I am on same team as Jhon btw) Right now there are too many concrete references and injecting stuff is not possible. Eg of customizations - 1) Spark UI which works differently on EMR than standalone, so that logic will

Re: Extending SparkInterpreter functionality

Hi Jhon, Do you mind to share what kind of custom function you want to add to spark interpreter ? One idea in my mind is that we could add extension point to the existing SparkInterpreter, and user can enhance SparkInterpreter via these extension point. That means we just open some interfaces and

Re: How to create security filter for Spark UI in Spark on YARN

I solve this by using the property hadoop.http.authentication.type to specify a custom Java Handler objects that contains the authentication logic. This class only has to implement the interface org.apache.hadoop.security.authentication.server.AuthenticationHandler. See: https://hadoop.apache.org

Extending SparkInterpreter functionality

Hello! I'm a software developer and as part of a project I require to extend the functionality of SparkInterpreter without modifying it. I need instead create a new interpreter that extends it or wrap its functionality. I also need the spark sub-interpreters to use my new custom interpreter, but

Re: Importing notes

Me neither. You're right again. =D On Thu, Feb 1, 2018 at 11:52 AM, Partridge, Lucas (GE Aviation) < lucas.partri...@ge.com> wrote: > Sorry, I probably shouldn’t have used the word ‘as-is’. You don’t need to > copy the whole notebook folder – only the folders beneath that that contain > the note

Importing notes

Sorry, I probably shouldn’t have used the word ‘as-is’. You don’t need to copy the whole notebook folder – only the folders beneath that that contain the notebook json files you want to make available to your new instance of Zeppelin. I usually only copy the notebooks I want, not all of them.

Re: Importing notes

You're right, Lucas. Copying the notebook folder as-is do the trick. I'm wondering now if there is a way to generate those unique ids to avoid copying the whole structure. Or maybe I can consider that a limitation of the import feature I'm doing now. On Thu, Feb 1, 2018 at 10:15 AM, Partridge, Lu

Importing notes

As far as I know all you have to do is copy the contents of the notebook folder as-is into the same folder of the new Zeppelin instance. That’s what I did when I moved from one Zeppelin version to another recently and it all appears to work! I think Zeppelin just reads the contents of that fold

Re: Importing notes

It doesn't matter where you store the notebooks (in a volume or directly in the container), what I'm trying to achieve is copy my notebooks somewhere that zeppelin could read and import all notebooks inside that path without having to manually click on the "Import Notebook" link. Imagine I have a f