Re: Inefficient third party JS library integration, e.g. plotly

[Andreas Weise]

Created https://issues.apache.org/jira/browse/ZEPPELIN-5019

Expect a PR soon for that.

On 2020/08/26 15:48:23, Jeff Zhang  wrote: 
> Adding custom js via configuration make sense to me.
> 
> Andreas Weise  于2020年8月26日周三 下午8:37写道：
> 
> > Bumping this one up again (after 2 years of silence), as its still not
> > possible to efficiently integrate plotly for multiple paragraphs in
> > zeppelin.
> >
> > WDYT about my suggestion. I can prepare a PR for that...
> >
> > > Therefore I'm again suggesting a new configuration property to allow
> > zeppelin administrators to include any kind of special JS code (most likely
> > referenced via an URL) to the html->head section. I'm not sure yet, if
> > thats possible at all, because configuration is some information from the
> > server, that needs to be loaded very early.
> >
> > Best Regards
> > Andreas
> >
> > On 2018/08/15 06:26:48, andreas.we...@gmail.com 
> > wrote:
> > >
> > >
> > > Thanks for your answers, but as long as custom JS code resides in
> > html->body (which is the case for any code loaded through notebooks) there
> > is no guarenteed loading sequence when using the default HTML outputs of
> > libs like plotly, which assume that all of the library code is already
> > loaded. Of course you can add more custom complex JS code (promise etc.) to
> > sort these issues per notebook, but that is not very enduser friendly.
> > >
> > > bokeh loads all of its JS resources in the html->head, which is why
> > things work there. And this is different to zeppelin, where everything is
> > loaded in the body.
> > >
> > https://github.com/bokeh/bokeh/blob/master/bokeh/core/_templates/file.html
> > >
> > > The point here is, that in zeppelin there is currently no feasible way
> > to add this kind of central customization to the html->head section,
> > neither for loading Libs from CDN nor loading JS code asynchronously (like
> > bokeh). (without rebuilding zeppelin-web.war)
> > >
> > > Therefore I'm again suggesting a new configuration property to allow
> > zeppelin administrators to include any kind of special JS code (most likely
> > referenced via an URL) to the html->head section. I'm not sure yet, if
> > thats possible at all, because configuration is some information from the
> > server, that needs to be loaded very early.
> > >
> > > Or can this be achieved via helium (or an extension to helium).
> > >
> > > Or any other idea how to solve this centrally and not on a per notebook
> > base?
> > >
> > > Regards
> > > Andreas
> > >
> > > On 2018/08/14 08:27:16, Jeff Zhang  wrote:
> > > > Here's the code of how bokeh loading js asynchronously, hope it is
> > useful
> > > > for you.
> > > >
> > > >
> > https://github.com/bokeh/bokeh/blob/master/bokeh/core/_templates/autoload_js.js
> > > >
> > > >
> > > > Partridge, Lucas (GE Aviation) 于2018年8月14日周二
> > > > 下午4:21写道：
> > > >
> > > > > I’m pretty rusty on this as it’s several years since I tried it but I
> > > > > remember having a similar problem when trying to use Google Maps
> > inside
> > > > > Zeppelin.  Unfortunately I can’t find the code I used but from
> > memory, my
> > > > > solution was to try invoking the library, and if, the library object
> > didn’t
> > > > > exist, load it. In the onload() callback (not sure I’ve remembered
> > the
> > > > > method’s name right) I would then draw my map. I certainly didn’t
> > embed the
> > > > > library in the notebook itself.
> > > > >
> > > > >
> > > > >
> > > > > *From:* Jeff Zhang 
> > > > > *Sent:* 14 August 2018 08:11
> > > > > *To:* users@zeppelin.apache.org
> > > > > *Subject:* EXT: Re: Inefficient third party JS library integration,
> > e.g.
> > > > > plotly
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Do you know how jupyter handle that ? And bokeh would load js from
> > CDN so
> > > > > it doesn't have the issue you mentioned, maybe plotly could use
> > similar
> > > > > approach.
> > > > >
> > > > >
> > > > >
> > > > > https://bokeh

Re: Inefficient third party JS library integration, e.g. plotly

[Andreas Weise]

Bumping this one up again (after 2 years of silence), as its still not possible 
to efficiently integrate plotly for multiple paragraphs in zeppelin. 

WDYT about my suggestion. I can prepare a PR for that...

> Therefore I'm again suggesting a new configuration property to allow zeppelin 
> administrators to include any kind of special JS code (most likely referenced 
> via an URL) to the html->head section. I'm not sure yet, if thats possible at 
> all, because configuration is some information from the server, that needs to 
> be loaded very early. 

Best Regards
Andreas

On 2018/08/15 06:26:48, andreas.we...@gmail.com  
wrote: 
> 
> 
> Thanks for your answers, but as long as custom JS code resides in html->body 
> (which is the case for any code loaded through notebooks) there is no 
> guarenteed loading sequence when using the default HTML outputs of libs like 
> plotly, which assume that all of the library code is already loaded. Of 
> course you can add more custom complex JS code (promise etc.) to sort these 
> issues per notebook, but that is not very enduser friendly.
> 
> bokeh loads all of its JS resources in the html->head, which is why things 
> work there. And this is different to zeppelin, where everything is loaded in 
> the body.
> https://github.com/bokeh/bokeh/blob/master/bokeh/core/_templates/file.html
> 
> The point here is, that in zeppelin there is currently no feasible way to add 
> this kind of central customization to the html->head section, neither for 
> loading Libs from CDN nor loading JS code asynchronously (like bokeh). 
> (without rebuilding zeppelin-web.war)
> 
> Therefore I'm again suggesting a new configuration property to allow zeppelin 
> administrators to include any kind of special JS code (most likely referenced 
> via an URL) to the html->head section. I'm not sure yet, if thats possible at 
> all, because configuration is some information from the server, that needs to 
> be loaded very early. 
> 
> Or can this be achieved via helium (or an extension to helium).
> 
> Or any other idea how to solve this centrally and not on a per notebook base?
> 
> Regards
> Andreas
> 
> On 2018/08/14 08:27:16, Jeff Zhang  wrote: 
> > Here's the code of how bokeh loading js asynchronously, hope it is useful
> > for you.
> > 
> > https://github.com/bokeh/bokeh/blob/master/bokeh/core/_templates/autoload_js.js
> > 
> > 
> > Partridge, Lucas (GE Aviation) 于2018年8月14日周二
> > 下午4:21写道：
> > 
> > > I’m pretty rusty on this as it’s several years since I tried it but I
> > > remember having a similar problem when trying to use Google Maps inside
> > > Zeppelin.  Unfortunately I can’t find the code I used but from memory, my
> > > solution was to try invoking the library, and if, the library object 
> > > didn’t
> > > exist, load it. In the onload() callback (not sure I’ve remembered the
> > > method’s name right) I would then draw my map. I certainly didn’t embed 
> > > the
> > > library in the notebook itself.
> > >
> > >
> > >
> > > *From:* Jeff Zhang 
> > > *Sent:* 14 August 2018 08:11
> > > *To:* users@zeppelin.apache.org
> > > *Subject:* EXT: Re: Inefficient third party JS library integration, e.g.
> > > plotly
> > >
> > >
> > >
> > >
> > >
> > > Do you know how jupyter handle that ? And bokeh would load js from CDN so
> > > it doesn't have the issue you mentioned, maybe plotly could use similar
> > > approach.
> > >
> > >
> > >
> > > https://bokeh.pydata.org/en/latest/
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > andreas.we...@gmail.com 于2018年8月14日周二 下午2:34写道：
> > >
> > > We are using plotly for charts quite often (plotly python in conjunction
> > > with pyspark) and it reavels a weakness regarding to third party JS 
> > > library
> > > integration.
> > >
> > > Unfortunately current plotly integration is not very efficient in terms of
> > > library integration, which leads to huge notebooks. This is because
> > > notebooks that contain plotly will have to output the full js code of
> > > plotly. (~3MB). When a Notebook contains several of that integration, the
> > > notebook becomes very slow. I think this also applies to other js 
> > > libraries.
> > >
> > > E.g. for plotly you can render charts in zeppelin the following way with
> > > pyspark (the first print is only for demoing the actual html output in
> > > cleartext):
> > > -
> > > %pyspark
> > > from plotly import graph_objs as go
> > > from plotly.offline import plot
> > >
> > > out = plot([go.Scatter(x=[1, 2, 3], y=[3, 1, 6])],include_plotlyjs=True,
> > > output_type='div')
> > >
> > > print(out)
> > > print('%html', out)
> > > -
> > >
> > > An alternative would be, to load plotly.js (or any other third party
> > > library) from a CDN, e.g.
> > >  https://cdn.plot.ly/plotly-latest.min.js";> (and
> > > settings include_plotlyjs in the example above to false, which would lead
> > > to the bare minimum of code for that specific chart)

Re: [DISCUSS] Release 0.8.2

[Andreas Weise]

+1.

Would you mind getting ZEPPELIN-4299 merged to branch-0.8?
https://issues.apache.org/jira/browse/ZEPPELIN-4299

PR already open
https://github.com/apache/zeppelin/pull/3423

The issue is pretty annoying for the end user experience.

Best regards
Andreas

On Tue, Sep 24, 2019 at 8:18 PM Miquel Angel Andreu Febrer <
miquelangeland...@gmail.com> wrote:

> +1
>
> El mar., 24 sept. 2019 18:28, moon soo Lee  escribió:
>
> > +1
> >
> > On Mon, Sep 23, 2019 at 8:16 PM Prabhjyot Singh <
> prabhjyotsi...@gmail.com>
> > wrote:
> >
> > > +1
> > >
> > > I see a lot of issues are fixed in the past few months. This sounds a
> > > good intermediate step before 0.9 release.
> > >
> > > On Tue, 24 Sep 2019 at 00:23, Eric Pugh <
> ep...@opensourceconnections.com
> > >
> > > wrote:
> > >
> > >> The iPython related fixes would address some issues I’ve run into!
> > >>
> > >> On Sep 22, 2019, at 11:31 PM, Jeff Zhang  wrote:
> > >>
> > >> Hi Folks,
> > >>
> > >> 0.8.1 is released half year ago (Jan 2019), I'd to start the
> discussion
> > >> about releasing of 0.8.2 which fixed several critical issues. Let me
> > know
> > >> your thoughts, thanks.
> > >>
> > >>
> > >>
> >
> https://jira.apache.org/jira/issues/?jql=project%20%3D%20ZEPPELIN%20AND%20fixVersion%20%3D%200.8.2
> > >>
> > >>
> > >>
> > >> --
> > >> Best Regards
> > >>
> > >> Jeff Zhang
> > >>
> > >>
> > >> ___
> > >> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC |
> > 434.466.1467
> > >> | http://www.opensourceconnections.com | My Free/Busy
> > >> 
> > >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > >> <
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> > >
> > >> This e-mail and all contents, including attachments, is considered to
> be
> > >> Company Confidential unless explicitly stated otherwise, regardless
> > >> of whether attachments are marked as such.
> > >>
> > >>
> > >
> > > --
> > > Regards,
> > > Prabhjyot Singh
> > >
> >
>

Re: [DISCUSS] ZEPPELIN-2619. Save note in [Title].zpln instead of [NOTEID]/note.json

[andreas . weise]

Somehow subject was deleted in earlier mail. So here again for adding my 
thoughts to the proper thread:

Sure, there might exist some kind of naming policy (or let's better call it 
naming convention) in zeppelin multiuser environments. But as long as there is 
no way to technically enforce a certain naming convention to turn it into a 
naming policy (which would be a nice idea BTW), it's IMHO a vague assumption, 
that there exist policies and users are following these. I'm thinking here of 
problems that might come up when changing the existing implementation and then 
deal with migration, because assumptions do not match reality.

My real life scenario here is, that zeppelin can be configured to make 
notebooks visible only to the owner (and invisible to any other user) by 
default: ZEPPELIN_NOTEBOOK_PUBLIC=false, which is IMHO a good idea when setting 
up zeppelin as multiuser environment in larger scenarios. In this case note 
owners can use any or no naming convention they like when creating and using a 
note for personal purposes only, because only the owner will see it - also and 
even if there exists certain naming policies on an global organisation level. A 
global naming convention must only be followed when users start sharing notes 
(means: adding at least reader permissions to any other user).

So I think noteId is a must have in the filename.

Andreas

On 2018/08/31 01:46:38, Jongyoul Lee  wrote: 
> Hi,
> 
> I have a bit different thoughts about the conflicts of the name of a new
> note created. In a multiuser environment, AFAIK, most teams and companies,
> generally, use a prefix for the group policy internally. In my case,
> user/{user_id}/{notebook_name_they_want}.zpln. In this case, naming
> conflicts rarely happen. And it will be stored under a specific folder. If
> someone needed two different same named notes in the same directory, I
> might not be appropriate. WDYT?
> 
> JL
> 
> On Fri, Aug 31, 2018 at 4:44 AM, andreas.we...@gmail.com <
> andreas.we...@gmail.com> wrote:
> 
> > another reason for keeping noteId is uniqueness in case of multi-user
> > environments. In that case users have separate zeppelin workspaces, which
> > is something we are using in production: see ZEPPELIN_NOTEBOOK_PUBLIC=false
> > in the doc [1]. In that case users might be very confused when they can not
> > create notebooks with a name that already exists, but they most likely
> > don't see (yet).
> >
> > So I like the proposal {note_name}_{note_id}.zpln. where note_name could
> > contains folders, e.g. folder_1/mynote_abcd.zpln. Even though I like
> > {note_name}.{note_id}.zpln (dot in between note_name and note_id) even
> > better :-)
> >
> > Regards
> > Andreas
> >
> >
> > [1] http://zeppelin.apache.org/docs/0.8.0/setup/security/
> > notebook_authorization.html#separate-notebook-workspaces-public-vs-private
> >
> > On 2018/08/18 08:42:44, Jeff Zhang  wrote:
> > > BTW, I also prefer to use note name as identify of note if the issue I
> > > mentioned before is acceptable for most of users.
> > >
> > >
> > >
> > > Jeff Zhang 于2018年8月18日周六 下午4:40写道：
> > >
> > > >
> > > > I am afraid we can not remove noteId, as noteId is the unique
> > identifier
> > > > of note and is immutable which is used in a lot places, such as
> > paragraph
> > > > share and rest api.
> > > > If we use note name as note id then it may break user's app if note
> > name
> > > > is changed
> > > >
> > > >
> > > > Jongyoul Lee 于2018年8月18日周六 下午2:33写道：
> > > >
> > > >> Hi, thanks for this kind of discussion.
> > > >>
> > > >> About noteId, How about changing note id to note name? AFAIK, Note id
> > is
> > > >> just an identifier and we can set any value to it.
> > > >>
> > > >> There’re two potential problems. We should be more careful to handle
> > note
> > > >> id as it could have very various type of characters. And Second, in
> > case
> > > >> where someone changes a note name, those who are seeing and updating
> > the
> > > >> same note wouldn’t access that note. We could handle it by using
> > websockets.
> > > >>
> > > >> WDYT?
> > > >>
> > > >> On Tue, 14 Aug 2018 at 6:14 PM Jeff Zhang  wrote:
> > > >>
> > > >>> >>> But I’m still not comfortable with note ids in the name of the
> > > >>> notebook itself.  Those names would look ugly if you shared your
> > notebooks
> > > >>> on github for example.  You don’t see Jupyter notebooks with names
> > like
> > > >>> that. If you have to keep the note ids with the notebooks could you
> > not
> > > >>> simply put the note id at the top of the notebook as Ruslan
> > suggested? Then
> > > >>> you’d only have to read the first line of each notebook.
> > > >>>
> > > >>> I know putting note_id in the note file name is not so elegant, but
> > this
> > > >>> is what we have to compromise to keep compatibility as we use noteId
> > to
> > > >>> uniquely identify note right now. And I don't think putting noteId
> > in the
> > > >>> top first line of note would help much. We still have to read note
> > 

[no subject]

[andreas . weise]

Sure, there might exist some kind of naming policy (or let's better call it 
naming convention) in zeppelin multiuser environments. But as long as there is 
no way to technically enforce a certain naming convention to turn it into a 
naming policy (which would be a nice idea BTW), it's IMHO a vague assumption, 
that there exist policies and users are following these. I'm thinking here of 
problems that might come up when changing the existing implementation and then 
deal with migration, because assumptions do not match reality.

My real life scenario here is, that zeppelin can be configured to make 
notebooks visible only to the owner (and invisible to any other user) by 
default: ZEPPELIN_NOTEBOOK_PUBLIC=false, which is IMHO a good idea when setting 
up zeppelin as multiuser environment in larger scenarios. In this case note 
owners can use any or no naming convention they like when creating and using a 
note for personal purposes only, because only the owner will see it - also and 
even if there exists certain naming policies on an global organisation level. A 
global naming convention must only be followed when users start sharing notes 
(means: adding at least reader permissions to any other user).

So I think noteId is a must have in the filename.

Andreas

On 2018/08/31 01:46:38, Jongyoul Lee  wrote: 
> Hi,
> 
> I have a bit different thoughts about the conflicts of the name of a new
> note created. In a multiuser environment, AFAIK, most teams and companies,
> generally, use a prefix for the group policy internally. In my case,
> user/{user_id}/{notebook_name_they_want}.zpln. In this case, naming
> conflicts rarely happen. And it will be stored under a specific folder. If
> someone needed two different same named notes in the same directory, I
> might not be appropriate. WDYT?
> 
> JL
> 
> On Fri, Aug 31, 2018 at 4:44 AM, andreas.we...@gmail.com <
> andreas.we...@gmail.com> wrote:
> 
> > another reason for keeping noteId is uniqueness in case of multi-user
> > environments. In that case users have separate zeppelin workspaces, which
> > is something we are using in production: see ZEPPELIN_NOTEBOOK_PUBLIC=false
> > in the doc [1]. In that case users might be very confused when they can not
> > create notebooks with a name that already exists, but they most likely
> > don't see (yet).
> >
> > So I like the proposal {note_name}_{note_id}.zpln. where note_name could
> > contains folders, e.g. folder_1/mynote_abcd.zpln. Even though I like
> > {note_name}.{note_id}.zpln (dot in between note_name and note_id) even
> > better :-)
> >
> > Regards
> > Andreas
> >
> >
> > [1] http://zeppelin.apache.org/docs/0.8.0/setup/security/
> > notebook_authorization.html#separate-notebook-workspaces-public-vs-private
> >
> > On 2018/08/18 08:42:44, Jeff Zhang  wrote:
> > > BTW, I also prefer to use note name as identify of note if the issue I
> > > mentioned before is acceptable for most of users.
> > >
> > >
> > >
> > > Jeff Zhang 于2018年8月18日周六 下午4:40写道：
> > >
> > > >
> > > > I am afraid we can not remove noteId, as noteId is the unique
> > identifier
> > > > of note and is immutable which is used in a lot places, such as
> > paragraph
> > > > share and rest api.
> > > > If we use note name as note id then it may break user's app if note
> > name
> > > > is changed
> > > >
> > > >
> > > > Jongyoul Lee 于2018年8月18日周六 下午2:33写道：
> > > >
> > > >> Hi, thanks for this kind of discussion.
> > > >>
> > > >> About noteId, How about changing note id to note name? AFAIK, Note id
> > is
> > > >> just an identifier and we can set any value to it.
> > > >>
> > > >> There’re two potential problems. We should be more careful to handle
> > note
> > > >> id as it could have very various type of characters. And Second, in
> > case
> > > >> where someone changes a note name, those who are seeing and updating
> > the
> > > >> same note wouldn’t access that note. We could handle it by using
> > websockets.
> > > >>
> > > >> WDYT?
> > > >>
> > > >> On Tue, 14 Aug 2018 at 6:14 PM Jeff Zhang  wrote:
> > > >>
> > > >>> >>> But I’m still not comfortable with note ids in the name of the
> > > >>> notebook itself.  Those names would look ugly if you shared your
> > notebooks
> > > >>> on github for example.  You don’t see Jupyter notebooks with names
> > like
> > > >>> that. If you have to keep the note ids with the notebooks could you
> > not
> > > >>> simply put the note id at the top of the notebook as Ruslan
> > suggested? Then
> > > >>> you’d only have to read the first line of each notebook.
> > > >>>
> > > >>> I know putting note_id in the note file name is not so elegant, but
> > this
> > > >>> is what we have to compromise to keep compatibility as we use noteId
> > to
> > > >>> uniquely identify note right now. And I don't think putting noteId
> > in the
> > > >>> top first line of note would help much. We still have to read note
> > files
> > > >>> which take much more time than just read the file names via file
> > system.
> > > >>>
> >

Re: [DISCUSS] ZEPPELIN-2619. Save note in [Title].zpln instead of [NOTEID]/note.json

[andreas . weise]

another reason for keeping noteId is uniqueness in case of multi-user 
environments. In that case users have separate zeppelin workspaces, which is 
something we are using in production: see ZEPPELIN_NOTEBOOK_PUBLIC=false in the 
doc [1]. In that case users might be very confused when they can not create 
notebooks with a name that already exists, but they most likely don't see (yet).

So I like the proposal {note_name}_{note_id}.zpln. where note_name could 
contains folders, e.g. folder_1/mynote_abcd.zpln. Even though I like 
{note_name}.{note_id}.zpln (dot in between note_name and note_id) even better 
:-)

Regards
Andreas


[1] 
http://zeppelin.apache.org/docs/0.8.0/setup/security/notebook_authorization.html#separate-notebook-workspaces-public-vs-private

On 2018/08/18 08:42:44, Jeff Zhang  wrote: 
> BTW, I also prefer to use note name as identify of note if the issue I
> mentioned before is acceptable for most of users.
> 
> 
> 
> Jeff Zhang 于2018年8月18日周六 下午4:40写道：
> 
> >
> > I am afraid we can not remove noteId, as noteId is the unique identifier
> > of note and is immutable which is used in a lot places, such as paragraph
> > share and rest api.
> > If we use note name as note id then it may break user's app if note name
> > is changed
> >
> >
> > Jongyoul Lee 于2018年8月18日周六 下午2:33写道：
> >
> >> Hi, thanks for this kind of discussion.
> >>
> >> About noteId, How about changing note id to note name? AFAIK, Note id is
> >> just an identifier and we can set any value to it.
> >>
> >> There’re two potential problems. We should be more careful to handle note
> >> id as it could have very various type of characters. And Second, in case
> >> where someone changes a note name, those who are seeing and updating the
> >> same note wouldn’t access that note. We could handle it by using 
> >> websockets.
> >>
> >> WDYT?
> >>
> >> On Tue, 14 Aug 2018 at 6:14 PM Jeff Zhang  wrote:
> >>
> >>> >>> But I’m still not comfortable with note ids in the name of the
> >>> notebook itself.  Those names would look ugly if you shared your notebooks
> >>> on github for example.  You don’t see Jupyter notebooks with names like
> >>> that. If you have to keep the note ids with the notebooks could you not
> >>> simply put the note id at the top of the notebook as Ruslan suggested? 
> >>> Then
> >>> you’d only have to read the first line of each notebook.
> >>>
> >>> I know putting note_id in the note file name is not so elegant, but this
> >>> is what we have to compromise to keep compatibility as we use noteId to
> >>> uniquely identify note right now. And I don't think putting noteId in the
> >>> top first line of note would help much. We still have to read note files
> >>> which take much more time than just read the file names via file system.
> >>>
> >>> Regarding the readability of note file name, I think it won't affect
> >>> much. E.g. This is the note book file name like:  *My Project/My Spark
> >>> Tutorial Note_2A94M5J1Z.zpln*
> >>> What user see in notebook menu is still *My Project/My Spark Tutorial* 
> >>> *Note
> >>> *which is no difference from what we see now.
> >>>
> >>> And thanks again for the feedback and comments, I am so glad to see so
> >>> many discussion in community.
> >>>
> >>>
> >>>
> >>> Partridge, Lucas (GE Aviation) 于2018年8月14日周二
> >>> 下午4:29写道：
> >>>
>  I agree you’re inviting consistency issues if you maintained a separate
>  note id-to-note name mapping file.
> 
> 
> 
>  But I’m still not comfortable with note ids in the name of the notebook
>  itself.  Those names would look ugly if you shared your notebooks on 
>  github
>  for example.  You don’t see Jupyter notebooks with names like that.  If 
>  you
>  have to keep the note ids with the notebooks could you not simply put the
>  note id at the top of the notebook as Ruslan suggested? Then you’d only
>  have to read the first line of each notebook.
> 
> 
> 
>  Presumably if you copied the notebooks to another Zeppelin server they
>  would be restored with the same note ids there too? And hopefully there
>  would be no id clash with notebooks already on that server…
> 
> 
> 
>  *From:* Jeff Zhang 
>  *Sent:* 14 August 2018 03:49
>  *To:* users@zeppelin.apache.org
> 
> 
>  *Subject:* EXT: Re: [DISCUSS] ZEPPELIN-2619. Save note in [Title].zpln
>  instead of [NOTEID]/note.json
> 
> 
> 
> 
> 
>  Thanks for the discussion.
> 
>  >>> I'm afraid about non-latin symbols in folder and note name. And
>  what about hieroglyphs?
> 
>  AFAIK, linux allow all the characters to be file name except `\0` and
>  '/'.  I can create file name with Chinese character in linux, I guess you
>  can use Russian as well.
> 
> 
> 
>  >>> If I understand correctly, this is being done solely to speed up
>  loading list of notebooks? What if a list of notebook names, their ids,
>  folder 

Re: Inefficient third party JS library integration, e.g. plotly

[andreas . weise]

Thanks for your answers, but as long as custom JS code resides in html->body 
(which is the case for any code loaded through notebooks) there is no 
guarenteed loading sequence when using the default HTML outputs of libs like 
plotly, which assume that all of the library code is already loaded. Of course 
you can add more custom complex JS code (promise etc.) to sort these issues per 
notebook, but that is not very enduser friendly.

bokeh loads all of its JS resources in the html->head, which is why things work 
there. And this is different to zeppelin, where everything is loaded in the 
body.
https://github.com/bokeh/bokeh/blob/master/bokeh/core/_templates/file.html

The point here is, that in zeppelin there is currently no feasible way to add 
this kind of central customization to the html->head section, neither for 
loading Libs from CDN nor loading JS code asynchronously (like bokeh). (without 
rebuilding zeppelin-web.war)

Therefore I'm again suggesting a new configuration property to allow zeppelin 
administrators to include any kind of special JS code (most likely referenced 
via an URL) to the html->head section. I'm not sure yet, if thats possible at 
all, because configuration is some information from the server, that needs to 
be loaded very early. 

Or can this be achieved via helium (or an extension to helium).

Or any other idea how to solve this centrally and not on a per notebook base?

Regards
Andreas

On 2018/08/14 08:27:16, Jeff Zhang  wrote: 
> Here's the code of how bokeh loading js asynchronously, hope it is useful
> for you.
> 
> https://github.com/bokeh/bokeh/blob/master/bokeh/core/_templates/autoload_js.js
> 
> 
> Partridge, Lucas (GE Aviation) 于2018年8月14日周二
> 下午4:21写道：
> 
> > I’m pretty rusty on this as it’s several years since I tried it but I
> > remember having a similar problem when trying to use Google Maps inside
> > Zeppelin.  Unfortunately I can’t find the code I used but from memory, my
> > solution was to try invoking the library, and if, the library object didn’t
> > exist, load it. In the onload() callback (not sure I’ve remembered the
> > method’s name right) I would then draw my map. I certainly didn’t embed the
> > library in the notebook itself.
> >
> >
> >
> > *From:* Jeff Zhang 
> > *Sent:* 14 August 2018 08:11
> > *To:* users@zeppelin.apache.org
> > *Subject:* EXT: Re: Inefficient third party JS library integration, e.g.
> > plotly
> >
> >
> >
> >
> >
> > Do you know how jupyter handle that ? And bokeh would load js from CDN so
> > it doesn't have the issue you mentioned, maybe plotly could use similar
> > approach.
> >
> >
> >
> > https://bokeh.pydata.org/en/latest/
> >
> >
> >
> >
> >
> >
> >
> > andreas.we...@gmail.com 于2018年8月14日周二 下午2:34写道：
> >
> > We are using plotly for charts quite often (plotly python in conjunction
> > with pyspark) and it reavels a weakness regarding to third party JS library
> > integration.
> >
> > Unfortunately current plotly integration is not very efficient in terms of
> > library integration, which leads to huge notebooks. This is because
> > notebooks that contain plotly will have to output the full js code of
> > plotly. (~3MB). When a Notebook contains several of that integration, the
> > notebook becomes very slow. I think this also applies to other js libraries.
> >
> > E.g. for plotly you can render charts in zeppelin the following way with
> > pyspark (the first print is only for demoing the actual html output in
> > cleartext):
> > -
> > %pyspark
> > from plotly import graph_objs as go
> > from plotly.offline import plot
> >
> > out = plot([go.Scatter(x=[1, 2, 3], y=[3, 1, 6])],include_plotlyjs=True,
> > output_type='div')
> >
> > print(out)
> > print('%html', out)
> > -
> >
> > An alternative would be, to load plotly.js (or any other third party
> > library) from a CDN, e.g.
> >  https://cdn.plot.ly/plotly-latest.min.js";> (and
> > settings include_plotlyjs in the example above to false, which would lead
> > to the bare minimum of code for that specific chart).
> > But this needs to be added to the html->head, otherwise any dependent code
> > using that library might run into 'Uncaught ReferenceError: ... is not
> > defined' issues, due to timing.
> >
> > I found that setup guide for repacking the zeppelin-web archive, to
> > include a custom library in the index.html, but IMHO that seems to be a
> > workaround:
> > https://github.com/beljun/zeppelin-plotly
> >
> > Therefore I would suggest to add some zeppelin server wide configuration
> > to allow zeppelin administrators to include additional 

Inefficient third party JS library integration, e.g. plotly

[andreas . weise]

We are using plotly for charts quite often (plotly python in conjunction with 
pyspark) and it reavels a weakness regarding to third party JS library 
integration.

Unfortunately current plotly integration is not very efficient in terms of 
library integration, which leads to huge notebooks. This is because notebooks 
that contain plotly will have to output the full js code of plotly. (~3MB). 
When a Notebook contains several of that integration, the notebook becomes very 
slow. I think this also applies to other js libraries.

E.g. for plotly you can render charts in zeppelin the following way with 
pyspark (the first print is only for demoing the actual html output in 
cleartext):
-
%pyspark
from plotly import graph_objs as go
from plotly.offline import plot

out = plot([go.Scatter(x=[1, 2, 3], y=[3, 1, 6])],include_plotlyjs=True, 
output_type='div')

print(out)
print('%html', out)
-

An alternative would be, to load plotly.js (or any other third party library) 
from a CDN, e.g.
 https://cdn.plot.ly/plotly-latest.min.js";> (and settings 
include_plotlyjs in the example above to false, which would lead to the bare 
minimum of code for that specific chart).
But this needs to be added to the html->head, otherwise any dependent code 
using that library might run into 'Uncaught ReferenceError: ... is not defined' 
issues, due to timing.

I found that setup guide for repacking the zeppelin-web archive, to include a 
custom library in the index.html, but IMHO that seems to be a workaround:
https://github.com/beljun/zeppelin-plotly

Therefore I would suggest to add some zeppelin server wide configuration to 
allow zeppelin administrators to include additional 

Re: Starting to Prepare 0.8.0 RC1

[andreas . weise]

We run into a non-deterministic issue related to new TimeoutLifecycle Feature
https://issues.apache.org/jira/browse/ZEPPELIN-3435


On 2018/04/27 12:51:26, Jeff Zhang  wrote: 
> Thanks Jongyoul. I will hold it until it is fixed, seems it is a quick fix.
> 
> 
> 
> Jongyoul Lee 于2018年4月27日周五 上午11:57写道：
> 
> > Unfortunately, I might find a critical error while running invalid sql
> > query with SparkSqlInterpreter. I'll file up the issue and submit a PR soon
> >
> > On Fri, Apr 27, 2018 at 12:36 PM, Mina Lee  wrote:
> >
> >> Nice! Let me know if you need any help
> >>
> >> On Fri, 27 Apr 2018 at 12:17 PM Jun Kim  wrote:
> >>
> >> > Awesome!
> >> > 2018년 4월 27일 (금) 오전 10:15, Jongyoul Lee 님이 작성:
> >> >
> >> >> Sounds great!!
> >> >>
> >> >> On Fri, Apr 27, 2018 at 10:13 AM, Xiaohui Liu 
> >> wrote:
> >> >>
> >> >>> Fantastic!
> >> >>>
> >> >>> On Fri, 27 Apr 2018 at 9:12 AM, Jeff Zhang  wrote:
> >> >>>
> >>  Hi Folks,
> >> 
> >>  All the block issues are fixed, I am starting to prepare 0.8.0 RC1
> >> 
> >> 
> >> 
> >> >>
> >> >>
> >> >> --
> >> >> 이종열, Jongyoul Lee, 李宗烈
> >> >> http://madeng.net
> >> >>
> >> > --
> >> > Taejun Kim
> >> >
> >> > Data Mining Lab.
> >> > School of Electrical and Computer Engineering
> >> > University of Seoul
> >> >
> >>
> >
> >
> >
> > --
> > 이종열, Jongyoul Lee, 李宗烈
> > http://madeng.net
> >
> 

Re: "IPython is available, use IPython for PySparkInterpreter"

[andreas . weise]

I already filed an issue: Checkout 
https://issues.apache.org/jira/browse/ZEPPELIN-3290

Jeff Zhang wanted to wait for other users feedback there.

On 2018/03/19 18:10:48, Ruslan Dautkhanov  wrote: 
> We're getting " IPython is available, use IPython for PySparkInterpreter "
> warning each time we start %pyspark notebooks.
> 
> Although there is no difference between %pyspark and %ipyspark afaik.
> At least we can use all ipython magic commands etc.
> (maybe becase we have zeppelin.pyspark.useIPython=true?)
> 
> If that's the case, how we can disable "IPython is available, use IPython
> for PySparkInterpreter" warning ?
> 
> 
> -- 
> Ruslan Dautkhanov
> 

Notebook not running via REST API after Zeppelin Restart: Already known issue in master branch?

[Andreas Weise]

Hi,

I came across issues when using the Notebook Rest API.

The run operations simply do not work after after fresh restarts of
zeppelin.

https://issues.apache.org/jira/browse/ZEPPELIN-3105

Is this a known issue in latest master branch?

Cheers
Andreas

Subject: Re: Zeppelin %dev Interpreter

[Andreas Weise]

Thanks. After building with -Phelium-dev I was able to add dev interpreter.

One follow up question:
Now after running a helium application (e.g. Clock in zeppelin-examples)
via dev mode, I'm trying to figure out, how a helium application is started
without %dev interpreter.

While for Helium Visualizations I find addtional chart buttons, there is
nothing similar available for Helium Applications. Of course I find it
listed on the /helium Page, where I can enable/disable it. If enabled, I
can not see, how to actually use the application.

Thanks.
Andreas


> -- Forwarded message --
> From: moon soo Lee 
> To: users@zeppelin.apache.org
> Cc:
> Bcc:
> Date: Sat, 11 Mar 2017 01:31:52 +
> Subject: Re: Zeppelin %dev Interpreter
> Have you tried add -Phelium-dev profile in your maven build command?
>
> Thanks,
> moon
>
> On Fri, Mar 10, 2017 at 12:11 PM Andreas Weise 
> wrote:
>
>> Hi,
>>
>> various documentations describe the usage of %dev interpreter, e.g. when
>> developing Helium Applications.
>> https://zeppelin.apache.org/docs/0.7.0/development/
>> writingzeppelinapplication.html
>>
>> I'm wondering how the %dev Interpreter gets added to zeppelin a server.
>>
>> I already tried starting zeppelin as described here:
>> https://zeppelin.apache.org/docs/0.7.0/development/
>> howtocontribute.html#run-zeppelin-server-in-development-mode
>>
>>  But without luck. Same for starting an zeppelin-web instance on
>> http://localhost:9000
>>
>> Any advice.
>>
>> Thanks in advance.
>> Andreas
>>
>
>

Zeppelin %dev Interpreter

[Andreas Weise]

Hi,

various documentations describe the usage of %dev interpreter, e.g. when
developing Helium Applications.
https://zeppelin.apache.org/docs/0.7.0/development/writingzeppelinapplication.html

I'm wondering how the %dev Interpreter gets added to zeppelin a server.

I already tried starting zeppelin as described here:
https://zeppelin.apache.org/docs/0.7.0/development/howtocontribute.html#run-zeppelin-server-in-development-mode

 But without luck. Same for starting an zeppelin-web instance on
http://localhost:9000

Any advice.

Thanks in advance.
Andreas

Re: [CANCEL][VOTE] Release Apache Zeppelin 0.7.0 (RC3)

[Andreas Weise]

As there seems to be RC4, any chance to also get PR 1804 [1] merged for
0.7.0.

It fixes an important issue [2] when customizing the zeppelin start page.
Without the bugfix, it is not possible to use a custom homepage which
includes the notebook list.

Thanks.
Andreas

[1] https://github.com/apache/zeppelin/pull/1804
[2] https://issues.apache.org/jira/browse/ZEPPELIN-1578

zeppelin front-end extensibility

[Andreas Weise]

Hi,

This is about the UI extensibility of zeppelin.

I've read the Helium Proposal [1] and looked at the referenced helium
examples in github. As far as I see, helium is an notebook centric approach
- which means that only notebook functionality can be used.

Right now it seems very hard to extend zeppelin overall functionality
properly without hacking the core application, e.g. when adding additional
visualization types to the core [2]. What seems to be missing is a kind of
plugin concept for extending the front-end more easily.

In another project I have worked with camunda [3] which is a backend system
written in java (for executing business processes) that also contains a UI
application [4] written in js with angular (called Cockpit for
administering business processes). The camunda guys found a very neat
approach [5, 6] allowing developers to extend their cockpit application in
a modular manner and I wanted to share this with you, as it might be
adaptable for the zeppelin-web project. Parts of their plugin concept:

* client side extensions written in angular
* server side extensions written in java (exposing REST apis)
* single deployment artifact (jar/war) that can be referenced via maven (or
any other dependency management tool)
* clear file structure containing server & client side files
* well-defined extension points for extending the UI [7]
* priorities for sorting multiple plugins on the same extension point
* plugin store for sharing plugins with the community [8]

This approach could also be used to make the core artifact itself
(zeppelin-web) become more modular - e.g. extract existing chart types into
separate module.

Let me know what you think.

Cheers
Andreas

[1] http://madhukaudantha.blogspot.de/2015/08/introducing-new-chart-library-
and-types.html
[2] https://cwiki.apache.org/confluence/display/ZEPPELIN/Helium+proposal
[3] https://camunda.org/
[4] https://docs.camunda.org/manual/7.5/webapps/cockpit/
[5] https://docs.camunda.org/manual/7.5/webapps/cockpit/extend/plugins/
[6]
https://docs.camunda.org/manual/7.5/examples/tutorials/develop-cockpit-plugin/
[7] https://docs.camunda.org/manual/7.5/webapps/cockpit/
extend/plugins/#plugin-points
[8] https://camunda.org/plugins/