[GitHub] zeppelin issue #3132: [ZEPPELIN-3701].Missing first several '0' and losing d...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/3132
  
this should be in 0.9. as for release date plan, please ask on 
dev@zeppelin.apache.org


---

[GitHub] zeppelin issue #3200: ZEPPELIN-3804. Separate log file for each interpreter ...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/3200
  
how to find out which log file is for which?


---

[jira] [Created] (ZEPPELIN-3806) Misleading x-axis visualisation when data has gaps

[Mikko Lipasti (JIRA)]

Mikko Lipasti created ZEPPELIN-3806:
---

 Summary: Misleading x-axis visualisation when data has gaps
 Key: ZEPPELIN-3806
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3806
 Project: Zeppelin
  Issue Type: Bug
  Components: GUI
Affects Versions: 0.8.0
 Environment: OSX, Chrome 69.0.3497.100, Zeppelin 0.8.0
Reporter: Mikko Lipasti
 Attachments: Bug.json

Given time-series data with gaps in the data (e.g. per-hour counts, some hours 
have no data), when plotted (especially when using scatterplot), gap in the 
data should be clearly visible as a gap at the missing hour.

Instead, Zeppelin visualisation completely ignores the missing data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ZEPPELIN-3807) Fix spark interpreter on Windows

[Alon Shoham (JIRA)]

Alon Shoham created ZEPPELIN-3807:
-

 Summary: Fix spark interpreter on Windows
 Key: ZEPPELIN-3807
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3807
 Project: Zeppelin
  Issue Type: Bug
 Environment: Windows
Reporter: Alon Shoham


# In ZEEPELIN-3111, changes were made to interpreter.sh script. These changes 
were not updated in interpreter.cmd script
 # In interprter.cmd, the sparkl submit commands passes custom dependencies via
{code:java}
--jars %CLASSPATH%{code}
The --jars flag should be changed to 
{code:java}
--driver-class-path %CLASSPATH%{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] zeppelin issue #2709: ZEPPELIN-3111. Refactor SparkInterpreter

[alonshoham]

Github user alonshoham commented on the issue:

https://github.com/apache/zeppelin/pull/2709
  
Hi I have created a JIRA iisue and corrected this locally. I cannot push my
local branch to github so I cannot create the PR.

How do i complete this process?

On Thu, Sep 27, 2018 at 5:02 AM Jeff Zhang  wrote:

> Thanks @alonshoham  feel free to create a
> PR for that.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---

Re: [DISCUSS] Is interpreter binding necessary ?

[andreas . weise]

Sorry I see a breaking change here: Or can you explain, how (from 0.9.0 on) 
users can change the default interpreter for existing notes, in case they only 
use short syntax (e.g. %spark) 

In 0.8.0 they are used to see all available intrepreters in the interpreter 
binding menu and move the default to the top of the list via drag and drop.

In current master branch the interpreter binding menu only shows the 
interpreters that are currently used. So with current state of implementation 
(master branch) end users are facing a breaking change when upgrading to 0.9.0.

Do you really want to drop support for changing default interpreter in existing 
notes (when short syntax is used)?



On 2018/09/26 22:33:58, Jongyoul Lee  wrote: 
> I think we can use default interpreter which would set when users create
> a note. It's a bit natural and won't break current behaviours
> 
> On Sat, Sep 22, 2018 at 8:16 PM, andreas.we...@gmail.com <
> andreas.we...@gmail.com> wrote:
> 
> > I agree with Jongyoul. Changing the default interpreter in an existing
> > note is vital, especially when a note has plenty of paragraphs.
> >
> > E.g. we are using spark interpreter per Org Unit and also for different
> > sizings.
> >
> > Our end users are using the respective Spark Interpreter just by setting
> > it to default and then using short syntax: %pyspark, %sql or %spark . After
> > some time, they want to change the default interpreter to use a different
> > Spark Executor Sizings or hand a note over to another Org Unit.
> >
> > In that situation changing full qualified names in ALL paragraphs is not
> > really feasible IMHO.
> >
> > As changing the default paragraph in existing notes is now still missing
> > in master branch, is there already a Jira Issue filed? I couldn't find it
> > in the umbrella story: https://issues.apache.org/jira/browse/ZEPPELIN-3594
> >
> > This issue is currently blocking us from early testing the master branch
> > with productive use cases.
> >
> > Thanks
> > Andreas
> >
> > On 2018/07/06 07:53:04, Jeff Zhang  wrote:
> > > We already allow setting default interpreter when creating note. Another
> > > way to set default interpreter is to reorder the interpreter setting
> > > binding in note page.
> > >
> > > But personally I don't recommend user to use short interpreter name
> > because
> > > of default interpreter. 2 Reaons:
> > > 1. It introduce in-accurate info. e.g. In our product, we have 2 spark
> > > interpreters (`spark`: for spark 1.x & `spark2` for spark 2.x).  Then
> > user
> > > often specify `%spark` for spark interpreter. But it could mean both
> > > `%spark.spark`  and `%spark2.spark`, So usually it is very hard to tell
> > > what's wrong when user expect to work spark2 but actually he still use
> > > spark 1.x. So usually we would recommend user to specify the full
> > qualified
> > > interpreter name. Just type several more characters which just cost 2
> > > seconds but make it more clear and readable.
> > > 2. Another issue is that interpreter binding is stored in
> > interpreter.json,
> > > that means if they export this note to another zeppelin instance, the
> > > default interpreter won't work.
> > >
> > > So I don't think setting default interpreter via interpreter binding is
> > > valuable for users. If user really want to do that, I would suggest to
> > > store it in note.json instead of interpreter.json
> > >
> > >
> > > Jongyoul Lee 于2018年7月6日周五 下午3:36写道：
> > >
> > > > There are two purposes of interpreter binding. One is what you
> > mentioned
> > > > and another one is to manage a default interpreter. If we provide a
> > new way
> > > > to set default interpreter, I think we can remove them :-) We could set
> > > > permissions in other ways.
> > > >
> > > > Overall, +1
> > > >
> > > > On Fri, Jul 6, 2018 at 4:24 PM, Jeff Zhang  wrote:
> > > >
> > > >> Hi Folks,
> > > >>
> > > >> I raise this thread to discuss whether we need the interpreter
> > binding.
> > > >> Currently when user create notes, they have to bind interpreters to
> > their
> > > >> notes in note page. Otherwise they will hit interpreter not found
> > issue.
> > > >> Besides that in zeppelin server side, we maintain the interpreter
> > binding
> > > >> info in memory as well as in interpreter.json.
> > > >>
> > > >> IMHO, it is not necessary to do interpreter binding. Because it just
> > add
> > > >> extra burden to maintain the interpreter binding info in zeppelin
> > server
> > > >> side, and doesn't introduce any benefits. The only benefit is that we
> > will
> > > >> check whether user have permission to use this interpreter, but
> > actually
> > > >> zeppelin will check the permission when running paragraph, so I don't
> > think
> > > >> we need to introduce interpreter binding just for this kind of
> > permission
> > > >> check that we will do later.
> > > >>
> > > >> So overall, I would suggest to remove interpreter binding feature.
> > What
> > > >> do you think ?
> > > >>
> > > >
> > > >
> > > >
> > > 

Re: [DISCUSS] Is interpreter binding necessary ?

[liuxun]

I also have some confusion about interpreter binding.  I have the following 
ideas:
Storage problem with interpreter configuration？
It is necessary to store the interpreter through the interpreter.json file. If 
there are multiple different zeppelin instances, we can unify the interpreter 
by copying the interpreter.json file.
Does the note require a default interpreter?
The default interpreter for each note binding, I find it easy to confuse users, 
users need to remember which note's default interpreter is what.
The note in the program can be bound to the default interpreter or not, which 
makes the code a bit more complicated.
I recommend that each note must specify an interpreter.
It takes only 2 seconds to specify the interpreter, so that the user is not 
confused, and it makes things clearer. I think it is more important.
Optimize how the interpreter is specified
The interpreter should be grouped by type(eg. spark, md ...). For example, the 
spark interpreter group contains all the interpreters for all spark1.x or 
spark2.x created by the system administrator. Different spark.yarn.queue in the 
spark interpreter of different user groups in our usage scenario requires 
different spark interpreters.
we can use a dynamic form to allow users to easily select different 
interpreters and parameters.
For example, select the spark by the drop-down box, select spark1.X, and then 
select %pyspark.



> 在 2018年10月3日，上午12:34，andreas.we...@gmail.com 写道：
> 
> Sorry I see a breaking change here: Or can you explain, how (from 0.9.0 on) 
> users can change the default interpreter for existing notes, in case they 
> only use short syntax (e.g. %spark) 
> 
> In 0.8.0 they are used to see all available intrepreters in the interpreter 
> binding menu and move the default to the top of the list via drag and drop.
> 
> In current master branch the interpreter binding menu only shows the 
> interpreters that are currently used. So with current state of implementation 
> (master branch) end users are facing a breaking change when upgrading to 
> 0.9.0.
> 
> Do you really want to drop support for changing default interpreter in 
> existing notes (when short syntax is used)?
> 
> 
> 
> On 2018/09/26 22:33:58, Jongyoul Lee  wrote: 
>> I think we can use default interpreter which would set when users create
>> a note. It's a bit natural and won't break current behaviours
>> 
>> On Sat, Sep 22, 2018 at 8:16 PM, andreas.we...@gmail.com <
>> andreas.we...@gmail.com> wrote:
>> 
>>> I agree with Jongyoul. Changing the default interpreter in an existing
>>> note is vital, especially when a note has plenty of paragraphs.
>>> 
>>> E.g. we are using spark interpreter per Org Unit and also for different
>>> sizings.
>>> 
>>> Our end users are using the respective Spark Interpreter just by setting
>>> it to default and then using short syntax: %pyspark, %sql or %spark . After
>>> some time, they want to change the default interpreter to use a different
>>> Spark Executor Sizings or hand a note over to another Org Unit.
>>> 
>>> In that situation changing full qualified names in ALL paragraphs is not
>>> really feasible IMHO.
>>> 
>>> As changing the default paragraph in existing notes is now still missing
>>> in master branch, is there already a Jira Issue filed? I couldn't find it
>>> in the umbrella story: https://issues.apache.org/jira/browse/ZEPPELIN-3594
>>> 
>>> This issue is currently blocking us from early testing the master branch
>>> with productive use cases.
>>> 
>>> Thanks
>>> Andreas
>>> 
>>> On 2018/07/06 07:53:04, Jeff Zhang  wrote:
 We already allow setting default interpreter when creating note. Another
 way to set default interpreter is to reorder the interpreter setting
 binding in note page.
 
 But personally I don't recommend user to use short interpreter name
>>> because
 of default interpreter. 2 Reaons:
 1. It introduce in-accurate info. e.g. In our product, we have 2 spark
 interpreters (`spark`: for spark 1.x & `spark2` for spark 2.x).  Then
>>> user
 often specify `%spark` for spark interpreter. But it could mean both
 `%spark.spark`  and `%spark2.spark`, So usually it is very hard to tell
 what's wrong when user expect to work spark2 but actually he still use
 spark 1.x. So usually we would recommend user to specify the full
>>> qualified
 interpreter name. Just type several more characters which just cost 2
 seconds but make it more clear and readable.
 2. Another issue is that interpreter binding is stored in
>>> interpreter.json,
 that means if they export this note to another zeppelin instance, the
 default interpreter won't work.
 
 So I don't think setting default interpreter via interpreter binding is
 valuable for users. If user really want to do that, I would suggest to
 store it in note.json instead of interpreter.json
 
 
 Jongyoul Lee 于2018年7月6日周五 下午3:36写道：
 
> There are two purposes of interprete

[GitHub] zeppelin pull request #3201: [ZEPPELIN-3805] Don't distribution shade jar to...

[liuxunorg]

GitHub user liuxunorg opened a pull request:

https://github.com/apache/zeppelin/pull/3201

[ZEPPELIN-3805] Don't distribution shade jar to lib/interpreter for 
zeppelin-interpreter

The zeppelin-interpreter-api module shades the conflicting package, Need to 
remove the package that issued these shades in zeppelin-distribution.

[Bug Fix]

* [x] Exclude shade JAR packages to lib/interpreter 
* [x] Add zeppelin-interpreter-api JAR package to lib/interpreter 

* https://issues.apache.org/jira/browse/ZEPPELIN-3805

[CI pass](https://travis-ci.org/liuxunorg/zeppelin/builds/436144528)

* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/liuxunorg/zeppelin ZEPPELIN-3805

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/3201.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3201


commit 5f073baaccacc913c1b925f14d67aa82d2528774
Author: liuxunorg <33611720@...>
Date:   2018-10-02T13:35:15Z

[ZEPPELIN-3805] Don't distribution shade jar to lib/interpreter for 
zeppelin-interpreter

The zeppelin-interpreter-api module shades the conflicting package, Need to 
remove the package that issued these conflicts in zeppelin-distribution.

[Bug Fix]

* [x] Exclude shade JAR packages
* [x] Add zeppelin-interpreter-api JAR package

* https://issues.apache.org/jira/browse/ZEPPELIN-3805

CI pass

* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No




---

[GitHub] zeppelin issue #3200: ZEPPELIN-3804. Separate log file for each interpreter ...

[liuxunorg]

Github user liuxunorg commented on the issue:

https://github.com/apache/zeppelin/pull/3200
  
Split logs are needed, Is it a separate log file for each interpreter 
process and user?
Display it in the Interpreter Process Monitoring 
Page(https://jira.apache.org/jira/browse/ZEPPELIN-3802) by looking up all log 
files that match the $InterpreterSettingName name, right?


---

[GitHub] zeppelin issue #2709: ZEPPELIN-3111. Refactor SparkInterpreter

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/2709
  
@alonshoham Sorry I didn't notice you created the ticket, I have fixed via 
this commit, 
https://github.com/apache/zeppelin/commit/8b2f4a7bd89aa69661e09ef9ad73c04d862029ff

Could you help verity it ?


---

[GitHub] zeppelin issue #3200: ZEPPELIN-3804. Separate log file for each interpreter ...

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/3200
  
That's right @liuxunorg  This is my plan, I'd like to show the log in the 
interpreter process monitoring page. 


---