[GitHub] zeppelin pull request #2869: ZEPPELIN-3330. Add more test for RemoteInterpre...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/zeppelin/pull/2869


---

[jira] [Created] (ZEPPELIN-3338) [Umbrella] Add more unit test

[Jeff Zhang (JIRA)]

Jeff Zhang created ZEPPELIN-3338:


 Summary: [Umbrella] Add more unit test
 Key: ZEPPELIN-3338
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3338
 Project: Zeppelin
  Issue Type: Improvement
Reporter: Jeff Zhang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] zeppelin pull request #2871: ZEPPELIN-3337. Add more test to SparkRInterpret...

[zjffdu]

GitHub user zjffdu opened a pull request:

https://github.com/apache/zeppelin/pull/2871

ZEPPELIN-3337. Add more test to SparkRInterpreter

### What is this PR for?
Add more test for SparkRInterpreter, and also some code refactoring in 
SparkRInterpreter. 


### What type of PR is it?
[ Improvement | Refactoring]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3337

### How should this be tested?
* CI pass

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-3337

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2871.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2871


commit 11419a0b20ad46c674ce9ea10a19ea5e290e0b28
Author: Jeff Zhang 
Date:   2018-03-15T04:38:01Z

ZEPPELIN-3337. Add more test to SparkRInterpreter




---

[jira] [Created] (ZEPPELIN-3337) Add more test to SparkRInterpreter

[Jeff Zhang (JIRA)]

Jeff Zhang created ZEPPELIN-3337:


 Summary: Add more test to SparkRInterpreter
 Key: ZEPPELIN-3337
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3337
 Project: Zeppelin
  Issue Type: Improvement
Reporter: Jeff Zhang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] zeppelin issue #2412: [ZEPPELIN-2641] Change encoding to UTF-8 when sending ...

[big-loser]

Github user big-loser commented on the issue:

https://github.com/apache/zeppelin/pull/2412
  
@zjffdu 
I solved this problem and it is not actually a Zeppelin issue.
The main problem is that the contentType in the livy service http header 
has no charset option.

Livy file location:
`Vim server/src/main/scala/org/apache/livy/server/SessionServlet.scala +67`

`before() {
 // contentType = "application/json"
 contentType = "application/json;charset=UTF-8"
  }`

rebuild livy server


---

[GitHub] zeppelin pull request #2862: [ZEPPELIN-3321] isRevisionSupported not working...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/zeppelin/pull/2862


---

[jira] [Created] (ZEPPELIN-3336) Remove sending spark web url to interpreter setting page

[Jeff Zhang (JIRA)]

Jeff Zhang created ZEPPELIN-3336:


 Summary: Remove sending spark web url to interpreter setting page
 Key: ZEPPELIN-3336
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3336
 Project: Zeppelin
  Issue Type: Improvement
Reporter: Jeff Zhang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] zeppelin pull request #2870: [ZEPPELIN-3335] trim property name of interpret...

[rockiee281]

GitHub user rockiee281 opened a pull request:

https://github.com/apache/zeppelin/pull/2870

[ZEPPELIN-3335] trim property name of interpreter settings

### What is this PR for?
trim property name of interpreter settings

### What type of PR is it?
Improvement 

### Todos
* [ ] - Task

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3335


### How should this be tested?
* First time? Setup Travis CI as described on 
https://zeppelin.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed 
behavior
* Outline any manual steps to test the PR here.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rockiee281/zeppelin ZEPPELIN-3335

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2870.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2870


commit df2ce972907c62a697761e4591afcf3b19489499
Author: yunl.li 
Date:   2018-03-15T02:53:10Z

trim properties when setting

commit d4adec1ca946c1a6d9671422b7aad60784c5c345
Author: yunl.li 
Date:   2018-03-15T02:56:00Z

remove blank line




---

[jira] [Created] (ZEPPELIN-3335) trim property name of interpreter settings

[richard.lee (JIRA)]

richard.lee created ZEPPELIN-3335:
-

 Summary: trim property name of interpreter settings
 Key: ZEPPELIN-3335
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3335
 Project: Zeppelin
  Issue Type: Improvement
  Components: Interpreters
Reporter: richard.lee


When setting the properties of interpreter, users maybe input some invisible 
chars by accident, like 'spark.yarn.keytab\t', which will cause zeppelin cannot 
work as expected. We should trim the properties when create or update the 
interpreters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] zeppelin pull request #2869: ZEPPELIN-3330. Add more test for RemoteInterpre...

[zjffdu]

GitHub user zjffdu opened a pull request:

https://github.com/apache/zeppelin/pull/2869

ZEPPELIN-3330. Add more test for RemoteInterpreterServer

### What is this PR for?
This PR add more test for RemoteInterpreterServer for the purpose of 
further changes on RemoveInterpreterServer. Besides, it did some code 
refactoring here. 


### What type of PR is it?
[ Improvement ]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3330

### How should this be tested?
* CI pass

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-3330

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2869.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2869


commit 8e8c219e5fe445aee8e2abbbe49cf09a4a8603fa
Author: Jeff Zhang 
Date:   2018-03-14T08:05:50Z

ZEPPELIN-3330. Add more test for RemoteInterpreterServer




---

[GitHub] zeppelin pull request #2868: [HOTFIX] Code style fix for branch-0.8

[zjffdu]

GitHub user zjffdu opened a pull request:

https://github.com/apache/zeppelin/pull/2868

[HOTFIX] Code style fix for branch-0.8

### What is this PR for?

HotFix for code style issue of branch-0.8


### What type of PR is it?
[Hot Fix]

### Todos
* [ ] - Task

### What is the Jira issue?
* No jira created

### How should this be tested?
* CI pass

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zjffdu/zeppelin HOTFIX_JavaDoc

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2868.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2868


commit 69f3fc9d4af89bec63c6764b87af39026e68850b
Author: Jeff Zhang 
Date:   2018-03-14T23:35:47Z

[HOTFIX] Code sytle fix for branch-0.8




---

[GitHub] zeppelin issue #2684: [Zeppelin-2572] multiple paragraphs actions

[Tagar]

Github user Tagar commented on the issue:

https://github.com/apache/zeppelin/pull/2684
  
Just noticed this.. I think it's a great feature! 

In Jupyter we can also select multiple paragraphs.
Although it is made more seamless.. I believe you could Ctrl+click anywhere 
on a cell to select it.
So no separate UI element was necessary (like a checkbox that you have).
Selected cells are just slightly highlighted then. 
Two pros for this approach:
- easier to select multiple cells as you could click anywhere on a cell to 
add it to a selection;
- no additional UI elements 
- easier to find what cells were selected.



---

Re: Zeppelin - Spark Driver location

[Jeff Zhang]

spark-submit would only run when you run the first paragraph using spark
interpreter. After that, paragraph would send code to the spark app to
execute.

>>> Also spark standalone cluster moder should work even before this new
release, right?
I didn't verify that, not sure whether other people veryfit.


ankit jain 于2018年3月15日周四 上午4:32写道：

> Also spark standalone cluster moder should work even before this new
> release, right?
>
> On Wed, Mar 14, 2018 at 8:43 AM, ankit jain 
> wrote:
>
>> Hi Jhang,
>> Not clear on that - I thought spark-submit was done when we run a
>> paragraph, how does the .sh file come into play?
>>
>> Thanks
>> Ankit
>>
>> On Tue, Mar 13, 2018 at 5:43 PM, Jeff Zhang  wrote:
>>
>>>
>>> spark-submit is called in bin/interpreter.sh,  I didn't try standalone
>>> cluster mode. It is expected to run driver in separate host, but didn't
>>> guaranteed zeppelin support this.
>>>
>>> Ankit Jain 于2018年3月14日周三 上午8:34写道：
>>>
 Hi Jhang,
 What is the expected behavior with standalone cluster mode? Should we
 see separate driver processes in the cluster(one per user) or multiple
 SparkSubmit processes?

 I was trying to dig in Zeppelin code & didn’t see where Zeppelin does
 the Spark-submit to the cluster? Can you please point to it?

 Thanks
 Ankit

 On Mar 13, 2018, at 5:25 PM, Jeff Zhang  wrote:


 ZEPPELIN-2898  is
 for yarn cluster model.  And Zeppelin have integration test for yarn mode,
 so guaranteed it would work. But don't' have test for standalone, so not
 sure the behavior of standalone mode.


 Ruslan Dautkhanov 于2018年3月14日周三 上午8:06写道：

> https://github.com/apache/zeppelin/pull/2577 pronounces yarn-cluster
> in it's title so I assume it's only yarn-cluster.
> Never used standalone-cluster myself.
>
> Which distro of Hadoop do you use?
> Cloudera desupported standalone in CDH 5.5 and will remove in CDH 6.
>
> https://www.cloudera.com/documentation/enterprise/release-notes/topics/rg_deprecated.html
>
>
>
> --
> Ruslan Dautkhanov
>
> On Tue, Mar 13, 2018 at 5:45 PM, Jhon Anderson Cardenas Diaz <
> jhonderson2...@gmail.com> wrote:
>
>> Does this new feature work only for yarn-cluster ?. Or for spark
>> standalone too ?
>>
> El mar., 13 de mar. de 2018 18:34, Ruslan Dautkhanov <
>> dautkha...@gmail.com> escribió:
>>
> > Zeppelin version: 0.8.0 (merged at September 2017 version)
>>>
>>> https://issues.apache.org/jira/browse/ZEPPELIN-2898 was merged end
>>> of September so not sure if you have that.
>>>
>>> Check out
>>> https://medium.com/@zjffdu/zeppelin-0-8-0-new-features-ea53e8810235
>>> how to set this up.
>>>
>>>
>>> --
>>> Ruslan Dautkhanov
>>>
>>> On Tue, Mar 13, 2018 at 5:24 PM, Jhon Anderson Cardenas Diaz <
>>> jhonderson2...@gmail.com> wrote:
>>>
>> Hi zeppelin users !

 I am working with zeppelin pointing to a spark in standalone. I am
 trying to figure out a way to make zeppelin runs the spark driver 
 outside
 of client process that submits the application.

 According with the documentation (
 http://spark.apache.org/docs/2.1.1/spark-standalone.html):

 *For standalone clusters, Spark currently supports two deploy
 modes. In client mode, the driver is launched in the same process as 
 the
 client that submits the application. In cluster mode, however, the 
 driver
 is launched from one of the Worker processes inside the cluster, and 
 the
 client process exits as soon as it fulfills its responsibility of
 submitting the application without waiting for the application to 
 finish.*

 The problem is that, even when I set the properties for
 spark-standalone cluster and deploy mode in cluster, the driver still 
 run
 inside zeppelin machine (according with spark UI/executors page). 
 These are
 properties that I am setting for the spark interpreter:

 master: spark://:7077
 spark.submit.deployMode: cluster
 spark.executor.memory: 16g

 Any ideas would be appreciated.

 Thank you

 Details:
 Spark version: 2.1.1
 Zeppelin version: 0.8.0 (merged at September 2017 version)

>>>
>>
>>
>> --
>> Thanks & Regards,
>> Ankit.
>>
>
>
>
> --
> Thanks & Regards,
> Ankit.
>

[jira] [Created] (ZEPPELIN-3334) Set spark.scheduler.pool to authenticate user name

[Ruslan Dautkhanov (JIRA)]

Ruslan Dautkhanov created ZEPPELIN-3334:
---

 Summary: Set spark.scheduler.pool to authenticate user name
 Key: ZEPPELIN-3334
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3334
 Project: Zeppelin
  Issue Type: Improvement
Reporter: Ruslan Dautkhanov


Setting *spark.scheduler.pool* to authenticated user name would allow to have 
multiple resource pools for different users when using shared Spark context / 
shared Spark Interpreter;

This improvement request is for "The interpreter will be instantiated 
*Globally* in *shared* process" Spark Interpreter mode.
 
 Per Spark documentation, 
[https://spark.apache.org/docs/latest/job-scheduling.html] 
  
{quote}" _within_ each Spark application, multiple “jobs” (Spark actions) may 
be running concurrently if they were submitted by different threads 
 ... /skip/
 threads. By “job”, in this section, we mean a Spark action (e.g. {{save}}, 
{{collect}}) and any tasks that need to run to evaluate that action. Spark’s 
scheduler is fully thread-safe and supports this use case to enable 
applications that serve multiple requests (e.g. queries for multiple users).
 ... /skip/
 Without any intervention, newly submitted jobs go into a _default pool_, but 
jobs’ pools can be set by adding the {{*spark.scheduler.pool*}} “local 
property” to the SparkContext in the thread that’s submitting them.    "
{quote}
Notice that setting *spark.scheduler.pool* to authenticated user name has to be 
done *in a separate thread* - assuming Zeppelin internally has a separate 
thread for each separate authenticated user.. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Zeppelin - Spark Driver location

[ankit jain]

Also spark standalone cluster moder should work even before this new
release, right?

On Wed, Mar 14, 2018 at 8:43 AM, ankit jain  wrote:

> Hi Jhang,
> Not clear on that - I thought spark-submit was done when we run a
> paragraph, how does the .sh file come into play?
>
> Thanks
> Ankit
>
> On Tue, Mar 13, 2018 at 5:43 PM, Jeff Zhang  wrote:
>
>>
>> spark-submit is called in bin/interpreter.sh,  I didn't try standalone
>> cluster mode. It is expected to run driver in separate host, but didn't
>> guaranteed zeppelin support this.
>>
>> Ankit Jain 于2018年3月14日周三 上午8:34写道：
>>
>>> Hi Jhang,
>>> What is the expected behavior with standalone cluster mode? Should we
>>> see separate driver processes in the cluster(one per user) or multiple
>>> SparkSubmit processes?
>>>
>>> I was trying to dig in Zeppelin code & didn’t see where Zeppelin does
>>> the Spark-submit to the cluster? Can you please point to it?
>>>
>>> Thanks
>>> Ankit
>>>
>>> On Mar 13, 2018, at 5:25 PM, Jeff Zhang  wrote:
>>>
>>>
>>> ZEPPELIN-2898  is
>>> for yarn cluster model.  And Zeppelin have integration test for yarn mode,
>>> so guaranteed it would work. But don't' have test for standalone, so not
>>> sure the behavior of standalone mode.
>>>
>>>
>>> Ruslan Dautkhanov 于2018年3月14日周三 上午8:06写道：
>>>
 https://github.com/apache/zeppelin/pull/2577 pronounces yarn-cluster
 in it's title so I assume it's only yarn-cluster.
 Never used standalone-cluster myself.

 Which distro of Hadoop do you use?
 Cloudera desupported standalone in CDH 5.5 and will remove in CDH 6.
 https://www.cloudera.com/documentation/enterprise/release-
 notes/topics/rg_deprecated.html



 --
 Ruslan Dautkhanov

 On Tue, Mar 13, 2018 at 5:45 PM, Jhon Anderson Cardenas Diaz <
 jhonderson2...@gmail.com> wrote:

> Does this new feature work only for yarn-cluster ?. Or for spark
> standalone too ?
>
 El mar., 13 de mar. de 2018 18:34, Ruslan Dautkhanov <
> dautkha...@gmail.com> escribió:
>
 > Zeppelin version: 0.8.0 (merged at September 2017 version)
>>
>> https://issues.apache.org/jira/browse/ZEPPELIN-2898 was merged end
>> of September so not sure if you have that.
>>
>> Check out https://medium.com/@zjffdu/zeppelin-0-8-0-new-features-
>> ea53e8810235 how to set this up.
>>
>>
>> --
>> Ruslan Dautkhanov
>>
>> On Tue, Mar 13, 2018 at 5:24 PM, Jhon Anderson Cardenas Diaz <
>> jhonderson2...@gmail.com> wrote:
>>
> Hi zeppelin users !
>>>
>>> I am working with zeppelin pointing to a spark in standalone. I am
>>> trying to figure out a way to make zeppelin runs the spark driver 
>>> outside
>>> of client process that submits the application.
>>>
>>> According with the documentation (http://spark.apache.org/docs/
>>> 2.1.1/spark-standalone.html):
>>>
>>> *For standalone clusters, Spark currently supports two deploy modes.
>>> In client mode, the driver is launched in the same process as the client
>>> that submits the application. In cluster mode, however, the driver is
>>> launched from one of the Worker processes inside the cluster, and the
>>> client process exits as soon as it fulfills its responsibility of
>>> submitting the application without waiting for the application to 
>>> finish.*
>>>
>>> The problem is that, even when I set the properties for
>>> spark-standalone cluster and deploy mode in cluster, the driver still 
>>> run
>>> inside zeppelin machine (according with spark UI/executors page). These 
>>> are
>>> properties that I am setting for the spark interpreter:
>>>
>>> master: spark://:7077
>>> spark.submit.deployMode: cluster
>>> spark.executor.memory: 16g
>>>
>>> Any ideas would be appreciated.
>>>
>>> Thank you
>>>
>>> Details:
>>> Spark version: 2.1.1
>>> Zeppelin version: 0.8.0 (merged at September 2017 version)
>>>
>>
>
>
> --
> Thanks & Regards,
> Ankit.
>



-- 
Thanks & Regards,
Ankit.

[jira] [Created] (ZEPPELIN-3333) docker image for v0.7.3 embeds spark 2.1.0 instead of 2.2.0

[Mathieu DESPRIEE (JIRA)]

Mathieu DESPRIEE created ZEPPELIN-:
--

 Summary: docker image for v0.7.3 embeds spark 2.1.0 instead of 
2.2.0
 Key: ZEPPELIN-
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-
 Project: Zeppelin
  Issue Type: Bug
Affects Versions: 0.7.3
Reporter: Mathieu DESPRIEE


ZEPPELIN-2768 mentions that spark version is bumped to 2.2.0 in Zeppelin 
version 0.7.3, although when I run it, I get
{{spark.version
res13: String = 2.1.0}}

The docker image
{{
REPOSITORYTAG   
 IMAGE IDCREATED SIZE
apache/zeppelin   0.7.3 
 393fb49385105 months ago2.42GB
}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] zeppelin issue #2862: [ZEPPELIN-3321] isRevisionSupported not working for im...

[prabhjyotsingh]

Github user prabhjyotsingh commented on the issue:

https://github.com/apache/zeppelin/pull/2862
  
Thanks for the review will merge this if no more discussion.


---

Re: Zeppelin - Spark Driver location

[ankit jain]

Hi Jhang,
Not clear on that - I thought spark-submit was done when we run a
paragraph, how does the .sh file come into play?

Thanks
Ankit

On Tue, Mar 13, 2018 at 5:43 PM, Jeff Zhang  wrote:

>
> spark-submit is called in bin/interpreter.sh,  I didn't try standalone
> cluster mode. It is expected to run driver in separate host, but didn't
> guaranteed zeppelin support this.
>
> Ankit Jain 于2018年3月14日周三 上午8:34写道：
>
>> Hi Jhang,
>> What is the expected behavior with standalone cluster mode? Should we see
>> separate driver processes in the cluster(one per user) or multiple
>> SparkSubmit processes?
>>
>> I was trying to dig in Zeppelin code & didn’t see where Zeppelin does the
>> Spark-submit to the cluster? Can you please point to it?
>>
>> Thanks
>> Ankit
>>
>> On Mar 13, 2018, at 5:25 PM, Jeff Zhang  wrote:
>>
>>
>> ZEPPELIN-2898  is
>> for yarn cluster model.  And Zeppelin have integration test for yarn mode,
>> so guaranteed it would work. But don't' have test for standalone, so not
>> sure the behavior of standalone mode.
>>
>>
>> Ruslan Dautkhanov 于2018年3月14日周三 上午8:06写道：
>>
>>> https://github.com/apache/zeppelin/pull/2577 pronounces yarn-cluster in
>>> it's title so I assume it's only yarn-cluster.
>>> Never used standalone-cluster myself.
>>>
>>> Which distro of Hadoop do you use?
>>> Cloudera desupported standalone in CDH 5.5 and will remove in CDH 6.
>>> https://www.cloudera.com/documentation/enterprise/
>>> release-notes/topics/rg_deprecated.html
>>>
>>>
>>>
>>> --
>>> Ruslan Dautkhanov
>>>
>>> On Tue, Mar 13, 2018 at 5:45 PM, Jhon Anderson Cardenas Diaz <
>>> jhonderson2...@gmail.com> wrote:
>>>
 Does this new feature work only for yarn-cluster ?. Or for spark
 standalone too ?

>>> El mar., 13 de mar. de 2018 18:34, Ruslan Dautkhanov <
 dautkha...@gmail.com> escribió:

>>> > Zeppelin version: 0.8.0 (merged at September 2017 version)
>
> https://issues.apache.org/jira/browse/ZEPPELIN-2898 was merged end of
> September so not sure if you have that.
>
> Check out https://medium.com/@zjffdu/zeppelin-0-8-0-new-
> features-ea53e8810235 how to set this up.
>
>
> --
> Ruslan Dautkhanov
>
> On Tue, Mar 13, 2018 at 5:24 PM, Jhon Anderson Cardenas Diaz <
> jhonderson2...@gmail.com> wrote:
>
 Hi zeppelin users !
>>
>> I am working with zeppelin pointing to a spark in standalone. I am
>> trying to figure out a way to make zeppelin runs the spark driver outside
>> of client process that submits the application.
>>
>> According with the documentation (http://spark.apache.org/docs/
>> 2.1.1/spark-standalone.html):
>>
>> *For standalone clusters, Spark currently supports two deploy modes.
>> In client mode, the driver is launched in the same process as the client
>> that submits the application. In cluster mode, however, the driver is
>> launched from one of the Worker processes inside the cluster, and the
>> client process exits as soon as it fulfills its responsibility of
>> submitting the application without waiting for the application to 
>> finish.*
>>
>> The problem is that, even when I set the properties for
>> spark-standalone cluster and deploy mode in cluster, the driver still run
>> inside zeppelin machine (according with spark UI/executors page). These 
>> are
>> properties that I am setting for the spark interpreter:
>>
>> master: spark://:7077
>> spark.submit.deployMode: cluster
>> spark.executor.memory: 16g
>>
>> Any ideas would be appreciated.
>>
>> Thank you
>>
>> Details:
>> Spark version: 2.1.1
>> Zeppelin version: 0.8.0 (merged at September 2017 version)
>>
>


-- 
Thanks & Regards,
Ankit.

[GitHub] zeppelin pull request #2867: ZEPPELIN-3332

[felixalbani]

GitHub user felixalbani opened a pull request:

https://github.com/apache/zeppelin/pull/2867

ZEPPELIN-3332

Zeppelin login fails with NPE if ldapRealm.authorizationEnabled is not set 
true

Issue was introduced by pull https://github.com/apache/zeppelin/pull/2498

### What is this PR for?
A few sentences describing the overall goals of the pull request's commits.
First time? Check out the contributing guide - 
https://zeppelin.apache.org/contribution/contributions.html

### What type of PR is it?
Bug Fix

### Todos
* [ ] - Task

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3332

### How should this be tested?
* First time? Setup Travis CI as described on 
https://zeppelin.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed 
behavior
* Outline any manual steps to test the PR here.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/felixalbani/zeppelin ZEPPELIN-3332

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2867.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2867


commit ceb7b6f837cfe196fd1a9b51abbffe0c31ee7e71
Author: bag_of_tricks 
Date:   2018-03-14T14:44:25Z

ZEPPELIN-3332

Zeppelin login fails with NPE if ldapRealm.authorizationEnabled is not set 
true

Issue was introduced by pull https://github.com/apache/zeppelin/pull/2498




---

[jira] [Created] (ZEPPELIN-3332) Zeppelin login fails with NPE if ldapRealm.authorizationEnabled is not set true

[Felix (JIRA)]

Felix created ZEPPELIN-3332:
---

 Summary: Zeppelin login fails with NPE if 
ldapRealm.authorizationEnabled is not set true
 Key: ZEPPELIN-3332
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3332
 Project: Zeppelin
  Issue Type: Bug
  Components: security
Affects Versions: 0.8.0
Reporter: Felix


NPE is caused due the return value of org.apache.zeppelin.realm.LdapRealm 
method queryForAuthorizationInfo is null if authorizationEnabled = false (false 
is the default)
 
The null is not correctly handled by org.apache.zeppelin.utils.SecurityUtils 
and the login attempt always fails with:
 
 
{noformat}
Caused by: java.lang.NullPointerException
at org.apache.zeppelin.utils.SecurityUtils.getRoles(SecurityUtils.java:144) 
at org.apache.zeppelin.rest.LoginRestApi.postLogin(LoginRestApi.java:82) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
at 
org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
... 50 more{noformat}
 
Even when the authentication was successful and authorizationEnabled=false we 
are not able to login to the zeppelin ui. 
 
Issue was introduced by pull [https://github.com/apache/zeppelin/pull/2498]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/2839
  
It is integer type, so you can set it as large as java int can be. e.g. 
31536000 which is one year (365*24*60*60)


---

[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...

[MeethuM]

Github user MeethuM commented on the issue:

https://github.com/apache/zeppelin/pull/2839
  
@zjffdu When I gave blank value for the parameter, it didnt work..What  
will be the best value if we want the interpreter to live forever.


---

[GitHub] zeppelin pull request #2839: ZEPPELIN-2515. After 100 minutes R process quit...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/zeppelin/pull/2839


---

[GitHub] zeppelin pull request #2858: ZEPPELIN-3315. Merge beforeStatusChange and aft...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/zeppelin/pull/2858


---

[GitHub] zeppelin issue #2862: [ZEPPELIN-3321] isRevisionSupported not working for im...

[r-kamath]

Github user r-kamath commented on the issue:

https://github.com/apache/zeppelin/pull/2862
  
LGTM


---

[jira] [Created] (ZEPPELIN-3331) Use NullLifecycleManager by default

[Jeff Zhang (JIRA)]

Jeff Zhang created ZEPPELIN-3331:


 Summary: Use NullLifecycleManager by default
 Key: ZEPPELIN-3331
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3331
 Project: Zeppelin
  Issue Type: Improvement
Reporter: Jeff Zhang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ZEPPELIN-3330) Add more test for RemoteInterpreterServer

[Jeff Zhang (JIRA)]

Jeff Zhang created ZEPPELIN-3330:


 Summary: Add more test for RemoteInterpreterServer
 Key: ZEPPELIN-3330
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3330
 Project: Zeppelin
  Issue Type: Improvement
Reporter: Jeff Zhang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/2839
  
@MeethuM There's no limitation on that. 


---

[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...

[MeethuM]

Github user MeethuM commented on the issue:

https://github.com/apache/zeppelin/pull/2839
  
@zjffdu its working. spark.r.backendConnectionTimeout value is taken into 
consideration for the r interpreter to quit. Is there any maximum limit for 
this parameter? I dont want my R interpreter to quit at any point of time.


---

[GitHub] zeppelin issue #2862: [ZEPPELIN-3321] isRevisionSupported not working for im...

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/2862
  
LGTM


---

[GitHub] zeppelin issue #2863: Hotfix: ambiguous reference to overloaded definition

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/2863
  
LGTM 


---

[GitHub] zeppelin issue #2863: Hotfix: ambiguous reference to overloaded definition

[necosta]

Github user necosta commented on the issue:

https://github.com/apache/zeppelin/pull/2863
  
Green CI (https://travis-ci.org/nokia/zeppelin/builds/352915331)

Adding main contributors for review: @Leemoonsoo , @zjffdu 

Thanks


---

[GitHub] zeppelin pull request #2863: Hotfix: ambiguous reference to overloaded defin...

[necosta]

Github user necosta closed the pull request at:

https://github.com/apache/zeppelin/pull/2863


---

[GitHub] zeppelin pull request #2863: Hotfix: ambiguous reference to overloaded defin...

[necosta]

GitHub user necosta reopened a pull request:

https://github.com/apache/zeppelin/pull/2863

Hotfix: ambiguous reference to overloaded definition

### What is this PR for?
Minor bugfix on Zeppelin build from sources


### What type of PR is it?
Bug Fix

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3324

### How should this be tested?
* mvn clean package \
  --batch-mode package \
  -Phelium-dev \
  -Pscala-2.11 \
  -Dscala.version=2.11.8 \
  -Dscala.binary.version=2.11 \
  -Pbuild-distr \
  -Pspark-2.1 \
  -Dspark.version=2.1.1 \
  -Pr \
  -Pcassandra-spark-1.5 \
  -Psparkr \
  -Ppyspark \
  -Phadoop-2.7 \
  -Dhadoop.version=2.7.3 \
  -Dmaven.findbugs.enable=false \
  -Drat.skip=true \
  -Dcheckstyle.skip=true \
  -DskipTests

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? N
* Is there breaking changes for older versions? N
* Does this needs documentation? N


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nokia/zeppelin ZEPPELIN-3324

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2863.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2863


commit dfac740e1dac1b4d144ce41676597a34d4763264
Author: Nelson Costa 
Date:   2018-03-07T14:58:26Z

Hotfix: ambiguous reference to overloaded definition




---

[jira] [Created] (ZEPPELIN-3329) Need bind interpreters inside note after restarting Zeppelin

[Maxim Belousov (JIRA)]

Maxim Belousov created ZEPPELIN-3329:


 Summary: Need bind interpreters inside note after restarting 
Zeppelin 
 Key: ZEPPELIN-3329
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3329
 Project: Zeppelin
  Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Maxim Belousov


After each reload of Zeppelin when user open note he see interpreter bindings 
list.
It's to be good to avoid this.

Seems it's consequence of [ZEPPELIN-3322].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] zeppelin pull request #2866: ZEPPELIN-3328. Add plotting test for LivyInterp...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/zeppelin/pull/2866


---

[GitHub] zeppelin pull request #2866: ZEPPELIN-3328. Add plotting test for LivyInterp...

[zjffdu]

GitHub user zjffdu opened a pull request:

https://github.com/apache/zeppelin/pull/2866

ZEPPELIN-3328. Add plotting test for LivyInterpreter

### What is this PR for?
Add plotting for LivyInterpreter (Python & R)


### What type of PR is it?
[Improvement]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3328

### How should this be tested?
* CI pass

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-3328

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2866.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2866


commit 1b2a5e43f3561f94a7cfee147d6c41e81710a5a5
Author: Jeff Zhang 
Date:   2018-03-14T05:27:09Z

ZEPPELIN-3328. Add plotting test for LivyInterpreter




---