[CANCEL][VOTE] Release Apache Zeppelin 0.7.0 (RC3)

[Mina Lee]

There are several critical issues found during the rc3 verification.
I am canceling vote to include below bug fixes:
   - Cron scheduling doesn't work after certain number of running [1]
   - Table is not rendered in certain condition [2]
   - Livy not working in anonymous mode [3]
   - Interpreter install script throws "NoClassDefFoundError" [4]

[1] https://github.com/apache/zeppelin/pull/1941
[2] https://github.com/apache/zeppelin/pull/1942
[3] https://github.com/apache/zeppelin/pull/1938
[4] https://github.com/apache/zeppelin/pull/1943

On Wed, Jan 25, 2017 at 3:44 PM Jeff Zhang  wrote:

> And seems ZEPPELIN-1982 also need to be included in 0.7
>
>
> Jeff Zhang 于2017年1月25日周三 下午1:38写道：
>
> Please help also merge ZEPPELIN-2006 (
> https://github.com/apache/zeppelin/pull/1938) If there's another RC
>
>
> moon soo Lee 于2017年1月25日周三 上午11:22写道：
>
> Currently, cron scheduling feature is broken [1] and patch is available at
> [2].
> Shell we include this patch in 0.7.0 release?
>
> [1] http://issues.apache.org/jira/browse/ZEPPELIN-2009
> [2] https://github.com/apache/zeppelin/pull/1941
>
> On Tue, Jan 24, 2017 at 6:45 PM Prabhjyot Singh 
> wrote:
>
> +1
>
> On Jan 24, 2017 6:25 PM, "Ahyoung Ryu"  wrote:
>
> +1
>
> On Tue, Jan 24, 2017 at 8:19 PM, Hyung Sung Shim 
> wrote:
>
> +1
> 2017년 1월 24일 (화) 오전 1:36, Windy Qin 님이 작성:
>
>
> +1
> On 2017-01-24 17:13 (+0800), Sora Lee  wrote:
> > +1
> >
> > On Tue, Jan 24, 2017 at 3:09 PM Jongyoul Lee  wrote:
> >
> > > +1 (binding)
> > >
> > > On Tue, Jan 24, 2017 at 2:43 PM, Mina Lee  wrote:
> > >
> > > I propose the following RC to be released for the Apache Zeppelin 0.7.0
> > > release.
> > >
> > > The commit id is 48ad70e8c62975bdb00779bed5919eaca98c5b5d which is
> > > corresponds to the tag v0.7.0-rc3:
> > > *
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=48ad70e8c62975bdb00779bed5919eaca98c5b5d
> > > <
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=48ad70e8c62975bdb00779bed5919eaca98c5b5d
> >*
> > >
> > > The release archives (tgz), signature, and checksums are here
> > > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc3/
> > >
> > > The release candidate consists of the following source distribution
> > > archive
> > > zeppelin-0.7.0.tgz
> > >
> > > In addition, the following supplementary binary distributions are
> provided
> > > for user convenience at the same location
> > > zeppelin-0.7.0-bin-all.tgz
> > > zeppelin-0.7.0-bin-netinst.tgz
> > >
> > > The maven artifacts are here
> > >
> https://repository.apache.org/content/repositories/orgapachezeppelin-1024
> > >
> > > You can find the KEYS file here:
> > > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> > >
> > > Release notes available at
> > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12336544=12316221
> > >
> > > Vote will be open for next 72 hours (close at 22:00 26/Jan PST).
> > >
> > > [ ] +1 approve
> > > [ ] 0 no opinion
> > > [ ] -1 disapprove (and reason why)
> > >
> > >
> > >
> > >
> > > --
> > > 이종열, Jongyoul Lee, 李宗烈
> > > http://madeng.net
> > >
> >
>
>
>

Re: notebook interpreter restart

[Alec Lee]

Thanks a lot, we miss that setting, will do the experiments right away, :)

best


AL

> On Jan 25, 2017, at 6:17 PM, Paul Brenner  wrote:
> 
> 
> Did you try setting your interpreter to “Isolated” mode? Is it currently in 
> “shared” mode? 
> 
> If you haven’t played with this setting before then: 
> 1. Open the interpreters page
> 2. Find your interpreter and click the edit button in the top right corner
> 3. Beneath the word “option” at the top left there should be a drop down that 
> says “shared”, “scoped”, or “isolated”.
> 4. Set that drop down to isolated and then scroll down and click save.
> 
> By doing that, we get a separate YARN application for each notebook running 
> on the same interpreter.
> 
>    
> Paul Brenner 
>   
>     
>    
>  
> 
> DATA SCIENTIST
> (217) 390-3033  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
> 
> On Wed, Jan 25, 2017 at 9:12 PM Alec Lee mailto:Alec Lee 
> >> wrote:
> Hello, Paul 
> 
> Thank you so much for prompt reply. Our understanding is each interpreter 
> takes one application on YARN, if multiple notebooks share same interpreter 
> will run through same YARN application. 
> 
> As you mentioned this “… which will kill any other YARN application 
> associated with that interpreter.” , we seem never see more than one YARN 
> application in YARN UI, the case we had, I use one interpreter but wrote many 
> notebooks, but no matter how many notebooks spark jobs, I only see one YARN 
> application. So I am curious how to make single interpreter being associated 
> with more than YARN applications.
> 
> 
> Thanks
> 
> 
> AL
> 
> 
> 
> 
> 
>> On Jan 25, 2017, at 5:39 PM, Paul Brenner > > wrote:
>> 
>> 
>> Alec,
>> 
>> The way we use zeppelin at our company is to set our interpreters to 
>> “isolated”. That way each notebook gets it’s own application on yarn. 
>> 
>> This mostly works well. The one downside is that if you stop a notebook 
>> (e.g. by calling sys.exit in a cell of a spark notebook) it does stop the 
>> YARN application most of the time but you can’t restart that notebook until 
>> you have restarted the interpreter… which will kill any other YARN 
>> application associated with that interpreter.
>> 
>> So our full setup is that we give each user an interpreter (which is good 
>> because we can set each user’s interpreter to have their username in 
>> spark.yarn.queue) and set each user’s interpreter to isolated.
>> 
>> Honestly I still don’t understand what scoped does… maybe that would work as 
>> well?
>> 
>>    
>>    Paul Brenner 
>>   
>>     
>>    
>>  
>> 
>> DATA SCIENTIST
>> (217) 390-3033  
>> 
>>  
>> 

Re: notebook interpreter restart

[Paul Brenner]

Did you try setting your interpreter to “Isolated” mode? Is it currently in 
“shared” mode? 

If you haven’t played with this setting before then: 

1. Open the interpreters page

2. Find your interpreter and click the edit button in the top right corner

3. Beneath the word “option” at the top left there should be a drop down that 
says “shared”, “scoped”, or “isolated”.

4. Set that drop down to isolated and then scroll down and click save.

By doing that, we get a separate YARN application for each notebook running on 
the same interpreter.

http://www.placeiq.com/ http://www.placeiq.com/ http://www.placeiq.com/

Paul Brenner

https://twitter.com/placeiq https://twitter.com/placeiq 
https://twitter.com/placeiq
https://www.facebook.com/PlaceIQ https://www.facebook.com/PlaceIQ
https://www.linkedin.com/company/placeiq 
https://www.linkedin.com/company/placeiq

DATA SCIENTIST

(217) 390-3033 

 

http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 
http://placeiq.com/2016/03/08/measuring-addressable-tv-campaigns-is-now-possible/
 
http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/
 
http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/
 
http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/
 
http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/
 
http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/
 
http://pages.placeiq.com/Location-Data-Accuracy-Whitepaper-Download.html?utm_source=Signature_medium=Email_campaign=AccuracyWP
 
http://placeiq.com/2016/08/03/placeiq-bolsters-location-intelligence-platform-with-mastercard-insights/
 
http://placeiq.com/2016/10/26/the-making-of-a-location-data-industry-milestone/ 
http://placeiq.com/2016/12/07/placeiq-introduces-landmark-a-groundbreaking-offering-that-delivers-access-to-the-highest-quality-location-data-for-insights-that-fuel-limitless-business-decisions/

On Wed, Jan 25, 2017 at 9:12 PM Alec Lee

<
mailto:Alec Lee 
> wrote:

a, pre, code, a:link, body { word-wrap: break-word !important; }

Hello, Paul 

Thank you so much for prompt reply. Our understanding is each interpreter takes 
one application on YARN, if multiple notebooks share same interpreter will run 
through same YARN application. 

As you mentioned this “… which will kill any other YARN application associated 
with that interpreter.” , we seem never see more than one YARN application in 
YARN UI, the case we had, I use one interpreter but wrote many notebooks, but 
no matter how many notebooks spark jobs, I only see one YARN application. So I 
am curious how to make single interpreter being associated with more than YARN 
applications.

Thanks

AL

On Jan 25, 2017, at 5:39 PM, Paul Brenner <
mailto:pbren...@placeiq.com
> wrote:

Alec,

The way we use zeppelin at our company is to set our interpreters to 
“isolated”. That way each notebook gets it’s own application on yarn. 

This mostly works well. The one downside is that if you stop a notebook (e.g. 
by calling sys.exit in a cell of a spark notebook) it does stop the YARN 
application most of the time but you can’t restart that notebook until you have 
restarted the interpreter… which will kill any other YARN application 
associated with that interpreter.

So our full setup is that we give each user an interpreter (which is good 
because we can set each user’s interpreter to have their username in 
spark.yarn.queue) and set each user’s interpreter to isolated.

Honestly I still don’t understand what scoped does… maybe that would work as 
well?

http://www.placeiq.com/ http://www.placeiq.com/ http://www.placeiq.com/

Paul Brenner

https://twitter.com/placeiq https://twitter.com/placeiq 
https://twitter.com/placeiq
https://www.facebook.com/PlaceIQ https://www.facebook.com/PlaceIQ
https://www.linkedin.com/company/placeiq 
https://www.linkedin.com/company/placeiq

DATA SCIENTIST

(217) 390-3033 

 

http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 

Re: notebook interpreter restart

[Alec Lee]

Hello, Paul 

Thank you so much for prompt reply. Our understanding is each interpreter takes 
one application on YARN, if multiple notebooks share same interpreter will run 
through same YARN application. 

As you mentioned this “… which will kill any other YARN application associated 
with that interpreter.” , we seem never see more than one YARN application in 
YARN UI, the case we had, I use one interpreter but wrote many notebooks, but 
no matter how many notebooks spark jobs, I only see one YARN application. So I 
am curious how to make single interpreter being associated with more than YARN 
applications.


Thanks


AL





> On Jan 25, 2017, at 5:39 PM, Paul Brenner  wrote:
> 
> 
> Alec,
> 
> The way we use zeppelin at our company is to set our interpreters to 
> “isolated”. That way each notebook gets it’s own application on yarn. 
> 
> This mostly works well. The one downside is that if you stop a notebook (e.g. 
> by calling sys.exit in a cell of a spark notebook) it does stop the YARN 
> application most of the time but you can’t restart that notebook until you 
> have restarted the interpreter… which will kill any other YARN application 
> associated with that interpreter.
> 
> So our full setup is that we give each user an interpreter (which is good 
> because we can set each user’s interpreter to have their username in 
> spark.yarn.queue) and set each user’s interpreter to isolated.
> 
> Honestly I still don’t understand what scoped does… maybe that would work as 
> well?
> 
>    
> Paul Brenner 
>   
>     
>    
>  
> 
> DATA SCIENTIST
> (217) 390-3033  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
>  
> 
> 
> On Wed, Jan 25, 2017 at 8:20 PM Alec Lee mailto:Alec Lee 
> >> wrote:
> Hi, all 
> 
> 
> Currently we are exploring feature of zeppelin, now the situation we are 
> using YARN to manage spark jobs. In terms of the experiments, we conclude 
> that one interpreter is corresponding to An application in YARN cluster, that 
> means all the notebooks from zeppelin with same interpreter go through single 
> Application in YARN. Also we found if out code shuts down the application in 
> YARN, then any notebooks fail to run after this point - error like this 
> “can’t call a stop spark context …..” . The only solution for this is to 
> restart the interpreter. How to get around this without restart the 
> interpreter? 
> 
> 
> 
> Thanks 
> 
> 
> AL
> 

notebook interpreter restart

[Paul Brenner]

Alec,

The way we use zeppelin at our company is to set our interpreters to 
“isolated”. That way each notebook gets it’s own application on yarn. 

This mostly works well. The one downside is that if you stop a notebook (e.g. 
by calling sys.exit in a cell of a spark notebook) it does stop the YARN 
application most of the time but you can’t restart that notebook until you have 
restarted the interpreter… which will kill any other YARN application 
associated with that interpreter.

So our full setup is that we give each user an interpreter (which is good 
because we can set each user’s interpreter to have their username in 
spark.yarn.queue) and set each user’s interpreter to isolated.

Honestly I still don’t understand what scoped does… maybe that would work as 
well?

http://www.placeiq.com/ http://www.placeiq.com/ http://www.placeiq.com/

Paul Brenner

https://twitter.com/placeiq https://twitter.com/placeiq 
https://twitter.com/placeiq
https://www.facebook.com/PlaceIQ https://www.facebook.com/PlaceIQ
https://www.linkedin.com/company/placeiq 
https://www.linkedin.com/company/placeiq

DATA SCIENTIST

(217) 390-3033 

 

http://www.placeiq.com/2015/05/26/placeiq-named-winner-of-prestigious-2015-oracle-data-cloud-activate-award/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 
http://placeiq.com/2015/12/18/accuracy-vs-precision-in-location-data-mma-webinar/
 
http://placeiq.com/2016/03/08/measuring-addressable-tv-campaigns-is-now-possible/
 
http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/
 
http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/
 
http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/
 
http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/
 
http://placeiq.com/2016/04/13/placeiq-joins-the-network-advertising-initiative-nai-as-100th-member/
 
http://pages.placeiq.com/Location-Data-Accuracy-Whitepaper-Download.html?utm_source=Signature_medium=Email_campaign=AccuracyWP
 
http://placeiq.com/2016/08/03/placeiq-bolsters-location-intelligence-platform-with-mastercard-insights/
 
http://placeiq.com/2016/10/26/the-making-of-a-location-data-industry-milestone/ 
http://placeiq.com/2016/12/07/placeiq-introduces-landmark-a-groundbreaking-offering-that-delivers-access-to-the-highest-quality-location-data-for-insights-that-fuel-limitless-business-decisions/

On Wed, Jan 25, 2017 at 8:20 PM Alec Lee

<
mailto:Alec Lee 
> wrote:

a, pre, code, a:link, body { word-wrap: break-word !important; }

Hi, all

Currently we are exploring feature of zeppelin, now the situation we are using 
YARN to manage spark jobs. In terms of the experiments, we conclude that one 
interpreter is corresponding to An application in YARN cluster, that means all 
the notebooks from zeppelin with same interpreter go through single Application 
in YARN. Also we found if out code shuts down the application in YARN, then any 
notebooks fail to run after this point - error like this “can’t call a stop 
spark context …..” . The only solution for this is to restart the interpreter. 
How to get around this without restart the interpreter?

Thanks

AL

notebook interpreter restart

[Alec Lee]

Hi, all


Currently we are exploring feature of zeppelin, now the situation we are using 
YARN to manage spark jobs. In terms of the experiments, we conclude that one 
interpreter is corresponding to An application in YARN cluster, that means all 
the notebooks from zeppelin with same interpreter go through single Application 
in YARN. Also we found if out code shuts down the application in YARN, then any 
notebooks fail to run after this point -  error like this “can’t call a stop 
spark context …..” . The only solution for this is to restart the interpreter. 
How to get around this without restart the interpreter?



Thanks


AL

Re: Monitoring Zeppelin Health Via Rest API

[Jongyoul Lee]

I think it's good idea and we need add more beans to be monitored, too.

On Thu, Jan 26, 2017 at 9:45 AM, Vinay Shukla  wrote:

> How about launching Zeppelin JVM with JConsole related config and trying
> to monitor JVM level stats?
>
> On Wed, Jan 25, 2017 at 9:12 AM, Rob Anderson 
> wrote:
>
>> Ok, thanks for the reply Jongyoul.
>>
>> On Wed, Jan 25, 2017 at 12:13 AM, Jongyoul Lee 
>> wrote:
>>
>>> AFAIK, Zeppelin doesn't have it for now. We have to develop that
>>> function.
>>>
>>> Regards,
>>> Jongyoul
>>>
>>> On Wed, Jan 25, 2017 at 3:18 AM, Rob Anderson <
>>> rockclimbings...@gmail.com> wrote:
>>>
 Hello,

 We're running Zeppelin 0.6.2  and authenticating against Active
 Directory via Shiro.  Everything is working pretty well, however, we do
 occasionally have issues, which is leading to a bad user experience, as
 operationally we're unaware of a problem.

 We'd like to monitor the health of Zeppelin via the rest api, however,
 I don't see a way to programmatically authenticate, so we can make the
 calls.  Does anyone have any recommendations?

 Thanks,

 Rob

>>>
>>>
>>>
>>> --
>>> 이종열, Jongyoul Lee, 李宗烈
>>> http://madeng.net
>>>
>>
>>
>


-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net