Any reason why PostgreSQL is not merged into jdbc interpreter

[Jeff Zhang]

Is there anything specific for postgreSQL so that zeppelin didn't merge it
into the general jdbc interpreter ?


-- 
Best Regards

Jeff Zhang

Re: Issues in Zeppelin 0.6.0

[Jeff Zhang]

Hi

Since you have many issues, let's focus one issue first.

>>> *not able to use the HiveContext to read the Hive table*


Can you paste your code of how you use HiveContext ? Do you create it by
yourself ? It should be created by zeppelin, so you don't need to create it.
What's  in interpreter log ?



On Thu, Aug 18, 2016 at 7:35 PM, Nagasravanthi, Valluri <
valluri.nagasravan...@pfizer.com> wrote:

> Hi,
>
>
>
> I am using Zeppelin 0.6.0. Please find below the issues along with their
> detailed explanation.
>
>
>
> *Zeppelin 0.6.0 Issues:*
>
> a.   *not able to execute DDL statements like Create/Drop tables
> using temptables derived from the hive table*
>
> · Error Log:* “*java.lang.RuntimeException: [1.1] failure:
> ``with'' expected but identifier drop found : When using sql interpreter to
> drop*”*
>
>
>
> b.  *not able to use the HiveContext to read the Hive table*
>
> · Error Log: *“*error: object HiveContext in package hive cannot
> be accessed in package org.apache.spark.sql.hive*”*
>
>
>
> *Detailed Explanation:*
>
> I upgraded to 0.6.0 from Zeppelin 0.5.6 last week. I am facing some issues
> while using notebooks on 0.6.0. I am using Ambari 2.4.2 as my Cluster
> Manager and Spark version is 1.6.
>
>
>
> The workflow of notebook is as follows:
>
> 1.   Create a spark scala dataframe by reading a hive table in
> parquet/text format using sqlContext (sqlContext.read.parquet(“/
> tablelocation/tablename”)
>
> 2.   Import sqlcontext_implicits
>
> 3.   Register the dataframe as a temp table
>
> 4.   Write queries using %sql interpreter or sqlContext.sql
>
>
>
> The issue which I am facing right now is that Even though I am able to
> execute “SELECT”  query on the temptables but *I am not able to execute
> DDL statements like Create/Drop tables using temptables derived from the
> hive table.  *
>
> Following is my code:
>
> 1st case:  sqlContext.sql(“drop if exists tablename”)
>
> 2nd case: %sql
>
>   drop if exists tablename
>
>
>
> I am getting the same error for both the cases:
> java.lang.RuntimeException: [1.1] failure: ``with'' expected but identifier
> drop found : When using sql interpreter to drop
>
>
>
> It is to be noted that, the same code used to work in Zeppelin 0.5.6.
>
>
>
> After researching a bit, I came across that I need to use HiveContext to
> query hive table.
>
>
>
> The second issue which I am facing is I was able to import HiveContext
> using “import org.apache.spark.sql.hive.HiveContext” *but I was not able
> to use the HiveContext to read the Hive table.*
>
>
>
> This is the code which I wrote :
>
> (HiveContext.read.parquet(“/tablelocation/tablename”)
>
>
>
> I got the following error:
>
> error: object HiveContext in package hive cannot be accessed in package
> org.apache.spark.sql.hive
>
>
>
> I am not able to deep dive into this error as there is not much support
> online.
>
>
>
> Could anyone please suggest any fix for the errors ?
>
>
>
> Thanks and Regards,
>
> **
>
> *[image: Description: cid:image001.png@01D1EBF4.36D373B0]*
>
> *Valluri Naga Sravanthi| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com* *; *
> *valluri.nagasravan...@pfizer.com* 
>
> *Website**: **http://pfizerWorks.pfizer.com*
> <http://pfizerworks.pfizer.com/>
>
>
>
> *……*
>
>
>



-- 
Best Regards

Jeff Zhang

Re: Issues in Zeppelin 0.6.0

[Jeff Zhang]

It should be "hiveContext" instead of "HiveContext"

On Fri, Aug 19, 2016 at 3:33 PM, Sundararajan, Pranav <
pranav.sundarara...@pfizer.com> wrote:

> Hi,
>
>
>
> PFB the code.
>
> import org.apache.spark.sql.hive.HiveContext
>
> val df_test_hive = HiveContext.read.parquet("/location/hivefile ")
>
>
>
>
>
> We are getting the following error in the log:
>
> import org.apache.spark.sql.hive.HiveContext
>
> :57: error: object HiveContext in package hive cannot be accessed
> in package org.apache.spark.sql.hive
>
> val df_test_hive = HiveContext.read.parquet("/location/hivefile”)
>
>
>
>
>
> PFB the embedded image of hive interpreter settings:
>
>
>
>
>
>
>
> Regards,
>
>
>
>
>
>
>
> [image: Description: cid:image001.png@01D1C173.60CDE0C0]
>
> *Pranav Sundararajan| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com **; 
> **pranav.sundarara...@pfizer.com
> *
>
> *Website**: **http://pfizerWorks.pfizer.com
> <http://pfizerworks.pfizer.com/>*
>
>
>
>
>
>
>
> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
> *Sent:* Thursday, August 18, 2016 8:31 PM
> *To:* dev@zeppelin.apache.org
> *Cc:* d...@zeppelin.incubator.apache.org; us...@zeppelin.apache.org;
> Sundararajan, Pranav; Jaisankar, Saurabh
> *Subject:* Re: Issues in Zeppelin 0.6.0
>
>
>
> Hi
>
>
>
> Since you have many issues, let's focus one issue first.
>
>
>
> >>> *not able to use the HiveContext to read the Hive table*
>
>
>
>
>
> Can you paste your code of how you use HiveContext ? Do you create it by
> yourself ? It should be created by zeppelin, so you don't need to create it.
>
> What's  in interpreter log ?
>
>
>
>
>
>
>
> On Thu, Aug 18, 2016 at 7:35 PM, Nagasravanthi, Valluri <
> valluri.nagasravan...@pfizer.com> wrote:
>
> Hi,
>
>
>
> I am using Zeppelin 0.6.0. Please find below the issues along with their
> detailed explanation.
>
>
>
> *Zeppelin 0.6.0 Issues:*
>
> a.   *not able to execute DDL statements like Create/Drop tables
> using temptables derived from the hive table*
>
> · Error Log:* “*java.lang.RuntimeException: [1.1] failure:
> ``with'' expected but identifier drop found : When using sql interpreter to
> drop*”*
>
>
>
> b.  *not able to use the HiveContext to read the Hive table*
>
> · Error Log: *“*error: object HiveContext in package hive cannot
> be accessed in package org.apache.spark.sql.hive*”*
>
>
>
> *Detailed Explanation:*
>
> I upgraded to 0.6.0 from Zeppelin 0.5.6 last week. I am facing some issues
> while using notebooks on 0.6.0. I am using Ambari 2.4.2 as my Cluster
> Manager and Spark version is 1.6.
>
>
>
> The workflow of notebook is as follows:
>
> 1.   Create a spark scala dataframe by reading a hive table in
> parquet/text format using sqlContext (sqlContext.read.parquet(“/
> tablelocation/tablename”)
>
> 2.   Import sqlcontext_implicits
>
> 3.   Register the dataframe as a temp table
>
> 4.   Write queries using %sql interpreter or sqlContext.sql
>
>
>
> The issue which I am facing right now is that Even though I am able to
> execute “SELECT”  query on the temptables but *I am not able to execute
> DDL statements like Create/Drop tables using temptables derived from the
> hive table.  *
>
> Following is my code:
>
> 1st case:  sqlContext.sql(“drop if exists tablename”)
>
> 2nd case: %sql
>
>   drop if exists tablename
>
>
>
> I am getting the same error for both the cases:
> java.lang.RuntimeException: [1.1] failure: ``with'' expected but identifier
> drop found : When using sql interpreter to drop
>
>
>
> It is to be noted that, the same code used to work in Zeppelin 0.5.6.
>
>
>
> After researching a bit, I came across that I need to use HiveContext to
> query hive table.
>
>
>
> The second issue which I am facing is I was able to import HiveContext
> using “import org.apache.spark.sql.hive.HiveContext” *but I was not able
> to use the HiveContext to read the Hive table.*
>
>
>
> This is the code which I wrote :
>
> (HiveContext.read.parquet(“/tablelocation/tablename”)
>
>
>
> I got the following error:
>
> error: object HiveContext in package hive cannot be accessed in package
> org.apache.spark.sql.hive
>
>
>
> I am not able to deep dive into this error as there is not much support
> online.
>
>
>
> Could anyone please suggest any fix for the errors ?
>
>
>
> Thanks and Regards,
>
> **
>
> *Valluri Naga Sravanthi| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com* *; *
> *valluri.nagasravan...@pfizer.com* 
>
> *Website**: **http://pfizerWorks.pfizer.com*
> <http://pfizerworks.pfizer.com/>
>
>
>
> *……*
>
>
>
>
>
>
>
> --
>
> Best Regards
>
> Jeff Zhang
>



-- 
Best Regards

Jeff Zhang

Re: Issues in Zeppelin 0.6.0

[Jeff Zhang]

Sorry, you should use sqlContext

On Fri, Aug 19, 2016 at 6:11 PM, Sundararajan, Pranav <
pranav.sundarara...@pfizer.com> wrote:

> Hi,
>
>
>
> We tried “hiveContext” instead of “HiveContext” but it didn’t work out.
>
> Please find below the code and the error log:
>
>
>
> *import org.apache.spark.sql.hive.HiveContext*
>
> *val df_test_hive = hiveContext.read.parquet("/location/hivefile")*
>
>
>
> *import org.apache.spark.sql.hive.HiveContext*
>
> *:67: error: not found: value hiveContext*
>
> *val df_test_hive = hiveContext.read.parquet("/location/hivefile")*
>
>
>
> It seems hiveContext is not an object of HiveContext package.
>
> Could you please suggest if there is any other way out?
>
>
>
>
>
> Regards,
>
>
>
>
>
>
>
> [image: Description: cid:image001.png@01D1C173.60CDE0C0]
>
> *Pranav Sundararajan| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com **; 
> **pranav.sundarara...@pfizer.com
> *
>
> *Website**: **http://pfizerWorks.pfizer.com
> <http://pfizerworks.pfizer.com/>*
>
>
>
>
>
>
>
> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
> *Sent:* Friday, August 19, 2016 3:35 AM
> *To:* Sundararajan, Pranav
> *Cc:* dev@zeppelin.apache.org; d...@zeppelin.incubator.apache.org;
> us...@zeppelin.apache.org; Jaisankar, Saurabh
>
> *Subject:* Re: Issues in Zeppelin 0.6.0
>
>
>
> It should be "hiveContext" instead of "HiveContext"
>
>
>
> On Fri, Aug 19, 2016 at 3:33 PM, Sundararajan, Pranav <
> pranav.sundarara...@pfizer.com> wrote:
>
> Hi,
>
>
>
> PFB the code.
>
> import org.apache.spark.sql.hive.HiveContext
>
> val df_test_hive = HiveContext.read.parquet("/location/hivefile ")
>
>
>
>
>
> We are getting the following error in the log:
>
> import org.apache.spark.sql.hive.HiveContext
>
> :57: error: object HiveContext in package hive cannot be accessed
> in package org.apache.spark.sql.hive
>
> val df_test_hive = HiveContext.read.parquet("/location/hivefile”)
>
>
>
>
>
> PFB the embedded image of hive interpreter settings:
>
>
>
>
>
>
>
> Regards,
>
>
>
>
>
>
>
> [image: Description: cid:image001.png@01D1C173.60CDE0C0]
>
> *Pranav Sundararajan| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com **; 
> **pranav.sundarara...@pfizer.com
> *
>
> *Website**: **http://pfizerWorks.pfizer.com
> <http://pfizerworks.pfizer.com/>*
>
>
>
>
>
>
>
> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
> *Sent:* Thursday, August 18, 2016 8:31 PM
> *To:* dev@zeppelin.apache.org
> *Cc:* d...@zeppelin.incubator.apache.org; us...@zeppelin.apache.org;
> Sundararajan, Pranav; Jaisankar, Saurabh
> *Subject:* Re: Issues in Zeppelin 0.6.0
>
>
>
> Hi
>
>
>
> Since you have many issues, let's focus one issue first.
>
>
>
> >>> *not able to use the HiveContext to read the Hive table*
>
>
>
>
>
> Can you paste your code of how you use HiveContext ? Do you create it by
> yourself ? It should be created by zeppelin, so you don't need to create it.
>
> What's  in interpreter log ?
>
>
>
>
>
>
>
> On Thu, Aug 18, 2016 at 7:35 PM, Nagasravanthi, Valluri <
> valluri.nagasravan...@pfizer.com> wrote:
>
> Hi,
>
>
>
> I am using Zeppelin 0.6.0. Please find below the issues along with their
> detailed explanation.
>
>
>
> *Zeppelin 0.6.0 Issues:*
>
> a.   *not able to execute DDL statements like Create/Drop tables
> using temptables derived from the hive table*
>
> · Error Log:* “*java.lang.RuntimeException: [1.1] failure:
> ``with'' expected but identifier drop found : When using sql interpreter to
> drop*”*
>
>
>
> b.  *not able to use the HiveContext to read the Hive table*
>
> · Error Log: *“*error: object HiveContext in package hive cannot
> be accessed in package org.apache.spark.sql.hive*”*
>
>
>
> *Detailed Explanation:*
>
> I upgraded to 0.6.0 from Zeppelin 0.5.6 last week. I am facing some issues
> while using notebooks on 0.6.0. I am using Ambari 2.4.2 as my Cluster
> Manager and Spark version is 1.6.
>
>
>
> The workflow of notebook is as follows:
>
> 1.   Create a spark scala dataframe by reading a hive table in
> parquet/text format using sqlContext (sqlContext.read.parquet(“/
> tablelocation/tablename”)
>

Re: Issues in Zeppelin 0.6.0

[Jeff Zhang]

Do you change spark version when upgrading zeppelin ?

On Fri, Aug 19, 2016 at 7:36 PM, Nagasravanthi, Valluri <
valluri.nagasravan...@pfizer.com> wrote:

>
>
> Hi,
>
>
>
> Actually, when we worked on Zeppelin 0.5.6, we used to use sqlContext
> itself for reading hive files, registering spark dataframes as temp tables
> and eventualy processing queries on the temp tables.
>
>
>
> The issue which I am facing on Zeppelin 0.6.0 right now  is that even
> though I am able to execute “SELECT”  query on the temptables but *I am
> not able to execute DDL statements like Create/Drop tables using temptables
> derived from the hive files.  *
>
>
>
> *To elaborate it further:*
>
> When I write the following codes, they work fine. PFB the sample codes:
>
>- sqlContext.sql(“select * from table name”)
>- sqlContext.sql(“Select count(column_name) from table_name”)
>
> But when I write the following codes, it fails to execute:
>
>- sqlContext.sql(“create tablename as select * from tablename_1”)
>- sqlContext.sql(“drop if exists tablename”)
>
>
>
> I am getting the error : *java.lang.RuntimeException: [1.1] failure:
> ``with'' expected but identifier drop found :*
>
> I even tried using sql interpreter.
>
>
>
> *PFB the code for it:*
>
> %sql
>
> Drop if exists tablename
>
> I am getting the same error again: : *java.lang.RuntimeException: [1.1]
> failure: ``with'' expected but identifier drop found :*
>
>
>
> Thanks a lot for being so responsive. Hope to fix this issue soon with
> your help.
>
>
>
> Thanks and Regards,
>
> **
>
> *[image: Description: cid:image001.png@01D1EBF4.36D373B0]*
>
> *Valluri Naga Sravanthi| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com* *; *
> *valluri.nagasravan...@pfizer.com* 
>
> *Website**: **http://pfizerWorks.pfizer.com*
> <http://pfizerworks.pfizer.com/>
>
>
>
> *……*
>
>
>
>
>
> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
>
>
>
>
> *Sent:* Friday, August 19, 2016 6:49 AM
> *To:* Sundararajan, Pranav
> *Cc:* dev@zeppelin.apache.org; d...@zeppelin.incubator.apache.org;
> us...@zeppelin.apache.org; Jaisankar, Saurabh; Nagasravanthi, Valluri
>
> *Subject:* Re: Issues in Zeppelin 0.6.0
>
>
>
> Sorry, you should use sqlContext
>
>
>
> On Fri, Aug 19, 2016 at 6:11 PM, Sundararajan, Pranav <
> pranav.sundarara...@pfizer.com> wrote:
>
> Hi,
>
>
>
> We tried “hiveContext” instead of “HiveContext” but it didn’t work out.
>
> Please find below the code and the error log:
>
>
>
> *import org.apache.spark.sql.hive.HiveContext*
>
> *val df_test_hive = hiveContext.read.parquet("/location/hivefile")*
>
>
>
> *import org.apache.spark.sql.hive.HiveContext*
>
> *:67: error: not found: value hiveContext*
>
> *val df_test_hive = hiveContext.read.parquet("/location/hivefile")*
>
>
>
> It seems hiveContext is not an object of HiveContext package.
>
> Could you please suggest if there is any other way out?
>
>
>
>
>
> Regards,
>
>
>
>
>
>
>
> [image: Description: cid:image001.png@01D1C173.60CDE0C0]
>
> *Pranav Sundararajan| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com **; 
> **pranav.sundarara...@pfizer.com
> *
>
> *Website**: **http://pfizerWorks.pfizer.com
> <http://pfizerworks.pfizer.com/>*
>
>
>
>
>
>
>
> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
> *Sent:* Friday, August 19, 2016 3:35 AM
> *To:* Sundararajan, Pranav
> *Cc:* dev@zeppelin.apache.org; d...@zeppelin.incubator.apache.org;
> us...@zeppelin.apache.org; Jaisankar, Saurabh
>
>
> *Subject:* Re: Issues in Zeppelin 0.6.0
>
>
>
> It should be "hiveContext" instead of "HiveContext"
>
>
>
> On Fri, Aug 19, 2016 at 3:33 PM, Sundararajan, Pranav <
> pranav.sundarara...@pfizer.com> wrote:
>
> Hi,
>
>
>
> PFB the code.
>
> import org.apache.spark.sql.hive.HiveContext
>
> val df_test_hive = HiveContext.read.parquet("/location/hivefile ")
>
>
>
>
>
> We are getting the following error in the log:
>
> import org.apache.spark.sql.hive.HiveContext
>
> :57: error: object HiveContext in package hive cannot be accessed
> in package org.apache.spark.sql.hive
>
> val df_test_hive = Hive

Re: Issues in Zeppelin 0.6.0

[Jeff Zhang]

I think it is your sql syntax issue. You miss the keyword "table"

It should be
create *table* tablename as select * from tablename_1
drop *table* if exists tablename

On Mon, Aug 22, 2016 at 8:10 PM, Nagasravanthi, Valluri <
valluri.nagasravan...@pfizer.com> wrote:

> Hi Jeff,
>
>
>
> We are still trying to sort out the issue that we highlighted before, so
> it’d be great if you could give your insight as soon as possible so that we
> can try and tackle it in a suitable manner.
>
>
>
> Thanks and Regards,
>
> **
>
> *[image: Description: cid:image001.png@01D1EBF4.36D373B0]*
>
> *Valluri Naga Sravanthi| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com* *; *
> *valluri.nagasravan...@pfizer.com* 
>
> *Website**: **http://pfizerWorks.pfizer.com*
> <http://pfizerworks.pfizer.com/>
>
>
>
> *………………*
>
> *From:* Nagasravanthi, Valluri
> *Sent:* Friday, August 19, 2016 8:39 AM
> *To:* 'Jeff Zhang'
> *Cc:* dev@zeppelin.apache.org; d...@zeppelin.incubator.apache.org;
> us...@zeppelin.apache.org; Jaisankar, Saurabh; Sundararajan, Pranav
> *Subject:* RE: Issues in Zeppelin 0.6.0
>
>
>
> We upgraded Ambari Cluster  while we were working with Zeppelin 0.5.6. At
> that point, we were using Spark 1.5.2. After the upgrade of Ambari, the
> Spark version got upgraded to 1.6. Post that we upgraded Zeppelin to 0.6.0.
>
>
>
> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
> *Sent:* Friday, August 19, 2016 7:48 AM
> *To:* Nagasravanthi, Valluri
> *Cc:* dev@zeppelin.apache.org; d...@zeppelin.incubator.apache.org;
> us...@zeppelin.apache.org; Jaisankar, Saurabh; Sundararajan, Pranav
>
> *Subject:* Re: Issues in Zeppelin 0.6.0
>
>
>
> Do you change spark version when upgrading zeppelin ?
>
>
>
> On Fri, Aug 19, 2016 at 7:36 PM, Nagasravanthi, Valluri <
> valluri.nagasravan...@pfizer.com> wrote:
>
>
>
> Hi,
>
>
>
> Actually, when we worked on Zeppelin 0.5.6, we used to use sqlContext
> itself for reading hive files, registering spark dataframes as temp tables
> and eventualy processing queries on the temp tables.
>
>
>
> The issue which I am facing on Zeppelin 0.6.0 right now  is that even
> though I am able to execute “SELECT”  query on the temptables but *I am
> not able to execute DDL statements like Create/Drop tables using temptables
> derived from the hive files.  *
>
>
>
> *To elaborate it further:*
>
> When I write the following codes, they work fine. PFB the sample codes:
>
>- sqlContext.sql(“select * from table name”)
>- sqlContext.sql(“Select count(column_name) from table_name”)
>
> But when I write the following codes, it fails to execute:
>
>- sqlContext.sql(“create tablename as select * from tablename_1”)
>- sqlContext.sql(“drop if exists tablename”)
>
>
>
> I am getting the error : *java.lang.RuntimeException: [1.1] failure:
> ``with'' expected but identifier drop found :*
>
> I even tried using sql interpreter.
>
>
>
> *PFB the code for it:*
>
> %sql
>
> Drop if exists tablename
>
> I am getting the same error again: : *java.lang.RuntimeException: [1.1]
> failure: ``with'' expected but identifier drop found :*
>
>
>
> Thanks a lot for being so responsive. Hope to fix this issue soon with
> your help.
>
>
>
> Thanks and Regards,
>
> **
>
> *[image: Description: cid:image001.png@01D1EBF4.36D373B0]*
>
> *Valluri Naga Sravanthi| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com* *; *
> *valluri.nagasravan...@pfizer.com* 
>
> *Website**: **http://pfizerWorks.pfizer.com*
> <http://pfizerworks.pfizer.com/>
>
>
>
> *……*
>
>
>
>
>
> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
>
>
>
>
> *Sent:* Friday, August 19, 2016 6:49 AM
> *To:* Sundararajan, Pranav
> *Cc:* dev@zeppelin.apache.org; d...@zeppelin.incubator.apache.org;
> us...@zeppelin.apache.org; Jaisankar, Saurabh; Nagasravanthi, Valluri
>
>
> *Subject:* Re: Issues in Zeppelin 0.6.0
>
>
>
> Sorry, you should use sqlContext
>
>
>
> On Fri, Aug 19, 2016 at 6:11 PM, Sundararajan, Pranav <
> pranav.sundarara...@pfizer.com> wrote:
>
> Hi,
>
>
>
> We tried “hiveContext” instead of “HiveContext” but it d

Re: Issues in Zeppelin 0.6.0

[Jeff Zhang]

You are using sqlContext to run sql, so it should not relate with hive
interpreter. It should run under spark interpreter.  I can ran your sql
successfully with zeppelin 0.6 & spark 1.6. Could you restart your zeppelin
and try again ? If possible, please attach the interpreter log

On Wed, Aug 24, 2016 at 4:19 PM, Sundararajan, Pranav <
pranav.sundarara...@pfizer.com> wrote:

> Hi Jeff,
>
>
>
> Thanks for your suggestion, but I believe we have worded our syntax
> exactly that same way, with the ‘Table’ keyword present, we merely missed
> out on capturing it on the mail.
>
>
>
> Coming back to the problem, we are fairly certain this issue has flared up
> due to the Zeppelin upgrade which we performed, as in the previous version
> of Zeppelin which we were using (0.5.6) the very same syntax for create and
> drop statements were working, it is only after the upgrade it is returning
> the error.
>
>
>
>
>
> We have checked the hive interpreter settings multiple times and also
> tried re-configuring it, and are able to successfully call it too. It is
> only with reference to trying to establish a connection between the Spark
> tables created in that sparkcontext instance and previously
> existing/created hive tables we are facing an issue. If you can give us
> come pointers or better yet remedies to this particular problem that’d be
> extremely helpful.
>
>
>
> Thanks a lot for your help!
>
>
>
> Regards,
>
>
>
>
>
>
>
> [image: Description: cid:image001.png@01D1C173.60CDE0C0]
>
> *Pranav Sundararajan| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com **; 
> **pranav.sundarara...@pfizer.com
> *
>
> *Website**: **http://pfizerWorks.pfizer.com
> <http://pfizerworks.pfizer.com/>*
>
>
>
>
>
>
>
> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
> *Sent:* Wednesday, August 24, 2016 4:01 AM
> *To:* Nagasravanthi, Valluri
> *Cc:* dev@zeppelin.apache.org; d...@zeppelin.incubator.apache.org;
> us...@zeppelin.apache.org; Jaisankar, Saurabh; Sundararajan, Pranav;
> Patel, Dhaval
>
> *Subject:* Re: Issues in Zeppelin 0.6.0
>
>
>
> I think it is your sql syntax issue. You miss the keyword "table"
>
>
>
> It should be
>
> create *table* tablename as select * from tablename_1
>
> drop *table* if exists tablename
>
>
>
> On Mon, Aug 22, 2016 at 8:10 PM, Nagasravanthi, Valluri <
> valluri.nagasravan...@pfizer.com> wrote:
>
> Hi Jeff,
>
>
>
> We are still trying to sort out the issue that we highlighted before, so
> it’d be great if you could give your insight as soon as possible so that we
> can try and tackle it in a suitable manner.
>
>
>
> Thanks and Regards,
>
> **
>
> *[image: Description: cid:image001.png@01D1EBF4.36D373B0]*
>
> *Valluri Naga Sravanthi| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com* *; *
> *valluri.nagasravan...@pfizer.com* 
>
> *Website**: **http://pfizerWorks.pfizer.com*
> <http://pfizerworks.pfizer.com/>
>
>
>
> *……*
>
> *From:* Nagasravanthi, Valluri
> *Sent:* Friday, August 19, 2016 8:39 AM
> *To:* 'Jeff Zhang'
> *Cc:* dev@zeppelin.apache.org; d...@zeppelin.incubator.apache.org;
> us...@zeppelin.apache.org; Jaisankar, Saurabh; Sundararajan, Pranav
> *Subject:* RE: Issues in Zeppelin 0.6.0
>
>
>
> We upgraded Ambari Cluster  while we were working with Zeppelin 0.5.6. At
> that point, we were using Spark 1.5.2. After the upgrade of Ambari, the
> Spark version got upgraded to 1.6. Post that we upgraded Zeppelin to 0.6.0.
>
>
>
> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
> *Sent:* Friday, August 19, 2016 7:48 AM
> *To:* Nagasravanthi, Valluri
> *Cc:* dev@zeppelin.apache.org; d...@zeppelin.incubator.apache.org;
> us...@zeppelin.apache.org; Jaisankar, Saurabh; Sundararajan, Pranav
>
>
> *Subject:* Re: Issues in Zeppelin 0.6.0
>
>
>
> Do you change spark version when upgrading zeppelin ?
>
>
>
> On Fri, Aug 19, 2016 at 7:36 PM, Nagasravanthi, Valluri <
> valluri.nagasravan...@pfizer.com> wrote:
>
>
>
> Hi,
>
>
>
> Actually, when we worked on Zeppelin 0.5.6, we used to use sqlContext
> itself for reading hive files, registering spark dataframes as temp tables
> and eventualy processing queries on the temp tables.
>
>
>
> The issue which I am facing on Zeppelin 0.6.0 right now  is that

Re: Issues in Zeppelin 0.6.0

[Jeff Zhang]

All right, then this is the real code you executed, So what kind of error
do you get ? And have you run the sql in spark 1.6 ? As I suspect it is
morely a spark issue.

On Wed, Aug 24, 2016 at 7:45 PM, Sundararajan, Pranav <
pranav.sundarara...@pfizer.com> wrote:

> No Jeff, that is not the issue here.
>
>
>
> For instance, in my previous zeppelin version, I was calling  %sql and
> then writing a create table statement below it, reading data from the spark
> tables I had created in that particular session.
>
>
>
> It reads something like this:
>
>
>
> *%sql*
>
>
>
>
>
> *CREATE TABLE SDS_DIAG_RESTRICTED_3YRS as*
>
>
>
> *select a.PTID, a.SDS_TERM, c.DIAGNOSIS_CD, c.DIAGNOSIS_CD_TYPE from *
>
>
>
>
>
> *(SELECT PTID, SDS_TERM from *
>
> *(*
>
> *SELECT PTID, SDS_TERM ,count(distinct NOTE_DATE) AS counted FROM
> sds201320142015*
>
>
>
> *where lower(SDS_TERM) rlike (${2. Terms of Interest='nash|pain'}) and
> substring(NOTE_DATE,7,10) rlike (${4. Years of Interest='2014'})*
>
>
>
> *GROUP BY PTID, SDS_TERM*
>
>
>
> *) AS B*
>
>
>
> *WHERE counted >=${3. Min Nr of Notes=2} ) as a*
>
>
>
> *inner join *
>
>
>
> *(select PTID,DIAGNOSIS_CD,DIAGNOSIS_CD_TYPE*
>
>
>
> *from diag201320142015*
>
>
>
> *where lower(DIAGNOSIS_CD) rlike (${1. Diagnosis codes='8820'})*
>
>
>
> *) as c *
>
>
>
> *on a.PTID=c.PTID*
>
>
>
>
>
> *group by a.PTID, a.SDS_TERM, c.DIAGNOSIS_CD, c.DIAGNOSIS_CD_TYPE*
>
>
>
> The yellow highlighted portion  of the code, i.e *sds201320142015* is the
> temporary spark table which I am trying to use to create my Hive table
> which is *SDS_DIAG_RESTRICTED_3YRS**. *So the main issue here is this
> code snippet which was working on Zeppelin 0.5.6, is not working on
> Zeppelin 0.6.0, in spite of carrying the same interpreter settings as I’ve
> told you before.
>
>
>
> I hope that you have a better understanding of the problem we are trying
> to resolve here now.
>
>
>
>
>
> Regards,
>
>
>
>
>
>
>
> [image: Description: cid:image001.png@01D1C173.60CDE0C0]
>
> *Pranav Sundararajan| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com **; 
> **pranav.sundarara...@pfizer.com
> *
>
> *Website**: **http://pfizerWorks.pfizer.com
> <http://pfizerworks.pfizer.com/>*
>
>
>
>
>
>
>
> *From:* Jeff Zhang [mailto:zjf...@gmail.com]
> *Sent:* Wednesday, August 24, 2016 4:45 AM
> *To:* Sundararajan, Pranav
> *Cc:* dev@zeppelin.apache.org; d...@zeppelin.incubator.apache.org;
> us...@zeppelin.apache.org; Jaisankar, Saurabh; Patel, Dhaval;
> Nagasravanthi, Valluri
>
> *Subject:* Re: Issues in Zeppelin 0.6.0
>
>
>
> You are using sqlContext to run sql, so it should not relate with hive
> interpreter. It should run under spark interpreter.  I can ran your sql
> successfully with zeppelin 0.6 & spark 1.6. Could you restart your zeppelin
> and try again ? If possible, please attach the interpreter log
>
>
>
> On Wed, Aug 24, 2016 at 4:19 PM, Sundararajan, Pranav <
> pranav.sundarara...@pfizer.com> wrote:
>
> Hi Jeff,
>
>
>
> Thanks for your suggestion, but I believe we have worded our syntax
> exactly that same way, with the ‘Table’ keyword present, we merely missed
> out on capturing it on the mail.
>
>
>
> Coming back to the problem, we are fairly certain this issue has flared up
> due to the Zeppelin upgrade which we performed, as in the previous version
> of Zeppelin which we were using (0.5.6) the very same syntax for create and
> drop statements were working, it is only after the upgrade it is returning
> the error.
>
>
>
>
>
> We have checked the hive interpreter settings multiple times and also
> tried re-configuring it, and are able to successfully call it too. It is
> only with reference to trying to establish a connection between the Spark
> tables created in that sparkcontext instance and previously
> existing/created hive tables we are facing an issue. If you can give us
> come pointers or better yet remedies to this particular problem that’d be
> extremely helpful.
>
>
>
> Thanks a lot for your help!
>
>
>
> Regards,
>
>
>
>
>
>
>
> [image: Description: cid:image001.png@01D1C173.60CDE0C0]
>
> *Pranav Sundararajan| On assignment to **P**fizerWorks*
>
> Cell*: +91 9008412366 <%2B91%209008412366>*
>
> *Email**: **pfizerwor...@pfizer.com **; 
> **pranav.sundarara...@pfizer.com
> *
>
&

Re: Running Zeppelin from my github checkout

[Jeff Zhang]

Can you check the logs to see what error you get ? It is located in
ZEPPELIN_HOME/logs

On Thu, Sep 1, 2016 at 1:00 AM, Luciano Resende 
wrote:

> In Spark, I can usually build the source code and run a test cluster from
> that checkout that I just built. I was trying the same with Zeppelin, and I
> can get things up but with some issues around some interpreters and I
> cannot even run the tutorial notebook.  Having said that, the distribution
> generated by the same build can be extracted and works fine.
>
> Before I spent more time on this, is this scenario something that works (or
> used to work) ? Is there any special profiles or something i need to invoke
> to enable this ?
>
> BTW, I was running my build as :
>
> mvn -Pspark-2.0 -Pr -Phadoop-2.6 -Psparkr -Ppyspark -Pscalding -Pexamples
> -Pbuild-distr -DskipTests -Dscala-2.11 clean install
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>



-- 
Best Regards

Jeff Zhang

Re: [DISCUSS] Zeppelin 0.6.2 release

[Jeff Zhang]

+1 for it.

On Fri, Oct 7, 2016 at 5:37 PM, Jongyoul Lee  wrote:

> Great job, Mina.
>
> I think it's enough to release 0.6.2.
>
> On Fri, Oct 7, 2016 at 4:39 PM, Mina Lee  wrote:
>
> > Hi devs,
> >
> > Release process has been delayed longer than we planned
> > since several critical issues have been reported continuously.
> > [1] All issues that were blocking 0.6.2 have been resolved finally,
> > so I want to hear thoughts about 0.6.2 release.
> >
> > Regards,
> > Mina
> >
> > [1] https://issues.apache.org/jira/browse/ZEPPELIN-1347
> >
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>



-- 
Best Regards

Jeff Zhang

Re: How Apache Zeppelin runs a paragraph

[Jeff Zhang]

Thanks Jougyoul for the effort. This is really helpful for new contributors.

Several comments.

1. Seems we didn't use noteId/notebookId correctly in code, do we have
ticket to track that ? Because the sample code is conflicted with the
definition of note/notebook in the post, it might confuse users.

@Path("job/{notebookId}/{paragraphId}")
@ZeppelinApi
public Response runParagraph(@PathParam("notebookId") String notebookId,
@PathParam("paragraphId") String paragraphId, String message)
throws IOException, IllegalArgumentException {
  LOG.info("run paragraph job asynchronously {} {} {}", notebookId,
paragraphId, message);

  Note note = notebook.getNote(notebookId);


2. It might be better to use diagram to visualize the differences
between shared/scoped/shared




On Sun, Oct 9, 2016 at 12:38 AM, Felix Cheung 
wrote:

> Great post!
>
>
>
>
>
> On Tue, Oct 4, 2016 at 8:56 PM -0700, "Jongyoul Lee" 
> wrote:
>
> Hello DuyHai,
>
> Thanks for the fixing the typo. I've fixed it.
>
> Concerning the debugging, I think writing posts or updating wiki would be
> better. I'm willing to write a simple post for that remote debugging.
>
> On Tue, Oct 4, 2016 at 5:53 PM, DuyHai Doan  wrote:
>
>> Hello Jongyoul
>>
>> Excellent write up to demystify the execution life-cycle of a paragraph.
>>
>> There is a small typo in the blog: "At last, it has two words separated
>> by comma, *getInterpreter* handles it as “%{group_name}.{interpreter_na
>> me}”"
>>
>> Instead of comma, shouldn't it be DOT (.) ?
>>
>> There is also another unknown details that may be interesting to expose:
>> how are interpreter JVM launched ?
>>
>> InterpreterFactory
>>
>> private Interpreter createRemoteRepl(String interpreterPath, String noteId, 
>> String className,
>> Properties property, String interpreterSettingId) {
>>   int connectTimeout = 
>> conf.getInt(ConfVars.ZEPPELIN_INTERPRETER_CONNECT_TIMEOUT);
>>   String localRepoPath = conf.getInterpreterLocalRepoPath() + "/" + 
>> interpreterSettingId;
>>   int maxPoolSize = conf.getInt(ConfVars.ZEPPELIN_INTERPRETER_MAX_POOL_SIZE);
>>
>>   RemoteInterpreter remoteInterpreter =
>>   new RemoteInterpreter(property, noteId, className, 
>> conf.getInterpreterRemoteRunnerPath(),
>>   interpreterPath, localRepoPath, connectTimeout, maxPoolSize,
>>   remoteInterpreterProcessListener, appEventListener);
>>   remoteInterpreter.addEnv(env);
>>
>>   return new LazyOpenInterpreter(remoteInterpreter);
>> }
>>
>>
>> The code will call the bin/intepreter.sh (or interpreter.cmd if on
>> Windows) to launch a new JVM for the interpreter.
>>
>> And depending on the type of interpreter, the command will be different:
>>
>> if [[ -n "${SPARK_SUBMIT}" ]]; then
>> ${SPARK_SUBMIT} --class ${ZEPPELIN_SERVER} --driver-class-path
>> "${ZEPPELIN_INTP_CLASSPATH_OVERRIDES}:${CLASSPATH}"
>> --driver-java-options "${JAVA_INTP_OPTS}" ${SPARK_SUBMIT_OPTIONS}
>> ${SPARK_APP_JAR} ${PORT} &
>> else
>> ${ZEPPELIN_RUNNER} ${JAVA_INTP_OPTS} ${ZEPPELIN_INTP_MEM} -cp
>> ${ZEPPELIN_INTP_CLASSPATH_OVERRIDES}:${CLASSPATH} ${ZEPPELIN_SERVER}
>> ${PORT} &
>> fi
>>
>> There is a special case for Spark because we execute a SparkSubmit.
>>
>> This detail is important for contributors because when they need to DEBUG
>> an interpreter code, they have to modify the interpreter.sh file to add an
>> "-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=xxx"
>>
>> Most of people naively add this debug argument to the zeppelin.sh file
>> but it only debugs the Zeppelin server itself, not the interpreter.
>>
>> Do you think we may create a page in the official doc for this ?
>>
>>
>> On Tue, Oct 4, 2016 at 9:00 AM, Jongyoul Lee  wrote:
>>
>>> Hello devs and users,
>>>
>>> I've submitted a post into Apache Zeppelin Stories on medium.com:
>>> https://medium.com/apache-zeppelin-stories/how-a
>>> pache-zeppelin-runs-a-paragraph-783a0a612ba9#.x9ym8fovv
>>>
>>> I hope all users and contributors understand Apache Zeppelin deeply.
>>>
>>> Regards,
>>> Jongyoul
>>>
>>> --
>>> 이종열, Jongyoul Lee, 李宗烈
>>> http://madeng.net
>>>
>>
>>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>



-- 
Best Regards

Jeff Zhang

Re: [DISCUSS] Zeppelin 0.7 Roadmap

[Jeff Zhang]

+1 for adding impersonation. Consider most of subtasks of ZEPPELIN-1337
(multiple user support) will involve large changes, I am afraid it may be a
little risky to release 0.7 in Nov, and I expect we would introduce new
bugs since the changes are big.

On Mon, Oct 10, 2016 at 10:30 AM, Corneau Damien 
wrote:

> We don't have many routes, and so far no need for subroutes, so
> angular-ui-router isn't really needed.
> However lazy loading could be nice
>
> On Mon, Oct 10, 2016 at 10:53 AM, Dennis Jung  wrote:
>
> > Hi~
> > In fronted part, how about update router to ui-router with lazy load
> > module?
> > ui-router is 3rd party module, but it is more powerful than ngRoute. And
> > using lazy-loading will help resource loading performance.
> >
> > http://stackoverflow.com/questions/21023763/angularjs-
> > difference-between-angular-route-and-angular-ui-router
> >
> > How do you think? If it seems okay, I could help on this.
> > Thanks.
> >
> > 2016-10-10 0:22 GMT+09:00 DuyHai Doan :
> >
> > > I'd also add in the Enterprise section a topic about monitoring (e.g.
> at
> > > least exposing JMX metrics as a 1st step). I guess we cannot pretend to
> > be
> > > enterprise ready without at least some means to monitor the Zeppelin
> > server
> > > state.
> > >
> > > Hopefully the monitoring topic will be covered by the big refactoring
> > JIRA
> > > epic: https://issues.apache.org/jira/browse/ZEPPELIN-1525
> > >
> > > On Sun, Oct 9, 2016 at 4:53 PM, Jongyoul Lee 
> wrote:
> > >
> > > > Thanks for the sharing this roadmap.
> > > >
> > > > I'd like to add impersonation for Spark/JDBC interpreter.
> > > >
> > > > How do you think of ti?
> > > >
> > > > Regards,
> > > > Jongyoul
> > > >
> > > > On Sat, Oct 8, 2016 at 6:46 AM, moon soo Lee 
> wrote:
> > > >
> > > >> Hi, Apache Zeppelin users and developers,
> > > >>
> > > >> We're about to start a release process for 0.6.2 and i think it's
> good
> > > >> time to discuss about future release, 0.7.0. There were great
> > discussion
> > > >> about roadmap [1] and we updated wiki page [2], but 0.7.0 section on
> > the
> > > >> roadmap wiki is empty at the moment. Having a 0.7.0 section on a
> wiki
> > > page,
> > > >> i think, doesn't mean neither rejecting other subjects nor 100%
> > > guarantee
> > > >> of them. However it's more for helping Zeppelin users and developers
> > > have
> > > >> reasonable expectations for the next release and helps community
> focus
> > > on
> > > >> main theme.
> > > >>
> > > >> Multi-tenancy related feature (interpreter authorization,
> > impersonation,
> > > >> per user interpreter, and so on) are the most active subject in the
> > > >> community at the moment i think. And we have a new menu, "Job" in
> the
> > > >> master branch is another big change. I think "Enterprise ready"
> > section
> > > on
> > > >> [2] can be main topic for 0.7 release.
> > > >>
> > > >> And there're many improvements around generic JDBC interpreter and
> > > Python
> > > >> support (matplotlib integration and so on). They can be another
> > > important
> > > >> subject with new Interpreters.
> > > >>
> > > >> Besides that, I've seen people struggle with front-end performance
> and
> > > we
> > > >> can address that on 0.7, that would be great.
> > > >> Also i'd like to keep working on pluggability for visualization,
> which
> > > >> was subject from 0.6 release.
> > > >> Therefore, i would suggest following draft as a roadmap for 0.7.0
> > > >>
> > > >> * Enterprise support
> > > >>   - Multi user support (ZEPPELIN-1337)
> > > >>   - Job management
> > > >> * Interpreter
> > > >>   - Improve JDBC / Python interpreter
> > > >>   - New interpreters
> > > >> * Front end performance improvement
> > > >> * Pluggable visualization
> > > >>
> > > >> Regarding timeline, although we're keep making series of 0.6.x
> > release,
> > > >> it's already been 3 months since 0.6 release. And i think many items
> > are
> > > >> already been addressed on master branch or patches are available.
> So i
> > > >> think we can target near future, like November.
> > > >>
> > > >> What do you think? And feedback would be appreciated.
> > > >>
> > > >> Thanks,
> > > >> moon
> > > >>
> > > >> [1] http://apache-zeppelin-users-incubating-mailing-li
> > > >> <http://apache-zeppelin-users-incubating-mailing-list.75479.
> > > x6.nabble.com/DISCUSS-Update-Roadmap-tp2452.html>
> > > >> st.75479.x6.nabble.com/DISCUSS-Update-Roadmap-tp2452.html
> > > >> <http://apache-zeppelin-users-incubating-mailing-list.75479.
> > > x6.nabble.com/DISCUSS-Update-Roadmap-tp2452.html>
> > > >> [2] https://cwiki.apache.org/confluence/display/ZEPPELIN/
> > > Zeppelin+Roadmap
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > 이종열, Jongyoul Lee, 李宗烈
> > > > http://madeng.net
> > > >
> > >
> >
>



-- 
Best Regards

Jeff Zhang

Re: [VOTE] Release Apache Zeppelin 0.6.2 (rc1)

[Jeff Zhang]

+1, teste with spark 2.0 with basic notes,  ZeppelinContext and UDF works
correctly.

On Tue, Oct 11, 2016 at 8:41 AM, Anthony Corbacho <
corbacho.anth...@gmail.com> wrote:

> +1
>
> I tested with my notebooks works as expected.
> Great job!
>
> On Mon, Oct 10, 2016 at 9:34 PM, Mina Lee  wrote:
>
> > Hi folks,
> >
> > I propose the following RC to be released for the Apache Zeppelin
> > 0.6.2 release.
> >
> > The commit id is 9a44590841d5d83da1a297d507f6e2e67784ba1f which is
> > corresponds to the tag v0.6.2-rc1:
> > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=
> > 9a44590841d5d83da1a297d507f6e2e67784ba1f
> >
> > The release archives (tgz), signature, and checksums are here
> > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.6.2-rc1/
> >
> > The release candidate consists of the following source distribution
> archive
> > zeppelin-0.6.2.tgz
> >
> > In addition, the following supplementary binary distributions are
> provided
> > for user convenience at the same location
> > zeppelin-0.6.2-bin-all.tgz
> > zeppelin-0.6.2-netinst-all.tgz
> >
> > The maven artifacts are here
> > https://repository.apache.org/content/repositories/
> orgapachezeppelin-1019
> >
> > You can find the KEYS file here:
> > https://people.apache.org/keys/committer/minalee.asc
> >
> > Release notes available at
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > projectId=12316221&version=12338135
> >
> > Vote will be open for next 72 hours (close at 6am 13/Oct PDT).
> >
> > [ ] +1 approve
> > [ ] 0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
>



-- 
Best Regards

Jeff Zhang

Re: [VOTE] Release Apache Zeppelin 0.6.2 (RC2)

[Jeff Zhang]

+1, Run tutorial note successfully in embedded mode and yarn-client.



Mina Lee 于2016年10月12日周三 下午6:29写道：

> Hi dev,
>
> I propose the following RC to be released for the Apache Zeppelin 0.6.2
> release.
>
> The commit id is 091086de9400dd1c02ca02acf4180b1bf1e9ede7 which is
> corresponds
> to the tag v0.6.2-rc2:
> *
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=091086de9400dd1c02ca02acf4180b1bf1e9ede7
> <
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=091086de9400dd1c02ca02acf4180b1bf1e9ede7
> >*
>
> The release archives (tgz), signature, and checksums are here
> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.6.2-rc2/
>
> The release candidate consists of the following source distribution archive
> zeppelin-0.6.2.tgz
>
> In addition, the following supplementary binary distributions are provided
> for user convenience at the same location
> zeppelin-0.6.2-bin-all.tgz
> zeppelin-0.6.2-netinst-all.tgz
>
> The maven artifacts are here
> https://repository.apache
> .org/content/repositories/orgapachezeppelin-1020/org/apache/zeppelin/
>
> You can find the KEYS file here:
> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
>
> Release notes available at
> *
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12336543&styleName=Html&projectId=12316221
> <
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12336543&styleName=Html&projectId=12316221
> >*
>
> Vote will be open for next 72 hours (close at 4am 15/Oct PDT).
>
> [ ] +1 approve
> [ ] 0 no opinion
> [ ] -1 disapprove (and reason why)
>

How to add custom syntax highlight ?

[Jeff Zhang]

I am working on Pig Interpreter, but unfortunately
https://github.com/ajaxorg/ace-builds/tree/master/src-noconflict doesn't
support pig, does anyone know other approaches to support custom syntax
hightlight except adding pig support in ACE editors ? Thanks.

Re: 0.7.0 zeppelin.interpreters change: can't make pyspark be default Spark interperter

[Jeff Zhang]

The default interpreter is now defined in interpreter-setting.json

You can update the following file to make pyspark as the default
interpreter and then copy it to folder interpreter/spark

https://github.com/apache/zeppelin/blob/master/spark/src/main/resources/interpreter-setting.json



Ruslan Dautkhanov 于2016年11月30日周三 上午8:49写道：

> After 0.6.2 -> 0.7 upgrade, pySpark isn't a default Spark interpreter;
> despite we have org.apache.zeppelin.spark.*PySparkInterpreter*
> listed first in zeppelin.interpreters.
>
> zeppelin.interpreters in zeppelin-site.xml:
>
> 
>   zeppelin.interpreters
>
> org.apache.zeppelin.spark.PySparkInterpreter,org.apache.zeppelin.spark.SparkInterpreter
> ...
> 
>
>
>
> Any ideas how to fix this?
>
>
> Thanks,
> Ruslan
>

Re: sparkContext to get Spark Driver's URL

[Jeff Zhang]

You can get ui by

sc.uiWebUrl

And community is working on to display it in paragraph.
https://github.com/apache/zeppelin/pull/1663


Ruslan Dautkhanov 于2016年12月1日周四 上午8:58写道：

> Any easy way to get Spark Driver's URL (i.e. from sparkContext )?
> I always have to go to CM -> YARN applications -> choose my Spark job ->
> click Application Master etc. to get Spark's Driver UI.
>
> Any way we could derive driver's URL programmatically from SparkContext
> variable?
>
>
> ps. Long haul - it would be super awesome to get a link staright in
> Zeppelin notebook (when SparkContext is instatiated).
>
>
> Thank you,
> Ruslan
>
>

Build error in docker

[Jeff Zhang]

I try to build travis in local, so I follow the instruction here to build
zeppelin in docker, but get the following error as following. Has anyone
hit this issue before or successfully run travis locally ?

https://docs.travis-ci.com/user/common-build-problems/#Troubleshooting-Locally-in-a-Docker-Image



  "id": "ECONFLICT",
  "level": "error",
  "data": {},
  "message": "Unable to find suitable version for angular",
  "stacktrace": "angular: Unable to find suitable version for angular\n
 at createError
(/home/travis/zeppelin/zeppelin-web/node_modules/bower/lib/util/createError.js:4:15)\n
   at Manager._electSuitable
(/home/travis/zeppelin/zeppelin-web/node_modules/bower/lib/core/Manager.js:825:15)\n
   at
/home/travis/zeppelin/zeppelin-web/node_modules/bower/lib/core/Manager.js:609:25\n
   at _fulfilled
(/home/travis/zeppelin/zeppelin-web/node_modules/bower/lib/node_modules/q/q.js:834:54)\n
   at self.promiseDispatch.done
(/home/travis/zeppelin/zeppelin-web/node_modules/bower/lib/node_modules/q/q.js:863:30)\n
   at Promise.promise.promiseDispatch
(/home/travis/zeppelin/zeppelin-web/node_modules/bower/lib/node_modules/q/q.js:796:13)\n
   at
/home/travis/zeppelin/zeppelin-web/node_modules/bower/lib/node_modules/q/q.js:556:49\n
   at runSingle
(/home/travis/zeppelin/zeppelin-web/node_modules/bower/lib/node_modules/q/q.js:137:13)\n
   at flush
(/home/travis/zeppelin/zeppelin-web/node_modules/bower/lib/node_modules/q/q.js:125:13)\n
   at process._tickCallback (node.js:458:13)"
}]

npm ERR! Linux 4.4.27-moby
npm ERR! argv "/usr/bin/node" "/usr/bin/npm" "install"
npm ERR! node v0.10.48
npm ERR! npm  v2.15.1
npm ERR! code ELIFECYCLE
npm ERR! zeppelin-web@0.0.0 postinstall: `bower install --silent`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the zeppelin-web@0.0.0 postinstall script 'bower install
--silent'.
npm ERR! This is most likely a problem with the zeppelin-web package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! bower install --silent
npm ERR! You can get information on how to open an issue for this project
with:
npm ERR! npm bugs zeppelin-web
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!
npm ERR! npm owner ls zeppelin-web
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! /home/travis/zeppelin/zeppelin-web/npm-debug.log

Re: Handling spark-submit errors

[Jeff Zhang]

Hi Luciano

Usually I change the log4j.properties to get the output of process
interpreter.sh, it can help most of the problems I hit in interpreter.sh

Here's the log4j.properties I use

log4j.rootLogger = INFO, dailyfile

log4j.appender.stdout = org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout = org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%d] ({%t} %F[%M]:%L) -
%m%n

log4j.appender.dailyfile.DatePattern=.-MM-dd
log4j.appender.dailyfile.Threshold = DEBUG
log4j.appender.dailyfile = org.apache.log4j.DailyRollingFileAppender
log4j.appender.dailyfile.File = ${zeppelin.log.file}
log4j.appender.dailyfile.layout = org.apache.log4j.PatternLayout
log4j.appender.dailyfile.layout.ConversionPattern=%5p [%d] ({%t} %F[%M]:%L)
- %m%n


log4j.logger.org.apache.zeppelin.interpreter.InterpreterFactory=DEBUG
log4j.logger.org.apache.zeppelin.notebook.Paragraph=DEBUG
log4j.logger.org.apache.zeppelin.scheduler=DEBUG
log4j.logger.org.apache.zeppelin.livy=DEBUG
log4j.logger.org.apache.zeppelin.flink=DEBUG
log4j.logger.org.apache.zeppelin.spark=DEBUG
log4j.logger.org.apache.zeppelin.interpreter.util=DEBUG
log4j.logger.org.apache.zeppelin.interpreter.remote=DEBUG
log4j.logger.org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer=DEBUG

Luciano Resende 于2016年12月9日周五 上午2:48写道：

> I was playing with some new data sources in Zeppelin (master) and had an
> issue on my --package declaration in zeppelin-env.sh.
>
> First, the error stack was a little misleading, as it was only reporting
> connection issues to Spark trying to retrieve paragraph status.
>
> From my initial investigation, we pretty much invoke the spark-submit from
> interpreter.sh... and I was wondering if there is a good way to trap this
> kind of issues and be able to provide a better response to the user (or
> even better logging)...
>
> One option is to have something similar to what spark does, and actually,
> have a class handling some of the interpreter integration logic.
>
> Thoughts  ? Any other possibilities ?
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Unstable travis CI recently

[Jeff Zhang]

Hi Folks,

As you may notice that our travis CI is not stable recently. There's many
flaky test, and it waste every developer's time to figure out whether the
failure is due to your PR or flaky test. So I think it is time for us to
make the CI stable. Here's tickets for all the flaky test.
https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20flaky%20and%20status%20!%3D%20RESOLVED%20ORDER%20%20BY%20status%20ASC%20

Fixing the flaky test may take some time and may not easy for some test but
I think it is worth to do that.  And it is better for these people who
familiar with that particular test case to fix it. What do you guys think ?
Thanks

Re: Interpreter zombie processes

[Jeff Zhang]

I believe I see this before too.


Luciano Resende 于2016年12月16日周五 上午10:17写道：

> I have also seen similar issues even using zeppelin-ddeamon but didn't
> have much time to investigate the issue when it was happening to me.
>
> On Thu, Dec 15, 2016 at 12:15 PM Ruslan Dautkhanov 
> wrote:
>
> Moon,
>
> > ZeppelinServer try to terminate interpreter process when shutting down
> [1].
>
> Unfortunatally, this does not happen all the time.
> I have seen zombie spark interpreter processes many times.
> As a double conirmation - I see spark yarn application was still running.
>
> > Also bin/zeppelin-deamon.sh script clean up if some processes are left
> [2].
>
> We were not using that script. We were just running like zeppelin.sh
> --config ..
> Will try to switch to zeppelin-deamon.sh and see if it makes a difference.
>
> Thanks!
>
>
>
>
> --
> Ruslan Dautkhanov
>
>
>
> On Thu, Dec 15, 2016 at 12:46 PM, moon  wrote:
>
> Hi,
>
>
>
>
>
> ZeppelinServer try to terminate interpreter process when shutting down [1].
>
>
> Also bin/zeppelin-deamon.sh script clean up if some processes are left [2].
>
>
>
>
>
> If some processes are remained after shutting down Zeppelin, that's not an
>
>
> expected result.
>
>
>
>
>
> Thanks,
>
>
> moon
>
>
>
>
>
> [1]
>
>
>
> https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java#L159
>
>
> [2]
>
>
> https://github.com/apache/zeppelin/blob/master/bin/zeppelin-daemon.sh#L218
>
>
>
>
>
> On Thu, Dec 15, 2016 at 11:25 AM Ruslan Dautkhanov 
>
>
> wrote:
>
>
>
>
>
> > How to make Interpreter processes terminate if Zeppelin server exits?
>
>
> >
>
>
> > When we restart Zeppelin server (main process), in many cases,
> Interpreter
>
>
> > process keeps running, essential becoming a zombie processes.
>
>
> >
>
>
> > In case of Spark interpreter, it also holds SparkContext - consuming
>
>
> > server-side
>
>
> > resources too.
>
>
> >
>
>
> > How do we configure Zeppelin to kill all interpreters before its main
>
>
> > process terminates?
>
>
> > Or is this a bug?
>
>
> >
>
>
> > We're running ~2 weeks old snapshot of 0.7.0 Zeppelin.
>
>
> >
>
>
> >
>
>
> > Thank you,
>
>
> > Ruslan Dautkhanov
>
>
> >
>
>
>
> --
> Sent from my Mobile device
>

Build error on windows

[Jeff Zhang]

I try to build zeppelin on windows, but hit the following error. Does
anyone know how to address this ? Thanks


[ERROR] npm ERR! Windows_NT 6.1.7601
[ERROR] npm ERR! argv
"C:\\Users\\jzhang\\zeppelin\\zeppelin-web\\node\\node.exe"
"C:\\Users\\jzhang\\zeppelin\\zeppelin
-web\\node\\node_modules\\npm\\bin\\npm-cli.js" "run" "build"
[ERROR] npm ERR! node v6.9.1
[ERROR] npm ERR! npm  v3.10.8
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! zeppelin-web@0.0.0 build: `grunt build`
[ERROR] npm ERR! Exit status 3
[ERROR] npm ERR!
[ERROR] npm ERR! Failed at the zeppelin-web@0.0.0 build script 'grunt
build'.
[ERROR] npm ERR! Make sure you have the latest version of node.js and npm
installed.
[ERROR] npm ERR! If you do, this is most likely a problem with the
zeppelin-web package,
[ERROR] npm ERR! not with npm itself.
[ERROR] npm ERR! Tell the author that this fails on your system:
[ERROR] npm ERR! grunt build
[ERROR] npm ERR! You can get information on how to open an issue for this
project with:
[ERROR] npm ERR! npm bugs zeppelin-web
[ERROR] npm ERR! Or if that isn't available, you can get their info via:
[ERROR] npm ERR! npm owner ls zeppelin-web
[ERROR] npm ERR! There is likely additional logging output above.
[ERROR]
[ERROR] npm ERR! Please include the following file with any support request:
[ERROR] npm ERR! C:\Users\jzhang\zeppelin\zeppelin-web\npm-debug.log
[INFO]

Re: Build error on windows

[Jeff Zhang]

Attach npm-debug.log


Alexander Bezzubov 于2016年12月18日周日 下午10:34写道：

Hi Jeff,


looks like something failed during `grunt build`.
Could you also show the content of
`C:\Users\jzhang\zeppelin\zeppelin-web\npm-debug.log`?
It might have some details to give a hint on the issue.

--
Alex

On Sun, Dec 18, 2016 at 10:46 PM, Jeff Zhang  wrote:

I try to build zeppelin on windows, but hit the following error. Does
anyone know how to address this ? Thanks


[ERROR] npm ERR! Windows_NT 6.1.7601
[ERROR] npm ERR! argv
"C:\\Users\\jzhang\\zeppelin\\zeppelin-web\\node\\node.exe"
"C:\\Users\\jzhang\\zeppelin\\zeppelin
-web\\node\\node_modules\\npm\\bin\\npm-cli.js" "run" "build"
[ERROR] npm ERR! node v6.9.1
[ERROR] npm ERR! npm  v3.10.8
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! zeppelin-web@0.0.0 build: `grunt build`
[ERROR] npm ERR! Exit status 3
[ERROR] npm ERR!
[ERROR] npm ERR! Failed at the zeppelin-web@0.0.0 build script 'grunt
build'.
[ERROR] npm ERR! Make sure you have the latest version of node.js and npm
installed.
[ERROR] npm ERR! If you do, this is most likely a problem with the
zeppelin-web package,
[ERROR] npm ERR! not with npm itself.
[ERROR] npm ERR! Tell the author that this fails on your system:
[ERROR] npm ERR! grunt build
[ERROR] npm ERR! You can get information on how to open an issue for this
project with:
[ERROR] npm ERR! npm bugs zeppelin-web
[ERROR] npm ERR! Or if that isn't available, you can get their info via:
[ERROR] npm ERR! npm owner ls zeppelin-web
[ERROR] npm ERR! There is likely additional logging output above.
[ERROR]
[ERROR] npm ERR! Please include the following file with any support request:
[ERROR] npm ERR! C:\Users\jzhang\zeppelin\zeppelin-web\npm-debug.log
[INFO]

Re: Re: Build error on windows

[Jeff Zhang]

Thanks, This PR resolve my issue.



zjp_j...@163.com 于2016年12月19日周一 上午10:21写道：

> I think the following issue can help u.
> - JIRA : https://issues.apache.org/jira/browse/ZEPPELIN-1239
> - PR : https://github.com/apache/zeppelin/pull/1731
>
> --
> zjp_j...@163.com
>
>
> *From:* Jeff Zhang 
> *Date:* 2016-12-19 09:27
> *To:* users ; dev 
> *Subject:* Re: Build error on windows
>
> Attach npm-debug.log
>
>
> Alexander Bezzubov 于2016年12月18日周日 下午10:34写道：
>
> Hi Jeff,
>
>
> looks like something failed during `grunt build`.
> Could you also show the content of 
> `C:\Users\jzhang\zeppelin\zeppelin-web\npm-debug.log`?
> It might have some details to give a hint on the issue.
>
> --
> Alex
>
> On Sun, Dec 18, 2016 at 10:46 PM, Jeff Zhang  wrote:
>
> I try to build zeppelin on windows, but hit the following error. Does
> anyone know how to address this ? Thanks
>
>
> [ERROR] npm ERR! Windows_NT 6.1.7601
> [ERROR] npm ERR! argv
> "C:\\Users\\jzhang\\zeppelin\\zeppelin-web\\node\\node.exe"
> "C:\\Users\\jzhang\\zeppelin\\zeppelin
> -web\\node\\node_modules\\npm\\bin\\npm-cli.js" "run" "build"
> [ERROR] npm ERR! node v6.9.1
> [ERROR] npm ERR! npm  v3.10.8
> [ERROR] npm ERR! code ELIFECYCLE
> [ERROR] npm ERR! zeppelin-web@0.0.0 build: `grunt build`
> [ERROR] npm ERR! Exit status 3
> [ERROR] npm ERR!
> [ERROR] npm ERR! Failed at the zeppelin-web@0.0.0 build script 'grunt
> build'.
> [ERROR] npm ERR! Make sure you have the latest version of node.js and npm
> installed.
> [ERROR] npm ERR! If you do, this is most likely a problem with the
> zeppelin-web package,
> [ERROR] npm ERR! not with npm itself.
> [ERROR] npm ERR! Tell the author that this fails on your system:
> [ERROR] npm ERR! grunt build
> [ERROR] npm ERR! You can get information on how to open an issue for this
> project with:
> [ERROR] npm ERR! npm bugs zeppelin-web
> [ERROR] npm ERR! Or if that isn't available, you can get their info via:
> [ERROR] npm ERR! npm owner ls zeppelin-web
> [ERROR] npm ERR! There is likely additional logging output above.
> [ERROR]
> [ERROR] npm ERR! Please include the following file with any support
> request:
> [ERROR] npm ERR! C:\Users\jzhang\zeppelin\zeppelin-web\npm-debug.log
> [INFO]
> 
>
>
>

Re: Exporting Spark paragraphs as Spark Applications

[Jeff Zhang]

Thanks Luciano. IIRC, what user want is to run the whole spark app, but
they don't care about whether it is in zeppelin or through a standard spark
app jar. I know zeppelin currently doesn't do well in converting note to
production spark app as Lee mentioned. But exporting note as jar seems a
short term solution, not a long term solution. I just feel when zeppelin
improve in this field, user might abandon this solution and transit to
zeppelin again. Here's some disadvantages I can see of this approach.

1.  If user want to change the code in iterative development , they have to
repeat the whole pipeline (change code in zeppelin -> export it to spark
jar -> redeploy this jar). This process is painful and wasting time.
2.  Hard to debug and diagnose as code is changed/restructured when
exporting to jar
3.  User have to leverage several distinct tools for the whole development
cycle (zeppelin, spark job server, and maybe cron job)

Besides,  the OOM issue of Spark REPL Lee mentioned might not be a problem.
Because we can shutdown the app (close interpreter) after the app is done.





Luciano Resende 于2017年1月5日周四 下午3:59写道：

Some use cases discussed earlier on this thread:

https://www.mail-archive.com/dev@zeppelin.apache.org/msg06323.html

https://www.mail-archive.com/dev@zeppelin.apache.org/msg06332.html

On Wed, Jan 4, 2017 at 4:51 PM, Jianfeng (Jeff) Zhang <
jzh...@hortonworks.com> wrote:

>
> I don¹t understand why user want to export zeppelin note as spark
> application.
>
> If they want to trigger the running of spark app, why not use zeppelin¹s
> rest api for that. Even user export it as spark application, most of time
> in reality, they need to submit it through spark job server, so why not
> use zeppelin as a spark job server.
> And if the spark app fails, it is pretty hard to debug it, because the
> exporting tool has changed/restructured the source code.
>
>
> If this is a pretty large and complicated spark application, I don¹t think
> zeppelin is a proper tool for that, they¹d better to use IDE for that
> project.
>
> BTW, After https://github.com/apache/zeppelin/pull/1799, user can define
> the dependency between paragraphs, and they can run one whole note which
> contains different interpreters.
>
>
>
> Best Regard,
> Jeff Zhang
>
>
>
>
>
> On 1/5/17, 2:25 AM, "Luciano Resende"  wrote:
>
> >I have made some progress with a tool to handle the points discussed in
> >this thread. It's currently a command line tool and given a Zeppelin
> >notebook (note.json) it generates a Spark scala application, compiles it
> >using the compiler embedded in the scala sdk and then package all these
> >resources into a jar that works with spark-submit command.
> >
> >I would like to start prototyping the integration into the Zeppelin UI
and
> >I was wondering if it would be ok to use the above jar as a dependency
> >(e.g. from a maven release) and integrate into zeppelin...
> >
> >Thoughts ?
> >
> >
> >On Mon, Sep 19, 2016 at 7:47 AM, Sourav Mazumder <
> >sourav.mazumde...@gmail.com> wrote:
> >
> >> To Moon's point, This is what my vision is around this feature -
> >>
> >> 1. Use should be able to package 1, more than one, all of the
> >>paragraphs in
> >> a Notebook to create a Jar file which can be used with Spark-Submit.
> >>
> >> 2. The tool should automatically remove the all the interactive
> >>statements
> >> like print, show etc.
> >>
> >> 3. The tool should automatically create a Main class in addition to the
> >>jar
> >> file(s) which will internally call the respective jar. User can then
> >>change
> >> this main class if needed for parameterization through Args.
> >>
> >> Regards,
> >> Sourav
> >>
> >> On Mon, Sep 19, 2016 at 7:33 AM, Sourav Mazumder <
> >> sourav.mazumde...@gmail.com> wrote:
> >>
> >> > I am also pretty much for this.
> >> >
> >> > I have got the similar request from each and every people/group who I
> >> > showcased Zeppelin.Regards,
> >> > Sourav
> >> >
> >> > On Fri, Sep 16, 2016 at 8:06 PM, moon soo Lee 
> wrote:
> >> >
> >> >> Hi Luciano,
> >> >>
> >> >> I've also got a lot of questions about "Productize the notebook"
> >>every
> >> >> time
> >> >> i meet users use Zeppelin in their work.
> >> >>
> >> >> I think it's actually about two different problems that Zeppelin
> >>need to
> >> >>

Re: Exporting Spark paragraphs as Spark Applications

[Jeff Zhang]

Thanks Luciano.  I am not saying the community don't feel this is a good
idea. It's just my personal opinion (maybe with some bias, I didn't talk
with many customers as you)  I just feel maybe you can spend time on
improving zeppelin to make zeppelin to do the job rather than exporting the
jar and leverage other tools to deploy the jar.  Because I don't want you
to waste time that maybe finally you find out customer are happy to do that
in one central place: zeppelin.  Anyway, this is just my personal thinking,
you can talk with your customers to hear their feedback.


Luciano Resende 于2017年1月6日周五 上午5:01写道：

> Hi Jeff,
>
> While I agree with you that what you mentioned is completely acceptable for
> some users, particularly regarding the data science personas. Having said
> that, while talking with multiple enterprise companies, that have their own
> scheduler infrastructure with different quality of service or just want to
> deploy this as an app into their production environment which will have
> much more resources for running these apps with complete data sets, and
> currently they finish the experiment/development of the application in an
> interactive environment and them move their final code into a native spark
> application.
>
> Zeppelin is evolving quickly in this area, and I think that export as an
> application might be a good option for users that want to actually deploy
> their notebooks as native applications into their own Spark cluster.
>
> Having said that, if the community feels that this is not a required
> function in Zeppelin anymore, then I can continue with the development of
> the tool as a standalone command line tool. I was even thinking about
> expanding the functionality and implementing what is described in
> ZEPPELIN-1793.
>
> Thoughts ?
>
> On Thu, Jan 5, 2017 at 12:38 AM, Jeff Zhang  wrote:
>
> > Thanks Luciano. IIRC, what user want is to run the whole spark app, but
> > they don't care about whether it is in zeppelin or through a standard
> spark
> > app jar. I know zeppelin currently doesn't do well in converting note to
> > production spark app as Lee mentioned. But exporting note as jar seems a
> > short term solution, not a long term solution. I just feel when zeppelin
> > improve in this field, user might abandon this solution and transit to
> > zeppelin again. Here's some disadvantages I can see of this approach.
> >
> > 1.  If user want to change the code in iterative development , they have
> to
> > repeat the whole pipeline (change code in zeppelin -> export it to spark
> > jar -> redeploy this jar). This process is painful and wasting time.
> > 2.  Hard to debug and diagnose as code is changed/restructured when
> > exporting to jar
> > 3.  User have to leverage several distinct tools for the whole
> development
> > cycle (zeppelin, spark job server, and maybe cron job)
> >
> > Besides,  the OOM issue of Spark REPL Lee mentioned might not be a
> problem.
> > Because we can shutdown the app (close interpreter) after the app is
> done.
> >
> >
> >
> >
> >
> > Luciano Resende 于2017年1月5日周四 下午3:59写道：
> >
> > Some use cases discussed earlier on this thread:
> >
> > https://www.mail-archive.com/dev@zeppelin.apache.org/msg06323.html
> >
> > https://www.mail-archive.com/dev@zeppelin.apache.org/msg06332.html
> >
> > On Wed, Jan 4, 2017 at 4:51 PM, Jianfeng (Jeff) Zhang <
> > jzh...@hortonworks.com> wrote:
> >
> > >
> > > I don¹t understand why user want to export zeppelin note as spark
> > > application.
> > >
> > > If they want to trigger the running of spark app, why not use
> zeppelin¹s
> > > rest api for that. Even user export it as spark application, most of
> time
> > > in reality, they need to submit it through spark job server, so why not
> > > use zeppelin as a spark job server.
> > > And if the spark app fails, it is pretty hard to debug it, because the
> > > exporting tool has changed/restructured the source code.
> > >
> > >
> > > If this is a pretty large and complicated spark application, I don¹t
> > think
> > > zeppelin is a proper tool for that, they¹d better to use IDE for that
> > > project.
> > >
> > > BTW, After https://github.com/apache/zeppelin/pull/1799, user can
> define
> > > the dependency between paragraphs, and they can run one whole note
> which
> > > contains different interpreters.
> > >
> > >
> > >
> > > Best Regard,
> > > Jeff Zhang
> > >
> > >
> > &

Re: Exporting Spark paragraphs as Spark Applications

[Jeff Zhang]

Hi Luciano, maybe I am wrong, just my two cents for your consideration.



Jeff Zhang 于2017年1月6日周五 上午8:32写道：

>
> Thanks Luciano.  I am not saying the community don't feel this is a good
> idea. It's just my personal opinion (maybe with some bias, I didn't talk
> with many customers as you)  I just feel maybe you can spend time on
> improving zeppelin to make zeppelin to do the job rather than exporting the
> jar and leverage other tools to deploy the jar.  Because I don't want you
> to waste time that maybe finally you find out customer are happy to do that
> in one central place: zeppelin.  Anyway, this is just my personal thinking,
> you can talk with your customers to hear their feedback.
>
>
> Luciano Resende 于2017年1月6日周五 上午5:01写道：
>
> Hi Jeff,
>
> While I agree with you that what you mentioned is completely acceptable for
> some users, particularly regarding the data science personas. Having said
> that, while talking with multiple enterprise companies, that have their own
> scheduler infrastructure with different quality of service or just want to
> deploy this as an app into their production environment which will have
> much more resources for running these apps with complete data sets, and
> currently they finish the experiment/development of the application in an
> interactive environment and them move their final code into a native spark
> application.
>
> Zeppelin is evolving quickly in this area, and I think that export as an
> application might be a good option for users that want to actually deploy
> their notebooks as native applications into their own Spark cluster.
>
> Having said that, if the community feels that this is not a required
> function in Zeppelin anymore, then I can continue with the development of
> the tool as a standalone command line tool. I was even thinking about
> expanding the functionality and implementing what is described in
> ZEPPELIN-1793.
>
> Thoughts ?
>
> On Thu, Jan 5, 2017 at 12:38 AM, Jeff Zhang  wrote:
>
> > Thanks Luciano. IIRC, what user want is to run the whole spark app, but
> > they don't care about whether it is in zeppelin or through a standard
> spark
> > app jar. I know zeppelin currently doesn't do well in converting note to
> > production spark app as Lee mentioned. But exporting note as jar seems a
> > short term solution, not a long term solution. I just feel when zeppelin
> > improve in this field, user might abandon this solution and transit to
> > zeppelin again. Here's some disadvantages I can see of this approach.
> >
> > 1.  If user want to change the code in iterative development , they have
> to
> > repeat the whole pipeline (change code in zeppelin -> export it to spark
> > jar -> redeploy this jar). This process is painful and wasting time.
> > 2.  Hard to debug and diagnose as code is changed/restructured when
> > exporting to jar
> > 3.  User have to leverage several distinct tools for the whole
> development
> > cycle (zeppelin, spark job server, and maybe cron job)
> >
> > Besides,  the OOM issue of Spark REPL Lee mentioned might not be a
> problem.
> > Because we can shutdown the app (close interpreter) after the app is
> done.
> >
> >
> >
> >
> >
> > Luciano Resende 于2017年1月5日周四 下午3:59写道：
> >
> > Some use cases discussed earlier on this thread:
> >
> > https://www.mail-archive.com/dev@zeppelin.apache.org/msg06323.html
> >
> > https://www.mail-archive.com/dev@zeppelin.apache.org/msg06332.html
> >
> > On Wed, Jan 4, 2017 at 4:51 PM, Jianfeng (Jeff) Zhang <
> > jzh...@hortonworks.com> wrote:
> >
> > >
> > > I don¹t understand why user want to export zeppelin note as spark
> > > application.
> > >
> > > If they want to trigger the running of spark app, why not use
> zeppelin¹s
> > > rest api for that. Even user export it as spark application, most of
> time
> > > in reality, they need to submit it through spark job server, so why not
> > > use zeppelin as a spark job server.
> > > And if the spark app fails, it is pretty hard to debug it, because the
> > > exporting tool has changed/restructured the source code.
> > >
> > >
> > > If this is a pretty large and complicated spark application, I don¹t
> > think
> > > zeppelin is a proper tool for that, they¹d better to use IDE for that
> > > project.
> > >
> > > BTW, After https://github.com/apache/zeppelin/pull/1799, user can
> define
> > > the dependency between paragraphs, and they can run one whole note
> which
&g

Re: Windows support ?

[Jeff Zhang]

I think most of the community development is on *nix. Although zeppelin
support windows,  it would be very probable to break the windows support as
zeppelin is evolving fast and very few developer use windows. Not sure
whether travis support windows, if yes, we can add one windows build to
prevent the break on windows.



Sourav Mazumder 于2017年1月6日周五 上午10:53写道：

> I ported successfully a previous version of Zeppelin.
> If needed we can start with that as baseline after making necessary
> changes for the latest version.
>
> Regards,
> Sourav
>
> > On Jan 6, 2017, at 3:23 AM, Luciano Resende 
> wrote:
> >
> > I see a few sporadic issues/questions related to windows, which seems to
> > get lower priority and I was trying the building Zeppelin in windows
> > environment and there are some clear issues.
> >
> > - What is the official word around Zeppelin supporting Windows?
> >
> > - Is Windows a supported development environment for Zeppelin (e.g.
> > building)?
> >
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
>

Re: Unstable travis CI recently

[Jeff Zhang]

Should we disable these flaky test now ? CI seems become more unstable
recently. It is almost useless for me, I never see a success CI recently.
Here's one screenshot of recent closed PRs. Most of them has CI failure.
IMO, this is pretty bad, especially for new contributors.



[image: pasted1]





Jun Kim 于2016年12月13日周二 下午11:27写道：

> @Hoon Thanks for your information :-) I should use that next time!
>
> 2016년 12월 13일 (화) 오후 8:23, Park Hoon <1am...@gmail.com>님이 작성:
>
> > I totally agree with your opinions. I will work on ZEPPELIN-1739,
> > ZEPPELIN-1749 first i reported before.
> >
> > @Jun Kim. So true. We have to wait long time. FYI, we can use our own
> > travis CI containers to test (I recently learned also!) by configuring
> > your-github-id/zeppelin-repo in travis CI
> >
> > Thanks!
> >
> > On Tue, Dec 13, 2016 at 8:19 PM, Jun Kim  wrote:
> >
> > > I definitely agree with you!
> > >
> > > I reopened my PR twice recently to pass CI and it wasn't because of me.
> > >
> > > CI takes about ~40min for a test, so I had to wait 1h and 20min to
> write
> > a
> > > comment after passing CI T_T
> > >
> > > And the worst of it is that I don't believe CI's result more and more.
> > >
> > > 2016년 12월 13일 (화) 오후 8:10, Jeff Zhang 님이 작성:
> > >
> > > > Hi Folks,
> > > >
> > > > As you may notice that our travis CI is not stable recently. There's
> > many
> > > > flaky test, and it waste every developer's time to figure out whether
> > the
> > > > failure is due to your PR or flaky test. So I think it is time for us
> > to
> > > > make the CI stable. Here's tickets for all the flaky test.
> > > >
> > > > https://issues.apache.org/jira/issues/?jql=project%20%
> > > 3D%20ZEPPELIN%20AND%20text%20~%20flaky%20and%20status%20!%
> > > 3D%20RESOLVED%20ORDER%20%20BY%20status%20ASC%20
> > > >
> > > > Fixing the flaky test may take some time and may not easy for some
> test
> > > but
> > > > I think it is worth to do that.  And it is better for these people
> who
> > > > familiar with that particular test case to fix it. What do you guys
> > > think ?
> > > > Thanks
> > > >
> > > --
> > > Taejun Kim
> > >
> > > Data Mining Lab.
> > > School of Electrical and Computer Engineering
> > > University of Seoul
> > >
> >
> --
> Taejun Kim
>
> Data Mining Lab.
> School of Electrical and Computer Engineering
> University of Seoul
>

Re: Unstable travis CI recently

[Jeff Zhang]

Hi Jongyoul,

Could you move all these tickets under this umbrella ticket ?

You can do bulk change by clicking the gear tool button on top right of
this page. ( I don't' have privilege to do that)


https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20flaky%20and%20status%20!%3D%20RESOLVED%20ORDER%20%20BY%20status%20ASC%20




Jongyoul Lee 于2017年1月19日周四 下午12:11写道：

> Thanks Jeff,
>
> I made a new PR for handling this issue.
>
> https://issues.apache.org/jira/browse/ZEPPELIN-1981
>
> On Thu, Jan 19, 2017 at 9:47 AM, Jianfeng (Jeff) Zhang <
> jzh...@hortonworks.com> wrote:
>
> > +1
> >
> > Best Regard,
> > Jeff Zhang
> >
> >
> >
> >
> >
> > On 1/19/17, 1:03 AM, "Jongyoul Lee"  wrote:
> >
> > >I also agree that ppl don't care of the result of CI anymore even it's
> > >real
> > >failure. One possible solution is making umbrella ticket, grabbing flaky
> > >tests, disabling at first and enabling when it solves. but it assumes we
> > >need to do our best to fix the flaky tests. Otherwise, we will lose some
> > >tests...
> > >
> > >How do you guys think of it?
> > >
> > >On Thu, Jan 19, 2017 at 1:56 AM, Felix Cheung <
> felixcheun...@hotmail.com>
> > >wrote:
> > >
> > >> I'd agree. Is there a course of actions you can propose? Disable all
> > >>these
> > >> tests is a not a long term solution, right?
> > >>
> > >>
> > >> _
> > >> From: Jeff Zhang mailto:zjf...@gmail.com>>
> > >> Sent: Tuesday, January 17, 2017 6:01 PM
> > >> Subject: Re: Unstable travis CI recently
> > >> To: mailto:dev@zeppelin.apache.org>>
> > >>
> > >>
> > >>
> > >> Should we disable these flaky test now ? CI seems become more unstable
> > >> recently. It is almost useless for me, I never see a success CI
> > >>recently.
> > >> Here's one screenshot of recent closed PRs. Most of them has CI
> failure.
> > >> IMO, this is pretty bad, especially for new contributors.
> > >>
> > >>
> > >>
> > >> [pasted1]
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> Jun Kim mailto:i2r@gmail.com>>?2016?12?13???
> > >> ??11:27???
> > >> @Hoon Thanks for your information :-) I should use that next time!
> > >>
> > >> 2016? 12? 13? (?) ?? 8:23, Park Hoon <1am...@gmail.com<mailto:1amb4
> > >> a...@gmail.com>>?? ??:
> > >>
> > >> > I totally agree with your opinions. I will work on ZEPPELIN-1739,
> > >> > ZEPPELIN-1749 first i reported before.
> > >> >
> > >> > @Jun Kim. So true. We have to wait long time. FYI, we can use our
> own
> > >> > travis CI containers to test (I recently learned also!) by
> configuring
> > >> > your-github-id/zeppelin-repo in travis CI
> > >> >
> > >> > Thanks!
> > >> >
> > >> > On Tue, Dec 13, 2016 at 8:19 PM, Jun Kim
> > >>mailto:i2r.
> > >> j...@gmail.com>> wrote:
> > >> >
> > >> > > I definitely agree with you!
> > >> > >
> > >> > > I reopened my PR twice recently to pass CI and it wasn't because
> of
> > >>me.
> > >> > >
> > >> > > CI takes about ~40min for a test, so I had to wait 1h and 20min to
> > >> write
> > >> > a
> > >> > > comment after passing CI T_T
> > >> > >
> > >> > > And the worst of it is that I don't believe CI's result more and
> > >>more.
> > >> > >
> > >> > > 2016? 12? 13? (?) ?? 8:10, Jeff Zhang  > zjffd
> > >> u...@gmail.com>>?? ??:
> > >> > >
> > >> > > > Hi Folks,
> > >> > > >
> > >> > > > As you may notice that our travis CI is not stable recently.
> > >>There's
> > >> > many
> > >> > > > flaky test, and it waste every developer's time to figure out
> > >>whether
> > >> > the
> > >> > > > failure is due to your PR or flaky test. So I think it is time
> > >>for us
> > >> > to
> > >> > > > make the CI stable. Here's tickets for all the flaky test.
> > >> > > >
> > >> > > > https://issues.apache.org/jira/issues/?jql=project%20%
> > >> > > 3D%20ZEPPELIN%20AND%20text%20~%20flaky%20and%20status%20!%
> > >> > > 3D%20RESOLVED%20ORDER%20%20BY%20status%20ASC%20
> > >> > > >
> > >> > > > Fixing the flaky test may take some time and may not easy for
> some
> > >> test
> > >> > > but
> > >> > > > I think it is worth to do that.  And it is better for these
> people
> > >> who
> > >> > > > familiar with that particular test case to fix it. What do you
> > >>guys
> > >> > > think ?
> > >> > > > Thanks
> > >> > > >
> > >> > > --
> > >> > > Taejun Kim
> > >> > >
> > >> > > Data Mining Lab.
> > >> > > School of Electrical and Computer Engineering
> > >> > > University of Seoul
> > >> > >
> > >> >
> > >> --
> > >> Taejun Kim
> > >>
> > >> Data Mining Lab.
> > >> School of Electrical and Computer Engineering
> > >> University of Seoul
> > >>
> > >>
> > >>
> > >
> > >
> > >--
> > >이종열, Jongyoul Lee, 李宗烈
> > >http://madeng.net
> >
> >
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: R interpreter

[Jeff Zhang]

That is SparkR, R interpreter is in the r folder under the root of zeppelin.



Igor Drozdov 于2017年1月19日周四 下午8:06写道：

> Hello!
>
> I'm working on https://issues.apache.org/jira/browse/ZEPPELIN-921 and I
> have some doubts about R Interpreter.
>
> Currently, R interpreter jar is in the same directory as Spark
> interpreter. InterpreterFactory searches for interpreters in spark dir,
> successfully loads interpreter-setting.json for spark and stops scanning
> this folder. So R interpreter is never loaded!
>
> Could anyone confirm that R interpreter does not work for now? Or it is my
> local misconfiguration?
>
> Thank you,
> Igor Drozdov
>

[Discuss] Move some interpreters out of zeppelin project

[Jeff Zhang]

As we talk in another thread [1] about moving some interpreters out of
zeppelin project. I open this thread to discuss it in more details. I'd
like to raise 4 questions for this.

1. Do we need to do this
2. If the answer is yes, which interpreters should be moved out
3. How do we integrate these interpreters into zeppelin
4. How does zeppelin work with these third party interpreters

I will first give my inputs on this.

*1. Do we need to do this ?*
Personally, I strongly +1 on this. Several reasons:

   - Keep the zeppelin project much smaller
   - Each interpreter's improvements won't be blocked by the release of
   zeppelin. Interpreters can has its own release cycle as long as
   zeppelin-interpreter doesn't break the compatibility.
   - Zeppelin developer don't have the knowledge of all interpreters.
   Sometimes it is very difficult for zeppelin committers to review a new
   interpreter that he doesn't know.


2. Which interpreters should be moved out ?
We can discuss it  in another thread about the min package.

3. How do we integrate these interpreters into zeppelin
Currently, user can install third party interpreter by running script (
http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/manual/interpreterinstallation.html#3rd-party-interpreters),
but this is not convienient, and it is hard to let every user to be aware
of this feature. So I think we should do that in zeppelin UI. We should
allow user to install/uninstall/upgrade/downgrade third party interpreters
in the interpreter page.

4. How does zeppelin work with these third party interpreters
Besides the interface zeppelin expose to the third party interpreter to be
install/uninstall/upgrade/downgrade, it is third party interpreter's own
responsibility to develop and make new release.

Please help comment on these 4 questions and feel free to add any things
that I miss.


[1]
https://lists.apache.org/thread.html/69f606409790d7ba11422e8c6df941a75c5dfae0aca63eccf2f840bf@%3Cusers.zeppelin.apache.org%3E

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

[Jeff Zhang]

-1,  Can we include ZEPPELIN-1985
 ? This issue cause
only user1 can see the pig tutorial note which is a very bad user
experience.



Prabhjyot Singh 于2017年1月21日周六 下午7:25写道：

> +1
>
> On Jan 21, 2017 4:52 PM, "Jun Kim"  wrote:
>
> +1
> 2017년 1월 21일 (토) 오후 8:21, CloverHearts 님이 작성:
>
> +1
>
>
>
> The Zeppelin 0.7 release includes some amazing features.
>
> In particular, a major bug fix for the remote interpreter.
>
> I am actively in favor of this release.
>
>
>
>
>
> *출발**: *Mina Lee 
> *회신* *대상**:* 
> *날짜**: *2017년 1월 21일 토요일 오전 3:17
> *받는* *사람**: *, 
> *주제**: *[VOTE] Release Apache Zeppelin 0.7.0 (RC1)
>
>
>
> Hi folks,
>
> I propose the following RC to be released for the Apache Zeppelin 0.7.0
> release.
>
> The commit id is bcbe0855175f55eddd10b918303806eb5e5db091 which is
> corresponds to the tag v0.7.0-rc1:
>
>
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=bcbe0855175f55eddd10b918303806eb5e5db091
>
> The release archives (tgz), signature, and checksums are here
> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc1/
>
> The release candidate consists of the following source distribution
> archive
>
> zeppelin-0.7.0.tgz
>
> In addition, the following supplementary binary distributions are provided
> for user convenience at the same location
>
> zeppelin-0.7.0-bin-all.tgz
> zeppelin-0.7.0-bin-netinst.tgz
>
>
> The maven artifacts are here
> https://repository.apache.org/content/repositories/orgapachezeppelin-1022
>
> You can find the KEYS file here:
> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
>
> Release notes available at
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12336544&projectId=12316221
>
> Vote will be open for next 72 hours (close at 3:30 am 24/Jan PST).
>
> [ ] +1 approve
> [ ] 0 no opinion
> [ ] -1 disapprove (and reason why)
>
> --
> Taejun Kim
>
> Data Mining Lab.
> School of Electrical and Computer Engineering
> University of Seoul
>
>

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

[Jeff Zhang]

Hi Alex & Mina,

Sorry for didn't fill the fix version (maybe my misunderstanding, usually I
use fix version for when it is fixed and target version as when it will be
fixed). It is fine for me to without this ticket in 0.7, I would cancel my
-1.



Alexander Bezzubov 于2017年1月22日周日 上午6:35写道：

> Hi Jeff,
>
> thank you for the notice, it is a good catch!
>
> But the issue that you mention does not have FixFor version assigned - if
> you want it to part of the release and even conside it as important enough
> to block the relese vote, could you please update that?
>
> This would simplify drastically the work of release manager.
>
> Also, as you know, the release branch has been unnounced a while a go, but
> for this issue as far as I can tell, there was no any indication that it
> was intended to be part of it, so seeing -1 is a bit surprising. Do you see
> what I mean?
>
> Release Manager can not read minds, as well as volonteer infinite ammount
> of time preparing a new Relese Candidate after each issue beeing solved.
>
> Cutting a release is a lot of work for Release Manger, so going further
> lets try to simplify life of the brave volonteer helping with release
> management! :)
>
> Hope it does make sense! Please, let us know what you think.
>
> --
> Alex
>
> On Sat, Jan 21, 2017, 21:55 Jeff Zhang  wrote:
>
> > -1,  Can we include ZEPPELIN-1985
> > <https://issues.apache.org/jira/browse/ZEPPELIN-1985> ? This issue cause
> > only user1 can see the pig tutorial note which is a very bad user
> > experience.
> >
> >
> >
> > Prabhjyot Singh 于2017年1月21日周六 下午7:25写道：
> >
> > > +1
> > >
> > > On Jan 21, 2017 4:52 PM, "Jun Kim"  wrote:
> > >
> > > +1
> > > 2017년 1월 21일 (토) 오후 8:21, CloverHearts 님이
> 작성:
> > >
> > > +1
> > >
> > >
> > >
> > > The Zeppelin 0.7 release includes some amazing features.
> > >
> > > In particular, a major bug fix for the remote interpreter.
> > >
> > > I am actively in favor of this release.
> > >
> > >
> > >
> > >
> > >
> > > *출발**: *Mina Lee 
> > > *회신* *대상**:* 
> > > *날짜**: *2017년 1월 21일 토요일 오전 3:17
> > > *받는* *사람**: *, 
> > > *주제**: *[VOTE] Release Apache Zeppelin 0.7.0 (RC1)
> > >
> > >
> > >
> > > Hi folks,
> > >
> > > I propose the following RC to be released for the Apache Zeppelin 0.7.0
> > > release.
> > >
> > > The commit id is bcbe0855175f55eddd10b918303806eb5e5db091 which is
> > > corresponds to the tag v0.7.0-rc1:
> > >
> > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=bcbe0855175f55eddd10b918303806eb5e5db091
> > >
> > > The release archives (tgz), signature, and checksums are here
> > > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc1/
> > >
> > > The release candidate consists of the following source distribution
> > > archive
> > >
> > > zeppelin-0.7.0.tgz
> > >
> > > In addition, the following supplementary binary distributions are
> > provided
> > > for user convenience at the same location
> > >
> > > zeppelin-0.7.0-bin-all.tgz
> > > zeppelin-0.7.0-bin-netinst.tgz
> > >
> > >
> > > The maven artifacts are here
> > >
> >
> https://repository.apache.org/content/repositories/orgapachezeppelin-1022
> > >
> > > You can find the KEYS file here:
> > > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> > >
> > > Release notes available at
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12336544&projectId=12316221
> > >
> > > Vote will be open for next 72 hours (close at 3:30 am 24/Jan PST).
> > >
> > > [ ] +1 approve
> > > [ ] 0 no opinion
> > > [ ] -1 disapprove (and reason why)
> > >
> > > --
> > > Taejun Kim
> > >
> > > Data Mining Lab.
> > > School of Electrical and Computer Engineering
> > > University of Seoul
> > >
> > >
> >
>

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC2)

[Jeff Zhang]

+1



DuyHai Doan 于2017年1月22日周日 下午4:01写道：

> +1
>
> On Sun, Jan 22, 2017 at 8:59 AM, Mina Lee  wrote:
>
> I propose the following RC to be released for the Apache Zeppelin 0.7.0
> release.
>
> The commit id is d0ddf8fdff75b4229543f7ebfd5eb001f685382d which is
> corresponds to the tag v0.7.0-rc2:
> *https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=d0ddf8fdff75b4229543f7ebfd5eb001f685382d
> *
>
> The release archives (tgz), signature, and checksums are here
> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc2/
>
> The release candidate consists of the following source distribution
> archive
> zeppelin-0.7.0.tgz
>
> In addition, the following supplementary binary distributions are provided
> for user convenience at the same location
> zeppelin-0.7.0-bin-all.tgz
> zeppelin-0.7.0-bin-netinst.tgz
>
> The maven artifacts are here
> https://repository.apache.org/content/repositories/orgapachezeppelin-1023
>
> You can find the KEYS file here:
> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
>
> Release notes available at
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12336544&projectId=12316221
>
> Vote will be open for next 72 hours (close at 23:59 24/Jan PST).
>
> [ ] +1 approve
> [ ] 0 no opinion
> [ ] -1 disapprove (and reason why)
>
>
>

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC4)

[Jeff Zhang]

+1

Ahyoung Ryu 于2017年2月2日周四 下午9:36写道：

+1

On Thu, Feb 2, 2017 at 10:07 PM, Jun Kim  wrote:

+1
2017년 2월 2일 (목) 오후 9:49, Sora Lee 님이 작성:

+1

On Thu, Feb 2, 2017 at 9:40 PM Khalid Huseynov  wrote:

> +1
>
> On Thu, Feb 2, 2017 at 9:21 PM, DuyHai Doan  wrote:
>
> +1
>
> On Thu, Feb 2, 2017 at 9:56 AM, Mina Lee  wrote:
>
> > I propose the following RC to be released for the Apache Zeppelin 0.7.0
> > release.
> >
> > The commit id is df007f2284a09caa7c8b35f8b59d5f1993fe8b64 which is
> > corresponds to the tag v0.7.0-rc4:
> > *
>
https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.7.0-rc4
> > <
>
https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.7.0-rc4
> >*
> >
> > The release archives (tgz), signature, and checksums are here
> > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc4/
> >
> > The release candidate consists of the following source distribution
> > archive
> > zeppelin-0.7.0.tgz
> >
> > In addition, the following supplementary binary distributions are
> provided
> > for user convenience at the same location
> > zeppelin-0.7.0-bin-all.tgz
> > zeppelin-0.7.0-bin-netinst.tgz
> >
> > The maven artifacts are here
> >
> https://repository.apache.org/content/repositories/orgapachezeppelin-1027
> >
> > You can find the KEYS file here:
> > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> >
> > Release notes available at
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > version=12336544&projectId=12316221
> >
> > Vote will be open for next 72 hours (close at 01:00 5/Feb PST).
> >
> > [ ] +1 approve
> > [ ] 0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
>
>
>

-- 
Taejun Kim

Data Mining Lab.
School of Electrical and Computer Engineering
University of Seoul

Re: [ANNOUNCE] Apache Zeppelin 0.7.0 released

[Jeff Zhang]

Congratulations, thanks Mina's endeavor and hard work of every release.



Ahyoung Ryu 于2017年2月6日周一 下午3:20写道：

> Congrats ALL the Apache Zeppelin community!
> Thanks for your effort Mina :D
>
> On Mon, Feb 6, 2017 at 4:11 PM, Jun Kim  wrote:
>
> Col!!!
>
> 2017년 2월 6일 (월) 오후 4:10, Anthony Corbacho 님이
> 작성:
>
> Awesome
>
> On Mon, Feb 6, 2017 at 4:07 PM, Mina Lee  wrote:
>
> The Apache Zeppelin community is pleased to announce the availability of
> the 0.7.0 release.
>
> Zeppelin is a collaborative data analytics and visualization tool for
> distributed, general-purpose data processing system such as Apache Spark,
> Apache Flink, etc.
>
> The community put significant effort into improving Apache Zeppelin since
> the last release, focusing on multi-user support, pluggable visualization,
> better interpreter support. More than 100+ contributors provided 700+
> patches
> for new features, improvements and bug fixes. More than 480+ issues have
> been resolved.
>
> We encourage you to download the latest release from
> http://zeppelin.apache.org/download.html
>
> Release note is available at
> http://zeppelin.apache.org/releases/zeppelin-release-0.7.0.html
>
> We welcome your help and feedback. For more information on the project and
> how to get involved, visit our website at http://zeppelin.apache.org/
>
> Thank you all users and contributors who have helped to improve Apache
> Zeppelin.
>
> Regards,
> The Apache Zeppelin community
>
>
> --
> Taejun Kim
>
> Data Mining Lab.
> School of Electrical and Computer Engineering
> University of Seoul
>
>
>

Is there any way to cancel travis build in queue ?

[Jeff Zhang]

There's lots of pending build job in travis build queue, I can cancel some
of my build as I would push a new commit. But I didn't find way to cancel
that. Does anyone know how to cancel it ? Thanks

Require wiki edit permission

[Jeff Zhang]

Hi I am working on PIG-5126 to add document about running pig in zeppelin.
But it is not good practise to include corporate link in apache project. So
I would like to copy the content of below link to zeppelin wiki page and
refer the zeppelin wiki in pig doc.

https://community.hortonworks.com/articles/79183/how-to-use-pig-in-zeppelin.html

Re: Require wiki edit permission

[Jeff Zhang]

Yes


moon soo Lee 于2017年2月10日周五 下午2:17写道：

> Your cwiki id is zjffdu, right?
>
> On Fri, Feb 10, 2017 at 11:05 AM Jeff Zhang  wrote:
>
> > Hi I am working on PIG-5126 to add document about running pig in
> zeppelin.
> > But it is not good practise to include corporate link in apache project.
> So
> > I would like to copy the content of below link to zeppelin wiki page and
> > refer the zeppelin wiki in pig doc.
> >
> >
> >
> https://community.hortonworks.com/articles/79183/how-to-use-pig-in-zeppelin.html
> >
>

Re: Require wiki edit permission

[Jeff Zhang]

Thanks moon.



moon soo Lee 于2017年2月10日周五 下午2:21写道：

> Done.
>
> On Fri, Feb 10, 2017 at 3:19 PM Jeff Zhang  wrote:
>
> > Yes
> >
> >
> > moon soo Lee 于2017年2月10日周五 下午2:17写道：
> >
> > > Your cwiki id is zjffdu, right?
> > >
> > > On Fri, Feb 10, 2017 at 11:05 AM Jeff Zhang  wrote:
> > >
> > > > Hi I am working on PIG-5126 to add document about running pig in
> > > zeppelin.
> > > > But it is not good practise to include corporate link in apache
> > project.
> > > So
> > > > I would like to copy the content of below link to zeppelin wiki page
> > and
> > > > refer the zeppelin wiki in pig doc.
> > > >
> > > >
> > > >
> > >
> >
> https://community.hortonworks.com/articles/79183/how-to-use-pig-in-zeppelin.html
> > > >
> > >
> >
>

Re: [DISCUSS] Scaling out CI capability

[Jeff Zhang]

Great, can't wait to try this.



moon soo Lee 于2017年2月16日周四 下午6:35写道：

> Hi Apache Zeppelin contributors!
>
> Recently, it's not very difficult to see pullrequest is waiting more than
> 10 hours to get travis-ci build slot.
>
> It's because of all ASF project shares the same travis account and the
> account's capability is limited. Although ASF assigned 5 concurrent build
> capacity to Zeppelin, it's not dedicated slot, it's more like max
> concurrent build limit, so when ASF travis is busy, Zeppelin pullrequest
> should wait.
>
> But each contributor can get 5 concurrent build capacity for opensource
> project. if we can leverage that, Zeppelin will no longer suffer from CI
> build capacity.
>
> I created https://issues.apache.org/jira/browse/ZEPPELIN-2123 and it
> includes proposed way, how Zeppelin can leverage individual contributors
> travis account.
> And created required script https://github.com/apache/zeppelin/pull/2026.
>
> Please take a look the proposal. Any feedback would be really appreciated!
>
> Thanks,
> moon
>

Re: [DISCUSS] Behaviour of Interpreter when it is restarted

[Jeff Zhang]

I think we can combine scenario 2 and 3 if user click yes button on the
popup window of whether you want to restart interpreter in scenario 3.

Regarding the restarting scenario of 1,2,3, IMHO I think we don't need to
differentiate them. Otherwise it might confuse users that restarting in
different places have different behavior. Zeppelin as a notebook should not
do too much assumption and do too much extra work implicitly for users, let
user to control what they want to do.

IMHO I think the behavior of restarting should be just restarting the
current user's interpreter and don't affect other users. If it's admin
perform the restarting operation in interpreter setting page, I also think
we should not restart all the users' interpreters by default. Because I
think the admin's intention of updating interpreter setting is just to
update the interpreter setting so that all the users can use the latest
interpreter setting (e.g. update SPARK_HOME in spark interpreter setting.
For now everyone share the same interpreter setting, but in the long term I
think everyone should has his own setting that extend from admin's setting.
But this is another story, not related to this thread ), admin doesn't want
to interrupt and close the current other users' active interpreters. Of
course, this is just my biased thinking, some customer may indeed want to
close all the interpreters when admin perform restarting operation. Then we
can provide one configuration in zeppelin-site to allow user to do that,
but by default I think we should not allow admin to close all users' active
interpreter.

Delete is a very special scenario among them, for now I think we can
terminate all the interpreter processes when interpreter is deleted.
Because after interpreter is deleted, there's no way to shutdown the
interpreter in zeppelin for now. If we don't close and shutdown them, then
that means resource leakage.

Besides these, another thing I want to mention is that there's no dedicated
component or concept in zeppelin to control lifecycle of interpreter. E.g.
for now if user don't restart interpreter, his interpreter will be alive
forever. This is almost unacceptable for enterprise usage.  I think we
should have some component to do that work to manage the lifecycle of
interpreter.




Prabhjyot Singh 于2017年2月22日周三 下午4:17写道：

> This is WRT the PR that I've created
> https://github.com/apache/zeppelin/pull/2034.
>
> The issue that I want to discuss over here is how should an Interpreter
> behave when it is;
>  - restarted from notebook
>  - restarted from Interpreter setting page
>  - edited from Interpreter setting page
>  - deleted from Interpreter setting page
>
>
> Assuming Zeppelin is being used in Enterprise world, where not all user may
> have access to Zeppelin's Interpreter setting page, say only restricted
> user say "admin-group" have access to this page. Now when a restart, edit
> or delete action is performed from Interpreter setting page; any of this
> operation should terminate all the processes of that particular
> Interpreter. On the other hand if it is restarted from the notebook page by
> any User, then only process of that logged-in User should get affected.
>
> How do you guys think of it?
>
> --
>
> Warm Regards,
>
> Prabhjyot Singh
>

Re: [DISCUSS] Admin feature

[Jeff Zhang]

I think in authenticated mode, admin is necessary.  What admin should take
charge can be in the following 4 areas:

1.  Interpreter setting. Admin can control a template for each interpreter,
and each user's template extend this base template but can also customize
his own setting. Besides that, admin can control which user can use which
interpreter. Because some interpreter has some potential security issue, we
should not allow everyone to use it (like shell interpreter)
2.  Note and user management. Admin can control the permission of notes and
users.  Like which user can use which note.
3.  Job management.
4.  Not sure about helium. But I think admin should be responsible for
helium plugins because plugins are external code which could bring some
potential security issue.

Besides that I have 2 concerns:
1.  Is admin mandatory in shrio ? Is it possible that I enable shrio but no
admin is defined ?
2. Should zeppelin take care of these or integrate the authorization of
shiro and let shrio to take care of these ? e.g. We can treat interpreter
and note as resource of shiro, and let shiro to control their permission
and relationship.



Alec Lee 于2017年2月23日周四 下午1:23写道：

> We have multiple uses in our organization share same anonymous account,
> which will potentially cause the problems, we hope to have true
> authenticated mode.
>
> thanks
>
> AL
>
> On Feb 22, 2017, at 9:14 PM, Jongyoul Lee  wrote:
>
> Hi folks,
>
> Recently, I've heard some new feature assumed that it needed the admin
> account or similar role. But Apache Zeppelin doesn't have any admin feature
> like hiding/showing menu and settings. I want to know how community thinks
> about that feature.
>
> My first concern is that we have to consider two modes: anonymous and
> authenticated.
>
> Feel free to start the discussion on pros and cons.
>
> Regards,
> Jongyoul
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
>

Re: Setting zeppelin.livy.url

[Jeff Zhang]

zeppelin-site.xml is for zeppelin server, not for interpreter. You should
set zeppelin.livy.url in interpreter setting page.

https://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/quickstart/explorezeppelinui.html#settings

Arun Khetarpal 于2017年3月15日周三 上午3:16写道：

> Hi,
>
> I am trying to setup the livy interpreter in zeppelin.
>
> For this, i have set zeppelin.livy.url property in zeppelin-site.xml - but
> in Zeppelin ui, the property of interpreter is still set to default.
>
> Alternatively, i tried also setting it zeppelin-env.sh as export result
> ZEPPELIN_LIVY_HOST_URL  but the same result.
>
> Any help?
>
> Regards,
> Arun
>

Re: Setting zeppelin.livy.url

[Jeff Zhang]

Why is it inconvenient for you ? If you have multiple livy servers, you can
create multiple livy interpreters for each livy server in interpreter
setting page.




Arun Khetarpal 于2017年3月15日周三 下午1:25写道：

> Is there a way to set it programmatically? I find it very inconvenient for
> users to go and modify the setting in the page - is there a better way?
>
> Regards,
> Arun
>
> On 15 March 2017 at 04:36, Jeff Zhang  wrote:
>
> > zeppelin-site.xml is for zeppelin server, not for interpreter. You should
> > set zeppelin.livy.url in interpreter setting page.
> >
> > https://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/
> > quickstart/explorezeppelinui.html#settings
> >
> > Arun Khetarpal 于2017年3月15日周三 上午3:16写道：
> >
> > > Hi,
> > >
> > > I am trying to setup the livy interpreter in zeppelin.
> > >
> > > For this, i have set zeppelin.livy.url property in zeppelin-site.xml -
> > but
> > > in Zeppelin ui, the property of interpreter is still set to default.
> > >
> > > Alternatively, i tried also setting it zeppelin-env.sh as export result
> > > ZEPPELIN_LIVY_HOST_URL  but the same result.
> > >
> > > Any help?
> > >
> > > Regards,
> > > Arun
> > >
> >
>

Re: Setting zeppelin.livy.url

[Jeff Zhang]

Still not clear to me, do you have multiple instances of livy server ? If
there is only one livy server, then you just need to change it one time.

If you have multiple livy instances, you always need to update
zeppelin.livy.url no matter through interpreter setting ui or
programmatically.




Arun Khetarpal 于2017年3月15日周三 下午3:35写道：

> The usecase we have is to create a service to spawn up the zepplin. Now
> since i cannot override zeppelin.livy.url  and the default value is set to
> "
> https://localhost:8998"; - users which are using my service need to
> manually
> go and change the interpreter url.
>
> I don't know if the scenario is clear, but i can explain more.
>
> Regards,
> Arun
>
> On 15 March 2017 at 12:58, Jeff Zhang  wrote:
>
> > Why is it inconvenient for you ? If you have multiple livy servers, you
> can
> > create multiple livy interpreters for each livy server in interpreter
> > setting page.
> >
> >
> >
> >
> > Arun Khetarpal 于2017年3月15日周三 下午1:25写道：
> >
> > > Is there a way to set it programmatically? I find it very inconvenient
> > for
> > > users to go and modify the setting in the page - is there a better way?
> > >
> > > Regards,
> > > Arun
> > >
> > > On 15 March 2017 at 04:36, Jeff Zhang  wrote:
> > >
> > > > zeppelin-site.xml is for zeppelin server, not for interpreter. You
> > should
> > > > set zeppelin.livy.url in interpreter setting page.
> > > >
> > > > https://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/
> > > > quickstart/explorezeppelinui.html#settings
> > > >
> > > > Arun Khetarpal 于2017年3月15日周三 上午3:16写道：
> > > >
> > > > > Hi,
> > > > >
> > > > > I am trying to setup the livy interpreter in zeppelin.
> > > > >
> > > > > For this, i have set zeppelin.livy.url property in
> zeppelin-site.xml
> > -
> > > > but
> > > > > in Zeppelin ui, the property of interpreter is still set to
> default.
> > > > >
> > > > > Alternatively, i tried also setting it zeppelin-env.sh as export
> > result
> > > > > ZEPPELIN_LIVY_HOST_URL  but the same result.
> > > > >
> > > > > Any help?
> > > > >
> > > > > Regards,
> > > > > Arun
> > > > >
> > > >
> > >
> >
>

Re: Setting zeppelin.livy.url

[Jeff Zhang]

I have an improved version of solution b.

You can copy interpreter-setting.json (with modified zeppelin.livy.url) to
ZEPPELIN_HOME/interpreter/livy/ to override the default one.



Arun Khetarpal 于2017年3月15日周三 下午10:43写道：

> Hi Jeff -
>
> Thanks for taking out the time to help me out.
>
> Actually i have a case of building a managed service and would like any
> user who spins up a cluster with zeppelin installed, to have it work with
> livy out of the box - since this use case also requires enterprise security
> - following this doc
> <
> https://community.hortonworks.com/articles/80059/how-to-configure-zeppelin-livy-interpreter-for-sec.html
> >
> -
> i would like to have zeppelin.livy.url to be set to a specific livy cluster
> by default and would not like the user to go and change it later.
>
> Now before the zeppelin server is even up, i tried setting the value of
> zeppelin.livy.url in zeppelin-site.xml which as you said, did not work. So
> my question is, how do i override the default value without me even going
> via UI.
>
> a. Go to the UI and change the interpreter setting? - Not an option for my
> use case - an external user would expect a working livy and zeppelin, and
> would not like to go and change the setting after i spin up a zeppelin
> instance for him
>
> b. Before i spin up the zeppelin server for a specific user - I go and
> change interpreter-settings.json by unjar'ing livy-interpreter.jar and
> changing the default value of zeppelin.livy.url? - Obviously a hack! and
> something i will be hanged for :(
>
> c. Override zeppelin.livy.url in some xml or env.sh - which would override
> the default value of interpreter - *Need help on how*.
>
> Regards,
> Arun
>
> On 15 March 2017 at 13:15, Jeff Zhang  wrote:
>
> > Still not clear to me, do you have multiple instances of livy server ? If
> > there is only one livy server, then you just need to change it one time.
> >
> > If you have multiple livy instances, you always need to update
> > zeppelin.livy.url no matter through interpreter setting ui or
> > programmatically.
> >
> >
> >
> >
> > Arun Khetarpal 于2017年3月15日周三 下午3:35写道：
> >
> > > The usecase we have is to create a service to spawn up the zepplin. Now
> > > since i cannot override zeppelin.livy.url  and the default value is set
> > to
> > > "
> > > https://localhost:8998"; - users which are using my service need to
> > > manually
> > > go and change the interpreter url.
> > >
> > > I don't know if the scenario is clear, but i can explain more.
> > >
> > > Regards,
> > > Arun
> > >
> > > On 15 March 2017 at 12:58, Jeff Zhang  wrote:
> > >
> > > > Why is it inconvenient for you ? If you have multiple livy servers,
> you
> > > can
> > > > create multiple livy interpreters for each livy server in interpreter
> > > > setting page.
> > > >
> > > >
> > > >
> > > >
> > > > Arun Khetarpal 于2017年3月15日周三 下午1:25写道：
> > > >
> > > > > Is there a way to set it programmatically? I find it very
> > inconvenient
> > > > for
> > > > > users to go and modify the setting in the page - is there a better
> > way?
> > > > >
> > > > > Regards,
> > > > > Arun
> > > > >
> > > > > On 15 March 2017 at 04:36, Jeff Zhang  wrote:
> > > > >
> > > > > > zeppelin-site.xml is for zeppelin server, not for interpreter.
> You
> > > > should
> > > > > > set zeppelin.livy.url in interpreter setting page.
> > > > > >
> > > > > > https://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/
> > > > > > quickstart/explorezeppelinui.html#settings
> > > > > >
> > > > > > Arun Khetarpal 于2017年3月15日周三 上午3:16写道：
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I am trying to setup the livy interpreter in zeppelin.
> > > > > > >
> > > > > > > For this, i have set zeppelin.livy.url property in
> > > zeppelin-site.xml
> > > > -
> > > > > > but
> > > > > > > in Zeppelin ui, the property of interpreter is still set to
> > > default.
> > > > > > >
> > > > > > > Alternatively, i tried also setting it zeppelin-env.sh as
> export
> > > > result
> > > > > > > ZEPPELIN_LIVY_HOST_URL  but the same result.
> > > > > > >
> > > > > > > Any help?
> > > > > > >
> > > > > > > Regards,
> > > > > > > Arun
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Setting zeppelin.livy.url

[Jeff Zhang]

Actually this is a bug, most of enviroment variable in interpreter-setting
doesn't work. So try to use property instead of enviroment variable.


Arun Khetarpal 于2017年3月16日周四 上午12:40写道：

Thanks Jeff.. In-general, IMO, there should be a better way to override
these settings atleast when i look at interpreter-settings.json (pasted
below the snippet)

It seems i should have, somehow been able to define the environment
variable, ZEPPELIN_LIVY_HOST_URL somewhere to override default.


Thoughts?

(Snippet of interpreter-settings.json )

{
"group": "livy",
"name": "spark",
"className": "org.apache.zeppelin.livy.LivySparkInterpreter",
"defaultInterpreter": true,
"properties": {
"zeppelin.livy.url": {
"envName": "ZEPPELIN_LIVY_HOST_URL",
"propertyName": "zeppelin.livy.url",
"defaultValue": "http://localhost:8998";,
"description": "The URL for Livy Server."
}, .

On 15 March 2017 at 20:21, Jeff Zhang  wrote:

> I have an improved version of solution b.
>
> You can copy interpreter-setting.json (with modified zeppelin.livy.url) to
> ZEPPELIN_HOME/interpreter/livy/ to override the default one.
>
>
>
> Arun Khetarpal 于2017年3月15日周三 下午10:43写道：
>
> > Hi Jeff -
> >
> > Thanks for taking out the time to help me out.
> >
> > Actually i have a case of building a managed service and would like any
> > user who spins up a cluster with zeppelin installed, to have it work
with
> > livy out of the box - since this use case also requires enterprise
> security
> > - following this doc
> > <
> > https://community.hortonworks.com/articles/80059/how-to-
> configure-zeppelin-livy-interpreter-for-sec.html
> > >
> > -
> > i would like to have zeppelin.livy.url to be set to a specific livy
> cluster
> > by default and would not like the user to go and change it later.
> >
> > Now before the zeppelin server is even up, i tried setting the value of
> > zeppelin.livy.url in zeppelin-site.xml which as you said, did not work.
> So
> > my question is, how do i override the default value without me even
going
> > via UI.
> >
> > a. Go to the UI and change the interpreter setting? - Not an option for
> my
> > use case - an external user would expect a working livy and zeppelin,
and
> > would not like to go and change the setting after i spin up a zeppelin
> > instance for him
> >
> > b. Before i spin up the zeppelin server for a specific user - I go and
> > change interpreter-settings.json by unjar'ing livy-interpreter.jar and
> > changing the default value of zeppelin.livy.url? - Obviously a hack! and
> > something i will be hanged for :(
> >
> > c. Override zeppelin.livy.url in some xml or env.sh - which would
> override
> > the default value of interpreter - *Need help on how*.
> >
> > Regards,
> > Arun
> >
> > On 15 March 2017 at 13:15, Jeff Zhang  wrote:
> >
> > > Still not clear to me, do you have multiple instances of livy server ?
> If
> > > there is only one livy server, then you just need to change it one
> time.
> > >
> > > If you have multiple livy instances, you always need to update
> > > zeppelin.livy.url no matter through interpreter setting ui or
> > > programmatically.
> > >
> > >
> > >
> > >
> > > Arun Khetarpal 于2017年3月15日周三 下午3:35写道：
> > >
> > > > The usecase we have is to create a service to spawn up the zepplin.
> Now
> > > > since i cannot override zeppelin.livy.url  and the default value is
> set
> > > to
> > > > "
> > > > https://localhost:8998"; - users which are using my service need to
> > > > manually
> > > > go and change the interpreter url.
> > > >
> > > > I don't know if the scenario is clear, but i can explain more.
> > > >
> > > > Regards,
> > > > Arun
> > > >
> > > > On 15 March 2017 at 12:58, Jeff Zhang  wrote:
> > > >
> > > > > Why is it inconvenient for you ? If you have multiple livy
servers,
> > you
> > > > can
> > > > > create multiple livy interpreters for each livy server in
> interpreter
> > > > > setting page.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Arun Khetarpal 于2017年3月15日周三 下午1:25写道：
> > > > >
> > > > > > Is there a way to set it programmatically? I 

Re: Setting zeppelin.livy.url

[Jeff Zhang]

Hi Arun,

Do you mind to create a ticket for this kind of bug ?


Jeff Zhang 于2017年3月16日周四 下午12:23写道：

> Actually this is a bug, most of enviroment variable in interpreter-setting
> doesn't work. So try to use property instead of enviroment variable.
>
>
> Arun Khetarpal 于2017年3月16日周四 上午12:40写道：
>
> Thanks Jeff.. In-general, IMO, there should be a better way to override
> these settings atleast when i look at interpreter-settings.json (pasted
> below the snippet)
>
> It seems i should have, somehow been able to define the environment
> variable, ZEPPELIN_LIVY_HOST_URL somewhere to override default.
>
>
> Thoughts?
>
> (Snippet of interpreter-settings.json )
>
> {
> "group": "livy",
> "name": "spark",
> "className": "org.apache.zeppelin.livy.LivySparkInterpreter",
> "defaultInterpreter": true,
> "properties": {
> "zeppelin.livy.url": {
> "envName": "ZEPPELIN_LIVY_HOST_URL",
> "propertyName": "zeppelin.livy.url",
> "defaultValue": "http://localhost:8998";,
> "description": "The URL for Livy Server."
> }, .
>
> On 15 March 2017 at 20:21, Jeff Zhang  wrote:
>
> > I have an improved version of solution b.
> >
> > You can copy interpreter-setting.json (with modified zeppelin.livy.url)
> to
> > ZEPPELIN_HOME/interpreter/livy/ to override the default one.
> >
> >
> >
> > Arun Khetarpal 于2017年3月15日周三 下午10:43写道：
> >
> > > Hi Jeff -
> > >
> > > Thanks for taking out the time to help me out.
> > >
> > > Actually i have a case of building a managed service and would like any
> > > user who spins up a cluster with zeppelin installed, to have it work
> with
> > > livy out of the box - since this use case also requires enterprise
> > security
> > > - following this doc
> > > <
> > > https://community.hortonworks.com/articles/80059/how-to-
> > configure-zeppelin-livy-interpreter-for-sec.html
> > > >
> > > -
> > > i would like to have zeppelin.livy.url to be set to a specific livy
> > cluster
> > > by default and would not like the user to go and change it later.
> > >
> > > Now before the zeppelin server is even up, i tried setting the value of
> > > zeppelin.livy.url in zeppelin-site.xml which as you said, did not work.
> > So
> > > my question is, how do i override the default value without me even
> going
> > > via UI.
> > >
> > > a. Go to the UI and change the interpreter setting? - Not an option for
> > my
> > > use case - an external user would expect a working livy and zeppelin,
> and
> > > would not like to go and change the setting after i spin up a zeppelin
> > > instance for him
> > >
> > > b. Before i spin up the zeppelin server for a specific user - I go and
> > > change interpreter-settings.json by unjar'ing livy-interpreter.jar and
> > > changing the default value of zeppelin.livy.url? - Obviously a hack!
> and
> > > something i will be hanged for :(
> > >
> > > c. Override zeppelin.livy.url in some xml or env.sh - which would
> > override
> > > the default value of interpreter - *Need help on how*.
> > >
> > > Regards,
> > > Arun
> > >
> > > On 15 March 2017 at 13:15, Jeff Zhang  wrote:
> > >
> > > > Still not clear to me, do you have multiple instances of livy server
> ?
> > If
> > > > there is only one livy server, then you just need to change it one
> > time.
> > > >
> > > > If you have multiple livy instances, you always need to update
> > > > zeppelin.livy.url no matter through interpreter setting ui or
> > > > programmatically.
> > > >
> > > >
> > > >
> > > >
> > > > Arun Khetarpal 于2017年3月15日周三 下午3:35写道：
> > > >
> > > > > The usecase we have is to create a service to spawn up the zepplin.
> > Now
> > > > > since i cannot override zeppelin.livy.url  and the default value is
> > set
> > > > to
> > > > > "
> > > > > https://localhost:8998"; - users which are using my service need to
> > > > > manually
> > > > > go and change the interpreter url.
> > > > >
> > > > > I don't know if the scenario is clear, but i can explain more.
> > > > >
> > &

Re: Roadmap for 0.8.0

[Jeff Zhang]

Here's some candidates for 0.8 IMO

   - Restructing InterpreterFacotry
   https://issues.apache.org/jira/browse/ZEPPELIN-2056Although it is
   refactoring ticket, I feel it is pretty important thing to do. As I see
   many bugs are due to interpreter factory component,  and I do feel it needs
   refactoring.
   - Admin Feature https://issues.apache.org/jira/browse/ZEPPELIN-2236
   - User Level Interpreter Setting
   https://issues.apache.org/jira/browse/ZEPPELIN-1338
   - Interpreter Lifecycle Control
   https://issues.apache.org/jira/browse/ZEPPELIN-2197



Jongyoul Lee 于2017年3月20日周一 下午12:03写道：

> Hi dev & users,
>
> Recently, community submits very new features for Apache Zeppelin. I think
> it's very positive signals to improve Apache Zeppelin and its community.
> But in another aspect, we should focus on what the next release includes. I
> think we need to summarize and prioritize them. Here is what I know:
>
> * Cluster management
> * Admin feature
> * Replace some context to separate users
> * Helium online
>
> Feel free to talk if you want to add more things. I think we need to choose
> which features will be included in 0.8.0, too.
>
> Regards,
> Jongyoul Lee
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: Roadmap for 0.8.0

[Jeff Zhang]

Yeah, make sense.


Jongyoul Lee 于2017年3月20日周一 下午7:21写道：

> Thanks for letting me know. I agree almost things we should develop.
> Personally, concerning refactoring it, I'm doing a bit with several PRs but
> we need to restructure InterpreterFactory. At first, list up all issues and
> make some groups and handle it. How do you think?
>
> On Mon, Mar 20, 2017 at 2:12 PM, Jeff Zhang  wrote:
>
>
>
> Here's some candidates for 0.8 IMO
>
>- Restructing InterpreterFacotry
>https://issues.apache.org/jira/browse/ZEPPELIN-2056Although it is
>refactoring ticket, I feel it is pretty important thing to do. As I see
>many bugs are due to interpreter factory component,  and I do feel it needs
>refactoring.
>- Admin Feature https://issues.apache.org/jira/browse/ZEPPELIN-2236
>- User Level Interpreter Setting
>https://issues.apache.org/jira/browse/ZEPPELIN-1338
>- Interpreter Lifecycle Control
>https://issues.apache.org/jira/browse/ZEPPELIN-2197
>
>
>
> Jongyoul Lee 于2017年3月20日周一 下午12:03写道：
>
> Hi dev & users,
>
> Recently, community submits very new features for Apache Zeppelin. I think
> it's very positive signals to improve Apache Zeppelin and its community.
> But in another aspect, we should focus on what the next release includes. I
> think we need to summarize and prioritize them. Here is what I know:
>
> * Cluster management
> * Admin feature
> * Replace some context to separate users
> * Helium online
>
> Feel free to talk if you want to add more things. I think we need to choose
> which features will be included in 0.8.0, too.
>
> Regards,
> Jongyoul Lee
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: [VOTE] Release Apache Zeppelin 0.7.1 (RC1)

[Jeff Zhang]

0

1. Build it from source successfully
2. Run pig tutorial using tez successfully, and dynamic form works
3. Run livy interpreter via 2 users, 2 livy sessions are created, and
restarting works in scoped per user mode
4. If interpreter fails to start, the exception in interpreter is not
propagated to frontend. I believe this is a regression. I try it in both
livy & spark interpreter.

Here's the error I see in frontend.


[image: pasted1]


Mina Lee 于2017年3月22日周三 上午9:51写道：

> I propose the following RC to be released for the Apache Zeppelin 0.7.1
> release.
>
> The commit id is 22b299b5c17c0c9ad8e54ee15bb7fde645472c65 which is
> corresponds to the tag v0.7.1-rc1:
>
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;h=refs/tags/v0.7.1-rc1
>
> The release archives (tgz), signature, and checksums are here
> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.1-rc1/
>
> The release candidate consists of the following source distribution archive
> zeppelin-0.7.1.tgz
>
> In addition, the following supplementary binary distributions are provided
> for user convenience at the same location
> zeppelin-0.7.1-bin-all.tgz
> zeppelin-0.7.1-bin-netinst.tgz
>
> The maven artifacts are here
> https://repository.apache.org/content/repositories/orgapachezeppelin-1033
>
> You can find the KEYS file here:
> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
>
> Release notes available at
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12339166&projectId=12316221
>
> Vote will be open for next 72 hours (close at 19:00 24/Mar PDT).
>
> [ ] +1 approve
> [ ] 0 no opinion
> [ ] -1 disapprove (and reason why)
>

Re: [VOTE] Release Apache Zeppelin 0.7.1 (RC1)

[Jeff Zhang]

I will change my vote to -1, as I feel this is a very critical bug which
cause bad user experience. User don't know what's wrong if interpreter
fails to start unless checking log.
But in enterprise usage, most likely user don't have permission to access
log.

I created ticket https://issues.apache.org/jira/browse/ZEPPELIN-2294


Jeff Zhang 于2017年3月22日周三 上午11:19写道：


0

1. Build it from source successfully
2. Run pig tutorial using tez successfully, and dynamic form works
3. Run livy interpreter via 2 users, 2 livy sessions are created, and
restarting works in scoped per user mode
4. If interpreter fails to start, the exception in interpreter is not
propagated to frontend. I believe this is a regression. I try it in both
livy & spark interpreter.

Here's the error I see in frontend.


[image: pasted1]


Mina Lee 于2017年3月22日周三 上午9:51写道：

I propose the following RC to be released for the Apache Zeppelin 0.7.1
release.

The commit id is 22b299b5c17c0c9ad8e54ee15bb7fde645472c65 which is
corresponds to the tag v0.7.1-rc1:
https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;h=refs/tags/v0.7.1-rc1

The release archives (tgz), signature, and checksums are here
https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.1-rc1/

The release candidate consists of the following source distribution archive
zeppelin-0.7.1.tgz

In addition, the following supplementary binary distributions are provided
for user convenience at the same location
zeppelin-0.7.1-bin-all.tgz
zeppelin-0.7.1-bin-netinst.tgz

The maven artifacts are here
https://repository.apache.org/content/repositories/orgapachezeppelin-1033

You can find the KEYS file here:
https://dist.apache.org/repos/dist/release/zeppelin/KEYS

Release notes available at
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12339166&projectId=12316221

Vote will be open for next 72 hours (close at 19:00 24/Mar PDT).

[ ] +1 approve
[ ] 0 no opinion
[ ] -1 disapprove (and reason why)

Re: Zeppelin Spark Sql Interpreter CompileException

[Jeff Zhang]

I suspect it is due to spark issue. Can you run it in spark shell local
mode to verify that ?


Chaoran Yu 于2017年3月23日周四 上午10:19写道：

> Hello,
>
>I’m running Zeppelin 0.7.0 with Mesos.
>
>   I tried to run the tutorial on this page
> https://zeppelin.apache.org/docs/0.7.0/quickstart/tutorial.html
>   The regular Spark code in Scala worked fine but I got a
> org.codehaus.commons.compiler.CompileException when running the Spark Sql
> code. Stacktrace is attached.
>
>   I thought this was a problem with Spark code generation (
> https://issues.apache.org/jira/browse/SPARK-18125), but running the same
> code in Zeppelin on my local machine worked fine. So I suspect this has
> something to do with Zeppelin on Mesos.
>
>   Has anyone run into the same problem and can offer me guidance?
>
> Thank you,
> Chaoran Yu
>
>

Re: [VOTE] Release Apache Zeppelin 0.7.1 (RC2)

[Jeff Zhang]

Sorry, I need to -1 again, as I just found that ZEPPELIN-1984 is also
needed to backport. Otherwise no error message displayed in frontend when
interpreter fail to start.



Sora Lee 于2017年3月24日周五 下午2:14写道：

> I faced a error when switching the scatter chart, the error message that is
> "cannot property 'x' of null" appear in web development tool of web
> browser. so I created this PR (
> https://github.com/apache/zeppelin/pull/2186
> ).
>
> Shell we include this patch in 0.7.1 release?
>
> Thanks,
> Sora
>
> On Fri, Mar 24, 2017 at 12:32 PM Mina Lee  wrote:
>
> > I propose the following RC to be released for the Apache Zeppelin 0.7.1
> > release.
> >
> > The commit id is 163f6730ea91486c581f502863fe62b3cd9b06d4 which is
> > corresponds to the tag v0.7.1-rc2:
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;h=refs/tags/v0.7.1-rc2
> >
> > The release archives (tgz), signature, and checksums are here
> > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.1-rc2/
> >
> > The release candidate consists of the following source distribution
> archive
> > zeppelin-0.7.1.tgz
> >
> > In addition, the following supplementary binary distributions are
> provided
> > for user convenience at the same location
> > zeppelin-0.7.1-bin-all.tgz
> > zeppelin-0.7.1-bin-netinst.tgz
> >
> > The maven artifacts are here
> >
> https://repository.apache.org/content/repositories/orgapachezeppelin-1034
> >
> > You can find the KEYS file here:
> > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> >
> > Release notes available at
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12339166&projectId=12316221
> >
> > Vote will be open for next 72 hours (close at 21:00 26/Mar PDT).
> >
> > [ ] +1 approve
> > [ ] 0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
>

Re: [VOTE] Release Apache Zeppelin 0.7.1 (RC3)

[Jeff Zhang]

Might find another bug about restarting button. The restarting button on
note page behaves differently from that of interpreter page.  This might
confuse users.

e.g. For livy interpreter, if I click the restart button in note page, then
only my livy session will be shutdown. But If I click the restart button in
interpreter page, all the sessions will be shut down. This kind of
differences will confuse users IMO.



Mina Lee 于2017年3月28日周二 下午6:53写道：

> I propose the following RC to be released for the Apache Zeppelin 0.7.1
> release.
>
> The commit id is 9eb77adfb3d9969f84d1657b78a95b58e44b5be8 which is
> corresponds to the tag v0.7.1-rc3:
>
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;h=refs/tags/v0.7.1-rc3
>
> The release archives (tgz), signature, and checksums are here
> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.1-rc3/
>
> The release candidate consists of the following source distribution archive
> zeppelin-0.7.1.tgz
>
> In addition, the following supplementary binary distributions are provided
> for user convenience at the same location
> zeppelin-0.7.1-bin-all.tgz
> zeppelin-0.7.1-bin-netinst.tgz
>
> The maven artifacts are here
> https://repository.apache.org/content/repositories/orgapachezeppelin-1035
>
> You can find the KEYS file here:
> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
>
> Release notes available at
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12339166&projectId=12316221
>
> Vote will be open for next 72 hours (close at 04:00 31/Mar PDT).
>
> [ ] +1 approve
> [ ] 0 no opinion
> [ ] -1 disapprove (and reason why)
>

Re: [VOTE] Release Apache Zeppelin 0.7.1 (RC3)

[Jeff Zhang]

Thanks Jongyoul for explanation, let's improve it later.

+1.

* Build zeppelin from source successfully
* Run pig tutorial using tez mode successfully
* Run spark tutorial using livy interpreter successfully.
* Verify interpreter fail error can be propagated to frontend.
* Verify the restart button works in several scenarios. Works properly.




Jongyoul Lee 于2017年3月29日周三 上午11:37写道：

> Jeff,
>
> Moon and I discuss the behaviors of clicking restart buttons from notebook
> page and interpreter page. In your case, it's intended fully. For more
> detail, In case of "per user", we can do the same thing in note and
> interpreter pages, but in case of "per note", we cannot get the exact note
> when users click restart button in interpreter tab. Thus Moon and I decided
> not to change original behaviors of interpreter page. It would be changed
> when we add admin feature.
>
> Thanks for testing it.
>
> Regards,
> Jongyoul
>
> On Wed, Mar 29, 2017 at 11:47 AM, Jeff Zhang  wrote:
>
> > Might find another bug about restarting button. The restarting button on
> > note page behaves differently from that of interpreter page.  This might
> > confuse users.
> >
> > e.g. For livy interpreter, if I click the restart button in note page,
> then
> > only my livy session will be shutdown. But If I click the restart button
> in
> > interpreter page, all the sessions will be shut down. This kind of
> > differences will confuse users IMO.
> >
> >
> >
> > Mina Lee 于2017年3月28日周二 下午6:53写道：
> >
> > > I propose the following RC to be released for the Apache Zeppelin 0.7.1
> > > release.
> > >
> > > The commit id is 9eb77adfb3d9969f84d1657b78a95b58e44b5be8 which is
> > > corresponds to the tag v0.7.1-rc3:
> > >
> > > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;h=
> > refs/tags/v0.7.1-rc3
> > >
> > > The release archives (tgz), signature, and checksums are here
> > > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.1-rc3/
> > >
> > > The release candidate consists of the following source distribution
> > archive
> > > zeppelin-0.7.1.tgz
> > >
> > > In addition, the following supplementary binary distributions are
> > provided
> > > for user convenience at the same location
> > > zeppelin-0.7.1-bin-all.tgz
> > > zeppelin-0.7.1-bin-netinst.tgz
> > >
> > > The maven artifacts are here
> > > https://repository.apache.org/content/repositories/
> > orgapachezeppelin-1035
> > >
> > > You can find the KEYS file here:
> > > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> > >
> > > Release notes available at
> > >
> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > version=12339166&projectId=12316221
> > >
> > > Vote will be open for next 72 hours (close at 04:00 31/Mar PDT).
> > >
> > > [ ] +1 approve
> > > [ ] 0 no opinion
> > > [ ] -1 disapprove (and reason why)
> > >
> >
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Could not find or load main class run when using yarn run dev

[Jeff Zhang]

Hi Folks,

I follow the instruction here to debug zeppelin-web, but got the following
error,
Do I miss something ? Thanks

   jzhangMBPr:zeppelin-web jzhang$ yarn run dev
   Error: Could not find or load main class run

Re: Could not find or load main class run when using yarn run dev

[Jeff Zhang]

That's it, thanks.



Remy Gayet 于2017年3月29日周三 下午5:14写道：

> Is there any chance you are using hadoop-yarn instead of yarn pkg manager (
> https://yarnpkg.com/lang/en/)?
>
> On Wed, Mar 29, 2017 at 9:48 AM, Jeff Zhang  wrote:
>
> > Hi Folks,
> >
> > I follow the instruction here to debug zeppelin-web, but got the
> following
> > error,
> > Do I miss something ? Thanks
> >
> >jzhangMBPr:zeppelin-web jzhang$ yarn run dev
> >Error: Could not find or load main class run
> >
>
>
>
> --
> Remy GAYET,
> +33 6 798 798 62.
>

Re: Roadmap for 0.8.0

[Jeff Zhang]

Hi Ruslan,

Regarding 'make zeppelinContext available in shell interpreter', you may
want to check https://issues.apache.org/jira/browse/ZEPPELIN-1595


Ruslan Dautkhanov 于2017年4月3日周一 下午12:05写道：

> That's exciting to see plans for 0.8.0 on the horizon.
>
> Here's my top list for 0.8 :
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle
> timeout"
>  This is a most-wanted feature by our Zeppelin admins. It was mentioned at
> least once on this email chain.
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z
> variables to Shell Interpreter"
>  We had several of our users asking about this functionality. %sh and some
> other interpreters can't be
>  parametrized by global variables. ZEPPELIN-1967 is one way of how this
> can be solved.
>
> - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory
> references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected"
>   Less of a critical compared to the above two, but it could complement
> the multi-tenancy feature very well.
>
>
> Best regards,
> Ruslan Dautkhanov
>
> On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung 
> wrote:
>
> +1 with latest/stable.
>
>
>
>
> --
> *From:* moon soo Lee 
> *Sent:* Tuesday, March 21, 2017 8:41:58 AM
> *To:* us...@zeppelin.apache.org
> *Cc:* dev@zeppelin.apache.org
>
> *Subject:* Re: Roadmap for 0.8.0
>
>
> And if i suggest simplest way for us to set quality expectation to user,
> which will be labeling release in download page.
>
> Currently releases are divided into 2 categories in download page. 'Latest
> release' and 'Old releases'. I think we can treat 'Latest' as unstable and
> add one more category 'Stable release'.
>
> For example, once 0.8.0 is released,
>
> Latest release : 0.8.0
> Stable release : 0.7.1
> Old release : 0.6.2, 0.6.1 
>
> Once we feel confident about the stability of latest release, we can just
> change label from latest to stable in the download page. (and previous
> stable goes to old releases)
> We can even include formal vote for moving release from 'latest' to
> 'stable' in our release process, if it is necessary.
>
> Thanks,
> moon
>
> On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee  wrote:
>
> Yes, having longer RC period will help.
>
> But if i recall 0.7.0 release, although 21 people participated verifying
> through 4 RC for 15days, it wasn't enough to catch all critical problems
> during the release process. After the release, we've got much more number
> of bug reports, in next few days.
>
> Basically, verifying RC is limited to people who subscribe mailing list +
> willing to contribute time to verify RC, which is much smaller number of
> people who download release from download page. So having longer RC period
> will definitely help and i think we should do, but I think it's still not
> enough to make sure the quality, considering past history.
>
> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project.
> ASF release process defines how to release source code, but it does not
> really restrict what kind of 'version' the project should have releases.
> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0.
>
> Thanks,
> moon
>
> [1] http://spark.apache.org/news/spark-2.0.0-preview.html
>
> On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee  wrote:
>
> I agree that it will help prolong RC period and use it actually. And also
> we need code freeze for the new features and spend time to stabilize RC.
>
> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung 
> wrote:
>
> +1 on quality and stabilization.
>
> I'm not sure if releasing as preview or calling it unstable fits with the
> ASF release process though.
>
> Other projects have code freeze, RC (and longer RC iteration time) etc. -
> do we think those will help improve quality when the release is finally cut?
>
>
> _
> From: Jianfeng (Jeff) Zhang 
> Sent: Monday, March 20, 2017 6:13 PM
> Subject: Re: Roadmap for 0.8.0
>
> To: , dev 
>
>
>
>
>
> Strongly +1 for adding system test for different interpreter modes and
> focus on bug fixing than new features. I do heard from some users complain
> about the bugs of zeppelin major release. A stabilized release is very
> necessary for community.
>
>
>
>
> Best Regard,
> Jeff Zhang
>
>
> From: moon soo Lee mailto:m...@apache.org
> >>
>
> Reply-To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org
>

Re: Zeppelin 0.7.1 unintentionally run a function on declaration

[Jeff Zhang]

Can you paste your code to reproduce it ? Your screenshot can not be
displayed.



Kevin Kim (Sangwoo) 于2017年4月10日周一 下午4:49写道：

[image: Screen Shot 2017-04-10 at 5.46.29 PM.png]

Hi everyone, I'm suffering from the problem Z is unintentionally running
functions on declaration.

(You may easily reproduce it)

Does anyone have idea on this?

Regards,
Kevin

Re: Zeppelin 0.7.1 unintentionally run a function on declaration

[Jeff Zhang]

hmm, I can reproduce it, please help file a ticket for it



Kevin Kim (Sangwoo) 于2017年4月10日周一 下午5:34写道：

> code
> ===
> def test() {
> println("this must not print.")
> }
>
>
> result
> ===
> test: ()Unit
> this must not print.
>
>
> 2017년 4월 10일 (월) 오후 5:55, Jeff Zhang 님이 작성:
>
> > Can you paste your code to reproduce it ? Your screenshot can not be
> > displayed.
> >
> >
> >
> > Kevin Kim (Sangwoo) 于2017年4月10日周一 下午4:49写道：
> >
> > [image: Screen Shot 2017-04-10 at 5.46.29 PM.png]
> >
> > Hi everyone, I'm suffering from the problem Z is unintentionally running
> > functions on declaration.
> >
> > (You may easily reproduce it)
> >
> > Does anyone have idea on this?
> >
> > Regards,
> > Kevin
> >
>

Re: Custom Interpreter group is not displayed on the GUI

[Jeff Zhang]

Please check the document here.
https://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/manual/interpreterinstallation.html



Minudika Malshan 于2017年4月11日周二 下午9:40写道：

> Hi,
>
> I am writing a custom interpreter for apache zeppelin. I added the
> interpreter class to the zeppelin-site.xml and also updated the
> interpreter-setting.json of my interpreter's module.
> But when build the project and start the zeppelin server, the interpreter
> group I given for my interpreter is not shown in the drop down.
> Could you please let me know how to resolve this?
>
> Please find my code here. [1]
>
> [1]
>
> https://github.com/minudika/zeppelin/tree/WSO2_Siddhi_Interpreter/wso2_receiver
>
> Also I have committed my current updated zeppelin-site.xml.
>
> Thank you!
>
> --
> *Minudika Malshan*
> Undergraduate
> Department of Computer Science and Engineering
> University of Moratuwa
> Sri Lanka.
>

CI status is lost in PR page

[Jeff Zhang]

I didn't see the CI status in the PR page recently, I need to check travis
manually. Does anyone experience the same thing ?



[image: image.png]

Re: CI status is lost in PR page

[Jeff Zhang]

It is fixed now, thanks moon.



moon soo Lee 于2017年4月26日周三 下午2:24写道：

> Yes, somehow, PR page does not display CI status. I don't know exact
> reason. I filed an INFRA issue
> https://issues.apache.org/jira/browse/INFRA-14005.
>
> On Tue, Apr 25, 2017 at 10:42 PM Jeff Zhang  wrote:
>
> > I didn't see the CI status in the PR page recently, I need to check
> travis
> > manually. Does anyone experience the same thing ?
> >
> >
> >
> > [image: image.png]
> >
>

Re: Windows Build Support

[Jeff Zhang]

Make sense for option 2


moon soo Lee 于2017年5月3日周三 上午9:21写道：

> Moving thread to dev@
>
> Currently CI on pullrequest is
>
>  - Individual contributor runs Travis - CI
>  - ASF's Jenkins connected to Apache Zeppelin github repo, and get
> information from each contributor's Travis build.
>  - ASF's Appveyor build each pullrequest.
>
> CI on branch (master, branch-0.7, ...) is
>  - ASF's Jenkins build new commits on branch
>  - ASF's Appveyor build new commits on branch
>
>
> Since Appveyor is enabled, I can see Zeppelin PR build on appveyor waits
> very long time in the queue. I think we can have several options on it.
>
> 1. Keep it as is,
>
> Pros - Each pullrequest will be verified on windows environment before
> merge
> Cons - Long wait to get a slot in Appveyor
>
> 2. Disable Appveyor on Pullreuqest build, keep enable Appveyor on branch
> build.
>
> Pros - Each pullrequest will not wait long time for Appveyor finish the
> job.
> Cons - If somethings broken on windows, we'll know after PR is merged.
>
> 3. Let individual contributor setup Appveyor and make Jenkins check it,
> like we do it with Travis
>
> Pros - Both Travis, Appveyor test can be finished quickly on each PR
> Cons - Each individual contributor need to setup both Travis, Appveyor.
>
>
> To me, 2) looks reasonable because most of PR are not related to the
> windows environment, so recognizing some PR break windows support after
> merge will be rare event. And of course we can just submit another patch to
> fix that.
>
> 3) is little bit too much for me, asking all contributors setup both travis
> and appveyor.
>
> What do you guys think?
>
> Thanks,
> moon
>
> On Wed, Apr 26, 2017 at 6:31 PM moon soo Lee  wrote:
>
> > Filed an infra issue to enable appveyor support.
> > https://issues.apache.org/jira/browse/INFRA-14019
> >
> > Thanks,
> > moon
> >
> >
> > On Fri, Apr 21, 2017 at 2:22 AM Park Hoon <1am...@gmail.com> wrote:
> >
> >> Thanks for reply @moon.
> >>
> >>
> >> Additionally, i create umbrella issue for windows support
> >>
> >> - https://issues.apache.org/jira/browse/ZEPPELIN-2440
> >>
> >>
> >> Regard,
> >>
> >> Hoon Park
> >>
> >> On Thu, Apr 20, 2017 at 12:28 PM, moon soo Lee  wrote:
> >>
> >>> Hi,
> >>>
> >>> Thanks for staring the thread.
> >>>
> >>> I've seen almost every month people report build broken on windows.
> >>> Windows CI env will definitely help solving the problem.
> >>>
> >>> I'm +1 on setting up appveyor for the project.
> >>>
> >>> Due to very high usage on ASF's travis account, Zeppelin project guide
> >>> each contributor setup travis account and check success/fail using
> Jenkins.
> >>>
> >>> I don't know how busy ASF's appveyor account. But I think we can first
> >>> try just turn on appveyor with ASF's account and see how busy it is.
> >>>
> >>> Thanks,
> >>> moon
> >>>
> >>>
> >>> On Wed, Apr 19, 2017 at 5:15 AM Park Hoon <1am...@gmail.com> wrote:
> >>>
>  Hi all,
> 
>  Some people are struggling with build Zeppelin in windows.
> 
>  -
> 
> http://stackoverflow.com/questions/40498638/install-zeppelin-on-windows/42560442#42560442
> 
>  IMO, it's enough to verify that zeppelin works well on windows.
> 
>  But it's not that hard to setup CI for windows and it's possible to
>  execute integration tests on windows, it's useful. As you know, we
> have
>  some issues related with windows env
> 
>  -
> 
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZEPPELIN%20and%20text%20~%20windows
> 
>  I found that some apache projects already use appveyor to run CI on
>  windows. (sadly, travis doesn't support)
> 
>  - (thrift) https://github.com/apache/thrift/blob/master/appveyor.yml
>  - (arrow) https://issues.apache.org/jira/browse/INFRA-12967
> 
> 
> 
>  Thanks.
> 
> 
> 
> >>
>

Re: Zeppelin at scale

[Jeff Zhang]

I don't think anyone has that scale of usage (1k users) of Zeppelin for
now. It is interesting to know your usage scenario.

Carlos Diogo  于2021年4月1日周四 上午1:56写道：

> Hi
> My two cents . The only way I know to scale this would be with a container
> based deployment like open shift . You would have isolation per user which
> makes each user process run in its own pod .
> In addition you could setup multiple zeppelin servers ( on the above
> mention way) and then have a load balancer in the middle ( Nginx for
> instance )
> You can have a common notebook repository such as S3 or an NFS share
> Finally , if you want to enable scheduling you would need to ensure that
> only one of the servers have the scheduling enabled so that the jobs only
> run in one instance .
> Hope it helps
> Carlos
>
> On Tue 30. Mar 2021 at 11:57, Great Info  wrote:
>
>> I see zeppelin does not have cluster deployment support.
>> Is there any work to support zeppelin access to multiple users.
>> Kindly share some links/guides if there is already discussion/solution
>> around this.
>>
>> A simple use case, many users(Atleast 1k users)  want to access sql
>> interpreter(jdbc postgres) write show read queries and select some
>> aggregation on the result then chart it.
>>
>> Thanks,
>> gub
>>
> --
> Os meus cumprimentos / Best regards /  Mit freundlichen Grüße
> Carlos Diogo
>


-- 
Best Regards

Jeff Zhang

Re: Zeppelin 0.9.0 Tag

[Jeff Zhang]

Yes, you can use that commit. Actually we had tag v0.9.0 before, but after
releasing 0.9.0, we found there's one issue in its docker file which cause
its docker image, so have create do delete that tag and create a new tag

Danny Cranmer  于2021年3月26日周五 下午11:35写道：

> Hello,
>
> I am trying to find the code commit that corresponds to the 0.9.0 Zeppelin
> release in Maven central, however I cannot see a tag. Can someone please
> confirm this is the correct commit:
> -
>
> https://github.com/apache/zeppelin/commit/9b839b5ae34ce42a350a78ec40e762ddf904a480
>
> And also, it would be good to add a tag!
>
> Thanks,
>


-- 
Best Regards

Jeff Zhang

CI is failed due to github reverse dns lookup

[Jeff Zhang]

Hi Folks,

Our CI is failed recently, it seems due to github reverse dns lookup. For
more information, please take at look at this thread.
https://github.community/t/github-action-and-oserror-errno-99-cannot-assign-requested-address/173973/9


-- 
Best Regards

Jeff Zhang

Re: CI is failed due to github reverse dns lookup

[Jeff Zhang]

I have created PR to fix this issue.

https://github.com/apache/zeppelin/pull/4096

Jeff Zhang  于2021年4月17日周六 下午11:49写道：

> Hi Folks,
>
> Our CI is failed recently, it seems due to github reverse dns lookup. For
> more information, please take at look at this thread.
>
> https://github.community/t/github-action-and-oserror-errno-99-cannot-assign-requested-address/173973/9
>
>
> --
> Best Regards
>
> Jeff Zhang
>


-- 
Best Regards

Jeff Zhang

Re: CI is failed due to github reverse dns lookup

[Jeff Zhang]

I have merged this PR, please rebase your PR as well.


Jeff Zhang  于2021年4月18日周日 下午6:38写道：

> I have created PR to fix this issue.
>
> https://github.com/apache/zeppelin/pull/4096
>
> Jeff Zhang  于2021年4月17日周六 下午11:49写道：
>
>> Hi Folks,
>>
>> Our CI is failed recently, it seems due to github reverse dns lookup. For
>> more information, please take at look at this thread.
>>
>> https://github.community/t/github-action-and-oserror-errno-99-cannot-assign-requested-address/173973/9
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>


-- 
Best Regards

Jeff Zhang

Re: CI is failed due to github reverse dns lookup

[Jeff Zhang]

It looks like azure update their environment, but introduces new issues, I
still see several failed tests, If you see these kind of failed tests, you
can just ignore them.


Tests in error:
1273
<https://github.com/zjffdu/zeppelin/runs/2425231811?check_suite_focus=true#step:9:1273>
PodPhaseWatcherTest.testPhase:48 » KubernetesClient Operation: [create] for
k...
1274
<https://github.com/zjffdu/zeppelin/runs/2425231811?check_suite_focus=true#step:9:1274>
PodPhaseWatcherTest.testPhaseWithError:78 » KubernetesClient Operation:
[creat...
1275
<https://github.com/zjffdu/zeppelin/runs/2425231811?check_suite_focus=true#step:9:1275>
K8sRemoteInterpreterProcessTest.testK8sStartFailed:468 » KubernetesClient
Oper...
1276
<https://github.com/zjffdu/zeppelin/runs/2425231811?check_suite_focus=true#step:9:1276>
K8sRemoteInterpreterProcessTest.testK8sStartSuccessful:427 »
KubernetesClient ...
1277
<https://github.com/zjffdu/zeppelin/runs/2425231811?check_suite_focus=true#step:9:1277>
K8sRemoteInterpreterProcessTest.testK8sStartTimeoutPending:529 »
KubernetesClient

Jeff Zhang  于2021年4月19日周一 下午12:08写道：

> I have merged this PR, please rebase your PR as well.
>
>
> Jeff Zhang  于2021年4月18日周日 下午6:38写道：
>
>> I have created PR to fix this issue.
>>
>> https://github.com/apache/zeppelin/pull/4096
>>
>> Jeff Zhang  于2021年4月17日周六 下午11:49写道：
>>
>>> Hi Folks,
>>>
>>> Our CI is failed recently, it seems due to github reverse dns lookup.
>>> For more information, please take at look at this thread.
>>>
>>> https://github.community/t/github-action-and-oserror-errno-99-cannot-assign-requested-address/173973/9
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>


-- 
Best Regards

Jeff Zhang

Re: CI is failed due to github reverse dns lookup

[Jeff Zhang]

Thanks Philipp, so should we upgrade kubernetes-client ?
BTW, I just fixed another CI issue of pty4j downloading issue. Please
rebase your PR if you want to try it in CI.
https://github.com/apache/zeppelin/commit/97874624a236aebfa0b9384be16be889f428c7b7




Philipp Dallig  于2021年5月3日周一 下午4:23写道：

> Hello developers,
>
> I have found the main problem for our test errors.
> https://github.com/fabric8io/kubernetes-client/issues/2145
>
> The main problem is that the new JDK 8u252 breaks okhttp.
>
> With best regards
>
> Philipp
>
> Am 03.05.21 um 09:47 schrieb Philipp Dallig:
> > Hi Jeff,
> >
> > I can reproduce the K8s test failures locally. I am working on a
> > solution.
> >
> > Best Regards
> >
> > Philipp
> >
> > Am 24.04.21 um 08:39 schrieb Jeff Zhang:
> >> It looks like azure update their environment, but introduces new
> >> issues, I
> >> still see several failed tests, If you see these kind of failed
> >> tests, you
> >> can just ignore them.
> >>
> >>
> >> Tests in error:
> >> 1273
> >> <
> https://github.com/zjffdu/zeppelin/runs/2425231811?check_suite_focus=true#step:9:1273>
>
> >>
> >> PodPhaseWatcherTest.testPhase:48 » KubernetesClient Operation:
> >> [create] for
> >> k...
> >> 1274
> >> <
> https://github.com/zjffdu/zeppelin/runs/2425231811?check_suite_focus=true#step:9:1274>
>
> >>
> >> PodPhaseWatcherTest.testPhaseWithError:78 » KubernetesClient Operation:
> >> [creat...
> >> 1275
> >> <
> https://github.com/zjffdu/zeppelin/runs/2425231811?check_suite_focus=true#step:9:1275>
>
> >>
> >> K8sRemoteInterpreterProcessTest.testK8sStartFailed:468 »
> >> KubernetesClient
> >> Oper...
> >> 1276
> >> <
> https://github.com/zjffdu/zeppelin/runs/2425231811?check_suite_focus=true#step:9:1276>
>
> >>
> >> K8sRemoteInterpreterProcessTest.testK8sStartSuccessful:427 »
> >> KubernetesClient ...
> >> 1277
> >> <
> https://github.com/zjffdu/zeppelin/runs/2425231811?check_suite_focus=true#step:9:1277>
>
> >>
> >> K8sRemoteInterpreterProcessTest.testK8sStartTimeoutPending:529 »
> >> KubernetesClient
> >>
> >> Jeff Zhang  于2021年4月19日周一 下午12:08写道：
> >>
> >>> I have merged this PR, please rebase your PR as well.
> >>>
> >>>
> >>> Jeff Zhang  于2021年4月18日周日 下午6:38写道：
> >>>
> >>>> I have created PR to fix this issue.
> >>>>
> >>>> https://github.com/apache/zeppelin/pull/4096
> >>>>
> >>>> Jeff Zhang  于2021年4月17日周六 下午11:49写道：
> >>>>
> >>>>> Hi Folks,
> >>>>>
> >>>>> Our CI is failed recently, it seems due to github reverse dns lookup.
> >>>>> For more information, please take at look at this thread.
> >>>>>
> >>>>>
> https://github.community/t/github-action-and-oserror-errno-99-cannot-assign-requested-address/173973/9
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Best Regards
> >>>>>
> >>>>> Jeff Zhang
> >>>>>
> >>>>
> >>>> --
> >>>> Best Regards
> >>>>
> >>>> Jeff Zhang
> >>>>
> >>>
> >>> --
> >>> Best Regards
> >>>
> >>> Jeff Zhang
> >>>
> >>
>


-- 
Best Regards

Jeff Zhang

Re: Slack channel for Zeppelin community

[Jeff Zhang]

done

Andreas Weise  于2021年5月26日周三 下午10:47写道：

> Hi Jeff,
>
> could you add me as well to the slack chan
> andreas.weise at gmail.com
>
> Thanks in advance.
> Andreas
>
> On 2021/02/10 15:17:42, Jeff Zhang  wrote:
> > Hi Folks,
> >
> > We have an apache slack channel for Zeppelin, but we didn't make lots of
> > discussion there,  but I think it is suitable for many kinds of
> discussion,
> > specially we recently talked about regular community sync up meeting. So
> if
> > you are interested, please leave your slack account in this thread, I
> will
> > invite you to join it.
> >
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>


-- 
Best Regards

Jeff Zhang

Re: Remote Interpreter

[Jeff Zhang]

Yes，it is supported. You can either k8s mode (interpreter runs in pod ) or
yarn mode (interpreter runs in yarn container)

http://zeppelin.apache.org/docs/0.9.0/quickstart/kubernetes.html
http://zeppelin.apache.org/docs/0.9.0/quickstart/yarn.html


denny wong  于2021年6月23日周三 下午10:14写道：

> Hi
> We are using Zeppelin 0.9 and running the remote interpreter as a docker
> container.  Currently, the Zeppelin server and the remote interpreter are
> running on the same machine.  We would like to know if the remote
> interpreter can be run on a different machine than the Zeppelin server.  Is
> this currently supported?
> Thanks
> Denny



-- 
Best Regards

Jeff Zhang

Re: Binding interpreter settings in s3

[Jeff Zhang]

Which version of zeppelin do you use ? If you are using 0.9, the
interpreter binding should not be necessary.


Great Info  于2021年6月28日周一 上午2:07写道：

> Hi,
>  I am storing the zeppelin notebook in s3 and due to some internal
> requirement, I need to rebuild my image and deploy the apps. Whenever I
> re-deploy the image(which has a zeppelin) and open any pre-created zeppelin
> notebook, the notebook asks to select and bind the interpreter.
>
> Is there a way to persist the interpreter binding other than the instance
> disk?
>
> Thanks,
> Cod
>


-- 
Best Regards

Jeff Zhang

Re: How to disable creating a new notebook for few users

[Jeff Zhang]

I don't think zeppelin has such a feature yet.

Great Info  于2021年7月2日周五 上午2:42写道：

> I am using Zeppelin  0.8.2, different kinds of users use our zeppelin.
> There is one category of users wherein they just need to run the notebook.
> I know how to set the authorization to disable 'edit or running' the
> notebook but I did not find the option to disable creating a new notebook
> for these users.
>
> Thanks,
> Cod
>


-- 
Best Regards

Jeff Zhang

Re: R Interpreter Problem

[Jeff Zhang]

What version of spark do you use ? I am not sure whether your spark version
supports R 4.0 ?

denny wong  于2021年7月2日周五 下午10:02写道：

> Hi
> We have problem with the R interpreter when using R version 4.0.5.  We are
> seeing this error when the zeppelin_sparkr.R is being run at the startup
> time.  How to resolve the issue?  Thanks.
> > .libPaths(c(file.path(libPath), .libPaths()))
> > library(SparkR) Error: package or namespace load failed for 'SparkR':
> package 'SparkR' was installed before R 4.0.0: please re-install it
> Execution halted
>


-- 
Best Regards

Jeff Zhang

Re: R Interpreter Problem

[Jeff Zhang]

Hi Denny,

I tried %r.ir on R 4.0.5, and it works well for me, please check the
following 2 links for how to use %r.ir

http://zeppelin.apache.org/docs/0.9.0/interpreter/r.html#using-the-r-interpreterrr-rir
http://zeppelin-notebook.com/#/notebook/2BWJFTXKJ


[image: image.png]

denny wong  于2021年7月2日周五 下午10:40写道：

>  Hi Jeff
> Initially, we used the Spark version that defined by the Zep 0.9.0 and got
> the error.
> When we changed the Spark to the latest version 3.1.2, we got a different
> error.  Fyi, we are not using Spark at all, but it seems the R interpreter
> has dependency on the SparkR.  Our requirement is to enable R interpreter
> with the latest R version.  Please advice.  Thanks.
> > .libPaths(c(file.path(libPath), .libPaths()))
> > library(SparkR)
> Error in readRDS(pfile) : unknown input format
> Calls: library -> find.package -> lapply -> FUN -> readRDS
> Execution halted
>
> On Friday, July 2, 2021, 10:26:09 AM EDT, Jeff Zhang 
> wrote:
>
>  What version of spark do you use ? I am not sure whether your spark
> version
> supports R 4.0 ?
>
> denny wong  于2021年7月2日周五 下午10:02写道：
>
> > Hi
> > We have problem with the R interpreter when using R version 4.0.5.  We
> are
> > seeing this error when the zeppelin_sparkr.R is being run at the startup
> > time.  How to resolve the issue?  Thanks.
> > > .libPaths(c(file.path(libPath), .libPaths()))
> > > library(SparkR) Error: package or namespace load failed for 'SparkR':
> > package 'SparkR' was installed before R 4.0.0: please re-install it
> > Execution halted
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
Best Regards

Jeff Zhang

[DISCUSS] Releasing 0.9.1

[Jeff Zhang]

Hi Folks,

It has been about half year since the last release of 0.9.0, after that we
have many fixes and improvements, what do you think of the release of 0.9.1
?

-- 
Best Regards

Jeff Zhang

Re: [DISCUSS] Releasing 0.9.1

[Jeff Zhang]

Hi Paul,

What kind of issue do you have when building from source code ?

Paul Brenner  于2021年7月6日周二 上午3:37写道：

> We have been having more and more trouble getting builds from source
> working so would love a new release.
>
> Paul Brenner
>
> Head of Data Science
> *pbren...@placeiq.com*  | (217) 390-3033 |
> *www.placeiq.com* <https://www.placeiq.com>
> twitter *@placeiq* <https://twitter.com/PlaceIQ> linkedin */placeiq*
> <https://www.linkedin.com/company/placeiq/>
> [image: Check out PlaceIQ's latest COVID-19 research]
> <https://www.placeiq.com/blog/?utm_source=email&utm_medium=signature&utm_campaign=blogsubscriber>
> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
> <http://go.placeiq.com/WC01DPG0l0N3Gq00e00>
> On Jul 5, 2021, 10:46 AM -0400, Jeff Zhang , wrote:
>
> Hi Folks,
>
> It has been about half year since the last release of 0.9.0, after that we
> have many fixes and improvements, what do you think of the release of 0.9.1
> ?
>
> --
> Best Regards
>
> Jeff Zhang
>
>

-- 
Best Regards

Jeff Zhang

Re: [ZEPPELIN-5414] set value for shell.command.timeout.millisecs

[Jeff Zhang]

Thanks Eric, it makes sense to me to remove the timeout threshold. Actually
user can cancel shell paragraph in Zeppelin UI, so I think it makes sense
to let user to decide whether he want to cancel his shell script execution.



Eric Gao  于2021年7月5日周一 下午4:29写道：

> Hi,
>
> The default threshold set for shell.command.timeout.millisecs is 60
> seconds, which seems confusing. Users will find their process get killed if
> they take more than 60 seconds. I’m wondering whether it is a better idea
> to change it to unlimited?
>
> Thanks!



-- 
Best Regards

Jeff Zhang

Re: Zeppelin 0.9.0 shanpshot build

[Jeff Zhang]

0.9.0 has been released, just download it here.
http://zeppelin.apache.org/download.html


Great Info  于2021年7月24日周六 上午9:52写道：

> I want to install Apache Zeppelin 0.9.0-SNAPSHOT but cannot seem to find
> the proper link.
> Is the build available for the latest snapshot or do I need to build on my
> own?
>
> Please guide me.
>


-- 
Best Regards

Jeff Zhang

Re: Zeppelin 0.9.0 shanpshot build

[Jeff Zhang]

run the following command to build zeppelin binary distribution:

mvn clean package -Pbuild-distr  -DskipRat -DskipTests -pl
zeppelin-distribution -am

Great Info  于2021年7月24日周六 上午11:49写道：

> I want the latest changes also, like
> https://github.com/apache/zeppelin/pull/4002.
>
> I don't think this build has these recent merges
>
> On Sat, Jul 24, 2021 at 8:33 AM Jeff Zhang  wrote:
>
>> 0.9.0 has been released, just download it here.
>> http://zeppelin.apache.org/download.html
>>
>>
>> Great Info  于2021年7月24日周六 上午9:52写道：
>>
>>> I want to install Apache Zeppelin 0.9.0-SNAPSHOT but cannot seem to find
>>> the proper link.
>>> Is the build available for the latest snapshot or do I need to build on
>>> my own?
>>>
>>> Please guide me.
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>

-- 
Best Regards

Jeff Zhang

Re: Zeppelin 0.9.0 shanpshot build

[Jeff Zhang]

zeppelin-distribution/target

Great Info  于2021年7月24日周六 下午2:04写道：

> thanks the build was successful, I don't see any dist folder, what are the
> folder I should copy ?. I just need lasted build with JDBC interpreter
> and s3notebook repo
>
>
> On Sat, Jul 24, 2021 at 9:35 AM Jeff Zhang  wrote:
>
>> run the following command to build zeppelin binary distribution:
>>
>> mvn clean package -Pbuild-distr  -DskipRat -DskipTests -pl
>> zeppelin-distribution -am
>>
>> Great Info  于2021年7月24日周六 上午11:49写道：
>>
>> > I want the latest changes also, like
>> > https://github.com/apache/zeppelin/pull/4002.
>> >
>> > I don't think this build has these recent merges
>> >
>> > On Sat, Jul 24, 2021 at 8:33 AM Jeff Zhang  wrote:
>> >
>> >> 0.9.0 has been released, just download it here.
>> >> http://zeppelin.apache.org/download.html
>> >>
>> >>
>> >> Great Info  于2021年7月24日周六 上午9:52写道：
>> >>
>> >>> I want to install Apache Zeppelin 0.9.0-SNAPSHOT but cannot seem to
>> find
>> >>> the proper link.
>> >>> Is the build available for the latest snapshot or do I need to build
>> on
>> >>> my own?
>> >>>
>> >>> Please guide me.
>> >>>
>> >>
>> >>
>> >> --
>> >> Best Regards
>> >>
>> >> Jeff Zhang
>> >>
>> >
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>

-- 
Best Regards

Jeff Zhang

Re: Disable notebook trash option

[Jeff Zhang]

I don't think there's such feature in Zeppelin right now, could you create
a ticket for it ?

Great Info  于2021年7月26日周一 上午2:08写道：

> Due to some reason I need to disable the trash option for notebooks.
> Just need to get the notebooks removed after deleting the notebooks
>


-- 
Best Regards

Jeff Zhang

Re: Delete notebook in s3notebook repo is not working

[Jeff Zhang]

Not sure what's the root cause, do you mind to help fix it ? I am not sure
whether others in the community familiar with s3 and has environment to
test it.

Great Info  于2021年7月26日周一 上午12:06写道：

> I have deployed zeppelin 0.9.0 on AWS e2 and configured the s3 Notebook
> store. I have IAM role created, ec2 uses that role and defined right bucket
> permission for that role.
> Create Notebook, modify notebook works but delete is not working, getting
> 403 error.
>
> I have the right policy defined, below is my policy JSON(Camel case here
> last due to some content move). How to know which action is used during
> Delete notebooks
>
>
>
> {
> "version": "2012-10-17",
> "id": "bucket_policy",
> "statement": [{
> "sid": "denyreadaccess",
> "effect": "deny",
> "principal": "*",
> "action": ["s3:getobject", "s3:getobjectversion"],
> "resource": "arn:aws:s3:::zeppelin-tes/*",
> "condition": {
> "arnnotlike": {
> "aws:principalarn": "arn:aws:iam::985767567532:role/app/zeppelin-tes"
> }
> }
> }, {
> "sid": "denywriteaccess",
> "effect": "deny",
> "principal": "*",
> "action": ["s3:putobject", "s3:putobjectacl"],
> "resource": "arn:aws:s3:::zeppelin-tes/*",
> "condition": {
> "arnnotlike": {
> "aws:principalarn": "arn:aws:iam::985767567532:role/app/zeppelin-tes"
> }
> }
> }, {
> "sid": "denydeleteaccess",
> "effect": "deny",
> "principal": "*",
> "action": ["s3:deleteobject", "s3:deleteobjectversion",
> "s3:abortmultipartupload"],
> "resource": "arn:aws:s3:::zeppelin-tes/*",
> "condition": {
> "arnnotlike": {
> "aws:principalarn": "arn:aws:iam::985767567532:role/app/zeppelin-tes"
> }
> }
> }, {
> "sid": "denyreplicateaccessallexceptmasterroles",
> "effect": "deny",
> "principal": "*",
> "action": ["s3:replicateobject", "s3:replicatetags", "s3:replicatedelete"],
> "resource": "arn:aws:s3:::zeppelin-tes/*",
> "condition": {
> "arnlike": {
> "aws:principalarn": "arn:aws:iam::985767567532:role/app/zeppelin-tes"
> }
> }
> }, {
> "sid": "denyaccessexceptformasterroles",
> "effect": "deny",
> "principal": "*",
> "action": ["s3:deletebucket", "s3:deletebucketpolicy",
> "s3:deletebucketwebsite", "s3:putbucketacl", "s3:putbucketcors",
> "s3:putbucketpolicy", "s3:putbucketlogging", "s3:putbucketnotification",
> "s3:putbucketobjectlockconfiguration", "s3:putbucketpublicaccessblock",
> "s3:putbucketrequestpayment", "s3:putbucketwebsite", "s3:restoreobject*"],
> "resource": ["arn:aws:s3:::zeppelin-tes", "arn:aws:s3:::zeppelin-tes/*"],
> "condition": {
> "arnlike": {
> "aws:principalarn": "arn:aws:iam::985767567532:role/app/zeppelin-tes"
> }
> }
> }, {
> "sid": "denynonsecuretraffic",
> "effect": "deny",
> "principal": "*",
> "action": "s3:*",
> "resource": ["arn:aws:s3:::zeppelin-tes", "arn:aws:s3:::zeppelin-tes/*"],
> "condition": {
> "bool": {
> "aws:securetransport": "false"
> }
> }
> }]
> }
>


-- 
Best Regards

Jeff Zhang

Re: [ZEPPELIN-5434] Upgrade jetty to 9.4.42.v20210604

[Jeff Zhang]

Do you want to help with this ticket ?

denny wong  于2021年7月26日周一 下午10:00写道：

> Hi
> Any progress on this JIRA?  Is this being actively worked on?
> https://github.com/apache/zeppelin/pull/4160
>
> Thanks
> Denny



-- 
Best Regards

Jeff Zhang

Re: z.show for R

[Jeff Zhang]

I don't this z.show is supported in R interpreter, but you can use R
package to visualize R data.

denny wong  于2021年7月29日周四 上午11:03写道：

> Hi
> In R, we want to leverage the Zeppelin's built in visualization, but not
> sure how it will work.  In python, we use z.show to display a data frame
> and leverage the built in visualization.  Is there an equivalent for R?
> Moreover, the table display system (%table) doesn't seem to work in R.
> Thanks
> Denny



-- 
Best Regards

Jeff Zhang

Re: z.show for R

[Jeff Zhang]

Hi Denny,

Could you create a ticket for it ?

denny wong  于2021年7月29日周四 下午8:08写道：

>  Hi Jeff
> It will be consistent to enable z.show for both python and R languages.
> The zeppelin doc mentions the display system is backend independent, but it
> doesn't work for R.  Is it easy to enable it for R?
>
> https://zeppelin.apache.org/docs/0.9.0/usage/display_system/basic.html#table
>
> Thanks
> Denny
>
> On Wednesday, July 28, 2021, 11:32:33 PM EDT, Jeff Zhang <
> zjf...@gmail.com> wrote:
>
>  I don't this z.show is supported in R interpreter, but you can use R
> package to visualize R data.
>
> denny wong 于2021年7月29日周四 上午11:03写道：
>
> > Hi
> > In R, we want to leverage the Zeppelin's built in visualization, but not
> > sure how it will work.  In python, we use z.show to display a data frame
> > and leverage the built in visualization.  Is there an equivalent for R?
> > Moreover, the table display system (%table) doesn't seem to work in R.
> > Thanks
> > Denny
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
Best Regards

Jeff Zhang

Re: z.show for R

[Jeff Zhang]

Thanks Denny, BTW could you share a little bit about how you use R
interpreter in Zeppelin ? Not many people use R in zeppelin, so your
feedback would be very helpful for the community.



denny wong  于2021年7月29日周四 下午9:24写道：

>  Here is the ticket.  It would be desirable to have this addressed soon.
> Thanks.
> [ZEPPELIN-5473] Enable z.show for R - ASF JIRA
>
>
> |
> |
> |  |
> [ZEPPELIN-5473] Enable z.show for R - ASF JIRA
>
>
>  |
>
>  |
>
>  |
>
>
>
>
> On Thursday, July 29, 2021, 08:37:56 AM EDT, Jeff Zhang <
> zjf...@gmail.com> wrote:
>
>  Hi Denny,
>
> Could you create a ticket for it ?
>
> denny wong  于2021年7月29日周四 下午8:08写道：
>
> >  Hi Jeff
> > It will be consistent to enable z.show for both python and R languages.
> > The zeppelin doc mentions the display system is backend independent, but
> it
> > doesn't work for R.  Is it easy to enable it for R?
> >
> >
> https://zeppelin.apache.org/docs/0.9.0/usage/display_system/basic.html#table
> >
> > Thanks
> > Denny
> >
> >On Wednesday, July 28, 2021, 11:32:33 PM EDT, Jeff Zhang <
> > zjf...@gmail.com> wrote:
> >
> >  I don't this z.show is supported in R interpreter, but you can use R
> > package to visualize R data.
> >
> > denny wong 于2021年7月29日周四 上午11:03写道：
> >
> > > Hi
> > > In R, we want to leverage the Zeppelin's built in visualization, but
> not
> > > sure how it will work.  In python, we use z.show to display a data
> frame
> > > and leverage the built in visualization.  Is there an equivalent for R?
> > > Moreover, the table display system (%table) doesn't seem to work in R.
> > > Thanks
> > > Denny
> >
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
Best Regards

Jeff Zhang

Re: How to pass data between paragraph in sql(jdbc)

[Jeff Zhang]

I am afraid it is not possible, unless you leverage other engines like
spark/flink you mentioned

Great Info  于2021年8月12日周四 上午1:29写道：

> I have two databases and I have configured two interpreters(jdbc1,jdbc2)
> belongs to JDBC group.
>
> jdbc1 has some basic reference data, I need to pull it  in paragraph 1 and
> pass few columns of data to the next paragraph
> %jdbc1
>   // pull the ref data and pass it to next
> %jdbc2
> // use the ref data and enrich some fields
>
> is this possible without using some backend processing framework like spark
> or flink..?
>


-- 
Best Regards

Jeff Zhang

[VOTE] Release Apache Zeppelin 0.10.0 (RC1)

[Jeff Zhang]

Hi folks,

I propose the following RC to be released for the Apache Zeppelin
0.10.0 release.


The commit id is 14cc810812ef2bd29a2c41ba37caa20f1aa3f3bc :
https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=14cc810812ef2bd29a2c41ba37caa20f1aa3f3bc

This corresponds to the tag: v0.10.0-rc1 :
https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.10.0-rc1

The release archives (tgz), signature, and checksums are here
https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.10.0-rc1/

The release candidate consists of the following source distribution archive
zeppelin-0.10.0.tgz

In addition, the following supplementary binary distributions are provided
for user convenience at the same location
zeppelin-0.10.0-bin-all.tgz


The maven artifacts are here
https://repository.apache.org/content/repositories/orgapachezeppelin-1291/org/apache/zeppelin/

You can find the KEYS file here:
https://dist.apache.org/repos/dist/release/zeppelin/KEYS

Release notes available at
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12349536&styleName=&projectId=12316221


Vote will be open for next 72 hours (close at 10PM 14/Aug PDT).

[ ] +1 approve
[ ] 0 no opinion

[ ] -1 disapprove (and reason why)


-- 
Best Regards

Jeff Zhang

[VOTE] Release Apache Zeppelin 0.10.0 RC2

[Jeff Zhang]

Hi folks,

I propose the following RC to be released for the Apache Zeppelin
0.10.0 release.


The commit id is 8f85476c40df3536718c5aa2c07b519b6bb6607e :
https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=8f85476c40df3536718c5aa2c07b519b6bb6607e


This corresponds to the tag: v0.10.0-rc2 :
https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.10.0-rc2

The release archives (tgz), signature, and checksums are here
https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.10.0-rc2/

The release candidate consists of the following source distribution archive
zeppelin-0.10.0.tgz

In addition, the following supplementary binary distributions are provided
for user convenience at the same location
zeppelin-0.10.0-bin-all.tgz


The maven artifacts are here
https://repository.apache.org/content/repositories/orgapachezeppelin-1302/org/apache/zeppelin/

You can find the KEYS file here:
https://dist.apache.org/repos/dist/release/zeppelin/KEYS

Release notes available at
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12349536&styleName=&projectId=12316221


Vote will be open for next 72 hours (close at 10PM 20/Aug PDT).

[ ] +1 approve
[ ] 0 no opinion

[ ] -1 disapprove (and reason why)



-- 
Best Regards

Jeff Zhang

Re: [VOTE] Release Apache Zeppelin 0.10.0 RC2

[Jeff Zhang]

+1，  Verified most of the tutorial notes

Jongyoul Lee  于2021年8月19日周四 下午12:20写道：

> +1
>
> I've checked the basic example about this RC
>
> 2021년 8월 18일 (수) 오후 3:28, 님이 작성:
>
>> +1
>>
>>
>>
>> *From:* Philipp Dallig 
>> *Sent:* Wednesday, August 18, 2021 3:24 PM
>> *To:* us...@zeppelin.apache.org
>> *Subject:* Re: [VOTE] Release Apache Zeppelin 0.10.0 RC2
>>
>>
>>
>> +1
>>
>> Am 18.08.21 um 08:01 schrieb Francois Reynald:
>>
>> +1
>>
>>
>>
>> On 18 Aug 2021, at 05:52, Jeff Zhang  wrote:
>>
>>
>>
>>
>> Hi folks,
>>
>>
>>
>> I propose the following RC to be released for the Apache Zeppelin 0.10.0 
>> release.
>>
>>
>>
>>
>>
>> The commit id is 8f85476c40df3536718c5aa2c07b519b6bb6607e :
>>
>> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=8f85476c40df3536718c5aa2c07b519b6bb6607e
>>
>>
>>
>> This corresponds to the tag: v0.10.0-rc2 :
>>
>> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.10.0-rc2
>>
>> The release archives (tgz), signature, and checksums are here
>>
>> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.10.0-rc2/
>>
>>
>>
>> The release candidate consists of the following source distribution archive
>>
>> zeppelin-0.10.0.tgz
>>
>>
>>
>> In addition, the following supplementary binary distributions are provided
>>
>> for user convenience at the same location
>>
>> zeppelin-0.10.0-bin-all.tgz
>>
>>
>>
>>
>>
>> The maven artifacts are here
>>
>> https://repository.apache.org/content/repositories/orgapachezeppelin-1302/org/apache/zeppelin/
>>
>>
>>
>> You can find the KEYS file here:
>>
>> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
>>
>>
>>
>> Release notes available at 
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12349536&styleName=&projectId=12316221
>>
>>
>>
>> Vote will be open for next 72 hours (close at 10PM 20/Aug PDT).
>>
>>
>>
>> [ ] +1 approve
>>
>> [ ] 0 no opinion
>>
>> [ ] -1 disapprove (and reason why)
>>
>>
>>
>>
>>
>> --
>>
>> Best Regards
>>
>> Jeff Zhang
>>
>>
>>
>> *François Reynald | Educational Services*
>> t. +33 (0) 624 200 125
>> cloudera.com <http://www.cloudera.com>
>>
>>
>>
>>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>


-- 
Best Regards

Jeff Zhang

[RESULT] [VOTE] Release Apache Zeppelin 0.10.0 RC2

[Jeff Zhang]

The vote passes with 4 binding +1 votes, 3 non-binding +1 votes, and no +0
or -1 votes. Thanks for everyone who verified rc and voted.


+1


Jeff Zhang *

Jongyoul Lee   *moon soo Lee *

Prabhjyot Singh   *
Francois Reynaldcloverheartsdev
Philipp Dallig


Prabhjyot Singh  于2021年8月21日周六 下午8:05写道：

> +1
>
> On Sat, Aug 21, 2021, 9:12 AM moon soo Lee  wrote:
>
> > +1
> >
> > Thank Jeff for preparing the RC.
> >
> >
> > On Thu, Aug 19, 2021 at 7:24 PM Jeff Zhang  wrote:
> >
> >> +1，  Verified most of the tutorial notes
> >>
> >> Jongyoul Lee  于2021年8月19日周四 下午12:20写道：
> >>
> >>> +1
> >>>
> >>> I've checked the basic example about this RC
> >>>
> >>> 2021년 8월 18일 (수) 오후 3:28, 님이 작성:
> >>>
> >>>> +1
> >>>>
> >>>>
> >>>>
> >>>> *From:* Philipp Dallig 
> >>>> *Sent:* Wednesday, August 18, 2021 3:24 PM
> >>>> *To:* us...@zeppelin.apache.org
> >>>> *Subject:* Re: [VOTE] Release Apache Zeppelin 0.10.0 RC2
> >>>>
> >>>>
> >>>>
> >>>> +1
> >>>>
> >>>> Am 18.08.21 um 08:01 schrieb Francois Reynald:
> >>>>
> >>>> +1
> >>>>
> >>>>
> >>>>
> >>>> On 18 Aug 2021, at 05:52, Jeff Zhang  wrote:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Hi folks,
> >>>>
> >>>>
> >>>>
> >>>> I propose the following RC to be released for the Apache Zeppelin
> 0.10.0 release.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> The commit id is 8f85476c40df3536718c5aa2c07b519b6bb6607e :
> >>>>
> >>>>
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=8f85476c40df3536718c5aa2c07b519b6bb6607e
> >>>>
> >>>>
> >>>>
> >>>> This corresponds to the tag: v0.10.0-rc2 :
> >>>>
> >>>>
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.10.0-rc2
> >>>>
> >>>> The release archives (tgz), signature, and checksums are here
> >>>>
> >>>> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.10.0-rc2/
> >>>>
> >>>>
> >>>>
> >>>> The release candidate consists of the following source distribution
> archive
> >>>>
> >>>> zeppelin-0.10.0.tgz
> >>>>
> >>>>
> >>>>
> >>>> In addition, the following supplementary binary distributions are
> provided
> >>>>
> >>>> for user convenience at the same location
> >>>>
> >>>> zeppelin-0.10.0-bin-all.tgz
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> The maven artifacts are here
> >>>>
> >>>>
> https://repository.apache.org/content/repositories/orgapachezeppelin-1302/org/apache/zeppelin/
> >>>>
> >>>>
> >>>>
> >>>> You can find the KEYS file here:
> >>>>
> >>>> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> >>>>
> >>>>
> >>>>
> >>>> Release notes available at
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12349536&styleName=&projectId=12316221
> >>>>
> >>>>
> >>>>
> >>>> Vote will be open for next 72 hours (close at 10PM 20/Aug PDT).
> >>>>
> >>>>
> >>>>
> >>>> [ ] +1 approve
> >>>>
> >>>> [ ] 0 no opinion
> >>>>
> >>>> [ ] -1 disapprove (and reason why)
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>>
> >>>> Best Regards
> >>>>
> >>>> Jeff Zhang
> >>>>
> >>>>
> >>>>
> >>>> *François Reynald | Educational Services*
> >>>> t. +33 (0) 624 200 125
> >>>> cloudera.com <http://www.cloudera.com>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>> 이종열, Jongyoul Lee, 李宗烈
> >>> http://madeng.net
> >>>
> >>
> >>
> >> --
> >> Best Regards
> >>
> >> Jeff Zhang
> >>
> >
>


-- 
Best Regards

Jeff Zhang

[ANNOUNCE] Apache Zeppelin 0.10.0 released

[Jeff Zhang]

The Apache Zeppelin community is pleased to announce the availability
of the 0.10.0 release.
Zeppelin is a collaborative data analytics and visualization tool for
distributed, general-purpose data processing system such as Apache
Spark, Apache Flink, Apache Cassandra and etc.

This is another major release after the last major release 0.9.0 half
year ago. The community put significant effort into improving Apache
Zeppelin since the last release.

We encourage you to download the latest release
fromhttp://zeppelin.apache.org/download.html

The easiest way to try Zeppelin is to run the shipped tutorial notes
via docker image

https://zeppelin.apache.org/download.html#using-the-official-docker-image

We welcome your help and feedback. For more information on the project
and how to get involved, visit our website at
http://zeppelin.apache.org/

Thank you all users and contributors who have helped to improve Apache
Zeppelin. Welcome to join our community to discuss with others
https://zeppelin.apache.org/community.html#mailing-list--slack-channel

Here's some highlights of this major release

https://zeppelin.apache.org/#whats-new-in


-- 
Best Regards

Jeff Zhang

[DISCUSS] Release of Zeppelin 0.10.1

[Jeff Zhang]

Hi Folks,

Since the release of Zeppelin 0.10.0, we have several critical fixes, I am
thinking of the release of Zeppelin 0.10.1, the only left issue in my mind
is the support fo Spark 3.2.0（
https://issues.apache.org/jira/browse/ZEPPELIN-5565）, let me know what your
thoughts are.

Here's the list of tickets we have fixed in 0.10.1

https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZEPPELIN%20AND%20fixVersion%20%3D%200.10.1


-- 
Best Regards

Jeff Zhang

Re: ipython interpreter prerequisites

[Jeff Zhang]

I don't think so, at least we need jupyter-client module as we use
jupyter-client api to connect with ipython kernel.

denny wong  于2021年10月28日周四 上午11:24写道：

> Hi
> Question about the ipython interpreter prerequisites.  The jupyter module
> version ( https://pypi.org/project/jupyter/) is pretty old.  Can it be
> replaced by ipykernel module (https://pypi.org/project/ipykernel/)?
>
> Prerequisites
> - Jupyter `pip install jupyter`
> - grpcio `pip install grpcio`
> - protobuf `pip install protobuf`
>
> Will this work instead?- ipykernel `pip install ipykernel`
> - grpcio `pip install grpcio`
> - protobuf `pip install protobuf`
> Thanks
> Denny



-- 
Best Regards

Jeff Zhang