Re: Run/install tensorframes on zeppelin pyspark

2018-08-08 Thread Jeff Zhang 
Make sure you use the correct python which has tensorframe installed.
Use PYSPARK_PYTHON
to configure the python



Spico Florin 于2018年8月8日周三 下午9:59写道：

> Hi!
>
> I would like to use tensorframes in my pyspark notebook.
>
> I have performed the following:
>
> 1. In the spark intepreter adde a new repository
> http://dl.bintray.com/spark-packages/maven
> 2. in the spark interpreter added the
> dependency databricks:tensorframes:0.2.9-s_2.11
> 3. pip install tensorframes
>
>
> In both 0.7.3 and 0.8.0:
> 1.  the following code resulted in error: "ImportError: No module named
> tensorframes"
>
> %pyspark
> import tensorframes as tfs
>
> 2. the following code succeeded
> %spark
> import org.tensorframes.{dsl => tf}
> import org.tensorframes.dsl.Implicits._
> val df = spark.createDataFrame(Seq(1.0->1.1, 2.0->2.2)).toDF("a", "b")
>
> // As in Python, scoping is recommended to prevent name collisions.
> val df2 = tf.withGraph {
> val a = df.block("a")
> // Unlike python, the scala syntax is more flexible:
> val out = a + 3.0 named "out"
> // The 'mapBlocks' method is added using implicits to dataframes.
> df.mapBlocks(out).select("a", "out")
> }
>
> // The transform is all lazy at this point, let's execute it with collect:
> df2.collect()
>
> I ran the code above directly with spark interpreter with the default
> configurations (master set up to local[*] - so not via spark-submit
> command) .
>
> Also, I have installed spark home locally and ran the command
>
> $SPARK_HOME/bin/pyspark --packages databricks:tensorframes:0.2.9-s_2.11
>
> and the code below worked as expcted
>
> import tensorframes as tfs
>
>  Can you please help to solve this?
>
> Thanks,
>
>  Florin
>
>
>
>
>
>
>
>
>

Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released

2018-08-08 Thread Jeff Zhang 
Yes, I am planning that, here's ticket
https://issues.apache.org/jira/browse/ZEPPELIN-3629
feel free to link any bug target for 0.8.1


Ruslan Dautkhanov 于2018年8月9日周四 上午3:46写道：

> Thanks Jeff! Should there be a Zeppelin 0.8.1 release sometime soon with
> all the fixes for issues that the users have faced in 0.8.0?
>
> --
> Ruslan Dautkhanov
>
>
> On Mon, Jul 23, 2018 at 12:24 AM Jeff Zhang  wrote:
>
>>
>> Thanks Ruslan, I will fix it.
>>
>> Ruslan Dautkhanov 于2018年7月23日周一 下午1:46写道：
>>
>>> https://zeppelin.apache.org/ home page still reads
>>> "WHAT'S NEW IN
>>>  Apache Zeppelin 0.7"
>>>
>>>
>>> --
>>> Ruslan Dautkhanov
>>>
>>>
>>> On Fri, Jun 29, 2018 at 4:56 AM Spico Florin 
>>> wrote:
>>>
 Hi!
   I tried to get the docker image for this version 0.8.0, but it seems
 that is not in the official docker hub repository:
 https://hub.docker.com/r/apache/zeppelin/tags/ there is no such as
 version 0.8.0
 Also, the commands
  docker pull apache/zeppelin:0.8.0
 or

 docker run -p 8080:8080 --rm --name zeppelin apache/zeppelin:0.8.0


 fails with
 Error response from daemon: manifest for apache/zeppelin:0.8.0 not found

 Can you please check? Or how should I get this version for docker
 (please instruct).

 Thanks.
 Regards,
  Florin



 On Fri, Jun 29, 2018 at 6:13 AM, Jongyoul Lee 
 wrote:

> Great work!!
>
> On Fri, Jun 29, 2018 at 9:49 AM, Jeff Zhang  wrote:
>
>> Thanks Patrick, I have fixed the broken link.
>>
>>
>>
>> Patrick Maroney 于2018年6月29日周五 上午7:13写道：
>>
>> > Install guides:
>> >
>> > http://zeppelin.apache.org/docs/0.8.0/install/install.html
>> > Not Found
>> >
>> > The requested URL /docs/0.8.0/install/install.html was not found
>> on this
>> > server.
>> >
>> >
>> http://zeppelin.apache.org/docs/0.8.0/manual/interpreterinstallation.html
>> >
>> > Not Found
>> >
>> > The requested URL /docs/0.8.0/manual/interpreterinstallation.html
>> was not
>> > found on this server.
>> >
>> >
>> > Patrick Maroney
>> > Principal Engineer - Data Science & Analytics
>> > Wapack Labs
>> >
>> >
>> > On Jun 28, 2018, at 6:59 PM, Jianfeng (Jeff) Zhang <
>> jzh...@hortonworks.com>
>> > wrote:
>> >
>> > Hi Patrick,
>> >
>> > Which link is broken ? I can access all the links.
>> >
>> > Best Regard,
>> > Jeff Zhang
>> >
>> >
>> > From: Patrick Maroney 
>> > Reply-To: 
>> > Date: Friday, June 29, 2018 at 4:59 AM
>> > To: 
>> > Cc: dev 
>> > Subject: Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released
>> >
>> > Great work Team/Community!
>> >
>> > Links on the main download page are broken:
>> >
>> > http://zeppelin.apache.org/download.html
>> >
>> > ...at least the ones I need ;-)
>> >
>> > *Patrick Maroney*
>>
>> > Principal Engineer - Data Science & Analytics
>> > Wapack Labs LLC
>> >
>> >
>> > Public Key:
>> http://pgp.mit.edu/pks/lookup?op=get&search=0x7C810C9769BD29AF
>> >
>> > On Jun 27, 2018, at 11:21 PM, Prabhjyot Singh <
>> prabhjyotsi...@gmail.com>
>> > wrote:
>> >
>> > Awesome! congratulations team.
>> >
>> >
>> >
>> > On Thu 28 Jun, 2018, 8:39 AM Taejun Kim,  wrote:
>> >
>> >> Awesome! Thanks for your great work :)
>> >>
>> >> 2018년 6월 28일 (목) 오후 12:07, Jeff Zhang 님이 작성:
>> >>
>> >>> The Apache Zeppelin community is pleased to announce the
>> availability of
>> >>> the 0.8.0 release.
>> >>>
>> >>> Zeppelin is a collaborative data analytics and visualization tool
>> for
>> >>> distributed, general-purpose data processing system such as
>> Apache Spark,
>> >>> Apache Flink, etc.
>> >>>
>> >>> This is another major release after the last minor release 0.7.3.
>> >>> The community put significant effort into improving Apache
>> Zeppelin since
>> >>> the last release. 122 contributors fixed totally 602 issues. Lots
>> of
>> >>> new features are introduced, such as inline configuration, ipython
>> >>> interpreter, yarn-cluster mode support , interpreter lifecycle
>> manager
>> >>> and etc.
>> >>>
>> >>> We encourage you to download the latest release
>> >>> fromhttp://zeppelin.apache.org/download.html
>> >>>
>> >>> Release note is available
>> >>> athttp://zeppelin.apache.org/releases/zeppelin-release-0.8.0.html
>> >>>
>> >>> We welcome your help and feedback. For more information on the
>> project
>> >>> and
>> >>> how to get involved, visit our website at
>> http://zeppelin.apache.org/
>> >>>
>> >>> Thank you all users and contributors who have helped to improve
>> Apache
>> >>> Zeppelin.
>> >>>
>> >>> Regards,
>> >>> The Apache Zeppelin co

Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released

2018-08-08 Thread Ruslan Dautkhanov 
Thanks Jeff! Should there be a Zeppelin 0.8.1 release sometime soon with
all the fixes for issues that the users have faced in 0.8.0?

-- 
Ruslan Dautkhanov


On Mon, Jul 23, 2018 at 12:24 AM Jeff Zhang  wrote:

>
> Thanks Ruslan, I will fix it.
>
> Ruslan Dautkhanov 于2018年7月23日周一 下午1:46写道：
>
>> https://zeppelin.apache.org/ home page still reads
>> "WHAT'S NEW IN
>>  Apache Zeppelin 0.7"
>>
>>
>> --
>> Ruslan Dautkhanov
>>
>>
>> On Fri, Jun 29, 2018 at 4:56 AM Spico Florin 
>> wrote:
>>
>>> Hi!
>>>   I tried to get the docker image for this version 0.8.0, but it seems
>>> that is not in the official docker hub repository:
>>> https://hub.docker.com/r/apache/zeppelin/tags/ there is no such as
>>> version 0.8.0
>>> Also, the commands
>>>  docker pull apache/zeppelin:0.8.0
>>> or
>>>
>>> docker run -p 8080:8080 --rm --name zeppelin apache/zeppelin:0.8.0
>>>
>>>
>>> fails with
>>> Error response from daemon: manifest for apache/zeppelin:0.8.0 not found
>>>
>>> Can you please check? Or how should I get this version for docker
>>> (please instruct).
>>>
>>> Thanks.
>>> Regards,
>>>  Florin
>>>
>>>
>>>
>>> On Fri, Jun 29, 2018 at 6:13 AM, Jongyoul Lee 
>>> wrote:
>>>
 Great work!!

 On Fri, Jun 29, 2018 at 9:49 AM, Jeff Zhang  wrote:

> Thanks Patrick, I have fixed the broken link.
>
>
>
> Patrick Maroney 于2018年6月29日周五 上午7:13写道：
>
> > Install guides:
> >
> > http://zeppelin.apache.org/docs/0.8.0/install/install.html
> > Not Found
> >
> > The requested URL /docs/0.8.0/install/install.html was not found on
> this
> > server.
> >
> >
> http://zeppelin.apache.org/docs/0.8.0/manual/interpreterinstallation.html
> >
> > Not Found
> >
> > The requested URL /docs/0.8.0/manual/interpreterinstallation.html
> was not
> > found on this server.
> >
> >
> > Patrick Maroney
> > Principal Engineer - Data Science & Analytics
> > Wapack Labs
> >
> >
> > On Jun 28, 2018, at 6:59 PM, Jianfeng (Jeff) Zhang <
> jzh...@hortonworks.com>
> > wrote:
> >
> > Hi Patrick,
> >
> > Which link is broken ? I can access all the links.
> >
> > Best Regard,
> > Jeff Zhang
> >
> >
> > From: Patrick Maroney 
> > Reply-To: 
> > Date: Friday, June 29, 2018 at 4:59 AM
> > To: 
> > Cc: dev 
> > Subject: Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released
> >
> > Great work Team/Community!
> >
> > Links on the main download page are broken:
> >
> > http://zeppelin.apache.org/download.html
> >
> > ...at least the ones I need ;-)
> >
> > *Patrick Maroney*
>
> > Principal Engineer - Data Science & Analytics
> > Wapack Labs LLC
> >
> >
> > Public Key:
> http://pgp.mit.edu/pks/lookup?op=get&search=0x7C810C9769BD29AF
> >
> > On Jun 27, 2018, at 11:21 PM, Prabhjyot Singh <
> prabhjyotsi...@gmail.com>
> > wrote:
> >
> > Awesome! congratulations team.
> >
> >
> >
> > On Thu 28 Jun, 2018, 8:39 AM Taejun Kim,  wrote:
> >
> >> Awesome! Thanks for your great work :)
> >>
> >> 2018년 6월 28일 (목) 오후 12:07, Jeff Zhang 님이 작성:
> >>
> >>> The Apache Zeppelin community is pleased to announce the
> availability of
> >>> the 0.8.0 release.
> >>>
> >>> Zeppelin is a collaborative data analytics and visualization tool
> for
> >>> distributed, general-purpose data processing system such as Apache
> Spark,
> >>> Apache Flink, etc.
> >>>
> >>> This is another major release after the last minor release 0.7.3.
> >>> The community put significant effort into improving Apache
> Zeppelin since
> >>> the last release. 122 contributors fixed totally 602 issues. Lots
> of
> >>> new features are introduced, such as inline configuration, ipython
> >>> interpreter, yarn-cluster mode support , interpreter lifecycle
> manager
> >>> and etc.
> >>>
> >>> We encourage you to download the latest release
> >>> fromhttp://zeppelin.apache.org/download.html
> >>>
> >>> Release note is available
> >>> athttp://zeppelin.apache.org/releases/zeppelin-release-0.8.0.html
> >>>
> >>> We welcome your help and feedback. For more information on the
> project
> >>> and
> >>> how to get involved, visit our website at
> http://zeppelin.apache.org/
> >>>
> >>> Thank you all users and contributors who have helped to improve
> Apache
> >>> Zeppelin.
> >>>
> >>> Regards,
> >>> The Apache Zeppelin community
> >>>
> >> --
> >> Taejun Kim
> >>
> >> Data Mining Lab.
> >> School of Electrical and Computer Engineering
> >> University of Seoul
> >>
> >
> >
>



 --
 이종열, Jongyoul Lee, 李宗烈
 http://madeng.net

>>>
>>>

Re: Sometimes tab deletes text

2018-08-08 Thread Ruslan Dautkhanov 
It was broken end of last year by this PR
https://github.com/apache/zeppelin/pull/2624

I already filed a jira for this
issue -
https://issues.apache.org/jira/browse/ZEPPELIN-3253

Zeppelin I built for our users has that change rolled back, see for example,
https://github.com/apache/zeppelin/pull/2812/files


--
Ruslan Dautkhanov


On Wed, Aug 8, 2018 at 10:01 AM Paul Brenner  wrote:

> ok, I went ahead and opened
> https://issues.apache.org/jira/browse/ZEPPELIN-3692
> 
>
> If I find a better way to reproduce, more details, or if anyone else
> chimes in, I'll add to the ticket. Sorry I'm not better able to contribute
> to fixing this.
>
>
> 
> 
> 
>  *Paul
> Brenner*
> 
> 
> 
> 
> 
> 
> 
> SR. DATA SCIENTIST
> *(217) 390-3033 *
>
> 
> 
> 
> 
>

Re: Sometimes tab deletes text

2018-08-08 Thread Paul Brenner 
ok, I went ahead and opened https://issues.apache.org/jira/browse/ZEPPELIN-3692 
( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErQEYA32hiB6svhZ6dNQnrKw9xG29x7IMjDaZ2VqzxHv9JNbHsi7yvZOUwB7I-b-7bEVRwLKBU_pob5z9ksWBo
 )

If I find a better way to reproduce, more details, or if anyone else chimes in, 
I'll add to the ticket. Sorry I'm not better able to contribute to fixing this.

( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErCVMAh3ZmWq4ml9OZLBmmIAUyW-c7fv5AmWhFLboiic7SSdI=
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErCVMAh3ZmWq4ml9OZLBmmIAUyW-c7fv5AmWhFLboiic7SSdI=
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErCVMAh3ZmWq4ml9OZLBmmIAUyW-c7fv5AmWhFLboiic7SSdI=
 ) *Paul Brenner* ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErQEYA33VmHao-k8LSJgflbBozFX5zqphNfrLucIjkqxaQ9XIJeaXa
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErQEYA33VmHao-k8LSJgflbBozFX5zqphNfrLucIjkqxaQ9XIJeaXa
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErQEYA33VmHao-k8LSJgflbBozFX5zqphNfrLucIjkqxaQ9XIJeaXa
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErQEYA33ZmA_Asl9OZJwfnKEQ8G3A5k4UrHKIQZalvdOOkilFQ7eNtx7whB80=
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErQEYA33ZmA_Asl9OZJwfnKEQ8G3A5k4UrHKIQZalvdOOkilFQ7eNtx7whB80=
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErQEYA33ZmA_Amn96XIAzhLUQ8G3A5oIYnD6Y3TfHuBe1Pf4LY4_VhaG02vRH4s_AIoB5RhA==
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErQEYA33ZmA_Amn96XIAzhLUQ8G3A5oIYnD6Y3TfHuBe1Pf4LY4_VhaG02vRH4s_AIoB5RhA==
 ) SR. DATA SCIENTIST (217) 390-3033 

( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErCVMAh3ZmWq4ml9OZLBmmIAUyWy8m8txlT_J2BuixGeBNeY7AqsGEScEbTqx_P7Jx2zzZQXvV52trzh6E8_iRykBOCxQ_zgJg4nz5t5Er_HGmBFEVx0a1AGahHQFULvOey_sp0YGqzd3SCkVpyXnaSDS6chcBwXHv
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErCVMAgG1wF7sjh56fKgWncVpuQTIn8cZ7R-g4V73rG-1PY8bfqMGaWskdQ_JhObIy1yDZQnKb9m1n0gTA8PCK3h4OVEUjjEpt-XP7qds49vc7kwYa2DGigzRfj0CU
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErCVMAgG1wF7sjh56fKgWncVpuQTIn8cZ7R-g4V73rG-1PY8bfqMGaWskdQ_JhObIy1yDZQnKb9m1n0gTA8PCK3h4OVEUjjEpt-XP7qds49vc7kwYa2DGigzRfj0CU
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErCVMAgG1wF7sjh56fKgWncVpuQTIn8cZ7R-g4V73rG-1PY8bfqMGaWskdQ_JhObIy1yDZQnKb9m1n0gTA8PCK3h4OVEUjjEpt-XP7qds49vc7kwYa2DGigzRfj0CU
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErCVMAgG1wF7sjh56fKgWncVpuQTIn8cZ7R-g4V73rG-1PY8bfqMGaWskdQ_JhObIy1yDZQnKb9m1n0gTA8PCK3h4OVEUjjEpt-XP7qds49vc7kwYa2DGigzRfj0CU
 ) ( 
https://share.polymail.io/v1/z/b/NWI2YjEzMWZlNjk0/KlQRBTEsQl6TBBvqq5P1ulqS2HpT7XuroULPqFkdrAJYNoqQhV95Y1HU1BfNTrXI1U3RwzkEO1Rjgxmq3iHJqsLp9-Y6ucg1HoNFuSH7xVkdZNUe5jite4eEd-mj4nsCjYFlHgK3evErCVMAgG1wF7sjh56fKgWncVpuQjIm8MZ6R-g0Ub_tHP5FdIyEuoiOWsk

Run/install tensorframes on zeppelin pyspark

2018-08-08 Thread Spico Florin 
Hi!

I would like to use tensorframes in my pyspark notebook.

I have performed the following:

1. In the spark intepreter adde a new repository
http://dl.bintray.com/spark-packages/maven
2. in the spark interpreter added the
dependency databricks:tensorframes:0.2.9-s_2.11
3. pip install tensorframes


In both 0.7.3 and 0.8.0:
1.  the following code resulted in error: "ImportError: No module named
tensorframes"

%pyspark
import tensorframes as tfs

2. the following code succeeded
%spark
import org.tensorframes.{dsl => tf}
import org.tensorframes.dsl.Implicits._
val df = spark.createDataFrame(Seq(1.0->1.1, 2.0->2.2)).toDF("a", "b")

// As in Python, scoping is recommended to prevent name collisions.
val df2 = tf.withGraph {
val a = df.block("a")
// Unlike python, the scala syntax is more flexible:
val out = a + 3.0 named "out"
// The 'mapBlocks' method is added using implicits to dataframes.
df.mapBlocks(out).select("a", "out")
}

// The transform is all lazy at this point, let's execute it with collect:
df2.collect()

I ran the code above directly with spark interpreter with the default
configurations (master set up to local[*] - so not via spark-submit
command) .

Also, I have installed spark home locally and ran the command

$SPARK_HOME/bin/pyspark --packages databricks:tensorframes:0.2.9-s_2.11

and the code below worked as expcted

import tensorframes as tfs

 Can you please help to solve this?

Thanks,

 Florin