Re: Change interpreter execution root path

2018-10-20 Thread Jeff Zhang 
You can set zeppelin.interpreter.dir in zeppelin-site.xml

Jhon Anderson Cardenas Diaz 于2018年10月20日周六
下午10:22写道：

> Hi
>
> Do you know how can I change the folder path where the interpreters are
> executed?.
>
> The reason why I want to change that default location (which is
> $ZEPPELIN_HOME) is because we are getting very large core dumps files in
> that location when the interpreter process die.
>
> As we are in a k8s ecosystem, If we change the run time location to a
> subfolder we could mount a volume in that location to avoid zeppelin die
> because of full disk space.
>
> We already tried to change the core dumps file location with 
> /proc/sys/kernel/core_pattern
> but this does not work because is running in a docker container inside a
> k8s ecosystem.
>
> Thanks!
>

Re: Zeppelin add hadoop submarine(machine learning framework) interpreter

2018-10-20 Thread Felix Cheung 
Very cool!



From: Jeff Zhang 
Sent: Friday, October 19, 2018 7:14 AM
To: dev@zeppelin.apache.org
Subject: Re: Zeppelin add hadoop submarine(machine learning framework) 
interpreter

Thanks xun. This would be a great addon for zeppelin to support deep
learning. I will check the design later.



liu xun 于2018年10月19日周五 下午3:56写道：

> Hi,
> Hadoop Submarine is the latest machine learning framework subproject in
> the Hadoop 3.2 release. It allows Hadoop to support Tensorflow,
> MXNet,Caffe, Spark, etc. A variety of deep learning frameworks provide a
> full-featured system framework for machine learning algorithm development,
> distributed model training, model management, and model publishing,
> combined with hadoop's intrinsic data storage and data processing
> capabilities to enable data scientists to Good mining and the value of the
> data.
>
>
> I was involved in the development of the hadoop submarine project. So I
> plan to add the interpreter module of hadoop submarine to zeppelin.
> Let zeppeline increase the development of deep learning. This is my design
> document, let's see if there is any opinion, you can put it directly in the
> document, thank you!
>
>
> https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit?ts=5bc6bfdd
>
>
>

Change interpreter execution root path

2018-10-20 Thread Jhon Anderson Cardenas Diaz 
Hi

Do you know how can I change the folder path where the interpreters are
executed?.

The reason why I want to change that default location (which is
$ZEPPELIN_HOME) is because we are getting very large core dumps files in
that location when the interpreter process die.

As we are in a k8s ecosystem, If we change the run time location to a
subfolder we could mount a volume in that location to avoid zeppelin die
because of full disk space.

We already tried to change the core dumps file location with
/proc/sys/kernel/core_pattern
but this does not work because is running in a docker container inside a
k8s ecosystem.

Thanks!

[GitHub] zeppelin issue #3206: [WIP][ZEPPELIN-3810] Support Spark 2.4

2018-10-20 Thread HyukjinKwon 
Github user HyukjinKwon commented on the issue:

https://github.com/apache/zeppelin/pull/3206
  
Nope, it will work for both 2.11.8 and 2.11.12. I manually checked. This 
change only uses the methods existing in both 2.11.8 and 2.11.12 at Scala.


---