[GitHub] liuxunorg commented on issue #3203: [ZEPPELIN-3626] Cluster server and client module design
liuxunorg commented on issue #3203: [ZEPPELIN-3626] Cluster server and client module design URL: https://github.com/apache/zeppelin/pull/3203#issuecomment-461672873 @zjffdu , CI(https://travis-ci.org/liuxunorg/zeppelin/builds/490335472) Pass, Please help me review the code. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Re: ipython
Great question. with Zeppelin-0.8.x, IPython server (in Python interpreter) can run inside its own docker container. You can check http://zeppelin.apache.org/docs/0.8.1/interpreter/python.html#docker to see how it works. with Zeppelin-0.9.x (not released yet, build from source), all interpreter including Zeppelin itself can run in the container on Kubernetes. You can check http://zeppelin.apache.org/docs/0.9.0-SNAPSHOT/quickstart/kubernetes.html to see how it works. Hope this helps. Thanks, moon On Mon, Feb 4, 2019 at 12:52 PM wrote: > Hi > > Can ipython server run inside its own docker container? How does ipython > server run in Zeppelin container? Does it run inside the Zeppelin > container? > > Thanks >
[jira] [Created] (ZEPPELIN-3996) Maven arifacts resolving for interpreter broken for artifacts with dots in name
Pavel Alexeev created ZEPPELIN-3996:
---
Summary: Maven arifacts resolving for interpreter broken for
artifacts with dots in name
Key: ZEPPELIN-3996
URL: https://issues.apache.org/jira/browse/ZEPPELIN-3996
Project: Zeppelin
Issue Type: Bug
Reporter: Pavel Alexeev
If f.e. for groovy interpreter I set dependency
{{org.apache.ivy:ivy:2.5.0-rc1}} it failed with:
{code}
Error while downloading repos for interpreter group : groovy,
go to interpreter setting page click on edit and save it again
to make this interpreter work properly:
Cannot fetch dependencies for org.apache.ivy:ivy:2.5.0-rc1
...
Caused by: org.sonatype.aether.resolution.DependencyResolutionException:
Could not find artifact com.jcraft:jsch.agentproxy:jar:0.0.9 in central
(http://repo1.maven.org/maven2/)
...
{code}
All works if I exclude {{com.jcraft:jsch.agentproxy}}
Please look [SO
question|https://stackoverflow.com/questions/54538104/apache-zeppelin-how-to-manage-dependencies-for-groovy-interpreter#54540995]
for details.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ZEPPELIN-3995) How to lock down sh, other OS access?
Jake created ZEPPELIN-3995: -- Summary: How to lock down sh, other OS access? Key: ZEPPELIN-3995 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3995 Project: Zeppelin Issue Type: Bug Components: zeppelin-server Affects Versions: 0.8.1 Reporter: Jake I'm running Zeppelin in docker based on the image on docker hub. I've noticed that the sh interpreter, and I guess all others, have access to the configuration files. For example I'm able to change the notebook permissions file using the sh interpreter. This is clearly a problem. Is there a way to change what user the interpreters, i guess, run as which won't have permissions to change the application's configuration? Otherwise, there really isn't any notebook security, right? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] Leemoonsoo commented on issue #3303: [ZEPPELIN-3989] Configure IPython Interpreter in Docker image
Leemoonsoo commented on issue #3303: [ZEPPELIN-3989] Configure IPython Interpreter in Docker image URL: https://github.com/apache/zeppelin/pull/3303#issuecomment-461571102 Ah branch-0.8 already has https://github.com/apache/zeppelin/commit/1b6b269d2c2ca4fb648a695d962a3c820807dad3. This PR is merged to master only. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit closed pull request #3303: [ZEPPELIN-3989] Configure IPython Interpreter in Docker image
asfgit closed pull request #3303: [ZEPPELIN-3989] Configure IPython Interpreter in Docker image URL: https://github.com/apache/zeppelin/pull/3303 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Leemoonsoo commented on issue #3303: [ZEPPELIN-3989] Configure IPython Interpreter in Docker image
Leemoonsoo commented on issue #3303: [ZEPPELIN-3989] Configure IPython Interpreter in Docker image URL: https://github.com/apache/zeppelin/pull/3303#issuecomment-461568824 Thanks @felixcheung for the review. Merging to master and branch-0.8 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (ZEPPELIN-3994) Notebook serving
Lee moon soo created ZEPPELIN-3994:
--
Summary: Notebook serving
Key: ZEPPELIN-3994
URL: https://issues.apache.org/jira/browse/ZEPPELIN-3994
Project: Zeppelin
Issue Type: New Feature
Reporter: Lee moon soo
h2. Motivation
Notebook is useful for interactive analysis. But bringing a model in a Note to
production is another challenge. Often, we see two approaches
a. Call paragraph using Rest API to ZeppelinServer
b. Reimplement model outside the notebook and deploy it without using Zeppelin.
Approach a) brings some headache because of Zeppelin server is not fault
tolerant and can be restarted at any moment intentionally (for changing
configuration, etc). Also notebook to be run through REST API can be changed at
any time without notice.
Approach b) has a clear down side. "re-implementation".
So, It'll be great if a Note can be deployed independently from ZeppelinServer
with RestAPI endpoint, with high availability support. It makes following
usecase really easy and reliable for production use
# Create model in Zeppelin (or any function)
# Click deploy button in a Note
# Access model (any function) using REST api.
h2. Requirements
* Note can be deployed through GUI in a single click ('Deploy' button on every
Note)
* Once Note is deployed, it runs independently from ZeppelinServer. Deployed
Note should keep up and running even if ZeppelinServer is restarting/stopped.
* ZeppelinServer provide a GUI to manage deployments
* Deployed note should be highly available.
* Runs on Kubernetes
h2. Design
h3. Deploy button
Each note have a 'Deploy' button.
h3. On deploy button click
On 'Deploy' button click, following will happen.
* ZeppelinServer creates a new Pod
** The pod snapshot Notebook directory mount it.
** The pod runs another ZeppelinServer
** Current login session information is transferred to the new ZeppelinServer
* ZeppelinServer in a new Pod runs paragraphs in the Note
** Depends on Note, it'll run multiple interpreters.
** Interpreters are running in Kubernetes Deployment Resource instead of Pod
resource.
* Once all paragraphs are successfully run, delete a new ZeppelinServer Pod
and all other Interpreter Deployment resource except for default Interpreter
Deployment resource of the Note.
For example, when A note has Markdown and Python paragraph with python as
default interpreter of a Note, then ZeppelinServer, python and Markdown
interpreter will be created after 'deploy' button click, but ZeppelinServer and
Markdown interpreter will be terminated after successful note run. And Python
interpreter will be remained for serving.
h3. Register model (or any function) to be served
Each interpreter has a
[ResourcePool|https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/resource/ResourcePool.java].
And ResourcePool is programmatically accessible by enduser. For example, user
can register arbitrary object into ResourcePool using API.
{code:java}
%spark
val myModel =
z.put("model_1", myModel){code}
h3. Access model (or any function) from Rest api Endpoint
Every object registered in the ResourcePool is wrapped by
[Resource|https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/resource/Resource.java#L151].
Resource provide method invocation using
[Resource.invokeMethod()|https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/resource/Resource.java#L151].
RemoteInterpreterServer provide a rest api endpoint to call method of Resource
in its ResourcePool. Such as
{code:java}
//{code}
and pass parameters of method using POST param or something.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ZEPPELIN-3993) Where is documentation on HA?
Jake created ZEPPELIN-3993: -- Summary: Where is documentation on HA? Key: ZEPPELIN-3993 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3993 Project: Zeppelin Issue Type: Bug Components: documentation Affects Versions: 0.8.1 Reporter: Jake Fix For: 0.8.1 I'm trying to understand how to deploy Zeppelin where there is more than one Zeppelin server. I've seen mention in various places (e.g. [here (around minute 19|[https://www.youtube.com/watch?v=J6Ei1RMG5Xo])] (or here: [HA Section|https://community.hortonworks.com/articles/141589/zeppelin-best-practices.html]) that is should be possible but I cannot find documentation about how. My case is that I both want HA and I want to be able to scale with demand. I'm unclear how to off-board that configuration (e.g. the notebook authorization conf) to a common location such as database so that it can be shared by the application server instances. It seems like the configuration is local to the instances or maybe I'm just missing something? In my case the plan would be to use s3 as the notebook storage (which I've got working) but I'm unclear how to put the configuration in a common location. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
