[GitHub] [zeppelin] cquptEthan commented on issue #3331: [Zeppelin-4049] Hadoop Submarine (Machine Learning) interpreter
cquptEthan commented on issue #3331: [Zeppelin-4049] Hadoop Submarine (Machine Learning) interpreter URL: https://github.com/apache/zeppelin/pull/3331#issuecomment-471867166 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] cquptEthan commented on issue #3330: [ZEPPELIN-4054] Fix LdapGroupRealm path in shiro doc
cquptEthan commented on issue #3330: [ZEPPELIN-4054] Fix LdapGroupRealm path in shiro doc URL: https://github.com/apache/zeppelin/pull/3330#issuecomment-471863718 @liuxunorg Thanks, I fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] liuxunorg opened a new pull request #3331: [Zeppelin-4049] Hadoop Submarine (Machine Learning) interpreter
liuxunorg opened a new pull request #3331: [Zeppelin-4049] Hadoop Submarine (Machine Learning) interpreter URL: https://github.com/apache/zeppelin/pull/3331 ### What is this PR for? Added the new submarine project for Hadoop in zeppelin to implement machine learning algorithm development and task scheduling in zeppelin. 1. Supports algorithm development and debugging of tensorflow 2. Support offline training of tensorflow 3. Support for Docker containers with YARN-3.1 ### What type of PR is it? [Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4049 ### How should this be tested? Doc: https://github.com/hadoopsubmarine/hadoop-submarine-ecosystem/tree/master/submarine-integration-zeppelin Youtebe：https://www.youtube.com/channel/UC4JBt8Y8VJ0BW0IM9YpdCyQ ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Re: Re: Cannot found MongoNotebookRepo in 0.9.0-SNAPSHOT
Please add ticket id or PR if you would like to contribute it. On Mon, Mar 11, 2019 at 6:28 PM yx91490 wrote: > thank you! I will try it. > > > > > > > 在2019年03月11 09时26分, "Jeff Zhang"写道: > > It is removed in https://issues.apache.org/jira/browse/ZEPPELIN-2619 as we > change the note file structure. I asked that in community, but no one seems > concern about removing that at that time, so I removed in ZEPPELIN-2619. > But I think you can still implement it via storing the file name in > mongodb. And welcome any contribution. > > @ 于2019年3月11日周一 上午9:15写道： > > > Hi everyone, we use MongoNotebookRepo in 0.8.0 in the past, but I found > > that in latest 0.9.0-SNAPSHOT the MongoNotebookRepo doesn't appear in > > zeppelin-plugins module, so I want to know if there any plan to restore > > the MongoNotebookRepo? > > > > > > > > > > > > > > -- > Best Regards > > Jeff Zhang > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [discuss] Zeppelin support workflow
Thanks for the sharing this kind of discussion. I'm interested in it. Will see it. On Mon, Mar 11, 2019 at 10:43 AM Xun Liu wrote: > Hello, everyone > > Because there are more than 20 interpreters in zeppelin, Data analysts > can be used to do a variety of data development, > A lot of data development is interdependent. > For example, the development of machine learning algorithms requires > relying on spark to preprocess data, and so on. > > Zeppelin should have built-in workflow capabilities. Instead of relying on > external software to schedule notes in zeppelin for the following reasons: > > 1. Now that we have upgraded from the data processing era to the algorithm > era, After zeppelin has its own workflow, > Will have a complete ecosystem of complete data processing and algorithmic > operations. > 2. zeppelin's powerful interactive processing capabilities help algorithm > engineers improve productivity and work. > Zeppelin should give the algorithm engineer more direct control. Instead > of handing the algorithm to other teams(or software) to do the workflow. > 3. zeppelin knows more about the processing status of data than Azkaban > and airflow. > So the built-in workflow will have better performance, user experience and > control. > > Typical use case > Especially in machine learning, Because machine learning generally has a > long task execution. > A typical example is as follows: > 1) First, obtain data from HDFS through spark; > 2) Clean and convert the data through sparksql; > 3) Feature extraction of data through spark; > 4) Tensorflow writing algorithm through hadoop submarine; > 5) Distribute the tensorflow algorithm as a job to YARN or k8s for batch > processing; > 6) Publish the training acquisition model and provide online prediction > services; > 7) Model prediction by flink; > 8) Receive incremental data through flink for incremental update of the > model; > Therefore, zeppelin is especially required to have the ability to arrange > workflows. > > I completed the draft of the zeppelin workflow system design, please > review, you can directly modify the document or fill in the comments. > > JIRA: https://issues.apache.org/jira/browse/ZEPPELIN-4018 < > https://issues.apache.org/jira/browse/ZEPPELIN-4018> > gdoc: > https://docs.google.com/document/d/1pQjVifOC1knPBuw3LVvby7GyNDXaeBq1ltRg6x4vDxM/edit > < > https://docs.google.com/document/d/1pQjVifOC1knPBuw3LVvby7GyNDXaeBq1ltRg6x4vDxM/edit> > > > :-) > > Xun Liu > 2019-03-11 -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
[jira] [Created] (ZEPPELIN-4056) DataFrame.registerTempTable(...) doesn't respect isolation levels.
Tyler created ZEPPELIN-4056: --- Summary: DataFrame.registerTempTable(...) doesn't respect isolation levels. Key: ZEPPELIN-4056 URL: https://issues.apache.org/jira/browse/ZEPPELIN-4056 Project: Zeppelin Issue Type: Bug Affects Versions: 0.8.0 Environment: OS X and also Linux, version 0.8.0 at least, and possible 0.8.1 as well (not verified). Reporter: Tyler Open two notebooks. Register a temp table in one, and then run "SELECT * FROM tableName" from a %sql environment. Now open a second notebook, and run the same command, you will see the same results. This is true regardless of the isolation level. In particular, scoped isolation doesn't prevent a notebook from reading temp tables from another notebook. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ZEPPELIN-4054) The LdapGroupRealm path in shiro doc should be update
EthanZhang created ZEPPELIN-4054: Summary: The LdapGroupRealm path in shiro doc should be update Key: ZEPPELIN-4054 URL: https://issues.apache.org/jira/browse/ZEPPELIN-4054 Project: Zeppelin Issue Type: Bug Components: documentation Reporter: EthanZhang Assignee: EthanZhang LdapGroupRealm has moved from org.apache.zeppelin.server to org.apache.zeppelin.realm -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ZEPPELIN-4055) Zeppelin 0.8.1 doesn't run on windows out of the box.
Tyler created ZEPPELIN-4055: --- Summary: Zeppelin 0.8.1 doesn't run on windows out of the box. Key: ZEPPELIN-4055 URL: https://issues.apache.org/jira/browse/ZEPPELIN-4055 Project: Zeppelin Issue Type: Bug Affects Versions: 0.8.1 Environment: Windows 10, JDK 8. Reporter: Tyler Running zeppelin out of the box on windows doesn't work. The symptoms is that you run zeppelin.cmd, and nothing happens, not even an error message. If you copy the file bin\common.cmd from a 0.8.0 install, then it works. This was tested using a fairly stock install of Windows 10. Those two files look almost identical to me, so I'm not sure exactly what the issue is. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [zeppelin] liuxunorg commented on issue #3330: Fix LdapGroupRealm path in shiro doc
liuxunorg commented on issue #3330: Fix LdapGroupRealm path in shiro doc URL: https://github.com/apache/zeppelin/pull/3330#issuecomment-471842131 @cquptEthan , Thanks for the contribution, please create jira and update the PR title. Reading the [Contribution guidelines](https://zeppelin.apache.org/contribution/contributions.html) will help you contribute code better. :-) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] cquptEthan opened a new pull request #3330: Fix LdapGroupRealm path in shiro doc
cquptEthan opened a new pull request #3330: Fix LdapGroupRealm path in shiro doc URL: https://github.com/apache/zeppelin/pull/3330 LdapGroupRealm has moved to org.apache.zeppelin.realm ### What is this PR for? Fix doc ### What type of PR is it? Documentation ### Todos [ ] - Fix LdapGroupRealm path in shiro doc ### What is the Jira issue? no need ### How should this be tested? no need ### Screenshots (if appropriate) no need ### Questions: This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] Leemoonsoo commented on issue #3329: [Website] fix some broken links. Update some link pointing old docs.
Leemoonsoo commented on issue #3329: [Website] fix some broken links. Update some link pointing old docs. URL: https://github.com/apache/zeppelin/pull/3329#issuecomment-471798493 I applied these changes to zeppelin.apache.org. Let me know if there're any other link need to be fixed or updated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] Leemoonsoo opened a new pull request #3329: [Website] fix some broken links. Update some link pointing old docs.
Leemoonsoo opened a new pull request #3329: [Website] fix some broken links. Update some link pointing old docs. URL: https://github.com/apache/zeppelin/pull/3329 ### What is this PR for? Zeppelin Website has some broken links and links that points an old version of docs. ### What type of PR is it? Bug Fix, Improvement ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] Leemoonsoo commented on issue #3312: [ZEPPELIN-4004] Fix RemoteResource.invokeMethod()
Leemoonsoo commented on issue #3312: [ZEPPELIN-4004] Fix RemoteResource.invokeMethod() URL: https://github.com/apache/zeppelin/pull/3312#issuecomment-471790630 Merge to master and branch-0.8 if no further comment. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] asfgit closed pull request #3295: Fix misspelled word
asfgit closed pull request #3295: Fix misspelled word URL: https://github.com/apache/zeppelin/pull/3295 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3297: [ZEPPELIN-3671] Add info about running interpreters to API and JMX
zjffdu commented on a change in pull request #3297: [ZEPPELIN-3671] Add info
about running interpreters to API and JMX
URL: https://github.com/apache/zeppelin/pull/3297#discussion_r264284798
##
File path:
zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
##
@@ -1851,6 +1855,48 @@ public void onSuccess(Note note, ServiceContext
context) throws IOException {
return connectionManager.getConnectedUsers();
}
+ @ManagedOperation
+ public List> getParagraphsInfo(String noteId) {
+Note note = getNotebook().getNote(noteId);
+return note.generateParagraphsInfo();
+ }
+
+ /**
+ * Extract info about running interpreters with additional paragraph info.
+ */
+ @ManagedOperation
+ public List> getRunningInterpretersParagraphInfo() {
Review comment:
Would `getRunningParagraphsInfo` be better ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3325: [ZEPPELIN-4030] spark-scala-2.10 won't be built under some profile
zjffdu commented on a change in pull request #3325: [ZEPPELIN-4030] spark-scala-2.10 won't be built under some profile URL: https://github.com/apache/zeppelin/pull/3325#discussion_r264279267 ## File path: spark/spark-scala-parent/pom.xml ## @@ -34,6 +34,11 @@ pom Zeppelin: Spark Scala Parent + + +2.2.0 Review comment: I changed it to 2.4.0, but actually it doesn't matter which version I specified here. Because type of spark-scala-parent is pom and the spark version will be override by its child module scala-2.10 and scala-2.11. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (ZEPPELIN-4053) Implement impersonation via c native api
Jeff Zhang created ZEPPELIN-4053: Summary: Implement impersonation via c native api Key: ZEPPELIN-4053 URL: https://issues.apache.org/jira/browse/ZEPPELIN-4053 Project: Zeppelin Issue Type: New Feature Reporter: Jeff Zhang Just like what hadoop implement impersonation https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [zeppelin] zjffdu opened a new pull request #3328: [ZEPPELIN-4051]. Commit note is broken
zjffdu opened a new pull request #3328: [ZEPPELIN-4051]. Commit note is broken URL: https://github.com/apache/zeppelin/pull/3328 ### What is this PR for? Commit note is broken due to wrong argument is passed (due to ZEPPELIN-2619). It should be note path instead of note name. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://jira.apache.org/jira/browse/ZEPPELIN-4051 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] Savalek closed pull request #3288: [ZEPPELIN-2274] - update cron UI
Savalek closed pull request #3288: [ZEPPELIN-2274] - update cron UI URL: https://github.com/apache/zeppelin/pull/3288 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] Savalek closed pull request #3255: [ZEPPELIN-3095] fix UI when paragraphs run sequential (in v0.8)
Savalek closed pull request #3255: [ZEPPELIN-3095] fix UI when paragraphs run sequential (in v0.8) URL: https://github.com/apache/zeppelin/pull/3255 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Re:Re: Cannot found MongoNotebookRepo in 0.9.0-SNAPSHOT
thank you! I will try it. 在2019年03月11 09时26分, "Jeff Zhang"写道: It is removed in https://issues.apache.org/jira/browse/ZEPPELIN-2619 as we change the note file structure. I asked that in community, but no one seems concern about removing that at that time, so I removed in ZEPPELIN-2619. But I think you can still implement it via storing the file name in mongodb. And welcome any contribution. @ 于2019年3月11日周一 上午9:15写道： > Hi everyone, we use MongoNotebookRepo in 0.8.0 in the past, but I found > that in latest 0.9.0-SNAPSHOT the MongoNotebookRepo doesn't appear in > zeppelin-plugins module, so I want to know if there any plan to restore > the MongoNotebookRepo? > > > > > > -- Best Regards Jeff Zhang
[GitHub] [zeppelin] zjffdu edited a comment on issue #3297: [ZEPPELIN-3671] Add info about running interpreters to API and JMX
zjffdu edited a comment on issue #3297: [ZEPPELIN-3671] Add info about running interpreters to API and JMX URL: https://github.com/apache/zeppelin/pull/3297#issuecomment-471451924 @egorklimov I am sorry that I miss your PR. Overall I think this is a very pretty cool feature for zeppelin, especially for zeppelin admin to do diagnose. I hope this could be a built-in feature of zeppelin finally (display running interpreter processes info in one built-in page), but exposing api is the first step, we could do the remaining thing in another PR. Could you do a rebase to resolve the conflict ? Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] zjffdu commented on issue #3297: [ZEPPELIN-3671] Add info about running interpreters to API and JMX
zjffdu commented on issue #3297: [ZEPPELIN-3671] Add info about running interpreters to API and JMX URL: https://github.com/apache/zeppelin/pull/3297#issuecomment-471451924 @egorklimov I am sorry that I miss your PR. Overall I think this is a very pretty cool feature for zeppelin, especially for zeppelin admin to do diagnose. I hope this could be a built-in feature of zeppelin finally (display running interpreter processes info in one built-in page), but exposing api is the first step, we could do the remaining thing in another PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] zjffdu opened a new pull request #3327: [ZEPPELIN-4048]. Rename SecurityService to AuthenticationService
zjffdu opened a new pull request #3327: [ZEPPELIN-4048]. Rename SecurityService to AuthenticationService URL: https://github.com/apache/zeppelin/pull/3327 ### What is this PR for? This is a refactoring PR which rename `SecurityService` to `AuthenticationService`, I would also rename `NotebookAuthorization` to `AuthorizationService` in #3316 ### What type of PR is it? [Refactoring] ### Todos * [ ] - Task ### What is the Jira issue? * https://jira.apache.org/jira/browse/ZEPPELIN-4048 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (ZEPPELIN-4052) Add capability to lock note
Jeff Zhang created ZEPPELIN-4052: Summary: Add capability to lock note Key: ZEPPELIN-4052 URL: https://issues.apache.org/jira/browse/ZEPPELIN-4052 Project: Zeppelin Issue Type: New Feature Reporter: Jeff Zhang In lock mode, user can not edit note, but only run note. This is used when moving note from development stage to production stage. Only the owner can lock/unlock note. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
