[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235572#comment-16235572 ] ASF subversion and git services commented on KNOX-842: -- Commit 4d60e4149a9fd484f8b54faeb082bf40fa3611d4 in knox's branch refs/heads/master from [~jeffrey...@gmail.com] [ https://git-wip-us.apache.org/repos/asf?p=knox.git;h=4d60e41 ] KNOX-842 Add support for Livy (Spark REST Service). > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez >Priority: Major > Fix For: 0.14.0 > > Attachments: KNOX-842.001.patch > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235356#comment-16235356 ] Jeffrey E Rodriguez commented on KNOX-842: --- VerifIed patch works with current master branch snapshot and can be merged. > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez >Priority: Major > Fix For: 0.14.0 > > Attachments: KNOX-842.001.patch > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235322#comment-16235322 ] Jeffrey Rodriguez commented on KNOX-842: Opened https://issues.apache.org/jira/browse/KNOX-1098 to address no proxyUser value pair limitation. > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez >Priority: Major > Fix For: 0.14.0 > > Attachments: KNOX-842.001.patch > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235268#comment-16235268 ] Jeffrey Rodriguez commented on KNOX-842: Larry McKay and Kevin Riesden. I agree with Kevin. Larry, I will get the following: 1. Update/Verify patch. 2. Document the service and the current limitation. 3. Open a Jira to address the userProxy limitation. > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez >Priority: Major > Fix For: 0.14.0 > > Attachments: KNOX-842.001.patch > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234489#comment-16234489 ] Larry McCay commented on KNOX-842: -- [~jeffreyr97] - as this is assigned to you, I am looking for your acceptance of the above comment and followup work to move this forward for 0.14.0. If we feel that this will work for many users then we should try and get it in with the limitations clearly stated. Please move this out of 0.14.0 or commit and file followup JIRAs. > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez >Priority: Major > Fix For: 0.14.0 > > Attachments: KNOX-842.001.patch > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16217029#comment-16217029 ] Larry McCay commented on KNOX-842: -- [~risdenk] and [~jeffreyr97] - if we can articulate and document what does work and what has to be done in order to not be missing userProxy then I am +1 on opening a JIRA for the gaps - if someone plans to address it. > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez > Fix For: 0.14.0 > > Attachments: KNOX-842.001.patch > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16214115#comment-16214115 ] Kevin Risden commented on KNOX-842: --- [~lmccay] - I agree with [~jeffrey...@gmail.com] about keeping this patch without adding userProxy if it is missing. This patch is working for us right now. > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez > Fix For: 0.14.0 > > Attachments: KNOX-842.001.patch > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16213288#comment-16213288 ] Jeffrey Rodriguez commented on KNOX-842: I will revalidate the patch and if we approve by reviewer I will commit it. > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez > Fix For: 0.14.0 > > Attachments: KNOX-842.001.patch > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16213286#comment-16213286 ] Jeffrey Rodriguez commented on KNOX-842: I will open a new Jira for adding the userProxy. I think we should get the Livy servers push. We should also open another Jira for the client . > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez > Fix For: 0.14.0 > > Attachments: KNOX-842.001.patch > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211845#comment-16211845 ] Larry McCay commented on KNOX-842: -- [~jeffreyr97] - are you working on this patch? Should we move it out of 0.14.0/1.0.0 release? > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez > Fix For: 0.14.0 > > Attachments: KNOX-842.001.patch > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166787#comment-16166787 ] Jeffrey E Rodriguez commented on KNOX-842: --- Hi Kevin. You are right I notice that too, I can also work on adding proxyUser to the request if it is not added. The json rewrite look for a json key value to replace, what we need is a way to add a 'proxyUser" to the livy json payload in case it doesn't exist. > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez > Fix For: Future > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166744#comment-16166744 ] Kevin Risden commented on KNOX-842: --- [~rkellogg] - I didn't get a chance to finish that repo yet. I ran into a few issues that I should be able to cleanup. [~jeffreyr97] - I agree that batches are supported. The issue I had previously when testing was that we weren't specifying proxyUser in the Livy request. Knox will automatically replace proxyUser but will not add proxyUser to the request if it is not specified. > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez > Fix For: Future > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166693#comment-16166693 ] Jeffrey E Rodriguez commented on KNOX-842: --- Will take the work done on the community. I've done some independing testing and batches are supported, and will add the patch to this Jira. I will also work on the documentation and any shell work we may need. > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt >Assignee: Jeffrey E Rodriguez > Fix For: Future > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157567#comment-16157567 ] Rick Kellogg commented on KNOX-842: --- The following project might be a good starting point on this: https://github.com/risdenk/knox_livy_testing (No idea if it works though). > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt > Fix For: Future > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115868#comment-16115868 ] Kevin Risden commented on KNOX-842: --- I was testing the linked Livy service for Knox and noticed it only applies to sessions and not to batches. I'm hoping to look at this closer in the next few weeks, but wanted to note in case someone gets here first. > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt > Fix For: Future > > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html > The Livy service will need to support Kerberos and load balancing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KNOX-842) Add support for Livy (Spark REST Service)
[ https://issues.apache.org/jira/browse/KNOX-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799344#comment-15799344 ] Larry McCay commented on KNOX-842: -- Hi [~samhjelmfelt] - thanks for filing this JIRA and for the article link. Do you plan to attach a tested patch for this implementation? > Add support for Livy (Spark REST Service) > -- > > Key: KNOX-842 > URL: https://issues.apache.org/jira/browse/KNOX-842 > Project: Apache Knox > Issue Type: Improvement > Components: Server >Reporter: Sam Hjelmfelt > > http://livy.io/ > Here is an example of a Livy Knox service implementation: > https://community.hortonworks.com/articles/70499/adding-livy-server-as-service-to-apache-knox.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
