Re: A new link request to my project and one question
oh, great news... I'll check the config. On Tue, Jun 26, 2018 at 10:30 AM Saisai Shao wrote: > Yes, has a configuration "livy.superusers". Here in this case, the sql > server user should be added as a superuser, who can impersonate other > different users. > > Marcelo Vanzin 于2018年6月26日周二 上午9:12写道： > > > You're talking about another service between the user and the > application. > > > > In that case a parameter probably makes sense. But then you'd need to > > add those config options, because this is a dangerous feature, and > > Livy should know who is allowed to impersonate who. In this case the > > service needs to authenticate to Livy as a privileged user, and Livy's > > configuration would say that the service's user is allowed to > > impersonate certain users or groups (same as the other services that > > allow impersonation like YARN). > > > > > > On Mon, Jun 25, 2018 at 5:41 PM, Takeshi Yamamuro > > > wrote: > > > Yea, I know the Livy supports impersonation. > > > I assume a case blow > > > [different users] ---Some protocols---> [the server applications > managing > > > multiple sessions for users] ---REST---> [Livy server] > > > In this case, Livy already has a way to pass proxyUser from the > > application > > > to Livy? > > > Sorry, but I'm not familiar with Livy internal logic. > > > > > > > > > On Tue, Jun 26, 2018 at 9:14 AM Marcelo Vanzin > > > > > wrote: > > > > > >> On Mon, Jun 25, 2018 at 5:09 PM, Takeshi Yamamuro < > > linguin@gmail.com> > > >> wrote: > > >> > In that case, I think Livy is useful; the application can pass > > proxyUser > > >> to > > >> > build LivyClient for each user > > >> > and run spark queries as each user authorization. > > >> > > >> But Livy already supports impersonation. It can impersonate the > > >> authenticated user. > > >> > > >> You're suggesting adding a parameter so the user can request > > >> impersonation of some specific user, which is a different thing. What > > >> is the use case for that? > > >> > > >> -- > > >> Marcelo > > >> > > > > > > > > > -- > > > --- > > > Takeshi Yamamuro > > > > > > > > -- > > Marcelo > > > -- --- Takeshi Yamamuro
Re: A new link request to my project and one question
> You're talking about another service between the user and the application. yes, I pointed out the case. > In that case a parameter probably makes sense. But then you'd need to > add those config options, yea, I see. Currently, any approach to avoid the dangerous case for the service? I think we can't control who is allowed to impersonate who in the Livy side. Or , In Livy, the use-case I pointed out is out-of-scope? On Tue, Jun 26, 2018 at 10:12 AM Marcelo Vanzin wrote: > You're talking about another service between the user and the application. > > In that case a parameter probably makes sense. But then you'd need to > add those config options, because this is a dangerous feature, and > Livy should know who is allowed to impersonate who. In this case the > service needs to authenticate to Livy as a privileged user, and Livy's > configuration would say that the service's user is allowed to > impersonate certain users or groups (same as the other services that > allow impersonation like YARN). > > > On Mon, Jun 25, 2018 at 5:41 PM, Takeshi Yamamuro > wrote: > > Yea, I know the Livy supports impersonation. > > I assume a case blow > > [different users] ---Some protocols---> [the server applications managing > > multiple sessions for users] ---REST---> [Livy server] > > In this case, Livy already has a way to pass proxyUser from the > application > > to Livy? > > Sorry, but I'm not familiar with Livy internal logic. > > > > > > On Tue, Jun 26, 2018 at 9:14 AM Marcelo Vanzin > > > wrote: > > > >> On Mon, Jun 25, 2018 at 5:09 PM, Takeshi Yamamuro < > linguin@gmail.com> > >> wrote: > >> > In that case, I think Livy is useful; the application can pass > proxyUser > >> to > >> > build LivyClient for each user > >> > and run spark queries as each user authorization. > >> > >> But Livy already supports impersonation. It can impersonate the > >> authenticated user. > >> > >> You're suggesting adding a parameter so the user can request > >> impersonation of some specific user, which is a different thing. What > >> is the use case for that? > >> > >> -- > >> Marcelo > >> > > > > > > -- > > --- > > Takeshi Yamamuro > > > > -- > Marcelo > -- --- Takeshi Yamamuro
Re: A new link request to my project and one question
Yea, I know the Livy supports impersonation. I assume a case blow [different users] ---Some protocols---> [the server applications managing multiple sessions for users] ---REST---> [Livy server] In this case, Livy already has a way to pass proxyUser from the application to Livy? Sorry, but I'm not familiar with Livy internal logic. On Tue, Jun 26, 2018 at 9:14 AM Marcelo Vanzin wrote: > On Mon, Jun 25, 2018 at 5:09 PM, Takeshi Yamamuro > wrote: > > In that case, I think Livy is useful; the application can pass proxyUser > to > > build LivyClient for each user > > and run spark queries as each user authorization. > > But Livy already supports impersonation. It can impersonate the > authenticated user. > > You're suggesting adding a parameter so the user can request > impersonation of some specific user, which is a different thing. What > is the use case for that? > > -- > Marcelo > -- --- Takeshi Yamamuro
Re: A new link request to my project and one question
>> Marcelo Sorry, I missed your response. Yea, thanks for your suggestion. >> Meisam I assume one application handles requests from different users, and the request in the application runs spark queries as each user authorization. Since SparkContext currently doesn't support impersionation, I think one solution is that the application manages multiples SparkContext for that. For example, spark thrift-server doesn't support impersonation because of the same reason above: e.g., SPARK-5159, SPARK-11248, and SPARK-21918 In that case, I think Livy is useful; the application can pass proxyUser to build LivyClient for each user and run spark queries as each user authorization. On Tue, Jun 26, 2018 at 5:33 AM Meisam Fathi wrote: > What is the use case for passing the proxy user to LivyClientBuilder? > > On Fri, Jun 15, 2018 at 9:02 AM Marcelo Vanzin > > wrote: > > > re: proxy user, you have to be extremely careful with that. > > > > Livy currently supports proxy user, but for the server only. It allows > > the server to impersonate anyone, so that sessions can run as the > > requesting user. > > > > If you let the user decide who the session will be run as, you'll need > > to add configuration, just as those available in HDFS, YARN, etc, to > > tell Livy which users can impersonate which other users. Otherwise > > you're basically making authentication meaningless. > > > > > > On Thu, Jun 14, 2018 at 7:36 PM, Saisai Shao > > wrote: > > > Sure, I will merge the website code, thanks! > > > > > > For proxyUser thing, I think there's no particular reason not adding > it, > > > maybe we just forgot to add the proxyUser support. > > > > > > It would be better if you could create a JIRA to track this issue. If > > > you're familiar with Livy code, you can also submit a PR about it. > > > > > > Thanks > > > Jerry > > > > > > Takeshi Yamamuro 于2018年6月15日周五 上午7:33写道： > > > > > >> Hi, Livy dev, > > >> > > >> I opened a new pr in incubator-livy-website to add a new link in > > >> third-party-projects.md. It'd be great if you could check this; > > >> https://github.com/apache/incubator-livy-website/pull/23 > > >> > > >> Btw, I have one question; currently, we cannot pass proxyUser > > >> in LivyClientBuilder. Any reason not to add code for that? > > >> I know we can handle this in an application side by adding a bit code > > like > > >> > > >> > > > https://github.com/maropu/spark-sql-server/blob/master/sql/sql-server/src/main/java/org/apache/livyclient/common/CreateClientRequestWithProxyUser.java > > >> But, If Livy itself supported this, it'd be nice to me. > > >> > > >> Best, > > >> takeshi > > >> > > >> -- > > >> --- > > >> Takeshi Yamamuro > > >> > > > > > > > > -- > > Marcelo > > > -- --- Takeshi Yamamuro
Re: A new link request to my project and one question
Thanks for quick reply! > It would be better if you could create a JIRA to track this issue. If > you're familiar with Livy code, you can also submit a PR about it. Aha, ok. ok, I'll file jira and try to make a pr later. Thanks! On Fri, Jun 15, 2018 at 11:36 AM, Saisai Shao wrote: > Sure, I will merge the website code, thanks! > > For proxyUser thing, I think there's no particular reason not adding it, > maybe we just forgot to add the proxyUser support. > > It would be better if you could create a JIRA to track this issue. If > you're familiar with Livy code, you can also submit a PR about it. > > Thanks > Jerry > > Takeshi Yamamuro 于2018年6月15日周五 上午7:33写道： > > > Hi, Livy dev, > > > > I opened a new pr in incubator-livy-website to add a new link in > > third-party-projects.md. It'd be great if you could check this; > > https://github.com/apache/incubator-livy-website/pull/23 > > > > Btw, I have one question; currently, we cannot pass proxyUser > > in LivyClientBuilder. Any reason not to add code for that? > > I know we can handle this in an application side by adding a bit code > like > > > > https://github.com/maropu/spark-sql-server/blob/master/ > sql/sql-server/src/main/java/org/apache/livyclient/common/ > CreateClientRequestWithProxyUser.java > > But, If Livy itself supported this, it'd be nice to me. > > > > Best, > > takeshi > > > > -- > > --- > > Takeshi Yamamuro > > > -- --- Takeshi Yamamuro
A new link request to my project and one question
Hi, Livy dev, I opened a new pr in incubator-livy-website to add a new link in third-party-projects.md. It'd be great if you could check this; https://github.com/apache/incubator-livy-website/pull/23 Btw, I have one question; currently, we cannot pass proxyUser in LivyClientBuilder. Any reason not to add code for that? I know we can handle this in an application side by adding a bit code like https://github.com/maropu/spark-sql-server/blob/master/sql/sql-server/src/main/java/org/apache/livyclient/common/CreateClientRequestWithProxyUser.java But, If Livy itself supported this, it'd be nice to me. Best, takeshi -- --- Takeshi Yamamuro
