Re: hadoop.job.ugi backwards compatibility
On Mon, Sep 13, 2010 at 1:04 PM, Owen O'Malley wrote: > On Mon, Sep 13, 2010 at 11:10 AM, Todd Lipcon wrote: > > Yep, but there are plenty of 10 node clusters out there that do important > > work at small startups or single-use-case installations, too. We need to > > provide scalability and security features that work for the 100+ node > > clusters but also not leave the beginners in the dust. > > 10 node clusters are an important use case, but creating the user > accounts on those clusters is very easy because of the few users. > Futhermore, if the accounts aren't there it just means the users have > no groups. Which for a single use system with security turned off > isn't the end of the world. > > > But I think there are plenty of people out there who have built small > > webapps, shell scripts, cron jobs, etc that use hadoop.job.ugi on some > > shared account to impersonate other users. > > I'd be surprised. At Yahoo, the primary problem came with people > screen scraping the jobtracker http. With security turned off that > isn't an issue. Again, it isn't hard, just the evolving interface of > UserGroupInformation changed. With security, we tried really hard to > maintain backwards compatibility and succeeded for the vast (99%+) > majority of the users. > > > Perhaps I am estimating > > incorrectly - that's why I wanted this discussion on a user-facing list > > rather than a dev-facing list. > > Obviously the pointer is there for them to follow into the rabbit hole > of the dev lists. *grin* > > > Another example use case that I do a lot on non-secure clusters is: > hadoop > > fs -Dhadoop.job.ugi=hadoop,hadoop superuser>. > > The permissions model we have in 0.20 obviously isn't secure, but it's > nice > > to avoid accidental mistakes, and making it easy to "sudo" like that is > > handy. > > It might make sense to add a new switch ( -user ?) to hadoop fs that > does a doAs before doing the > shell command. You could even make it fancy and try to be a proxy user > if security is turned on. > Yep, I agree - I think either (ab)using proxyuser functionality or adding some new "sudoers" like configuration would be very handy and we should do it. > > > Regardless of our particular opinions, isn't our policy that we cannot > break > > API compatibility between versions without a one-version deprecation > period? > > There wasn't a way to keep UGI stable. It was a broken design before > the security work. It is marked evolving so we try to minimize > breakage, but it isn't prohibited. > > I agree that keeping API compatibility for UGI was probably impossible, and respect that. But it would certainly be very easy to do a patch like the following: JobClient(Configuration conf) { if (conf.get("hadoop.job.ugi") != null && UserGroupInformation.isSecurityEnabled()) { LOG.warn("Stop being evil. Don't use hadoop.job.ugi! RAAWR"); UserGroupInformation.createRemoteUser(...).doAs() { create proxy } } else { create normal RPC proxy; } } ... and the same on the HDFS side. Would you -1 such a compatibility layer? -Todd -- Todd Lipcon Software Engineer, Cloudera
Re: hadoop.job.ugi backwards compatibility
On Mon, Sep 13, 2010 at 11:10 AM, Todd Lipcon wrote: > Yep, but there are plenty of 10 node clusters out there that do important > work at small startups or single-use-case installations, too. We need to > provide scalability and security features that work for the 100+ node > clusters but also not leave the beginners in the dust. 10 node clusters are an important use case, but creating the user accounts on those clusters is very easy because of the few users. Futhermore, if the accounts aren't there it just means the users have no groups. Which for a single use system with security turned off isn't the end of the world. > But I think there are plenty of people out there who have built small > webapps, shell scripts, cron jobs, etc that use hadoop.job.ugi on some > shared account to impersonate other users. I'd be surprised. At Yahoo, the primary problem came with people screen scraping the jobtracker http. With security turned off that isn't an issue. Again, it isn't hard, just the evolving interface of UserGroupInformation changed. With security, we tried really hard to maintain backwards compatibility and succeeded for the vast (99%+) majority of the users. > Perhaps I am estimating > incorrectly - that's why I wanted this discussion on a user-facing list > rather than a dev-facing list. Obviously the pointer is there for them to follow into the rabbit hole of the dev lists. *grin* > Another example use case that I do a lot on non-secure clusters is: hadoop > fs -Dhadoop.job.ugi=hadoop,hadoop . > The permissions model we have in 0.20 obviously isn't secure, but it's nice > to avoid accidental mistakes, and making it easy to "sudo" like that is > handy. It might make sense to add a new switch ( -user ?) to hadoop fs that does a doAs before doing the shell command. You could even make it fancy and try to be a proxy user if security is turned on. > Regardless of our particular opinions, isn't our policy that we cannot break > API compatibility between versions without a one-version deprecation period? There wasn't a way to keep UGI stable. It was a broken design before the security work. It is marked evolving so we try to minimize breakage, but it isn't prohibited. -- Owen
Re: hadoop.job.ugi backwards compatibility
On Mon, Sep 13, 2010 at 10:59 AM, Owen O'Malley wrote: > On Mon, Sep 13, 2010 at 10:05 AM, Todd Lipcon wrote: > > > This is not MR-specific, since the strangely named hadoop.job.ugi > determines > > HDFS permissions as well. > > Yeah, after I hit send, I realized that I should have used common-dev. > This is really a dev issue. > > > "or the user must write a custom group mapper" above refers to this > plugin > > capability. But I think most users do not want to spend the time to write > > (or even setup) such a plugin beyond the default shell-based mapping > > service. > > Sure, which is why it is easiest to just have the (hopefully disabled) > user accounts on the jt/nn. Any installs > 100 nodes should be using > HADOOP-6864 to avoid the fork in the JT/NN. > Yep, but there are plenty of 10 node clusters out there that do important work at small startups or single-use-case installations, too. We need to provide scalability and security features that work for the 100+ node clusters but also not leave the beginners in the dust. > > > As someone who spends an awful lot of time doing downstream support of > lots > > of different clusters, I actually disagree. > > Normal applications never need to do doAs. They run as the default > user. This only comes up in servers that deal with multiple users. In > *that* context, it sucks having servers that only work in non-secure > mode. If some server X only works without security that sucks. Doing > doAs isn't harder, it is just different. Having two different > semantics models *will* cause lots of grief. > I agree that all real (ie community) projects should support both security and non-security and shouldn't be using hadoop.job.ugi to impersonate users. But I think there are plenty of people out there who have built small webapps, shell scripts, cron jobs, etc that use hadoop.job.ugi on some shared account to impersonate other users. Perhaps I am estimating incorrectly - that's why I wanted this discussion on a user-facing list rather than a dev-facing list. Another example use case that I do a lot on non-secure clusters is: hadoop fs -Dhadoop.job.ugi=hadoop,hadoop . The permissions model we have in 0.20 obviously isn't secure, but it's nice to avoid accidental mistakes, and making it easy to "sudo" like that is handy. Regardless of our particular opinions, isn't our policy that we cannot break API compatibility between versions without a one-version deprecation period? I see this as an important API (even if it isn't one we like) and breaking it without such a transition period is against our own rules. Like you said, doAs() isn't any harder, but we need to give people a grace period to switch over, and we probably need to write some command line tools to allow fs operations as superuser, etc. -Todd -- Todd Lipcon Software Engineer, Cloudera
Re: hadoop.job.ugi backwards compatibility
On Mon, Sep 13, 2010 at 10:05 AM, Todd Lipcon wrote: > This is not MR-specific, since the strangely named hadoop.job.ugi determines > HDFS permissions as well. Yeah, after I hit send, I realized that I should have used common-dev. This is really a dev issue. > "or the user must write a custom group mapper" above refers to this plugin > capability. But I think most users do not want to spend the time to write > (or even setup) such a plugin beyond the default shell-based mapping > service. Sure, which is why it is easiest to just have the (hopefully disabled) user accounts on the jt/nn. Any installs > 100 nodes should be using HADOOP-6864 to avoid the fork in the JT/NN. > As someone who spends an awful lot of time doing downstream support of lots > of different clusters, I actually disagree. Normal applications never need to do doAs. They run as the default user. This only comes up in servers that deal with multiple users. In *that* context, it sucks having servers that only work in non-secure mode. If some server X only works without security that sucks. Doing doAs isn't harder, it is just different. Having two different semantics models *will* cause lots of grief. -- Owen
Re: hadoop.job.ugi backwards compatibility
On Sep 13, 2010, at 10:05 AM, Todd Lipcon wrote: > On Mon, Sep 13, 2010 at 9:31 AM, Owen O'Malley wrote: > >> Moving the discussion over to the more appropriate mapreduce-dev. >> > > This is not MR-specific, since the strangely named hadoop.job.ugi determines > HDFS permissions as well. +CC hdfs-dev... though I actually think this is an > issue that users will have interest in, which is why I posted to general > initially rather than a dev list. +1
Re: hadoop.job.ugi backwards compatibility
On Mon, Sep 13, 2010 at 9:31 AM, Owen O'Malley wrote: > Moving the discussion over to the more appropriate mapreduce-dev. > This is not MR-specific, since the strangely named hadoop.job.ugi determines HDFS permissions as well. +CC hdfs-dev... though I actually think this is an issue that users will have interest in, which is why I posted to general initially rather than a dev list. > On Mon, Sep 13, 2010 at 9:08 AM, Todd Lipcon wrote: > > > 1) Groups resolution happens on the server side, where it used to happen > on > > the client. Thus, all Hadoop users must exist on the NN/JT machines in > order > > for group mapping to succeed (or the user must write a custom group > mapper). > > There is a plugin that performs the group lookup. See HADOOP-4656. > There is no requirement for having the user accounts on the NN/JT > although that is the easiest approach. It is not recommended that the > users be allowed to login. > "or the user must write a custom group mapper" above refers to this plugin capability. But I think most users do not want to spend the time to write (or even setup) such a plugin beyond the default shell-based mapping service. > I think it is important that turning security on and off doesn't > drastically change the semantics or protocols. That will become much > much harder to support downstream. > > As someone who spends an awful lot of time doing downstream support of lots of different clusters, I actually disagree. I believe the majority of users do *not* plan on turning on security, so keeping things simpler for them is worth a lot. In many of these clusters the users and the ops team and the developers are all one and the same - it's not the multitenant "internal service" model that we see at the larger installations like Yahoo or Facebook. > > 2) The hadoop.job.ugi parameter is ignored - instead the user has to use > the > > new UGI.createRemoteUser("foo").doAs() API, even in simple security. > > User code that counts on hadoop.job.ugi working will be horribly > broken once you turn on security. Turning on and off security should > not involve testing all of your applications. It is unfortunate that > we ever used the configuration value as the user, but continuing to > support it will make our user's code much much more brittle. > The assumption above is "once you turn on security" - but many users will not and probably never will turn on security. Providing a transition plan for one version is our usual policy here - I agree that long term we would like to do away with this hack of a configuration parameter. Since it's not hard to provide a backwards compatibility path with a deprecation warning for one version, are you against it? Or just saying that on your particular clusters you will choose not to take advantage of it? -Todd -- Todd Lipcon Software Engineer, Cloudera