[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284943#comment-15284943
]
Chris Nauroth commented on HADOOP-12942:
bq. I thought that I used to be able to do that.
Apache Infrastructure recently has needed to tighten up permissions on JIRA as
a spam counter-measure. I suspect things that used to be accessible to you
through the Contributors role went away. Adding you to the Committers role
should solve it, and you belong in that role now anyway. :-)
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Fix For: 2.8.0
>
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch, HADOOP-12942.005.patch,
> HADOOP-12942.006.patch, HADOOP-12942.007.patch, HADOOP-12942.008.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284926#comment-15284926
]
Larry McCay commented on HADOOP-12942:
--
Thanks, [~cnauroth]!
I thought that I used to be able to do that.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Fix For: 2.8.0
>
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch, HADOOP-12942.005.patch,
> HADOOP-12942.006.patch, HADOOP-12942.007.patch, HADOOP-12942.008.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284910#comment-15284910
]
Chris Nauroth commented on HADOOP-12942:
[~lmccay], I have set fix version to 2.8.0. I also added you to the Committers
role in JIRA, so you should be set for future JIRA edits.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Fix For: 2.8.0
>
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch, HADOOP-12942.005.patch,
> HADOOP-12942.006.patch, HADOOP-12942.007.patch, HADOOP-12942.008.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284422#comment-15284422
]
Larry McCay commented on HADOOP-12942:
--
[~yoderme] - Can you set the fix version on this to 2.8? It doesn't seem that I
am able to do it.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch, HADOOP-12942.005.patch,
> HADOOP-12942.006.patch, HADOOP-12942.007.patch, HADOOP-12942.008.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280621#comment-15280621
]
Hudson commented on HADOOP-12942:
-
FAILURE: Integrated in Hadoop-trunk-Commit #9746 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/9746/])
HADOOP-12942. hadoop credential commands non-obviously use password of (lmccay:
rev acb509b2fa0bbe6e00f8a90aec37f63a09463afa)
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/JavaKeyStoreProvider.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/JavaKeyStoreProvider.java
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/alias/TestCredentialProviderFactory.java
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestKeyProviderFactory.java
* hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/CredentialShell.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyProvider.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyShell.java
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/alias/TestCredShell.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/CredentialProvider.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/LocalJavaKeyStoreProvider.java
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestKeyShell.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/AbstractJavaKeyStoreProvider.java
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch, HADOOP-12942.005.patch,
> HADOOP-12942.006.patch, HADOOP-12942.007.patch, HADOOP-12942.008.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280602#comment-15280602
]
Larry McCay commented on HADOOP-12942:
--
This has been committed to trunk, branch-2 and branch-2.8.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch, HADOOP-12942.005.patch,
> HADOOP-12942.006.patch, HADOOP-12942.007.patch, HADOOP-12942.008.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280583#comment-15280583
]
Larry McCay commented on HADOOP-12942:
--
+1 - I will commit this to trunk, branch-2 and branch-2.8.
Thanks for the patch, [~yoderme]!
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch, HADOOP-12942.005.patch,
> HADOOP-12942.006.patch, HADOOP-12942.007.patch, HADOOP-12942.008.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279565#comment-15279565
]
Hadoop QA commented on HADOOP-12942:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
45s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 53s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 43s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 0s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
36s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 52s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s {color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 111 unchanged - 78 fixed = 111 total (was 189) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 44s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.8.0_91.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 54s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 55s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12803345/HADOOP-12942.008.patch
|
| JIRA Issue | HADOOP-12942 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 54186218ac46 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279347#comment-15279347
]
Larry McCay commented on HADOOP-12942:
--
Same thing happened to me yesterday.
I think that reporting on checkstyle errors in test classes must have just been
turned back on or something.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch, HADOOP-12942.005.patch,
> HADOOP-12942.006.patch, HADOOP-12942.007.patch, HADOOP-12942.008.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279047#comment-15279047
]
Larry McCay commented on HADOOP-12942:
--
[~yoderme] - can you address the checkstyle and whitespace issues above?
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch, HADOOP-12942.005.patch,
> HADOOP-12942.006.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15277467#comment-15277467
]
Hadoop QA commented on HADOOP-12942:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
3s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 1s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 48s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 2s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
37s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} trunk passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 54s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 54s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 50s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 25s
{color} | {color:red} hadoop-common-project/hadoop-common: The patch generated
5 new + 109 unchanged - 78 fixed = 114 total (was 187) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 16s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 48s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 50s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_91 Timed out junit tests |
org.apache.hadoop.http.TestHttpServerLifecycle |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12803090/HADOOP-12942.006.patch
|
| JIRA Issue | HADOOP-12942 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux acf7768804cc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15277098#comment-15277098
]
Mike Yoder commented on HADOOP-12942:
-
Patch 6: now only show the warnings on the create command.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch, HADOOP-12942.005.patch,
> HADOOP-12942.006.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15273414#comment-15273414
]
Larry McCay commented on HADOOP-12942:
--
Hi [~yoderme] - this is looking pretty good.
* I don't like that the warnings are displayed on commands other than create
however. In fact, it really should only be displayed when the keystore is being
created because it doesn't exist yet.
However I could be convinced that they should be warned that they are adding a
new credential to a provider that is using the default password.
* It seems that there are a couple lines with trailing whitespace in the
command manual change as well.
I think if we can change the above we are good to go!
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch, HADOOP-12942.005.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15252248#comment-15252248
]
Hadoop QA commented on HADOOP-12942:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
45s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 9s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 34s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 41s
{color} | {color:green} the patch passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s {color} | {color:green} hadoop-common-project/hadoop-common: patch
generated 0 new + 38 unchanged - 70 fixed = 38 total (was 108) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 54s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_77. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 7s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 16s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_77 Failed junit tests |
hadoop.security.ssl.TestReloadingX509TrustManager |
| | hadoop.net.TestDNS |
| JDK v1.8.0_77 Timed out junit tests |
org.apache.hadoop.http.TestHttpServerLifecycle |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:fbe3e86 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/1271/HADOOP-12942.005.patch
|
| JIRA Issue | HADOOP-12942 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
|
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240196#comment-15240196
]
Mike Yoder commented on HADOOP-12942:
-
So it's not just the absolute number of checkstyle violations, it knows which
ones were yours. Ow!
Regarding the latest patch... it differs in only 4 whitespace characters from
the previous patch, which did pass the unit tests. The
hadoop.security.ssl.TestReloadingX509TrustManager failure passes for me; looks
unrelated.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240111#comment-15240111
]
Hadoop QA commented on HADOOP-12942:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 9s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
43s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 13s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 11s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 1s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 29s
{color} | {color:green} the patch passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 29s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 15s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} hadoop-common-project/hadoop-common: patch
generated 0 new + 38 unchanged - 70 fixed = 38 total (was 108) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 11s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_77. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 26s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 67m 39s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_77 Failed junit tests | hadoop.net.TestClusterTopology |
| | hadoop.security.ssl.TestReloadingX509TrustManager |
| JDK v1.7.0_95 Failed junit tests | hadoop.ha.TestZKFailoverController |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:fbe3e86 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12798569/HADOOP-12942.004.patch
|
| JIRA Issue | HADOOP-12942 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f8c7ca5a3aba 3.13.0-36-lowlatency
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240096#comment-15240096
]
Larry McCay commented on HADOOP-12942:
--
Hi [~yoderme] - thanks for the new patch.
I will try and review it tonight or tomorrow.
Looks like you got flagged for adding a new checkstyle violation - even though
you fixed 70. :)
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch,
> HADOOP-12942.003.patch, HADOOP-12942.004.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238393#comment-15238393
]
Hadoop QA commented on HADOOP-12942:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 29s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 42s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 59s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 40s
{color} | {color:green} the patch passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 31s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 31s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 22s
{color} | {color:red} hadoop-common-project/hadoop-common: patch generated 1
new + 39 unchanged - 70 fixed = 40 total (was 109) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 29s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.8.0_77.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 8s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 74m 22s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:fbe3e86 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12798407/HADOOP-12942.003.patch
|
| JIRA Issue | HADOOP-12942 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux bd0fe18c537d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 35f0770 |
|
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233560#comment-15233560
]
Larry McCay commented on HADOOP-12942:
--
The link to the Keystore Passwords section will be:
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html#Keystore_Passwords
- once HADOOP-13011 is committed.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232832#comment-15232832
]
Larry McCay commented on HADOOP-12942:
--
We may want to wait till we have a link for the "documentation regarding
provider passwords available here..." which I am working on before putting
together another patch.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232519#comment-15232519
]
Mike Yoder commented on HADOOP-12942:
-
OK, I'll take your last suggestion. Will have the patch in a bit.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229651#comment-15229651
]
Larry McCay commented on HADOOP-12942:
--
{code}
WARNING: You have accepted the use of the default password by not configuring a
password in either:
o The environment variable ...
o File referred to by the configuration entry ...
Please review the documentation regarding provider passwords available here...
Continuing with default provider password.
{code}
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229646#comment-15229646
]
Larry McCay commented on HADOOP-12942:
--
Warning should be something like:
bq. WARNING: You have accepted the use of the default password by not
configuring a password in either:
o The environment variable ...
o File referred to by the configuration entry ...
Please review the documentation regarding provider passwords available here...
Continuing with default provider password.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228816#comment-15228816
]
Larry McCay commented on HADOOP-12942:
--
This statement is a good place start:
bq. Or to come at this a different way - I can think of no other secure system
involving a password where the use of a default hardcoded password is common.
This actually is done in a number of systems that I have seen. Calling this a
secure mechanism is would be too strong even with your proposed change. We are
talking about levels of protection.
What the keystore based providers are doing is making the clear text password
in a file permissions protected file a bit better. Both mechanisms require file
permissions to be thwarted in order to get to the file. However, the clear text
password in a file can then easily be gotten to with standard tools. There are
no standard tools that can be used to get to the password in a keystore.
Keytool doesn't even allow you to read a secret stored this way - whether you
have the password or not. Not that you can't write a tool to get to it but it
is still better than the clear text password in a file. Claiming that the
protection of the keystore with a clear text password in a file or environment
variable makes it a secure system is a stretch.
The other benefit of the credential provider API is that it provides an
abstraction and API to easily migrate to a better provider when it comes online.
Warning that the default password is being used is sufficient when the -strict
flag is absent. The language should make it clear that they have accepted the
default password but not make it seem like they have done something wrong.
Dumping the default password into the command line seems a little too public to
me. What do you think about pointing them to documentation to find out more
about the default password? We will need to put docs together for this anyway
and we can pull what I have in trunk and branch-2 in with new language for the
password protection of the keystore providers.
bq. Please note that when this provider is used in the future, the password
must also be available to it in the same manner.
I fear that "in the future" is too vague. It reads like a temporal thing
instead of an "all consumers" thing. It needs to be clear that any clients or
jobs that require this password have access to the file and the password. I
think that it only needs to be presented as part of the ERROR language as well.
If they are accepting the use of the default password then other consumers will
not have a problem.
This is actually the trickiest part of having the non-default password and will
be the root cause of lots of support calls/tickets. This is why I am reluctant
make the use of the default password seem like an error.
I am looking to provide a patch that will move the secrets required by a job
into the Credentials file during job setup and will make this a moot point
eventually. Once we have that in place, the only parties that need access to
the keystores are the ones setting up environments for jobs/applications.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228718#comment-15228718
]
Mike Yoder commented on HADOOP-12942:
-
Thanks for having a look.
{quote}
WARN messages (without strict flag): read too much like ERRORs \[...] It is
perfectly legitimate to use the static/hardcoded password.
{quote}
See, here's where we disagree. Using the CredentialProvider or KeyProvider
indicates that the user cares about security. Otherwise they wouldn't use the
feature at all - for example just providing a cleartext password instead of
getting it through the CredentialProvider. So if the user cares about
security, they are going to care that the provider is actually protecting the
information.
Or to come at this a different way - I can think of no other secure system
involving a password where the use of a default hardcoded password is common.
So yeah, given my assumptions above the WARN messages are pretty severe on
purpose. It's difficult for me to fathom a (security conscious) user who, upon
learning that they were using a static hardcoded password, would say "meh".
{quote}
a provider *requires* a password
{quote}
Well, it requires a password for an attempt at secure operation.
{quote}
It would also be good to let the user know that when a custom password is
being used that it must be available to the runtime consumers of it as well.
The trick is communicating all of this without spitting out a book.
{quote}
Quite true. How about the following two new lines:
{noformat}
WARNING: The provider cannot find a password in the expected locations.
Please supply a password using one of the following two mechanisms:
o In the environment variable ...
o In a file referred to by the configuration entry ...
Please note that when this provider is used in the future, the password must
also be available to it in the same manner.
Continuing with default provider password "none"
{noformat}
{quote}
I'm not sure that the hardcoded password needs to be emitted on the command
line in order to satisfy "obviousness".
{quote}
My thinking was that the user might want to figure out what the default
password is, and so if the information is public, I might as well be helpful
right on the command line.
{quote}
I think we should take this opportunity to revisit the 700 file permissions
and change it to 600
{quote}
OK, makes me a little nervous to lump that in, but sure.
{quote}
the consolidation of some caught keystore exceptions
{quote}
There was one place I changed
{noformat}
-} catch (NoSuchAlgorithmException e) {
- throw new IOException("Can't load keystore " + getPathAsString(), e);
-} catch (CertificateException e) {
- throw new IOException("Can't load keystore " + getPathAsString(), e);
-}
{noformat}
to this
{noformat}
+} catch (GeneralSecurityException e) {
+ throw new IOException("Can't load keystore " + getPathAsString(), e);
+}
{noformat}
just to collapse the to dups into one.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15227089#comment-15227089
]
Larry McCay commented on HADOOP-12942:
--
Hi [~yoderme] - Sorry for the delay in review. I am really underwater with
other things. I have taken a quick look at this and think that it is looking a
lot better.
Comments:
1. WARN messages (without strict flag): read too much like ERRORs. It should be
talking about the fact that a unique password could be used by provisioning it
in one of the following ways instead of it was expected to be found here or
there. It is perfectly legitimate to use the static/hardcoded password. I think
that this can be done easily with minor changes to the language.
2. There is a bit more language that is too strongly leaning toward ERROR. For
instance, there are cases where it is being communicated that a provider
*requires* a password and none is given. Interesting word play aside, there is
one given it happens to be the default one. Basically, I would like to see the
semantics of the -strict flag as indicating a desire for a unique or custom
password to be used and the lack of the -strict flag as accepting the default
password.
3. It would also be good to let the user know that when a custom password is
being used that it must be available to the runtime consumers of it as well.
The trick is communicating all of this without spitting out a book.
4. I'm not sure that the hardcoded password needs to be emitted on the command
line in order to satisfy "obviousness". I would rather see it be referred to as
the default password. The default password can easily be documented in the
command or credential provider docs so that it can be found, when needed.
5. I think we should take this opportunity to revisit the 700 file permissions
and change it to 600 - unless there is some reason that 700 is needed.
That's as deep as I could dig in today. There are a couple things that I would
like to dig deeper into like the consolidation of some caught keystore
exceptions and minor loss of context. I can't tell if some refactoring made the
previous exceptions not possible anymore or what.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch, HADOOP-12942.002.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15222630#comment-15222630
]
Hadoop QA commented on HADOOP-12942:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
36s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 49s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 40s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} trunk passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 42s
{color} | {color:green} the patch passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 45s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} hadoop-common-project/hadoop-common: patch
generated 0 new + 38 unchanged - 70 fixed = 38 total (was 108) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed with JDK v1.8.0_77 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 53s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_77. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 5s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_95.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 13s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_77 Timed out junit tests |
org.apache.hadoop.http.TestHttpServerLifecycle |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:fbe3e86 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12796629/HADOOP-12942.002.patch
|
| JIRA Issue | HADOOP-12942 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 07bde5f83451 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
|
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212533#comment-15212533
]
Larry McCay commented on HADOOP-12942:
--
HA - yeah, we are stepping on each other
bq. So that means that the config file would have to change so that the name of
the file is provided... and the command can't do that itself. Right? This has
to be an independent step taken by the user I assume.
Right.
bq. by this you mean creating the file with the password, assuming that the
config file mentions a file?
Correct.
bq. Well, it does give the user the flexibility to set up the password in the
file or use the environment variable at their leisure at a later date.
Absolutely, and in other systems, I would agree with this. I would much rather
fail early than later here though.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212513#comment-15212513
]
Larry McCay commented on HADOOP-12942:
--
bq. They aren't - but they never were assured of this in the first place.
They were assured of the default password always being available.
If we are going to define the best practice as using the password-in-a-file
approach then it should be documented clearly that this config setting needs to
be in place otherwise the default will be used.
bq. There is a higher probability of that with my patch, yes. I believe this to
be better than setting the user up for unintentional insecure storage of
secrets. I don't know how to handle this better, and I'm not sure that we can
since we don't know how the cred store will be accessed in the future.
Not if we warn them AND allow them to be protected from it with a -strict flag.
bq. See if I agreed with this I never would have filed this jira. I feel that
it is a bug to give the user the impression that a value is being securely
stored when in fact it is not. Hardcoded "none" provides no protection.
I can agree that the non-obvious aspect of this JIRA is a bug.
A hardcoded password actually provides more protection than that afforded to
the password-in-the-file itself.
It will only be protected by file permissions.
Printing a warning, documenting it clearly and providing a -strict switch would
satisfy the non-obviousness bug.
If we didn't have this existing pattern of referencing password-files and had a
way to connect all the dots for the password that we get through the prompt
then this would be perfect. Unfortunately, I think that prompting like this
adds unnecessary complexity in order to add a password that we know isn't
provisioned in the config and will likely lead to a failure at runtime rather
than provisioning time. I believe that we want to fail early.
I should clearly state that I don't think that the password-in-a-file actually
makes security much better but I can appreciate the idea of using it. But we
really are just moving the problem around until we have a credential server.
We are talking about protecting an encoded credential store which is protected
with file permissions with a password that is stored in clear text in a file
that is protected with file permissions. Add, to that, the complexity of
ensuring that file is configured properly in all environments and it becomes an
availability problem.
My preference is to try and clearly define a best practice until we have a
credential server that uses existing functionality and makes it very clear that
a default password is being used otherwise.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212489#comment-15212489
]
Mike Yoder commented on HADOOP-12942:
-
Oh, hey, I didn't see your second comment before posting. We're getting
closer...
You say
{quote}
provision a password to a file for your credential providers
{quote}
So that means that the config file would have to change so that the name of the
file is provided... and the command can't do that itself. Right? This has to
be an independent step taken by the user I assume.
{quote}
use the hadoop credential CLI to provision the actual credential required by MR
jobs, etc
{quote}
by this you mean creating the file with the password, assuming that the config
file mentions a file?
{quote}
I wouldn't be opposed to a -strict switch that doesn't allow the default
password to be used either.
{quote}
Yeah that's a good idea.
{quote}
Prompting for a password that has not been provisioned yet will lead to runtime
problems.
{quote}
Well, it does give the user the flexibility to set up the password in the file
or use the environment variable at their leisure at a later date.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212473#comment-15212473
]
Mike Yoder commented on HADOOP-12942:
-
I guess I didn't explain my intent to prompt the user for a password very
clearly. My (admittedly simplistic) thinking was "hey there's no password".
"We should therefore make sure there's a password."
{quote}
If we are leveraging the password-in-a-file approach then why are we going to
prompt the user for a password? It should be in the config if that is what is
going to be used.
{quote}
So if there is in fact a password in a file referred to in the config, it takes
priority and the user will never be prompted for a password. That's why the
providers' needsPassword() has to exist. We aren't doing anything new with the
password-in-a-file approach with this patch; it's has been there and continues
to be there.
{quote}
Additionally, how is the MR job going be assured to have the password to access
the keystore by this?
{quote}
They aren't - but they never were assured of this in the first place. If you're
reading from a file pointed to by the config, you're assuming that the same
config will exist in the context in which it's later used (and that the file
exists, too). If you're using an environment variable, you're assuming the
environment variable is going to exist in the future context in which it's
later used. Neither of these are guaranteed.
{quote}
If you are setting a password without it being first provisioned in the file
then you are setting them up for a credential store that can't be opened.
{quote}
There is a higher probability of that with my patch, yes. I believe this to be
better than setting the user up for unintentional insecure storage of secrets.
I don't know how to handle this better, and I'm not sure that we can since we
don't know how the cred store will be accessed in the future.
{quote}
The current behavior should find the provisioned keystore password from the
file and create the credential store appropriately with no need to prompt the
user. This is the intended behavior by design and keeps the config aligned with
the keystore password.
{quote}
I see what you're getting at, but I guess I have not felt that they are as
"aligned" as you feel they are.
So instead of prompting the user for a password, you would instead check for
either the password-in-a-file or the environment variable, and if they don't
exist, error out with a message stating that the provider couldn't find the
password and here's how to provide it?
That would achieve the same sort of goal, but it just seemed easier and a
better interface to just ask the user for the password. I suppose my patch
doesn't give the user any hints on how to set things up so future stuff could
read the keystore, though, which isn't great.
{quote}
The current behavior isn't a bug "none" is a real password
{quote}
See if I agreed with this I never would have filed this jira. :-) I feel that
it is a bug to give the user the impression that a value is being securely
stored when in fact it is not. Hardcoded "none" provides no protection.
{quote}
I also see the backward compatibility issue as a non-starter
{quote}
I view the current _interface_ as having the bug - that interface being the
non-obvious use of a password "none". As such, the interface ought to change,
and as such that means a backwards compatibility issue. But...ergh. If we must
keep the interface safe for scripts and the like... how about the following
algorithm
.
* if there are no new command line arguments
** if file or env var found
*** great, continue as before
** else
*** print a big WARNING that they are using a password of "none" and
instructions on how to set it; continue as before
* else if "-password" or "-askMeTheProviderPassword" is found on the command
line obtain the provider password, and
** if the provider _already_ has a password via file or env var, print a
WARNING that the file or env var exists, and that the user supplied password
will be ignored
** else pass the given password into the provider.
This gives us backwards compatibility, notification to the user that they're
doing something insecure, and a way to provide the password in the command
itself. Your thoughts?
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212444#comment-15212444
]
Hadoop QA commented on HADOOP-12942:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
36s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 43s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 33s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 40s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m 39s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m 39s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 22s
{color} | {color:red} hadoop-common-project/hadoop-common: patch generated 19
new + 103 unchanged - 4 fixed = 122 total (was 107) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 40s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_74. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 54s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 24s
{color} | {color:red} Patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 21s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_74 Failed junit tests | hadoop.crypto.key.TestKeyProviderFactory |
| JDK v1.8.0_74 Timed out junit tests |
org.apache.hadoop.util.TestNativeLibraryChecker |
| JDK v1.7.0_95 Failed junit tests | hadoop.crypto.key.TestKeyProviderFactory |
| JDK v1.7.0_95 Timed out junit tests |
org.apache.hadoop.util.TestNativeLibraryChecker |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:fbe3e86 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12795453/HADOOP-12942.001.patch
|
| JIRA Issue | HADOOP-12942 |
| Optional Tests | asflicense compile javac
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212372#comment-15212372
]
Larry McCay commented on HADOOP-12942:
--
I see that I did misunderstand your conclusion earlier. I apologize for that.
I don't see any point in prompting for a password that will successfully create
a keystore and not fail until access is attempted from a running MR job.
I think that the core issue can be addressed with a warning at credential
creation time that the default password is being used and that they might want
to consider provisioning a password in a file and add that filename to the
config.
Current behavior + what we should consider best practice according to this JIRA
is:
1. provision a password to a file for your credential providers
2. set this file location as the configured password file (really part of
provisioning above...)
3. use the hadoop credential CLI to provision the actual credential required by
MR jobs, etc
4. if default password is used warning the user
The combination of the warning from the CLI and better documentation should be
all that we need here.
I wouldn't be opposed to a -strict switch that doesn't allow the default
password to be used either.
So, when that is set we don't fallback to the default but fail out of the CLI
with appropriate error message.
An explicit switch to be -strict about it retains backward compatibility too.
Prompting for a password that has not been provisioned yet will lead to runtime
problems.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212334#comment-15212334
]
Larry McCay commented on HADOOP-12942:
--
[~yoderme] - I'm not clear on the intent here. If we are leveraging the
password-in-a-file approach then why are we going to prompt the user for a
password? It should be in the config if that is what is going to be used.
Additionally, how is the MR job going be assured to have the password to access
the keystore by this?
If you are setting a password without it being first provisioned in the file
then you are setting them up for a credential store that can't be opened. The
current behavior should find the provisioned keystore password from the file
and create the credential store appropriately with no need to prompt the user.
This is the intended behavior by design and keeps the config aligned with the
keystore password.
I also see the backward compatibility issue as a non-starter. The current
behavior isn't a *bug* "none" is a real password and there is code that is
likely depending on that behavior. If we want to add the ability to prompt for
the password then it has to be the other way round. If there is no password
provided on the command line and no --interactive (-i) switch then you have to
use "none".
I'm sorry if I misunderstood our previous discussion on this but I am -1 on
this as it stands - unless I also misunderstand the patch.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212291#comment-15212291
]
Mike Yoder commented on HADOOP-12942:
-
At last, a patch. I fixed both the KeyShell and CredentialShell, since they
have the same problem. I also noticed that the CredentialShell threw an NPE
with the "-help" commands, so I fixed that while I was in there. The new code
will prompt for the password for the provider if one is needed, and it will
also accept "-password xxx" on the command line. Note that there is a
backwards compatibility issue here: the user has to give a password where none
was required before. I don't see a way around this, however, since not having a
real password was the root cause of this bug. I did set it up so that if the
user just hits 'enter' (no password) when prompted, the default "none" is used
instead, which is the prior behavior.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>Assignee: Mike Yoder
> Attachments: HADOOP-12942.001.patch
>
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207612#comment-15207612
]
Larry McCay commented on HADOOP-12942:
--
It's actually a "password in a file that is referenced from config" and is a
specific pattern in hadoop.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207610#comment-15207610
]
Larry McCay commented on HADOOP-12942:
--
Agreed. :)
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207603#comment-15207603
]
Mike Yoder commented on HADOOP-12942:
-
Oh goodness. When you expand it to the general paradigm of "a password in a
file..." yeah, I do recognize most of those. I was just thinking of the concept
as applied to the providers in the discussion so far. Let me start without the
pwdfile command at all. On some level, an "echo asdf > file && chmod 400 file"
isn't that hard. Or at least not implement in the first pass - it's a separate
problem from the rest.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207546#comment-15207546
]
Larry McCay commented on HADOOP-12942:
--
There are at least the following additional password files throughout the
ecosystem - I'm sure that there are probably more:
Hadoop:
hadoop-http-auth-signature-secret
hadoop.security.group.mapping.ldap.ssl.keystore.password.file
hadoop.security.group.mapping.ldap.bind.password.file
HBase JMX Remote:
HBASE_JMX_OPTS="$HBASE_JMX_OPTS
-Dcom.sun.management.jmxremote.password.file=$HBASE_HOME/conf/jmxremote.passwd"
HBase Web UIs
TLS/SSL Server Keystore File Password - Password for the server keystore file
used for encrypted web UIs.
TLS/SSL Server Keystore Key Password- Password that protects the private
key contained in the server keystore used for encrypted web UIs.
HBase REST Server
HBase REST Server TLS/SSL Server JKS Keystore File Password - The password
for the HBase REST Server JKS keystore file.
HBase REST Server TLS/SSL Server JKS Keystore Key Password - The password
that protects the private key contained in the JKS keystore used when HBase
REST Server is acting as a TLS/SSL server.
HBase Thrift Server
HBase Thrift Server over HTTP TLS/SSL Server JKS Keystore File Password - The
password for the HBase Thrift Server JKS keystore file.
HBase Thrift Server over HTTP TLS/SSL Server JKS Keystore Key Password - The
password that protects the private key contained in the JKS keystore used when
HBase Thrift Server over HTTP is acting as a TLS/SSL server.
Oozie SSL/TLS
Oozie TLS/SSL Server JKS Keystore File Password - Password for the keystore.
I'd rather not add this work to the Key and Credential provider commands.
The keystore providers are both just consumers of the same password file
pattern found else where through out hadoop.
I believe that it is generally part of the administrative platforms like Ambari
and Cloudera Manager but if you would like a CLI management tool then I think
that may add some value. Like I described it could take care of the permissions
settings, etc. Which would all be separate manual steps from the command line.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207098#comment-15207098
]
Mike Yoder commented on HADOOP-12942:
-
{quote}
I have heard reluctance from folks in the past for having commands prompt for
passwords and would certainly break the scriptability of it. We would have to
add a switch that enabled the prompting for a password - if we were to add it
to the credential create subcommand.
{quote}
Agreed. Today as you know the credential create command prompts for a password
but there is an undocumented "-value" argument that can be used. I'd stick
with the same scheme where either a prompt or command line argument were
possible.
{quote}
This same password file is used in lots of scenarios though: KMS, javakeystore
providers for key provider API, oozie, signing secret providers,e tc. I wonder
whether a separate command for it would make sense.
{quote}
Conceptually, yes, but aren't config values different? I'm aware of two:
* alias/AbstractJavaKeyStoreProvider:
hadoop.security.credstore.java-keystore-provider.password-file
* key/JavaKeyStoreProvider:
hadoop.security.keystore.java-keystore-provider.password-file
{quote}
Keep in mind that we would need to do a number of things for this.
1. prompt for the password
2. persist it
3. set appropriate permissions on the file
4. somehow determine the filename to use (probably based on the password file
name configuration) which would need to be provided by the user as well
5. allow for use of the same password file for multiple keystores or scenarios
6. allow for random-ish generated password without prompt
{quote}
I think it's even more complicated. :-) The user could want to use the
environment variable when the credential is consumed, and so would want to
provide it to the command but would not want to deal with anything
file-related.
Also it's conceivable that the user could have constructed the file themselves;
although this doesn't seem particularly user friendly.
So we have scenarios for hadoop credential create|list|etc that look like
# Here is the credstore password from a prompt
# Here is the credstore password on the command line
# The credstore password is already in a file in the "expected" location (set
up either by hand or via your new pwdfile command).
Making a command to manage the password file makes sense. I think that we
shouldn't ask the user to give it the property name though: you could modify
KeyShell and CredentialShell to have a new subcommand of 'pwdfile', thusly:
* hadoop credential pwdfile \[args\]
* hadoop key pwdfile \[args\]
And they could share an implementation. This way the user does not have to
remember "hadoop.security.credstore.java-keystore-provider.password-file" or
the like. This also means that the provider selected needs a new interface to
create said file, if applicable.
I like the auto-generate-password option for the file. I think the default
would be to still prompt for the password, though. So yeah, adding a pwdfile
command seems like a good idea.
The thing about the existing design that I'm going back and forth on is that
the CredentialShell is high-level, and selects a provider and then simply
passes information to the provider. The password is implied and not passed
directly, so the CredentialShell has no notion of whether or not the underlying
provider actually has a password or not.
So, for example, it would be daft of CredentialShell to accept a password on
the command line if one is provided in a file, and it would also be even more
daft if no password was specifed on the command line and the password wasn't in
the password file either. Furthermore it would be silly to accept a password
when the underlying provider does not need a password at all for proper
operation (example: the UserProvider). There has to be some amount of
communication between the CredentialShell and the provider in order to get the
"is a password required" and "where precisely is the password" cases correct.
To make this even more interesting, in the various providers with a key store,
the keyStore is either created or opened in the constructor, requiring that all
the information be presented up front - without scope for the back and forth of
"do you need a password and where" from the provider.
So... one way to deal with this is to move the keyStore.load() call out of the
constructor and defer it until the first get/set/delete credential entry call.
Then expose interfaces along the lines of "does this provider already have the
password somehow?" and "set the password directly". We'd have to add default
behavior in CredentialProvider (and KeyProvider) and then implement in the ones
that matter.
The downside to this approach is that we move around a few error conditions.
However everything can throw an IOException, so maybe this isn't a big deal.
Seem
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205626#comment-15205626
]
Larry McCay commented on HADOOP-12942:
--
Let's walk through that proposal.
I think that the password file is marginally more secure because both files
would have to be accessible in order to access the keystore and some folks may
be willing to manage more files in order to get that additional protection. In
addition, gaining access to one of those password files will only provide
access to keystores that an attacker has access to *and* are protected by that
particular password.
The AbstractJavaKeyStoreProvider already has support for a password file and
can easily be used - we definitely need to document this clearly.
I have heard reluctance from folks in the past for having commands prompt for
passwords and would certainly break the scriptability of it. We would have to
add a switch that enabled the prompting for a password - if we were to add it
to the credential create subcommand.
This same password file is used in lots of scenarios though: KMS, javakeystore
providers for key provider API, oozie, signing secret providers,e tc. I wonder
whether a separate command for it would make sense.
Keep in mind that we would need to do a number of things for this.
1. prompt for the password
2. persist it
3. set appropriate permissions on the file
4. somehow determine the filename to use (probably based on the password file
name configuration) which would need to be provided by the user as well
5. allow for use of the same password file for multiple keystores or scenarios
6. allow for random-ish generated password without prompt
So, something like:
hadoop pwdfile -pwdfile.property.name
hadoop.security.credstore.java-keystore-provider.password-file [-generate true]
[-permissions 400]
This would check the Configuration for the provided pwdfile.property.name to
get the file to persist the password to.
If generate is set to true then it doesn't prompt and generates a password to
use - otherwise, prompts for a password.
(I could also see the opposite approach which would be default to generate
unless a -interactive --i type switch is provided.)
If permissions are provided the file is created with those permissions
otherwise, defaults to 400.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
>
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205487#comment-15205487
]
Mike Yoder commented on HADOOP-12942:
-
{quote}
We could:
{quote}
This is becoming bigger than the intended scope of this jira. :-)
{quote}
Add a command that provisions an encrypted master secret to a well-known
location in HDFS
{quote}
We'd have to carefully think through what users would be able to perform this
action. And if something like this could be automated instead. And where that
"well-known location" might be - could it be configured (I think we'd have to).
And what about recursion issues if that location was inside an Encryption Zone?
{quote}
Obviously, this approach would require KMS to be in use and a new manual step
to provision a master secret.
{quote}
I think what you propose is workable, but these new requirements do concern me.
We'd also have to think through what users could perform this action (for this
action and for making the key in the KMS). There are lot of moving parts. Seems
like a case for a credential server (or credential server functionality in the
KMS).
Back to the issue in this jira - regardless of the difficulty of handling the
credential store password throughout the entire workflow, I still believe that
the credential shell should ask for that password. It's got to be better than
silently using "none" everywhere. And given that the key store provider has the
ability to get the password from a file, it seems like it would be possible to
put the password into a file for basically all use cases.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200686#comment-15200686
]
Larry McCay commented on HADOOP-12942:
--
For what it's worth, I considered an approach that would require having an
encrypted master secret file and that all keystores be protected with the
password that is encrypted in that file. But alas, that gets us back to the
question of how we protect the key/secret used to encrypt the master? I think
that as long as we are file based that we are using file permissions as the
main protection.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202400#comment-15202400
]
Larry McCay commented on HADOOP-12942:
--
The other thing to keep in mind is that the current situation is still better
than clear text files.
* like clear text files the main protection is file permissions
* unlike clear text files - given a file permission breach, there are no
standard tools that can get to the password value in the keystore
Even with the password to the keystore, keytool can not get to the value and
display it.
You could certainly get to it using the credential provider API directly but
this is better than clear text files that are solely protected with file
permissions.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200639#comment-15200639
]
Mike Yoder commented on HADOOP-12942:
-
Need some advice with this one [~lmccay]. I'm going to attempt a patch for this.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202393#comment-15202393
]
Larry McCay commented on HADOOP-12942:
--
Yes, exactly.
We can certainly document this aspect more clearly.
There is a Credential API page in the docs already that will be published with
2.8 - I will take a look and see how much I said about that aspect.
Something that has occurred to me is that we could possibly leverage the KMS
for the "master" secret idea.
We could:
* add a command that provisions an encrypted master secret to a well-known
location in HDFS
* add code in the credential provider factory that acquires the key from KMS
and decrypts the password from the master file
* If the master secret can be found and encrypted then that can be used for the
keystore password - if not, it falls back to "none" with a warning
* the credential provider factory would then also be used within the credential
provider API runtime use and would do the same thing
We would have to think through possible recursive issues with requiring access
to HDFS in order to get credentials from keystores in HDFS. The fact that the
master is in a file rather than a keystore may eliminate that problem though.
Obviously, this approach would require KMS to be in use and a new manual step
to provision a master secret.
It may be slightly odd that this is all just for the keystore based providers
and wouldn't be needed for a credential server based solution but I think that
can be justified.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202320#comment-15202320
]
Mike Yoder commented on HADOOP-12942:
-
{quote}
Otherwise, we just keep moving the problem.
{quote}
Oh, I agree. It's turtles all the way down. And you're right - as part of this
work I'm looking at where this is used (in the use case we saw, at least) and
how we can protect the password. I'm not sure we will be able to solve that
problem, though.
{quote}
more or less obfuscated
{quote}
I don't know if encrypting with the same hardcoded password meets the level of
even "obfuscation". :-) Of course, you could probably direct the same charge
against using a password that's easy to find.
{quote}
I would love it if you have an idea for something else.
{quote}
Yeah, me too.
I think that one of the problems I want to call out here is that the command,
as is, gives the user a false sense of security. Since there's no way to
obviously specify the credential provider password, it's easy for the user to
believe that whatever is going on behind the scenes is secure, because hey we
must know what we're doing. If our position is that the security of that jceks
file is no better than that of a plaintext file then I think we've done the
user a disservice.
I mean, let's imagine that the command outputted a warning saying "hey, that
provider you just used encrypted the file with a hardcoded default password".
Of course that will prompt the user to not be happy and demand a patch or
something. But at least we'd be up front about the issue. :-)
Better, I think, to do the right thing from the perspective of this command,
and then work on making the later consumers of the provider do "something". But
you're right, we have to think hard about end to end security with the
password. I don't know if we will have a really good answer, though.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12942) hadoop credential commands non-obviously use password of "none"
[
https://issues.apache.org/jira/browse/HADOOP-12942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200657#comment-15200657
]
Larry McCay commented on HADOOP-12942:
--
The problem is that the password to protect the store needs to be protected
somehow as well.
The only real fix to this issue that I can envision is to have a credential
server that we authenticate to rather than have a password like this.
Otherwise, we just keep moving the problem.
The environment variable doesn't really work either except for when both sides
of the equation - the provisioner and the consumer of the secret - can set the
environment variable without it being exposed in a script, in a file or
provided on the command line and thus available from things like ps. For
instance, we provision a password using the CLI or programmatically with the
API and then it needs to be acquired by a MR job. How does the environment
variable get set for the MR job? The idea was that setting up the job would
involve collecting known passwords and moving them into the User provider.
Again, the code that is setting up the job will need to have the env
variable
Until we have a secure credential server, the main protections with the
keystore credential files are still file permissions. The keystore must be set
with appropriate file permissions. In addition, it is more or less obfuscated
with the encryption of the keystore and the default password.
I would love it if you have an idea for something else.
> hadoop credential commands non-obviously use password of "none"
> ---
>
> Key: HADOOP-12942
> URL: https://issues.apache.org/jira/browse/HADOOP-12942
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Mike Yoder
>
> The "hadoop credential create" command, when using a jceks provider, defaults
> to using the value of "none" for the password that protects the jceks file.
> This is not obvious in the command or in documentation - to users or to other
> hadoop developers - and leads to jceks files that essentially are not
> protected.
> In this example, I'm adding a credential entry with name of "foo" and a value
> specified by the password entered:
> {noformat}
> # hadoop credential create foo -provider localjceks://file/bar.jceks
> Enter password:
> Enter password again:
> foo has been successfully created.
> org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider has been updated.
> {noformat}
> However, the password that protects the file bar.jceks is "none", and there
> is no obvious way to change that. The practical way of supplying the password
> at this time is something akin to
> {noformat}
> HADOOP_CREDSTORE_PASSWORD=credpass hadoop credential create --provider ...
> {noformat}
> That is, stuffing HADOOP_CREDSTORE_PASSWORD into the environment of the
> command.
> This is more than a documentation issue. I believe that the password ought to
> be _required_. We have three implementations at this point, the two
> JavaKeystore ones and the UserCredential. The latter is "transient" which
> does not make sense to use in this context. The former need some sort of
> password, and it's relatively easy to envision that any non-transient
> implementation would need a mechanism by which to protect the store that it's
> creating.
> The implementation gets interesting because the password in the
> AbstractJavaKeyStoreProvider is determined in the constructor, and changing
> it after the fact would get messy. So this probably means that the
> CredentialProviderFactory should have another factory method like the first
> that additionally takes the password, and an additional constructor exist in
> all the implementations that takes the password.
> Then we just ask for the password in getCredentialProvider() and that gets
> passed down to via the factory to the implementation. The code does have
> logic in the factory to try multiple providers, but I don't really see how
> multiple providers would be rationaly be used in the command shell context.
> This issue was brought to light when a user stored credentials for a Sqoop
> action in Oozie; upon trying to figure out where the password was coming from
> we discovered it to be the default value of "none".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
