[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583217#comment-17583217 ] ASF GitHub Bot commented on HDFS-4043: -- jojochuang merged PR #4785: URL: https://github.com/apache/hadoop/pull/4785 > Namenode Kerberos Login does not use proper hostname for host qualified hdfs > principal name. > > > Key: HDFS-4043 > URL: https://issues.apache.org/jira/browse/HDFS-4043 > Project: Hadoop HDFS > Issue Type: Bug > Components: security >Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha, > 3.4.0, 3.3.9 > Environment: CDH4U1 on Ubuntu 12.04 >Reporter: Ahad Rana >Assignee: Steve Vaughan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > Original Estimate: 24h > Time Spent: 50m > Remaining Estimate: 23h 10m > > The Namenode uses the loginAsNameNodeUser method in NameNode.java to login > using the hdfs principal. This method in turn invokes SecurityUtil.login with > a hostname (last parameter) obtained via a call to InetAddress.getHostName. > This call does not always return the fully qualified host name, and thus > causes the namenode to login to fail due to kerberos's inability to find a > matching hdfs principal in the hdfs.keytab file. Instead it should use > InetAddress.getCanonicalHostName. This is consistent with what is used > internally by SecurityUtil.java to login in other services, such as the > DataNode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583169#comment-17583169
]
ASF GitHub Bot commented on HDFS-4043:
--
hadoop-yetus commented on PR #4785:
URL: https://github.com/apache/hadoop/pull/4785#issuecomment-1222856319
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 11m 43s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 35s | | branch-3.3 passed |
| +1 :green_heart: | compile | 18m 58s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 1m 15s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 1m 52s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 1m 9s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 2m 59s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 28m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 4s | | the patch passed |
| +1 :green_heart: | compile | 18m 20s | | the patch passed |
| +1 :green_heart: | javac | 18m 20s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 10s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4785/1/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 2 new + 93
unchanged - 0 fixed = 95 total (was 93) |
| +1 :green_heart: | mvnsite | 1m 53s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed |
| +1 :green_heart: | spotbugs | 3m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 28m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 49s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 16s | | The patch does not
generate ASF License warnings. |
| | | 179m 52s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4785/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4785 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 97116f2d2c6a 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / a099a30a84f608aa0e06a50a64e6c4be577c61fe |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~18.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4785/1/testReport/ |
| Max. process+thread count | 2868 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4785/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Namenode Kerberos Login does not use proper hostname for host qualified hdfs
> principal name.
>
>
> Key: HDFS-4043
> URL: https://issues.apache.org/jira/browse/HDFS-4043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha,
> 3.4.0, 3.3.9
> Environment: CDH4U1 on Ubuntu 12.04
>Reporter: Ahad Rana
>Assignee: Steve Vaughan
>Priority: Major
>
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583086#comment-17583086 ] ASF GitHub Bot commented on HDFS-4043: -- snmvaughan opened a new pull request, #4785: URL: https://github.com/apache/hadoop/pull/4785 Backport of the changes from trunk. Use the existing DomainNameResolver to leverage the pluggable resolution framework. This provides a means to perform a reverse lookup if needed. Update default implementation of DNSDomainNameResolver to protect against returning the IP address as a string from a cached value. - [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Namenode Kerberos Login does not use proper hostname for host qualified hdfs > principal name. > > > Key: HDFS-4043 > URL: https://issues.apache.org/jira/browse/HDFS-4043 > Project: Hadoop HDFS > Issue Type: Bug > Components: security >Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha, > 3.4.0, 3.3.9 > Environment: CDH4U1 on Ubuntu 12.04 >Reporter: Ahad Rana >Assignee: Steve Vaughan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > Original Estimate: 24h > Time Spent: 50m > Remaining Estimate: 23h 10m > > The Namenode uses the loginAsNameNodeUser method in NameNode.java to login > using the hdfs principal. This method in turn invokes SecurityUtil.login with > a hostname (last parameter) obtained via a call to InetAddress.getHostName. > This call does not always return the fully qualified host name, and thus > causes the namenode to login to fail due to kerberos's inability to find a > matching hdfs principal in the hdfs.keytab file. Instead it should use > InetAddress.getCanonicalHostName. This is consistent with what is used > internally by SecurityUtil.java to login in other services, such as the > DataNode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580953#comment-17580953 ] ASF GitHub Bot commented on HDFS-4043: -- sunchao merged PR #4693: URL: https://github.com/apache/hadoop/pull/4693 > Namenode Kerberos Login does not use proper hostname for host qualified hdfs > principal name. > > > Key: HDFS-4043 > URL: https://issues.apache.org/jira/browse/HDFS-4043 > Project: Hadoop HDFS > Issue Type: Bug > Components: security >Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha, > 3.4.0, 3.3.9 > Environment: CDH4U1 on Ubuntu 12.04 >Reporter: Ahad Rana >Assignee: Steve Vaughan >Priority: Major > Labels: pull-request-available > Original Estimate: 24h > Time Spent: 50m > Remaining Estimate: 23h 10m > > The Namenode uses the loginAsNameNodeUser method in NameNode.java to login > using the hdfs principal. This method in turn invokes SecurityUtil.login with > a hostname (last parameter) obtained via a call to InetAddress.getHostName. > This call does not always return the fully qualified host name, and thus > causes the namenode to login to fail due to kerberos's inability to find a > matching hdfs principal in the hdfs.keytab file. Instead it should use > InetAddress.getCanonicalHostName. This is consistent with what is used > internally by SecurityUtil.java to login in other services, such as the > DataNode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580954#comment-17580954 ] ASF GitHub Bot commented on HDFS-4043: -- sunchao commented on PR #4693: URL: https://github.com/apache/hadoop/pull/4693#issuecomment-1218386748 Thanks, merged to trunk > Namenode Kerberos Login does not use proper hostname for host qualified hdfs > principal name. > > > Key: HDFS-4043 > URL: https://issues.apache.org/jira/browse/HDFS-4043 > Project: Hadoop HDFS > Issue Type: Bug > Components: security >Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha, > 3.4.0, 3.3.9 > Environment: CDH4U1 on Ubuntu 12.04 >Reporter: Ahad Rana >Assignee: Steve Vaughan >Priority: Major > Labels: pull-request-available > Original Estimate: 24h > Time Spent: 50m > Remaining Estimate: 23h 10m > > The Namenode uses the loginAsNameNodeUser method in NameNode.java to login > using the hdfs principal. This method in turn invokes SecurityUtil.login with > a hostname (last parameter) obtained via a call to InetAddress.getHostName. > This call does not always return the fully qualified host name, and thus > causes the namenode to login to fail due to kerberos's inability to find a > matching hdfs principal in the hdfs.keytab file. Instead it should use > InetAddress.getCanonicalHostName. This is consistent with what is used > internally by SecurityUtil.java to login in other services, such as the > DataNode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579894#comment-17579894
]
ASF GitHub Bot commented on HDFS-4043:
--
hadoop-yetus commented on PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#issuecomment-1215750317
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 27s | | trunk passed |
| +1 :green_heart: | compile | 26m 42s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 23m 59s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 58s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 31s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 27m 5s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 27m 32s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 6s | | the patch passed |
| +1 :green_heart: | compile | 24m 32s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 24m 32s | | the patch passed |
| +1 :green_heart: | compile | 21m 53s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 53s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 25s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/8/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 2 new + 92
unchanged - 0 fixed = 94 total (was 92) |
| +1 :green_heart: | mvnsite | 1m 58s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 23s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 5s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 32s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 22s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 17s | | The patch does not
generate ASF License warnings. |
| | | 233m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4693 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux bb8f8c1c3dfe 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 31572d6318a5d14e01787298a09f6b84d57b1f9f |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579761#comment-17579761
]
ASF GitHub Bot commented on HDFS-4043:
--
snmvaughan commented on code in PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#discussion_r945856788
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/DNSDomainNameResolver.java:
##
@@ -40,6 +48,16 @@ public String getHostnameByIP(InetAddress address) {
&& host.charAt(host.length()-1) == '.') {
host = host.substring(0, host.length()-1);
}
+// Protect against the Java behaviour of returning the IP address as a
string from a cache
+// instead of performing a reverse lookup.
+if (host.equals(address.getHostAddress())) {
Review Comment:
It either returns what it figured out, or returns the IP address as a
string. There is no way of knowing when it is failing to provide an actual
FQDN, which is why there is the additional check to see if it is just the IP
address.
> Namenode Kerberos Login does not use proper hostname for host qualified hdfs
> principal name.
>
>
> Key: HDFS-4043
> URL: https://issues.apache.org/jira/browse/HDFS-4043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha,
> 3.4.0, 3.3.9
> Environment: CDH4U1 on Ubuntu 12.04
>Reporter: Ahad Rana
>Priority: Major
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 50m
> Remaining Estimate: 23h 10m
>
> The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
> using the hdfs principal. This method in turn invokes SecurityUtil.login with
> a hostname (last parameter) obtained via a call to InetAddress.getHostName.
> This call does not always return the fully qualified host name, and thus
> causes the namenode to login to fail due to kerberos's inability to find a
> matching hdfs principal in the hdfs.keytab file. Instead it should use
> InetAddress.getCanonicalHostName. This is consistent with what is used
> internally by SecurityUtil.java to login in other services, such as the
> DataNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579760#comment-17579760
]
ASF GitHub Bot commented on HDFS-4043:
--
ndimiduk commented on code in PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#discussion_r945854695
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/DNSDomainNameResolver.java:
##
@@ -40,6 +48,16 @@ public String getHostnameByIP(InetAddress address) {
&& host.charAt(host.length()-1) == '.') {
host = host.substring(0, host.length()-1);
}
+// Protect against the Java behaviour of returning the IP address as a
string from a cache
+// instead of performing a reverse lookup.
+if (host.equals(address.getHostAddress())) {
Review Comment:
In fact, it should never be null according to the docs on
`getCanonicalHostName`.
> Namenode Kerberos Login does not use proper hostname for host qualified hdfs
> principal name.
>
>
> Key: HDFS-4043
> URL: https://issues.apache.org/jira/browse/HDFS-4043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha,
> 3.4.0, 3.3.9
> Environment: CDH4U1 on Ubuntu 12.04
>Reporter: Ahad Rana
>Priority: Major
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 50m
> Remaining Estimate: 23h 10m
>
> The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
> using the hdfs principal. This method in turn invokes SecurityUtil.login with
> a hostname (last parameter) obtained via a call to InetAddress.getHostName.
> This call does not always return the fully qualified host name, and thus
> causes the namenode to login to fail due to kerberos's inability to find a
> matching hdfs principal in the hdfs.keytab file. Instead it should use
> InetAddress.getCanonicalHostName. This is consistent with what is used
> internally by SecurityUtil.java to login in other services, such as the
> DataNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579759#comment-17579759
]
ASF GitHub Bot commented on HDFS-4043:
--
ndimiduk commented on code in PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#discussion_r945852367
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/DNSDomainNameResolver.java:
##
@@ -40,6 +48,16 @@ public String getHostnameByIP(InetAddress address) {
&& host.charAt(host.length()-1) == '.') {
host = host.substring(0, host.length()-1);
}
+// Protect against the Java behaviour of returning the IP address as a
string from a cache
+// instead of performing a reverse lookup.
+if (host.equals(address.getHostAddress())) {
Review Comment:
What's the expected behavior here? does the called expect us to throw or is
it acceptable to return `null` ?
> Namenode Kerberos Login does not use proper hostname for host qualified hdfs
> principal name.
>
>
> Key: HDFS-4043
> URL: https://issues.apache.org/jira/browse/HDFS-4043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha,
> 3.4.0, 3.3.9
> Environment: CDH4U1 on Ubuntu 12.04
>Reporter: Ahad Rana
>Priority: Major
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 50m
> Remaining Estimate: 23h 10m
>
> The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
> using the hdfs principal. This method in turn invokes SecurityUtil.login with
> a hostname (last parameter) obtained via a call to InetAddress.getHostName.
> This call does not always return the fully qualified host name, and thus
> causes the namenode to login to fail due to kerberos's inability to find a
> matching hdfs principal in the hdfs.keytab file. Instead it should use
> InetAddress.getCanonicalHostName. This is consistent with what is used
> internally by SecurityUtil.java to login in other services, such as the
> DataNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579195#comment-17579195
]
ASF GitHub Bot commented on HDFS-4043:
--
hadoop-yetus commented on PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#issuecomment-1213619876
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 29s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 12s | | trunk passed |
| +1 :green_heart: | compile | 28m 31s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 23m 42s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 10s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 5s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 34s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 27m 0s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 9s | | the patch passed |
| +1 :green_heart: | compile | 24m 39s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 24m 39s | | the patch passed |
| +1 :green_heart: | compile | 22m 7s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 22m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 31s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/7/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 2 new + 92
unchanged - 0 fixed = 94 total (was 92) |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 23s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 7s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | spotbugs | 3m 7s |
[/new-spotbugs-hadoop-common-project_hadoop-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/7/artifact/out/new-spotbugs-hadoop-common-project_hadoop-common.html)
| hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | shadedclient | 25m 56s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 21s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 18s | | The patch does not
generate ASF License warnings. |
| | | 237m 46s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-common-project/hadoop-common |
| | Possible null pointer dereference of host in
org.apache.hadoop.net.DNSDomainNameResolver.getHostnameByIP(InetAddress)
Dereferenced at DNSDomainNameResolver.java:host in
org.apache.hadoop.net.DNSDomainNameResolver.getHostnameByIP(InetAddress)
Dereferenced at DNSDomainNameResolver.java:[line 53] |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4693 |
| Optional
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579175#comment-17579175
]
ASF GitHub Bot commented on HDFS-4043:
--
hadoop-yetus commented on PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#issuecomment-1213576016
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 9s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 45s | | trunk passed |
| +1 :green_heart: | compile | 25m 25s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 22m 16s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 1s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 33s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 6s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 22s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 26m 48s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 8s | | the patch passed |
| +1 :green_heart: | compile | 26m 55s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 26m 56s | | the patch passed |
| +1 :green_heart: | compile | 23m 40s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 23m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 37s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/6/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 2 new + 92
unchanged - 0 fixed = 94 total (was 92) |
| +1 :green_heart: | mvnsite | 2m 54s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 9s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | spotbugs | 3m 34s |
[/new-spotbugs-hadoop-common-project_hadoop-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/6/artifact/out/new-spotbugs-hadoop-common-project_hadoop-common.html)
| hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | shadedclient | 28m 50s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 44s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 17s | | The patch does not
generate ASF License warnings. |
| | | 237m 45s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-common-project/hadoop-common |
| | Possible null pointer dereference of host in
org.apache.hadoop.net.DNSDomainNameResolver.getHostnameByIP(InetAddress)
Dereferenced at DNSDomainNameResolver.java:host in
org.apache.hadoop.net.DNSDomainNameResolver.getHostnameByIP(InetAddress)
Dereferenced at DNSDomainNameResolver.java:[line 53] |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4693 |
| Optional
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579159#comment-17579159
]
ASF GitHub Bot commented on HDFS-4043:
--
sunchao commented on code in PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#discussion_r944892561
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/DNSDomainNameResolver.java:
##
@@ -40,6 +48,16 @@ public String getHostnameByIP(InetAddress address) {
&& host.charAt(host.length()-1) == '.') {
host = host.substring(0, host.length()-1);
}
+// Protect against the Java behaviour of returning the IP address as a
string from a cache
+// instead of performing a reverse lookup.
+if (host.equals(address.getHostAddress())) {
Review Comment:
should we check that the `host` is not null? I saw we check that at line 47.
> Namenode Kerberos Login does not use proper hostname for host qualified hdfs
> principal name.
>
>
> Key: HDFS-4043
> URL: https://issues.apache.org/jira/browse/HDFS-4043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha,
> 3.4.0, 3.3.9
> Environment: CDH4U1 on Ubuntu 12.04
>Reporter: Ahad Rana
>Priority: Major
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 50m
> Remaining Estimate: 23h 10m
>
> The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
> using the hdfs principal. This method in turn invokes SecurityUtil.login with
> a hostname (last parameter) obtained via a call to InetAddress.getHostName.
> This call does not always return the fully qualified host name, and thus
> causes the namenode to login to fail due to kerberos's inability to find a
> matching hdfs principal in the hdfs.keytab file. Instead it should use
> InetAddress.getCanonicalHostName. This is consistent with what is used
> internally by SecurityUtil.java to login in other services, such as the
> DataNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579133#comment-17579133
]
ASF GitHub Bot commented on HDFS-4043:
--
snmvaughan commented on code in PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#discussion_r944850020
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/DNSDomainNameResolver.java:
##
@@ -40,6 +48,16 @@ public String getHostnameByIP(InetAddress address) {
&& host.charAt(host.length()-1) == '.') {
host = host.substring(0, host.length()-1);
}
+// Protect against the Java behaviour of returning the IP address as a
string from a cache
+// instead of performing a reverse lookup.
+if (host.equals(address.getHostAddress())) {
+ LOG.debug("IP address returned for FQDN detected");
Review Comment:
I'll make that change.
> Namenode Kerberos Login does not use proper hostname for host qualified hdfs
> principal name.
>
>
> Key: HDFS-4043
> URL: https://issues.apache.org/jira/browse/HDFS-4043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha,
> 3.4.0, 3.3.9
> Environment: CDH4U1 on Ubuntu 12.04
>Reporter: Ahad Rana
>Priority: Major
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 50m
> Remaining Estimate: 23h 10m
>
> The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
> using the hdfs principal. This method in turn invokes SecurityUtil.login with
> a hostname (last parameter) obtained via a call to InetAddress.getHostName.
> This call does not always return the fully qualified host name, and thus
> causes the namenode to login to fail due to kerberos's inability to find a
> matching hdfs principal in the hdfs.keytab file. Instead it should use
> InetAddress.getCanonicalHostName. This is consistent with what is used
> internally by SecurityUtil.java to login in other services, such as the
> DataNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579132#comment-17579132 ] ASF GitHub Bot commented on HDFS-4043: -- snmvaughan commented on code in PR #4693: URL: https://github.com/apache/hadoop/pull/4693#discussion_r944849715 ## hadoop-common-project/hadoop-common/src/main/resources/core-default.xml: ## @@ -130,6 +130,14 @@ + + hadoop.security.resolver.impl + Review Comment: I agree that your suggestion sounds like a good idea. I followed the example of `dfs.namenode.edits.qjournals.resolver.impl` which doesn't provide the value. > Namenode Kerberos Login does not use proper hostname for host qualified hdfs > principal name. > > > Key: HDFS-4043 > URL: https://issues.apache.org/jira/browse/HDFS-4043 > Project: Hadoop HDFS > Issue Type: Bug > Components: security >Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha, > 3.4.0, 3.3.9 > Environment: CDH4U1 on Ubuntu 12.04 >Reporter: Ahad Rana >Priority: Major > Labels: pull-request-available > Original Estimate: 24h > Time Spent: 50m > Remaining Estimate: 23h 10m > > The Namenode uses the loginAsNameNodeUser method in NameNode.java to login > using the hdfs principal. This method in turn invokes SecurityUtil.login with > a hostname (last parameter) obtained via a call to InetAddress.getHostName. > This call does not always return the fully qualified host name, and thus > causes the namenode to login to fail due to kerberos's inability to find a > matching hdfs principal in the hdfs.keytab file. Instead it should use > InetAddress.getCanonicalHostName. This is consistent with what is used > internally by SecurityUtil.java to login in other services, such as the > DataNode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579129#comment-17579129
]
ASF GitHub Bot commented on HDFS-4043:
--
goiri commented on code in PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#discussion_r944820454
##
hadoop-common-project/hadoop-common/src/main/resources/core-default.xml:
##
@@ -130,6 +130,14 @@
+
+ hadoop.security.resolver.impl
+
Review Comment:
Should we specify the default just in case?
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/DNSDomainNameResolver.java:
##
@@ -40,6 +48,16 @@ public String getHostnameByIP(InetAddress address) {
&& host.charAt(host.length()-1) == '.') {
host = host.substring(0, host.length()-1);
}
+// Protect against the Java behaviour of returning the IP address as a
string from a cache
+// instead of performing a reverse lookup.
+if (host.equals(address.getHostAddress())) {
+ LOG.debug("IP address returned for FQDN detected");
Review Comment:
It would be good to log the ip
> Namenode Kerberos Login does not use proper hostname for host qualified hdfs
> principal name.
>
>
> Key: HDFS-4043
> URL: https://issues.apache.org/jira/browse/HDFS-4043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha,
> 3.4.0, 3.3.9
> Environment: CDH4U1 on Ubuntu 12.04
>Reporter: Ahad Rana
>Priority: Major
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 50m
> Remaining Estimate: 23h 10m
>
> The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
> using the hdfs principal. This method in turn invokes SecurityUtil.login with
> a hostname (last parameter) obtained via a call to InetAddress.getHostName.
> This call does not always return the fully qualified host name, and thus
> causes the namenode to login to fail due to kerberos's inability to find a
> matching hdfs principal in the hdfs.keytab file. Instead it should use
> InetAddress.getCanonicalHostName. This is consistent with what is used
> internally by SecurityUtil.java to login in other services, such as the
> DataNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579105#comment-17579105 ] ASF GitHub Bot commented on HDFS-4043: -- sunchao commented on PR #4693: URL: https://github.com/apache/hadoop/pull/4693#issuecomment-1213433652 cc @fengnanli @goiri since you authored the original code > Namenode Kerberos Login does not use proper hostname for host qualified hdfs > principal name. > > > Key: HDFS-4043 > URL: https://issues.apache.org/jira/browse/HDFS-4043 > Project: Hadoop HDFS > Issue Type: Bug > Components: security >Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha, > 3.4.0, 3.3.9 > Environment: CDH4U1 on Ubuntu 12.04 >Reporter: Ahad Rana >Priority: Major > Labels: pull-request-available > Original Estimate: 24h > Time Spent: 50m > Remaining Estimate: 23h 10m > > The Namenode uses the loginAsNameNodeUser method in NameNode.java to login > using the hdfs principal. This method in turn invokes SecurityUtil.login with > a hostname (last parameter) obtained via a call to InetAddress.getHostName. > This call does not always return the fully qualified host name, and thus > causes the namenode to login to fail due to kerberos's inability to find a > matching hdfs principal in the hdfs.keytab file. Instead it should use > InetAddress.getCanonicalHostName. This is consistent with what is used > internally by SecurityUtil.java to login in other services, such as the > DataNode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579075#comment-17579075 ] ASF GitHub Bot commented on HDFS-4043: -- hadoop-yetus commented on PR #4693: URL: https://github.com/apache/hadoop/pull/4693#issuecomment-1213379155 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 0s | | Docker mode activated. | | -1 :x: | patch | 0m 31s | | https://github.com/apache/hadoop/pull/4693 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/4693 | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/5/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org | This message was automatically generated. > Namenode Kerberos Login does not use proper hostname for host qualified hdfs > principal name. > > > Key: HDFS-4043 > URL: https://issues.apache.org/jira/browse/HDFS-4043 > Project: Hadoop HDFS > Issue Type: Bug > Components: security >Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha, > 3.4.0, 3.3.9 > Environment: CDH4U1 on Ubuntu 12.04 >Reporter: Ahad Rana >Priority: Major > Labels: pull-request-available > Original Estimate: 24h > Time Spent: 50m > Remaining Estimate: 23h 10m > > The Namenode uses the loginAsNameNodeUser method in NameNode.java to login > using the hdfs principal. This method in turn invokes SecurityUtil.login with > a hostname (last parameter) obtained via a call to InetAddress.getHostName. > This call does not always return the fully qualified host name, and thus > causes the namenode to login to fail due to kerberos's inability to find a > matching hdfs principal in the hdfs.keytab file. Instead it should use > InetAddress.getCanonicalHostName. This is consistent with what is used > internally by SecurityUtil.java to login in other services, such as the > DataNode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578987#comment-17578987
]
ASF GitHub Bot commented on HDFS-4043:
--
ndimiduk commented on code in PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#discussion_r944499525
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestInetAddressUtils.java:
##
@@ -0,0 +1,48 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.net;
+
+import org.junit.Test;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotEquals;
+
+
+public class TestInetAddressUtils {
+
+ @Test
+ public void testGetCanonicalHostName() throws UnknownHostException {
+InetAddress localhost = InetAddress.getLocalHost();
+InetAddress unresolved =
InetAddress.getByAddress(localhost.getHostAddress(),
+localhost.getAddress());
+
+// Precondition: host name and canonical host name for unresolved returns
an IP address.
+assertEquals(localhost.getHostAddress(), unresolved.getHostName());
Review Comment:
Hmm yes, I see no mention of `jdk.net.hosts.file` in the `InetAddress` of
OpenJDK8.
> Namenode Kerberos Login does not use proper hostname for host qualified hdfs
> principal name.
>
>
> Key: HDFS-4043
> URL: https://issues.apache.org/jira/browse/HDFS-4043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha
> Environment: CDH4U1 on Ubuntu 12.04
>Reporter: Ahad Rana
>Priority: Major
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 50m
> Remaining Estimate: 23h 10m
>
> The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
> using the hdfs principal. This method in turn invokes SecurityUtil.login with
> a hostname (last parameter) obtained via a call to InetAddress.getHostName.
> This call does not always return the fully qualified host name, and thus
> causes the namenode to login to fail due to kerberos's inability to find a
> matching hdfs principal in the hdfs.keytab file. Instead it should use
> InetAddress.getCanonicalHostName. This is consistent with what is used
> internally by SecurityUtil.java to login in other services, such as the
> DataNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577053#comment-17577053
]
ASF GitHub Bot commented on HDFS-4043:
--
hadoop-yetus commented on PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#issuecomment-1208748137
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 8s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 20s | | trunk passed |
| +1 :green_heart: | compile | 25m 25s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 22m 4s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 1s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 26s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 6s | | the patch passed |
| +1 :green_heart: | compile | 24m 35s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 24m 35s | | the patch passed |
| +1 :green_heart: | compile | 22m 0s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 22m 0s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 26s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 57s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 4s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 57s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 17s | | The patch does not
generate ASF License warnings. |
| | | 229m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4693 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 87ec85eb51bd 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 86ffe289cb0a0d6f0e2b5fcf83f760be5655fa38 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/4/testReport/ |
| Max. process+thread count | 1253 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4693/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
>
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576942#comment-17576942
]
ASF GitHub Bot commented on HDFS-4043:
--
snmvaughan commented on code in PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#discussion_r940569507
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestInetAddressUtils.java:
##
@@ -0,0 +1,48 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.net;
+
+import org.junit.Test;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotEquals;
+
+
+public class TestInetAddressUtils {
+
+ @Test
+ public void testGetCanonicalHostName() throws UnknownHostException {
+InetAddress localhost = InetAddress.getLocalHost();
+InetAddress unresolved =
InetAddress.getByAddress(localhost.getHostAddress(),
+localhost.getAddress());
+
+// Precondition: host name and canonical host name for unresolved returns
an IP address.
+assertEquals(localhost.getHostAddress(), unresolved.getHostName());
Review Comment:
We're still support Java 8, so I avoided using Java 11 classes.
> Namenode Kerberos Login does not use proper hostname for host qualified hdfs
> principal name.
>
>
> Key: HDFS-4043
> URL: https://issues.apache.org/jira/browse/HDFS-4043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha
> Environment: CDH4U1 on Ubuntu 12.04
>Reporter: Ahad Rana
>Priority: Major
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 50m
> Remaining Estimate: 23h 10m
>
> The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
> using the hdfs principal. This method in turn invokes SecurityUtil.login with
> a hostname (last parameter) obtained via a call to InetAddress.getHostName.
> This call does not always return the fully qualified host name, and thus
> causes the namenode to login to fail due to kerberos's inability to find a
> matching hdfs principal in the hdfs.keytab file. Instead it should use
> InetAddress.getCanonicalHostName. This is consistent with what is used
> internally by SecurityUtil.java to login in other services, such as the
> DataNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576691#comment-17576691
]
ASF GitHub Bot commented on HDFS-4043:
--
ndimiduk commented on code in PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#discussion_r940079026
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestInetAddressUtils.java:
##
@@ -0,0 +1,48 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.net;
+
+import org.junit.Test;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotEquals;
+
+
+public class TestInetAddressUtils {
+
+ @Test
+ public void testGetCanonicalHostName() throws UnknownHostException {
+InetAddress localhost = InetAddress.getLocalHost();
+InetAddress unresolved =
InetAddress.getByAddress(localhost.getHostAddress(),
+localhost.getAddress());
+
+// Precondition: host name and canonical host name for unresolved returns
an IP address.
+assertEquals(localhost.getHostAddress(), unresolved.getHostName());
Review Comment:
I should specify. At least in OpenJDK11, in the `InetAddress` class, there's
a `private static NameService createNameService()` that makes use of this
property. I don't know if this is formally documented someplace on the JVM.
> Namenode Kerberos Login does not use proper hostname for host qualified hdfs
> principal name.
>
>
> Key: HDFS-4043
> URL: https://issues.apache.org/jira/browse/HDFS-4043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha
> Environment: CDH4U1 on Ubuntu 12.04
>Reporter: Ahad Rana
>Priority: Major
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 50m
> Remaining Estimate: 23h 10m
>
> The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
> using the hdfs principal. This method in turn invokes SecurityUtil.login with
> a hostname (last parameter) obtained via a call to InetAddress.getHostName.
> This call does not always return the fully qualified host name, and thus
> causes the namenode to login to fail due to kerberos's inability to find a
> matching hdfs principal in the hdfs.keytab file. Instead it should use
> InetAddress.getCanonicalHostName. This is consistent with what is used
> internally by SecurityUtil.java to login in other services, such as the
> DataNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576672#comment-17576672
]
ASF GitHub Bot commented on HDFS-4043:
--
ndimiduk commented on code in PR #4693:
URL: https://github.com/apache/hadoop/pull/4693#discussion_r940043984
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/InetAddressUtils.java:
##
@@ -0,0 +1,55 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.net;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.naming.NamingException;
+import java.net.InetAddress;
+
+public final class InetAddressUtils {
Review Comment:
I believe that you need InterfaceAudience and InterfaceStability annotations
on the new class.
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestInetAddressUtils.java:
##
@@ -0,0 +1,48 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.net;
+
+import org.junit.Test;
+
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotEquals;
+
+
+public class TestInetAddressUtils {
+
+ @Test
+ public void testGetCanonicalHostName() throws UnknownHostException {
+InetAddress localhost = InetAddress.getLocalHost();
+InetAddress unresolved =
InetAddress.getByAddress(localhost.getHostAddress(),
+localhost.getAddress());
+
+// Precondition: host name and canonical host name for unresolved returns
an IP address.
+assertEquals(localhost.getHostAddress(), unresolved.getHostName());
Review Comment:
The behavior of this test will vary based on the dns environment at the time
it's run. At the very least, you can make this an `assume` statement so that
the test will make no assertions when the environmental conditions are
inappropriate.
For more rigorous testing, it seems like mocking of a static method becomes
possible as of Mockito 3.4.0. Alternatively, I wonder if you can create an
environment where you manipulate the property `jdk.net.hosts.file` and provide
a file that you populate for the duration of the test.
> Namenode Kerberos Login does not use proper hostname for host qualified hdfs
> principal name.
>
>
> Key: HDFS-4043
> URL: https://issues.apache.org/jira/browse/HDFS-4043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha
> Environment: CDH4U1 on Ubuntu 12.04
>Reporter: Ahad Rana
>Priority: Major
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 50m
> Remaining Estimate: 23h 10m
>
> The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
> using the hdfs principal. This method in turn invokes SecurityUtil.login with
> a hostname (last parameter) obtained via a call to InetAddress.getHostName.
> This call does not always return the fully qualified host name, and thus
> causes the namenode to login to fail due to kerberos's inability to find a
> matching hdfs principal in the hdfs.keytab
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17570928#comment-17570928 ] Steve Vaughan commented on HDFS-4043: - I've been testing a fix that detects when getCanonicalHostName() returns the IP address as a string, and then performs a DNS reverse name lookup to fix the issue. > Namenode Kerberos Login does not use proper hostname for host qualified hdfs > principal name. > > > Key: HDFS-4043 > URL: https://issues.apache.org/jira/browse/HDFS-4043 > Project: Hadoop HDFS > Issue Type: Bug > Components: security >Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha > Environment: CDH4U1 on Ubuntu 12.04 >Reporter: Ahad Rana >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > The Namenode uses the loginAsNameNodeUser method in NameNode.java to login > using the hdfs principal. This method in turn invokes SecurityUtil.login with > a hostname (last parameter) obtained via a call to InetAddress.getHostName. > This call does not always return the fully qualified host name, and thus > causes the namenode to login to fail due to kerberos's inability to find a > matching hdfs principal in the hdfs.keytab file. Instead it should use > InetAddress.getCanonicalHostName. This is consistent with what is used > internally by SecurityUtil.java to login in other services, such as the > DataNode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13479668#comment-13479668 ] Ahad Rana commented on HDFS-4043: - Hi Brahma, Please disregard my last suggestion. Setting dfs.namenode.kerberos.principal or dfs.namenode.kerberos.internal.spnego.principal to and explicit principal name (instead of a pattern name with _HOST in it) triggers other bugs (see HDFS-4081). The bottom line is that it is probably best to set the hostname of the namenode to match exactly the name returned via a reverse-dns query (getCanonicalName). You are right however, that your problems are a manifestation of the same general bug (inconsistent resolution of canonical principal name via different code paths). Most definitely, incoming IP based connections need to use getCanonicalName to get back a host name that can be used to form the proper principal name. Otherwise you will need to probably go with IP based principal names ? As mentioned above, I have reverted to setting the internal hostname for the namenodes/secondary namenodes to exactly match the fully qualified hostname returned via reverse-dns. And so far, things seems to be working properly now. Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name. Key: HDFS-4043 URL: https://issues.apache.org/jira/browse/HDFS-4043 Project: Hadoop HDFS Issue Type: Bug Components: security Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha Environment: CDH4U1 on Ubuntu 12.04 Reporter: Ahad Rana Original Estimate: 24h Remaining Estimate: 24h The Namenode uses the loginAsNameNodeUser method in NameNode.java to login using the hdfs principal. This method in turn invokes SecurityUtil.login with a hostname (last parameter) obtained via a call to InetAddress.getHostName. This call does not always return the fully qualified host name, and thus causes the namenode to login to fail due to kerberos's inability to find a matching hdfs principal in the hdfs.keytab file. Instead it should use InetAddress.getCanonicalHostName. This is consistent with what is used internally by SecurityUtil.java to login in other services, such as the DataNode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13479954#comment-13479954
]
Brahma Reddy Battula commented on HDFS-4043:
[~ahadr]
Let's go ahead and close this JIRA.
{quote}
You are right however, that your problems are a manifestation of the same
general bug (inconsistent resolution of canonical principal name via different
code paths). Most definitely, incoming IP based connections need to use
getCanonicalName to get back a host name that can be used to form the proper
principal name. Otherwise you will need to probably go with IP based principal
names ?
{quote}
can we discuss this point in HDF-3980..?
Namenode Kerberos Login does not use proper hostname for host qualified hdfs
principal name.
Key: HDFS-4043
URL: https://issues.apache.org/jira/browse/HDFS-4043
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha
Environment: CDH4U1 on Ubuntu 12.04
Reporter: Ahad Rana
Original Estimate: 24h
Remaining Estimate: 24h
The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
using the hdfs principal. This method in turn invokes SecurityUtil.login with
a hostname (last parameter) obtained via a call to InetAddress.getHostName.
This call does not always return the fully qualified host name, and thus
causes the namenode to login to fail due to kerberos's inability to find a
matching hdfs principal in the hdfs.keytab file. Instead it should use
InetAddress.getCanonicalHostName. This is consistent with what is used
internally by SecurityUtil.java to login in other services, such as the
DataNode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478067#comment-13478067 ] Ahad Rana commented on HDFS-4043: - Hi, Can you explicitly set the namenode principal name (to the proper canonical name) in your hdfs-site via the following properties and see if this resolves your issues ? property namedfs.namenode.kerberos.principal/name valuehdfs/n01.prod.company@company.com/value /property property namedfs.namenode.kerberos.internal.spnego.principal/name valueHTTP/n01.prod.company@company.com/value /property Best, Ahad. On Sun, Oct 14, 2012 at 9:35 PM, Brahma Reddy Battula (JIRA) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name. Key: HDFS-4043 URL: https://issues.apache.org/jira/browse/HDFS-4043 Project: Hadoop HDFS Issue Type: Bug Components: security Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha Environment: CDH4U1 on Ubuntu 12.04 Reporter: Ahad Rana Original Estimate: 24h Remaining Estimate: 24h The Namenode uses the loginAsNameNodeUser method in NameNode.java to login using the hdfs principal. This method in turn invokes SecurityUtil.login with a hostname (last parameter) obtained via a call to InetAddress.getHostName. This call does not always return the fully qualified host name, and thus causes the namenode to login to fail due to kerberos's inability to find a matching hdfs principal in the hdfs.keytab file. Instead it should use InetAddress.getCanonicalHostName. This is consistent with what is used internally by SecurityUtil.java to login in other services, such as the DataNode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478670#comment-13478670
]
Brahma Reddy Battula commented on HDFS-4043:
HI Ahad thanks a lot for reply..
{quote}
property
namedfs.namenode.kerberos.principal/name
valuehdfs/n01.prod.company@company.com/value
/property
property
namedfs.namenode.kerberos.internal.spnego.principal/name
valueHTTP/n01.prod.company@company.com/value
/property
{quote}
two properties I had configured as
hdfs/hostname of mach...@hadoop.com(RELAM)
HTTP/hostname of mach...@hadoop.com
I added both principal's (hdfs/hostname of mach...@hadoop.com and HTTP/hostname
of mach...@hadoop.com) to KDC and generated keytab
but it's failing for HTTP/(IP of machine) which is not added in the KDC and
then ticket is failing like following while doing checkpoint..
Oct 04 03:07:43 host-***-168 krb5kdc[24598](info): TGS_REQ (6 etypes {3 1
23 16 17 18}) ***.168: ISSUE: authtime 1349300202, etypes {rep=23 tkt=18
ses=23}, hdfs/had...@hadoop.com for hdfs/had...@hadoop.com
Oct 04 03:07:43 host-***-168 krb5kdc[24598](info): TGS_REQ (6 etypes {3 1
23 16 17 18}) ***.168: ISSUE: authtime 1349300202, etypes {rep=23 tkt=18
ses=23}, hdfs/had...@hadoop.com for hdfs/had...@hadoop.com
Oct 04 03:07:43 host-***-168 krb5kdc[24598](info): TGS_REQ (6 etypes {3 1
23 16 17 18}) ***.168: ISSUE: authtime 1349300202, etypes {rep=23 tkt=18
ses=23}, hdfs/had...@hadoop.com for HTTP/***.1...@hadoop.com
Oct 04 03:07:43 host-***-168 krb5kdc[24598](info): TGS_REQ (6 etypes {3 1
23 16 17 18}) ***.168: ISSUE: authtime 1349300202, etypes {rep=23 tkt=18
ses=23}, hdfs/had...@hadoop.com for HTTP/***.1...@hadoop.com
Mostly your's and mine(HDP-3980) is same,I think...Since
KerberosAuthenticator.this*.url.getHost() is always retunring IP of the machine
and then principal coming like HTTP/(ip of the machine)..
Please correct me If I am wrong...
Namenode Kerberos Login does not use proper hostname for host qualified hdfs
principal name.
Key: HDFS-4043
URL: https://issues.apache.org/jira/browse/HDFS-4043
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha
Environment: CDH4U1 on Ubuntu 12.04
Reporter: Ahad Rana
Original Estimate: 24h
Remaining Estimate: 24h
The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
using the hdfs principal. This method in turn invokes SecurityUtil.login with
a hostname (last parameter) obtained via a call to InetAddress.getHostName.
This call does not always return the fully qualified host name, and thus
causes the namenode to login to fail due to kerberos's inability to find a
matching hdfs principal in the hdfs.keytab file. Instead it should use
InetAddress.getCanonicalHostName. This is consistent with what is used
internally by SecurityUtil.java to login in other services, such as the
DataNode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[
https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13475956#comment-13475956
]
Brahma Reddy Battula commented on HDFS-4043:
Hi Ahad,
thanks for reply..
{quote}
Have you verified that
the generated service principal name does not match the one stored in your
kdc's database ?
{quote}
generated principal is HTTP/hostname which is not added in the KDC(I mean not
stored in kdc database) and not configured..
Namenode Kerberos Login does not use proper hostname for host qualified hdfs
principal name.
Key: HDFS-4043
URL: https://issues.apache.org/jira/browse/HDFS-4043
Project: Hadoop HDFS
Issue Type: Bug
Components: security
Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha
Environment: CDH4U1 on Ubuntu 12.04
Reporter: Ahad Rana
Original Estimate: 24h
Remaining Estimate: 24h
The Namenode uses the loginAsNameNodeUser method in NameNode.java to login
using the hdfs principal. This method in turn invokes SecurityUtil.login with
a hostname (last parameter) obtained via a call to InetAddress.getHostName.
This call does not always return the fully qualified host name, and thus
causes the namenode to login to fail due to kerberos's inability to find a
matching hdfs principal in the hdfs.keytab file. Instead it should use
InetAddress.getCanonicalHostName. This is consistent with what is used
internally by SecurityUtil.java to login in other services, such as the
DataNode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13475717#comment-13475717 ] Ahad Rana commented on HDFS-4043: - Hi Brahma, Not sure if this is exactly the same bug. It could definitely be the case that the host name derived via KerberosAuthenticator.this*.url.getHost() *is not the Canonical Hostname used to generate the server principal, so it is definitely similar in nature to the bug I reported. Have you verified that the generated service principal name does not match the one stored in your kdc's database ? Ahad. On Fri, Oct 12, 2012 at 9:25 PM, Brahma Reddy Battula (JIRA) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name. Key: HDFS-4043 URL: https://issues.apache.org/jira/browse/HDFS-4043 Project: Hadoop HDFS Issue Type: Bug Components: security Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha Environment: CDH4U1 on Ubuntu 12.04 Reporter: Ahad Rana Original Estimate: 24h Remaining Estimate: 24h The Namenode uses the loginAsNameNodeUser method in NameNode.java to login using the hdfs principal. This method in turn invokes SecurityUtil.login with a hostname (last parameter) obtained via a call to InetAddress.getHostName. This call does not always return the fully qualified host name, and thus causes the namenode to login to fail due to kerberos's inability to find a matching hdfs principal in the hdfs.keytab file. Instead it should use InetAddress.getCanonicalHostName. This is consistent with what is used internally by SecurityUtil.java to login in other services, such as the DataNode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4043) Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name.
[ https://issues.apache.org/jira/browse/HDFS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13475524#comment-13475524 ] Brahma Reddy Battula commented on HDFS-4043: Hi Ahad Rana, I think,,this is same as HDFS-3980..Please refer following comment.. https://issues.apache.org/jira/browse/HDFS-3980?focusedCommentId=13469267page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13469267.. Can I duplicate this...? Please correct me If I am wrong.. Namenode Kerberos Login does not use proper hostname for host qualified hdfs principal name. Key: HDFS-4043 URL: https://issues.apache.org/jira/browse/HDFS-4043 Project: Hadoop HDFS Issue Type: Bug Components: security Affects Versions: 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.0.3-alpha Environment: CDH4U1 on Ubuntu 12.04 Reporter: Ahad Rana Original Estimate: 24h Remaining Estimate: 24h The Namenode uses the loginAsNameNodeUser method in NameNode.java to login using the hdfs principal. This method in turn invokes SecurityUtil.login with a hostname (last parameter) obtained via a call to InetAddress.getHostName. This call does not always return the fully qualified host name, and thus causes the namenode to login to fail due to kerberos's inability to find a matching hdfs principal in the hdfs.keytab file. Instead it should use InetAddress.getCanonicalHostName. This is consistent with what is used internally by SecurityUtil.java to login in other services, such as the DataNode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
