[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13844303#comment-13844303
]
Hudson commented on HDFS-4983:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1634 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1634/])
Move HDFS-4983 to 2.3 in CHANGES.txt (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1549747)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.3.0
>
> Attachments: HDFS-4983-branch2.3.001.patch, HDFS-4983.001.patch,
> HDFS-4983.002.patch, HDFS-4983.003.patch, HDFS-4983.004.patch,
> HDFS-4983.005.patch, HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13844268#comment-13844268
]
Hudson commented on HDFS-4983:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1608 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1608/])
Move HDFS-4983 to 2.3 in CHANGES.txt (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1549747)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.3.0
>
> Attachments: HDFS-4983-branch2.3.001.patch, HDFS-4983.001.patch,
> HDFS-4983.002.patch, HDFS-4983.003.patch, HDFS-4983.004.patch,
> HDFS-4983.005.patch, HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13844186#comment-13844186
]
Hudson commented on HDFS-4983:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #417 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/417/])
Move HDFS-4983 to 2.3 in CHANGES.txt (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1549747)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.3.0
>
> Attachments: HDFS-4983-branch2.3.001.patch, HDFS-4983.001.patch,
> HDFS-4983.002.patch, HDFS-4983.003.patch, HDFS-4983.004.patch,
> HDFS-4983.005.patch, HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843932#comment-13843932
]
Yongjun Zhang commented on HDFS-4983:
-
Thanks a lot Andrew!
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.3.0
>
> Attachments: HDFS-4983-branch2.3.001.patch, HDFS-4983.001.patch,
> HDFS-4983.002.patch, HDFS-4983.003.patch, HDFS-4983.004.patch,
> HDFS-4983.005.patch, HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843873#comment-13843873
]
Hudson commented on HDFS-4983:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #4857 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/4857/])
Move HDFS-4983 to 2.3 in CHANGES.txt (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1549747)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.3.0
>
> Attachments: HDFS-4983-branch2.3.001.patch, HDFS-4983.001.patch,
> HDFS-4983.002.patch, HDFS-4983.003.patch, HDFS-4983.004.patch,
> HDFS-4983.005.patch, HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843836#comment-13843836
]
Andrew Wang commented on HDFS-4983:
---
+1 looks good to me, thanks Yongjun. Typically we don't put the CHANGES.txt in
the generated patch, but that's easy for me to fixup.
Will commit to branch-2.3 shortly and shuffle CHANGES.txt accordingly.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.4.0
>
> Attachments: HDFS-4983-branch2.3.001.patch, HDFS-4983.001.patch,
> HDFS-4983.002.patch, HDFS-4983.003.patch, HDFS-4983.004.patch,
> HDFS-4983.005.patch, HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843292#comment-13843292
]
Yongjun Zhang commented on HDFS-4983:
-
Hi Daryn,
Please see the comment Alejandro Abdelnur made on 05/Dec/13 01:01 above.
Thanks.
--Yongjun
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.4.0
>
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843218#comment-13843218
]
Daryn Sharp commented on HDFS-4983:
---
I've questioned the necessity of this pattern before because it's required
multiple tweaks since being added. Like RPC, why not rely on the UGI to
delegate to the underlying OS for determining the validity of a username?
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.4.0
>
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842522#comment-13842522
]
Hudson commented on HDFS-4983:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1632 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1632/])
HDFS-4983. Numeric usernames do not work with WebHDFS FS. Contributed by
Yongjun Zhang. (jing9:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1548968)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.4.0
>
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842504#comment-13842504
]
Hudson commented on HDFS-4983:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #1606 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1606/])
HDFS-4983. Numeric usernames do not work with WebHDFS FS. Contributed by
Yongjun Zhang. (jing9:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1548968)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.4.0
>
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842475#comment-13842475
]
Hudson commented on HDFS-4983:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #415 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/415/])
HDFS-4983. Numeric usernames do not work with WebHDFS FS. Contributed by
Yongjun Zhang. (jing9:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1548968)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.4.0
>
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842399#comment-13842399
]
Yongjun Zhang commented on HDFS-4983:
-
Thanks a lot Jing! and many thanks to you all!
--Yongjun
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Fix For: 2.4.0
>
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842329#comment-13842329
]
Hudson commented on HDFS-4983:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #4849 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/4849/])
HDFS-4983. Numeric usernames do not work with WebHDFS FS. Contributed by
Yongjun Zhang. (jing9:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1548968)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842149#comment-13842149
]
Hadoop QA commented on HDFS-4983:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12617544/HDFS-4983.006.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5673//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5673//console
This message is automatically generated.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842105#comment-13842105
]
Yongjun Zhang commented on HDFS-4983:
-
Thanks Haohui.
I suspect the above test failure is caused by a glitch in the test env,
resubmitted previous version to trigger another run.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch, HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842054#comment-13842054
]
Hadoop QA commented on HDFS-4983:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12617515/HDFS-4983.006.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5669//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5669//console
This message is automatically generated.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13841991#comment-13841991
]
Haohui Mai commented on HDFS-4983:
--
+1. Thanks [~yzhangal]!
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13841947#comment-13841947
]
Yongjun Zhang commented on HDFS-4983:
-
HI Haohui,
Many thanks for the comments. I just uploaded a new version with all of them
addressed.
Best regards,
--Yongjun
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch,
> HDFS-4983.006.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13841655#comment-13841655
]
Haohui Mai commented on HDFS-4983:
--
bq. About the static block, the intention is that it always get executed when
the class is loaded, so to assure the default is initialized correctly.
The execution order of static blocks has been a long headache for us. The order
is determined by the Java class loader and it is unpredictable. I would much
rather to make the initialization explicit. It makes things much easier to
debug.
bq. I made the userPattern string static, and made it retrievable by the test
code (please refer to the change in TestParam.java), and annotated as
visibleForTesting.
What I'm saying is that you don't need the userPattern field at all -- you can
just compile the pattern and store it in the domain field. The code seems
redundant to me and can go away.
bq. About the property name, I think "webhdfs.", meant to be consistent
with the counterpart "httpfs...". And this is what the original bug requested.
What httpfs does is irrelevant. We have dfs.webhdfs.enabled already, so please
make it consistent.
bq. I made the public interface setUserPattern(Configuration) instead of
setUserPattern(String), so if in the future we want to set something
differently based on other configuration, we can just change inside UserParam
class without changing the caller.
Configuration in UserPattern seems a wrong abstraction to me. For no apparent
reasons UserParams depends on Configuration now. If later we want to build a
thin webhdfs client and to reuse the class, it can be problematic. If this is
not a big deal for you. I appreciate you can fix it.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13841597#comment-13841597
]
Yongjun Zhang commented on HDFS-4983:
-
Many thanks to you all for the reviews and comments.
Hi Haohui,
Thanks for your detailed review. Some clarification here.
About the static block, the intention is that it always get executed when the
class is loaded, so to assure the default is initialized correctly.
I made the userPattern string static, and made it retrievable by the test code
(please refer to the change in TestParam.java), and annotated as
visibleForTesting.
About the property name, I think "webhdfs.", meant to be consistent with
the counterpart "httpfs...". And this is what the original bug requested.
I made the public interface setUserPattern(Configuration) instead of
setUserPattern(String), so if in the future we want to set something
differently based on other configuration, we can just change inside UserParam
class without changing the caller.
Does that make sense to you?
Thanks.
--Yongjun
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13841560#comment-13841560
]
Haohui Mai commented on HDFS-4983:
--
Sorry, I didn't see userPattern is a static field. To clarify, here is a sketch:
{code}
// Some javadocs
public static void setUserNameDomain(String pattern) {
domain = Pattern.compile(pattern);
}
{code}
{code}
if (webhdfs is enabled) {
UserParam.setUserPattern(conf.get(DFS_WEBHDFS_USER_PATTERN_KEY,
DFS_WEBHDFS_USER_PATTERN_DEFAULT));
...
}
{code}
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13841537#comment-13841537
]
Haohui Mai commented on HDFS-4983:
--
Good patch. A couple comments:
{code}
+ static {
+setUserPattern(DFS_WEBHDFS_USER_PATTERN_DEFAULT);
+ }
{code}
The execution of static code block for a class is fairly unpredictable. Since
you've initialized it explicitly (thus no NPE), you can delete it.
{code}
+ private static String userPattern = null;
+ private static Domain domain = null;
+ public static void setUserPattern(Configuration conf) {
+String pattern = conf.get(DFS_WEBHDFS_USER_PATTERN_KEY,
DFS_WEBHDFS_USER_PATTERN_DEFAULT);
+setUserPattern(pattern);
+ }
+ @VisibleForTesting
+ public static String getUserPattern() {
+return userPattern;
+ }
+
+ @VisibleForTesting
+ public static void setUserPattern(String pattern) {
+userPattern = pattern;
+Pattern pt = Pattern.compile(userPattern);
+domain = new Domain(NAME, pt);
+ }
{code}
A better way to do it is to change the signature to setUserPattern(String). By
doing this you can remove the userPattern field from the class, which reduces
the overhead as jersey will construct an instance of this class for each
request.
{code}
+
+ webhdfs.user.provider.user.pattern
...
{code}
Can you rename the configuration to dfs.webhdfs.user.provider.user.pattern to
make things consistent?
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13841508#comment-13841508
]
Jing Zhao commented on HDFS-4983:
-
+1 for the new patch. Thanks Yongjun!
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13841482#comment-13841482
]
Hadoop QA commented on HDFS-4983:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12617409/HDFS-4983.005.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5659//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5659//console
This message is automatically generated.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13841360#comment-13841360
]
Yongjun Zhang commented on HDFS-4983:
-
Thanks a lot Andrew, just uploaded a version with this slight change you
pointed out.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch, HDFS-4983.005.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840860#comment-13840860
]
Andrew Wang commented on HDFS-4983:
---
Looks good Yongjun, +1 once this little nit is addressed:
{code}
+ static {
+ setUserPattern(DFS_WEBHDFS_USER_PATTERN_DEFAULT);
+ }
{code}
Indentation here should be 2 not 4 spaces.
I'll wait a day or two before committing to give Jing and Haohui time to review.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840756#comment-13840756
]
Hadoop QA commented on HDFS-4983:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12617248/HDFS-4983.004.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5649//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5649//console
This message is automatically generated.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840653#comment-13840653
]
Yongjun Zhang commented on HDFS-4983:
-
Many thanks to all. I just uploaded a new version to address the comments.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch, HDFS-4983.004.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840597#comment-13840597
]
Haohui Mai commented on HDFS-4983:
--
[~andrew.wang], this sounds good to me. Let's move forward.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840596#comment-13840596
]
Jing Zhao commented on HDFS-4983:
-
bq. How about Yongjun revs his patch again based on your review feedback, then
we file another JIRA where we discuss changing the default regex for
HttpFs/WebHDFS to be more accepting?
Sounds good to me. Let's keep moving and fix the issue first.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840587#comment-13840587
]
Andrew Wang commented on HDFS-4983:
---
Jing, Haohui, thanks for your comments thus far. Is it that big a deal to make
this configurable though? Personally, I will always choose a configurable value
with a good default over a hardcoded constant, simply because the kinds of
issues Harsh is talking about can crop up in production, and tweaking a conf
option is far better than having to ship a custom build.
How about Yongjun revs his patch again based on your review feedback, then we
file another JIRA where we discuss changing the default regex for
HttpFs/WebHDFS to be more accepting? This seems like a good compromise to me.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840120#comment-13840120
]
Hudson commented on HDFS-4983:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1629 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1629/])
Revert HDFS-4983 (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1547970)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
HDFS-4983. Numeric usernames do not work with WebHDFS FS. Contributed by
Yongjun Zhang. (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1547935)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840081#comment-13840081
]
Hudson commented on HDFS-4983:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1603 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1603/])
Revert HDFS-4983 (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1547970)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
HDFS-4983. Numeric usernames do not work with WebHDFS FS. Contributed by
Yongjun Zhang. (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1547935)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840019#comment-13840019
]
Hudson commented on HDFS-4983:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #412 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/412/])
Revert HDFS-4983 (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1547970)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
HDFS-4983. Numeric usernames do not work with WebHDFS FS. Contributed by
Yongjun Zhang. (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1547935)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839873#comment-13839873
]
Haohui Mai commented on HDFS-4983:
--
bq. I disagree but as you wish. I simply do not want a difference between
WebHDFS and HTTPFS, which is what I filed this ticket for. If you see removal
of any form of username restriction as a valid step for this, then lets also
remove it from HTTPFS at the same time.
I think that this is a good thing to shoot for. I just check the default user
pattern of HTTPFS, it doesn't support numeric usernames either. It'll make both
WebHDFS and HTTPFS simpler.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839845#comment-13839845
]
Harsh J commented on HDFS-4983:
---
bq. For example, what does it mean if I making this configuration empty?
It means you are configuring the said property incorrectly? Same argument can
be used for a whole lot of different configurations we offer (all of which have
saved ops at one time or another when a need has arose). I don't see your
point, sorry.
bq. I think that this change does bring in some configuration burdens.
We have a default that is flexible via configuration. Its not a mandatory
configuration.
bq. let's focus on supporting numeric usernames in this jira and move forward.
If you're interested in making this configurable, let's open a separate jira
for it.
I disagree but as you wish. I simply do not want a difference between WebHDFS
and HTTPFS, which is what I filed this ticket for. If you see removal of any
form of username restriction as a valid step for this, then lets also remove it
from HTTPFS at the same time.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839801#comment-13839801
]
Haohui Mai commented on HDFS-4983:
--
My concern is why the flexibility is required in WebHDFS at all. Particularly,
I'm concerned that the users hang themselves when they get this configuration
wrong. For example, what does it mean if I making this configuration empty?
Isn't a empty regex will reject all non-empty username? Suddenly the client
can't create any files. In summary, I think that this change does bring in some
configuration burdens.
Although I can definitely see that the value of using numbers as the usernames,
as well as other use cased mentioned in HDFS-4733, I'm yet to be convinced that
placing additional restrictions inside it. As I said before, WebHDFS is
basically a DFSClient speaks HTTP protocols, its behavior should stay as close
as the original DFSClient.
What about removing the checks from WebHDFS, and create a separate jira to
support the user pattern in HDFS? I'm not worried about special characters
since jersey takes care of all the escaping / unescaping automatically. It
seems that this requires minimal amount of work, and it should solve all the
use cases you guys have in mind.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839784#comment-13839784
]
Yongjun Zhang commented on HDFS-4983:
-
Thanks all for the discussion and comments.
My 2 cents: If we simply change the default to support numerical user name but
without configurability, then for users who don't want to use numerical user
name (and want to treat numerical user name illegal), s/he doesn't have the
flexibility.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839757#comment-13839757
]
Haohui Mai commented on HDFS-4983:
--
[~qwertymaniac], let's focus on supporting numeric usernames in this jira and
move forward. If you're interested in making this configurable, let's open a
separate jira for it.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839674#comment-13839674
]
Harsh J commented on HDFS-4983:
---
Yes I requested a configurability for the same reason as Alejandro has already
pointed out (on-par with HTTPFS, which had solved this issue much earlier). If
needed, via a separate JIRA, we may change the default to be the more relaxed
version for users, while continuing to keep it configurable. Its not an
"overkill".
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839630#comment-13839630
]
Alejandro Abdelnur commented on HDFS-4983:
--
Allowed usernames are the OS allowed user names. Different versions of
Unix/Linux have different restrictions by default. This was discussed when this
was done for httpfs. Refer to HDFS-4733 for details.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839547#comment-13839547
]
Haohui Mai commented on HDFS-4983:
--
Here is a link that shows how adduser addresses this:
http://lists.debian.org/debian-devel/2005/02/msg00269.html
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839544#comment-13839544
]
Andrew Wang commented on HDFS-4983:
---
I'd kind of like to hear from [~qwertymaniac] as the original reporter of this
JIRA, so let's wait a bit.
In any case, it'd be also be nice to let Yongjun provide the patch. He's put in
the work so far, so it'd be polite to have his name on the commit.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839532#comment-13839532
]
Hudson commented on HDFS-4983:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #4829 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/4829/])
Revert HDFS-4983 (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1547970)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839528#comment-13839528
]
Haohui Mai commented on HDFS-4983:
--
Jing's proposal should allow us to fix this in 2.3. +1 for the idea. I
voluntarily to do the work.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839526#comment-13839526
]
Jing Zhao commented on HDFS-4983:
-
bq. Let's figure out the right answer here.
Thanks Andrew! This sounds like a good plan.
So since to make a one-line change is very simple and direct and will not bring
other critical issues except not consistent with httpFS, how about we first
make this change in this jira? We can always make it configurable in the future
if we have this requirement from end users.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839514#comment-13839514
]
Andrew Wang commented on HDFS-4983:
---
Folks, I'm just going to revert this out since it'd look pretty messy to
replace it with a one-liner, and 2.3 is potentially getting cut soon. Let's
figure out the right answer here.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839499#comment-13839499
]
Haohui Mai commented on HDFS-4983:
--
bq. I did a quick search, but wasn't able to find any restrictions on the
usernames which HDFS or Hadoop allows. In UserGroupInformation#getLoginUser,
for example, you can see that it reads certain environment variables and just
uses them directly to get a username in some cases.
org.apache.hadoop.security.User doesn't seem to have any validation either.
I think it might be a good idea to open a new jira to address it. In general
you don't want control characters to be parts of the user / group name. For
example, if hdfs allows \0 in the username, libhdfs might break.
bq. my impression was that user name validation was added to httpfs to help
prevent problems caused by usernames which included HTTP metacharacters such as
%. The same problems apply to both webhdfs and httpfs, since they both use HTTP
and both are susceptible to those metacharacters.
My impression is that in webhdfs jersey is escaping / unescaping the parameters
automatically. It shouldn't be a problem of WebHDFS. I think it's okay to have
these additional checks around, but I don't quite understand the values of
making these checks generic and configurable.
Do you think the new pattern will be good enough for both everyday uses and
migration? If this is the case we can just tweak the pattern and claim victory.
We should avoid adding a new configuration when it is possible, as it
complicates the deployment and operations.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839481#comment-13839481
]
Colin Patrick McCabe commented on HDFS-4983:
bq. At the end of the day both of them are writing to HDFS using a DFSClient,
thus their user patterns must be a subset of the patterns that HDFS allows.
I did a quick search, but wasn't able to find any restrictions on the usernames
which HDFS or Hadoop allows. In {{UserGroupInformation#getLoginUser}}, for
example, you can see that it reads certain environment variables and just uses
them directly to get a username in some cases.
{{org.apache.hadoop.security.User}} doesn't seem to have any validation either.
bq. Httpfs is a proxy, so it might make sense for the administrators to
configure a more restrictive pattern.
Maybe someone else can comment more on this, but my impression was that user
name validation was added to httpfs to help prevent problems caused by
usernames which included HTTP metacharacters such as %. The same problems
apply to both webhdfs and httpfs, since they both use HTTP and both are
susceptible to those metacharacters.
I don't think these issues have anything to do with whether you're proxying or
not, just the fact that these names are difficult to deal with in the context
of HTTP. I guess you could argue that we should escape them rather than
disallowing them... but that seems outside the scope of this JIRA.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839463#comment-13839463
]
Yongjun Zhang commented on HDFS-4983:
-
Thank you all. Before I introduce the new file WebHdfsConfigKeys.java to hold
the new key "webhdfs.user.provider.user.pattern", I did examine
DFSConfigKeys.java, and saw all keys there start with prefix "dfs.". I wonder
whether adding a new key with different prefix there would "pollute" it. Or
should we change the new key to "dfs" given that we already have
"httpfs.user.provider.user.pattern".
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839460#comment-13839460
]
Haohui Mai commented on HDFS-4983:
--
bq. I don't think it makes sense to have this configurable in httpfs, but not
in webhdfs. It would make it hard for people to migrate between the two when
they need to. So I would argue that we should make this configurable for
webhdfs as well. What do you think?
At the end of the day both of them are writing to HDFS using a DFSClient, thus
their user patterns must be a subset of the patterns that HDFS allows.
Httpfs is a proxy, so it might make sense for the administrators to configure a
more restrictive pattern.
WebHDFS, however, should be seen as a DFSClient that speaks HTTP protocol. It
should follow the exact behavior of HDFS. Furthermore, the new pattern is more
generic than the default value of httpfs.user.provider.user.pattern, I doubt
that someone will ever hit this issue during migration in practice. Does it
make sense to you?
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839438#comment-13839438
]
Jing Zhao commented on HDFS-4983:
-
bq. Should we open a follow-up JIRA to move the config key to DFSConfigKeys.java
Yeah, since Andrew has already committed the patch. Let's continue the
discussion and work there.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839434#comment-13839434
]
Jing Zhao commented on HDFS-4983:
-
bq. whereas webHDFS (despite the name) exists only in hadoop-common.
WebHDFS and the corresponding Param are all in hadoop-hdfs.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839436#comment-13839436
]
Andrew Wang commented on HDFS-4983:
---
Jing and Haohui, sorry that I missed your comments before committing. Do you
want to revert me to revert this out while we settle this?
I guess we could stick this in DFSConfigKeys rather than making a new file, and
add it to hdfs-default.xml. I think it should still be configurable though to
match httpfs, and because apparently not all authentication systems support the
new pattern string.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839433#comment-13839433
]
Colin Patrick McCabe commented on HDFS-4983:
ignore my previous comment about webhdfs being in hadoop-common. It's in
hadoop-hdfs. httpfs is the one in hadoop-common.
Should we open a follow-up JIRA to move the config key to DFSConfigKeys.java?
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839426#comment-13839426
]
Andrew Wang commented on HDFS-4983:
---
Committed this to trunk and branch-2 after a few trivial patch conflicts.
Yongjun, do you mind preparing a branch-2.3 patch as well? This one was
non-trivial.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839424#comment-13839424
]
Colin Patrick McCabe commented on HDFS-4983:
bq. I also think we do not need to make it configurable here.
We already have {{httpfs.user.provider.user.pattern}} for httpfs.
I don't think it makes sense to have this configurable in httpfs, but not in
webhdfs. It would make it hard for people to migrate between the two when they
need to. So I would argue that we should make this configurable for webhdfs as
well. What do you think?
bq. Even if we want it configurable, we can simply put it into
DFSConfigKeys.java, and do not need to create a new WebHdfsConfigKeys.
DFSConfigKeys is in the hadoop-hdfs project, whereas webHDFS (despite the name)
exists only in hadoop-common. You can use webHDFS with filesystems other than
HDFS. Perhaps we could put this in CommonConfigurationKeys, if you feel a new
file is overkill?
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839422#comment-13839422
]
Hudson commented on HDFS-4983:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #4828 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/4828/])
HDFS-4983. Numeric usernames do not work with WebHDFS FS. Contributed by
Yongjun Zhang. (wang:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1547935)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsConfigKeys.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/resources/TestParam.java
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839411#comment-13839411
]
Jing Zhao commented on HDFS-4983:
-
I also think we do not need to make it configurable here.
Even if we want it configurable, we can simply put it into DFSConfigKeys.java,
and do not need to create a new WebHdfsConfigKeys. Also, if it is configurable,
is it supposed to be exposed to administrators? In that case, we may also want
to document it in hdfs-default.xml?
But still, I think to make the pattern configurable may be unnecessary here.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839394#comment-13839394
]
Haohui Mai commented on HDFS-4983:
--
I don't think the pattern should be configurable. I see this is an overkill. It
should be an one-liner.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839391#comment-13839391
]
Andrew Wang commented on HDFS-4983:
---
+1 looks good to me. As a future note, we don't wrap lines in the config keys
files since it'd get excessive, but I guess there's no harm to doing it here.
Thanks [~yzhangal] for the patch and [~azuryy] for the review, will commit
shortly.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839381#comment-13839381
]
Yongjun Zhang commented on HDFS-4983:
-
Thanks.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839338#comment-13839338
]
Fengdong Yu commented on HDFS-4983:
---
yes, ok for me.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839297#comment-13839297
]
Yongjun Zhang commented on HDFS-4983:
-
Sure Fengdong, I assume you are ok with my last submission after the
clarification, right? Thanks.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839283#comment-13839283
]
Fengdong Yu commented on HDFS-4983:
---
OH, Thanks for clarification.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839039#comment-13839039
]
Yongjun Zhang commented on HDFS-4983:
-
He Fengdong,
The fix is not trying to change the default "^[A-Za-z_][A-Za-z0-9._-]*[$]?$",
but allow user to specify different value such as
"^[A-Za-z0-9_][A-Za-z0-9._-]*[$]?$" in configuration file. The hard-coded value
you pointed out is only in the testcases. Does this make sense to you?
Thanks a lot.
--Yongjun
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838756#comment-13838756
]
Hadoop QA commented on HDFS-4983:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12616947/HDFS-4983.003.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5631//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5631//console
This message is automatically generated.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838719#comment-13838719
]
Fengdong Yu commented on HDFS-4983:
---
[~yzhangal] :
Thanks for fix, but I still have some comments here:
a. please use USER_PATTERN_DEFAULT in all your test cases, insteadof hard code.
b. WebHdfsConfigKeys.java
{code}
public static final String USER_PATTERN_DEFAULT =
"^[A-Za-z_][A-Za-z0-9._-]*[$]?$";
{code}
It should be
{code}
"^[A-Za-z0-9_][A-Za-z0-9._-]*[$]?$"
{code}
You passed all test cased because hard coded in all test cases, but it cannot
succeed actually if you use USER_PATTERN_DEFAULT to test.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838672#comment-13838672
]
Yongjun Zhang commented on HDFS-4983:
-
Many thanks to Fengdong for the review and comments, I submitted a new version
to address them.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch,
> HDFS-4983.003.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838608#comment-13838608
]
Hadoop QA commented on HDFS-4983:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12616932/HDFS-4983.002.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5630//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5630//console
This message is automatically generated.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838559#comment-13838559
]
Fengdong Yu commented on HDFS-4983:
---
Hi [~yzhangal] ,
We often limit 80 characters one line. so please wrap lines such as :
{code}
public static final String USER_PATTERN_KEY =
"webhdfs.user.provider.user.pattern";
{code}
{code}
private static Domain DOMAIN = null;
{code}
don't use uppercase for a non-final variable here.
{code}
+
+ @Test
+ public void testUserNameOkAfterResettingPattern() {
+String oldPattern = UserParam.getUserPattern();
+String newPattern = "^[A-Za-z0-9_][A-Za-z0-9._-]*[$]?$";
+UserParam.setUserPattern(newPattern);
+UserParam userParam = new UserParam("1x");
+assertNotNull(userParam.getValue());
+userParam = new UserParam("123");
+assertNotNull(userParam.getValue());
+UserParam.setUserPattern(oldPattern);
+ }
{code}
please add some new lines to identify different groups, such as:
{code}
+
+ @Test
+ public void testUserNameOkAfterResettingPattern() {
+String oldPattern = UserParam.getUserPattern();
+String newPattern = "^[A-Za-z0-9_][A-Za-z0-9._-]*[$]?$";
+UserParam.setUserPattern(newPattern);
+UserParam userParam = new UserParam("1x");
+assertNotNull(userParam.getValue());
+userParam = new UserParam("123");
+assertNotNull(userParam.getValue());
+UserParam.setUserPattern(oldPattern);
+ }
{code}
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838544#comment-13838544
]
Yongjun Zhang commented on HDFS-4983:
-
HI Andrew,
Thanks a lot for the review and comments. I just updated a new version to
address them.
Best regards,
--Yongjun
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch, HDFS-4983.002.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838019#comment-13838019
]
Andrew Wang commented on HDFS-4983:
---
Hey Yongjun, thanks for providing the patch. A few review comments, mostly
nitty stuff, otherwise looks good.
It's worth checking out
http://blog.cloudera.com/blog/2013/05/how-to-configure-eclipse-for-hadoop-contributions/
to get the eclipse auto-formatter if you aren't using it yet, then these
things are mostly done for you:
{code}
//set user pattern based on configuration file
{code}
Usually we put a space between the slashes and the comment. I see the comment
below also lacks a space, you could add one there too if you like.
{code}
public static final String USER_PATTERN_KEY
= "webhdfs.user.provider.user.pattern";
/** Default user name pattern value */
public static final String USER_PATTERN_DEFAULT
= "^[A-Za-z_][A-Za-z0-9._-]*[$]?$";
{code}
We normally double indent wrapped lines.
In the new proxy user test, I think we can chop out the not-superuser and
permission stuff copied from the other test. Basically, doing any WebHDFS
operation with a numeric proxy user should suffice (but please do verify!).
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13837160#comment-13837160
]
Hadoop QA commented on HDFS-4983:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12616621/HDFS-4983.001.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5613//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5613//console
This message is automatically generated.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Yongjun Zhang
> Labels: patch
> Attachments: HDFS-4983.001.patch
>
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13713815#comment-13713815
]
Kousuke Saruta commented on HDFS-4983:
--
Hi Harsh,
I understood what you said.
It seems to that there is httpfs.user.provider.user.pattern for free-er regex
for HttpFs but there is not any property for that for WebHDFS.
So, can we add a property like "webhdfs.user.provider.user.pattern" ?
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13712172#comment-13712172
]
Harsh J commented on HDFS-4983:
---
Hi Kousuke,
Thanks for looking into this!
If you mean WebHDFS'
{{hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserProvider.java}},
then I don't see an RegEx in it currently. The only RegEx I've noticed is in
Domain there which am pretty sure is the reason it fails.
For HttpFs, we configure httpfs.user.provider.user.pattern with a more free-er
regex and then there's no problem with it. But WebHDFS has a problem.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4983) Numeric usernames do not work with WebHDFS FS
[
https://issues.apache.org/jira/browse/HDFS-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13712115#comment-13712115
]
Kousuke Saruta commented on HDFS-4983:
--
Hi Harsh,
I think the reason is that USER_PATTERN_DEFAULT field in UserProvider is set to
"^[A-Za-z_][A-Za-z0-9._-]*[$]?$".
But some authentication systems like shadow allow less restricted user name.
> Numeric usernames do not work with WebHDFS FS
> -
>
> Key: HDFS-4983
> URL: https://issues.apache.org/jira/browse/HDFS-4983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>
> Per the file
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java,
> the DOMAIN pattern is set to: {{^[A-Za-z_][A-Za-z0-9._-]*[$]?$}}.
> Given this, using a username such as "123" seems to fail for some reason
> (tried on insecure setup):
> {code}
> [123@host-1 ~]$ whoami
> 123
> [123@host-1 ~]$ hadoop fs -fs webhdfs://host-2.domain.com -ls /
> -ls: Invalid value: "123" does not belong to the domain
> ^[A-Za-z_][A-Za-z0-9._-]*[$]?$
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [ ...]
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
