[jira] [Comment Edited] (HDFS-14423) Percent (%) and plus (+) characters no longer work in WebHDFS
[
https://issues.apache.org/jira/browse/HDFS-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904728#comment-16904728
]
Masatake Iwasaki edited comment on HDFS-14423 at 8/13/19 4:11 AM:
--
java.net.URLDecoder converts "\+" into " " (since java.net.URLEncoder converts
" " into "\+"). The code path of {{create}} seems to convert "a+b" into "a b"
while {{mkdir}} does not.
was (Author: iwasakims):
java.net.URLDecoder converts "+" into " " (since java.net.URLEncoder converts "
" into "+"). The code path of {{create}} seems to convert "a+b" into "a b"
while {{mkdir}} does not.
> Percent (%) and plus (+) characters no longer work in WebHDFS
> -
>
> Key: HDFS-14423
> URL: https://issues.apache.org/jira/browse/HDFS-14423
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 3.2.0, 3.1.2
> Environment: Ubuntu 16.04, but I believe this is irrelevant.
>Reporter: Jing Wang
>Assignee: Masatake Iwasaki
>Priority: Major
> Attachments: HDFS-14423.001.patch, HDFS-14423.002.patch,
> HDFS-14423.003.patch
>
>
> The following commands with percent (%) no longer work starting with version
> 3.1:
> {code:java}
> $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/%
> $ hadoop/bin/hdfs dfs -cat webhdfs://localhost/%
> cat: URLDecoder: Incomplete trailing escape (%) pattern
> {code}
> Also, plus (+ ) characters get turned into spaces when doing DN operations:
> {code:java}
> $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/a+b
> $ hadoop/bin/hdfs dfs -mkdir webhdfs://localhost/c+d
> $ hadoop/bin/hdfs dfs -ls /
> Found 4 items
> -rw-r--r-- 1 jing supergroup 0 2019-04-12 11:20 /a b
> drwxr-xr-x - jing supergroup 0 2019-04-12 11:21 /c+d
> {code}
> I can confirm that these commands work correctly on 2.9 and 3.0. Also, the
> usual hdfs:// client works as expected.
> I suspect a relation with HDFS-13176 or HDFS-13582, but I'm not sure what the
> right fix is. Note that Hive uses % to escape special characters in partition
> values, so banning % might not be a good option. For example, Hive will
> create a paths like {{table_name/partition_key=%2F}} when
> {{partition_key='/'}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14423) Percent (%) and plus (+) characters no longer work in WebHDFS
[
https://issues.apache.org/jira/browse/HDFS-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904728#comment-16904728
]
Masatake Iwasaki edited comment on HDFS-14423 at 8/13/19 4:10 AM:
--
java.net.URLDecoder converts "+" into " " (since java.net.URLEncoder converts "
" into "+"). The code path of {{create}} seems to convert "a+b" into "a b"
while {{mkdir}} does not.
was (Author: iwasakims):
java.net.URLEncoder converts "\+" into " " (since java.net.URLEncoder converts
" " into "\+"). The code path of {{create}} seems to convert "a+b" into "a b"
while {{mkdir}} does not.
> Percent (%) and plus (+) characters no longer work in WebHDFS
> -
>
> Key: HDFS-14423
> URL: https://issues.apache.org/jira/browse/HDFS-14423
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 3.2.0, 3.1.2
> Environment: Ubuntu 16.04, but I believe this is irrelevant.
>Reporter: Jing Wang
>Assignee: Masatake Iwasaki
>Priority: Major
> Attachments: HDFS-14423.001.patch, HDFS-14423.002.patch,
> HDFS-14423.003.patch
>
>
> The following commands with percent (%) no longer work starting with version
> 3.1:
> {code:java}
> $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/%
> $ hadoop/bin/hdfs dfs -cat webhdfs://localhost/%
> cat: URLDecoder: Incomplete trailing escape (%) pattern
> {code}
> Also, plus (+ ) characters get turned into spaces when doing DN operations:
> {code:java}
> $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/a+b
> $ hadoop/bin/hdfs dfs -mkdir webhdfs://localhost/c+d
> $ hadoop/bin/hdfs dfs -ls /
> Found 4 items
> -rw-r--r-- 1 jing supergroup 0 2019-04-12 11:20 /a b
> drwxr-xr-x - jing supergroup 0 2019-04-12 11:21 /c+d
> {code}
> I can confirm that these commands work correctly on 2.9 and 3.0. Also, the
> usual hdfs:// client works as expected.
> I suspect a relation with HDFS-13176 or HDFS-13582, but I'm not sure what the
> right fix is. Note that Hive uses % to escape special characters in partition
> values, so banning % might not be a good option. For example, Hive will
> create a paths like {{table_name/partition_key=%2F}} when
> {{partition_key='/'}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14423) Percent (%) and plus (+) characters no longer work in WebHDFS
[
https://issues.apache.org/jira/browse/HDFS-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904728#comment-16904728
]
Masatake Iwasaki edited comment on HDFS-14423 at 8/12/19 1:06 AM:
--
java.net.URLEncoder converts "\+" into " " (since java.net.URLEncoder converts
" " into "\+"). The code path of {{create}} seems to convert "a+b" into "a b"
while {{mkdir}} does not.
was (Author: iwasakims):
java.net.URLEncoder converts "\+" into " " (since java.net.URLEncoder converts
" " into "\+"). The code path of {{open}} seems to convert "a+b" into "a b"
while {{mkdir}} does not.
> Percent (%) and plus (+) characters no longer work in WebHDFS
> -
>
> Key: HDFS-14423
> URL: https://issues.apache.org/jira/browse/HDFS-14423
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 3.2.0, 3.1.2
> Environment: Ubuntu 16.04, but I believe this is irrelevant.
>Reporter: Jing Wang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HDFS-14423.001.patch
>
>
> The following commands with percent (%) no longer work starting with version
> 3.1:
> {code:java}
> $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/%
> $ hadoop/bin/hdfs dfs -cat webhdfs://localhost/%
> cat: URLDecoder: Incomplete trailing escape (%) pattern
> {code}
> Also, plus (+ ) characters get turned into spaces when doing DN operations:
> {code:java}
> $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/a+b
> $ hadoop/bin/hdfs dfs -mkdir webhdfs://localhost/c+d
> $ hadoop/bin/hdfs dfs -ls /
> Found 4 items
> -rw-r--r-- 1 jing supergroup 0 2019-04-12 11:20 /a b
> drwxr-xr-x - jing supergroup 0 2019-04-12 11:21 /c+d
> {code}
> I can confirm that these commands work correctly on 2.9 and 3.0. Also, the
> usual hdfs:// client works as expected.
> I suspect a relation with HDFS-13176 or HDFS-13582, but I'm not sure what the
> right fix is. Note that Hive uses % to escape special characters in partition
> values, so banning % might not be a good option. For example, Hive will
> create a paths like {{table_name/partition_key=%2F}} when
> {{partition_key='/'}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14423) Percent (%) and plus (+) characters no longer work in WebHDFS
[
https://issues.apache.org/jira/browse/HDFS-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904731#comment-16904731
]
Masatake Iwasaki edited comment on HDFS-14423 at 8/11/19 8:28 PM:
--
The reason why [WebHdfs file path gets truncated when having semicolon (\;)
inside|HDFS-13176] is that Jersey treats the part after ";" as parameters. I
think using URLEncoder/URLDecoder is not the right way for HDFS-13176. I'm
going to try refatctoring here.
was (Author: iwasakims):
The reason why WebHdfs file path gets truncated when having semicolon (\;)
inside is that Jersey treats the part after ";" as parameters. I think using
URLEncoder/URLDecoder is not the right way for HDFS-13176. I'm going to try
refatctoring here.
> Percent (%) and plus (+) characters no longer work in WebHDFS
> -
>
> Key: HDFS-14423
> URL: https://issues.apache.org/jira/browse/HDFS-14423
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 3.2.0, 3.1.2
> Environment: Ubuntu 16.04, but I believe this is irrelevant.
>Reporter: Jing Wang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HDFS-14423.001.patch
>
>
> The following commands with percent (%) no longer work starting with version
> 3.1:
> {code:java}
> $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/%
> $ hadoop/bin/hdfs dfs -cat webhdfs://localhost/%
> cat: URLDecoder: Incomplete trailing escape (%) pattern
> {code}
> Also, plus (+ ) characters get turned into spaces when doing DN operations:
> {code:java}
> $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/a+b
> $ hadoop/bin/hdfs dfs -mkdir webhdfs://localhost/c+d
> $ hadoop/bin/hdfs dfs -ls /
> Found 4 items
> -rw-r--r-- 1 jing supergroup 0 2019-04-12 11:20 /a b
> drwxr-xr-x - jing supergroup 0 2019-04-12 11:21 /c+d
> {code}
> I can confirm that these commands work correctly on 2.9 and 3.0. Also, the
> usual hdfs:// client works as expected.
> I suspect a relation with HDFS-13176 or HDFS-13582, but I'm not sure what the
> right fix is. Note that Hive uses % to escape special characters in partition
> values, so banning % might not be a good option. For example, Hive will
> create a paths like {{table_name/partition_key=%2F}} when
> {{partition_key='/'}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14423) Percent (%) and plus (+) characters no longer work in WebHDFS
[
https://issues.apache.org/jira/browse/HDFS-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904731#comment-16904731
]
Masatake Iwasaki edited comment on HDFS-14423 at 8/11/19 8:11 PM:
--
The reason why WebHdfs file path gets truncated when having semicolon (\;)
inside is that Jersey treats the part after ";" as parameters. I think using
URLEncoder/URLDecoder is not the right way for HDFS-13176. I'm going to try
refatctoring here.
was (Author: iwasakims):
The reason why WebHdfs file path gets truncated when having semicolon (;)
inside is that Jersey treats the part after ";" as parameters. I think using
URLEncoder/URLDecoder is not the right way for HDFS-13176. I'm going to try
refatctoring here.
> Percent (%) and plus (+) characters no longer work in WebHDFS
> -
>
> Key: HDFS-14423
> URL: https://issues.apache.org/jira/browse/HDFS-14423
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 3.2.0, 3.1.2
> Environment: Ubuntu 16.04, but I believe this is irrelevant.
>Reporter: Jing Wang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HDFS-14423.001.patch
>
>
> The following commands with percent (%) no longer work starting with version
> 3.1:
> {code:java}
> $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/%
> $ hadoop/bin/hdfs dfs -cat webhdfs://localhost/%
> cat: URLDecoder: Incomplete trailing escape (%) pattern
> {code}
> Also, plus (+ ) characters get turned into spaces when doing DN operations:
> {code:java}
> $ hadoop/bin/hdfs dfs -touchz webhdfs://localhost/a+b
> $ hadoop/bin/hdfs dfs -mkdir webhdfs://localhost/c+d
> $ hadoop/bin/hdfs dfs -ls /
> Found 4 items
> -rw-r--r-- 1 jing supergroup 0 2019-04-12 11:20 /a b
> drwxr-xr-x - jing supergroup 0 2019-04-12 11:21 /c+d
> {code}
> I can confirm that these commands work correctly on 2.9 and 3.0. Also, the
> usual hdfs:// client works as expected.
> I suspect a relation with HDFS-13176 or HDFS-13582, but I'm not sure what the
> right fix is. Note that Hive uses % to escape special characters in partition
> values, so banning % might not be a good option. For example, Hive will
> create a paths like {{table_name/partition_key=%2F}} when
> {{partition_key='/'}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
