[jira] [Updated] (HDFS-6143) WebHdfsFileSystem open should throw FileNotFoundException for non-existing paths
[ https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-6143: - Target Version/s: (was: 2.4.1) > WebHdfsFileSystem open should throw FileNotFoundException for non-existing > paths > > > Key: HDFS-6143 > URL: https://issues.apache.org/jira/browse/HDFS-6143 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Blocker > Fix For: 2.5.0 > > Attachments: HDFS-6143-branch-2.4.0.v01.patch, > HDFS-6143-trunk-after-HDFS-5570.v01.patch, > HDFS-6143-trunk-after-HDFS-5570.v02.patch, HDFS-6143.v01.patch, > HDFS-6143.v02.patch, HDFS-6143.v03.patch, HDFS-6143.v04.patch, > HDFS-6143.v04.patch, HDFS-6143.v05.patch, HDFS-6143.v06.patch > > > WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles > non-existing paths. > - 'open', does not really open anything, i.e., it does not contact the > server, and therefore cannot discover FileNotFound, it's deferred until next > read. It's counterintuitive and not how local FS or HDFS work. In POSIX you > get ENOENT on open. > [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java] > is an example of the code that's broken because of this. > - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST > instead of SC_NOT_FOUND for non-exitsing paths -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6143) WebHdfsFileSystem open should throw FileNotFoundException for non-existing paths
[ https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-6143: - Resolution: Fixed Fix Version/s: 2.5.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I've committed the patch to trunk and branch-2. Thanks [~jira.shegalov] for the contribution. > WebHdfsFileSystem open should throw FileNotFoundException for non-existing > paths > > > Key: HDFS-6143 > URL: https://issues.apache.org/jira/browse/HDFS-6143 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Blocker > Fix For: 2.5.0 > > Attachments: HDFS-6143-branch-2.4.0.v01.patch, > HDFS-6143-trunk-after-HDFS-5570.v01.patch, > HDFS-6143-trunk-after-HDFS-5570.v02.patch, HDFS-6143.v01.patch, > HDFS-6143.v02.patch, HDFS-6143.v03.patch, HDFS-6143.v04.patch, > HDFS-6143.v04.patch, HDFS-6143.v05.patch, HDFS-6143.v06.patch > > > WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles > non-existing paths. > - 'open', does not really open anything, i.e., it does not contact the > server, and therefore cannot discover FileNotFound, it's deferred until next > read. It's counterintuitive and not how local FS or HDFS work. In POSIX you > get ENOENT on open. > [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java] > is an example of the code that's broken because of this. > - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST > instead of SC_NOT_FOUND for non-exitsing paths -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6143) WebHdfsFileSystem open should throw FileNotFoundException for non-existing paths
[ https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated HDFS-6143: Attachment: HDFS-6143-trunk-after-HDFS-5570.v02.patch Removing the ExceptionHandler change to be addressed as a separate JIRA > WebHdfsFileSystem open should throw FileNotFoundException for non-existing > paths > > > Key: HDFS-6143 > URL: https://issues.apache.org/jira/browse/HDFS-6143 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Blocker > Attachments: HDFS-6143-branch-2.4.0.v01.patch, > HDFS-6143-trunk-after-HDFS-5570.v01.patch, > HDFS-6143-trunk-after-HDFS-5570.v02.patch, HDFS-6143.v01.patch, > HDFS-6143.v02.patch, HDFS-6143.v03.patch, HDFS-6143.v04.patch, > HDFS-6143.v04.patch, HDFS-6143.v05.patch, HDFS-6143.v06.patch > > > WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles > non-existing paths. > - 'open', does not really open anything, i.e., it does not contact the > server, and therefore cannot discover FileNotFound, it's deferred until next > read. It's counterintuitive and not how local FS or HDFS work. In POSIX you > get ENOENT on open. > [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java] > is an example of the code that's broken because of this. > - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST > instead of SC_NOT_FOUND for non-exitsing paths -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6143) WebHdfsFileSystem open should throw FileNotFoundException for non-existing paths
[ https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated HDFS-6143: Attachment: HDFS-6143-branch-2.4.0.v01.patch > WebHdfsFileSystem open should throw FileNotFoundException for non-existing > paths > > > Key: HDFS-6143 > URL: https://issues.apache.org/jira/browse/HDFS-6143 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Blocker > Attachments: HDFS-6143-branch-2.4.0.v01.patch, > HDFS-6143-trunk-after-HDFS-5570.v01.patch, HDFS-6143.v01.patch, > HDFS-6143.v02.patch, HDFS-6143.v03.patch, HDFS-6143.v04.patch, > HDFS-6143.v04.patch, HDFS-6143.v05.patch, HDFS-6143.v06.patch > > > WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles > non-existing paths. > - 'open', does not really open anything, i.e., it does not contact the > server, and therefore cannot discover FileNotFound, it's deferred until next > read. It's counterintuitive and not how local FS or HDFS work. In POSIX you > get ENOENT on open. > [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java] > is an example of the code that's broken because of this. > - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST > instead of SC_NOT_FOUND for non-exitsing paths -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6143) WebHdfsFileSystem open should throw FileNotFoundException for non-existing paths
[ https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated HDFS-6143: Attachment: HDFS-6143-trunk-after-HDFS-5570.v01.patch [~wheat9], I am uploading a new patch for trunk, and will follow up with patches for branch-2.4.0. The change to ExceptionHandler was done specifically to address [~jingzhao]'s comments in the original patch. > WebHdfsFileSystem open should throw FileNotFoundException for non-existing > paths > > > Key: HDFS-6143 > URL: https://issues.apache.org/jira/browse/HDFS-6143 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Blocker > Attachments: HDFS-6143-trunk-after-HDFS-5570.v01.patch, > HDFS-6143.v01.patch, HDFS-6143.v02.patch, HDFS-6143.v03.patch, > HDFS-6143.v04.patch, HDFS-6143.v04.patch, HDFS-6143.v05.patch, > HDFS-6143.v06.patch > > > WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles > non-existing paths. > - 'open', does not really open anything, i.e., it does not contact the > server, and therefore cannot discover FileNotFound, it's deferred until next > read. It's counterintuitive and not how local FS or HDFS work. In POSIX you > get ENOENT on open. > [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java] > is an example of the code that's broken because of this. > - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST > instead of SC_NOT_FOUND for non-exitsing paths -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6143) WebHdfsFileSystem open should throw FileNotFoundException for non-existing paths
[ https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated HDFS-6143: Attachment: HDFS-6143.v06.patch The v05 patch did not apply because HDFS-5570 removed TestByteRangeInputStream. Was it intentional? > WebHdfsFileSystem open should throw FileNotFoundException for non-existing > paths > > > Key: HDFS-6143 > URL: https://issues.apache.org/jira/browse/HDFS-6143 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Blocker > Attachments: HDFS-6143.v01.patch, HDFS-6143.v02.patch, > HDFS-6143.v03.patch, HDFS-6143.v04.patch, HDFS-6143.v04.patch, > HDFS-6143.v05.patch, HDFS-6143.v06.patch > > > WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles > non-existing paths. > - 'open', does not really open anything, i.e., it does not contact the > server, and therefore cannot discover FileNotFound, it's deferred until next > read. It's counterintuitive and not how local FS or HDFS work. In POSIX you > get ENOENT on open. > [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java] > is an example of the code that's broken because of this. > - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST > instead of SC_NOT_FOUND for non-exitsing paths -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6143) WebHdfsFileSystem open should throw FileNotFoundException for non-existing paths
[
https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated HDFS-6143:
Attachment: HDFS-6143.v05.patch
I split the patch as requested. However, I hope that the fix for both hftp and
webhdfs will be merged. It's pretty straightforward because they share the
logic of {{ByteRangeInputStream}}.
> WebHdfsFileSystem open should throw FileNotFoundException for non-existing
> paths
>
>
> Key: HDFS-6143
> URL: https://issues.apache.org/jira/browse/HDFS-6143
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
>Priority: Blocker
> Attachments: HDFS-6143.v01.patch, HDFS-6143.v02.patch,
> HDFS-6143.v03.patch, HDFS-6143.v04.patch, HDFS-6143.v04.patch,
> HDFS-6143.v05.patch
>
>
> WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles
> non-existing paths.
> - 'open', does not really open anything, i.e., it does not contact the
> server, and therefore cannot discover FileNotFound, it's deferred until next
> read. It's counterintuitive and not how local FS or HDFS work. In POSIX you
> get ENOENT on open.
> [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java]
> is an example of the code that's broken because of this.
> - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST
> instead of SC_NOT_FOUND for non-exitsing paths
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HDFS-6143) WebHdfsFileSystem open should throw FileNotFoundException for non-existing paths
[ https://issues.apache.org/jira/browse/HDFS-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated HDFS-6143: Summary: WebHdfsFileSystem open should throw FileNotFoundException for non-existing paths (was: (WebHdfs|Hftp)FileSystem open should throw FileNotFoundException for non-existing paths) > WebHdfsFileSystem open should throw FileNotFoundException for non-existing > paths > > > Key: HDFS-6143 > URL: https://issues.apache.org/jira/browse/HDFS-6143 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.3.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Blocker > Attachments: HDFS-6143.v01.patch, HDFS-6143.v02.patch, > HDFS-6143.v03.patch, HDFS-6143.v04.patch, HDFS-6143.v04.patch > > > WebHdfsFileSystem.open and HftpFileSystem.open incorrectly handles > non-existing paths. > - 'open', does not really open anything, i.e., it does not contact the > server, and therefore cannot discover FileNotFound, it's deferred until next > read. It's counterintuitive and not how local FS or HDFS work. In POSIX you > get ENOENT on open. > [LzoInputFormat.getSplits|https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/LzoInputFormat.java] > is an example of the code that's broken because of this. > - On the server side, FileDataServlet incorrectly sends SC_BAD_REQUEST > instead of SC_NOT_FOUND for non-exitsing paths -- This message was sent by Atlassian JIRA (v6.2#6252)
