[jira] [Updated] (HDFS-3794) WebHDFS Open used with Offset returns the original (and incorrect) Content Length in the HTTP Header.
[ https://issues.apache.org/jira/browse/HDFS-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-3794: Target Version/s: 0.23.3, 3.0.0, 1-win, 1.3.0 (was: 0.23.3, 3.0.0) Affects Version/s: 1.3.0 1-win +1 for the branch-1 patch. I'm also going to run some tests on Windows with this patch applied to branch-1-win. Assuming that goes well, I'll commit to branch-1 and branch-1-win soon. > WebHDFS Open used with Offset returns the original (and incorrect) Content > Length in the HTTP Header. > - > > Key: HDFS-3794 > URL: https://issues.apache.org/jira/browse/HDFS-3794 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 0.23.3, 2.0.0-alpha, 1-win, 1.3.0 >Reporter: Ravi Prakash >Assignee: Ravi Prakash > Fix For: 0.23.3, 2.0.2-alpha > > Attachments: HDFS-3794_branch-1.patch, HDFS-3794.patch, > HDFS-3794.patch > > > When an offset is specified, the HTTP header Content Length still contains > the original file size. e.g. if the original file is 100 bytes, and the > offset specified it 10, then HTTP Content Length ought to be 90. Currently it > is still returned as 100. > This causes curl to give error 18, and JAVA to throw ConnectionClosedException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3794) WebHDFS Open used with Offset returns the original (and incorrect) Content Length in the HTTP Header.
[ https://issues.apache.org/jira/browse/HDFS-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-3794: - Attachment: HDFS-3794_branch-1.patch HDFS-3794_branch-1.patch: for branch-1. > WebHDFS Open used with Offset returns the original (and incorrect) Content > Length in the HTTP Header. > - > > Key: HDFS-3794 > URL: https://issues.apache.org/jira/browse/HDFS-3794 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Ravi Prakash >Assignee: Ravi Prakash > Fix For: 0.23.3, 2.0.2-alpha > > Attachments: HDFS-3794_branch-1.patch, HDFS-3794.patch, > HDFS-3794.patch > > > When an offset is specified, the HTTP header Content Length still contains > the original file size. e.g. if the original file is 100 bytes, and the > offset specified it 10, then HTTP Content Length ought to be 90. Currently it > is still returned as 100. > This causes curl to give error 18, and JAVA to throw ConnectionClosedException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3794) WebHDFS Open used with Offset returns the original (and incorrect) Content Length in the HTTP Header.
[ https://issues.apache.org/jira/browse/HDFS-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-3794: -- Fix Version/s: 0.23.3 Merged to branch 23. > WebHDFS Open used with Offset returns the original (and incorrect) Content > Length in the HTTP Header. > - > > Key: HDFS-3794 > URL: https://issues.apache.org/jira/browse/HDFS-3794 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 0.23.3, 2.0.0-alpha, 2.1.0-alpha >Reporter: Ravi Prakash >Assignee: Ravi Prakash > Fix For: 0.23.3, 2.2.0-alpha > > Attachments: HDFS-3794.patch, HDFS-3794.patch > > > When an offset is specified, the HTTP header Content Length still contains > the original file size. e.g. if the original file is 100 bytes, and the > offset specified it 10, then HTTP Content Length ought to be 90. Currently it > is still returned as 100. > This causes curl to give error 18, and JAVA to throw ConnectionClosedException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3794) WebHDFS Open used with Offset returns the original (and incorrect) Content Length in the HTTP Header.
[ https://issues.apache.org/jira/browse/HDFS-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-3794: - Resolution: Fixed Fix Version/s: 2.2.0-alpha Status: Resolved (was: Patch Available) I have committed this. Thanks, Ravi! > WebHDFS Open used with Offset returns the original (and incorrect) Content > Length in the HTTP Header. > - > > Key: HDFS-3794 > URL: https://issues.apache.org/jira/browse/HDFS-3794 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 0.23.3, 2.0.0-alpha, 2.1.0-alpha >Reporter: Ravi Prakash >Assignee: Ravi Prakash > Fix For: 2.2.0-alpha > > Attachments: HDFS-3794.patch, HDFS-3794.patch > > > When an offset is specified, the HTTP header Content Length still contains > the original file size. e.g. if the original file is 100 bytes, and the > offset specified it 10, then HTTP Content Length ought to be 90. Currently it > is still returned as 100. > This causes curl to give error 18, and JAVA to throw ConnectionClosedException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3794) WebHDFS Open used with Offset returns the original (and incorrect) Content Length in the HTTP Header.
[ https://issues.apache.org/jira/browse/HDFS-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-3794: - Hadoop Flags: Reviewed +1 The new patch looks good. The failed test is not related. > WebHDFS Open used with Offset returns the original (and incorrect) Content > Length in the HTTP Header. > - > > Key: HDFS-3794 > URL: https://issues.apache.org/jira/browse/HDFS-3794 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 0.23.3, 2.0.0-alpha, 2.1.0-alpha >Reporter: Ravi Prakash >Assignee: Ravi Prakash > Attachments: HDFS-3794.patch, HDFS-3794.patch > > > When an offset is specified, the HTTP header Content Length still contains > the original file size. e.g. if the original file is 100 bytes, and the > offset specified it 10, then HTTP Content Length ought to be 90. Currently it > is still returned as 100. > This causes curl to give error 18, and JAVA to throw ConnectionClosedException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3794) WebHDFS Open used with Offset returns the original (and incorrect) Content Length in the HTTP Header.
[
https://issues.apache.org/jira/browse/HDFS-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash updated HDFS-3794:
---
Attachment: HDFS-3794.patch
Thanks a lot Nicholas! I'm afraid I don't know enough about the code. I'll
defer to you on this! I'm attaching the modified patch with the change you
suggested.
Thanks Daryn. It discovers an out of range offset and throws an exception
before reaching this method.
{noformat}
$ curl -L
"http://HOST:PORT/webhdfs/v1/somePath/someFile?op=OPEN&offset=457236547";
{"RemoteException":{"exception":"IOException","javaClassName":"java.io.IOException","message":"Offset=457236547
out of the range [0, 457236477); OPEN, path=/somePath/someFile"}}
{noformat}
> WebHDFS Open used with Offset returns the original (and incorrect) Content
> Length in the HTTP Header.
> -
>
> Key: HDFS-3794
> URL: https://issues.apache.org/jira/browse/HDFS-3794
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 0.23.3, 2.0.0-alpha, 2.1.0-alpha
>Reporter: Ravi Prakash
>Assignee: Ravi Prakash
> Attachments: HDFS-3794.patch, HDFS-3794.patch
>
>
> When an offset is specified, the HTTP header Content Length still contains
> the original file size. e.g. if the original file is 100 bytes, and the
> offset specified it 10, then HTTP Content Length ought to be 90. Currently it
> is still returned as 100.
> This causes curl to give error 18, and JAVA to throw ConnectionClosedException
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3794) WebHDFS Open used with Offset returns the original (and incorrect) Content Length in the HTTP Header.
[ https://issues.apache.org/jira/browse/HDFS-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-3794: --- Attachment: HDFS-3794.patch Attaching a patch that fixes the issue. Its too trivial to write a unit test (which will have to be pretty complicated :'( ... I tried briefly) Here's the testing I did 1. Small file with offset. Worked 2. Big file (multiple blocks) with offset. Worked 3. Big file with offset greater than file size. Correctly threw a RemoteException > WebHDFS Open used with Offset returns the original (and incorrect) Content > Length in the HTTP Header. > - > > Key: HDFS-3794 > URL: https://issues.apache.org/jira/browse/HDFS-3794 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 0.23.3, 2.0.0-alpha, 2.1.0-alpha >Reporter: Ravi Prakash >Assignee: Ravi Prakash > Attachments: HDFS-3794.patch > > > When an offset is specified, the HTTP header Content Length still contains > the original file size. e.g. if the original file is 100 bytes, and the > offset specified it 10, then HTTP Content Length ought to be 90. Currently it > is still returned as 100. > This causes curl to give error 18, and JAVA to throw ConnectionClosedException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3794) WebHDFS Open used with Offset returns the original (and incorrect) Content Length in the HTTP Header.
[ https://issues.apache.org/jira/browse/HDFS-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-3794: --- Status: Patch Available (was: Open) > WebHDFS Open used with Offset returns the original (and incorrect) Content > Length in the HTTP Header. > - > > Key: HDFS-3794 > URL: https://issues.apache.org/jira/browse/HDFS-3794 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.0.0-alpha, 0.23.3, 2.1.0-alpha >Reporter: Ravi Prakash >Assignee: Ravi Prakash > Attachments: HDFS-3794.patch > > > When an offset is specified, the HTTP header Content Length still contains > the original file size. e.g. if the original file is 100 bytes, and the > offset specified it 10, then HTTP Content Length ought to be 90. Currently it > is still returned as 100. > This causes curl to give error 18, and JAVA to throw ConnectionClosedException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
