[ https://issues.apache.org/jira/browse/HADOOP-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16660784#comment-16660784 ]
ASF GitHub Bot commented on HADOOP-15870: ----------------------------------------- Github user steveloughran commented on the issue: https://github.com/apache/hadoop/pull/433 1. I think the same problem surfaces in `remainingInCurrentRequest()` too, which is where some inspection is going to be needed to carefully understand what's up. 1. For this method, I think maybe we could cut it completely, and fix up `available()` to return a default value when `wrappedStream==null` (0 or maybe 1 except when at EOF), and when the stream isn't null, delegate to it. That pushes down the problem of estimating the number of non-blocking bytes into the http connector. Tests. In `AbstractContractSeekTest` there's some existing tests which could take some more asserts on that available call * `testSeekZeroByteFile`: `available() == 0`, always * `testSeekReadClosedFile` : call available() on an empty file, it should raise some IllegalStateException or IOE, but not an NPE For some of the other tests, I think you insert a couple of checks to say " available > 0" after a seek + read() Call. This also clarifies that "what should be available after a seek but before a read()?" As for cloudstores with lazy seek, available == 0, though if that breaks gzip then maybe they should return 1, so the assert should be available >1 with an option in the contract to say "actually returns 0". It's a tricky one to test on because really, the sole asserts should be 1. fails if the stream is closed (unless its a 0?) 1. returns >= if the stream is open 1. is always < remaining file length. Probably assert #3 is the one to check for: available() is valid when it is >=0 and <= (filelength -getPos()), with care taken about that second clause to make sure there's no off-by-one errors in the assert/code > S3AInputStream.remainingInFile should use nextReadPos > ----------------------------------------------------- > > Key: HADOOP-15870 > URL: https://issues.apache.org/jira/browse/HADOOP-15870 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 2.8.4, 3.1.1 > Reporter: Shixiong Zhu > Priority: Major > > Otherwise `remainingInFile` will not change after `seek`. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org