[
https://issues.apache.org/jira/browse/HADOOP-14747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807995#comment-16807995
]
Steve Loughran commented on HADOOP-14747:
-----------------------------------------
those tests sound good and yes, a section in writing down what valid
preconditions are is exactly
hadoop-common-project/hadoop-common/src/site/markdown/filesystem/fsdatainputstream.md
I think you've got everything there. Maybe also
* discuss concurrency, like "if you are doing a read in a different thread,
outcome is undefined"
* assert that the input stream position doesn't change after the unbuffer
regarding the s3a tests, if you find the mock one too hard, just go straight to
the S3A one ... that new method sounds a good way to test.
if you call toString() on the stream you get all its statistics; while
getS3AStreamStatistics() lets you at the counters of how many times there's
been open/close events, bytes discarded etc.
> S3AInputStream to implement CanUnbuffer
> ---------------------------------------
>
> Key: HADOOP-14747
> URL: https://issues.apache.org/jira/browse/HADOOP-14747
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.8.1
> Reporter: Steve Loughran
> Assignee: Sahil Takiar
> Priority: Major
>
> HBase relies on FileSystems implementing {{CanUnbuffer.unbuffer()}} to force
> input streams to free up remote connections (HBASE-9393). This works for
> HDFS, but not elsewhere.
> S3A input stream can implement {{CanUnbuffer.unbuffer()}} by closing the
> input stream and relying on lazy seek to reopen it on demand.
> Needs
> * Contract specification of unbuffer. As in "who added a new feature to
> filesystems but forgot to mention what it should do?"
> * Contract test for filesystems which declare their support.
> * S3AInputStream to call {{closeStream()}} on a call to {{unbuffer()}}.
> * Test case
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
