[ https://issues.apache.org/jira/browse/HADOOP-17038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212916#comment-17212916 ]
Anoop Sam John commented on HADOOP-17038: ----------------------------------------- New PR based on the suggestion from [~ste...@apache.org]. Using new openFile API to disable buffered reads while preads. The API is marked InterfaceStability.Unstable as of now. Will this be changed ? Thanks for the suggestions. Tests passed in Azure ADL Gen2 premium storage account in East US. I have an HBase PE test results on a 3 node cluster. Will give that charts in a while. We see 2x gains. Will give cluster details and hbase file details. > Support disabling buffered reads in ABFS positional reads > --------------------------------------------------------- > > Key: HADOOP-17038 > URL: https://issues.apache.org/jira/browse/HADOOP-17038 > Project: Hadoop Common > Issue Type: Sub-task > Reporter: Anoop Sam John > Assignee: Anoop Sam John > Priority: Major > Labels: HBase, abfsactive, pull-request-available > Attachments: HBase Perf Test Report.xlsx, screenshot-1.png > > Time Spent: 50m > Remaining Estimate: 0h > > Right now it will do a seek to the position , read and then seek back to the > old position. (As per the impl in the super class) > In HBase kind of workloads we rely mostly on short preads. (like 64 KB size > by default). So would be ideal to support a pure pos read API which will not > even keep the data in a buffer but will only read the required data as what > is asked for by the caller. (Not reading ahead more data as per the read size > config) > Allow an optional boolean config to be specified while opening file for read > using which buffered pread can be disabled. > FutureDataInputStreamBuilder openFile(Path path) -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org