[
https://issues.apache.org/jira/browse/HBASE-23584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17401448#comment-17401448
]
Anoop Sam John commented on HBASE-23584:
----------------------------------------
I came across such a need recently. This is more critical when we deal with
cloud FS.
Also one more idea. When its scan, we will switch from pread to stream read
(need basis) and that time we will open new IS for the files. As part of that
also, we might end up fetching FileStatus (at the hadoop client layer ). Now
that we have new open API in hadoop common, ( FutureDataInputStreamBuilder
builder = FS.openFile(Path)) and this builder support to pass the FileStatus
(we already have it after this patch) , we can utilize it.
This API is not available in older versions. So need to see the version based
checks etc.
Just parking the idea here and not to add all as part of this Jira. I can work
on that in another Jira.
> Decrease rpc getFileStatus count when open a storefile (cache filestatus in
> storefileinfo rather than load each time)
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-23584
> URL: https://issues.apache.org/jira/browse/HBASE-23584
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 2.1.1
> Reporter: yuhuiyang
> Priority: Minor
> Attachments: HBASE-23584-branch-2.1-01.patch,
> HBASE-23584-master-001.patch
>
>
> When a store needs to open a storefile , it will create getFileStatus rpc
> twice or more . So open a region with too many files or open too many regions
> at once will cost very much time. if namenode wastes too much time in rpc
> process every time (in my case 5s sometime) due to namenode itself's problem
> . So i think we can descrease the times for getFileStatus , this will reduce
> stress to namenode and consume less time when store open a storefile .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
