[
https://issues.apache.org/jira/browse/HBASE-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915094#action_12915094
]
jinglong.liujl commented on HBASE-3040:
---------------------------------------
It sounds great. If index size very huge, BufferedInputStream will be more
helpful to control number of each read.
But currently, loadStoreFile in HBase read the whole indexs and keep all the
index in memory.(size of index currently is about 200K.) We can not suppose
which index is not interested by user.
So, as andy's patch, batch read index may be more helpful in performance. (keep
read index in one HDFS read operation)
> BlockIndex readIndex too slowly in heavy write scenario
> -------------------------------------------------------
>
> Key: HBASE-3040
> URL: https://issues.apache.org/jira/browse/HBASE-3040
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.20.6
> Environment: 1master, 7 region servers, 4 * 7 clients(all clients run
> on region server host), sequential put
> Reporter: andychen
> Attachments: load-all-index-in-one-dfs-read.patch
>
>
> region size is configured with 128M, block size is 64K, the table has 5
> column families
> at the beginning, when region split, master assigns daughters to new region
> servers, new region server open region, readIndex of this region's
> storefile(about 1000 blocks) spent 30~50ms, with the data import region
> server spent more and more time (sometimes up to several seconds) to load
> 1000 block indices
> at right now, we resolve this issue by getting all indices of one hfile
> within one DFS read instead of 1000 reads.
> is there any other better resolution?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
