[
https://issues.apache.org/jira/browse/HBASE-23634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161118#comment-17161118
]
Anoop Sam John commented on HBASE-23634:
----------------------------------------
Ya the validate step can be turned off for these kind of HFiles bulkload.
Create a subtask?
On createReader ya I agree. We will need to open readers on recovered edits
files then rather than HFiles. Also at the end of replay there will be flush.
(if in between replay we reach 128 MB flush size we will have more)
The worry of not getting compacted, we can handle. We have a subtask for that
already.
[~zghao] How it can affect write is it can cause the #files under CF to reach
blocking store files. Because some times the #HFiles created by WAL split may
be really high. That is where I thought getting a compaction which can compact
all these small files in one go (irrespective of max files of compaction
config) is imp. Also for this compaction, we better dont apply the throttling
? Even one more thing is when the region is opened in another RS, that RS may
have a large compaction Q already. These compaction req will get added to the
tail of the Q and when it really gets a chance, it may be too late. So we need
to think in those directions. But thinking all those, now I wonder how far
this helps us E2E
> Enable "Split WAL to HFile" by default
> --------------------------------------
>
> Key: HBASE-23634
> URL: https://issues.apache.org/jira/browse/HBASE-23634
> Project: HBase
> Issue Type: Task
> Affects Versions: 3.0.0-alpha-1, 2.3.0
> Reporter: Guanghao Zhang
> Priority: Blocker
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
