[
https://issues.apache.org/jira/browse/HBASE-1364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12876425#action_12876425
]
Todd Lipcon commented on HBASE-1364:
------------------------------------
bq. Is their any reason why we are targeting 22.0 vs 21.0 for this patch?
My personal opinion is that this should not be default in 0.21, since it
doesn't quite fit with the current priorities (durability, stability,
correctness) and may actually introduce new complicated bugs in the above. The
most carefully programmed distributed operation is usually buggier than the
least carefully programmed single-node operation :)
Having it be configurable seems like a great thing for 0.21 so people can try
it out, and if it seems bug-free we can make it default in the next release. Of
course that's just my opinion, and if others disagree, happy to be outvoted.
> [performance] Distributed splitting of regionserver commit logs
> ---------------------------------------------------------------
>
> Key: HBASE-1364
> URL: https://issues.apache.org/jira/browse/HBASE-1364
> Project: HBase
> Issue Type: Improvement
> Reporter: stack
> Assignee: Alex Newman
> Priority: Critical
> Fix For: 0.22.0
>
> Attachments: 1364-v2.patch, 1364.patch
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> HBASE-1008 has some improvements to our log splitting on regionserver crash;
> but it needs to run even faster.
> (Below is from HBASE-1008)
> In bigtable paper, the split is distributed. If we're going to have 1000
> logs, we need to distribute or at least multithread the splitting.
> 1. As is, regions starting up expect to find one reconstruction log only.
> Need to make it so pick up a bunch of edit logs and it should be fine that
> logs are elsewhere in hdfs in an output directory written by all split
> participants whether multithreaded or a mapreduce-like distributed process
> (Lets write our distributed sort first as a MR so we learn whats involved;
> distributed sort, as much as possible should use MR framework pieces). On
> startup, regions go to this directory and pick up the files written by split
> participants deleting and clearing the dir when all have been read in. Making
> it so can take multiple logs for input, can also make the split process more
> robust rather than current tenuous process which loses all edits if it
> doesn't make it to the end without error.
> 2. Each column family rereads the reconstruction log to find its edits. Need
> to fix that. Split can sort the edits by column family so store only reads
> its edits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
