[
https://issues.apache.org/jira/browse/HBASE-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14613361#comment-14613361
]
Andrew Purtell commented on HBASE-12988:
----------------------------------------
bq. In 1.1+ I plan to default the number of threads to 10. Before that (1.0 and
0.98) to 1, so that the behavior and performance characteristics do not change.
Sounds reasonable.
Can you pull this out into a constant and javadoc it? Should go into
hbase-defaults.xml too I think.
{quote}
{code}
"replication.source.maxthreads"
{code}
{quote}
What do you think about DEBUG level logging where we are submitting futures and
then handling errors here?
{quote}
{code}
+ List<Future<Integer>> futures = new
ArrayList<Future<Integer>>(entryLists.size());
+ for (int i=0; i<entryLists.size(); i++) {
+ if (!entryLists.get(i).isEmpty()) {
+ futures.add(exec.submit(new Replicator(entryLists.get(i), i)));
+ }
+ }
+ IOException iox = null;
+ for (Future<Integer> f : futures) {
+ try {
+ // wait for all futures remove successful parts
+ entryLists.remove(f.get());
+ } catch (InterruptedException ie) {
+ iox = new IOException(ie);
+ } catch (ExecutionException ee) {
+ // cause must be an IOException
+ iox = (IOException)ee.getCause();
+ }
+ }
+ if (iox != null) {
+ // if we had any exception, try again
+ throw iox;
+ }
{code}
{quote}
Have you by chance had a chance to run ITR with the latest changes applied?
It's a bit of a PITA to set up, you'll need to have two single node clusters
running on the same node as minimum. If not don't worry I'll do it as part of
checking the 0.98.14 RC (this change will be in it for sure).
> [Replication]Parallel apply edits across regions
> ------------------------------------------------
>
> Key: HBASE-12988
> URL: https://issues.apache.org/jira/browse/HBASE-12988
> Project: HBase
> Issue Type: Improvement
> Components: Replication
> Reporter: hongyu bi
> Assignee: Lars Hofhansl
> Attachments: 12988-v2.txt, 12988-v3.txt, 12988-v4.txt, 12988.txt,
> HBASE-12988-0.98.patch, ParallelReplication-v2.txt
>
>
> we can apply edits to slave cluster in parallel on table-level to speed up
> replication .
> update : per conversation blow , it's better to apply edits on row-level in
> parallel
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
