ndimiduk commented on code in PR #7041:
URL: https://github.com/apache/hbase/pull/7041#discussion_r2121010210
##########
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java:
##########
@@ -131,6 +134,9 @@ public class VerifyReplication extends Configured
implements Tool {
*/
public static class Verifier extends TableMapper<ImmutableBytesWritable,
Put> {
+ private ThreadPoolExecutor reCompareExecutor = null;
Review Comment:
This Executor and its helper methods are only used from within the Map task,
so moved them in here. It's possible that some MR environment will instantiate
multiple map task instances per JVM, which would result in all tasks sharing
the same pool, but I don't think that's how MRv2 works.
##########
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java:
##########
@@ -340,20 +405,6 @@ protected void cleanup(Context context) {
throw new RuntimeException("Failed to await executor termination in
cleanup", e);
}
}
- if (replicatedScanner != null) {
Review Comment:
Not sure why we closed down the threadpool while there was still work to do.
Re-ordered those steps so that the final counter work can also run in parallel.
##########
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java:
##########
@@ -156,7 +162,96 @@ public enum Counters {
private int sleepMsBeforeReCompare;
private String delimiter = "";
private boolean verbose = false;
- private int batch = -1;
+
+ private ResultScanner initializeMapperOnFirstValue(Context context, Result
value)
Review Comment:
I moved the whole if-block up into this method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
