joshelser commented on a change in pull request #2849:
URL: https://github.com/apache/hbase/pull/2849#discussion_r554118044
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceShipper.java
##########
@@ -330,4 +333,56 @@ public boolean sleepForRetries(String msg, int
sleepMultiplier) {
}
return sleepMultiplier < maxRetriesMultiplier;
}
+
+ /**
+ * Attempts to properly update
<code>ReplicationSourceManager.totalBufferUser</code>,
+ * in case there were unprocessed entries batched by the reader to the
shipper,
+ * but the shipper didn't manage to ship those because the replication
source is being terminated.
+ * In that case, it iterates through the batched entries and decrease the
pending
+ * entries size from <code>ReplicationSourceManager.totalBufferUser</code>
+ * <p/>
+ * <b>NOTES</b>
+ * 1) This method should only be called upon replication source termination.
+ * It blocks waiting for both shipper and reader threads termination,
+ * to make sure no race conditions
+ * when updating <code>ReplicationSourceManager.totalBufferUser</code>.
+ *
+ * 2) It <b>does not</b> attempt to terminate reader and shipper threads.
Those <b>must</b>
+ * have been triggered interruption/termination prior to calling this method.
+ */
+ void clearWALEntryBatch() {
+ long timeout = System.currentTimeMillis() + this.shipEditsTimeout;
+ while(this.isAlive() || this.entryReader.isAlive()){
+ try {
+ if (System.currentTimeMillis() >= timeout) {
+ LOG.warn("Shipper clearWALEntryBatch method timed out whilst waiting
reader/shipper "
+ + "thread to stop. Not cleaning buffer usage. Shipper alive: {};
Reader alive: {}",
+ this.source.getPeerId(), this.isAlive(),
this.entryReader.isAlive());
+ return;
+ } else {
+ // Wait both shipper and reader threads to stop
+ Thread.sleep(this.sleepForRetries);
+ }
+ } catch (InterruptedException e) {
+ LOG.warn("{} Interrupted while waiting {} to stop on
clearWALEntryBatch. "
+ + "Not cleaning buffer usage: {}", this.source.getPeerId(),
this.getName(), e);
+ return;
+ }
+ }
+ LongAccumulator totalToDecrement = new LongAccumulator((a,b) -> a + b, 0);
+ entryReader.entryBatchQueue.forEach(w -> {
+ entryReader.entryBatchQueue.remove(w);
Review comment:
Is this iteration safe on a Queue? Usually, the safer thing to do is to
user an `iterator` on that `Collection` to remove the element as well. The
`iterator.remove()` might also be more efficient.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
