[hbase] branch branch-2.1 updated: HBASE-21612, Add developer debug options in HBase Config for REST server
This is an automated email from the ASF dual-hosted git repository. stack pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new e11f0d6 HBASE-21612, Add developer debug options in HBase Config for REST server e11f0d6 is described below commit e11f0d6a755de55d670664d2c12cde6f52109728 Author: Pankaj AuthorDate: Wed Dec 19 00:03:21 2018 +0530 HBASE-21612, Add developer debug options in HBase Config for REST server --- conf/hbase-env.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/conf/hbase-env.sh b/conf/hbase-env.sh index 1ac93cc..9c0e93f 100644 --- a/conf/hbase-env.sh +++ b/conf/hbase-env.sh @@ -106,6 +106,7 @@ export HBASE_OPTS="$HBASE_OPTS -XX:+UseConcMarkSweepGC" # export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8071" # export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8072" # export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8073" +# export HBASE_REST_OPTS="$HBASE_REST_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8074" # A string representing this instance of hbase. $USER by default. # export HBASE_IDENT_STRING=$USER
[hbase] branch branch-2 updated: HBASE-21612, Add developer debug options in HBase Config for REST server
This is an automated email from the ASF dual-hosted git repository. stack pushed a commit to branch branch-2 in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/branch-2 by this push: new 9990ae9 HBASE-21612, Add developer debug options in HBase Config for REST server 9990ae9 is described below commit 9990ae9a13d434c499caba9c9aa0386752210136 Author: Pankaj AuthorDate: Wed Dec 19 00:03:21 2018 +0530 HBASE-21612, Add developer debug options in HBase Config for REST server --- conf/hbase-env.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/conf/hbase-env.sh b/conf/hbase-env.sh index 3ca670f..ae54da7 100644 --- a/conf/hbase-env.sh +++ b/conf/hbase-env.sh @@ -106,6 +106,7 @@ export HBASE_OPTS="$HBASE_OPTS -XX:+UseConcMarkSweepGC" # export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8071" # export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8072" # export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8073" +# export HBASE_REST_OPTS="$HBASE_REST_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8074" # A string representing this instance of hbase. $USER by default. # export HBASE_IDENT_STRING=$USER
[hbase] branch master updated: HBASE-21612, Add developer debug options in HBase Config for REST server
This is an automated email from the ASF dual-hosted git repository. stack pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/master by this push: new 5ca1b64 HBASE-21612, Add developer debug options in HBase Config for REST server 5ca1b64 is described below commit 5ca1b64be5f1e0fa678dfe6d4c8dd787013ea2e2 Author: Pankaj AuthorDate: Wed Dec 19 00:03:21 2018 +0530 HBASE-21612, Add developer debug options in HBase Config for REST server --- conf/hbase-env.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/conf/hbase-env.sh b/conf/hbase-env.sh index 9851736..b05d9a3 100644 --- a/conf/hbase-env.sh +++ b/conf/hbase-env.sh @@ -107,6 +107,7 @@ export HBASE_OPTS="$HBASE_OPTS -XX:+UseConcMarkSweepGC" # export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8071" # export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8072" # export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8073" +# export HBASE_REST_OPTS="$HBASE_REST_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8074" # A string representing this instance of hbase. $USER by default. # export HBASE_IDENT_STRING=$USER
[hbase] branch branch-2 updated: HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case
This is an automated email from the ASF dual-hosted git repository.
openinx pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new fdfbd9a HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has
been the bottleneck in 100% scan case
fdfbd9a is described below
commit fdfbd9a21cdd84f083a11e4d7750c96ed2e76b1a
Author: huzheng
AuthorDate: Mon Jan 7 14:41:18 2019 +0800
HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the
bottleneck in 100% scan case
---
.../hadoop/hbase/client/ConnectionUtils.java | 2 +-
.../org/apache/hadoop/hbase/client/Mutation.java | 4 +-
.../org/apache/hadoop/hbase/client/Result.java | 2 +-
.../apache/hadoop/hbase/filter/KeyOnlyFilter.java | 10
.../hbase/ipc/TestHBaseRpcControllerImpl.java | 10
.../apache/hadoop/hbase/ByteBufferKeyValue.java| 5 ++
.../main/java/org/apache/hadoop/hbase/Cell.java| 8 +++-
.../java/org/apache/hadoop/hbase/CellUtil.java | 6 +--
.../java/org/apache/hadoop/hbase/ExtendedCell.java | 1 +
.../java/org/apache/hadoop/hbase/KeyValue.java | 5 ++
.../java/org/apache/hadoop/hbase/KeyValueUtil.java | 5 +-
.../org/apache/hadoop/hbase/PrivateCellUtil.java | 53 ++
.../apache/hadoop/hbase/SizeCachedKeyValue.java| 10
.../hadoop/hbase/io/encoding/RowIndexSeekerV1.java | 5 +-
.../java/org/apache/hadoop/hbase/TestCellUtil.java | 20
.../hadoop/hbase/util/MapReduceExtendedCell.java | 2 +-
.../hadoop/hbase/io/hfile/HFileBlockIndex.java | 2 +-
.../hadoop/hbase/io/hfile/HFileReaderImpl.java | 4 +-
.../apache/hadoop/hbase/regionserver/HRegion.java | 2 +-
.../hadoop/hbase/regionserver/RSRpcServices.java | 11 ++---
.../apache/hadoop/hbase/regionserver/Segment.java | 5 +-
.../hadoop/hbase/regionserver/StoreScanner.java| 3 +-
.../java/org/apache/hadoop/hbase/wal/WALEdit.java | 2 +-
.../apache/hadoop/hbase/wal/WALPrettyPrinter.java | 2 +-
.../hbase/TestPartialResultsFromClientSide.java| 3 +-
.../TestServerSideScanMetricsFromClientSide.java | 2 +-
.../TestPassCustomCellViaRegionObserver.java | 11 +
27 files changed, 114 insertions(+), 81 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
index 122d754..8e050df 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
@@ -323,7 +323,7 @@ public final class ConnectionUtils {
long estimatedHeapSizeOfResult = 0;
// We don't make Iterator here
for (Cell cell : rs.rawCells()) {
- estimatedHeapSizeOfResult += PrivateCellUtil.estimatedSizeOfCell(cell);
+ estimatedHeapSizeOfResult += cell.heapSize();
}
return estimatedHeapSizeOfResult;
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
index dc8199d..8bfdf89 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
@@ -499,8 +499,8 @@ public abstract class Mutation extends
OperationWithAttributes implements Row, C
heapsize += ClassSize.align(ClassSize.ARRAY +
size * ClassSize.REFERENCE);
- for(Cell cell : entry.getValue()) {
-heapsize += PrivateCellUtil.estimatedSizeOfCell(cell);
+ for (Cell cell : entry.getValue()) {
+heapsize += cell.heapSize();
}
}
heapsize += getAttributeSize();
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
index 832689e..5d56e83 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
@@ -859,7 +859,7 @@ public class Result implements CellScannable, CellScanner {
return size;
}
for (Cell c : result.rawCells()) {
- size += PrivateCellUtil.estimatedSizeOfCell(c);
+ size += c.heapSize();
}
return size;
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/KeyOnlyFilter.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/KeyOnlyFilter.java
index dc0b089..3368078 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/KeyOnlyFilter.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/KeyOnlyFilter.java
@@ -244,6 +244,11 @@ public class KeyOnlyFilter extends FilterBase {
}
@Override
+public int getSerializedSize() {
+ return cell.getSerializedSize();
+}
+
+
[hbase] branch master updated: HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case
This is an automated email from the ASF dual-hosted git repository.
openinx pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 15031c5 HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has
been the bottleneck in 100% scan case
15031c5 is described below
commit 15031c5bd52f4c8cc733ffd943b35da452fa0ec4
Author: huzheng
AuthorDate: Mon Jan 7 14:41:18 2019 +0800
HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the
bottleneck in 100% scan case
---
.../hadoop/hbase/client/ConnectionUtils.java | 2 +-
.../org/apache/hadoop/hbase/client/Mutation.java | 4 +-
.../org/apache/hadoop/hbase/client/Result.java | 2 +-
.../apache/hadoop/hbase/filter/KeyOnlyFilter.java | 10
.../hbase/ipc/TestHBaseRpcControllerImpl.java | 10
.../apache/hadoop/hbase/ByteBufferKeyValue.java| 5 ++
.../main/java/org/apache/hadoop/hbase/Cell.java| 8 +++-
.../java/org/apache/hadoop/hbase/CellUtil.java | 2 +-
.../java/org/apache/hadoop/hbase/ExtendedCell.java | 1 +
.../java/org/apache/hadoop/hbase/KeyValue.java | 5 ++
.../java/org/apache/hadoop/hbase/KeyValueUtil.java | 5 +-
.../org/apache/hadoop/hbase/PrivateCellUtil.java | 53 ++
.../apache/hadoop/hbase/SizeCachedKeyValue.java| 10
.../hadoop/hbase/io/encoding/RowIndexSeekerV1.java | 5 +-
.../java/org/apache/hadoop/hbase/TestCellUtil.java | 20
.../hadoop/hbase/util/MapReduceExtendedCell.java | 2 +-
.../hadoop/hbase/io/hfile/HFileBlockIndex.java | 2 +-
.../hadoop/hbase/io/hfile/HFileReaderImpl.java | 4 +-
.../apache/hadoop/hbase/regionserver/HRegion.java | 2 +-
.../hadoop/hbase/regionserver/RSRpcServices.java | 11 ++---
.../apache/hadoop/hbase/regionserver/Segment.java | 5 +-
.../hadoop/hbase/regionserver/StoreScanner.java| 3 +-
.../java/org/apache/hadoop/hbase/wal/WALEdit.java | 2 +-
.../apache/hadoop/hbase/wal/WALPrettyPrinter.java | 2 +-
.../hbase/TestPartialResultsFromClientSide.java| 3 +-
.../TestServerSideScanMetricsFromClientSide.java | 2 +-
.../TestPassCustomCellViaRegionObserver.java | 11 +
27 files changed, 114 insertions(+), 77 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
index 122d754..8e050df 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java
@@ -323,7 +323,7 @@ public final class ConnectionUtils {
long estimatedHeapSizeOfResult = 0;
// We don't make Iterator here
for (Cell cell : rs.rawCells()) {
- estimatedHeapSizeOfResult += PrivateCellUtil.estimatedSizeOfCell(cell);
+ estimatedHeapSizeOfResult += cell.heapSize();
}
return estimatedHeapSizeOfResult;
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
index dc8199d..8bfdf89 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
@@ -499,8 +499,8 @@ public abstract class Mutation extends
OperationWithAttributes implements Row, C
heapsize += ClassSize.align(ClassSize.ARRAY +
size * ClassSize.REFERENCE);
- for(Cell cell : entry.getValue()) {
-heapsize += PrivateCellUtil.estimatedSizeOfCell(cell);
+ for (Cell cell : entry.getValue()) {
+heapsize += cell.heapSize();
}
}
heapsize += getAttributeSize();
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
index 832689e..5d56e83 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java
@@ -859,7 +859,7 @@ public class Result implements CellScannable, CellScanner {
return size;
}
for (Cell c : result.rawCells()) {
- size += PrivateCellUtil.estimatedSizeOfCell(c);
+ size += c.heapSize();
}
return size;
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/KeyOnlyFilter.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/KeyOnlyFilter.java
index dc0b089..3368078 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/KeyOnlyFilter.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/KeyOnlyFilter.java
@@ -244,6 +244,11 @@ public class KeyOnlyFilter extends FilterBase {
}
@Override
+public int getSerializedSize() {
+ return cell.getSerializedSize();
+}
+
+@Override
[hbase] branch branch-2 updated: HBASE-21647 Add status track for splitting WAL tasks
This is an automated email from the ASF dual-hosted git repository.
stack pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 87d4ab8 HBASE-21647 Add status track for splitting WAL tasks
87d4ab8 is described below
commit 87d4ab866915d5bb4f28a4d0e5636e9f18b79062
Author: tianjingyun
AuthorDate: Tue Jan 15 20:19:13 2019 +0800
HBASE-21647 Add status track for splitting WAL tasks
---
.../assignment/TransitRegionStateProcedure.java| 3 ++
.../master/procedure/ServerCrashProcedure.java | 39 ++
.../hbase/master/procedure/SplitWALProcedure.java | 1 +
3 files changed, 43 insertions(+)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
index 0885a6a..2d022b7 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
@@ -32,6 +32,7 @@ import
org.apache.hadoop.hbase.exceptions.UnexpectedStateException;
import org.apache.hadoop.hbase.master.RegionState.State;
import
org.apache.hadoop.hbase.master.procedure.AbstractStateMachineRegionProcedure;
import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv;
+import org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure;
import org.apache.hadoop.hbase.procedure2.Procedure;
import org.apache.hadoop.hbase.procedure2.ProcedureMetrics;
import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer;
@@ -398,6 +399,8 @@ public class TransitRegionStateProcedure
regionNode.setOpenSeqNum(openSeqNum);
env.getAssignmentManager().regionOpened(regionNode);
if (lastState ==
RegionStateTransitionState.REGION_STATE_TRANSITION_CONFIRM_OPENED) {
+ // if parent procedure is ServerCrashProcedure, update progress
+ ServerCrashProcedure.updateProgress(env, getParentProcId());
// we are done
regionNode.unsetProcedure(this);
}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
index bee90cda..292fa45 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
@@ -35,6 +35,8 @@ import org.apache.hadoop.hbase.master.SplitWALManager;
import org.apache.hadoop.hbase.master.assignment.AssignmentManager;
import org.apache.hadoop.hbase.master.assignment.RegionStateNode;
import org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure;
+import org.apache.hadoop.hbase.monitoring.MonitoredTask;
+import org.apache.hadoop.hbase.monitoring.TaskMonitor;
import org.apache.hadoop.hbase.procedure2.Procedure;
import org.apache.hadoop.hbase.procedure2.ProcedureMetrics;
import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer;
@@ -82,6 +84,10 @@ public class ServerCrashProcedure
private boolean carryingMeta = false;
private boolean shouldSplitWal;
+ private MonitoredTask status;
+ // currentRunningState is updated when ServerCrashProcedure get scheduled,
child procedures update
+ // progress will not update the state because the actual state is
overwritten by its next state
+ private ServerCrashState currentRunningState = getInitialState();
/**
* Call this constructor queuing up a Procedure.
@@ -113,6 +119,7 @@ public class ServerCrashProcedure
throws ProcedureSuspendedException, ProcedureYieldException {
final MasterServices services = env.getMasterServices();
final AssignmentManager am = env.getAssignmentManager();
+updateProgress(true);
// HBASE-14802
// If we have not yet notified that we are processing a dead server, we
should do now.
if (!notifiedDeadServer) {
@@ -224,6 +231,7 @@ public class ServerCrashProcedure
LOG.info("removed crashed server {} after splitting done",
serverName);
services.getAssignmentManager().getRegionStates().removeServer(serverName);
services.getServerManager().getDeadServers().finish(serverName);
+ updateProgress(true);
return Flow.NO_MORE_STATE;
default:
throw new UnsupportedOperationException("unhandled state=" + state);
@@ -299,6 +307,25 @@ public class ServerCrashProcedure
LOG.debug("Done splitting WALs {}", this);
}
+ void updateProgress(boolean updateState) {
+String msg = "Processing ServerCrashProcedure of " + serverName;
+if (status == null) {
+ status = TaskMonitor.get().createStatus(msg);
+ return;
+}
+if
[hbase] branch master updated: HBASE-21647 Add status track for splitting WAL tasks
This is an automated email from the ASF dual-hosted git repository.
stack pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new c2c1af4 HBASE-21647 Add status track for splitting WAL tasks
c2c1af4 is described below
commit c2c1af4927b1e1c4443998a92946b05ffafd5937
Author: tianjingyun
AuthorDate: Tue Jan 15 20:19:13 2019 +0800
HBASE-21647 Add status track for splitting WAL tasks
---
.../assignment/TransitRegionStateProcedure.java| 3 ++
.../master/procedure/ServerCrashProcedure.java | 39 ++
.../hbase/master/procedure/SplitWALProcedure.java | 1 +
3 files changed, 43 insertions(+)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
index 0885a6a..2d022b7 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
@@ -32,6 +32,7 @@ import
org.apache.hadoop.hbase.exceptions.UnexpectedStateException;
import org.apache.hadoop.hbase.master.RegionState.State;
import
org.apache.hadoop.hbase.master.procedure.AbstractStateMachineRegionProcedure;
import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv;
+import org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure;
import org.apache.hadoop.hbase.procedure2.Procedure;
import org.apache.hadoop.hbase.procedure2.ProcedureMetrics;
import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer;
@@ -398,6 +399,8 @@ public class TransitRegionStateProcedure
regionNode.setOpenSeqNum(openSeqNum);
env.getAssignmentManager().regionOpened(regionNode);
if (lastState ==
RegionStateTransitionState.REGION_STATE_TRANSITION_CONFIRM_OPENED) {
+ // if parent procedure is ServerCrashProcedure, update progress
+ ServerCrashProcedure.updateProgress(env, getParentProcId());
// we are done
regionNode.unsetProcedure(this);
}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
index bee90cda..292fa45 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
@@ -35,6 +35,8 @@ import org.apache.hadoop.hbase.master.SplitWALManager;
import org.apache.hadoop.hbase.master.assignment.AssignmentManager;
import org.apache.hadoop.hbase.master.assignment.RegionStateNode;
import org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure;
+import org.apache.hadoop.hbase.monitoring.MonitoredTask;
+import org.apache.hadoop.hbase.monitoring.TaskMonitor;
import org.apache.hadoop.hbase.procedure2.Procedure;
import org.apache.hadoop.hbase.procedure2.ProcedureMetrics;
import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer;
@@ -82,6 +84,10 @@ public class ServerCrashProcedure
private boolean carryingMeta = false;
private boolean shouldSplitWal;
+ private MonitoredTask status;
+ // currentRunningState is updated when ServerCrashProcedure get scheduled,
child procedures update
+ // progress will not update the state because the actual state is
overwritten by its next state
+ private ServerCrashState currentRunningState = getInitialState();
/**
* Call this constructor queuing up a Procedure.
@@ -113,6 +119,7 @@ public class ServerCrashProcedure
throws ProcedureSuspendedException, ProcedureYieldException {
final MasterServices services = env.getMasterServices();
final AssignmentManager am = env.getAssignmentManager();
+updateProgress(true);
// HBASE-14802
// If we have not yet notified that we are processing a dead server, we
should do now.
if (!notifiedDeadServer) {
@@ -224,6 +231,7 @@ public class ServerCrashProcedure
LOG.info("removed crashed server {} after splitting done",
serverName);
services.getAssignmentManager().getRegionStates().removeServer(serverName);
services.getServerManager().getDeadServers().finish(serverName);
+ updateProgress(true);
return Flow.NO_MORE_STATE;
default:
throw new UnsupportedOperationException("unhandled state=" + state);
@@ -299,6 +307,25 @@ public class ServerCrashProcedure
LOG.debug("Done splitting WALs {}", this);
}
+ void updateProgress(boolean updateState) {
+String msg = "Processing ServerCrashProcedure of " + serverName;
+if (status == null) {
+ status = TaskMonitor.get().createStatus(msg);
+ return;
+}
+if
[hbase] 01/02: HBASE-19722 Meta query statistics metrics source
This is an automated email from the ASF dual-hosted git repository.
busbey pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit d89e5270aa2a5ef50a472798c7b74ef22e8be07b
Author: Xu Cang
AuthorDate: Thu May 31 14:02:26 2018 -0700
HBASE-19722 Meta query statistics metrics source
Signed-off-by: Andrew Purtell
(cherry picked from commit 58ccd3dc7eb59ce41a2a48af792adb342520e626)
---
.../java/org/apache/hadoop/hbase/HConstants.java | 2 +
.../hadoop/hbase/coprocessor/MetaTableMetrics.java | 335 +
.../apache/hadoop/hbase/util/LossyCounting.java| 135 +
.../hbase/coprocessor/TestMetaTableMetrics.java| 231 ++
.../hadoop/hbase/util/TestLossyCounting.java | 88 ++
5 files changed, 791 insertions(+)
diff --git a/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
index 42288c5..19581f5 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
@@ -1402,6 +1402,8 @@ public final class HConstants {
public static final String DEFAULT_SNAPSHOT_RESTORE_FAILSAFE_NAME =
"hbase-failsafe-{snapshot.name}-{restore.timestamp}";
+ public static final String DEFAULT_LOSSY_COUNTING_ERROR_RATE =
+ "hbase.util.default.lossycounting.errorrate";
public static final String NOT_IMPLEMENTED = "Not implemented";
private HConstants() {
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MetaTableMetrics.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MetaTableMetrics.java
new file mode 100644
index 000..9bf35c0
--- /dev/null
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MetaTableMetrics.java
@@ -0,0 +1,335 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
contributor license
+ * agreements. See the NOTICE file distributed with this work for additional
information regarding
+ * copyright ownership. The ASF licenses this file to you under the Apache
License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
License. You may obtain a
+ * copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless
required by applicable
+ * law or agreed to in writing, software distributed under the License is
distributed on an "AS IS"
+ * BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License
+ * for the specific language governing permissions and limitations under the
License.
+ */
+
+package org.apache.hadoop.hbase.coprocessor;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.CoprocessorEnvironment;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Get;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Row;
+import org.apache.hadoop.hbase.ipc.RpcServer;
+import org.apache.hadoop.hbase.metrics.Meter;
+import org.apache.hadoop.hbase.metrics.Metric;
+import org.apache.hadoop.hbase.metrics.MetricRegistry;
+import org.apache.hadoop.hbase.util.LossyCounting;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.hbase.thirdparty.com.google.common.collect.ImmutableMap;
+
+
+/**
+ * A coprocessor that collects metrics from meta table.
+ *
+ * These metrics will be available through the regular Hadoop metrics2 sinks
(ganglia, opentsdb,
+ * etc) as well as JMX output.
+ *
+ * @see MetaTableMetrics
+ */
+
+@InterfaceAudience.Private
+public class MetaTableMetrics implements RegionCoprocessor {
+
+ private ExampleRegionObserverMeta observer;
+ private Map> requestsMap;
+ private RegionCoprocessorEnvironment regionCoprocessorEnv;
+ private LossyCounting clientMetricsLossyCounting;
+ private boolean active = false;
+
+ enum MetaTableOps {
+GET, PUT, DELETE;
+ }
+
+ private ImmutableMap opsNameMap =
+ ImmutableMap.builder()
+ .put(Put.class, MetaTableOps.PUT)
+ .put(Get.class, MetaTableOps.GET)
+ .put(Delete.class, MetaTableOps.DELETE)
+ .build();
+
+ class ExampleRegionObserverMeta implements RegionCoprocessor, RegionObserver
{
+
+@Override
+public Optional getRegionObserver() {
+ return Optional.of(this);
+}
+
+@Override
+public void preGetOp(ObserverContext e, Get
get,
+List results) throws IOException {
+ if (!active || !isMetaTableOp(e)) {
+return;
+
[hbase] branch branch-2.1 updated (47099d1 -> 6c18d97)
This is an automated email from the ASF dual-hosted git repository. busbey pushed a change to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/hbase.git. from 47099d1 HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit. (addendum) new d89e527 HBASE-19722 Meta query statistics metrics source new 6c18d97 HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for non-meta region The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../java/org/apache/hadoop/hbase/HConstants.java | 2 + .../hadoop/hbase/coprocessor/MetaTableMetrics.java | 335 + .../apache/hadoop/hbase/util/LossyCounting.java| 135 + .../hbase/coprocessor/TestMetaTableMetrics.java| 231 ++ .../hadoop/hbase/util/TestLossyCounting.java | 88 ++ 5 files changed, 791 insertions(+) create mode 100644 hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MetaTableMetrics.java create mode 100644 hbase-server/src/main/java/org/apache/hadoop/hbase/util/LossyCounting.java create mode 100644 hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMetaTableMetrics.java create mode 100644 hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestLossyCounting.java
[hbase] 02/02: HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for non-meta region
This is an automated email from the ASF dual-hosted git repository.
busbey pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 6c18d9735701b94dad7396d5ce1e2702ba1a6d48
Author: tedyu
AuthorDate: Sun Jul 22 20:51:58 2018 -0700
HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for
non-meta region
with addendum
(cherry picked from commit 654480326b9a8a5fa9d9314d1cfced6a45cfa941)
(cherry picked from commit 3787c729530ab7cf7f759decf2cbe961ff95f497)
---
.../hadoop/hbase/coprocessor/MetaTableMetrics.java | 20 ++--
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MetaTableMetrics.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MetaTableMetrics.java
index 9bf35c0..3bb47ae 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MetaTableMetrics.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MetaTableMetrics.java
@@ -197,9 +197,8 @@ public class MetaTableMetrics implements RegionCoprocessor {
}
private boolean
isMetaTableOp(ObserverContext e) {
- return TableName.META_TABLE_NAME.toString()
- .equals(new
String(e.getEnvironment().getRegionInfo().getTable().getName(),
- StandardCharsets.UTF_8));
+ return TableName.META_TABLE_NAME
+ .equals(e.getEnvironment().getRegionInfo().getTable());
}
private void
clientMetricRegisterAndMark(ObserverContext e) {
@@ -309,9 +308,8 @@ public class MetaTableMetrics implements RegionCoprocessor {
public void start(CoprocessorEnvironment env) throws IOException {
if (env instanceof RegionCoprocessorEnvironment
&& ((RegionCoprocessorEnvironment) env).getRegionInfo().getTable() !=
null
-&& ((RegionCoprocessorEnvironment)
env).getRegionInfo().getTable().getName() != null
-&& new String(((RegionCoprocessorEnvironment)
env).getRegionInfo().getTable().getName(),
-
StandardCharsets.UTF_8).equals(TableName.META_TABLE_NAME.toString())) {
+&& ((RegionCoprocessorEnvironment) env).getRegionInfo().getTable()
+ .equals(TableName.META_TABLE_NAME)) {
regionCoprocessorEnv = (RegionCoprocessorEnvironment) env;
observer = new ExampleRegionObserverMeta();
requestsMap = new ConcurrentHashMap<>();
@@ -324,11 +322,13 @@ public class MetaTableMetrics implements
RegionCoprocessor {
}
@Override
- public void stop(CoprocessorEnvironment e) throws IOException {
+ public void stop(CoprocessorEnvironment env) throws IOException {
// since meta region can move around, clear stale metrics when stop.
-for (String meterName : requestsMap.keySet()) {
- MetricRegistry registry =
regionCoprocessorEnv.getMetricRegistryForRegionServer();
- registry.remove(meterName);
+if (requestsMap != null) {
+ for (String meterName : requestsMap.keySet()) {
+MetricRegistry registry =
regionCoprocessorEnv.getMetricRegistryForRegionServer();
+registry.remove(meterName);
+ }
}
}
[hbase] 01/02: HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit.
This is an automated email from the ASF dual-hosted git repository.
busbey pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 5c7c425e34a8306f70354f4910a2e686230320f4
Author: Lars Hofhansl
AuthorDate: Thu Dec 13 11:56:39 2018 -0800
HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit.
(cherry picked from commit 11193d7cc1f5a0a7ffb73777da7ce5c1b6af6c8c)
---
.../org/apache/hadoop/hbase/regionserver/StoreScanner.java | 12 ++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
index be226fe..bc457df 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
@@ -808,12 +808,16 @@ public class StoreScanner extends
NonReversedNonLazyKeyValueScanner
@VisibleForTesting
protected boolean trySkipToNextRow(Cell cell) throws IOException {
Cell nextCell = null;
+// used to guard against a changed next indexed key by doing a identity
comparison
+// when the identity changes we need to compare the bytes again
+Cell previousIndexedKey = null;
do {
Cell nextIndexedKey = getNextIndexedKey();
if (nextIndexedKey != null && nextIndexedKey !=
KeyValueScanner.NO_NEXT_INDEXED_KEY
- && matcher.compareKeyForNextRow(nextIndexedKey, cell) >= 0) {
+ && (nextIndexedKey == previousIndexedKey ||
matcher.compareKeyForNextRow(nextIndexedKey, cell) >= 0)) {
this.heap.next();
++kvsScanned;
+previousIndexedKey = nextIndexedKey;
} else {
return false;
}
@@ -829,12 +833,16 @@ public class StoreScanner extends
NonReversedNonLazyKeyValueScanner
@VisibleForTesting
protected boolean trySkipToNextColumn(Cell cell) throws IOException {
Cell nextCell = null;
+// used to guard against a changed next indexed key by doing a identity
comparison
+// when the identity changes we need to compare the bytes again
+Cell previousIndexedKey = null;
do {
Cell nextIndexedKey = getNextIndexedKey();
if (nextIndexedKey != null && nextIndexedKey !=
KeyValueScanner.NO_NEXT_INDEXED_KEY
- && matcher.compareKeyForNextColumn(nextIndexedKey, cell) >= 0) {
+ && (nextIndexedKey == previousIndexedKey ||
matcher.compareKeyForNextColumn(nextIndexedKey, cell) >= 0)) {
this.heap.next();
++kvsScanned;
+previousIndexedKey = nextIndexedKey;
} else {
return false;
}
[hbase] 02/02: HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit. (addendum)
This is an automated email from the ASF dual-hosted git repository.
busbey pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 47099d1b17ec2d69e8ebd9b4bfb26ca25af1725f
Author: Sean Busbey
AuthorDate: Fri Dec 14 11:23:36 2018 -0600
HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit. (addendum)
(cherry picked from commit a284c493b127fbf785fbee401f0854ad1979eb26)
(cherry picked from commit aa36c3f6b69ef47b6f53158eda4a87e1a72e4fad)
---
.../org/apache/hadoop/hbase/regionserver/StoreScanner.java | 10 ++
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
index bc457df..d06 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
@@ -813,8 +813,9 @@ public class StoreScanner extends
NonReversedNonLazyKeyValueScanner
Cell previousIndexedKey = null;
do {
Cell nextIndexedKey = getNextIndexedKey();
- if (nextIndexedKey != null && nextIndexedKey !=
KeyValueScanner.NO_NEXT_INDEXED_KEY
- && (nextIndexedKey == previousIndexedKey ||
matcher.compareKeyForNextRow(nextIndexedKey, cell) >= 0)) {
+ if (nextIndexedKey != null && nextIndexedKey !=
KeyValueScanner.NO_NEXT_INDEXED_KEY &&
+ (nextIndexedKey == previousIndexedKey ||
+ matcher.compareKeyForNextRow(nextIndexedKey, cell) >= 0)) {
this.heap.next();
++kvsScanned;
previousIndexedKey = nextIndexedKey;
@@ -838,8 +839,9 @@ public class StoreScanner extends
NonReversedNonLazyKeyValueScanner
Cell previousIndexedKey = null;
do {
Cell nextIndexedKey = getNextIndexedKey();
- if (nextIndexedKey != null && nextIndexedKey !=
KeyValueScanner.NO_NEXT_INDEXED_KEY
- && (nextIndexedKey == previousIndexedKey ||
matcher.compareKeyForNextColumn(nextIndexedKey, cell) >= 0)) {
+ if (nextIndexedKey != null && nextIndexedKey !=
KeyValueScanner.NO_NEXT_INDEXED_KEY &&
+ (nextIndexedKey == previousIndexedKey ||
+ matcher.compareKeyForNextColumn(nextIndexedKey, cell) >= 0)) {
this.heap.next();
++kvsScanned;
previousIndexedKey = nextIndexedKey;
[hbase] branch branch-2.1 updated (12dfc8b -> 47099d1)
This is an automated email from the ASF dual-hosted git repository. busbey pushed a change to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/hbase.git. from 12dfc8b HBASE-21639 maxHeapUsage value not read properly from config during EntryBuffers initialization new 5c7c425 HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit. new 47099d1 HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit. (addendum) The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../apache/hadoop/hbase/regionserver/StoreScanner.java | 18 ++ 1 file changed, 14 insertions(+), 4 deletions(-)
[hbase] branch branch-2.0 updated: HBASE-21639 maxHeapUsage value not read properly from config during EntryBuffers initialization
This is an automated email from the ASF dual-hosted git repository.
stack pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 27987e8 HBASE-21639 maxHeapUsage value not read properly from config
during EntryBuffers initialization
27987e8 is described below
commit 27987e850eaa5f5e0b114aa66eadeb2ab164ee1c
Author: Pankaj
AuthorDate: Sat Dec 29 00:54:48 2018 +0530
HBASE-21639 maxHeapUsage value not read properly from config during
EntryBuffers initialization
---
.../replication/regionserver/RegionReplicaReplicationEndpoint.java | 3 +--
.../src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java | 2 +-
2 files changed, 2 insertions(+), 3 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
index cb755fe..e0e58c3 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
@@ -131,8 +131,7 @@ public class RegionReplicaReplicationEndpoint extends
HBaseReplicationEndpoint {
"hbase.region.replica.replication.writer.threads", 3);
controller = new PipelineController();
entryBuffers = new EntryBuffers(controller,
- this.conf.getInt("hbase.region.replica.replication.buffersize",
- 128*1024*1024));
+this.conf.getLong("hbase.region.replica.replication.buffersize", 128 *
1024 * 1024));
// use the regular RPC timeout for replica replication RPC's
this.operationTimeout =
conf.getInt(HConstants.HBASE_CLIENT_OPERATION_TIMEOUT,
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
index 9ed7f6c..d936561 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
@@ -166,7 +166,7 @@ public class WALSplitter {
this.splitWriterCreationBounded =
conf.getBoolean(SPLIT_WRITER_CREATION_BOUNDED, false);
entryBuffers = new EntryBuffers(controller,
-this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 *
1024 * 1024),
+this.conf.getLong("hbase.regionserver.hlog.splitlog.buffersize", 128 *
1024 * 1024),
splitWriterCreationBounded);
int numWriterThreads =
this.conf.getInt("hbase.regionserver.hlog.splitlog.writer.threads", 3);
[hbase] branch branch-2.1 updated: HBASE-21639 maxHeapUsage value not read properly from config during EntryBuffers initialization
This is an automated email from the ASF dual-hosted git repository.
stack pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 12dfc8b HBASE-21639 maxHeapUsage value not read properly from config
during EntryBuffers initialization
12dfc8b is described below
commit 12dfc8bafd9054969deb0d6261380287d5977143
Author: Pankaj
AuthorDate: Sat Dec 29 00:54:48 2018 +0530
HBASE-21639 maxHeapUsage value not read properly from config during
EntryBuffers initialization
---
.../replication/regionserver/RegionReplicaReplicationEndpoint.java | 3 +--
.../src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java | 2 +-
2 files changed, 2 insertions(+), 3 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
index cb755fe..e0e58c3 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
@@ -131,8 +131,7 @@ public class RegionReplicaReplicationEndpoint extends
HBaseReplicationEndpoint {
"hbase.region.replica.replication.writer.threads", 3);
controller = new PipelineController();
entryBuffers = new EntryBuffers(controller,
- this.conf.getInt("hbase.region.replica.replication.buffersize",
- 128*1024*1024));
+this.conf.getLong("hbase.region.replica.replication.buffersize", 128 *
1024 * 1024));
// use the regular RPC timeout for replica replication RPC's
this.operationTimeout =
conf.getInt(HConstants.HBASE_CLIENT_OPERATION_TIMEOUT,
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
index 4a18ce5..638c574 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
@@ -166,7 +166,7 @@ public class WALSplitter {
this.splitWriterCreationBounded =
conf.getBoolean(SPLIT_WRITER_CREATION_BOUNDED, false);
entryBuffers = new EntryBuffers(controller,
-this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 *
1024 * 1024),
+this.conf.getLong("hbase.regionserver.hlog.splitlog.buffersize", 128 *
1024 * 1024),
splitWriterCreationBounded);
int numWriterThreads =
this.conf.getInt("hbase.regionserver.hlog.splitlog.writer.threads", 3);
[hbase] branch branch-2 updated: HBASE-21639 maxHeapUsage value not read properly from config during EntryBuffers initialization
This is an automated email from the ASF dual-hosted git repository.
stack pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 11a0793 HBASE-21639 maxHeapUsage value not read properly from config
during EntryBuffers initialization
11a0793 is described below
commit 11a0793d7265a4cb1c7ab3caa8d36b24ebd9c05e
Author: Pankaj
AuthorDate: Sat Dec 29 00:54:48 2018 +0530
HBASE-21639 maxHeapUsage value not read properly from config during
EntryBuffers initialization
---
.../replication/regionserver/RegionReplicaReplicationEndpoint.java | 3 +--
.../src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java | 2 +-
2 files changed, 2 insertions(+), 3 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
index dc83eb7..f7721e0 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
@@ -132,8 +132,7 @@ public class RegionReplicaReplicationEndpoint extends
HBaseReplicationEndpoint {
"hbase.region.replica.replication.writer.threads", 3);
controller = new PipelineController();
entryBuffers = new EntryBuffers(controller,
- this.conf.getInt("hbase.region.replica.replication.buffersize",
- 128*1024*1024));
+this.conf.getLong("hbase.region.replica.replication.buffersize", 128 *
1024 * 1024));
// use the regular RPC timeout for replica replication RPC's
this.operationTimeout =
conf.getInt(HConstants.HBASE_CLIENT_OPERATION_TIMEOUT,
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
index 4a18ce5..638c574 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
@@ -166,7 +166,7 @@ public class WALSplitter {
this.splitWriterCreationBounded =
conf.getBoolean(SPLIT_WRITER_CREATION_BOUNDED, false);
entryBuffers = new EntryBuffers(controller,
-this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 *
1024 * 1024),
+this.conf.getLong("hbase.regionserver.hlog.splitlog.buffersize", 128 *
1024 * 1024),
splitWriterCreationBounded);
int numWriterThreads =
this.conf.getInt("hbase.regionserver.hlog.splitlog.writer.threads", 3);
[hbase] branch master updated: HBASE-21639 maxHeapUsage value not read properly from config during EntryBuffers initialization
This is an automated email from the ASF dual-hosted git repository.
stack pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 6da0b4e HBASE-21639 maxHeapUsage value not read properly from config
during EntryBuffers initialization
6da0b4e is described below
commit 6da0b4ec34727240e433825382cfc30366340097
Author: Pankaj
AuthorDate: Sat Dec 29 00:54:48 2018 +0530
HBASE-21639 maxHeapUsage value not read properly from config during
EntryBuffers initialization
---
.../replication/regionserver/RegionReplicaReplicationEndpoint.java | 3 +--
.../src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java | 2 +-
2 files changed, 2 insertions(+), 3 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
index dc83eb7..f7721e0 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
@@ -132,8 +132,7 @@ public class RegionReplicaReplicationEndpoint extends
HBaseReplicationEndpoint {
"hbase.region.replica.replication.writer.threads", 3);
controller = new PipelineController();
entryBuffers = new EntryBuffers(controller,
- this.conf.getInt("hbase.region.replica.replication.buffersize",
- 128*1024*1024));
+this.conf.getLong("hbase.region.replica.replication.buffersize", 128 *
1024 * 1024));
// use the regular RPC timeout for replica replication RPC's
this.operationTimeout =
conf.getInt(HConstants.HBASE_CLIENT_OPERATION_TIMEOUT,
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
index bc67d98..6679597 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
@@ -167,7 +167,7 @@ public class WALSplitter {
this.splitWriterCreationBounded =
conf.getBoolean(SPLIT_WRITER_CREATION_BOUNDED, false);
entryBuffers = new EntryBuffers(controller,
-this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 *
1024 * 1024),
+this.conf.getLong("hbase.regionserver.hlog.splitlog.buffersize", 128 *
1024 * 1024),
splitWriterCreationBounded);
int numWriterThreads =
this.conf.getInt("hbase.regionserver.hlog.splitlog.writer.threads", 3);
[hbase] branch branch-2 updated: HBASE-21595, Print thread's information and stack traces when RS is aborting forcibly
This is an automated email from the ASF dual-hosted git repository.
stack pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new a9f40d3 HBASE-21595, Print thread's information and stack traces when
RS is aborting forcibly
a9f40d3 is described below
commit a9f40d3bf72fc618ffe30e76ea6e03d460918ac5
Author: Pankaj
AuthorDate: Tue Dec 18 01:03:08 2018 +0530
HBASE-21595, Print thread's information and stack traces when RS is
aborting forcibly
---
.../main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
index bbae268..5443a8c 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
@@ -3787,7 +3787,8 @@ public class HRegionServer extends HasThread implements
@Override
public void run() {
- LOG.warn("Aborting region server timed out, terminate forcibly...");
+ LOG.warn("Aborting region server timed out, terminating forcibly. Thread
dump to stdout.");
+ Threads.printThreadInfo(System.out, "Zombie HRegionServer");
System.exit(1);
}
}
[hbase] branch master updated: HBASE-21595, Print thread's information and stack traces when RS is aborting forcibly
This is an automated email from the ASF dual-hosted git repository.
stack pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new bc0fb66 HBASE-21595, Print thread's information and stack traces when
RS is aborting forcibly
bc0fb66 is described below
commit bc0fb66e224c639f5a0b0d5ed940446a667cab7f
Author: Pankaj
AuthorDate: Tue Dec 18 01:03:08 2018 +0530
HBASE-21595, Print thread's information and stack traces when RS is
aborting forcibly
---
.../main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
index a99ded3..34a6c13 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
@@ -3849,7 +3849,8 @@ public class HRegionServer extends HasThread implements
@Override
public void run() {
- LOG.warn("Aborting region server timed out, terminate forcibly...");
+ LOG.warn("Aborting region server timed out, terminating forcibly. Thread
dump to stdout.");
+ Threads.printThreadInfo(System.out, "Zombie HRegionServer");
System.exit(1);
}
}
[hbase-site] branch asf-site updated: INFRA-10751 Empty commit
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hbase-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 13e4495 INFRA-10751 Empty commit 13e4495 is described below commit 13e44956ff493f794732e9486a55eaab8f2fd859 Author: jenkins AuthorDate: Tue Jan 15 14:48:25 2019 + INFRA-10751 Empty commit
[hbase] 07/08: HBASE-21537 Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch HBASE-21512
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 446f9adf51a593b790a50ccb89514ae833887b88
Author: Duo Zhang
AuthorDate: Thu Jan 10 11:47:41 2019 +0800
HBASE-21537 Rewrite ServerManager.closeRegionSilentlyAndWait to use
AsyncClusterConnection
Signed-off-by: Michael Stack
---
.../hadoop/hbase/master/MasterMetaBootstrap.java | 2 +-
.../apache/hadoop/hbase/master/ServerManager.java | 41 +-
.../apache/hadoop/hbase/util/HBaseFsckRepair.java | 22 +++-
3 files changed, 38 insertions(+), 27 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
index e57817e..6e38bdd 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
@@ -101,7 +101,7 @@ class MasterMetaBootstrap {
RegionState r = MetaTableLocator.getMetaRegionState(zooKeeper,
replicaId);
LOG.info("Closing excess replica of meta region " + r.getRegion());
// send a close and wait for a max of 30 seconds
-
ServerManager.closeRegionSilentlyAndWait(master.getClusterConnection(),
+
ServerManager.closeRegionSilentlyAndWait(master.getAsyncClusterConnection(),
r.getServerName(), r.getRegion(), 3);
ZKUtil.deleteNode(zooKeeper,
zooKeeper.getZNodePaths().getZNodeForReplica(replicaId));
}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
index c26ef6c..44a0770 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
@@ -48,13 +48,15 @@ import org.apache.hadoop.hbase.ServerMetrics;
import org.apache.hadoop.hbase.ServerMetricsBuilder;
import org.apache.hadoop.hbase.ServerName;
import org.apache.hadoop.hbase.YouAreDeadException;
-import org.apache.hadoop.hbase.client.ClusterConnection;
+import org.apache.hadoop.hbase.client.AsyncClusterConnection;
+import org.apache.hadoop.hbase.client.AsyncRegionServerAdmin;
import org.apache.hadoop.hbase.client.RegionInfo;
import org.apache.hadoop.hbase.ipc.HBaseRpcController;
import org.apache.hadoop.hbase.master.assignment.RegionStates;
import org.apache.hadoop.hbase.monitoring.MonitoredTask;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.util.FutureUtils;
import org.apache.hadoop.hbase.zookeeper.ZKUtil;
import org.apache.hadoop.hbase.zookeeper.ZKWatcher;
import org.apache.yetus.audience.InterfaceAudience;
@@ -67,6 +69,7 @@ import
org.apache.hbase.thirdparty.com.google.protobuf.ByteString;
import org.apache.hbase.thirdparty.com.google.protobuf.UnsafeByteOperations;
import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.RequestConverter;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionStoreSequenceIds;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.StoreSequenceId;
@@ -660,35 +663,39 @@ public class ServerManager {
}
/**
- * Contacts a region server and waits up to timeout ms
- * to close the region. This bypasses the active hmaster.
+ * Contacts a region server and waits up to timeout ms to close the region.
This bypasses the
+ * active hmaster.
*/
- public static void closeRegionSilentlyAndWait(ClusterConnection connection,
-ServerName server, RegionInfo region, long timeout) throws IOException,
InterruptedException {
-AdminService.BlockingInterface rs = connection.getAdmin(server);
-HBaseRpcController controller =
connection.getRpcControllerFactory().newController();
+ public static void closeRegionSilentlyAndWait(AsyncClusterConnection
connection,
+ ServerName server, RegionInfo region, long timeout) throws IOException,
InterruptedException {
+AsyncRegionServerAdmin admin = connection.getRegionServerAdmin(server);
try {
- ProtobufUtil.closeRegion(controller, rs, server, region.getRegionName());
+ FutureUtils.get(
+admin.closeRegion(ProtobufUtil.buildCloseRegionRequest(server,
region.getRegionName(;
} catch (IOException e) {
LOG.warn("Exception when closing region: " +
region.getRegionNameAsString(), e);
}
long expiration = timeout + System.currentTimeMillis();
while (System.currentTimeMillis() < expiration) {
- controller.reset();
try {
-
[hbase] 06/08: HBASE-21671 Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch HBASE-21512
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 6786c4952776d3798247bae0a42312e7cfbdca09
Author: Duo Zhang
AuthorDate: Fri Jan 11 16:22:24 2019 +0800
HBASE-21671 Rewrite RegionReplicaReplicationEndpoint to use
AsyncClusterConnection
---
.../hadoop/hbase/client/AsyncConnectionImpl.java | 24 +-
.../hbase/client/AsyncClusterConnection.java | 17 +
.../hbase/client/AsyncClusterConnectionImpl.java | 80 +++
.../AsyncRegionReplicaReplayRetryingCaller.java| 146
.../hbase/client/AsyncRegionServerAdmin.java | 5 +-
.../hbase/client/ClusterConnectionFactory.java | 2 +-
.../hbase/protobuf/ReplicationProtbufUtil.java | 31 +-
.../handler/RegionReplicaFlushHandler.java | 3 +-
.../hbase/replication/ReplicationEndpoint.java | 35 +-
.../RegionReplicaReplicationEndpoint.java | 783 +++--
.../regionserver/ReplicationSource.java| 2 +-
.../hbase/client/TestAsyncTableNoncedRetry.java| 2 +-
.../TestRegionReplicaReplicationEndpoint.java | 56 +-
...stRegionReplicaReplicationEndpointNoMaster.java | 99 ++-
14 files changed, 627 insertions(+), 658 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
index ce6bfac..f6a2149 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
@@ -55,7 +55,6 @@ import
org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer;
import org.apache.hadoop.hbase.shaded.protobuf.RequestConverter;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService;
-import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.FlushRegionResponse;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ClientService;
import org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsMasterRunningResponse;
@@ -65,7 +64,7 @@ import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.MasterServ
* The implementation of AsyncConnection.
*/
@InterfaceAudience.Private
-class AsyncConnectionImpl implements AsyncClusterConnection {
+class AsyncConnectionImpl implements AsyncConnection {
private static final Logger LOG =
LoggerFactory.getLogger(AsyncConnectionImpl.class);
@@ -85,7 +84,7 @@ class AsyncConnectionImpl implements AsyncClusterConnection {
private final int rpcTimeout;
- private final RpcClient rpcClient;
+ protected final RpcClient rpcClient;
final RpcControllerFactory rpcControllerFactory;
@@ -160,16 +159,10 @@ class AsyncConnectionImpl implements
AsyncClusterConnection {
}
// ditto
- @Override
- public NonceGenerator getNonceGenerator() {
+ NonceGenerator getNonceGenerator() {
return nonceGenerator;
}
- @Override
- public RpcClient getRpcClient() {
-return rpcClient;
- }
-
private ClientService.Interface createRegionServerStub(ServerName
serverName) throws IOException {
return ClientService.newStub(rpcClient.createRpcChannel(serverName, user,
rpcTimeout));
}
@@ -360,15 +353,4 @@ class AsyncConnectionImpl implements
AsyncClusterConnection {
return new HBaseHbck(MasterProtos.HbckService.newBlockingStub(
rpcClient.createBlockingRpcChannel(masterServer, user, rpcTimeout)),
rpcControllerFactory);
}
-
- public AsyncRegionServerAdmin getRegionServerAdmin(ServerName serverName) {
-return new AsyncRegionServerAdmin(serverName, this);
- }
-
- @Override
- public CompletableFuture flush(byte[] regionName,
- boolean writeFlushWALMarker) {
-RawAsyncHBaseAdmin admin = (RawAsyncHBaseAdmin) getAdmin();
-return admin.flushRegionInternal(regionName, writeFlushWALMarker);
- }
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
similarity index 72%
rename from
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
rename to
hbase-server/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
index f1f64ca..0ad77ba 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
@@ -17,9 +17,13 @@
*/
package org.apache.hadoop.hbase.client;
+import java.util.List;
import java.util.concurrent.CompletableFuture;
+import org.apache.hadoop.hbase.RegionLocations;
import org.apache.hadoop.hbase.ServerName;
+import
[hbase] 03/08: HBASE-21526 Use AsyncClusterConnection in ServerManager for getRsAdmin
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch HBASE-21512
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit aeda3f8fdefd0a57ae899091657ca0a5f0affcb5
Author: zhangduo
AuthorDate: Thu Dec 6 21:25:34 2018 +0800
HBASE-21526 Use AsyncClusterConnection in ServerManager for getRsAdmin
---
.../hbase/client/AsyncClusterConnection.java | 6 +
.../hadoop/hbase/client/AsyncConnectionImpl.java | 4 +
.../hbase/client/AsyncRegionServerAdmin.java | 210 +
.../org/apache/hadoop/hbase/util/FutureUtils.java | 2 +-
.../org/apache/hadoop/hbase/master/HMaster.java| 15 +-
.../apache/hadoop/hbase/master/ServerManager.java | 67 ---
.../master/procedure/RSProcedureDispatcher.java| 44 +++--
7 files changed, 262 insertions(+), 86 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
index c7dea25..1327fd7 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
@@ -17,6 +17,7 @@
*/
package org.apache.hadoop.hbase.client;
+import org.apache.hadoop.hbase.ServerName;
import org.apache.hadoop.hbase.ipc.RpcClient;
import org.apache.yetus.audience.InterfaceAudience;
@@ -27,6 +28,11 @@ import org.apache.yetus.audience.InterfaceAudience;
public interface AsyncClusterConnection extends AsyncConnection {
/**
+ * Get the admin service for the given region server.
+ */
+ AsyncRegionServerAdmin getRegionServerAdmin(ServerName serverName);
+
+ /**
* Get the nonce generator for this connection.
*/
NonceGenerator getNonceGenerator();
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
index 50e27c4..9bead83 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
@@ -359,4 +359,8 @@ class AsyncConnectionImpl implements AsyncClusterConnection
{
return new HBaseHbck(MasterProtos.HbckService.newBlockingStub(
rpcClient.createBlockingRpcChannel(masterServer, user, rpcTimeout)),
rpcControllerFactory);
}
+
+ public AsyncRegionServerAdmin getRegionServerAdmin(ServerName serverName) {
+return new AsyncRegionServerAdmin(serverName, this);
+ }
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionServerAdmin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionServerAdmin.java
new file mode 100644
index 000..9accd89
--- /dev/null
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionServerAdmin.java
@@ -0,0 +1,210 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import java.io.IOException;
+import java.util.concurrent.CompletableFuture;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.ipc.HBaseRpcController;
+import org.apache.yetus.audience.InterfaceAudience;
+
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcCallback;
+import org.apache.hbase.thirdparty.com.google.protobuf.RpcController;
+
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.ClearCompactionQueuesRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.ClearCompactionQueuesResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.ClearRegionBlockCacheRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.ClearRegionBlockCacheResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.CloseRegionRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.CloseRegionResponse;
+import
[hbase] 04/08: HBASE-21579 Use AsyncClusterConnection for HBaseInterClusterReplicationEndpoint
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch HBASE-21512
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit c6ddd86a71e8581a8af8d03d818b720392a8ca23
Author: zhangduo
AuthorDate: Tue Jan 1 21:27:14 2019 +0800
HBASE-21579 Use AsyncClusterConnection for
HBaseInterClusterReplicationEndpoint
---
.../hbase/client/AsyncRegionServerAdmin.java | 14 +---
.../hbase/protobuf/ReplicationProtbufUtil.java | 35 ++-
.../HBaseInterClusterReplicationEndpoint.java | 31 +
.../regionserver/ReplicationSinkManager.java | 40 --
.../hbase/replication/SyncReplicationTestBase.java | 12 +++
.../regionserver/TestReplicationSinkManager.java | 21 +---
6 files changed, 74 insertions(+), 79 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionServerAdmin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionServerAdmin.java
index 9accd89..b9141a9 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionServerAdmin.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionServerAdmin.java
@@ -19,6 +19,7 @@ package org.apache.hadoop.hbase.client;
import java.io.IOException;
import java.util.concurrent.CompletableFuture;
+import org.apache.hadoop.hbase.CellScanner;
import org.apache.hadoop.hbase.ServerName;
import org.apache.hadoop.hbase.ipc.HBaseRpcController;
import org.apache.yetus.audience.InterfaceAudience;
@@ -94,9 +95,9 @@ public class AsyncRegionServerAdmin {
void call(AdminService.Interface stub, HBaseRpcController controller,
RpcCallback done);
}
- private CompletableFuture call(RpcCall rpcCall) {
+ private CompletableFuture call(RpcCall rpcCall,
CellScanner cellScanner) {
CompletableFuture future = new CompletableFuture<>();
-HBaseRpcController controller = conn.rpcControllerFactory.newController();
+HBaseRpcController controller =
conn.rpcControllerFactory.newController(cellScanner);
try {
rpcCall.call(conn.getAdminStub(server), controller, new
RpcCallback() {
@@ -115,6 +116,10 @@ public class AsyncRegionServerAdmin {
return future;
}
+ private CompletableFuture call(RpcCall rpcCall) {
+return call(rpcCall, null);
+ }
+
public CompletableFuture
getRegionInfo(GetRegionInfoRequest request) {
return call((stub, controller, done) -> stub.getRegionInfo(controller,
request, done));
}
@@ -154,8 +159,9 @@ public class AsyncRegionServerAdmin {
}
public CompletableFuture replicateWALEntry(
- ReplicateWALEntryRequest request) {
-return call((stub, controller, done) -> stub.replicateWALEntry(controller,
request, done));
+ ReplicateWALEntryRequest request, CellScanner cellScanner) {
+return call((stub, controller, done) -> stub.replicateWALEntry(controller,
request, done),
+ cellScanner);
}
public CompletableFuture
replay(ReplicateWALEntryRequest request) {
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java
index c1b3911..74fad26 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java
@@ -20,51 +20,54 @@ package org.apache.hadoop.hbase.protobuf;
import java.io.IOException;
+import java.io.InterruptedIOException;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
+import java.util.concurrent.ExecutionException;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CellScanner;
import org.apache.hadoop.hbase.PrivateCellUtil;
+import org.apache.hadoop.hbase.client.AsyncRegionServerAdmin;
import org.apache.hadoop.hbase.io.SizedCellScanner;
-import org.apache.hadoop.hbase.ipc.HBaseRpcController;
-import org.apache.hadoop.hbase.ipc.HBaseRpcControllerImpl;
import org.apache.hadoop.hbase.regionserver.wal.WALCellCodec;
import org.apache.hadoop.hbase.util.Pair;
import org.apache.hadoop.hbase.wal.WAL.Entry;
import org.apache.hadoop.hbase.wal.WALEdit;
import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.hbase.thirdparty.com.google.common.base.Throwables;
import org.apache.hbase.thirdparty.com.google.protobuf.UnsafeByteOperations;
import org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos;
-import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService;
import org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos;
@InterfaceAudience.Private
public class ReplicationProtbufUtil {
+
/**
- * A helper to replicate a list of WAL entries using admin protocol.
- * @param admin Admin service
+ * A helper to
[hbase] branch HBASE-21512 updated (be37dab -> c93f47c)
This is an automated email from the ASF dual-hosted git repository. zhangduo pushed a change to branch HBASE-21512 in repository https://gitbox.apache.org/repos/asf/hbase.git. discard be37dab HBASE-21537 Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection discard fa0ee54 HBASE-21671 Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection discard ea64a91 HBASE-21538 Rewrite RegionReplicaFlushHandler to use AsyncClusterConnection discard a34f83c HBASE-21579 Use AsyncClusterConnection for HBaseInterClusterReplicationEndpoint discard 680327f HBASE-21526 Use AsyncClusterConnection in ServerManager for getRsAdmin discard 3300b61 HBASE-21516 Use AsyncConnection instead of Connection in SecureBulkLoadManager discard d95304e HBASE-21515 Also initialize an AsyncClusterConnection in HRegionServer add 63a1fcb HBASE-21712 : Make submit-patch.py python3 compatible add e69ab24 HBASE-21626 : log the regions blocking WAL from being archived add 54c724f HBASE-21707 Addendum remove debug logs add 93d4b95 HBASE-21225: Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE' add 0fd4243 HBASE-21634: Print error message when user uses unacceptable values for LIMIT while setting quotas. new 910a6b4 HBASE-21515 Also initialize an AsyncClusterConnection in HRegionServer new 6faad94 HBASE-21516 Use AsyncConnection instead of Connection in SecureBulkLoadManager new aeda3f8 HBASE-21526 Use AsyncClusterConnection in ServerManager for getRsAdmin new c6ddd86 HBASE-21579 Use AsyncClusterConnection for HBaseInterClusterReplicationEndpoint new dd41313 HBASE-21538 Rewrite RegionReplicaFlushHandler to use AsyncClusterConnection new 6786c49 HBASE-21671 Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection new 446f9ad HBASE-21537 Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection new c93f47c HBASE-21719 Rewrite RegionPlacementMaintainer to use AsyncClusterConnection This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (be37dab) \ N -- N -- N refs/heads/HBASE-21512 (c93f47c) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 8 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .pylintrc | 3 + dev-support/submit-patch.py| 7 +- .../hadoop/hbase/quotas/SpaceLimitSettings.java| 15 +- .../hbase/rsgroup/RSGroupBasedLoadBalancer.java| 2 - .../hbase/master/RegionPlacementMaintainer.java| 225 +++-- .../hbase/quotas/GlobalQuotaSettingsImpl.java | 23 +-- .../hbase/regionserver/wal/AbstractFSWAL.java | 46 - .../regionserver/wal/SequenceIdAccounting.java | 15 +- .../hbase/quotas/TestMasterQuotasObserver.java | 61 +- .../regionserver/wal/TestSequenceIdAccounting.java | 24 ++- hbase-shell/src/main/ruby/hbase/quotas.rb | 23 ++- hbase-shell/src/test/ruby/hbase/quotas_test.rb | 86 +++- 12 files changed, 363 insertions(+), 167 deletions(-)
[hbase] 02/08: HBASE-21516 Use AsyncConnection instead of Connection in SecureBulkLoadManager
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch HBASE-21512
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 6faad94cd3e9ea807e2fa5d7872a6b97c6bd5e7b
Author: zhangduo
AuthorDate: Sat Dec 1 21:15:48 2018 +0800
HBASE-21516 Use AsyncConnection instead of Connection in
SecureBulkLoadManager
---
.../apache/hadoop/hbase/protobuf/ProtobufUtil.java | 5 +-
.../hadoop/hbase/shaded/protobuf/ProtobufUtil.java | 7 ++-
.../hadoop/hbase/regionserver/HRegionServer.java | 2 +-
.../hbase/regionserver/SecureBulkLoadManager.java | 24 -
.../hadoop/hbase/security/token/TokenUtil.java | 57 +-
.../hadoop/hbase/security/token/TestTokenUtil.java | 42
6 files changed, 96 insertions(+), 41 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
index a3d49b5..d9e620b 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
@@ -261,13 +261,12 @@ public final class ProtobufUtil {
* just {@link ServiceException}. Prefer this method to
* {@link #getRemoteException(ServiceException)} because trying to
* contain direct protobuf references.
- * @param e
*/
- public static IOException handleRemoteException(Exception e) {
+ public static IOException handleRemoteException(Throwable e) {
return makeIOExceptionOfException(e);
}
- private static IOException makeIOExceptionOfException(Exception e) {
+ private static IOException makeIOExceptionOfException(Throwable e) {
Throwable t = e;
if (e instanceof ServiceException ||
e instanceof
org.apache.hbase.thirdparty.com.google.protobuf.ServiceException) {
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
index fea81f1..de2fb7d 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
@@ -40,7 +40,6 @@ import java.util.concurrent.TimeUnit;
import java.util.function.Function;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
-
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.ByteBufferExtendedCell;
@@ -123,6 +122,7 @@ import
org.apache.hbase.thirdparty.com.google.protobuf.Service;
import org.apache.hbase.thirdparty.com.google.protobuf.ServiceException;
import org.apache.hbase.thirdparty.com.google.protobuf.TextFormat;
import org.apache.hbase.thirdparty.com.google.protobuf.UnsafeByteOperations;
+
import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.CloseRegionRequest;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.GetOnlineRegionRequest;
@@ -343,13 +343,12 @@ public final class ProtobufUtil {
* just {@link ServiceException}. Prefer this method to
* {@link #getRemoteException(ServiceException)} because trying to
* contain direct protobuf references.
- * @param e
*/
- public static IOException handleRemoteException(Exception e) {
+ public static IOException handleRemoteException(Throwable e) {
return makeIOExceptionOfException(e);
}
- private static IOException makeIOExceptionOfException(Exception e) {
+ private static IOException makeIOExceptionOfException(Throwable e) {
Throwable t = e;
if (e instanceof ServiceException) {
t = e.getCause();
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
index b4b1d3e..bbc9d3c 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
@@ -1937,7 +1937,7 @@ public class HRegionServer extends HasThread implements
if (!isStopped() && !isAborted()) {
initializeThreads();
}
-this.secureBulkLoadManager = new SecureBulkLoadManager(this.conf,
clusterConnection);
+this.secureBulkLoadManager = new SecureBulkLoadManager(this.conf,
asyncClusterConnection);
this.secureBulkLoadManager.start();
// Health checker thread.
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SecureBulkLoadManager.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SecureBulkLoadManager.java
index 566a6b6..add6519 100644
---
[hbase] 05/08: HBASE-21538 Rewrite RegionReplicaFlushHandler to use AsyncClusterConnection
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch HBASE-21512
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit dd41313bdfa63d817136f03d3ff81dd93a575585
Author: Duo Zhang
AuthorDate: Wed Dec 12 09:33:33 2018 +0800
HBASE-21538 Rewrite RegionReplicaFlushHandler to use AsyncClusterConnection
---
.../hbase/client/AsyncClusterConnection.java | 8 ++
.../hadoop/hbase/client/AsyncConnectionImpl.java | 8 ++
.../hbase/client/ClusterConnectionFactory.java | 16 +--
.../hadoop/hbase/client/RawAsyncHBaseAdmin.java| 36 ---
.../org/apache/hadoop/hbase/util/FutureUtils.java | 22 +
.../master/procedure/RSProcedureDispatcher.java| 34 +--
.../hbase/protobuf/ReplicationProtbufUtil.java | 15 +--
.../hadoop/hbase/regionserver/HRegionServer.java | 3 +-
.../handler/RegionReplicaFlushHandler.java | 110 -
9 files changed, 132 insertions(+), 120 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
index 1327fd7..f1f64ca 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
@@ -17,10 +17,13 @@
*/
package org.apache.hadoop.hbase.client;
+import java.util.concurrent.CompletableFuture;
import org.apache.hadoop.hbase.ServerName;
import org.apache.hadoop.hbase.ipc.RpcClient;
import org.apache.yetus.audience.InterfaceAudience;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.FlushRegionResponse;
+
/**
* The asynchronous connection for internal usage.
*/
@@ -41,4 +44,9 @@ public interface AsyncClusterConnection extends
AsyncConnection {
* Get the rpc client we used to communicate with other servers.
*/
RpcClient getRpcClient();
+
+ /**
+ * Flush a region and get the response.
+ */
+ CompletableFuture flush(byte[] regionName, boolean
writeFlushWALMarker);
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
index 9bead83..ce6bfac 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
@@ -55,6 +55,7 @@ import
org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer;
import org.apache.hadoop.hbase.shaded.protobuf.RequestConverter;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.FlushRegionResponse;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ClientService;
import org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsMasterRunningResponse;
@@ -363,4 +364,11 @@ class AsyncConnectionImpl implements
AsyncClusterConnection {
public AsyncRegionServerAdmin getRegionServerAdmin(ServerName serverName) {
return new AsyncRegionServerAdmin(serverName, this);
}
+
+ @Override
+ public CompletableFuture flush(byte[] regionName,
+ boolean writeFlushWALMarker) {
+RawAsyncHBaseAdmin admin = (RawAsyncHBaseAdmin) getAdmin();
+return admin.flushRegionInternal(regionName, writeFlushWALMarker);
+ }
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClusterConnectionFactory.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClusterConnectionFactory.java
index 68c0630..79484db 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClusterConnectionFactory.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClusterConnectionFactory.java
@@ -18,15 +18,12 @@
package org.apache.hadoop.hbase.client;
import java.io.IOException;
-import java.io.InterruptedIOException;
import java.net.SocketAddress;
-import java.util.concurrent.ExecutionException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.util.FutureUtils;
import org.apache.yetus.audience.InterfaceAudience;
-import org.apache.hbase.thirdparty.com.google.common.base.Throwables;
-
/**
* The factory for creating {@link AsyncClusterConnection}.
*/
@@ -48,16 +45,7 @@ public final class ClusterConnectionFactory {
public static AsyncClusterConnection
createAsyncClusterConnection(Configuration conf,
SocketAddress localAddress, User user) throws IOException {
AsyncRegistry registry = AsyncRegistryFactory.getRegistry(conf);
-String clusterId;
-try {
- clusterId = registry.getClusterId().get();
-} catch
[hbase] 01/08: HBASE-21515 Also initialize an AsyncClusterConnection in HRegionServer
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch HBASE-21512
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 910a6b446517bd10cb3fd064706b12d925d6668f
Author: zhangduo
AuthorDate: Fri Nov 30 08:23:47 2018 +0800
HBASE-21515 Also initialize an AsyncClusterConnection in HRegionServer
---
.../hbase/client/AsyncClusterConnection.java | 38 +
.../hadoop/hbase/client/AsyncConnectionImpl.java | 14 +++--
.../hbase/client/ClusterConnectionFactory.java | 63 ++
.../hadoop/hbase/client/ConnectionFactory.java | 5 +-
.../apache/hadoop/hbase/util/ReflectionUtils.java | 22
.../main/java/org/apache/hadoop/hbase/Server.java | 20 +++
.../org/apache/hadoop/hbase/master/HMaster.java| 3 ++
.../hadoop/hbase/regionserver/HRegionServer.java | 56 +--
.../regionserver/ReplicationSyncUp.java| 6 +++
.../hadoop/hbase/MockRegionServerServices.java | 5 ++
.../client/TestAsyncNonMetaRegionLocator.java | 2 +-
...stAsyncNonMetaRegionLocatorConcurrenyLimit.java | 2 +-
.../client/TestAsyncRegionLocatorTimeout.java | 2 +-
.../TestAsyncSingleRequestRpcRetryingCaller.java | 4 +-
.../hbase/client/TestAsyncTableNoncedRetry.java| 2 +-
.../hbase/master/MockNoopMasterServices.java | 6 +++
.../hadoop/hbase/master/MockRegionServer.java | 5 ++
.../hbase/master/TestActiveMasterManager.java | 6 +++
.../hbase/master/cleaner/TestHFileCleaner.java | 6 +++
.../hbase/master/cleaner/TestHFileLinkCleaner.java | 6 +++
.../hbase/master/cleaner/TestLogsCleaner.java | 6 +++
.../cleaner/TestReplicationHFileCleaner.java | 6 +++
.../hbase/regionserver/TestHeapMemoryManager.java | 6 +++
.../hbase/regionserver/TestSplitLogWorker.java | 6 +++
.../hadoop/hbase/regionserver/TestWALLockup.java | 6 +++
.../replication/TestReplicationTrackerZKImpl.java | 6 +++
.../regionserver/TestReplicationSourceManager.java | 6 +++
.../security/token/TestTokenAuthentication.java| 6 +++
.../org/apache/hadoop/hbase/util/MockServer.java | 6 +++
29 files changed, 290 insertions(+), 37 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
new file mode 100644
index 000..c7dea25
--- /dev/null
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClusterConnection.java
@@ -0,0 +1,38 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import org.apache.hadoop.hbase.ipc.RpcClient;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * The asynchronous connection for internal usage.
+ */
+@InterfaceAudience.Private
+public interface AsyncClusterConnection extends AsyncConnection {
+
+ /**
+ * Get the nonce generator for this connection.
+ */
+ NonceGenerator getNonceGenerator();
+
+ /**
+ * Get the rpc client we used to communicate with other servers.
+ */
+ RpcClient getRpcClient();
+}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
index 3cbd950..50e27c4 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
@@ -23,6 +23,7 @@ import static
org.apache.hadoop.hbase.client.NonceGenerator.CLIENT_NONCES_ENABLE
import static org.apache.hadoop.hbase.util.FutureUtils.addListener;
import java.io.IOException;
+import java.net.SocketAddress;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
@@ -63,7 +64,7 @@ import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.MasterServ
* The implementation of AsyncConnection.
*/
@InterfaceAudience.Private
-class AsyncConnectionImpl implements AsyncConnection {
+class AsyncConnectionImpl implements
[hbase] 08/08: HBASE-21719 Rewrite RegionPlacementMaintainer to use AsyncClusterConnection
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch HBASE-21512
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit c93f47cfd137748dc01006acdc4249cf582ddcb1
Author: Duo Zhang
AuthorDate: Tue Jan 15 11:43:41 2019 +0800
HBASE-21719 Rewrite RegionPlacementMaintainer to use AsyncClusterConnection
Signed-off-by: Michael Stack
---
.../hbase/master/RegionPlacementMaintainer.java| 225 +++--
1 file changed, 113 insertions(+), 112 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionPlacementMaintainer.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionPlacementMaintainer.java
index faf5e4a..fda0a9c 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionPlacementMaintainer.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionPlacementMaintainer.java
@@ -1,5 +1,4 @@
/**
- *
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
@@ -16,9 +15,9 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-
package org.apache.hadoop.hbase.master;
+import java.io.Closeable;
import java.io.IOException;
import java.text.DecimalFormat;
import java.util.ArrayList;
@@ -39,29 +38,30 @@ import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HConstants;
import org.apache.hadoop.hbase.ServerName;
import org.apache.hadoop.hbase.TableName;
-import org.apache.hadoop.hbase.client.Admin;
-import org.apache.hadoop.hbase.client.ClusterConnection;
-import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.AsyncClusterConnection;
+import org.apache.hadoop.hbase.client.AsyncRegionServerAdmin;
+import org.apache.hadoop.hbase.client.ClusterConnectionFactory;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.RegionInfo;
import org.apache.hadoop.hbase.favored.FavoredNodeAssignmentHelper;
import org.apache.hadoop.hbase.favored.FavoredNodesPlan;
+import org.apache.hadoop.hbase.security.User;
import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hbase.util.FutureUtils;
import org.apache.hadoop.hbase.util.MunkresAssignment;
import org.apache.hadoop.hbase.util.Pair;
import org.apache.yetus.audience.InterfaceAudience;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import org.apache.hbase.thirdparty.com.google.common.io.Closeables;
import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine;
import org.apache.hbase.thirdparty.org.apache.commons.cli.GnuParser;
import org.apache.hbase.thirdparty.org.apache.commons.cli.HelpFormatter;
import org.apache.hbase.thirdparty.org.apache.commons.cli.Options;
import org.apache.hbase.thirdparty.org.apache.commons.cli.ParseException;
-import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
import org.apache.hadoop.hbase.shaded.protobuf.RequestConverter;
-import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService.BlockingInterface;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.UpdateFavoredNodesRequest;
import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.UpdateFavoredNodesResponse;
@@ -71,7 +71,7 @@ import
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.UpdateFavor
*/
@InterfaceAudience.Private
// TODO: Remove? Unused. Partially implemented only.
-public class RegionPlacementMaintainer {
+public class RegionPlacementMaintainer implements Closeable {
private static final Logger LOG =
LoggerFactory.getLogger(RegionPlacementMaintainer.class
.getName());
//The cost of a placement that should never be assigned.
@@ -96,9 +96,9 @@ public class RegionPlacementMaintainer {
private final boolean enforceMinAssignmentMove;
private RackManager rackManager;
private Set targetTableSet;
- private final Connection connection;
+ private AsyncClusterConnection connection;
- public RegionPlacementMaintainer(Configuration conf) {
+ public RegionPlacementMaintainer(Configuration conf) throws IOException {
this(conf, true, true);
}
@@ -109,11 +109,6 @@ public class RegionPlacementMaintainer {
this.enforceMinAssignmentMove = enforceMinAssignmentMove;
this.targetTableSet = new HashSet<>();
this.rackManager = new RackManager(conf);
-try {
- this.connection = ConnectionFactory.createConnection(this.conf);
-} catch (IOException e) {
- throw new RuntimeException(e);
-}
}
private static void printHelp(Options opt) {
@@ -124,6 +119,14 @@ public class RegionPlacementMaintainer {
" [-fs hdfs://a.b.c.d:9000] [-hbase_root /HBASE]", opt);
}
+ private AsyncClusterConnection getConnection() throws
[hbase] branch branch-1.2 updated: HBASE-21685 Change repository urls to Gitbox
This is an automated email from the ASF dual-hosted git repository. psomogyi pushed a commit to branch branch-1.2 in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/branch-1.2 by this push: new 1784b14 HBASE-21685 Change repository urls to Gitbox 1784b14 is described below commit 1784b1427a738a0761acf72274b100c2e5889456 Author: Peter Somogyi AuthorDate: Tue Jan 15 10:44:45 2019 +0100 HBASE-21685 Change repository urls to Gitbox Signed-off-by: Sean Busbey --- pom.xml | 4 ++-- src/main/asciidoc/_chapters/rpc.adoc | 8 +++- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/pom.xml b/pom.xml index 2c2ce3b..9c07078 100644 --- a/pom.xml +++ b/pom.xml @@ -91,8 +91,8 @@ scm:git:git://git.apache.org/hbase.git - scm:git:https://git-wip-us.apache.org/repos/asf/hbase.git -https://git-wip-us.apache.org/repos/asf?p=hbase.git + scm:git:https://gitbox.apache.org/repos/asf/hbase.git +https://gitbox.apache.org/repos/asf?p=hbase.git JIRA diff --git a/src/main/asciidoc/_chapters/rpc.adoc b/src/main/asciidoc/_chapters/rpc.adoc index 1d363eb..ee73c53 100644 --- a/src/main/asciidoc/_chapters/rpc.adoc +++ b/src/main/asciidoc/_chapters/rpc.adoc @@ -66,13 +66,11 @@ The client will send setup information on connection establish. Thereafter, the client invokes methods against the remote server sending a protobuf Message and receiving a protobuf Message in response. Communication is synchronous. All back and forth is preceded by an int that has the total length of the request/response. -Optionally, Cells(KeyValues) can be passed outside of protobufs in follow-behind Cell blocks (because link:https://docs.google.com/document/d/1WEtrq-JTIUhlnlnvA0oYRLp0F8MKpEBeBSCFcQiacdw/edit#[we -can't protobuf megabytes of KeyValues] or Cells). These CellBlocks are encoded and optionally compressed. - - +Optionally, Cells(KeyValues) can be passed outside of protobufs in follow-behind Cell blocks +(because link:https://docs.google.com/document/d/1WEtrq-JTIUhlnlnvA0oYRLp0F8MKpEBeBSCFcQiacdw/edit#[we can't protobuf megabytes of KeyValues] or Cells). These CellBlocks are encoded and optionally compressed. For more detail on the protobufs involved, see the -link:https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=blob;f=hbase-protocol/src/main/protobuf/RPC.proto;hb=HEAD[RPC.proto] file in master. +link:https://github.com/apache/hbase/blob/master/hbase-protocol/src/main/protobuf/RPC.proto[RPC.proto] file in master. Connection Setup
[hbase] branch branch-1.3 updated: HBASE-21685 Change repository urls to Gitbox
This is an automated email from the ASF dual-hosted git repository. psomogyi pushed a commit to branch branch-1.3 in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/branch-1.3 by this push: new c13a2ff HBASE-21685 Change repository urls to Gitbox c13a2ff is described below commit c13a2ff7fa24105000baf42cbbe4df502359d31e Author: Peter Somogyi AuthorDate: Tue Jan 15 10:44:45 2019 +0100 HBASE-21685 Change repository urls to Gitbox Signed-off-by: Sean Busbey --- pom.xml | 4 ++-- src/main/asciidoc/_chapters/rpc.adoc | 8 +++- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/pom.xml b/pom.xml index 27f0864..7b84e92 100644 --- a/pom.xml +++ b/pom.xml @@ -92,8 +92,8 @@ scm:git:git://git.apache.org/hbase.git - scm:git:https://git-wip-us.apache.org/repos/asf/hbase.git -https://git-wip-us.apache.org/repos/asf?p=hbase.git + scm:git:https://gitbox.apache.org/repos/asf/hbase.git +https://gitbox.apache.org/repos/asf?p=hbase.git JIRA diff --git a/src/main/asciidoc/_chapters/rpc.adoc b/src/main/asciidoc/_chapters/rpc.adoc index 1d363eb..ee73c53 100644 --- a/src/main/asciidoc/_chapters/rpc.adoc +++ b/src/main/asciidoc/_chapters/rpc.adoc @@ -66,13 +66,11 @@ The client will send setup information on connection establish. Thereafter, the client invokes methods against the remote server sending a protobuf Message and receiving a protobuf Message in response. Communication is synchronous. All back and forth is preceded by an int that has the total length of the request/response. -Optionally, Cells(KeyValues) can be passed outside of protobufs in follow-behind Cell blocks (because link:https://docs.google.com/document/d/1WEtrq-JTIUhlnlnvA0oYRLp0F8MKpEBeBSCFcQiacdw/edit#[we -can't protobuf megabytes of KeyValues] or Cells). These CellBlocks are encoded and optionally compressed. - - +Optionally, Cells(KeyValues) can be passed outside of protobufs in follow-behind Cell blocks +(because link:https://docs.google.com/document/d/1WEtrq-JTIUhlnlnvA0oYRLp0F8MKpEBeBSCFcQiacdw/edit#[we can't protobuf megabytes of KeyValues] or Cells). These CellBlocks are encoded and optionally compressed. For more detail on the protobufs involved, see the -link:https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=blob;f=hbase-protocol/src/main/protobuf/RPC.proto;hb=HEAD[RPC.proto] file in master. +link:https://github.com/apache/hbase/blob/master/hbase-protocol/src/main/protobuf/RPC.proto[RPC.proto] file in master. Connection Setup
[hbase] branch branch-1.4 updated: HBASE-21685 Change repository urls to Gitbox
This is an automated email from the ASF dual-hosted git repository. psomogyi pushed a commit to branch branch-1.4 in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/branch-1.4 by this push: new 7cf53cc HBASE-21685 Change repository urls to Gitbox 7cf53cc is described below commit 7cf53cca103f42cbb561f76f7d59146e1860201c Author: Peter Somogyi AuthorDate: Tue Jan 15 10:44:45 2019 +0100 HBASE-21685 Change repository urls to Gitbox Signed-off-by: Sean Busbey --- pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index 4a88765..76103af 100644 --- a/pom.xml +++ b/pom.xml @@ -94,8 +94,8 @@ scm:git:git://git.apache.org/hbase.git - scm:git:https://git-wip-us.apache.org/repos/asf/hbase.git -https://git-wip-us.apache.org/repos/asf?p=hbase.git + scm:git:https://gitbox.apache.org/repos/asf/hbase.git +https://gitbox.apache.org/repos/asf?p=hbase.git JIRA
[hbase] branch branch-1 updated: HBASE-21685 Change repository urls to Gitbox
This is an automated email from the ASF dual-hosted git repository. psomogyi pushed a commit to branch branch-1 in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/branch-1 by this push: new 936ff59 HBASE-21685 Change repository urls to Gitbox 936ff59 is described below commit 936ff59b76cc4f63fe7dff540a5c6eb22e8fe89d Author: Peter Somogyi AuthorDate: Tue Jan 15 10:44:45 2019 +0100 HBASE-21685 Change repository urls to Gitbox Signed-off-by: Sean Busbey --- pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index 10ac12b..8a8fc00 100644 --- a/pom.xml +++ b/pom.xml @@ -94,8 +94,8 @@ scm:git:git://git.apache.org/hbase.git - scm:git:https://git-wip-us.apache.org/repos/asf/hbase.git -https://git-wip-us.apache.org/repos/asf?p=hbase.git + scm:git:https://gitbox.apache.org/repos/asf/hbase.git +https://gitbox.apache.org/repos/asf?p=hbase.git JIRA
[hbase] branch branch-2.0 updated: HBASE-21685 Change repository urls to Gitbox
This is an automated email from the ASF dual-hosted git repository. psomogyi pushed a commit to branch branch-2.0 in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/branch-2.0 by this push: new a183d9d HBASE-21685 Change repository urls to Gitbox a183d9d is described below commit a183d9d665fa92fc02990df78c5a361466e00db9 Author: Peter Somogyi AuthorDate: Tue Jan 8 11:15:54 2019 +0100 HBASE-21685 Change repository urls to Gitbox Signed-off-by: Sean Busbey --- dev-support/jenkins-scripts/generate-hbase-website.sh | 2 +- pom.xml | 4 ++-- src/main/asciidoc/_chapters/developer.adoc| 19 +-- src/main/asciidoc/_chapters/rpc.adoc | 8 +++- src/main/site/resources/doap_Hbase.rdf| 2 +- 5 files changed, 16 insertions(+), 19 deletions(-) diff --git a/dev-support/jenkins-scripts/generate-hbase-website.sh b/dev-support/jenkins-scripts/generate-hbase-website.sh index 0ef9b2d..3b9ce7a 100644 --- a/dev-support/jenkins-scripts/generate-hbase-website.sh +++ b/dev-support/jenkins-scripts/generate-hbase-website.sh @@ -142,7 +142,7 @@ echo "Current HBase commit: $CURRENT_HBASE_COMMIT" # Clone the hbase-site repo manually so it doesn't trigger spurious # commits in Jenkins. -git clone --depth 1 --branch asf-site https://git-wip-us.apache.org/repos/asf/hbase-site.git +git clone --depth 1 --branch asf-site https://gitbox.apache.org/repos/asf/hbase-site.git # Figure out if the commit of the hbase repo has already been built and bail if so. declare -i PUSHED diff --git a/pom.xml b/pom.xml index 6e92991..e7f6552 100755 --- a/pom.xml +++ b/pom.xml @@ -101,8 +101,8 @@ scm:git:git://git.apache.org/hbase.git - scm:git:https://git-wip-us.apache.org/repos/asf/hbase.git -https://git-wip-us.apache.org/repos/asf?p=hbase.git + scm:git:https://gitbox.apache.org/repos/asf/hbase.git +https://gitbox.apache.org/repos/asf?p=hbase.git JIRA diff --git a/src/main/asciidoc/_chapters/developer.adoc b/src/main/asciidoc/_chapters/developer.adoc index 51ed461..f728236 100644 --- a/src/main/asciidoc/_chapters/developer.adoc +++ b/src/main/asciidoc/_chapters/developer.adoc @@ -168,13 +168,14 @@ code snippet [[repos]] == Apache HBase Repositories -There are two different repositories for Apache HBase: Subversion (SVN) and Git. -GIT is our repository of record for all but the Apache HBase website. -We used to be on SVN. -We migrated. -See link:https://issues.apache.org/jira/browse/INFRA-7768[Migrate Apache HBase SVN Repos to Git]. -See link:https://hbase.apache.org/source-repository.html[Source Code -Management] page for contributor and committer links or search for HBase on the link:https://git.apache.org/[Apache Git] page. +Apache HBase consists of multiple repositories which are hosted on https://gitbox.apache.org/[Apache GitBox]. +These are the following: + +* https://gitbox.apache.org/repos/asf?p=hbase.git[hbase] - main Apache HBase repository +* https://gitbox.apache.org/repos/asf?p=hbase-connectors.git[hbase-connectors] - connectors to Apache Kafka and Apache Spark +* https://gitbox.apache.org/repos/asf?p=hbase-operator-tools.git[hbase-operator-tools] - operability and supportability tools, such as <> +* https://gitbox.apache.org/repos/asf?p=hbase-site.git[hbase-site] - hbase.apache.org website +* https://gitbox.apache.org/repos/asf?p=hbase-thirdparty.git[hbase-thirdparty] - relocated versions of popular third-party libraries == IDEs @@ -1794,7 +1795,7 @@ If you submit a patch for one thing, don't do auto-reformatting or unrelated ref Likewise, don't add unrelated cleanup or refactorings outside the scope of your Jira. [[common.patch.feedback.tests]] -= Ambigious Unit Tests += Ambiguous Unit Tests Make sure that you're clear about what you are testing in your unit tests and why. @@ -1920,8 +1921,6 @@ If you are new to submitting patches to open source or new to submitting patches reading the link:https://commons.apache.org/patches.html[On Contributing Patches] page from link:https://commons.apache.org/[Apache Commons Project]. It provides a nice overview that applies equally to the Apache HBase Project. -link:https://accumulo.apache.org/git.html[Accumulo doc on how to contribute and develop] is also -good read to understand development workflow. [[submitting.patches.create]] Create Patch diff --git a/src/main/asciidoc/_chapters/rpc.adoc b/src/main/asciidoc/_chapters/rpc.adoc index fbfba6c..1926c6f 100644 --- a/src/main/asciidoc/_chapters/rpc.adoc +++ b/src/main/asciidoc/_chapters/rpc.adoc @@ -66,13 +66,11 @@ The client will send setup information on connection establish. Thereafter, the client invokes methods against the remote server sending a protobuf Message and receiving a protobuf Message in response. Communication is
[hbase] branch branch-2.1 updated: HBASE-21685 Change repository urls to Gitbox
This is an automated email from the ASF dual-hosted git repository. psomogyi pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new e1b5dd5 HBASE-21685 Change repository urls to Gitbox e1b5dd5 is described below commit e1b5dd5d9145be18748b393e904293c943410dbd Author: Peter Somogyi AuthorDate: Tue Jan 8 11:15:54 2019 +0100 HBASE-21685 Change repository urls to Gitbox Signed-off-by: Sean Busbey --- dev-support/jenkins-scripts/generate-hbase-website.sh | 2 +- pom.xml | 4 ++-- src/main/asciidoc/_chapters/developer.adoc| 19 +-- src/main/asciidoc/_chapters/rpc.adoc | 8 +++- src/site/resources/doap_Hbase.rdf | 2 +- 5 files changed, 16 insertions(+), 19 deletions(-) diff --git a/dev-support/jenkins-scripts/generate-hbase-website.sh b/dev-support/jenkins-scripts/generate-hbase-website.sh index 0ef9b2d..3b9ce7a 100644 --- a/dev-support/jenkins-scripts/generate-hbase-website.sh +++ b/dev-support/jenkins-scripts/generate-hbase-website.sh @@ -142,7 +142,7 @@ echo "Current HBase commit: $CURRENT_HBASE_COMMIT" # Clone the hbase-site repo manually so it doesn't trigger spurious # commits in Jenkins. -git clone --depth 1 --branch asf-site https://git-wip-us.apache.org/repos/asf/hbase-site.git +git clone --depth 1 --branch asf-site https://gitbox.apache.org/repos/asf/hbase-site.git # Figure out if the commit of the hbase repo has already been built and bail if so. declare -i PUSHED diff --git a/pom.xml b/pom.xml index 53065f2..ea5816b 100755 --- a/pom.xml +++ b/pom.xml @@ -101,8 +101,8 @@ scm:git:git://git.apache.org/hbase.git - scm:git:https://git-wip-us.apache.org/repos/asf/hbase.git -https://git-wip-us.apache.org/repos/asf?p=hbase.git + scm:git:https://gitbox.apache.org/repos/asf/hbase.git +https://gitbox.apache.org/repos/asf?p=hbase.git JIRA diff --git a/src/main/asciidoc/_chapters/developer.adoc b/src/main/asciidoc/_chapters/developer.adoc index 51ed461..f728236 100644 --- a/src/main/asciidoc/_chapters/developer.adoc +++ b/src/main/asciidoc/_chapters/developer.adoc @@ -168,13 +168,14 @@ code snippet [[repos]] == Apache HBase Repositories -There are two different repositories for Apache HBase: Subversion (SVN) and Git. -GIT is our repository of record for all but the Apache HBase website. -We used to be on SVN. -We migrated. -See link:https://issues.apache.org/jira/browse/INFRA-7768[Migrate Apache HBase SVN Repos to Git]. -See link:https://hbase.apache.org/source-repository.html[Source Code -Management] page for contributor and committer links or search for HBase on the link:https://git.apache.org/[Apache Git] page. +Apache HBase consists of multiple repositories which are hosted on https://gitbox.apache.org/[Apache GitBox]. +These are the following: + +* https://gitbox.apache.org/repos/asf?p=hbase.git[hbase] - main Apache HBase repository +* https://gitbox.apache.org/repos/asf?p=hbase-connectors.git[hbase-connectors] - connectors to Apache Kafka and Apache Spark +* https://gitbox.apache.org/repos/asf?p=hbase-operator-tools.git[hbase-operator-tools] - operability and supportability tools, such as <> +* https://gitbox.apache.org/repos/asf?p=hbase-site.git[hbase-site] - hbase.apache.org website +* https://gitbox.apache.org/repos/asf?p=hbase-thirdparty.git[hbase-thirdparty] - relocated versions of popular third-party libraries == IDEs @@ -1794,7 +1795,7 @@ If you submit a patch for one thing, don't do auto-reformatting or unrelated ref Likewise, don't add unrelated cleanup or refactorings outside the scope of your Jira. [[common.patch.feedback.tests]] -= Ambigious Unit Tests += Ambiguous Unit Tests Make sure that you're clear about what you are testing in your unit tests and why. @@ -1920,8 +1921,6 @@ If you are new to submitting patches to open source or new to submitting patches reading the link:https://commons.apache.org/patches.html[On Contributing Patches] page from link:https://commons.apache.org/[Apache Commons Project]. It provides a nice overview that applies equally to the Apache HBase Project. -link:https://accumulo.apache.org/git.html[Accumulo doc on how to contribute and develop] is also -good read to understand development workflow. [[submitting.patches.create]] Create Patch diff --git a/src/main/asciidoc/_chapters/rpc.adoc b/src/main/asciidoc/_chapters/rpc.adoc index fbfba6c..1926c6f 100644 --- a/src/main/asciidoc/_chapters/rpc.adoc +++ b/src/main/asciidoc/_chapters/rpc.adoc @@ -66,13 +66,11 @@ The client will send setup information on connection establish. Thereafter, the client invokes methods against the remote server sending a protobuf Message and receiving a protobuf Message in response. Communication is
[hbase] branch branch-2 updated: HBASE-21685 Change repository urls to Gitbox
This is an automated email from the ASF dual-hosted git repository. psomogyi pushed a commit to branch branch-2 in repository https://gitbox.apache.org/repos/asf/hbase.git The following commit(s) were added to refs/heads/branch-2 by this push: new 1200733 HBASE-21685 Change repository urls to Gitbox 1200733 is described below commit 1200733cd9ad4ec2bd1949cc82cd03fb817c14b5 Author: Peter Somogyi AuthorDate: Tue Jan 8 11:15:54 2019 +0100 HBASE-21685 Change repository urls to Gitbox Signed-off-by: Sean Busbey --- dev-support/jenkins-scripts/generate-hbase-website.sh | 2 +- pom.xml | 4 ++-- src/main/asciidoc/_chapters/developer.adoc| 19 +-- src/main/asciidoc/_chapters/rpc.adoc | 8 +++- src/site/resources/doap_Hbase.rdf | 2 +- 5 files changed, 16 insertions(+), 19 deletions(-) diff --git a/dev-support/jenkins-scripts/generate-hbase-website.sh b/dev-support/jenkins-scripts/generate-hbase-website.sh index 0ef9b2d..3b9ce7a 100644 --- a/dev-support/jenkins-scripts/generate-hbase-website.sh +++ b/dev-support/jenkins-scripts/generate-hbase-website.sh @@ -142,7 +142,7 @@ echo "Current HBase commit: $CURRENT_HBASE_COMMIT" # Clone the hbase-site repo manually so it doesn't trigger spurious # commits in Jenkins. -git clone --depth 1 --branch asf-site https://git-wip-us.apache.org/repos/asf/hbase-site.git +git clone --depth 1 --branch asf-site https://gitbox.apache.org/repos/asf/hbase-site.git # Figure out if the commit of the hbase repo has already been built and bail if so. declare -i PUSHED diff --git a/pom.xml b/pom.xml index 91cc740..f54de61 100755 --- a/pom.xml +++ b/pom.xml @@ -101,8 +101,8 @@ scm:git:git://git.apache.org/hbase.git - scm:git:https://git-wip-us.apache.org/repos/asf/hbase.git -https://git-wip-us.apache.org/repos/asf?p=hbase.git + scm:git:https://gitbox.apache.org/repos/asf/hbase.git +https://gitbox.apache.org/repos/asf?p=hbase.git JIRA diff --git a/src/main/asciidoc/_chapters/developer.adoc b/src/main/asciidoc/_chapters/developer.adoc index 935d6e6..32432ff 100644 --- a/src/main/asciidoc/_chapters/developer.adoc +++ b/src/main/asciidoc/_chapters/developer.adoc @@ -168,13 +168,14 @@ code snippet [[repos]] == Apache HBase Repositories -There are two different repositories for Apache HBase: Subversion (SVN) and Git. -GIT is our repository of record for all but the Apache HBase website. -We used to be on SVN. -We migrated. -See link:https://issues.apache.org/jira/browse/INFRA-7768[Migrate Apache HBase SVN Repos to Git]. -See link:https://hbase.apache.org/source-repository.html[Source Code -Management] page for contributor and committer links or search for HBase on the link:https://git.apache.org/[Apache Git] page. +Apache HBase consists of multiple repositories which are hosted on https://gitbox.apache.org/[Apache GitBox]. +These are the following: + +* https://gitbox.apache.org/repos/asf?p=hbase.git[hbase] - main Apache HBase repository +* https://gitbox.apache.org/repos/asf?p=hbase-connectors.git[hbase-connectors] - connectors to Apache Kafka and Apache Spark +* https://gitbox.apache.org/repos/asf?p=hbase-operator-tools.git[hbase-operator-tools] - operability and supportability tools, such as <> +* https://gitbox.apache.org/repos/asf?p=hbase-site.git[hbase-site] - hbase.apache.org website +* https://gitbox.apache.org/repos/asf?p=hbase-thirdparty.git[hbase-thirdparty] - relocated versions of popular third-party libraries == IDEs @@ -1794,7 +1795,7 @@ If you submit a patch for one thing, don't do auto-reformatting or unrelated ref Likewise, don't add unrelated cleanup or refactorings outside the scope of your Jira. [[common.patch.feedback.tests]] -= Ambigious Unit Tests += Ambiguous Unit Tests Make sure that you're clear about what you are testing in your unit tests and why. @@ -1920,8 +1921,6 @@ If you are new to submitting patches to open source or new to submitting patches reading the link:https://commons.apache.org/patches.html[On Contributing Patches] page from link:https://commons.apache.org/[Apache Commons Project]. It provides a nice overview that applies equally to the Apache HBase Project. -link:https://accumulo.apache.org/git.html[Accumulo doc on how to contribute and develop] is also -good read to understand development workflow. [[submitting.patches.create]] Create Patch diff --git a/src/main/asciidoc/_chapters/rpc.adoc b/src/main/asciidoc/_chapters/rpc.adoc index fbfba6c..1926c6f 100644 --- a/src/main/asciidoc/_chapters/rpc.adoc +++ b/src/main/asciidoc/_chapters/rpc.adoc @@ -66,13 +66,11 @@ The client will send setup information on connection establish. Thereafter, the client invokes methods against the remote server sending a protobuf Message and receiving a protobuf Message in response. Communication is
