HBASE-15638 Shade protobuf
Which includes
HBASE-16742 Add chapter for devs on how we do protobufs going forward
HBASE-16741 Amend the generate protobufs out-of-band build step
to include shade, pulling in protobuf source and a hook for patching
protobuf
Removed ByteStringer from hbase-protocol-shaded. Use the protobuf-3.1.0
trick directly instead. Makes stuff cleaner. All under 'shaded' dir is
now generated.
HBASE-16567 Upgrade to protobuf-3.1.x
Regenerate all protos in this module with protoc3.
Redo ByteStringer to use new pb3.1.0 unsafebytesutil
instead of HBaseZeroCopyByteString
HBASE-16264 Figure how to deal with endpoints and shaded pb Shade our
protobufs.
Do it in a manner that makes it so we can still have in our API references
to
com.google.protobuf (and in REST). The c.g.p in API is for Coprocessor
Endpoints (CPEP)
This patch is Tactic #4 from Shading Doc attached to the referenced
issue.
Figuring an appoach took a while because we have Coprocessor
Endpoints
mixed in with the core of HBase that are tough to untangle (FIX).
Tactic #4 (the fourth attempt at addressing this issue) is COPY all
but
the CPEP .proto files currently in hbase-protocol to a new module
named
hbase-protocol-shaded. Generate .protos again in the new location
and
then relocate/shade the generated files. Let CPEPs keep on with the
old references at com.google.protobuf.* and
org.apache.hadoop.hbase.protobuf.* but change the hbase core so all
instead refer to the relocated files in their new location at
org.apache.hadoop.hbase.shaded.com.google.protobuf.*.
Let the new module also shade protobufs themselves and change hbase
core to pick up this shaded protobuf rather than directly reference
com.google.protobuf.
This approach allows us to explicitly refer to either the shaded or
non-shaded version of a protobuf class in any particular context
(though
usually context dictates one or the other). Core runs on shaded
protobuf.
CPEPs continue to use whatever is on the classpath with
com.google.protobuf.* which is pb2.5.0 for the near future at least.
See above cited doc for follow-ons and downsides. In short, IDEs
will complain
about not being able to find the shaded protobufs since shading
happens at package
time; will fix by checking in all generated classes and relocated
protobuf in
a follow-on. Also, CPEPs currently suffer an extra-copy as
marshalled from
non-shaded to shaded. To fix. Finally, our .protos are duplicated;
once
shaded, and once not. Pain, but how else to reveal our protos to
CPEPs or
C++ client that wants to talk with HBase AND shade protobuf.
Details:
Add a new hbase-protocol-shaded module. It is a copy of
hbase-protocol
i with all relocated offset from o.a.h.h. to o.a.h.h.shaded. The new
module
also includes the relocated pb. It does not include CPEPs. They
stay in
their old location.
Add another module hbase-endpoint which has in it all the endpoints
that ship as part of hbase -- at least the ones that are not
entangled with core such as AccessControl and Auth. Move all protos
for these CPEPs here as well as their unit tests (mostly moving a
bunch of stuff out of hbase-server module)
Much of the change looks like this:
-import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
-import
org.apache.hadoop.hbase.protobuf.generated.ClusterIdProtos;
+import org.apache.hadoop.hbase.protobuf.shaded.ProtobufUtil;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterIdProtos;
In HTable and in HBaseAdmin, regularize the way Callables are used
and also hide
protobuf usage as much as possible moving it up into Callable super
classes or out
to utility classes. Still TODO is adding in of retries, etc., but
can wait on
procedure which will redo all this.
Also in HTable and HBaseAdmin as well as in HRegionServer and
Server, be explicit
when using non-shaded protobuf. Do the full-path so it is clear.
This is around
endpoint coprocessors registration of services and execution of
CPEP methods.
Shrunk ProtobufUtil by moving methods used by one CPEP only back to
the CPEP either
into Client class or as new Util class; e.g. AccessControlUtil.
There are actually two versions of ProtobufUtil now; a shaded one
and a subset
that is used by CPEPs doing non-shaded work.
Made it so hbase-common no longer depends on hbase-protocol (with
Matteo's help)
R*Converter classes got moved down under shaded package -- they are
for internal
use only. There are no non-shaded versions of these classes.
D
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRegionServerCallable
D RetryingCallableBase
Not used anymore and we have too many tiers of Callables so
removed/cleaned-up.
A ClientServicecallable
Had to add this one. RegionServerCallable was made generic so it
could be used
for a few Interfaces (Client and Admin). Then added
ClientServiceCallable to
implement RegionServerCallable with the Client Interface.
Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/95c1dc93
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/95c1dc93
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/95c1dc93
Branch: refs/heads/hbase-12439
Commit: 95c1dc93fb6780f66fd8ebb57914a050b75b9b11
Parents: cc237c4
Author: stack <st...@apache.org>
Authored: Mon Oct 3 21:37:32 2016 -0700
Committer: stack <st...@apache.org>
Committed: Mon Oct 3 21:37:32 2016 -0700
----------------------------------------------------------------------
hbase-client/pom.xml | 4 +
.../java/org/apache/hadoop/hbase/ClusterId.java | 4 +-
.../org/apache/hadoop/hbase/ClusterStatus.java | 16 +-
.../apache/hadoop/hbase/HColumnDescriptor.java | 4 +-
.../org/apache/hadoop/hbase/HRegionInfo.java | 12 +-
.../apache/hadoop/hbase/HTableDescriptor.java | 4 +-
.../apache/hadoop/hbase/MetaTableAccessor.java | 115 +-
.../org/apache/hadoop/hbase/RegionLoad.java | 4 +-
.../org/apache/hadoop/hbase/ServerLoad.java | 6 +-
.../client/AbstractRegionServerCallable.java | 138 -
.../hadoop/hbase/client/AsyncProcess.java | 45 +-
.../hbase/client/AsyncRequestFutureImpl.java | 7 +-
.../client/CancellableRegionServerCallable.java | 59 +-
.../hadoop/hbase/client/ClientScanner.java | 4 +-
.../hbase/client/ClientServiceCallable.java | 56 +
.../hadoop/hbase/client/ClientSmallScanner.java | 10 +-
.../hadoop/hbase/client/ClusterConnection.java | 6 +-
.../hbase/client/ClusterStatusListener.java | 4 +-
.../hbase/client/ConnectionImplementation.java | 47 +-
.../hadoop/hbase/client/ConnectionUtils.java | 4 +-
.../client/FastFailInterceptorContext.java | 11 +-
.../hbase/client/FlushRegionCallable.java | 6 +-
.../apache/hadoop/hbase/client/HBaseAdmin.java | 262 +-
.../org/apache/hadoop/hbase/client/HTable.java | 148 +-
.../hadoop/hbase/client/MasterCallable.java | 2 +-
.../hbase/client/MasterKeepAliveConnection.java | 2 +-
.../hadoop/hbase/client/MetricsConnection.java | 10 +-
.../hadoop/hbase/client/MultiResponse.java | 2 +-
.../hbase/client/MultiServerCallable.java | 21 +-
.../apache/hadoop/hbase/client/Mutation.java | 56 +-
.../client/NoOpRetryableCallerInterceptor.java | 8 +-
.../client/NoOpRetryingInterceptorContext.java | 9 +-
.../client/NoncedRegionServerCallable.java | 9 +-
.../org/apache/hadoop/hbase/client/Query.java | 7 +-
.../client/RegionAdminServiceCallable.java | 4 +-
.../client/RegionCoprocessorRpcChannel.java | 103 +
.../client/RegionCoprocessorServiceExec.java | 4 +-
.../hbase/client/RegionServerCallable.java | 202 +-
.../org/apache/hadoop/hbase/client/Result.java | 2 +-
.../hadoop/hbase/client/RetryingCallable.java | 38 +-
.../hbase/client/RetryingCallableBase.java | 60 -
.../RetryingCallerInterceptorContext.java | 8 +-
.../hbase/client/ReversedScannerCallable.java | 12 +-
.../hbase/client/RpcRetryingCallable.java | 2 +-
.../hbase/client/RpcRetryingCallerImpl.java | 2 +-
.../RpcRetryingCallerWithReadReplicas.java | 56 +-
.../org/apache/hadoop/hbase/client/Scan.java | 2 +-
.../hadoop/hbase/client/ScannerCallable.java | 18 +-
.../hbase/client/SecureBulkLoadClient.java | 31 +-
.../hbase/client/SyncCoprocessorRpcChannel.java | 80 +
.../apache/hadoop/hbase/client/TableState.java | 4 +-
.../client/coprocessor/AggregationClient.java | 823 -
.../BigDecimalColumnInterpreter.java | 7 +-
.../replication/ReplicationSerDeHelper.java | 10 +-
.../hadoop/hbase/filter/BinaryComparator.java | 10 +-
.../hbase/filter/BinaryPrefixComparator.java | 7 +-
.../hadoop/hbase/filter/BitComparator.java | 7 +-
.../hbase/filter/ColumnCountGetFilter.java | 4 +-
.../hbase/filter/ColumnPaginationFilter.java | 8 +-
.../hadoop/hbase/filter/ColumnPrefixFilter.java | 8 +-
.../hadoop/hbase/filter/ColumnRangeFilter.java | 12 +-
.../hadoop/hbase/filter/CompareFilter.java | 8 +-
.../hbase/filter/DependentColumnFilter.java | 13 +-
.../hadoop/hbase/filter/FamilyFilter.java | 7 +-
.../apache/hadoop/hbase/filter/FilterList.java | 10 +-
.../hadoop/hbase/filter/FilterWrapper.java | 7 +-
.../hadoop/hbase/filter/FirstKeyOnlyFilter.java | 4 +-
.../FirstKeyValueMatchingQualifiersFilter.java | 10 +-
.../hadoop/hbase/filter/FuzzyRowFilter.java | 12 +-
.../hbase/filter/InclusiveStopFilter.java | 9 +-
.../hadoop/hbase/filter/KeyOnlyFilter.java | 4 +-
.../hadoop/hbase/filter/LongComparator.java | 8 +-
.../hbase/filter/MultiRowRangeFilter.java | 11 +-
.../filter/MultipleColumnPrefixFilter.java | 9 +-
.../hadoop/hbase/filter/NullComparator.java | 4 +-
.../apache/hadoop/hbase/filter/PageFilter.java | 4 +-
.../hadoop/hbase/filter/PrefixFilter.java | 8 +-
.../hadoop/hbase/filter/QualifierFilter.java | 7 +-
.../hadoop/hbase/filter/RandomRowFilter.java | 4 +-
.../hbase/filter/RegexStringComparator.java | 4 +-
.../apache/hadoop/hbase/filter/RowFilter.java | 7 +-
.../filter/SingleColumnValueExcludeFilter.java | 7 +-
.../hbase/filter/SingleColumnValueFilter.java | 32 +-
.../apache/hadoop/hbase/filter/SkipFilter.java | 7 +-
.../hbase/filter/SubstringComparator.java | 4 +-
.../hadoop/hbase/filter/TimestampsFilter.java | 4 +-
.../apache/hadoop/hbase/filter/ValueFilter.java | 7 +-
.../hadoop/hbase/filter/WhileMatchFilter.java | 7 +-
.../hadoop/hbase/ipc/AbstractRpcClient.java | 45 +-
.../hadoop/hbase/ipc/BlockingRpcCallback.java | 4 +-
.../hadoop/hbase/ipc/BlockingRpcConnection.java | 18 +-
.../java/org/apache/hadoop/hbase/ipc/Call.java | 10 +-
.../hadoop/hbase/ipc/CoprocessorRpcChannel.java | 5 +-
.../hadoop/hbase/ipc/CoprocessorRpcUtils.java | 199 +-
.../hbase/ipc/DelegatingHBaseRpcController.java | 4 +-
.../hadoop/hbase/ipc/HBaseRpcController.java | 4 +-
.../hbase/ipc/HBaseRpcControllerImpl.java | 2 +-
.../org/apache/hadoop/hbase/ipc/IPCUtil.java | 12 +-
.../hbase/ipc/MasterCoprocessorRpcChannel.java | 80 -
.../hadoop/hbase/ipc/NettyRpcConnection.java | 8 +-
.../hadoop/hbase/ipc/NettyRpcDuplexHandler.java | 14 +-
.../hbase/ipc/RegionCoprocessorRpcChannel.java | 121 -
.../ipc/RegionServerCoprocessorRpcChannel.java | 76 -
.../org/apache/hadoop/hbase/ipc/RpcClient.java | 4 +-
.../apache/hadoop/hbase/ipc/RpcConnection.java | 6 +-
.../hbase/ipc/SyncCoprocessorRpcChannel.java | 79 -
.../apache/hadoop/hbase/master/RegionState.java | 2 +-
.../hadoop/hbase/protobuf/ProtobufMagic.java | 90 +
.../hadoop/hbase/protobuf/ProtobufUtil.java | 2294 +-
.../hadoop/hbase/protobuf/RequestConverter.java | 1794 -
.../hbase/protobuf/ResponseConverter.java | 442 -
.../hadoop/hbase/quotas/QuotaRetriever.java | 2 +-
.../hadoop/hbase/quotas/QuotaSettings.java | 4 +-
.../hbase/quotas/QuotaSettingsFactory.java | 8 +-
.../hadoop/hbase/quotas/QuotaTableUtil.java | 4 +-
.../hadoop/hbase/quotas/ThrottleSettings.java | 6 +-
.../replication/ReplicationPeerConfig.java | 1 -
.../replication/ReplicationPeerZKImpl.java | 4 +-
.../replication/ReplicationPeersZKImpl.java | 2 +-
.../replication/ReplicationQueuesZKImpl.java | 2 +
.../replication/ReplicationStateZKBase.java | 24 +-
.../hadoop/hbase/security/EncryptionUtil.java | 10 +-
.../hadoop/hbase/security/SecurityInfo.java | 8 +-
.../security/access/AccessControlClient.java | 36 +-
.../security/access/AccessControlUtil.java | 766 +
.../security/visibility/VisibilityClient.java | 34 +-
.../hbase/shaded/protobuf/ProtobufUtil.java | 3161 +
.../hbase/shaded/protobuf/RequestConverter.java | 1498 +
.../shaded/protobuf/ResponseConverter.java | 427 +
.../ClientSnapshotDescriptionUtils.java | 2 +-
.../hbase/zookeeper/MasterAddressTracker.java | 15 +-
.../hbase/zookeeper/MetaTableLocator.java | 16 +-
.../apache/hadoop/hbase/zookeeper/ZKUtil.java | 18 +-
.../hbase/TestInterfaceAudienceAnnotations.java | 11 +
.../client/TestClientExponentialBackoff.java | 4 +-
.../hbase/client/TestClientNoCluster.java | 71 +-
.../org/apache/hadoop/hbase/client/TestGet.java | 4 +-
.../hbase/client/TestMetricsConnection.java | 20 +-
.../hbase/client/TestProcedureFuture.java | 4 +-
.../apache/hadoop/hbase/client/TestScan.java | 5 +-
.../hbase/client/TestSnapshotFromAdmin.java | 10 +-
.../exceptions/TestClientExceptionsUtil.java | 2 +-
hbase-common/pom.xml | 4 -
.../org/apache/hadoop/hbase/ProcedureInfo.java | 27 +-
.../org/apache/hadoop/hbase/ProcedureUtil.java | 103 -
.../org/apache/hadoop/hbase/ServerName.java | 55 +-
.../hbase/filter/ByteArrayComparable.java | 9 -
.../hbase/io/encoding/EncodedDataBlock.java | 4 +-
.../hadoop/hbase/rsgroup/RSGroupInfo.java | 3 +-
.../hadoop/hbase/util/ForeignExceptionUtil.java | 128 -
hbase-endpoint/README.txt | 42 +
hbase-endpoint/pom.xml | 321 +
.../client/coprocessor/AggregationClient.java | 864 +
.../coprocessor/AggregateImplementation.java | 530 +
.../example/generated/BulkDeleteProtos.java | 1792 +
.../generated/ColumnAggregationProtos.java | 1277 +
.../ColumnAggregationWithErrorsProtos.java | 1290 +
...ColumnAggregationWithNullResponseProtos.java | 1283 +
.../DummyRegionServerEndpointProtos.java | 1225 +
.../IncrementCounterProcessorTestProtos.java | 4059 +
.../protobuf/generated/AggregateProtos.java | 2375 +
.../generated/SecureBulkLoadProtos.java | 2088 +
.../security/access/SecureBulkLoadEndpoint.java | 171 +
.../src/main/protobuf/Aggregate.proto | 63 +
.../src/main/protobuf/BulkDelete.proto | 51 +
.../ColumnAggregationNullResponseProtocol.proto | 38 +
.../protobuf/ColumnAggregationProtocol.proto | 35 +
.../ColumnAggregationWithErrorsProtocol.proto | 38 +
.../protobuf/DummyRegionServerEndpoint.proto | 37 +
.../protobuf/IncrementCounterProcessor.proto | 55 +
.../src/main/protobuf/SecureBulkLoad.proto | 48 +
.../hbase/client/TestRpcControllerFactory.java | 221 +
.../coprocessor/ColumnAggregationEndpoint.java | 117 +
.../ColumnAggregationEndpointNullResponse.java | 129 +
.../ColumnAggregationEndpointWithErrors.java | 126 +
.../coprocessor/ProtobufCoprocessorService.java | 97 +
.../TestBatchCoprocessorEndpoint.java | 283 +
.../hbase/coprocessor/TestClassLoading.java | 563 +
.../coprocessor/TestCoprocessorEndpoint.java | 349 +
.../TestCoprocessorTableEndpoint.java | 182 +
.../TestRegionServerCoprocessorEndpoint.java | 137 +
.../coprocessor/TestRowProcessorEndpoint.java | 665 +
.../hbase/ipc/TestCoprocessorRpcUtils.java | 45 +
.../SecureBulkLoadEndpointClient.java | 167 +
...gionServerBulkLoadWithOldSecureEndpoint.java | 179 +
.../regionserver/TestServerCustomProtocol.java | 480 +
...ReplicationSyncUpToolWithBulkLoadedData.java | 235 +
hbase-examples/README.txt | 4 +
hbase-examples/pom.xml | 40 +
.../coprocessor/example/BulkDeleteEndpoint.java | 4 +-
.../coprocessor/example/RowCountEndpoint.java | 6 +-
hbase-examples/src/main/protobuf/Examples.proto | 38 +
.../example/TestBulkDeleteProtocol.java | 443 -
.../example/TestRowCountEndpoint.java | 107 -
.../TestZooKeeperScanPolicyObserver.java | 130 -
.../hadoop/hbase/DistributedHBaseCluster.java | 10 +-
.../hbase/IntegrationTestMetaReplicas.java | 3 +-
.../hbase/ipc/IntegrationTestRpcClient.java | 12 +-
hbase-procedure/pom.xml | 2 +-
.../hadoop/hbase/procedure2/Procedure.java | 26 +-
.../hbase/procedure2/ProcedureExecutor.java | 6 +-
.../hadoop/hbase/procedure2/ProcedureUtil.java | 108 +
.../procedure2/RemoteProcedureException.java | 7 +-
.../hbase/procedure2/RootProcedureState.java | 2 +-
.../hbase/procedure2/SequentialProcedure.java | 2 +-
.../hbase/procedure2/StateMachineProcedure.java | 2 +-
.../procedure2/store/ProcedureStoreTracker.java | 6 +-
.../procedure2/store/wal/ProcedureWALFile.java | 6 +-
.../store/wal/ProcedureWALFormat.java | 8 +-
.../store/wal/ProcedureWALFormatReader.java | 8 +-
.../store/wal/ProcedureWALPrettyPrinter.java | 4 +-
.../procedure2/store/wal/WALProcedureStore.java | 6 +-
.../procedure2/ProcedureTestingUtility.java | 28 +-
.../hbase/procedure2/TestChildProcedures.java | 2 +-
.../procedure2/TestProcedureExecution.java | 6 +-
.../hbase/procedure2/TestProcedureToString.java | 4 +-
hbase-protocol-shaded/README.txt | 48 +
hbase-protocol-shaded/pom.xml | 347 +
.../com/google/protobuf/AbstractMessage.java | 646 +
.../google/protobuf/AbstractMessageLite.java | 384 +
.../com/google/protobuf/AbstractParser.java | 258 +
.../google/protobuf/AbstractProtobufList.java | 180 +
.../hbase/shaded/com/google/protobuf/Any.java | 899 +
.../com/google/protobuf/AnyOrBuilder.java | 70 +
.../shaded/com/google/protobuf/AnyProto.java | 59 +
.../hbase/shaded/com/google/protobuf/Api.java | 2473 +
.../com/google/protobuf/ApiOrBuilder.java | 258 +
.../shaded/com/google/protobuf/ApiProto.java | 97 +
.../com/google/protobuf/BlockingRpcChannel.java | 51 +
.../com/google/protobuf/BlockingService.java | 64 +
.../shaded/com/google/protobuf/BoolValue.java | 452 +
.../com/google/protobuf/BoolValueOrBuilder.java | 18 +
.../com/google/protobuf/BooleanArrayList.java | 272 +
.../com/google/protobuf/ByteBufferWriter.java | 145 +
.../shaded/com/google/protobuf/ByteOutput.java | 116 +
.../shaded/com/google/protobuf/ByteString.java | 1558 +
.../shaded/com/google/protobuf/BytesValue.java | 454 +
.../google/protobuf/BytesValueOrBuilder.java | 18 +
.../com/google/protobuf/CodedInputStream.java | 2895 +
.../com/google/protobuf/CodedOutputStream.java | 3001 +
.../com/google/protobuf/DescriptorProtos.java | 38670 +++++++++
.../shaded/com/google/protobuf/Descriptors.java | 2547 +
.../com/google/protobuf/DoubleArrayList.java | 273 +
.../shaded/com/google/protobuf/DoubleValue.java | 454 +
.../google/protobuf/DoubleValueOrBuilder.java | 18 +
.../shaded/com/google/protobuf/Duration.java | 618 +
.../com/google/protobuf/DurationOrBuilder.java | 33 +
.../com/google/protobuf/DurationProto.java | 59 +
.../com/google/protobuf/DynamicMessage.java | 684 +
.../hbase/shaded/com/google/protobuf/Empty.java | 386 +
.../com/google/protobuf/EmptyOrBuilder.java | 9 +
.../shaded/com/google/protobuf/EmptyProto.java | 58 +
.../hbase/shaded/com/google/protobuf/Enum.java | 1745 +
.../com/google/protobuf/EnumOrBuilder.java | 157 +
.../shaded/com/google/protobuf/EnumValue.java | 1044 +
.../com/google/protobuf/EnumValueOrBuilder.java | 80 +
.../com/google/protobuf/ExperimentalApi.java | 66 +
.../shaded/com/google/protobuf/Extension.java | 86 +
.../com/google/protobuf/ExtensionLite.java | 63 +
.../com/google/protobuf/ExtensionRegistry.java | 397 +
.../protobuf/ExtensionRegistryFactory.java | 95 +
.../google/protobuf/ExtensionRegistryLite.java | 226 +
.../hbase/shaded/com/google/protobuf/Field.java | 2450 +
.../shaded/com/google/protobuf/FieldMask.java | 903 +
.../com/google/protobuf/FieldMaskOrBuilder.java | 44 +
.../com/google/protobuf/FieldMaskProto.java | 58 +
.../com/google/protobuf/FieldOrBuilder.java | 189 +
.../shaded/com/google/protobuf/FieldSet.java | 908 +
.../com/google/protobuf/FloatArrayList.java | 272 +
.../shaded/com/google/protobuf/FloatValue.java | 454 +
.../google/protobuf/FloatValueOrBuilder.java | 18 +
.../com/google/protobuf/GeneratedMessage.java | 3047 +
.../google/protobuf/GeneratedMessageLite.java | 2280 +
.../com/google/protobuf/GeneratedMessageV3.java | 2840 +
.../shaded/com/google/protobuf/Int32Value.java | 451 +
.../google/protobuf/Int32ValueOrBuilder.java | 18 +
.../shaded/com/google/protobuf/Int64Value.java | 452 +
.../google/protobuf/Int64ValueOrBuilder.java | 18 +
.../com/google/protobuf/IntArrayList.java | 272 +
.../shaded/com/google/protobuf/Internal.java | 751 +
.../InvalidProtocolBufferException.java | 146 +
.../shaded/com/google/protobuf/LazyField.java | 154 +
.../com/google/protobuf/LazyFieldLite.java | 458 +
.../google/protobuf/LazyStringArrayList.java | 423 +
.../com/google/protobuf/LazyStringList.java | 174 +
.../shaded/com/google/protobuf/ListValue.java | 814 +
.../com/google/protobuf/ListValueOrBuilder.java | 53 +
.../com/google/protobuf/LongArrayList.java | 272 +
.../shaded/com/google/protobuf/MapEntry.java | 449 +
.../com/google/protobuf/MapEntryLite.java | 226 +
.../shaded/com/google/protobuf/MapField.java | 624 +
.../com/google/protobuf/MapFieldLite.java | 224 +
.../shaded/com/google/protobuf/Message.java | 292 +
.../shaded/com/google/protobuf/MessageLite.java | 341 +
.../google/protobuf/MessageLiteOrBuilder.java | 60 +
.../google/protobuf/MessageLiteToString.java | 239 +
.../com/google/protobuf/MessageOrBuilder.java | 143 +
.../com/google/protobuf/MessageReflection.java | 990 +
.../shaded/com/google/protobuf/Method.java | 1541 +
.../com/google/protobuf/MethodOrBuilder.java | 142 +
.../hbase/shaded/com/google/protobuf/Mixin.java | 812 +
.../com/google/protobuf/MixinOrBuilder.java | 47 +
.../com/google/protobuf/MutabilityOracle.java | 48 +
.../com/google/protobuf/NioByteString.java | 291 +
.../shaded/com/google/protobuf/NullValue.java | 109 +
.../shaded/com/google/protobuf/Option.java | 761 +
.../com/google/protobuf/OptionOrBuilder.java | 52 +
.../shaded/com/google/protobuf/Parser.java | 272 +
.../com/google/protobuf/ProtobufArrayList.java | 105 +
.../google/protobuf/ProtocolMessageEnum.java | 59 +
.../com/google/protobuf/ProtocolStringList.java | 48 +
.../google/protobuf/RepeatedFieldBuilder.java | 708 +
.../google/protobuf/RepeatedFieldBuilderV3.java | 708 +
.../com/google/protobuf/RopeByteString.java | 897 +
.../shaded/com/google/protobuf/RpcCallback.java | 47 +
.../shaded/com/google/protobuf/RpcChannel.java | 71 +
.../com/google/protobuf/RpcController.java | 118 +
.../shaded/com/google/protobuf/RpcUtil.java | 136 +
.../shaded/com/google/protobuf/Service.java | 117 +
.../com/google/protobuf/ServiceException.java | 52 +
.../com/google/protobuf/SingleFieldBuilder.java | 241 +
.../google/protobuf/SingleFieldBuilderV3.java | 241 +
.../com/google/protobuf/SmallSortedMap.java | 670 +
.../com/google/protobuf/SourceContext.java | 539 +
.../google/protobuf/SourceContextOrBuilder.java | 29 +
.../com/google/protobuf/SourceContextProto.java | 58 +
.../shaded/com/google/protobuf/StringValue.java | 532 +
.../google/protobuf/StringValueOrBuilder.java | 27 +
.../shaded/com/google/protobuf/Struct.java | 701 +
.../com/google/protobuf/StructOrBuilder.java | 63 +
.../shaded/com/google/protobuf/StructProto.java | 103 +
.../shaded/com/google/protobuf/Syntax.java | 124 +
.../shaded/com/google/protobuf/TextFormat.java | 2108 +
.../com/google/protobuf/TextFormatEscaper.java | 137 +
.../protobuf/TextFormatParseInfoTree.java | 226 +
.../protobuf/TextFormatParseLocation.java | 104 +
.../shaded/com/google/protobuf/Timestamp.java | 616 +
.../com/google/protobuf/TimestampOrBuilder.java | 32 +
.../com/google/protobuf/TimestampProto.java | 59 +
.../hbase/shaded/com/google/protobuf/Type.java | 1967 +
.../com/google/protobuf/TypeOrBuilder.java | 192 +
.../shaded/com/google/protobuf/TypeProto.java | 140 +
.../shaded/com/google/protobuf/UInt32Value.java | 451 +
.../google/protobuf/UInt32ValueOrBuilder.java | 18 +
.../shaded/com/google/protobuf/UInt64Value.java | 452 +
.../google/protobuf/UInt64ValueOrBuilder.java | 18 +
.../protobuf/UninitializedMessageException.java | 99 +
.../com/google/protobuf/UnknownFieldSet.java | 1037 +
.../google/protobuf/UnknownFieldSetLite.java | 378 +
.../protobuf/UnmodifiableLazyStringList.java | 210 +
.../google/protobuf/UnsafeByteOperations.java | 120 +
.../shaded/com/google/protobuf/UnsafeUtil.java | 295 +
.../hbase/shaded/com/google/protobuf/Utf8.java | 1573 +
.../hbase/shaded/com/google/protobuf/Value.java | 1410 +
.../com/google/protobuf/ValueOrBuilder.java | 98 +
.../shaded/com/google/protobuf/WireFormat.java | 260 +
.../com/google/protobuf/WrappersProto.java | 153 +
.../google/protobuf/compiler/PluginProtos.java | 4278 +
.../protobuf/generated/TestProcedureProtos.java | 608 +
.../ipc/protobuf/generated/TestProtos.java | 2987 +
.../generated/TestRpcServiceProtos.java | 575 +
.../shaded/protobuf/generated/AdminProtos.java | 25475 ++++++
.../shaded/protobuf/generated/CellProtos.java | 2287 +
.../shaded/protobuf/generated/ClientProtos.java | 40662 ++++++++++
.../protobuf/generated/ClusterIdProtos.java | 668 +
.../protobuf/generated/ClusterStatusProtos.java | 15191 ++++
.../protobuf/generated/ComparatorProtos.java | 5719 ++
.../protobuf/generated/EncryptionProtos.java | 980 +
.../protobuf/generated/ErrorHandlingProtos.java | 2997 +
.../shaded/protobuf/generated/FSProtos.java | 1311 +
.../shaded/protobuf/generated/FilterProtos.java | 18739 +++++
.../shaded/protobuf/generated/HBaseProtos.java | 20673 +++++
.../shaded/protobuf/generated/HFileProtos.java | 2449 +
.../protobuf/generated/LoadBalancerProtos.java | 524 +
.../protobuf/generated/MapReduceProtos.java | 1804 +
.../generated/MasterProcedureProtos.java | 22959 ++++++
.../shaded/protobuf/generated/MasterProtos.java | 70426 +++++++++++++++++
.../protobuf/generated/ProcedureProtos.java | 7826 ++
.../shaded/protobuf/generated/QuotaProtos.java | 4552 ++
.../shaded/protobuf/generated/RPCProtos.java | 6480 ++
.../generated/RegionNormalizerProtos.java | 524 +
.../generated/RegionServerStatusProtos.java | 9594 +++
.../protobuf/generated/SnapshotProtos.java | 4950 ++
.../protobuf/generated/TracingProtos.java | 628 +
.../shaded/protobuf/generated/WALProtos.java | 12308 +++
.../protobuf/generated/ZooKeeperProtos.java | 9909 +++
.../hadoop/hbase/util/ForeignExceptionUtil.java | 146 +
.../src/main/protobuf/Admin.proto | 309 +
.../src/main/protobuf/Cell.proto | 67 +
.../src/main/protobuf/CellSetMessage.proto | 28 +
.../src/main/protobuf/Client.proto | 511 +
.../src/main/protobuf/ClusterId.proto | 34 +
.../src/main/protobuf/ClusterStatus.proto | 227 +
.../src/main/protobuf/Comparator.proto | 74 +
.../src/main/protobuf/Encryption.proto | 33 +
.../src/main/protobuf/ErrorHandling.proto | 58 +
.../src/main/protobuf/FS.proto | 45 +
.../src/main/protobuf/Filter.proto | 171 +
.../src/main/protobuf/HBase.proto | 252 +
.../src/main/protobuf/HFile.proto | 49 +
.../src/main/protobuf/LoadBalancer.proto | 29 +
.../src/main/protobuf/MapReduce.proto | 37 +
.../src/main/protobuf/Master.proto | 831 +
.../src/main/protobuf/MasterProcedure.proto | 300 +
.../src/main/protobuf/Procedure.proto | 120 +
.../src/main/protobuf/Quota.proto | 76 +
.../src/main/protobuf/RPC.proto | 137 +
.../src/main/protobuf/RegionNormalizer.proto | 28 +
.../src/main/protobuf/RegionServerStatus.proto | 158 +
.../src/main/protobuf/RowProcessor.proto | 45 +
.../src/main/protobuf/Snapshot.proto | 66 +
.../src/main/protobuf/TestProcedure.proto | 24 +
.../src/main/protobuf/Tracing.proto | 33 +
.../src/main/protobuf/WAL.proto | 173 +
.../src/main/protobuf/ZooKeeper.proto | 162 +
.../src/main/protobuf/test.proto | 43 +
.../src/main/protobuf/test_rpc_service.proto | 35 +
hbase-protocol/README.txt | 17 +-
hbase-protocol/pom.xml | 36 +-
.../generated/ColumnAggregationProtos.java | 1277 -
.../ColumnAggregationWithErrorsProtos.java | 1290 -
...ColumnAggregationWithNullResponseProtos.java | 1283 -
.../DummyRegionServerEndpointProtos.java | 1225 -
.../IncrementCounterProcessorTestProtos.java | 4059 -
.../protobuf/generated/TestProcedureProtos.java | 530 -
.../hadoop/hbase/protobuf/ProtobufMagic.java | 90 -
.../protobuf/generated/AggregateProtos.java | 2375 -
.../generated/MasterProcedureProtos.java | 22424 ------
.../hbase/protobuf/generated/MasterProtos.java | 66862 ----------------
.../generated/MultiRowMutationProtos.java | 238 +-
.../protobuf/generated/ProcedureProtos.java | 7599 --
.../protobuf/generated/RSGroupAdminProtos.java | 11855 ---
.../hbase/protobuf/generated/RSGroupProtos.java | 1331 -
.../generated/RegionNormalizerProtos.java | 485 -
.../generated/RegionServerStatusProtos.java | 9239 ---
.../generated/SecureBulkLoadProtos.java | 2088 -
.../rest/protobuf/generated/CellMessage.java | 731 -
.../rest/protobuf/generated/CellSetMessage.java | 1521 -
.../protobuf/generated/ColumnSchemaMessage.java | 1904 -
.../generated/NamespacePropertiesMessage.java | 1394 -
.../protobuf/generated/NamespacesMessage.java | 547 -
.../rest/protobuf/generated/ScannerMessage.java | 1578 -
.../generated/StorageClusterStatusMessage.java | 3955 -
.../protobuf/generated/TableInfoMessage.java | 1802 -
.../protobuf/generated/TableListMessage.java | 547 -
.../protobuf/generated/TableSchemaMessage.java | 2125 -
.../rest/protobuf/generated/VersionMessage.java | 1147 -
.../protobuf/generated/SparkFilterProtos.java | 2006 -
.../src/main/protobuf/Aggregate.proto | 63 -
.../src/main/protobuf/BulkDelete.proto | 51 -
.../src/main/protobuf/CellMessage.proto | 25 -
.../src/main/protobuf/CellSetMessage.proto | 28 -
.../ColumnAggregationNullResponseProtocol.proto | 38 -
.../protobuf/ColumnAggregationProtocol.proto | 35 -
.../ColumnAggregationWithErrorsProtocol.proto | 38 -
.../src/main/protobuf/ColumnSchemaMessage.proto | 31 -
.../protobuf/DummyRegionServerEndpoint.proto | 37 -
hbase-protocol/src/main/protobuf/Examples.proto | 38 -
.../protobuf/IncrementCounterProcessor.proto | 55 -
hbase-protocol/src/main/protobuf/Master.proto | 831 -
.../src/main/protobuf/MasterProcedure.proto | 300 -
.../src/main/protobuf/MultiRowMutation.proto | 4 +-
.../protobuf/NamespacePropertiesMessage.proto | 26 -
.../src/main/protobuf/NamespacesMessage.proto | 22 -
.../src/main/protobuf/Procedure.proto | 120 -
hbase-protocol/src/main/protobuf/RSGroup.proto | 34 -
.../src/main/protobuf/RSGroupAdmin.proto | 136 -
.../src/main/protobuf/RegionNormalizer.proto | 28 -
.../src/main/protobuf/RegionServerStatus.proto | 158 -
.../src/main/protobuf/ScannerMessage.proto | 32 -
.../src/main/protobuf/SecureBulkLoad.proto | 48 -
.../src/main/protobuf/SparkFilter.proto | 40 -
.../protobuf/StorageClusterStatusMessage.proto | 51 -
.../src/main/protobuf/TableInfoMessage.proto | 30 -
.../src/main/protobuf/TableListMessage.proto | 22 -
.../src/main/protobuf/TableSchemaMessage.proto | 33 -
.../src/main/protobuf/VersionMessage.proto | 26 -
hbase-rest/pom.xml | 50 +
.../hadoop/hbase/rest/model/CellModel.java | 4 +-
.../hadoop/hbase/rest/model/CellSetModel.java | 14 +-
.../hadoop/hbase/rest/model/ScannerModel.java | 30 +-
.../rest/model/StorageClusterStatusModel.java | 2 +-
.../hadoop/hbase/rest/model/TableInfoModel.java | 10 +-
.../hadoop/hbase/rest/model/TableListModel.java | 4 +-
.../hbase/rest/model/TableSchemaModel.java | 18 +-
.../hadoop/hbase/rest/model/VersionModel.java | 4 +-
.../rest/protobuf/generated/CellMessage.java | 731 +
.../rest/protobuf/generated/CellSetMessage.java | 1521 +
.../protobuf/generated/ColumnSchemaMessage.java | 1904 +
.../generated/NamespacePropertiesMessage.java | 1394 +
.../protobuf/generated/NamespacesMessage.java | 547 +
.../rest/protobuf/generated/ScannerMessage.java | 1578 +
.../generated/StorageClusterStatusMessage.java | 3955 +
.../protobuf/generated/TableInfoMessage.java | 1802 +
.../protobuf/generated/TableListMessage.java | 547 +
.../protobuf/generated/TableSchemaMessage.java | 2125 +
.../rest/protobuf/generated/VersionMessage.java | 1147 +
hbase-rest/src/main/protobuf/CellMessage.proto | 25 +
.../src/main/protobuf/CellSetMessage.proto | 28 +
.../src/main/protobuf/ColumnSchemaMessage.proto | 31 +
.../protobuf/NamespacePropertiesMessage.proto | 26 +
.../src/main/protobuf/NamespacesMessage.proto | 22 +
.../src/main/protobuf/ScannerMessage.proto | 32 +
.../protobuf/StorageClusterStatusMessage.proto | 51 +
.../src/main/protobuf/TableInfoMessage.proto | 30 +
.../src/main/protobuf/TableListMessage.proto | 22 +
.../src/main/protobuf/TableSchemaMessage.proto | 33 +
.../src/main/protobuf/VersionMessage.proto | 26 +
hbase-rsgroup/README.txt | 30 +
.../protobuf/generated/RSGroupAdminProtos.java | 11855 +++
.../hbase/protobuf/generated/RSGroupProtos.java | 1331 +
.../hbase/rsgroup/RSGroupAdminClient.java | 28 +-
.../hbase/rsgroup/RSGroupAdminEndpoint.java | 32 +-
.../hbase/rsgroup/RSGroupInfoManagerImpl.java | 19 +-
.../hadoop/hbase/rsgroup/RSGroupSerDe.java | 45 +-
hbase-rsgroup/src/main/protobuf/RSGroup.proto | 34 +
.../src/main/protobuf/RSGroupAdmin.proto | 136 +
.../hadoop/hbase/rsgroup/TestRSGroups.java | 4 +-
.../hadoop/hbase/rsgroup/TestRSGroupsBase.java | 4 +-
.../rsgroup/VerifyingRSGroupAdminClient.java | 4 +-
.../hbase/tmpl/regionserver/RSStatusTmpl.jamon | 4 +-
.../tmpl/regionserver/RegionListTmpl.jamon | 8 +-
.../org/apache/hadoop/hbase/SplitLogTask.java | 8 +-
.../apache/hadoop/hbase/ZKNamespaceManager.java | 4 +-
.../hbase/client/CoprocessorHConnection.java | 4 +-
.../hadoop/hbase/client/VersionInfoUtil.java | 2 +-
.../apache/hadoop/hbase/codec/MessageCodec.java | 13 +-
.../SplitLogManagerCoordination.java | 2 +-
.../SplitLogWorkerCoordination.java | 4 +-
.../ZKSplitLogManagerCoordination.java | 2 +-
.../ZkSplitLogWorkerCoordination.java | 4 +-
.../coprocessor/AggregateImplementation.java | 528 -
.../BaseMasterAndRegionObserver.java | 4 +-
.../hbase/coprocessor/BaseMasterObserver.java | 4 +-
.../coprocessor/BaseRegionServerObserver.java | 2 +-
.../coprocessor/BaseRowProcessorEndpoint.java | 4 +-
.../hbase/coprocessor/BulkLoadObserver.java | 4 +-
.../hbase/coprocessor/MasterObserver.java | 4 +-
.../coprocessor/MultiRowMutationEndpoint.java | 4 +-
.../hbase/coprocessor/RegionServerObserver.java | 2 +-
.../hbase/errorhandling/ForeignException.java | 11 +-
.../org/apache/hadoop/hbase/io/Reference.java | 8 +-
.../hadoop/hbase/io/hfile/FixedFileTrailer.java | 10 +-
.../org/apache/hadoop/hbase/io/hfile/HFile.java | 20 +-
.../org/apache/hadoop/hbase/ipc/CallRunner.java | 2 +-
.../ipc/FastPathBalancedQueueRpcExecutor.java | 1 -
.../hadoop/hbase/ipc/PriorityFunction.java | 4 +-
.../hadoop/hbase/ipc/RWQueueRpcExecutor.java | 17 +-
.../apache/hadoop/hbase/ipc/RpcCallContext.java | 2 +-
.../apache/hadoop/hbase/ipc/RpcCallback.java | 3 +-
.../org/apache/hadoop/hbase/ipc/RpcServer.java | 33 +-
.../hadoop/hbase/ipc/RpcServerInterface.java | 8 +-
.../hadoop/hbase/ipc/SimpleRpcScheduler.java | 3 +-
.../hbase/mapreduce/LoadIncrementalHFiles.java | 6 +-
.../hbase/mapreduce/MutationSerialization.java | 6 +-
.../hbase/mapreduce/ResultSerialization.java | 4 +-
.../hbase/mapreduce/TableMapReduceUtil.java | 45 +-
.../mapreduce/TableSnapshotInputFormatImpl.java | 8 +-
.../hbase/master/ActiveMasterManager.java | 3 +-
.../hadoop/hbase/master/AssignmentManager.java | 4 +-
.../hbase/master/ClusterStatusPublisher.java | 4 +-
.../org/apache/hadoop/hbase/master/HMaster.java | 25 +-
...MasterAnnotationReadingPriorityFunction.java | 10 +-
.../hbase/master/MasterCoprocessorHost.java | 4 +-
.../hbase/master/MasterMetaBootstrap.java | 2 +-
.../hadoop/hbase/master/MasterRpcServices.java | 125 +-
.../hadoop/hbase/master/MasterWalManager.java | 2 +-
.../hbase/master/RegionPlacementMaintainer.java | 10 +-
.../hadoop/hbase/master/ServerManager.java | 27 +-
.../hadoop/hbase/master/SnapshotSentinel.java | 2 +-
.../hadoop/hbase/master/SplitLogManager.java | 2 +-
.../hadoop/hbase/master/TableLockManager.java | 4 +-
.../hbase/master/TableNamespaceManager.java | 4 +-
.../balancer/FavoredNodeAssignmentHelper.java | 11 +-
.../normalizer/SimpleRegionNormalizer.java | 20 +-
.../procedure/AddColumnFamilyProcedure.java | 6 +-
.../procedure/CloneSnapshotProcedure.java | 21 +-
.../procedure/CreateNamespaceProcedure.java | 6 +-
.../master/procedure/CreateTableProcedure.java | 13 +-
.../procedure/DeleteColumnFamilyProcedure.java | 11 +-
.../procedure/DeleteNamespaceProcedure.java | 6 +-
.../master/procedure/DeleteTableProcedure.java | 9 +-
.../master/procedure/DisableTableProcedure.java | 8 +-
.../DispatchMergingRegionsProcedure.java | 6 +-
.../master/procedure/EnableTableProcedure.java | 7 +-
.../master/procedure/MasterProcedureUtil.java | 2 +-
.../procedure/ModifyColumnFamilyProcedure.java | 7 +-
.../procedure/ModifyNamespaceProcedure.java | 6 +-
.../master/procedure/ModifyTableProcedure.java | 8 +-
.../master/procedure/ProcedureSyncWait.java | 7 +-
.../procedure/RestoreSnapshotProcedure.java | 14 +-
.../master/procedure/ServerCrashProcedure.java | 10 +-
.../procedure/TruncateTableProcedure.java | 17 +-
.../snapshot/DisabledTableSnapshotHandler.java | 2 +-
.../snapshot/EnabledTableSnapshotHandler.java | 2 +-
.../master/snapshot/MasterSnapshotVerifier.java | 6 +-
.../hbase/master/snapshot/SnapshotManager.java | 14 +-
.../master/snapshot/TakeSnapshotHandler.java | 2 +-
.../mob/mapreduce/SweepJobNodeTracker.java | 4 +-
.../hbase/monitoring/MonitoredRPCHandler.java | 2 +-
.../monitoring/MonitoredRPCHandlerImpl.java | 2 +-
.../hbase/procedure/MasterProcedureManager.java | 2 +-
.../procedure/ZKProcedureCoordinatorRpcs.java | 6 +-
.../hbase/procedure/ZKProcedureMemberRpcs.java | 6 +-
.../flush/MasterFlushTableProcedureManager.java | 2 +-
.../hbase/protobuf/ReplicationProtbufUtil.java | 22 +-
.../hadoop/hbase/quotas/MasterQuotaManager.java | 14 +-
.../hbase/quotas/QuotaLimiterFactory.java | 2 +-
.../apache/hadoop/hbase/quotas/QuotaState.java | 2 +-
.../apache/hadoop/hbase/quotas/QuotaUtil.java | 2 +-
.../hbase/quotas/RegionServerQuotaManager.java | 2 +-
.../hadoop/hbase/quotas/TimeBasedLimiter.java | 6 +-
.../hadoop/hbase/quotas/UserQuotaState.java | 2 +-
.../AnnotationReadingPriorityFunction.java | 26 +-
.../regionserver/FavoredNodesForRegion.java | 2 +-
.../hadoop/hbase/regionserver/HRegion.java | 112 +-
.../hbase/regionserver/HRegionServer.java | 128 +-
.../hadoop/hbase/regionserver/HStore.java | 4 +-
.../hbase/regionserver/LastSequenceId.java | 2 +-
.../hbase/regionserver/RSRpcServices.java | 186 +-
.../hadoop/hbase/regionserver/Region.java | 22 +-
.../RegionMergeTransactionImpl.java | 2 +-
.../hbase/regionserver/RegionScanner.java | 2 +-
.../RegionServerCoprocessorHost.java | 2 +-
.../regionserver/RegionServerServices.java | 2 +-
.../regionserver/ReplicationSinkService.java | 2 +-
.../regionserver/SecureBulkLoadManager.java | 8 +-
.../hbase/regionserver/SplitLogWorker.java | 2 +-
.../regionserver/SplitTransactionImpl.java | 2 +-
.../apache/hadoop/hbase/regionserver/Store.java | 2 +-
.../handler/CloseRegionHandler.java | 2 +-
.../regionserver/handler/OpenRegionHandler.java | 2 +-
.../handler/RegionReplicaFlushHandler.java | 2 +-
.../handler/WALSplitterHandler.java | 2 +-
.../snapshot/FlushSnapshotSubprocedure.java | 2 +-
.../snapshot/RegionServerSnapshotManager.java | 12 +-
.../wal/AbstractProtobufLogWriter.java | 8 +-
.../wal/AsyncProtobufLogWriter.java | 4 +-
.../regionserver/wal/ProtobufLogReader.java | 10 +-
.../regionserver/wal/ProtobufLogWriter.java | 4 +-
.../wal/SecureAsyncProtobufLogWriter.java | 2 +-
.../wal/SecureProtobufLogReader.java | 2 +-
.../wal/SecureProtobufLogWriter.java | 2 +-
.../hbase/regionserver/wal/WALCellCodec.java | 2 +-
.../hadoop/hbase/regionserver/wal/WALEdit.java | 8 +-
.../regionserver/wal/WALEditsReplaySink.java | 13 +-
.../hadoop/hbase/regionserver/wal/WALUtil.java | 10 +-
.../hbase/replication/BulkLoadCellFilter.java | 4 +-
.../replication/master/TableCFsUpdater.java | 2 +-
.../HBaseInterClusterReplicationEndpoint.java | 2 +-
.../RegionReplicaReplicationEndpoint.java | 4 +-
.../replication/regionserver/Replication.java | 8 +-
.../regionserver/ReplicationLoad.java | 2 +-
.../regionserver/ReplicationSink.java | 8 +-
.../regionserver/ReplicationSinkManager.java | 2 +-
.../regionserver/ReplicationSource.java | 6 +-
.../hbase/security/HBasePolicyProvider.java | 8 +-
.../security/access/AccessControlLists.java | 194 +-
.../hbase/security/access/AccessController.java | 34 +-
.../HbaseObjectWritableFor96Migration.java | 4 +-
.../security/access/SecureBulkLoadEndpoint.java | 133 -
.../hbase/security/token/TokenProvider.java | 16 +-
.../hadoop/hbase/security/token/TokenUtil.java | 38 +-
.../visibility/VisibilityController.java | 45 +-
.../security/visibility/VisibilityUtils.java | 6 +-
.../hadoop/hbase/snapshot/CreateSnapshot.java | 4 +-
.../hadoop/hbase/snapshot/ExportSnapshot.java | 6 +-
.../hbase/snapshot/RestoreSnapshotHelper.java | 4 +-
.../snapshot/SnapshotDescriptionUtils.java | 4 +-
.../hadoop/hbase/snapshot/SnapshotInfo.java | 6 +-
.../hadoop/hbase/snapshot/SnapshotManifest.java | 12 +-
.../hbase/snapshot/SnapshotManifestV1.java | 12 +-
.../hbase/snapshot/SnapshotManifestV2.java | 11 +-
.../hbase/snapshot/SnapshotReferenceUtil.java | 6 +-
.../org/apache/hadoop/hbase/util/FSUtils.java | 4 +-
.../org/apache/hadoop/hbase/util/HBaseFsck.java | 4 +-
.../hadoop/hbase/util/ZKDataMigrator.java | 4 +-
.../hbase/util/hbck/TableLockChecker.java | 4 +-
.../org/apache/hadoop/hbase/wal/WALKey.java | 22 +-
.../apache/hadoop/hbase/wal/WALSplitter.java | 49 +-
.../hbase/zookeeper/ClusterStatusTracker.java | 4 +-
.../hbase/zookeeper/LoadBalancerTracker.java | 4 +-
.../zookeeper/RegionNormalizerTracker.java | 4 +-
.../hbase/zookeeper/RegionServerTracker.java | 4 +-
.../hbase/zookeeper/SplitOrMergeTracker.java | 4 +-
.../hadoop/hbase/zookeeper/ZKSplitLog.java | 2 +-
.../hbase-webapps/master/procedures.jsp | 2 +-
.../hbase-webapps/master/snapshotsStats.jsp | 2 +-
.../resources/hbase-webapps/master/table.jsp | 6 +-
.../org/apache/hadoop/hbase/HBaseCluster.java | 6 +-
.../hadoop/hbase/HBaseTestingUtility.java | 2 +-
.../apache/hadoop/hbase/MiniHBaseCluster.java | 8 +-
.../hadoop/hbase/MockRegionServerServices.java | 5 +-
.../org/apache/hadoop/hbase/QosTestHelper.java | 4 +-
.../hadoop/hbase/TestGlobalMemStoreSize.java | 2 +-
.../org/apache/hadoop/hbase/TestIOFencing.java | 4 +-
.../hbase/TestMetaTableAccessorNoCluster.java | 10 +-
.../hadoop/hbase/TestMetaTableLocator.java | 14 +-
.../hadoop/hbase/TestRegionRebalancing.java | 2 +-
.../apache/hadoop/hbase/TestSerialization.java | 6 +-
.../org/apache/hadoop/hbase/TestServerLoad.java | 6 +-
.../hbase/client/HConnectionTestingUtility.java | 4 +-
.../apache/hadoop/hbase/client/TestAdmin1.java | 12 +-
.../apache/hadoop/hbase/client/TestAdmin2.java | 2 +-
.../client/TestClientScannerRPCTimeout.java | 8 +-
.../hadoop/hbase/client/TestClientTimeouts.java | 10 +-
.../hbase/client/TestFromClientSide3.java | 4 +-
.../hbase/client/TestHBaseAdminNoCluster.java | 28 +-
.../org/apache/hadoop/hbase/client/TestHCM.java | 6 +-
.../hadoop/hbase/client/TestMetaCache.java | 10 +-
.../hbase/client/TestMetaWithReplicas.java | 11 +-
.../hbase/client/TestReplicaWithCluster.java | 6 +-
.../hadoop/hbase/client/TestReplicasClient.java | 7 +-
.../hbase/client/TestRpcControllerFactory.java | 221 -
.../client/TestScannersFromClientSide.java | 2 +-
.../client/TestShortCircuitConnection.java | 4 +-
.../hbase/client/TestSnapshotFromClient.java | 2 +-
.../coprocessor/ColumnAggregationEndpoint.java | 117 -
.../ColumnAggregationEndpointNullResponse.java | 129 -
.../ColumnAggregationEndpointWithErrors.java | 126 -
.../coprocessor/ProtobufCoprocessorService.java | 97 -
.../coprocessor/TestAggregateProtocol.java | 831 -
.../TestBatchCoprocessorEndpoint.java | 284 -
.../TestBigDecimalColumnInterpreter.java | 717 -
.../hbase/coprocessor/TestClassLoading.java | 563 -
.../coprocessor/TestCoprocessorEndpoint.java | 345 -
.../TestCoprocessorTableEndpoint.java | 182 -
.../TestDoubleColumnInterpreter.java | 716 -
.../hbase/coprocessor/TestMasterObserver.java | 12 +-
.../TestRegionObserverInterface.java | 2 +-
.../TestRegionServerCoprocessorEndpoint.java | 136 -
.../coprocessor/TestRowProcessorEndpoint.java | 665 -
.../TestForeignExceptionSerialization.java | 12 +-
.../filter/TestColumnPaginationFilter.java | 4 +-
.../filter/TestComparatorSerialization.java | 2 +-
.../hadoop/hbase/filter/TestFilterList.java | 2 +-
.../hbase/filter/TestFilterSerialization.java | 2 +-
.../hadoop/hbase/ipc/AbstractTestIPC.java | 24 +-
.../hbase/ipc/DelegatingRpcScheduler.java | 2 +-
.../hbase/ipc/TestCoprocessorRpcUtils.java | 44 -
.../hadoop/hbase/ipc/TestProtoBufRpc.java | 24 +-
.../hbase/ipc/TestProtobufRpcServiceImpl.java | 27 +-
.../hbase/ipc/TestRpcHandlerException.java | 11 +-
.../hbase/ipc/TestSimpleRpcScheduler.java | 50 +-
.../TestLoadIncrementalHFilesSplitRecovery.java | 13 +-
.../hadoop/hbase/master/MockRegionServer.java | 117 +-
.../master/TestAssignmentManagerOnCluster.java | 2 +-
.../hadoop/hbase/master/TestCatalogJanitor.java | 27 +-
.../hbase/master/TestClockSkewDetection.java | 2 +-
.../master/TestDistributedLogSplitting.java | 2 +-
.../master/TestGetLastFlushedSequenceId.java | 2 +-
.../hbase/master/TestHMasterRPCException.java | 15 +-
.../hadoop/hbase/master/TestMasterFailover.java | 4 +-
.../hadoop/hbase/master/TestMasterMetrics.java | 6 +-
.../hbase/master/TestMasterNoCluster.java | 11 +-
.../hbase/master/TestMasterQosFunction.java | 22 +-
.../hadoop/hbase/master/TestRegionState.java | 2 +-
.../hadoop/hbase/master/TestRollingRestart.java | 2 +-
.../hbase/master/TestSplitLogManager.java | 2 +-
.../hbase/master/TestTableStateManager.java | 4 +-
.../master/cleaner/TestSnapshotFromMaster.java | 17 +-
.../normalizer/TestSimpleRegionNormalizer.java | 32 +-
.../procedure/TestAddColumnFamilyProcedure.java | 2 +-
.../procedure/TestCloneSnapshotProcedure.java | 6 +-
.../procedure/TestCreateNamespaceProcedure.java | 2 +-
.../TestDeleteColumnFamilyProcedure.java | 2 +-
.../procedure/TestDeleteNamespaceProcedure.java | 2 +-
.../procedure/TestDisableTableProcedure.java | 2 +-
.../TestDispatchMergingRegionsProcedure.java | 2 +-
.../procedure/TestEnableTableProcedure.java | 2 +-
.../TestMasterFailoverWithProcedures.java | 10 +-
.../procedure/TestMasterProcedureEvents.java | 2 +-
.../TestModifyColumnFamilyProcedure.java | 2 +-
.../procedure/TestModifyNamespaceProcedure.java | 2 +-
.../procedure/TestModifyTableProcedure.java | 2 +-
.../procedure/TestRestoreSnapshotProcedure.java | 6 +-
.../procedure/TestTruncateTableProcedure.java | 1 +
.../master/snapshot/TestSnapshotFileCache.java | 2 +-
.../procedure/SimpleMasterProcedureManager.java | 2 +-
.../procedure/TestZKProcedureControllers.java | 2 +-
.../hadoop/hbase/protobuf/TestProtobufUtil.java | 13 +-
.../hadoop/hbase/quotas/TestQuotaState.java | 6 +-
.../hadoop/hbase/quotas/TestQuotaTableUtil.java | 9 +-
.../hbase/regionserver/OOMERegionServer.java | 16 +-
.../SecureBulkLoadEndpointClient.java | 168 -
.../hadoop/hbase/regionserver/TestBulkLoad.java | 6 +-
.../TestEndToEndSplitTransaction.java | 23 +-
.../hadoop/hbase/regionserver/TestHRegion.java | 71 +-
.../hbase/regionserver/TestHRegionInfo.java | 19 +-
.../regionserver/TestHRegionReplayEvents.java | 65 +-
.../regionserver/TestHRegionServerBulkLoad.java | 18 +-
.../TestHRegionServerBulkLoadWithOldClient.java | 20 +-
...gionServerBulkLoadWithOldSecureEndpoint.java | 179 -
.../hbase/regionserver/TestPriorityRpc.java | 21 +-
.../hbase/regionserver/TestQosFunction.java | 2 +-
.../TestRSKilledWhenInitializing.java | 4 +-
.../hbase/regionserver/TestRSStatusServlet.java | 19 +-
.../regionserver/TestRegionFavoredNodes.java | 9 +-
.../TestRegionMergeTransactionOnCluster.java | 10 +-
.../hbase/regionserver/TestRegionReplicas.java | 22 +-
.../regionserver/TestRegionServerNoMaster.java | 27 +-
.../TestScannerHeartbeatMessages.java | 8 +-
.../regionserver/TestServerCustomProtocol.java | 480 -
.../hbase/regionserver/TestSplitLogWorker.java | 2 +-
.../TestSplitTransactionOnCluster.java | 14 +-
.../regionserver/wal/AbstractTestWALReplay.java | 2 +-
.../replication/TestPerTableCFReplication.java | 2 +-
.../replication/TestReplicationSmallTests.java | 2 +-
...ReplicationSyncUpToolWithBulkLoadedData.java | 235 -
...egionReplicaReplicationEndpointNoMaster.java | 2 +-
.../regionserver/TestReplicationSink.java | 19 +-
.../TestReplicationSinkManager.java | 2 +-
.../TestReplicationSourceManager.java | 8 +-
.../hadoop/hbase/security/TestSecureIPC.java | 16 +-
.../hbase/security/access/SecureTestUtil.java | 18 +-
.../security/access/TestAccessController.java | 28 +-
.../security/access/TestNamespaceCommands.java | 13 +-
.../access/TestWithDisabledAuthorization.java | 4 +-
.../token/TestGenerateDelegationToken.java | 6 +-
.../security/token/TestTokenAuthentication.java | 214 +-
.../hbase/snapshot/SnapshotTestingUtils.java | 20 +-
.../hbase/snapshot/TestExportSnapshot.java | 5 +-
.../snapshot/TestExportSnapshotHelpers.java | 9 +-
.../snapshot/TestFlushSnapshotFromClient.java | 4 +-
.../TestRestoreFlushSnapshotFromClient.java | 4 +-
.../snapshot/TestRestoreSnapshotHelper.java | 2 +-
.../snapshot/TestSnapshotClientRetries.java | 2 +-
.../snapshot/TestSnapshotDescriptionUtils.java | 2 +-
.../hbase/snapshot/TestSnapshotManifest.java | 23 +-
.../hadoop/hbase/util/BaseTestHBaseFsck.java | 4 +-
.../hadoop/hbase/util/MultiThreadedAction.java | 2 +-
.../hadoop/hbase/util/MultiThreadedUpdater.java | 2 +-
.../hadoop/hbase/util/TestHBaseFsckTwoRS.java | 3 +-
.../hadoop/hbase/wal/TestWALFiltering.java | 12 +-
.../apache/hadoop/hbase/wal/TestWALMethods.java | 2 +-
.../hbase/wal/TestWALReaderOnSecureWAL.java | 2 +-
.../apache/hadoop/hbase/wal/TestWALSplit.java | 6 +-
hbase-spark/README.txt | 35 +
hbase-spark/pom.xml | 40 +
.../protobuf/generated/SparkFilterProtos.java | 2006 +
hbase-spark/src/main/protobuf/SparkFilter.proto | 40 +
...TestThriftHBaseServiceHandlerWithLabels.java | 3 +-
pom.xml | 21 +-
src/main/asciidoc/_chapters/cp.adoc | 9 +-
src/main/asciidoc/_chapters/protobuf.adoc | 169 +
src/main/asciidoc/book.adoc | 1 +
846 files changed, 474237 insertions(+), 172053 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/pom.xml
----------------------------------------------------------------------
diff --git a/hbase-client/pom.xml b/hbase-client/pom.xml
index acde9de..f3e27bc 100644
--- a/hbase-client/pom.xml
+++ b/hbase-client/pom.xml
@@ -134,6 +134,10 @@
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
+ <artifactId>hbase-protocol-shaded</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.hbase</groupId>
<artifactId>hbase-protocol</artifactId>
</dependency>
<!-- General dependencies -->
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterId.java
----------------------------------------------------------------------
diff --git a/hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterId.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterId.java
index c127627..6d47da6 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterId.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterId.java
@@ -23,8 +23,8 @@ import java.util.UUID;
import org.apache.hadoop.hbase.classification.InterfaceAudience;
import org.apache.hadoop.hbase.exceptions.DeserializationException;
-import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
-import org.apache.hadoop.hbase.protobuf.generated.ClusterIdProtos;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterIdProtos;
import org.apache.hadoop.hbase.util.Bytes;
/**
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
----------------------------------------------------------------------
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
index ffeb51a..f00016d 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
@@ -30,14 +30,14 @@ import java.util.Set;
import org.apache.hadoop.hbase.classification.InterfaceAudience;
import org.apache.hadoop.hbase.classification.InterfaceStability;
import org.apache.hadoop.hbase.master.RegionState;
-import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
-import org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos;
-import
org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos.LiveServerInfo;
-import
org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos.RegionInTransition;
-import
org.apache.hadoop.hbase.protobuf.generated.FSProtos.HBaseVersionFileContent;
-import org.apache.hadoop.hbase.protobuf.generated.HBaseProtos;
-import org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.RegionSpecifier;
-import
org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.RegionSpecifier.RegionSpecifierType;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.LiveServerInfo;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionInTransition;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.FSProtos.HBaseVersionFileContent;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.RegionSpecifier;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.RegionSpecifier.RegionSpecifierType;
import org.apache.hadoop.hbase.util.ByteStringer;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.io.VersionedWritable;
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
----------------------------------------------------------------------
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
index b75e8cd..d940509 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
@@ -34,8 +34,8 @@ import
org.apache.hadoop.hbase.exceptions.DeserializationException;
import org.apache.hadoop.hbase.exceptions.HBaseException;
import org.apache.hadoop.hbase.io.compress.Compression;
import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
-import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
-import
org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.ColumnFamilySchema;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ColumnFamilySchema;
import org.apache.hadoop.hbase.regionserver.BloomType;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.PrettyPrinter;
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
----------------------------------------------------------------------
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
index ab9045d..da0d941 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
@@ -33,11 +33,11 @@ import org.apache.hadoop.hbase.client.RegionReplicaUtil;
import org.apache.hadoop.hbase.KeyValue.KVComparator;
import org.apache.hadoop.hbase.exceptions.DeserializationException;
import org.apache.hadoop.hbase.master.RegionState;
-import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
-import org.apache.hadoop.hbase.protobuf.generated.HBaseProtos;
-import org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.RegionInfo;
+import org.apache.hadoop.hbase.shaded.com.google.protobuf.UnsafeByteOperations;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.RegionInfo;
import org.apache.hadoop.hbase.util.ByteArrayHashKey;
-import org.apache.hadoop.hbase.util.ByteStringer;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.HashKey;
import org.apache.hadoop.hbase.util.JenkinsHash;
@@ -873,10 +873,10 @@ public class HRegionInfo implements
Comparable<HRegionInfo> {
builder.setTableName(ProtobufUtil.toProtoTableName(info.getTable()));
builder.setRegionId(info.getRegionId());
if (info.getStartKey() != null) {
- builder.setStartKey(ByteStringer.wrap(info.getStartKey()));
+ builder.setStartKey(UnsafeByteOperations.unsafeWrap(info.getStartKey()));
}
if (info.getEndKey() != null) {
- builder.setEndKey(ByteStringer.wrap(info.getEndKey()));
+ builder.setEndKey(UnsafeByteOperations.unsafeWrap(info.getEndKey()));
}
builder.setOffline(info.isOffline());
builder.setSplit(info.isSplit());
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
----------------------------------------------------------------------
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
index 9abdf42..3d8d7da 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
@@ -40,8 +40,8 @@ import
org.apache.hadoop.hbase.classification.InterfaceStability;
import org.apache.hadoop.hbase.client.Durability;
import org.apache.hadoop.hbase.client.RegionReplicaUtil;
import org.apache.hadoop.hbase.exceptions.DeserializationException;
-import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
-import org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.TableSchema;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.TableSchema;
import org.apache.hadoop.hbase.security.User;
import org.apache.hadoop.hbase.util.Bytes;
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
----------------------------------------------------------------------
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
index 2b50829..09980c0 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
@@ -17,9 +17,6 @@
*/
package org.apache.hadoop.hbase;
-import com.google.common.annotations.VisibleForTesting;
-import com.google.protobuf.ServiceException;
-
import java.io.Closeable;
import java.io.IOException;
import java.io.InterruptedIOException;
@@ -36,8 +33,6 @@ import java.util.TreeMap;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
-import edu.umd.cs.findbugs.annotations.NonNull;
-import edu.umd.cs.findbugs.annotations.Nullable;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.conf.Configuration;
@@ -51,6 +46,7 @@ import org.apache.hadoop.hbase.client.Mutation;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.RegionLocator;
import org.apache.hadoop.hbase.client.RegionReplicaUtil;
+import org.apache.hadoop.hbase.client.RegionServerCallable;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
@@ -60,13 +56,22 @@ import
org.apache.hadoop.hbase.exceptions.DeserializationException;
import org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel;
import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
import org.apache.hadoop.hbase.protobuf.generated.ClientProtos;
+import org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.RegionSpecifier;
+import
org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.RegionSpecifier.RegionSpecifierType;
import org.apache.hadoop.hbase.protobuf.generated.MultiRowMutationProtos;
+import
org.apache.hadoop.hbase.protobuf.generated.MultiRowMutationProtos.MutateRowsRequest;
+import
org.apache.hadoop.hbase.protobuf.generated.MultiRowMutationProtos.MutateRowsResponse;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
import org.apache.hadoop.hbase.util.ExceptionUtil;
import org.apache.hadoop.hbase.util.Pair;
import org.apache.hadoop.hbase.util.PairOfSameType;
+import com.google.common.annotations.VisibleForTesting;
+
+import edu.umd.cs.findbugs.annotations.NonNull;
+import edu.umd.cs.findbugs.annotations.Nullable;
+
/**
* Read/write operations on region and assignment information store in
* <code>hbase:meta</code>.
@@ -1677,7 +1682,7 @@ public class MetaTableAccessor {
} else {
mutations = new Mutation[] { putOfMerged, deleteA, deleteB };
}
- multiMutate(meta, tableRow, mutations);
+ multiMutate(connection, meta, tableRow, mutations);
} finally {
meta.close();
}
@@ -1732,7 +1737,7 @@ public class MetaTableAccessor {
mutations = new Mutation[]{putParent, putA, putB};
}
byte[] tableRow = Bytes.toBytes(parent.getRegionNameAsString() +
HConstants.DELIMITER);
- multiMutate(meta, tableRow, mutations);
+ multiMutate(connection, meta, tableRow, mutations);
} finally {
meta.close();
}
@@ -1777,37 +1782,74 @@ public class MetaTableAccessor {
LOG.info("Deleted table " + table + " state from META");
}
+ private static void multiMutate(Connection connection, Table table, byte[]
row,
+ Mutation... mutations)
+ throws IOException {
+ multiMutate(connection, table, row, Arrays.asList(mutations));
+ }
+
/**
- * Performs an atomic multi-Mutate operation against the given table.
+ * Performs an atomic multi-mutate operation against the given table.
*/
- private static void multiMutate(Table table, byte[] row, Mutation...
mutations)
- throws IOException {
- CoprocessorRpcChannel channel = table.coprocessorService(row);
- MultiRowMutationProtos.MutateRowsRequest.Builder mmrBuilder
- = MultiRowMutationProtos.MutateRowsRequest.newBuilder();
+ // Used by the RSGroup Coprocessor Endpoint. It had a copy/paste of the
below. Need to reveal
+ // this facility for CPEP use or at least those CPEPs that are on their way
to becoming part of
+ // core as is the intent for RSGroup eventually.
+ public static void multiMutate(Connection connection, final Table table,
byte[] row,
+ final List<Mutation> mutations)
+ throws IOException {
if (METALOG.isDebugEnabled()) {
METALOG.debug(mutationsToString(mutations));
}
- for (Mutation mutation : mutations) {
- if (mutation instanceof Put) {
- mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(
- ClientProtos.MutationProto.MutationType.PUT, mutation));
- } else if (mutation instanceof Delete) {
- mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(
- ClientProtos.MutationProto.MutationType.DELETE, mutation));
- } else {
- throw new DoNotRetryIOException("multi in MetaEditor doesn't support "
- + mutation.getClass().getName());
+ // TODO: Need rollback!!!!
+ // TODO: Need Retry!!!
+ // TODO: What for a timeout? Default write timeout? GET FROM HTABLE?
+ // TODO: Review when we come through with ProcedureV2.
+ RegionServerCallable<MutateRowsResponse,
+ MultiRowMutationProtos.MultiRowMutationService.BlockingInterface>
callable =
+ new RegionServerCallable<MutateRowsResponse,
+ MultiRowMutationProtos.MultiRowMutationService.BlockingInterface>(
+ connection, table.getName(), row, null/*RpcController not used
in this CPEP!*/) {
+ @Override
+ protected MutateRowsResponse rpcCall() throws Exception {
+ final MutateRowsRequest.Builder builder =
MutateRowsRequest.newBuilder();
+ for (Mutation mutation : mutations) {
+ if (mutation instanceof Put) {
+ builder.addMutationRequest(ProtobufUtil.toMutation(
+ ClientProtos.MutationProto.MutationType.PUT, mutation));
+ } else if (mutation instanceof Delete) {
+ builder.addMutationRequest(ProtobufUtil.toMutation(
+ ClientProtos.MutationProto.MutationType.DELETE, mutation));
+ } else {
+ throw new DoNotRetryIOException("multi in MetaEditor doesn't
support "
+ + mutation.getClass().getName());
+ }
+ }
+ // The call to #prepare that ran before this invocation will have
populated HRegionLocation.
+ HRegionLocation hrl = getLocation();
+ RegionSpecifier region = ProtobufUtil.buildRegionSpecifier(
+ RegionSpecifierType.REGION_NAME,
hrl.getRegionInfo().getRegionName());
+ builder.setRegion(region);
+ // The rpcController here is awkward. The Coprocessor Endpoint wants
an instance of a
+ // com.google.protobuf but we are going over an rpc that is all shaded
protobuf so it
+ // wants a org.apache.h.h.shaded.com.google.protobuf.RpcController.
Set up a factory
+ // that makes com.google.protobuf.RpcController and then copy into it
configs.
+ return getStub().mutateRows(null, builder.build());
}
- }
- MultiRowMutationProtos.MultiRowMutationService.BlockingInterface service =
- MultiRowMutationProtos.MultiRowMutationService.newBlockingStub(channel);
- try {
- service.mutateRows(null, mmrBuilder.build());
- } catch (ServiceException ex) {
- ProtobufUtil.toIOException(ex);
- }
+ @Override
+ // Called on the end of the super.prepare call. Set the stub.
+ protected void setStubByServiceName(ServerName serviceName/*Ignored*/)
throws IOException {
+ CoprocessorRpcChannel channel = table.coprocessorService(getRow());
+
setStub(MultiRowMutationProtos.MultiRowMutationService.newBlockingStub(channel));
+ }
+ };
+ int writeTimeout =
connection.getConfiguration().getInt(HConstants.HBASE_RPC_WRITE_TIMEOUT_KEY,
+ connection.getConfiguration().getInt(HConstants.HBASE_RPC_TIMEOUT_KEY,
+ HConstants.DEFAULT_HBASE_RPC_TIMEOUT));
+ // The region location should be cached in connection. Call prepare so
this callable picks
+ // up the region location (see super.prepare method).
+ callable.prepare(false);
+ callable.call(writeTimeout);
}
/**
@@ -2026,16 +2068,6 @@ public class MetaTableAccessor {
return p;
}
- private static String mutationsToString(Mutation ... mutations) throws
IOException {
- StringBuilder sb = new StringBuilder();
- String prefix = "";
- for (Mutation mutation : mutations) {
- sb.append(prefix).append(mutationToString(mutation));
- prefix = ", ";
- }
- return sb.toString();
- }
-
private static String mutationsToString(List<? extends Mutation> mutations)
throws IOException {
StringBuilder sb = new StringBuilder();
String prefix = "";
@@ -2169,5 +2201,4 @@ public class MetaTableAccessor {
}
return null;
}
-
}
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/src/main/java/org/apache/hadoop/hbase/RegionLoad.java
----------------------------------------------------------------------
diff --git a/hbase-client/src/main/java/org/apache/hadoop/hbase/RegionLoad.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/RegionLoad.java
index 1702767..befb2de 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/RegionLoad.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/RegionLoad.java
@@ -24,8 +24,8 @@ import java.util.List;
import org.apache.hadoop.hbase.classification.InterfaceAudience;
import org.apache.hadoop.hbase.classification.InterfaceStability;
-import org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos;
-import
org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos.StoreSequenceId;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.StoreSequenceId;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.Strings;
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/src/main/java/org/apache/hadoop/hbase/ServerLoad.java
----------------------------------------------------------------------
diff --git a/hbase-client/src/main/java/org/apache/hadoop/hbase/ServerLoad.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ServerLoad.java
index 3ea59db..d16c90f 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/ServerLoad.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/ServerLoad.java
@@ -28,9 +28,9 @@ import java.util.TreeSet;
import org.apache.hadoop.hbase.classification.InterfaceAudience;
import org.apache.hadoop.hbase.classification.InterfaceStability;
-import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
-import org.apache.hadoop.hbase.protobuf.generated.ClusterStatusProtos;
-import org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.Coprocessor;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.Coprocessor;
import org.apache.hadoop.hbase.replication.ReplicationLoadSink;
import org.apache.hadoop.hbase.replication.ReplicationLoadSource;
import org.apache.hadoop.hbase.util.Bytes;
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRegionServerCallable.java
----------------------------------------------------------------------
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRegionServerCallable.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRegionServerCallable.java
deleted file mode 100644
index 5a1f5cc..0000000
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRegionServerCallable.java
+++ /dev/null
@@ -1,138 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.hadoop.hbase.client;
-
-import java.io.IOException;
-
-import org.apache.hadoop.hbase.HRegionInfo;
-import org.apache.hadoop.hbase.HRegionLocation;
-import org.apache.hadoop.hbase.ServerName;
-import org.apache.hadoop.hbase.TableName;
-import org.apache.hadoop.hbase.TableNotEnabledException;
-import org.apache.hadoop.hbase.classification.InterfaceAudience;
-import org.apache.hadoop.hbase.util.Bytes;
-
-/**
- * Added by HBASE-15745 Refactor of RPC classes to better accept async changes.
- * Temporary.
- */
-@InterfaceAudience.Private
-abstract class AbstractRegionServerCallable<T> implements RetryingCallable<T> {
- protected final Connection connection;
- protected final TableName tableName;
- protected final byte[] row;
- protected HRegionLocation location;
- protected final static int MIN_WAIT_DEAD_SERVER = 10000;
-
- /**
- * @param connection Connection to use.
- * @param tableName Table name to which <code>row</code> belongs.
- * @param row The row we want in <code>tableName</code>.
- */
- public AbstractRegionServerCallable(Connection connection, TableName
tableName, byte[] row) {
- this.connection = connection;
- this.tableName = tableName;
- this.row = row;
- }
-
- /**
- * @return {@link ClusterConnection} instance used by this Callable.
- */
- ClusterConnection getConnection() {
- return (ClusterConnection) this.connection;
- }
-
- protected HRegionLocation getLocation() {
- return this.location;
- }
-
- protected void setLocation(final HRegionLocation location) {
- this.location = location;
- }
-
- public TableName getTableName() {
- return this.tableName;
- }
-
- public byte [] getRow() {
- return this.row;
- }
-
- @Override
- public void throwable(Throwable t, boolean retrying) {
- if (location != null) {
- getConnection().updateCachedLocations(tableName,
location.getRegionInfo().getRegionName(),
- row, t, location.getServerName());
- }
- }
-
- @Override
- public String getExceptionMessageAdditionalDetail() {
- return "row '" + Bytes.toString(row) + "' on table '" + tableName + "' at
" + location;
- }
-
- @Override
- public long sleep(long pause, int tries) {
- long sleep = ConnectionUtils.getPauseTime(pause, tries);
- if (sleep < MIN_WAIT_DEAD_SERVER
- && (location == null ||
getConnection().isDeadServer(location.getServerName()))) {
- sleep = ConnectionUtils.addJitter(MIN_WAIT_DEAD_SERVER, 0.10f);
- }
- return sleep;
- }
-
- /**
- * @return the HRegionInfo for the current region
- */
- public HRegionInfo getHRegionInfo() {
- if (this.location == null) {
- return null;
- }
- return this.location.getRegionInfo();
- }
-
- /**
- * Prepare for connection to the server hosting region with row from
tablename. Does lookup
- * to find region location and hosting server.
- * @param reload Set to true to re-check the table state
- * @throws IOException e
- */
- @Override
- public void prepare(final boolean reload) throws IOException {
- // check table state if this is a retry
- if (reload && !tableName.equals(TableName.META_TABLE_NAME) &&
- getConnection().isTableDisabled(tableName)) {
- throw new TableNotEnabledException(tableName.getNameAsString() + " is
disabled.");
- }
- try (RegionLocator regionLocator = connection.getRegionLocator(tableName))
{
- this.location = regionLocator.getRegionLocation(row);
- }
- if (this.location == null) {
- throw new IOException("Failed to find location, tableName=" + tableName +
- ", row=" + Bytes.toString(row) + ", reload=" + reload);
- }
- setClientByServiceName(this.location.getServerName());
- }
-
- /**
- * Set the Rpc client for Client services
- * @param serviceName to get client for
- * @throws IOException When client could not be created
- */
- abstract void setClientByServiceName(ServerName serviceName) throws
IOException;
-}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
----------------------------------------------------------------------
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
index 2ffb2e3..f2d9546 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
@@ -19,24 +19,6 @@
package org.apache.hadoop.hbase.client;
-import com.google.common.annotations.VisibleForTesting;
-import org.apache.commons.logging.Log;
-import org.apache.commons.logging.LogFactory;
-import org.apache.hadoop.conf.Configuration;
-import org.apache.hadoop.hbase.HConstants;
-import org.apache.hadoop.hbase.HRegionInfo;
-import org.apache.hadoop.hbase.HRegionLocation;
-import org.apache.hadoop.hbase.RegionLocations;
-import org.apache.hadoop.hbase.ServerName;
-import org.apache.hadoop.hbase.TableName;
-import org.apache.hadoop.hbase.classification.InterfaceAudience;
-import org.apache.hadoop.hbase.client.AsyncProcess.RowChecker.ReturnCode;
-import org.apache.hadoop.hbase.client.coprocessor.Batch;
-import org.apache.hadoop.hbase.ipc.RpcControllerFactory;
-import org.apache.hadoop.hbase.util.Bytes;
-import org.apache.hadoop.hbase.util.EnvironmentEdge;
-import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
-
import java.io.IOException;
import java.io.InterruptedIOException;
import java.util.ArrayList;
@@ -56,6 +38,25 @@ import java.util.concurrent.ExecutorService;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.RegionLocations;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.classification.InterfaceAudience;
+import org.apache.hadoop.hbase.client.AsyncProcess.RowChecker.ReturnCode;
+import org.apache.hadoop.hbase.client.coprocessor.Batch;
+import org.apache.hadoop.hbase.ipc.RpcControllerFactory;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.EnvironmentEdge;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+
+import com.google.common.annotations.VisibleForTesting;
+
/**
* This class allows a continuous flow of requests. It's written to be
compatible with a
* synchronous caller such as HTable.
@@ -125,7 +126,7 @@ class AsyncProcess {
public static final String HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE =
"hbase.client.max.perrequest.heapsize";
/**
- * Default value of {@link #HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE}.
+ * Default value of #HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE
*/
public static final long DEFAULT_HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE =
4194304;
@@ -134,7 +135,7 @@ class AsyncProcess {
*/
public static final String HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE =
"hbase.client.max.submit.heapsize";
/**
- * Default value of {@link #HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE}.
+ * Default value of #HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE
*/
public static final long DEFAULT_HBASE_CLIENT_MAX_SUBMIT_HEAPSIZE =
DEFAULT_HBASE_CLIENT_MAX_PERREQUEST_HEAPSIZE;
@@ -306,7 +307,7 @@ class AsyncProcess {
}
/**
- * See {@link #submit(ExecutorService, TableName, RowAccess, boolean,
Batch.Callback, boolean)}.
+ * See #submit(ExecutorService, TableName, RowAccess, boolean,
Batch.Callback, boolean).
* Uses default ExecutorService for this AP (must have been created with
one).
*/
public <CResult> AsyncRequestFuture submit(TableName tableName,
@@ -740,7 +741,7 @@ class AsyncProcess {
/**
* Provide a way to control the flow of rows iteration.
*/
- @VisibleForTesting
+ // Visible for Testing. Adding @VisibleForTesting here doesn't work for some
reason.
interface RowChecker {
enum ReturnCode {
/**
http://git-wip-us.apache.org/repos/asf/hbase/blob/95c1dc93/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRequestFutureImpl.java
----------------------------------------------------------------------
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRequestFutureImpl.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRequestFutureImpl.java
index c6b2a53..3894d58 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRequestFutureImpl.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRequestFutureImpl.java
@@ -33,8 +33,8 @@ import
org.apache.hadoop.hbase.classification.InterfaceAudience;
import org.apache.hadoop.hbase.client.backoff.ServerStatistics;
import org.apache.hadoop.hbase.client.coprocessor.Batch;
import org.apache.hadoop.hbase.exceptions.ClientExceptionsUtil;
-import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
-import org.apache.hadoop.hbase.protobuf.generated.ClientProtos;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
import org.apache.htrace.Trace;
@@ -1284,7 +1284,6 @@ class AsyncRequestFutureImpl<CResult> implements
AsyncRequestFuture {
private MultiServerCallable<Row> createCallable(final ServerName server,
TableName tableName, final
MultiAction<Row> multi) {
return new MultiServerCallable<Row>(asyncProcess.connection, tableName,
server,
- asyncProcess.rpcFactory, multi);
+ multi, asyncProcess.rpcFactory.newController());
}
-
}
\ No newline at end of file
