[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024923#comment-17024923
 ] 

Hudson commented on HBASE-22514:


Results for branch HBASE-22514
[build #255 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/255/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/255//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/255//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/255//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
>  Issue Type: Umbrella
>  Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch, 
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public 
> we need to use java api  RSGroupAdminClient  to manager RSG 
> so  RSGroupAdminClient should be public
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23746) [Flakey Tests] Caused by: org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: hflush and hsync

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024911#comment-17024911
 ] 

Hudson commented on HBASE-23746:


Results for branch branch-2
[build #2435 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2435/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2435//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2435//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2435//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> [Flakey Tests] Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
> -
>
> Key: HBASE-23746
> URL: https://issues.apache.org/jira/browse/HBASE-23746
> Project: HBase
>  Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 23746.patch
>
>
> Tests that use the new RegionProcedureStoreTestBase utility fail in branch-2 
> nightlies complaining...
> {code}
> Error Message
> cannot get log writer
> Stacktrace
> java.io.IOException: cannot get log writer
> Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
> {code}
> They are using local fs which doesn't have support for above. Only shows when 
> you run the tests with hadoop3 profile.  Let me do for these tests what was 
> done over in HBASE-19289
> It has been going on for a while now, probably since the move to region 
> procedure store (the old wal procedure store had turned off the checks for 
> sync/hflush).
> {code}
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestHFileProcedurePrettyPrinter.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStore.testLoad
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStore.testCleanup
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreCompaction.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreMigration.testMigrateWithUnsupportedProcedures
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreMigration.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreWALCleaner.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestWALProcedurePrettyPrinter.test
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371619365
 
 

 ##
 File path: hbase-protocol-shaded/src/main/protobuf/TooSlowLog.proto
 ##
 @@ -0,0 +1,45 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+syntax = "proto2";
+
+// This file contains protocol buffers that are used for Online TooSlowLogs
+// To be used as Ring Buffer payload
+package hbase.pb;
+
+option java_package = "org.apache.hadoop.hbase.shaded.protobuf.generated";
+option java_outer_classname = "TooSlowLog";
+option java_generate_equals_and_hash = true;
+option optimize_for = SPEED;
+
+message SlowLogPayload {
+required int64 start_time = 1;
+required int32 processing_time = 2;
+required int32 queue_time = 3;
+required int64 response_size = 4;
+required string client_address = 5;
+required string server_class = 6;
+required string method_name = 7;
+required string call_details = 8;
+required string param = 9;
+required string user_name = 10;
 
 Review comment:
   Based on the Jira, we need to provide filter for user so definitely we need 
user here, even if it is just `hbase` :)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371628695
 
 

 ##
 File path: 
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
 ##
 @@ -426,4 +437,55 @@ public void testAsyncTimeout() throws IOException {
   rpcServer.stop();
 }
   }
+
+  @Test
 
 Review comment:
   I am actually looking for unit testing `rpcServer.logResponse()` which 
ideally should land here, it's just that while testing this method, we are also 
testing whether or not ringBuffer was filled up with records


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371608183
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/SlowLogPayload.java
 ##
 @@ -0,0 +1,318 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.client;
+
+import org.apache.hadoop.hbase.util.GsonUtil;
+import org.apache.yetus.audience.InterfaceAudience;
+
+import org.apache.hbase.thirdparty.com.google.gson.Gson;
+import org.apache.hbase.thirdparty.com.google.gson.JsonObject;
+import org.apache.hbase.thirdparty.com.google.gson.JsonSerializer;
+
+/**
+ * SlowLog payload for client
+ */
+@InterfaceAudience.Private
+final public class SlowLogPayload {
+
+  private static final Gson GSON = GsonUtil.createGson()
+.setPrettyPrinting()
+.registerTypeAdapter(SlowLogPayload.class, (JsonSerializer)
+  (slowLogPayload, type, jsonSerializationContext) -> {
 
 Review comment:
   Since we are using shaded Gson and we want to exclude multiXYZ if they have 
value 0, this is an easy way but still I am not sure of any filter that can 
look better and do the same for us. Open for suggestion if you know of any.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371637044
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
 ##
 @@ -3818,6 +3828,40 @@ public ExecuteProceduresResponse 
executeProcedures(RpcController controller,
 }
   }
 
+  @Override
+  @QosPriority(priority = HConstants.ADMIN_QOS)
+  public SlowLogResponses getSlowLogResponses(final RpcController controller,
+  final SlowLogResponseRequest request) throws ServiceException {
+final OnlineSlowLogProvider onlineSlowLogProvider =
+  this.regionServer.getOnlineSlowLogProvider();
+List slowLogPayloads;
+if (onlineSlowLogProvider != null) {
+  slowLogPayloads = onlineSlowLogProvider.getSlowLogPayloads();
+} else {
+  slowLogPayloads = new ArrayList<>();
+}
+SlowLogResponses slowLogResponses = SlowLogResponses.newBuilder()
+  .addAllSlowLogPayloads(slowLogPayloads)
+  .build();
+return slowLogResponses;
+  }
+
+  @Override
+  @QosPriority(priority = HConstants.ADMIN_QOS)
+  public ClearSlowLogResponses clearSlowLogsResponses(final RpcController 
controller,
+  final ClearSlowLogResponseRequest request) throws ServiceException {
+final OnlineSlowLogProvider onlineSlowLogProvider =
+  this.regionServer.getOnlineSlowLogProvider();
+boolean slowLogsCleaned = false;
 
 Review comment:
   Thanks for the suggestion


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371607074
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 ##
 @@ -590,6 +596,10 @@ public HRegionServer(final Configuration conf) throws 
IOException {
   this.abortRequested = false;
   this.stopped = false;
 
+  // initiate online slowlog ringbuffer only for RegionServers
+  if (!(this instanceof HMaster)) {
 
 Review comment:
   Yes because even when the conf is disabled, we can let get_slow_responses() 
work and return empty list at least? User needs to take care of enabling it if 
they want to use. Fine?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371610260
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/SlowLogPayload.java
 ##
 @@ -0,0 +1,318 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.client;
+
+import org.apache.hadoop.hbase.util.GsonUtil;
+import org.apache.yetus.audience.InterfaceAudience;
+
+import org.apache.hbase.thirdparty.com.google.gson.Gson;
+import org.apache.hbase.thirdparty.com.google.gson.JsonObject;
+import org.apache.hbase.thirdparty.com.google.gson.JsonSerializer;
+
+/**
+ * SlowLog payload for client
+ */
+@InterfaceAudience.Private
+final public class SlowLogPayload {
+
+  private static final Gson GSON = GsonUtil.createGson()
+.setPrettyPrinting()
+.registerTypeAdapter(SlowLogPayload.class, (JsonSerializer)
+  (slowLogPayload, type, jsonSerializationContext) -> {
+Gson gson = new Gson();
+JsonObject jsonObj = (JsonObject) gson.toJsonTree(slowLogPayload);
+if (slowLogPayload.getMultiGetsCount() == 0) {
+  jsonObj.remove("multiGetsCount");
+}
+if (slowLogPayload.getMultiMutationsCount() == 0) {
+  jsonObj.remove("multiMutationsCount");
+}
+if (slowLogPayload.getMultiServiceCalls() == 0) {
+  jsonObj.remove("multiServiceCalls");
+}
+return jsonObj;
+  }).create();
+
+  private long startTime;
+  private int processingTime;
+  private int queueTime;
+  private long responseSize;
+  private String clientAddress;
+  private String serverClass;
+  private String methodName;
+  private String callDetails;
+  private String param;
+  private transient String regionName;
+  private String userName;
+  private int multiGetsCount;
+  private int multiMutationsCount;
+  private int multiServiceCalls;
+
+  public long getStartTime() {
+return startTime;
+  }
+
+  public int getProcessingTime() {
+return processingTime;
+  }
+
+  public int getQueueTime() {
+return queueTime;
+  }
+
+  public long getResponseSize() {
+return responseSize;
+  }
+
+  public String getClientAddress() {
+return clientAddress;
+  }
+
+  public String getServerClass() {
+return serverClass;
+  }
+
+  public String getMethodName() {
+return methodName;
+  }
+
+  public String getCallDetails() {
+return callDetails;
+  }
+
+  public String getParam() {
+return param;
+  }
+
+  public String getRegionName() {
+return regionName;
+  }
+
+  public String getUserName() {
+return userName;
+  }
+
+  public int getMultiGetsCount() {
+return multiGetsCount;
+  }
+
+  public int getMultiMutationsCount() {
+return multiMutationsCount;
+  }
+
+  public int getMultiServiceCalls() {
+return multiServiceCalls;
+  }
+
+  private SlowLogPayload(final long startTime, final int processingTime, final 
int queueTime,
+  final long responseSize, final String clientAddress, final String 
serverClass,
+  final String methodName, final String callDetails, final String param,
+  final String regionName, final String userName, final int multiGetsCount,
+  final int multiMutationsCount, final int multiServiceCalls) {
+this.startTime = startTime;
+this.processingTime = processingTime;
+this.queueTime = queueTime;
+this.responseSize = responseSize;
+this.clientAddress = clientAddress;
+this.serverClass = serverClass;
+this.methodName = methodName;
+this.callDetails = callDetails;
+this.param = param;
+this.regionName = regionName;
+this.userName = userName;
+this.multiGetsCount = multiGetsCount;
+this.multiMutationsCount = multiMutationsCount;
+this.multiServiceCalls = multiServiceCalls;
+  }
+
+  public static class SlowLogPayloadBuilder {
+private long startTime;
+private int processingTime;
+private int queueTime;
+private long responseSize;
+private String clientAddress;
+private String serverClass;
+private String methodName;
+private String callDetails;
+private String param;
+private String regionName;
+p

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371620037
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
 ##
 @@ -215,6 +224,17 @@
*/
   private RSRpcServices rsRpcServices;
 
+
+  /**
+   * Use to add online slowlog responses
+   */
+  private OnlineSlowLogProvider onlineSlowLogProvider;
 
 Review comment:
   If we make, we add up in constructor arg which I don't think we should do, 
so better have setter method and let it be non-final please? :(


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371633649
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
 ##
 @@ -1251,6 +1258,9 @@ public RSRpcServices(final HRegionServer rs) throws 
IOException {
 ConnectionUtils.setServerSideHConnectionRetriesConfig(conf, name, LOG);
 rpcServer = createRpcServer(rs, rpcSchedulerFactory, bindAddress, name);
 rpcServer.setRsRpcServices(this);
+if (!(rs instanceof HMaster)) {
+  rpcServer.setOnlineSlowLogProvider(rs.getOnlineSlowLogProvider());
 
 Review comment:
   Well this is the main logic, RSRpcService is the one that should know this 
better regardless of whether we instantiate OnlineSlowLog in HMaster.
   The one in HRegionServer class just initializes OnlineSlowLog only in 
HRegionServer and not in HMaster. But even if it does, then this if condition 
in RSRpcServices will ensure we set the expected instance of OnlineSlowLog in 
rpcServer since it is rpcServer that is going to use it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371618023
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
 ##
 @@ -2213,6 +2224,57 @@ private static String getStringForByteString(ByteString 
bs) {
 return Bytes.toStringBinary(bs.toByteArray());
   }
 
+  /**
+   * Return SlowLogParams to maintain recent online slowlog responses
+   *
+   * @param message Message object {@link Message}
+   * @return SlowLogParams with regionName(for filter queries) and params
+   */
+  public static SlowLogParams getSlowLogParams(Message message) {
+if (message == null) {
+  return null;
+}
+if (message instanceof ScanRequest) {
+  ScanRequest scanRequest = (ScanRequest) message;
+  String regionName = 
getStringForByteString(scanRequest.getRegion().getValue());
+  String params = TextFormat.shortDebugString(message);
+  return new SlowLogParams(regionName, params);
+} else if (message instanceof MutationProto) {
+  MutationProto mutationProto = (MutationProto) message;
+  String params = "type= " + mutationProto.getMutateType().toString();
+  return new SlowLogParams(StringUtils.EMPTY, params);
+} else if (message instanceof GetRequest) {
+  GetRequest getRequest = (GetRequest) message;
+  String regionName = 
getStringForByteString(getRequest.getRegion().getValue());
+  String params = "region= " + regionName + ", row= "
++ getStringForByteString(getRequest.getGet().getRow());
+  return new SlowLogParams(regionName, params);
+} else if (message instanceof MultiRequest) {
+  MultiRequest multiRequest = (MultiRequest) message;
+  int actionsCount = multiRequest.getRegionActionList()
+.stream()
+.mapToInt(ClientProtos.RegionAction::getActionCount)
+.sum();
+  RegionAction actions = multiRequest.getRegionActionList().get(0);
+  String regionName = 
getStringForByteString(actions.getRegion().getValue());
+  String params = "region= " + regionName + ", for " + actionsCount + " 
action(s)";
+  return new SlowLogParams(regionName, params);
+} else if (message instanceof MutateRequest) {
+  MutateRequest mutateRequest = (MutateRequest) message;
+  String regionName = 
getStringForByteString(mutateRequest.getRegion().getValue());
+  String params = "region= " + regionName;
+  return new SlowLogParams(regionName, params);
+} else if (message instanceof CoprocessorServiceRequest) {
+  CoprocessorServiceRequest coprocessorServiceRequest = 
(CoprocessorServiceRequest) message;
+  String params = "coprocessorService= "
++ coprocessorServiceRequest.getCall().getServiceName()
++ ":" + coprocessorServiceRequest.getCall().getMethodName();
+  return new SlowLogParams(StringUtils.EMPTY, params);
 
 Review comment:
   Yes, but not all the times, so let me have another constructor


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371608218
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/SlowLogPayload.java
 ##
 @@ -0,0 +1,318 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.client;
+
+import org.apache.hadoop.hbase.util.GsonUtil;
+import org.apache.yetus.audience.InterfaceAudience;
+
+import org.apache.hbase.thirdparty.com.google.gson.Gson;
+import org.apache.hbase.thirdparty.com.google.gson.JsonObject;
+import org.apache.hbase.thirdparty.com.google.gson.JsonSerializer;
+
+/**
+ * SlowLog payload for client
+ */
+@InterfaceAudience.Private
+final public class SlowLogPayload {
+
+  private static final Gson GSON = GsonUtil.createGson()
+.setPrettyPrinting()
+.registerTypeAdapter(SlowLogPayload.class, (JsonSerializer)
+  (slowLogPayload, type, jsonSerializationContext) -> {
+Gson gson = new Gson();
+JsonObject jsonObj = (JsonObject) gson.toJsonTree(slowLogPayload);
+if (slowLogPayload.getMultiGetsCount() == 0) {
+  jsonObj.remove("multiGetsCount");
+}
+if (slowLogPayload.getMultiMutationsCount() == 0) {
+  jsonObj.remove("multiMutationsCount");
+}
+if (slowLogPayload.getMultiServiceCalls() == 0) {
+  jsonObj.remove("multiServiceCalls");
+}
+return jsonObj;
+  }).create();
+
+  private long startTime;
+  private int processingTime;
+  private int queueTime;
+  private long responseSize;
+  private String clientAddress;
+  private String serverClass;
+  private String methodName;
+  private String callDetails;
+  private String param;
+  private transient String regionName;
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371618142
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
 ##
 @@ -2205,6 +2216,57 @@ private static String getStringForByteString(ByteString 
bs) {
 return Bytes.toStringBinary(bs.toByteArray());
   }
 
+  /**
+   * Return SlowLogParams to maintain recent online slowlog responses
+   *
+   * @param message Message object {@link Message}
+   * @return SlowLogParams with regionName(for filter queries) and params
+   */
+  public static SlowLogParams getSlowLogParams(Message message) {
+if (message == null) {
+  return null;
+}
+if (message instanceof ScanRequest) {
+  ScanRequest scanRequest = (ScanRequest) message;
+  String regionName = 
getStringForByteString(scanRequest.getRegion().getValue());
+  String params = TextFormat.shortDebugString(message);
+  return new SlowLogParams(regionName, params);
+} else if (message instanceof MutationProto) {
+  MutationProto mutationProto = (MutationProto) message;
+  String params = "type= " + mutationProto.getMutateType().toString();
+  return new SlowLogParams(StringUtils.EMPTY, params);
+} else if (message instanceof GetRequest) {
+  GetRequest getRequest = (GetRequest) message;
+  String regionName = 
getStringForByteString(getRequest.getRegion().getValue());
+  String params = "region= " + regionName + ", row= "
++ getStringForByteString(getRequest.getGet().getRow());
+  return new SlowLogParams(regionName, params);
+} else if (message instanceof MultiRequest) {
+  MultiRequest multiRequest = (MultiRequest) message;
+  int actionsCount = multiRequest.getRegionActionList()
+.stream()
+.mapToInt(ClientProtos.RegionAction::getActionCount)
+.sum();
+  RegionAction actions = multiRequest.getRegionActionList().get(0);
+  String regionName = 
getStringForByteString(actions.getRegion().getValue());
+  String params = "region= " + regionName + ", for " + actionsCount + " 
action(s)";
+  return new SlowLogParams(regionName, params);
+} else if (message instanceof MutateRequest) {
+  MutateRequest mutateRequest = (MutateRequest) message;
+  String regionName = 
getStringForByteString(mutateRequest.getRegion().getValue());
+  String params = "region= " + regionName;
+  return new SlowLogParams(regionName, params);
+} else if (message instanceof CoprocessorServiceRequest) {
+  CoprocessorServiceRequest coprocessorServiceRequest = 
(CoprocessorServiceRequest) message;
+  String params = "coprocessorService= "
++ coprocessorServiceRequest.getCall().getServiceName()
++ ":" + coprocessorServiceRequest.getCall().getMethodName();
+  return new SlowLogParams(StringUtils.EMPTY, params);
+}
+String params = "TODO: " + message.getClass().toString();
 
 Review comment:
   Okk let me display just the className in that case


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on a change in pull request #754: HBASE-22978 : Online 
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371631514
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 ##
 @@ -590,6 +596,10 @@ public HRegionServer(final Configuration conf) throws 
IOException {
   this.abortRequested = false;
   this.stopped = false;
 
+  // initiate online slowlog ringbuffer only for RegionServers
+  if (!(this instanceof HMaster)) {
 
 Review comment:
   Honestly I don't like it either but do we really have a better way? Like do 
we have some config to check this or some param may be?
   e.g. I see lot of these codes:
   ```
   if (server instanceof HRegionServer) {
 rsServerHost = ((HRegionServer) 
server).getRegionServerCoprocessorHost();
   }
   ```
   ```
   if (server instanceof HRegionServer) {
 NettyEventLoopGroupConfig config = ((HRegionServer) 
server).getEventLoopGroupConfig();
 eventLoopGroup = config.group();
 channelClass = config.serverChannelClass();
   } else {
 eventLoopGroup = new NioEventLoopGroup(0,
 new DefaultThreadFactory("NettyRpcServer", true, 
Thread.MAX_PRIORITY));
 channelClass = NioServerSocketChannel.class;
   }
   ```
   ```
 boolean isMasterNotCarryTable =
 this instanceof HMaster && !LoadBalancer.isTablesOnMaster(conf);
   ```
   
   and so on. My point is we are trying to intialize something for RS and not 
for HM and may be I am not aware of better decision making conditions that we 
might already have. Could you please point me to one? Would be really helpful. 
Here, a `protected void postConstruction` method not help right? Since we have 
to initialize something in parent class and not in child class. We don't want 
to make super() from child class initialize something without knowing the 
instance and then let child re-initialize(override) it with null or something 
similar?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024908#comment-17024908
 ] 

Hudson commented on HBASE-18095:


Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #53 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/53/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/53//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/53//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/53//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for 
details|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/53//artifact/output-integration/hadoop-2.log].
 (note that this means we didn't run on Hadoop 3)


> Provide an option for clients to find the server hosting META that does not 
> involve the ZooKeeper client
> 
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
>  Issue Type: New Feature
>  Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the 
> regionserver hosting the meta table region. Site configuration provides the 
> client a list of ZK quorum peers and the client uses an embedded ZK client to 
> query meta location. Timeouts and retry behavior of this embedded ZK client 
> are managed orthogonally to HBase layer settings and in some cases the ZK 
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage 
> or network partition.
> We should consider new configuration settings that provide a list of 
> well-known master and backup master locations, and with this information the 
> client can contact any of the master processes directly. Any master in either 
> active or passive state will track meta location and respond to requests for 
> it with its cached last known location. If this location is stale, the client 
> can ask again with a flag set that requests the master refresh its location 
> cache and return the up-to-date location. Every client interaction with the 
> cluster thus uses only HBase RPC as transport, with appropriate settings 
> applied to the connection. The configuration toggle that enables this 
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and 
> contact the ZK service directly at the beginning of the connection lifecycle. 
> This has several benefits. ZK service need not be exposed to clients, and 
> their potential abuse, yet no benefit ZK provides the HBase server cluster is 
> compromised. Normalizing HBase client and ZK client timeout settings and 
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer 
> necessary. 
> And, from [~ghelmling]: There is an additional complication here for 
> token-based authentication. When a delegation token is used for SASL 
> authentication, the client uses the cluster ID obtained from Zookeeper to 
> select the token identifier to use. So there would also need to be some 
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] Apache-HBase commented on issue #1101: HBASE-23305: Master based registry implementation (#954)

[GitBox]

Apache-HBase commented on issue #1101: HBASE-23305: Master based registry 
implementation (#954)
URL: https://github.com/apache/hbase/pull/1101#issuecomment-579099014
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 12s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  No case conflicting files 
found.  |
   | +0 :ok: |  prototool  |   0m  0s |  prototool was not available.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
50 new or modified test files.  |
   ||| _ branch-2 Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  Maven dependency ordering for branch  |
   | -1 :x: |  mvninstall  |   0m 41s |  root in branch-2 failed.  |
   | -1 :x: |  compile  |   0m 10s |  hbase-client in branch-2 failed.  |
   | -1 :x: |  compile  |   0m  8s |  hbase-common in branch-2 failed.  |
   | -1 :x: |  compile  |   0m  7s |  hbase-protocol-shaded in branch-2 failed. 
 |
   | -1 :x: |  compile  |   0m 16s |  hbase-server in branch-2 failed.  |
   | -1 :x: |  compile  |   0m  7s |  hbase-zookeeper in branch-2 failed.  |
   | -1 :x: |  checkstyle  |   0m  5s |  The patch fails to run checkstyle in 
hbase-client  |
   | -1 :x: |  checkstyle  |   0m  6s |  The patch fails to run checkstyle in 
hbase-common  |
   | -1 :x: |  checkstyle  |   0m  5s |  The patch fails to run checkstyle in 
hbase-protocol-shaded  |
   | -1 :x: |  checkstyle  |   0m  6s |  The patch fails to run checkstyle in 
hbase-server  |
   | -1 :x: |  checkstyle  |   0m  6s |  The patch fails to run checkstyle in 
hbase-zookeeper  |
   | -1 :x: |  shadedjars  |   0m 28s |  branch has 11 errors when building our 
shaded downstream artifacts.  |
   | -1 :x: |  javadoc  |   0m  8s |  hbase-client in branch-2 failed.  |
   | -1 :x: |  javadoc  |   0m  7s |  hbase-common in branch-2 failed.  |
   | -1 :x: |  javadoc  |   0m  8s |  hbase-protocol-shaded in branch-2 failed. 
 |
   | -1 :x: |  javadoc  |   0m  8s |  hbase-server in branch-2 failed.  |
   | -1 :x: |  javadoc  |   0m  8s |  hbase-zookeeper in branch-2 failed.  |
   | +0 :ok: |  spotbugs  |   1m 46s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | -1 :x: |  findbugs  |   0m  7s |  hbase-client in branch-2 failed.  |
   | -1 :x: |  findbugs  |   0m  8s |  hbase-common in branch-2 failed.  |
   | -1 :x: |  findbugs  |   0m  7s |  hbase-protocol-shaded in branch-2 
failed.  |
   | -1 :x: |  findbugs  |   0m  8s |  hbase-server in branch-2 failed.  |
   | -1 :x: |  findbugs  |   0m  8s |  hbase-zookeeper in branch-2 failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m  7s |  Maven dependency ordering for patch  |
   | -1 :x: |  mvninstall  |   0m 28s |  root in the patch failed.  |
   | -1 :x: |  compile  |   0m  8s |  hbase-client in the patch failed.  |
   | -1 :x: |  compile  |   0m  7s |  hbase-common in the patch failed.  |
   | -1 :x: |  compile  |   0m  8s |  hbase-protocol-shaded in the patch 
failed.  |
   | -1 :x: |  compile  |   0m  8s |  hbase-server in the patch failed.  |
   | -1 :x: |  compile  |   0m  7s |  hbase-zookeeper in the patch failed.  |
   | -1 :x: |  cc  |   0m  8s |  hbase-client in the patch failed.  |
   | -1 :x: |  cc  |   0m  7s |  hbase-common in the patch failed.  |
   | -1 :x: |  cc  |   0m  8s |  hbase-protocol-shaded in the patch failed.  |
   | -1 :x: |  cc  |   0m  8s |  hbase-server in the patch failed.  |
   | -1 :x: |  cc  |   0m  7s |  hbase-zookeeper in the patch failed.  |
   | -1 :x: |  javac  |   0m  8s |  hbase-client in the patch failed.  |
   | -1 :x: |  javac  |   0m  7s |  hbase-common in the patch failed.  |
   | -1 :x: |  javac  |   0m  8s |  hbase-protocol-shaded in the patch failed.  
|
   | -1 :x: |  javac  |   0m  8s |  hbase-server in the patch failed.  |
   | -1 :x: |  javac  |   0m  7s |  hbase-zookeeper in the patch failed.  |
   | -1 :x: |  checkstyle  |   0m  6s |  The patch fails to run checkstyle in 
hbase-client  |
   | -1 :x: |  checkstyle  |   0m  5s |  The patch fails to run checkstyle in 
hbase-common  |
   | -1 :x: |  checkstyle  |   0m  6s |  The patch fails to run checkstyle in 
hbase-protocol-shaded  |
   | -1 :x: |  checkstyle  |   0m  6s |  The patch fails to run checkstyle in 
hbase-server  |
   | -1 :x: |  checkstyle  |   0m  6s |  The patch fails to run checkstyle in 
hbase-zookeeper  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | -1 :x: |  shadedjars  |   0m 27s |  patch has 11 errors when building our 
shaded downstream artifacts.  |
   | -1 :x: |  hadoopcheck  |   0m 11s |  The patch causes 10 errors with 
Hadoop v2.8

[jira] [Resolved] (HBASE-23716) [hbase-thirdparty] Make release 3.2.0

[Michael Stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack resolved HBASE-23716.
---
  Assignee: Michael Stack
Resolution: Fixed

Pushed it out in maven repo and added it to our release dir. Resolving.

> [hbase-thirdparty] Make release 3.2.0
> -
>
> Key: HBASE-23716
> URL: https://issues.apache.org/jira/browse/HBASE-23716
> Project: HBase
>  Issue Type: Bug
>  Components: thirdparty
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: hbase-thirdparty-3.2.0
>
>
> Update the dependencies in hbase-thirdparty. In particular, pb goes from 3.7 
> to 3.11 with a few perf improvements.
> Would like to update the hbase-thirdparty to include in the coming 
> hbase-2.3.0.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23750) [hbase-thirdparty] Add nexus-staging-maven-plugin to apache-release profile

[Michael Stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack updated HBASE-23750:
--
Fix Version/s: 3.3.0
   3.2.1

> [hbase-thirdparty] Add nexus-staging-maven-plugin to apache-release profile
> ---
>
> Key: HBASE-23750
> URL: https://issues.apache.org/jira/browse/HBASE-23750
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Michael Stack
>Priority: Major
> Fix For: 3.2.1, 3.3.0
>
>
> Parent issue added use of nexus-staging-maven-plugin manipulating 
> maven.apache.org repos. Add plugin reference over in hbase-thirdparty so when 
> we make RCs using create-release from hbase core, the repo manipulation 
> 'works' (failed when I made the 2.3.0RC0.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23750) [hbase-thirdparty] Add nexus-staging-maven-plugin to apache-release profile

[Michael Stack (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024897#comment-17024897
 ] 

Michael Stack commented on HBASE-23750:
---

Test attached patch next time we make an RC to make sure it works. Then commit.

> [hbase-thirdparty] Add nexus-staging-maven-plugin to apache-release profile
> ---
>
> Key: HBASE-23750
> URL: https://issues.apache.org/jira/browse/HBASE-23750
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Michael Stack
>Priority: Major
>
> Parent issue added use of nexus-staging-maven-plugin manipulating 
> maven.apache.org repos. Add plugin reference over in hbase-thirdparty so when 
> we make RCs using create-release from hbase core, the repo manipulation 
> 'works' (failed when I made the 2.3.0RC0.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase-thirdparty] saintstack opened a new pull request #11: HBASE-23750 [hbase-thirdparty] Add nexus-staging-maven-plugin to apac…

[GitBox]

saintstack opened a new pull request #11: HBASE-23750 [hbase-thirdparty] Add 
nexus-staging-maven-plugin to apac…
URL: https://github.com/apache/hbase-thirdparty/pull/11
 
 
   …he-release profile


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (HBASE-23751) Move core to hbase-thirdparty 3.2.0

[Michael Stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack resolved HBASE-23751.
---
  Assignee: Michael Stack
Resolution: Fixed

Pushed the one-liner patch (attached) to master and branch-2. Resolving.

> Move core to hbase-thirdparty 3.2.0
> ---
>
> Key: HBASE-23751
> URL: https://issues.apache.org/jira/browse/HBASE-23751
> Project: HBase
>  Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 
> 0001-HBASE-23751-Move-core-to-hbase-thirdparty-3.2.0.patch
>
>
> Move core to use just-released 3.2.0 hbase-thirdparty.  The change has 
> already been through build courtesy of 
> https://github.com/apache/hbase/pull/1086



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23751) Move core to hbase-thirdparty 3.2.0

[Michael Stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack updated HBASE-23751:
--
Attachment: 0001-HBASE-23751-Move-core-to-hbase-thirdparty-3.2.0.patch

> Move core to hbase-thirdparty 3.2.0
> ---
>
> Key: HBASE-23751
> URL: https://issues.apache.org/jira/browse/HBASE-23751
> Project: HBase
>  Issue Type: Bug
>Reporter: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 
> 0001-HBASE-23751-Move-core-to-hbase-thirdparty-3.2.0.patch
>
>
> Move core to use just-released 3.2.0 hbase-thirdparty.  The change has 
> already been through build courtesy of 
> https://github.com/apache/hbase/pull/1086



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] bharathv opened a new pull request #1101: HBASE-23305: Master based registry implementation (#954)

[GitBox]

bharathv opened a new pull request #1101: HBASE-23305: Master based registry 
implementation (#954)
URL: https://github.com/apache/hbase/pull/1101
 
 
  HBASE-23305: Master based registry implementation (#954)
   
   Implements a master based registry for clients.
   
- Supports hedged RPCs (fan out configured via configs).
- Parameterized existing client tests to run with multiple registry 
combinations.
- Added unit-test coverage for the new registry implementation.
   
   Signed-off-by: Nick Dimiduk 
   Signed-off-by: stack 
   Signed-off-by: Andrew Purtell 
   (cherry picked from commit 62da419b23f4409837c57b61c42db18b90d9d259)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (HBASE-23751) Move core to hbase-thirdparty 3.2.0

[Michael Stack (Jira)]

Michael Stack created HBASE-23751:
-

 Summary: Move core to hbase-thirdparty 3.2.0
 Key: HBASE-23751
 URL: https://issues.apache.org/jira/browse/HBASE-23751
 Project: HBase
  Issue Type: Bug
Reporter: Michael Stack
 Fix For: 3.0.0, 2.3.0


Move core to use just-released 3.2.0 hbase-thirdparty.  The change has already 
been through build courtesy of https://github.com/apache/hbase/pull/1086



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23750) [hbase-thirdparty] Add nexus-staging-maven-plugin to apache-release profile

[Michael Stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack updated HBASE-23750:
--
Summary: [hbase-thirdparty] Add nexus-staging-maven-plugin to 
apache-release profile  (was: [hbase-operator-tools] Add 
nexus-staging-maven-plugin to apache-release profile)

> [hbase-thirdparty] Add nexus-staging-maven-plugin to apache-release profile
> ---
>
> Key: HBASE-23750
> URL: https://issues.apache.org/jira/browse/HBASE-23750
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Michael Stack
>Priority: Major
>
> Parent issue added use of nexus-staging-maven-plugin manipulating 
> maven.apache.org repos. Add plugin reference over in hbase-operator-tools so 
> when we make RCs using create-release from hbase core, the repo manipulation 
> 'works' (failed when I made the 2.3.0RC0.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23750) [hbase-thirdparty] Add nexus-staging-maven-plugin to apache-release profile

[Michael Stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack updated HBASE-23750:
--
Description: Parent issue added use of nexus-staging-maven-plugin 
manipulating maven.apache.org repos. Add plugin reference over in 
hbase-thirdparty so when we make RCs using create-release from hbase core, the 
repo manipulation 'works' (failed when I made the 2.3.0RC0.  (was: Parent issue 
added use of nexus-staging-maven-plugin manipulating maven.apache.org repos. 
Add plugin reference over in hbase-operator-tools so when we make RCs using 
create-release from hbase core, the repo manipulation 'works' (failed when I 
made the 2.3.0RC0.)

> [hbase-thirdparty] Add nexus-staging-maven-plugin to apache-release profile
> ---
>
> Key: HBASE-23750
> URL: https://issues.apache.org/jira/browse/HBASE-23750
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Michael Stack
>Priority: Major
>
> Parent issue added use of nexus-staging-maven-plugin manipulating 
> maven.apache.org repos. Add plugin reference over in hbase-thirdparty so when 
> we make RCs using create-release from hbase core, the repo manipulation 
> 'works' (failed when I made the 2.3.0RC0.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23750) [hbase-operator-tools] Add nexus-staging-maven-plugin to apache-release profile

[Michael Stack (Jira)]

Michael Stack created HBASE-23750:
-

 Summary: [hbase-operator-tools] Add nexus-staging-maven-plugin to 
apache-release profile
 Key: HBASE-23750
 URL: https://issues.apache.org/jira/browse/HBASE-23750
 Project: HBase
  Issue Type: Sub-task
Reporter: Michael Stack


Parent issue added use of nexus-staging-maven-plugin manipulating 
maven.apache.org repos. Add plugin reference over in hbase-operator-tools so 
when we make RCs using create-release from hbase core, the repo manipulation 
'works' (failed when I made the 2.3.0RC0.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] bharathv opened a new pull request #1100: HBASE-23648: Re-use underlying connection registry in RawAsyncHBaseAd… …min (#994)

[GitBox]

bharathv opened a new pull request #1100: HBASE-23648: Re-use underlying 
connection registry in RawAsyncHBaseAd…  …min (#994)
URL: https://github.com/apache/hbase/pull/1100
 
 
   * HBASE-23648: Re-use underlying connection registry in RawAsyncHBaseAdmin
   
   No need to create and close a new registry on demand. Audited other
   usages of getRegistry() and the code looks fine.
   
   * Fix checkstyle issues in RawAsyncHBaseAdmin


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] bharathv opened a new pull request #1099: HBASE-23604: Clarify AsyncRegistry usage in the code. (#957)

[GitBox]

bharathv opened a new pull request #1099: HBASE-23604: Clarify AsyncRegistry 
usage in the code. (#957)
URL: https://github.com/apache/hbase/pull/1099
 
 
   * HBASE-23604: Cleanup AsyncRegistry interface
   
   - Cleans up the method names to make more sense and adds a little
   more javadocs for context. In future patches we can revisit
   the name of the actual class to make it more self explanatory.
   
   - Does AsyncRegistry -> ConnectionRegistry rename.
   "async" ness of the registry is kind of implicit based on
   the interface contents and need not be reflected in the name.
   
   Signed-off-by: Nick Dimiduk 
   Signed-off-by: stack 
   Signed-off-by: Viraj Jasani 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23349) Low refCount preventing archival of compacted away files

[Andrew Kyle Purtell (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024863#comment-17024863
 ] 

Andrew Kyle Purtell commented on HBASE-23349:
-

Some regions are hot, with readers mostly always active, and also taking 
writes, enough to generate flush and compaction activity.

> Low refCount preventing archival of compacted away files
> 
>
> Key: HBASE-23349
> URL: https://issues.apache.org/jira/browse/HBASE-23349
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0, 2.3.0, 1.6.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
>
> We have observed that refCount on compacted away store files as low as 1 is 
> prevent archival.
> {code:java}
> regionserver.HStore - Can't archive compacted file 
> hdfs://{{root-dir}}/hbase/data/default/t1/12a9e1112e0371955b3db8d3ebb2d298/cf1/73b72f5ddfce4a34a9e01afe7b83c1f9
>  because of either isCompactedAway=true or file has reference, 
> isReferencedInReads=true, refCount=1, skipping for now.
> {code}
> We should come up with core code (run as part of discharger thread) 
> gracefully resolve reader lock issue by resetting ongoing scanners to start 
> pointing to new store files instead of compacted away store files.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23281) Track meta region changes on masters

[HBase QA (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024856#comment-17024856
 ] 

HBase QA commented on HBASE-23281:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 18 new or modified test 
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
58s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
56s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
28s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
53s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
20s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  0m 
44s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  6m 
30s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
42s{color} | {color:green} The patch passed checkstyle in hbase-client {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} The patch passed checkstyle in hbase-zookeeper 
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
25s{color} | {color:green} hbase-server: The patch generated 0 new + 114 
unchanged - 6 fixed = 114 total (was 120) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
36s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
20m 30s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  6m 
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  4m  
4s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
54s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}167m 
33s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  1m 
37s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{c

[GitHub] [hbase] Apache-HBase commented on issue #1097: HBASE-23281: Track meta region locations in masters (#830)

[GitBox]

Apache-HBase commented on issue #1097: HBASE-23281: Track meta region locations 
in masters (#830)
URL: https://github.com/apache/hbase/pull/1097#issuecomment-579072989
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 18s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
18 new or modified test files.  |
   ||| _ branch-2 Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 18s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   6m 58s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   1m 56s |  branch-2 passed  |
   | +1 :green_heart: |  checkstyle  |   2m 28s |  branch-2 passed  |
   | +1 :green_heart: |  shadedjars  |   4m 53s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m 20s |  branch-2 passed  |
   | +0 :ok: |  spotbugs  |   0m 44s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   6m 30s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 16s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   6m 32s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 57s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 57s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 42s |  The patch passed checkstyle 
in hbase-client  |
   | +1 :green_heart: |  checkstyle  |   0m 14s |  The patch passed checkstyle 
in hbase-zookeeper  |
   | +1 :green_heart: |  checkstyle  |   1m 25s |  hbase-server: The patch 
generated 0 new + 114 unchanged - 6 fixed = 114 total (was 120)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   5m 36s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  20m 30s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   1m 28s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   6m 26s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   4m  4s |  hbase-client in the patch passed.  
|
   | +1 :green_heart: |  unit  |   0m 54s |  hbase-zookeeper in the patch 
passed.  |
   | +1 :green_heart: |  unit  | 167m 33s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   1m 37s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 253m 46s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1097/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1097 |
   | JIRA Issue | HBASE-23281 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 893a96afaca9 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 
17:06:04 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1097/out/precommit/personality/provided.sh
 |
   | git revision | branch-2 / e0f913323e |
   | Default Java | 1.8.0_181 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1097/1/testReport/
 |
   | Max. process+thread count | 4812 (vs. ulimit of 1) |
   | modules | C: hbase-client hbase-zookeeper hbase-server U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1097/1/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23349) Low refCount preventing archival of compacted away files

[ramkrishna.s.vasudevan (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024855#comment-17024855
 ] 

ramkrishna.s.vasudevan commented on HBASE-23349:


bq.Now the issue is that if there are readers active on a region always it will 
never be allowed to discharge compacted files
Ya if readers are active discharger will never come into play. Is there any 
specific reason why readers are always active? 
Are you suggesting we go back to the locking way only? If not then atleast i 
prefer the approach here where CPs can make the store scanner reset itself on 
compaction. 

> Low refCount preventing archival of compacted away files
> 
>
> Key: HBASE-23349
> URL: https://issues.apache.org/jira/browse/HBASE-23349
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0, 2.3.0, 1.6.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
>
> We have observed that refCount on compacted away store files as low as 1 is 
> prevent archival.
> {code:java}
> regionserver.HStore - Can't archive compacted file 
> hdfs://{{root-dir}}/hbase/data/default/t1/12a9e1112e0371955b3db8d3ebb2d298/cf1/73b72f5ddfce4a34a9e01afe7b83c1f9
>  because of either isCompactedAway=true or file has reference, 
> isReferencedInReads=true, refCount=1, skipping for now.
> {code}
> We should come up with core code (run as part of discharger thread) 
> gracefully resolve reader lock issue by resetting ongoing scanners to start 
> pointing to new store files instead of compacted away store files.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23304) Implement RPCs needed for master based registry

[HBase QA (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024852#comment-17024852
 ] 

HBase QA commented on HBASE-23304:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} prototool {color} | {color:blue}  0m  
0s{color} | {color:blue} prototool was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 19 new or modified test 
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
22s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
25s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
15s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
16s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
34s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  0m 
34s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  7m 
51s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green}  2m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
12s{color} | {color:green} The patch passed checkstyle in hbase-protocol-shaded 
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
37s{color} | {color:green} hbase-client: The patch generated 0 new + 194 
unchanged - 1 fixed = 194 total (was 195) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
13s{color} | {color:green} The patch passed checkstyle in hbase-zookeeper 
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
12s{color} | {color:green} hbase-server: The patch generated 0 new + 119 
unchanged - 13 fixed = 119 total (was 132) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
19s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
15m 39s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}  
2m 17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  8m 
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {co

[GitHub] [hbase] Apache-HBase commented on issue #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904)

[GitBox]

Apache-HBase commented on issue #1098: HBASE-23304: RPCs needed for client meta 
information lookup (apache#904)
URL: https://github.com/apache/hbase/pull/1098#issuecomment-579071234
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 35s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  No case conflicting files 
found.  |
   | +0 :ok: |  prototool  |   0m  0s |  prototool was not available.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
19 new or modified test files.  |
   ||| _ branch-2 Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 16s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 22s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   2m 25s |  branch-2 passed  |
   | +1 :green_heart: |  checkstyle  |   2m 15s |  branch-2 passed  |
   | +1 :green_heart: |  shadedjars  |   4m 16s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m 34s |  branch-2 passed  |
   | +0 :ok: |  spotbugs  |   0m 34s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   7m 51s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 15s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   4m 55s |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 24s |  the patch passed  |
   | +1 :green_heart: |  cc  |   2m 24s |  the patch passed  |
   | +1 :green_heart: |  javac  |   2m 24s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 12s |  The patch passed checkstyle 
in hbase-protocol-shaded  |
   | +1 :green_heart: |  checkstyle  |   0m 37s |  hbase-client: The patch 
generated 0 new + 194 unchanged - 1 fixed = 194 total (was 195)  |
   | +1 :green_heart: |  checkstyle  |   0m 13s |  The patch passed checkstyle 
in hbase-zookeeper  |
   | +1 :green_heart: |  checkstyle  |   1m 12s |  hbase-server: The patch 
generated 0 new + 119 unchanged - 13 fixed = 119 total (was 132)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   4m 19s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  15m 39s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  hbaseprotoc  |   2m 17s |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 33s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   8m 20s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   0m 42s |  hbase-protocol-shaded in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   3m 45s |  hbase-client in the patch passed.  
|
   | +1 :green_heart: |  unit  |   0m 53s |  hbase-zookeeper in the patch 
passed.  |
   | +1 :green_heart: |  unit  | 153m 34s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   2m 12s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 235m  0s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1098/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1098 |
   | JIRA Issue | HBASE-23304 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool |
   | uname | Linux 1aaa31531bb2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1098/out/precommit/personality/provided.sh
 |
   | git revision | branch-2 / e0f913323e |
   | Default Java | 1.8.0_181 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1098/1/testReport/
 |
   | Max. process+thread count | 6066 (vs. ulimit of 1) |
   | modules | C: hbase-protocol-shaded hbase-client hbase-zookeeper 
hbase-server U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1098/1/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To

[jira] [Commented] (HBASE-23749) TestHFileWriterV3 should have tests for all data block encodings

[Viraj Jasani (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024848#comment-17024848
 ] 

Viraj Jasani commented on HBASE-23749:
--

FYI [~ram_krish]

> TestHFileWriterV3 should have tests for all data block encodings
> 
>
> Key: HBASE-23749
> URL: https://issues.apache.org/jira/browse/HBASE-23749
> Project: HBase
>  Issue Type: Test
>Affects Versions: 3.0.0, 2.3.0, 1.6.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>
> TestHFileWriterV3 tests include writing and reading HFiles for default data 
> blocks encoding. However, since this test has efficient way of testing by 
> looping through encoded/un-encoded keys and values and keeping offsets 
> updated etc, it would be really good to have such test for each encoded data 
> blocks. For that, we should explore using respective seekers e.g. 
> RowIndexV1Seeker for ROW_INDEX_V1 encoded data blocks etc.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23749) TestHFileWriterV3 should have tests for all data block encodings

[Viraj Jasani (Jira)]

Viraj Jasani created HBASE-23749:


 Summary: TestHFileWriterV3 should have tests for all data block 
encodings
 Key: HBASE-23749
 URL: https://issues.apache.org/jira/browse/HBASE-23749
 Project: HBase
  Issue Type: Test
Affects Versions: 3.0.0, 2.3.0, 1.6.0
Reporter: Viraj Jasani
Assignee: Viraj Jasani


TestHFileWriterV3 tests include writing and reading HFiles for default data 
blocks encoding. However, since this test has efficient way of testing by 
looping through encoded/un-encoded keys and values and keeping offsets updated 
etc, it would be really good to have such test for each encoded data blocks. 
For that, we should explore using respective seekers e.g. RowIndexV1Seeker for 
ROW_INDEX_V1 encoded data blocks etc.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23275) Track active master server name in ActiveMasterManager

[HBase QA (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024845#comment-17024845
 ] 

HBase QA commented on HBASE-23275:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
56s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
57s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
17s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
38s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
41s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  3m 
30s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
28s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
14s{color} | {color:green} hbase-server: The patch generated 0 new + 106 
unchanged - 6 fixed = 106 total (was 112) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
41s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
17m  4s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}165m  0s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
27s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}224m  8s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAsyncTableGetMultiThreaded |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1096/1/artifact/out/Dockerfile
 |
| GITHUB PR | https://github.com/apache/hbase/pull/1096 |
| JIRA Issue | HBASE-23275 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 3474081bae23 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1096/out/precommit/personality/provided.sh
 |
| git revision | branch-2 / e0f913323e |
| Default Java | 1.8.0_181 |

[GitHub] [hbase] Apache-HBase commented on issue #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812)

[GitBox]

Apache-HBase commented on issue #1096: HBASE-23275: Track active master's 
address in ActiveMasterManager (#812)
URL: https://github.com/apache/hbase/pull/1096#issuecomment-579065532
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 11s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
2 new or modified test files.  |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 56s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   0m 57s |  branch-2 passed  |
   | +1 :green_heart: |  checkstyle  |   1m 17s |  branch-2 passed  |
   | +1 :green_heart: |  shadedjars  |   4m 38s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 41s |  branch-2 passed  |
   | +0 :ok: |  spotbugs  |   3m 30s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   3m 28s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 26s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 59s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 59s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 14s |  hbase-server: The patch 
generated 0 new + 106 unchanged - 6 fixed = 106 total (was 112)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   4m 41s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  17m  4s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   0m 36s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   3m 40s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 165m  0s |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  asflicense  |   0m 27s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 224m  8s |   |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.hbase.client.TestAsyncTableGetMultiThreaded |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1096/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1096 |
   | JIRA Issue | HBASE-23275 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 3474081bae23 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1096/out/precommit/personality/provided.sh
 |
   | git revision | branch-2 / e0f913323e |
   | Default Java | 1.8.0_181 |
   | unit | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1096/1/artifact/out/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1096/1/testReport/
 |
   | Max. process+thread count | 4823 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1096/1/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23710) Priority configuration for system coprocessors

[Anoop Sam John (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024844#comment-17024844
 ] 

Anoop Sam John commented on HBASE-23710:


Pls add release notes regarding usage. Tks

> Priority configuration for system coprocessors
> --
>
> Key: HBASE-23710
> URL: https://issues.apache.org/jira/browse/HBASE-23710
> Project: HBase
>  Issue Type: New Feature
>  Components: Coprocessors
>Reporter: Geoffrey Jacoby
>Assignee: Geoffrey Jacoby
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0, 2.2.3, 2.1.9
>
> Attachments: HBASE-23710-master.v01.patch, 
> HBASE-23710.branch-1.000.patch, HBASE-23710.v1.branch-1.patch
>
>
> Currenty HBase allows operators to set system region coprocessors via 
> hbase-site.xml to be loaded on each table in a cluster (or alternately, all 
> tables but system tables). HBase assumes that the first loaded system 
> coprocessor gets the first, or SYSTEM priority, with each subsequent system 
> coproc getting incremented by 1. As a reminder, in HBase _lower_ priorities 
> go first. 
> It can be useful for an operator to be able to define a coprocessor on each 
> table that needs a different priority. For example, an operator might want a 
> coproc to load on each table _last_, so that it can enforce some system 
> invariant and know that no other coproc will interfere with it. 
> I propose adding optional priority config to the hbase-site.xml 
> configuration, separated from each coproc class in the comma-separated list 
> by a special character (perhaps a pipe, such as table coprocs use) that's not 
> used in class names. 
> The region coprocessor host will parse the priority if present and use it 
> when instantiating the coproc. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-22775) Enhance logging for peer related operations

[HBase QA (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024841#comment-17024841
 ] 

HBase QA commented on HBASE-22775:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}  
0m  0s{color} | {color:orange} The patch doesn't appear to include any new or 
modified tests. Please justify why no new tests are needed for this patch. Also 
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
 5s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  0m 
30s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
 7s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
17m  5s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
24s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 9s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 53s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.4 Server=19.03.4 base: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1108/artifact/patchprocess/Dockerfile
 |
| JIRA Issue | HBASE-22775 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12977410/HBASE-22775.master.002.patch
 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 29b9e2feb322 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / fb06ca5ba4 |
| Default Java | 1.8.0_181 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-

[jira] [Commented] (HBASE-23257) Track ClusterID in stand by masters

[HBase QA (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024840#comment-17024840
 ] 

HBase QA commented on HBASE-23257:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
37s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
32s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
54s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
11s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
18s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
39s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  2m 
53s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
52s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
15s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
15m 38s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}151m 
41s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
37s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}205m 24s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1095/1/artifact/out/Dockerfile
 |
| GITHUB PR | https://github.com/apache/hbase/pull/1095 |
| JIRA Issue | HBASE-23257 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 215f7abb36e8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1095/out/precommit/personality/provided.sh
 |
| git revision | branch-2 / e0f913323e |
| Default Java | 1.8.0_181 |
|  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1095/1/testReport/
 |
| Max. process+thread count | 6126 (vs. ulimit of 1) |
| mo

[GitHub] [hbase] Apache-HBase commented on issue #1095: HBASE-23257: Track clusterID in stand by masters (#798)

[GitBox]

Apache-HBase commented on issue #1095: HBASE-23257: Track clusterID in stand by 
masters (#798)
URL: https://github.com/apache/hbase/pull/1095#issuecomment-579060018
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 37s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
1 new or modified test files.  |
   ||| _ branch-2 Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 32s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   0m 54s |  branch-2 passed  |
   | +1 :green_heart: |  checkstyle  |   1m 11s |  branch-2 passed  |
   | +1 :green_heart: |  shadedjars  |   4m 18s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  branch-2 passed  |
   | +0 :ok: |  spotbugs  |   2m 53s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   2m 52s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   4m 51s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 55s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 55s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 10s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   4m 15s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  15m 38s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   0m 35s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   3m 18s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 151m 41s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   0m 37s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 205m 24s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1095/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1095 |
   | JIRA Issue | HBASE-23257 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 215f7abb36e8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1095/out/precommit/personality/provided.sh
 |
   | git revision | branch-2 / e0f913323e |
   | Default Java | 1.8.0_181 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1095/1/testReport/
 |
   | Max. process+thread count | 6126 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1095/1/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] virajjasani commented on issue #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on issue #754: HBASE-22978 : Online slow response log
URL: https://github.com/apache/hbase/pull/754#issuecomment-579052854
 
 
   > I sent a note to dev@ giving a status update. If you don’t find there what 
you need, or have some comments, please reply on that thread.
   > […](#)
   > On Mon, Jan 27, 2020 at 16:42 Viraj Jasani ***@***.***> wrote: Thanks for 
the review @ndimiduk  I will try to be on top of 
this. Could you please let me know if you have any estimate of 2.3.0 release? 
May be by end of next week? — You are receiving this because you were 
mentioned. Reply to this email directly, view it on GitHub 
<#754?email_source=notifications&email_token=AAALDLBPT5FPVY6OIEYVSRLQ755PDA5CNFSM4JEQSSC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKBTWZQ#issuecomment-579025766>,
 or unsubscribe 

 .
   
   Sure @ndimiduk , I got your dev@ message immediately after asking the 
question here :)
   Thanks


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (HBASE-23748) Include HBASE-21284 to branch-2.2

[Sakthi (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sakthi updated HBASE-23748:
---
Fix Version/s: 2.2.3
   Status: Patch Available  (was: In Progress)

> Include HBASE-21284 to branch-2.2
> -
>
> Key: HBASE-23748
> URL: https://issues.apache.org/jira/browse/HBASE-23748
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 2.2.3
>
> Attachments: hbase-23748.branch-2.2.001.patch
>
>
> HBASE-21284 was ought to be present in 2.2. But by the time the commit was 
> done, the branch had been cut already. Hence this Jira to track it's 
> inclusion.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23748) Include HBASE-21284 to branch-2.2

[Sakthi (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sakthi updated HBASE-23748:
---
Attachment: hbase-23748.branch-2.2.001.patch

> Include HBASE-21284 to branch-2.2
> -
>
> Key: HBASE-23748
> URL: https://issues.apache.org/jira/browse/HBASE-23748
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-23748.branch-2.2.001.patch
>
>
> HBASE-21284 was ought to be present in 2.2. But by the time the commit was 
> done, the branch had been cut already. Hence this Jira to track it's 
> inclusion.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work started] (HBASE-23748) Include HBASE-21284 to branch-2.2

[Sakthi (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-23748 started by Sakthi.
--
> Include HBASE-21284 to branch-2.2
> -
>
> Key: HBASE-23748
> URL: https://issues.apache.org/jira/browse/HBASE-23748
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-23748.branch-2.2.001.patch
>
>
> HBASE-21284 was ought to be present in 2.2. But by the time the commit was 
> done, the branch had been cut already. Hence this Jira to track it's 
> inclusion.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23748) Include HBASE-21284 to branch-2.2

[Sakthi (Jira)]

Sakthi created HBASE-23748:
--

 Summary: Include HBASE-21284 to branch-2.2
 Key: HBASE-23748
 URL: https://issues.apache.org/jira/browse/HBASE-23748
 Project: HBase
  Issue Type: Sub-task
Reporter: Sakthi
Assignee: Sakthi


HBASE-21284 was ought to be present in 2.2. But by the time the commit was 
done, the branch had been cut already. Hence this Jira to track it's inclusion.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-22775) Enhance logging for peer related operations

[Viraj Jasani (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024827#comment-17024827
 ] 

Viraj Jasani commented on HBASE-22775:
--

[~xucang] we can use {} for slf4j logging, otherwise the changes look good 
based on your recent explanation on why you want to put logs in 
ZKReplicationPeerStorage.

 

[~apurtell] hope, you are fine with this.

> Enhance logging for peer related operations
> ---
>
> Key: HBASE-22775
> URL: https://issues.apache.org/jira/browse/HBASE-22775
> Project: HBase
>  Issue Type: Improvement
>Reporter: Xu Cang
>Assignee: Xu Cang
>Priority: Minor
> Attachments: HBASE-22775.master.001.patch, 
> HBASE-22775.master.002.patch
>
>
> Now we don't have good logging regarding peer operations, for example addPeer 
> does not log itself:
> [https://github.com/apache/hbase/blob/master/hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ZKReplicationPeerStorage.java#L102]
> This Jira is aiming to enhancing this area



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-21284) Forward port HBASE-21000 to branch-2

[Sakthi (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-21284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024826#comment-17024826
 ] 

Sakthi commented on HBASE-21284:


Missing from 2.2. Let me create a subtask for the porting.

> Forward port HBASE-21000 to branch-2
> 
>
> Key: HBASE-21284
> URL: https://issues.apache.org/jira/browse/HBASE-21284
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Mingliang Liu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21284.001.patch, HBASE-21284.002.patch
>
>
> See parent for details.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] ndimiduk commented on issue #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on issue #754: HBASE-22978 : Online slow response log
URL: https://github.com/apache/hbase/pull/754#issuecomment-579042263
 
 
   I sent a note to dev@ giving a status update. If you don’t find there what
   you need, or have some comments, please reply on that thread.
   
   On Mon, Jan 27, 2020 at 16:42 Viraj Jasani  wrote:
   
   > Thanks for the review @ndimiduk 
   > I will try to be on top of this. Could you please let me know if you have
   > any estimate of 2.3.0 release? May be by end of next week?
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or unsubscribe
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23746) [Flakey Tests] Caused by: org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: hflush and hsync

[Michael Stack (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024805#comment-17024805
 ] 

Michael Stack commented on HBASE-23746:
---

Had to set flag on a few more tests. Attached the combined patch pushed on 
branch-2. Started up a nightly build.

> [Flakey Tests] Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
> -
>
> Key: HBASE-23746
> URL: https://issues.apache.org/jira/browse/HBASE-23746
> Project: HBase
>  Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 23746.patch
>
>
> Tests that use the new RegionProcedureStoreTestBase utility fail in branch-2 
> nightlies complaining...
> {code}
> Error Message
> cannot get log writer
> Stacktrace
> java.io.IOException: cannot get log writer
> Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
> {code}
> They are using local fs which doesn't have support for above. Only shows when 
> you run the tests with hadoop3 profile.  Let me do for these tests what was 
> done over in HBASE-19289
> It has been going on for a while now, probably since the move to region 
> procedure store (the old wal procedure store had turned off the checks for 
> sync/hflush).
> {code}
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestHFileProcedurePrettyPrinter.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStore.testLoad
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStore.testCleanup
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreCompaction.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreMigration.testMigrateWithUnsupportedProcedures
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreMigration.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreWALCleaner.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestWALProcedurePrettyPrinter.test
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23746) [Flakey Tests] Caused by: org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: hflush and hsync

[Michael Stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack updated HBASE-23746:
--
Attachment: 23746.patch

> [Flakey Tests] Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
> -
>
> Key: HBASE-23746
> URL: https://issues.apache.org/jira/browse/HBASE-23746
> Project: HBase
>  Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 23746.patch
>
>
> Tests that use the new RegionProcedureStoreTestBase utility fail in branch-2 
> nightlies complaining...
> {code}
> Error Message
> cannot get log writer
> Stacktrace
> java.io.IOException: cannot get log writer
> Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
> {code}
> They are using local fs which doesn't have support for above. Only shows when 
> you run the tests with hadoop3 profile.  Let me do for these tests what was 
> done over in HBASE-19289
> It has been going on for a while now, probably since the move to region 
> procedure store (the old wal procedure store had turned off the checks for 
> sync/hflush).
> {code}
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestHFileProcedurePrettyPrinter.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStore.testLoad
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStore.testCleanup
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreCompaction.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreMigration.testMigrateWithUnsupportedProcedures
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreMigration.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreWALCleaner.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestWALProcedurePrettyPrinter.test
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23746) [Flakey Tests] Caused by: org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: hflush and hsync

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024794#comment-17024794
 ] 

Hudson commented on HBASE-23746:


Results for branch branch-2
[build #2434 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2434/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2434//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2434//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2434//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> [Flakey Tests] Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
> -
>
> Key: HBASE-23746
> URL: https://issues.apache.org/jira/browse/HBASE-23746
> Project: HBase
>  Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Tests that use the new RegionProcedureStoreTestBase utility fail in branch-2 
> nightlies complaining...
> {code}
> Error Message
> cannot get log writer
> Stacktrace
> java.io.IOException: cannot get log writer
> Caused by: 
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: 
> hflush and hsync
> {code}
> They are using local fs which doesn't have support for above. Only shows when 
> you run the tests with hadoop3 profile.  Let me do for these tests what was 
> done over in HBASE-19289
> It has been going on for a while now, probably since the move to region 
> procedure store (the old wal procedure store had turned off the checks for 
> sync/hflush).
> {code}
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestHFileProcedurePrettyPrinter.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStore.testLoad
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStore.testCleanup
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreCompaction.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreMigration.testMigrateWithUnsupportedProcedures
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreMigration.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestRegionProcedureStoreWALCleaner.test
> health checks / yetus jdk8 hadoop3 checks / 
> org.apache.hadoop.hbase.procedure2.store.region.TestWALProcedurePrettyPrinter.test
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-22972) [JDK11] Support JDK11 LTS in HBase

[Duo Zhang (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-22972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024793#comment-17024793
 ] 

Duo Zhang commented on HBASE-22972:
---

+1 on splitting runtime and compile.

> [JDK11] Support JDK11 LTS in HBase
> --
>
> Key: HBASE-22972
> URL: https://issues.apache.org/jira/browse/HBASE-22972
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Blocker
>  Labels: jdk11
> Fix For: 3.0.0, 2.3.0
>
>
> This is an umbrella issue for tracking all the problems for JDK11 in HBase. 
> And we also rely on hadoop.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] joshelser commented on issue #936: HBASE-17115 Define UI admins via an ACL

[GitBox]

joshelser commented on issue #936: HBASE-17115 Define UI admins via an ACL
URL: https://github.com/apache/hbase/pull/936#issuecomment-579031784
 
 
   HBASE-20950 isn't contained on branch-2.1 which means I have to fix up 
TestInfoServersAcl.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (HBASE-23747) Support HBase getconf in hbase shell similar to hdfs

[Swaroopa Kadam (Jira)]

Swaroopa Kadam created HBASE-23747:
--

 Summary: Support HBase getconf in hbase shell similar to hdfs
 Key: HBASE-23747
 URL: https://issues.apache.org/jira/browse/HBASE-23747
 Project: HBase
  Issue Type: Improvement
Reporter: Swaroopa Kadam
Assignee: Swaroopa Kadam


Similar to HDFS getconf confkey -[key], HBase should provide a way to read 
configurations from HBase shell 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] virajjasani commented on issue #754: HBASE-22978 : Online slow response log

[GitBox]

virajjasani commented on issue #754: HBASE-22978 : Online slow response log
URL: https://github.com/apache/hbase/pull/754#issuecomment-579025766
 
 
   Thanks for the review @ndimiduk 
   I will try to be on top of this. Could you please let me know if you have 
any estimate of 2.3.0 release? May be by end of next week?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client

[Nick Dimiduk (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-18095:
-
Fix Version/s: 1.6.0
   2.3.0
   3.0.0

> Provide an option for clients to find the server hosting META that does not 
> involve the ZooKeeper client
> 
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
>  Issue Type: New Feature
>  Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the 
> regionserver hosting the meta table region. Site configuration provides the 
> client a list of ZK quorum peers and the client uses an embedded ZK client to 
> query meta location. Timeouts and retry behavior of this embedded ZK client 
> are managed orthogonally to HBase layer settings and in some cases the ZK 
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage 
> or network partition.
> We should consider new configuration settings that provide a list of 
> well-known master and backup master locations, and with this information the 
> client can contact any of the master processes directly. Any master in either 
> active or passive state will track meta location and respond to requests for 
> it with its cached last known location. If this location is stale, the client 
> can ask again with a flag set that requests the master refresh its location 
> cache and return the up-to-date location. Every client interaction with the 
> cluster thus uses only HBase RPC as transport, with appropriate settings 
> applied to the connection. The configuration toggle that enables this 
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and 
> contact the ZK service directly at the beginning of the connection lifecycle. 
> This has several benefits. ZK service need not be exposed to clients, and 
> their potential abuse, yet no benefit ZK provides the HBase server cluster is 
> compromised. Normalizing HBase client and ZK client timeout settings and 
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer 
> necessary. 
> And, from [~ghelmling]: There is an additional complication here for 
> token-based authentication. When a delegation token is used for SASL 
> authentication, the client uses the cluster ID obtained from Zookeeper to 
> select the token identifier to use. So there would also need to be some 
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-22972) [JDK11] Support JDK11 LTS in HBase

[Nick Dimiduk (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-22972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-22972:
-
Priority: Blocker  (was: Major)

> [JDK11] Support JDK11 LTS in HBase
> --
>
> Key: HBASE-22972
> URL: https://issues.apache.org/jira/browse/HBASE-22972
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Blocker
>  Labels: jdk11
> Fix For: 3.0.0, 2.3.0
>
>
> This is an umbrella issue for tracking all the problems for JDK11 in HBase. 
> And we also rely on hadoop.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-22972) [JDK11] Support JDK11 LTS in HBase

[Nick Dimiduk (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-22972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-22972:
-
Fix Version/s: 2.3.0
   3.0.0

> [JDK11] Support JDK11 LTS in HBase
> --
>
> Key: HBASE-22972
> URL: https://issues.apache.org/jira/browse/HBASE-22972
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
>  Labels: jdk11
> Fix For: 3.0.0, 2.3.0
>
>
> This is an umbrella issue for tracking all the problems for JDK11 in HBase. 
> And we also rely on hadoop.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-22972) [JDK11] Support JDK11 LTS in HBase

[Nick Dimiduk (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-22972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024781#comment-17024781
 ] 

Nick Dimiduk commented on HBASE-22972:
--

Would be nice to get this one closed down for 2.3.0. At least, the runtime side 
anyway. Is it worth breaking runtime from compile time like they did in Hadoop?

> [JDK11] Support JDK11 LTS in HBase
> --
>
> Key: HBASE-22972
> URL: https://issues.apache.org/jira/browse/HBASE-22972
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
>  Labels: jdk11
>
> This is an umbrella issue for tracking all the problems for JDK11 in HBase. 
> And we also rely on hadoop.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] bharathv opened a new pull request #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904)

[GitBox]

bharathv opened a new pull request #1098: HBASE-23304: RPCs needed for client 
meta information lookup (apache#904)
URL: https://github.com/apache/hbase/pull/1098
 
 
   This patch implements the RPCs needed for the meta information
   lookup during connection init. New tests added to cover the RPC
   code paths. HBASE-23305 builds on this to implement the client
   side logic.
   
   Fixed a bunch of checkstyle nits around the places the patch
   touches.
   
   Signed-off-by: Andrew Purtell 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] bharathv opened a new pull request #1097: HBASE-23281: Track meta region locations in masters (#830)

[GitBox]

bharathv opened a new pull request #1097: HBASE-23281: Track meta region 
locations in masters (#830)
URL: https://github.com/apache/hbase/pull/1097
 
 
   This patch adds a simple cache that tracks the meta region replica
   locations. It keeps an eye on the region movements so that the
   cached locations are not stale.
   
   This information is used for servicing client RPCs for connections
   that use master based registry (HBASE-18095). The RPC end points
   will be added in a separate patch.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] bharathv opened a new pull request #1096: HBASE-23275: Track active master's address in ActiveMasterManager (#812)

[GitBox]

bharathv opened a new pull request #1096: HBASE-23275: Track active master's 
address in ActiveMasterManager (#812)
URL: https://github.com/apache/hbase/pull/1096
 
 
   Currently we just track whether an active master exists.
   It helps to also track the address of the active master in
   all the masters to help serve the client RPC requests to
   know which master is active.
   
   Signed-off-by: Nick Dimiduk 
   Signed-off-by: Andrew Purtell 
   (cherry picked from commit efebb84)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] bharathv opened a new pull request #1095: HBASE-23257: Track clusterID in stand by masters (#798)

[GitBox]

bharathv opened a new pull request #1095: HBASE-23257: Track clusterID in stand 
by masters (#798)
URL: https://github.com/apache/hbase/pull/1095
 
 
   This patch implements a simple cache that all the masters
   can lookup to serve cluster ID to clients. Active HMaster
   is still responsible for creating it but all the masters
   will read it from fs to serve clients.
   
   RPCs exposing it will come in a separate patch as a part of
   HBASE-18095.
   
   Signed-off-by: Andrew Purtell 
   Signed-off-by: Wellington Chevreuil 
   Signed-off-by: Guangxu Cheng 
   (cherry picked from commit c2e01f2398424104a6faae901b12f782ca74c284)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-23737) [Flakey Tests] TestFavoredNodeTableImport fails 30% of the time

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024738#comment-17024738
 ] 

Hudson commented on HBASE-23737:


Results for branch branch-2.2
[build #767 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> [Flakey Tests] TestFavoredNodeTableImport fails 30% of the time
> ---
>
> Key: HBASE-23737
> URL: https://issues.apache.org/jira/browse/HBASE-23737
> Project: HBase
>  Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.9, 2.2.4
>
> Attachments: 
> 0001-HBASE-23737-Flakey-Tests-TestFavoredNodeTableImport-.patch, 
> 0001-HBASE-23737-Flakey-Tests-TestFavoredNodeTableImport.addendum.fix.patch
>
>
> Spent time on TestFavoredNodeTableImport. It fails w/ an NPE when we go to 
> get favorednodes for one of the regions. It is sporadic. Fails for me locally 
> too about 30% of the time.
> I tried to study where we are going wrong. The balancer is disabled when we 
> start the cluster up again on FN balancer... but this don't seem to be the 
> problem.
> Looks like laggard Regions taking their time to open means they don't show in 
> the global list of favored nodes when the checking runs. Adding a wait till 
> no RIT seems to stabilize the test.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024740#comment-17024740
 ] 

Hudson commented on HBASE-21874:


Results for branch branch-2.2
[build #767 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Bucket cache on Persistent memory
> -
>
> Key: HBASE-21874
> URL: https://issues.apache.org/jira/browse/HBASE-21874
> Project: HBase
>  Issue Type: New Feature
>  Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21874.branch-2.2.001.patch, HBASE-21874.patch, 
> HBASE-21874.patch, HBASE-21874_V2.patch, HBASE-21874_V4.patch, 
> HBASE-21874_V5.patch, HBASE-21874_V6.patch, Pmem_BC.png
>
>
> Non volatile persistent memory devices are byte addressable like DRAM (for 
> eg. Intel DCPMM). Bucket cache implementation can take advantage of this new 
> memory type and can make use of the existing offheap data structures to serve 
> data directly from this memory area without having to bring the data to 
> onheap.
> The patch is a new IOEngine implementation that works with the persistent 
> memory.
> Note : Here we don't make use of the persistence nature of the device and 
> just make use of the big memory it provides.
> Performance numbers to follow. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23734) Backport [HBASE-21874 Bucket cache on Persistent memory] to branch-2.2

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024739#comment-17024739
 ] 

Hudson commented on HBASE-23734:


Results for branch branch-2.2
[build #767 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/767//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Backport [HBASE-21874 Bucket cache on Persistent memory] to branch-2.2
> --
>
> Key: HBASE-23734
> URL: https://issues.apache.org/jira/browse/HBASE-23734
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.2.2, 2.2.3
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Fix For: 2.2.4
>
>
> We had patch for backporting HBASE-21874 into branch-2.2 attached to that 
> original jira, but it never went through and per discussions there, decided 
> to close it and open later a separate Jira for the backport.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase-operator-tools] jatsakthi commented on a change in pull request #47: HBASE-23180 hbck2 testing tool

[GitBox]

jatsakthi commented on a change in pull request #47: HBASE-23180 hbck2 testing 
tool
URL: 
https://github.com/apache/hbase-operator-tools/pull/47#discussion_r371527263
 
 

 ##
 File path: dev-support/testingtool/scripts/deleteRegionInMeta.sh
 ##
 @@ -0,0 +1,119 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This action script does the following:
+# 1. Delete one of the regions of the given table from hbase:meta using 
HBCKActions
+# 2. Runs hbck2's 'reportMissingRegionsInMeta' to make sure the deleted 
regions is reported missing
+# 3. Attemps to add the missing region in 'hbase:meta' using hbck2's 
'addFsRegionsMissingInMeta'
+# 4. Restarts the master as the above commands requires a master restart 
for cache refresh
+# 5. Runs hbck2's assigns command to finally assign the missing region
+# 6. Verifies the assignment using row count
+
+set -e
+function usage {
+  echo "Usage: ${0} hbase_client working_dir component_install hbck2_jar_path 
hadoop_jars hbase_version table_name"
+}
+
+function redirect_and_run {
+  log_base=$1
+  shift
+  echo "$*" >"${log_base}.err"
+  "$@" >"${log_base}.out" 2>>"${log_base}.err"
+}
+
+# if no args specified, show usage
+if [ $# -lt 7 ]; then
+  usage
+  exit 1
+fi
+
+# Carry forward the arguments from the control script
+declare hbase_client=$1
+declare working_dir=$2
+declare component_install=$3
+declare hbck2_jar_path=$4
+declare hadoop_jars=$5
+declare hbase_version=$6
+# The table name to operate with
+declare table_name=$7;
+
+
+echo "= Running hbckActions. It'll delete the region from meta table"
+redirect_and_run "${working_dir}/hbckActions_deleteRegionFromMeta" \
+java -cp 
"${working_dir}/hbase-conf/:${hbase_client}/lib/shaded-clients/hbase-shaded-client-byo-hadoop-${hbase_version}.jar:${working_dir}:${hadoop_jars}:${hbck2_jar_path}"
 org.apache.hbase.HBCKActions deleteRegionFromMeta "${table_name}"
+
+echo "= Running hbck2's reportMissingRegionsInMeta."
+HBASE_CLASSPATH_PREFIX="${hadoop_jars}" 
HBASE_CONF_DIR="${working_dir}/hbase-conf/" redirect_and_run \
+"${working_dir}/hbck2_reportMissingRegions" 
"${component_install}/bin/hbase" hbck -j "${hbck2_jar_path}" 
reportMissingRegionsInMeta "${table_name}"
+
+if [ ! -f "${working_dir}/hbck2_reportMissingRegions.out" ] ; then
+echo "ERROR: Logfile not found to grep/verify reportMissingRegions command"
+exit 2
+elif grep -q "\-\> No missing regions" 
"${working_dir}/hbck2_reportMissingRegions.out"  ; then
+echo "ERROR: No missing region. hbck2 failed. Exiting.."
+exit 2
+else
+echo "A region is missing as expected"
+grep "Missing Regions for each table" -A2 
"${working_dir}/hbck2_reportMissingRegions.out"
+fi
+
+echo "= Running hbck2's addFsRegionsMissingInMeta."
+HBASE_CLASSPATH_PREFIX="${hadoop_jars}" 
HBASE_CONF_DIR="${working_dir}/hbase-conf/" redirect_and_run \
+"${working_dir}/hbck2_addMissingRegions" "${component_install}/bin/hbase" 
hbck -j "${hbck2_jar_path}" addFsRegionsMissingInMeta "${table_name}"
+
+if [ ! -f "${working_dir}/hbck2_addMissingRegions.out" ] ; then
+echo "ERROR: Logfile not found to grep/verify addMissingRegions command"
+exit 2
+elif grep -q "regions were added to META" 
"${working_dir}/hbck2_addMissingRegions.out"; then
+echo "Region/s were added in META as expected"
+assigns_output=$(grep -Eo 'assigns [0-9a-z]+' 
"${working_dir}/hbck2_addMissingRegions.out")
+grep "Regions re-added into Meta" -A4 
"${working_dir}/hbck2_addMissingRegions.out"
+echo "The assigns command extraced from output is: $assigns_output"
+else
+echo "ERROR: No region added in META. hbck2 failed. Exiting.."
+exit 2
+fi
+
+echo "= Restarting HBase master..."
+HBASE_CONF_DIR="${working_dir}/hbase-conf/" 
"${component_install}/bin/rolling-restart.sh"
+
+sleep_time=2
+until "${component_install}/bin/hbase" --config "${working_dir}/hbase-conf/" 
shell --noninteractive >"${working_dir}/waiting_hbase_restart.log" 2>&1 <

[GitHub] [hbase] Apache-HBase commented on issue #1094: HBASE-23744 - FastPathBalancedQueueRpcExecutor should enforce queue l…

[GitBox]

Apache-HBase commented on issue #1094: HBASE-23744 - 
FastPathBalancedQueueRpcExecutor should enforce queue l…
URL: https://github.com/apache/hbase/pull/1094#issuecomment-578988099
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 32s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
1 new or modified test files.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 31s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 58s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m  7s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m 40s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  master passed  |
   | +0 :ok: |  spotbugs  |   4m 23s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   4m 20s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m  2s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 58s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 58s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m  7s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  1s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   4m 43s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  16m  1s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   0m 36s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   4m 13s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 156m 52s |  hbase-server in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   0m 32s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 214m 17s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1094/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1094 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux b99e1f3cf3b6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1094/out/precommit/personality/provided.sh
 |
   | git revision | master / fb06ca5ba4 |
   | Default Java | 1.8.0_181 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1094/1/testReport/
 |
   | Max. process+thread count | 5569 (vs. ulimit of 1) |
   | modules | C: hbase-server U: hbase-server |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1094/1/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (HBASE-21593) closing flags show be set false in HRegion

[Xu Cang (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024718#comment-17024718
 ] 

Xu Cang commented on HBASE-21593:
-

Where is the github PR for this issue? [~stack]

I came to cleanup my Jira issues and wanted to follow up on this one. If it has 
been solved. I can close it. Otherwise I can give another try. thanks.

> closing flags show be set false in HRegion
> --
>
> Key: HBASE-21593
> URL: https://issues.apache.org/jira/browse/HBASE-21593
> Project: HBase
>  Issue Type: Bug
>Reporter: xiaolerzheng
>Assignee: Xu Cang
>Priority: Minor
> Attachments: HBASE-21593.branch-1.001.patch, 
> image-2018-12-13-16-04-51-892.png, image-2018-12-13-16-05-09-246.png, 
> image-2018-12-13-16-05-36-404.png
>
>
> in HRegion.java
>  
>  
> 1429 // block waiting for the lock for closing
> 1430 lock.writeLock().lock();
> 1431 this.closing.set(true);
> 1432 status.setStatus("Disabling writes for close");
>  
> 
>  
>  
> 1557 } finally {
>        {color:#FF}  //should here add {color}
>     {color:#FF}    this.closing.set(false); {color}
> 1558  lock.writeLock().unlock();
> 1559 }



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-23731) TestFromClientSide flakey after junit upgrade

[HBase QA (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-23731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024702#comment-17024702
 ] 

HBase QA commented on HBASE-23731:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  2m  
0s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} HBASE-18095/client-locate-meta-no-zookeeper Compile 
Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
17s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed 
{color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
40s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed 
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
47s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed 
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  5m 
42s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
7s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed 
{color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  5m  
7s{color} | {color:blue} Used deprecated FindBugs config; considering switching 
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  6m  
0s{color} | {color:green} HBASE-18095/client-locate-meta-no-zookeeper passed 
{color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
 7s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
22m 37s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  6m 
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  3m 
43s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}180m 15s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  1m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}267m 32s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/4/artifact/out/Dockerfile
 |
| GITHUB PR | https://github.com/apache/hbase/pull/1091 |
| JIRA Issue | 

[GitHub] [hbase] Apache-HBase commented on issue #1091: HBASE-23731: De-flake TestFromClientSide

[GitBox]

Apache-HBase commented on issue #1091: HBASE-23731: De-flake TestFromClientSide
URL: https://github.com/apache/hbase/pull/1091#issuecomment-578974055
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   2m  0s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
2 new or modified test files.  |
   ||| _ HBASE-18095/client-locate-meta-no-zookeeper Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 34s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   7m 17s |  
HBASE-18095/client-locate-meta-no-zookeeper passed  |
   | +1 :green_heart: |  compile  |   1m 40s |  
HBASE-18095/client-locate-meta-no-zookeeper passed  |
   | +1 :green_heart: |  checkstyle  |   1m 47s |  
HBASE-18095/client-locate-meta-no-zookeeper passed  |
   | +1 :green_heart: |  shadedjars  |   5m 42s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  7s |  
HBASE-18095/client-locate-meta-no-zookeeper passed  |
   | +0 :ok: |  spotbugs  |   5m  7s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   6m  0s |  
HBASE-18095/client-locate-meta-no-zookeeper passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 16s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   6m 32s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 27s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 27s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 48s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   6m  7s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  22m 37s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   1m 11s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   6m 44s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   3m 43s |  hbase-common in the patch passed.  
|
   | -1 :x: |  unit  | 180m 15s |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  asflicense  |   1m 15s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 267m 32s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.5 Server=19.03.5 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/4/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1091 |
   | JIRA Issue | HBASE-23731 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 046df39cb8f1 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1091/out/precommit/personality/provided.sh
 |
   | git revision | HBASE-18095/client-locate-meta-no-zookeeper / 39ff6f82d9 |
   | Default Java | 1.8.0_181 |
   | unit | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/4/artifact/out/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/4/testReport/
 |
   | Max. process+thread count | 5244 (vs. ulimit of 1) |
   | modules | C: hbase-common hbase-server U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1091/4/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (HBASE-23731) TestFromClientSide flakey after junit upgrade

[Bharath Vissapragada (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bharath Vissapragada resolved HBASE-23731.
--
Fix Version/s: HBASE-18095
   Resolution: Fixed

Thanks [~stack] and [~janh] for the review.

> TestFromClientSide flakey after junit upgrade
> -
>
> Key: HBASE-23731
> URL: https://issues.apache.org/jira/browse/HBASE-23731
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: HBASE-18095
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
>  Labels: flaky-test
> Fix For: HBASE-18095
>
>
> The cluster shutdown and restart between parameterized runs causes a thread 
> leak with stacks like follows.
> {noformat}
> 2020-01-22 09:52:10,900 WARN  [LeaseRenewer:hostname@localhost:62252] 
> impl.LeaseRenewer(438): Failed to renew lease for 
> [DFSClient_NONMAPREDUCE_-1123200667_19] for 73 seconds.  Will retry shortly 
> ...
> java.net.ConnectException: Call From x/ to localhost:62252 failed on 
> connection exception: java.net.ConnectException: Connection refused; For more 
> details see:  http://wiki.apache.org/hadoop/ConnectionRefused
>   at sun.reflect.GeneratedConstructorAccessor64.newInstance(Unknown 
> Source)
>   at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>   at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:801)
>   at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
>   at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1493)
>   at org.apache.hadoop.ipc.Client.call(Client.java:1435)
>   at org.apache.hadoop.ipc.Client.call(Client.java:1345)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
>   at com.sun.proxy.$Proxy29.renewLease(Unknown Source)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.renewLease(ClientNamenodeProtocolTranslatorPB.java:613)
>   at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
>   at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
>   at com.sun.proxy.$Proxy30.renewLease(Unknown Source)
>   at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
>   at com.sun.proxy.$Proxy33.renewLease(Unknown Source)
>   at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
>   at com.sun.proxy.$Proxy33.renewLease(Unknown Source)
>   at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:560)
>   at 
> org.apache.hadoop.hdfs.client.impl.LeaseRenewer.renew(LeaseRenewer.java:395)
>   at 
> org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:415)
>   at 
> org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76)
>   at 
> org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307)
>   at java.lang.Thread.run(Thread.java:748)
> {noformat}
> While the issue existed even before, it looks like junit 4.12 somehow was 
> masking it. Upgrade to 4.13 made it worse.  The host:port refers to already 
> shutdown hdfs cluster, so ideally this thread shouldn't be there in the first 
> place. A culprit could be leaking filesystem / dfsclient objects.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] ndimiduk commented on a change in pull request #1039: HBASE-23647: Make MasterRegistry the default impl.

[GitBox]

ndimiduk commented on a change in pull request #1039: HBASE-23647: Make 
MasterRegistry the default impl.
URL: https://github.com/apache/hbase/pull/1039#discussion_r371497725
 
 

 ##
 File path: 
hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseConfiguration.java
 ##
 @@ -268,14 +268,19 @@ public static Configuration 
createClusterConf(Configuration baseConf, String clu
* used to communicate with distant clusters
* @param conf configuration object to configure
* @param key string that contains the 3 required configuratins
-   * @throws IOException
*/
   private static void applyClusterKeyToConf(Configuration conf, String key)
-  throws IOException{
+  throws IOException {
 ZKConfig.ZKClusterKey zkClusterKey = ZKConfig.transformClusterKey(key);
 conf.set(HConstants.ZOOKEEPER_QUORUM, zkClusterKey.getQuorumString());
 conf.setInt(HConstants.ZOOKEEPER_CLIENT_PORT, 
zkClusterKey.getClientPort());
 conf.set(HConstants.ZOOKEEPER_ZNODE_PARENT, zkClusterKey.getZnodeParent());
+// Without the right registry, the above configs are useless. Also, we 
don't use setClass()
+// here because the ConnectionRegistry* classes are not resolvable from 
this module.
+// This will be broken if ZkConnectionRegistry class gets renamed or 
moved. Is there a better
+// way?
 
 Review comment:
   > Overall, I think all MR jobs running on a single cluster will use master 
registry. I think that answers the MR/Spark/Flink usecases.
   
   Sounds good.
   
   > I think these need to be rewritten with new config params like 
hbase.[source|target].master.addrs for clients to pass the addresses so that 
they can use master registry.
   
   I think it's best to reduce the ZK-exposed surface area as much as possible. 
This seems a reasonable solution to me.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (HBASE-23647) Make MasterRegistry the default registry impl

[Nick Dimiduk (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-23647:
-
Fix Version/s: HBASE-18095

> Make MasterRegistry the default registry impl
> -
>
> Key: HBASE-23647
> URL: https://issues.apache.org/jira/browse/HBASE-23647
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Affects Versions: 3.0.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: HBASE-18095
>
>
> Now that we are close to getting the registry implementation patch in, the 
> idea here is to make it the default implementation in 3.0.0 and this means
> - No known bugs with the implementation
> - No known performance issues
> - Entire nightly test suite is green (and without flakes).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HBASE-23647) Make MasterRegistry the default registry impl

[Nick Dimiduk (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk updated HBASE-23647:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Make MasterRegistry the default registry impl
> -
>
> Key: HBASE-23647
> URL: https://issues.apache.org/jira/browse/HBASE-23647
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Affects Versions: 3.0.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: HBASE-18095
>
>
> Now that we are close to getting the registry implementation patch in, the 
> idea here is to make it the default implementation in 3.0.0 and this means
> - No known bugs with the implementation
> - No known performance issues
> - Entire nightly test suite is green (and without flakes).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client

[Hudson (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17024678#comment-17024678
 ] 

Hudson commented on HBASE-18095:


Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #52 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/52/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/52//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/52//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/52//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Provide an option for clients to find the server hosting META that does not 
> involve the ZooKeeper client
> 
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
>  Issue Type: New Feature
>  Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the 
> regionserver hosting the meta table region. Site configuration provides the 
> client a list of ZK quorum peers and the client uses an embedded ZK client to 
> query meta location. Timeouts and retry behavior of this embedded ZK client 
> are managed orthogonally to HBase layer settings and in some cases the ZK 
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage 
> or network partition.
> We should consider new configuration settings that provide a list of 
> well-known master and backup master locations, and with this information the 
> client can contact any of the master processes directly. Any master in either 
> active or passive state will track meta location and respond to requests for 
> it with its cached last known location. If this location is stale, the client 
> can ask again with a flag set that requests the master refresh its location 
> cache and return the up-to-date location. Every client interaction with the 
> cluster thus uses only HBase RPC as transport, with appropriate settings 
> applied to the connection. The configuration toggle that enables this 
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and 
> contact the ZK service directly at the beginning of the connection lifecycle. 
> This has several benefits. ZK service need not be exposed to clients, and 
> their potential abuse, yet no benefit ZK provides the HBase server cluster is 
> compromised. Normalizing HBase client and ZK client timeout settings and 
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer 
> necessary. 
> And, from [~ghelmling]: There is an additional complication here for 
> token-based authentication. When a delegation token is used for SASL 
> authentication, the client uses the cluster ID obtained from Zookeeper to 
> select the token identifier to use. So there would also need to be some 
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [hbase] ndimiduk commented on issue #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on issue #754: HBASE-22978 : Online slow response log
URL: https://github.com/apache/hbase/pull/754#issuecomment-578952421
 
 
   A couple high-level questions:
   
   1. Since the detailed info is going through this ring buffer subsystem, why 
do we continue to log at all? Why not replace the logging with a metric? 
   1. Should entries of the ring buffer expire after a period of time?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371445755
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
 ##
 @@ -3889,4 +3890,72 @@ private void getProcedureResult(long procId, 
CompletableFuture future, int
 .call();
   }
 
+  @Override
+  public CompletableFuture> 
getSlowLogResponses(List serverNames)
+  throws IOException {
+CompletableFuture> slowLoadPayloads = null;
+if (CollectionUtils.isNotEmpty(serverNames)) {
+  for (ServerName serverName : serverNames) {
+CompletableFuture> slowLogPayloadFromServer =
+  getSlowLogResponseFromServer(serverName);
+if (slowLoadPayloads == null) {
+  slowLoadPayloads = slowLogPayloadFromServer;
+  continue;
+}
+slowLoadPayloads = 
slowLoadPayloads.thenCombine(slowLogPayloadFromServer, (l1, l2) -> {
+  List finalSlowLogPayloads = new ArrayList<>();
 
 Review comment:
   We have no clue what the length of these lists will be? How do we expect the 
caller to consume the results? Should this be a `LinkedList` instead?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371447304
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
 ##
 @@ -3889,4 +3890,72 @@ private void getProcedureResult(long procId, 
CompletableFuture future, int
 .call();
   }
 
+  @Override
+  public CompletableFuture> 
getSlowLogResponses(List serverNames)
+  throws IOException {
+CompletableFuture> slowLoadPayloads = null;
 
 Review comment:
   Should the default case instead return a `completedFuture` over an empty 
list?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371433786
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
 ##
 @@ -440,25 +463,28 @@ public void setSecretManager(SecretManager secretMana
* @param param The parameters received in the call.
* @param methodName The name of the method invoked
* @param call The string representation of the call
-   * @param tag  The tag that will be used to indicate this event in the log.
+   * @param tooLarge To indicate if the event is tooLarge or tooSlow
 
 Review comment:
   An event that has `tooLarge=false` does not imply to me that `tooSlow=true`. 
These are independent concepts that should not be mashed into a single 
parameter. The previous parameter "tag" was simply a log annotation, not a 
decision criteria. Here you've replaced it with a decision condition that lacks 
expression. If you want to pursue this change, please include parameters for 
both boolean flags, or implement something like a bit-mask that allows for 
multiple states to be expressed simultaneously.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371427340
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
 ##
 @@ -85,6 +90,10 @@
   protected static final CallQueueTooBigException CALL_QUEUE_TOO_BIG_EXCEPTION
   = new CallQueueTooBigException();
 
+  private static final String MULTI_GETS = "multi.gets";
+  private static final String MULTI_MUTATIONS = "multi.mutations";
+  private static final String MULTI_SERVICECALLS = "multi.servicecalls";
 
 Review comment:
   "MULTI_SERVICE_CALLS," "multi.service_calls".


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371431623
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
 ##
 @@ -215,6 +224,17 @@
*/
   private RSRpcServices rsRpcServices;
 
+
+  /**
+   * Use to add online slowlog responses
+   */
+  private OnlineSlowLogProvider onlineSlowLogProvider;
 
 Review comment:
   This cannot be final? :(


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371457333
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
 ##
 @@ -3818,6 +3828,40 @@ public ExecuteProceduresResponse 
executeProcedures(RpcController controller,
 }
   }
 
+  @Override
+  @QosPriority(priority = HConstants.ADMIN_QOS)
+  public SlowLogResponses getSlowLogResponses(final RpcController controller,
+  final SlowLogResponseRequest request) throws ServiceException {
 
 Review comment:
   > `throws ServiceException`
   
   No it doesn't.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371439546
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 ##
 @@ -590,6 +596,10 @@ public HRegionServer(final Configuration conf) throws 
IOException {
   this.abortRequested = false;
   this.stopped = false;
 
+  // initiate online slowlog ringbuffer only for RegionServers
+  if (!(this instanceof HMaster)) {
 
 Review comment:
   Always instantiate this field? Even when the configuration is disabled?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371448948
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
 ##
 @@ -1507,4 +1507,26 @@
*/
   CompletableFuture isSnapshotCleanupEnabled();
 
+  /**
+   * Retrieves online slowlog responses from in memory ringbuffer maintained by
+   * RegionServers
+   *
+   * @param serverNames Server names to get slowlog responses from
+   * @return Online slowlog response list. The return value wrapped by a 
{@link CompletableFuture}
+   * @throws IOException if a remote or network exception occurs
 
 Review comment:
   Does it?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371458942
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
 ##
 @@ -3818,6 +3828,40 @@ public ExecuteProceduresResponse 
executeProcedures(RpcController controller,
 }
   }
 
+  @Override
+  @QosPriority(priority = HConstants.ADMIN_QOS)
+  public SlowLogResponses getSlowLogResponses(final RpcController controller,
+  final SlowLogResponseRequest request) throws ServiceException {
+final OnlineSlowLogProvider onlineSlowLogProvider =
+  this.regionServer.getOnlineSlowLogProvider();
+List slowLogPayloads;
+if (onlineSlowLogProvider != null) {
+  slowLogPayloads = onlineSlowLogProvider.getSlowLogPayloads();
+} else {
+  slowLogPayloads = new ArrayList<>();
+}
+SlowLogResponses slowLogResponses = SlowLogResponses.newBuilder()
+  .addAllSlowLogPayloads(slowLogPayloads)
+  .build();
+return slowLogResponses;
+  }
+
+  @Override
+  @QosPriority(priority = HConstants.ADMIN_QOS)
+  public ClearSlowLogResponses clearSlowLogsResponses(final RpcController 
controller,
+  final ClearSlowLogResponseRequest request) throws ServiceException {
+final OnlineSlowLogProvider onlineSlowLogProvider =
+  this.regionServer.getOnlineSlowLogProvider();
+boolean slowLogsCleaned = false;
 
 Review comment:
   You have here an ideal use of `Optional`; please use it.
   
   ```
   final boolean slowLogsCleaned = 
Optional.ofNullable(onlineSlowLogProvider)
 .map(OnlineSlowLogProvider::clearSlowLogPayloads)
 .orElse(false);
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371453733
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/SlowLogPayload.java
 ##
 @@ -0,0 +1,318 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.client;
+
+import org.apache.hadoop.hbase.util.GsonUtil;
+import org.apache.yetus.audience.InterfaceAudience;
+
+import org.apache.hbase.thirdparty.com.google.gson.Gson;
+import org.apache.hbase.thirdparty.com.google.gson.JsonObject;
+import org.apache.hbase.thirdparty.com.google.gson.JsonSerializer;
+
+/**
+ * SlowLog payload for client
+ */
+@InterfaceAudience.Private
+final public class SlowLogPayload {
+
+  private static final Gson GSON = GsonUtil.createGson()
+.setPrettyPrinting()
+.registerTypeAdapter(SlowLogPayload.class, (JsonSerializer)
+  (slowLogPayload, type, jsonSerializationContext) -> {
+Gson gson = new Gson();
+JsonObject jsonObj = (JsonObject) gson.toJsonTree(slowLogPayload);
+if (slowLogPayload.getMultiGetsCount() == 0) {
+  jsonObj.remove("multiGetsCount");
+}
+if (slowLogPayload.getMultiMutationsCount() == 0) {
+  jsonObj.remove("multiMutationsCount");
+}
+if (slowLogPayload.getMultiServiceCalls() == 0) {
+  jsonObj.remove("multiServiceCalls");
+}
+return jsonObj;
+  }).create();
+
+  private long startTime;
+  private int processingTime;
+  private int queueTime;
+  private long responseSize;
+  private String clientAddress;
+  private String serverClass;
+  private String methodName;
+  private String callDetails;
+  private String param;
+  private transient String regionName;
+  private String userName;
+  private int multiGetsCount;
+  private int multiMutationsCount;
+  private int multiServiceCalls;
+
+  public long getStartTime() {
+return startTime;
+  }
+
+  public int getProcessingTime() {
+return processingTime;
+  }
+
+  public int getQueueTime() {
+return queueTime;
+  }
+
+  public long getResponseSize() {
+return responseSize;
+  }
+
+  public String getClientAddress() {
+return clientAddress;
+  }
+
+  public String getServerClass() {
+return serverClass;
+  }
+
+  public String getMethodName() {
+return methodName;
+  }
+
+  public String getCallDetails() {
+return callDetails;
+  }
+
+  public String getParam() {
+return param;
+  }
+
+  public String getRegionName() {
+return regionName;
+  }
+
+  public String getUserName() {
+return userName;
+  }
+
+  public int getMultiGetsCount() {
+return multiGetsCount;
+  }
+
+  public int getMultiMutationsCount() {
+return multiMutationsCount;
+  }
+
+  public int getMultiServiceCalls() {
+return multiServiceCalls;
+  }
+
+  private SlowLogPayload(final long startTime, final int processingTime, final 
int queueTime,
+  final long responseSize, final String clientAddress, final String 
serverClass,
+  final String methodName, final String callDetails, final String param,
+  final String regionName, final String userName, final int multiGetsCount,
+  final int multiMutationsCount, final int multiServiceCalls) {
+this.startTime = startTime;
+this.processingTime = processingTime;
+this.queueTime = queueTime;
+this.responseSize = responseSize;
+this.clientAddress = clientAddress;
+this.serverClass = serverClass;
+this.methodName = methodName;
+this.callDetails = callDetails;
+this.param = param;
+this.regionName = regionName;
+this.userName = userName;
+this.multiGetsCount = multiGetsCount;
+this.multiMutationsCount = multiMutationsCount;
+this.multiServiceCalls = multiServiceCalls;
+  }
+
+  public static class SlowLogPayloadBuilder {
+private long startTime;
+private int processingTime;
+private int queueTime;
+private long responseSize;
+private String clientAddress;
+private String serverClass;
+private String methodName;
+private String callDetails;
+private String param;
+private String regionName;
+priv

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371426646
 
 

 ##
 File path: hbase-protocol-shaded/src/main/protobuf/TooSlowLog.proto
 ##
 @@ -0,0 +1,45 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+syntax = "proto2";
+
+// This file contains protocol buffers that are used for Online TooSlowLogs
+// To be used as Ring Buffer payload
+package hbase.pb;
+
+option java_package = "org.apache.hadoop.hbase.shaded.protobuf.generated";
+option java_outer_classname = "TooSlowLog";
+option java_generate_equals_and_hash = true;
+option optimize_for = SPEED;
+
+message SlowLogPayload {
+required int64 start_time = 1;
+required int32 processing_time = 2;
+required int32 queue_time = 3;
+required int64 response_size = 4;
+required string client_address = 5;
+required string server_class = 6;
+required string method_name = 7;
+required string call_details = 8;
+required string param = 9;
+required string user_name = 10;
 
 Review comment:
   What is recorded when there is no user authentication enabled in the cluster 
configuration? Just `hbase`? Should this field be made optional so as to save 
the space?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371457420
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
 ##
 @@ -3818,6 +3828,40 @@ public ExecuteProceduresResponse 
executeProcedures(RpcController controller,
 }
   }
 
+  @Override
+  @QosPriority(priority = HConstants.ADMIN_QOS)
+  public SlowLogResponses getSlowLogResponses(final RpcController controller,
+  final SlowLogResponseRequest request) throws ServiceException {
+final OnlineSlowLogProvider onlineSlowLogProvider =
+  this.regionServer.getOnlineSlowLogProvider();
+List slowLogPayloads;
+if (onlineSlowLogProvider != null) {
+  slowLogPayloads = onlineSlowLogProvider.getSlowLogPayloads();
+} else {
+  slowLogPayloads = new ArrayList<>();
+}
+SlowLogResponses slowLogResponses = SlowLogResponses.newBuilder()
+  .addAllSlowLogPayloads(slowLogPayloads)
+  .build();
+return slowLogResponses;
+  }
+
+  @Override
+  @QosPriority(priority = HConstants.ADMIN_QOS)
+  public ClearSlowLogResponses clearSlowLogsResponses(final RpcController 
controller,
+  final ClearSlowLogResponseRequest request) throws ServiceException {
 
 Review comment:
   > `throws ServiceException`
   
   No it doesn't.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371431837
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
 ##
 @@ -215,6 +224,17 @@
*/
   private RSRpcServices rsRpcServices;
 
+
+  /**
+   * Use to add online slowlog responses
+   */
+  private OnlineSlowLogProvider onlineSlowLogProvider;
+
+  private final boolean isOnlineSlowLogProviderEnabled;
+
 
 Review comment:
   nit: trim all these extra newlines between conceptually related fields.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371450989
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/SlowLogParams.java
 ##
 @@ -0,0 +1,55 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.client;
+
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * SlowLog params object that contains detailed info as params and region name 
: to be used
+ * for filter purpose
+ */
+@InterfaceAudience.Private
+public class SlowLogParams {
+
+  private String regionName;
+  private String params;
+
+  public SlowLogParams(String regionName, String params) {
+this.regionName = regionName;
+this.params = params;
+  }
+
+  public String getRegionName() {
+return regionName;
+  }
+
+  public String getParams() {
+return params;
+  }
+
+  @Override
+  public String toString() {
+final StringBuilder sb = new StringBuilder("SlowLogParams{");
+sb.append("regionName='").append(regionName).append('\'');
+sb.append(", params='").append(params).append('\'');
+sb.append('}');
+return sb.toString();
+  }
+}
 
 Review comment:
   Do you expect this class to be used in a collection object? If so, it should 
include `hashCode` and `equals` implementations... which are also easily built 
with Apache Commons Lang3 Builder helpers.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371450271
 
 

 ##
 File path: 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/SlowLogParams.java
 ##
 @@ -0,0 +1,55 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.client;
+
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * SlowLog params object that contains detailed info as params and region name 
: to be used
+ * for filter purpose
+ */
+@InterfaceAudience.Private
+public class SlowLogParams {
+
+  private String regionName;
+  private String params;
+
+  public SlowLogParams(String regionName, String params) {
+this.regionName = regionName;
+this.params = params;
+  }
+
+  public String getRegionName() {
+return regionName;
+  }
+
+  public String getParams() {
+return params;
+  }
+
+  @Override
+  public String toString() {
+final StringBuilder sb = new StringBuilder("SlowLogParams{");
 
 Review comment:
   Please use `ToStringBuilder` with `ToStringStyle.SHORT_PREFIX_STYLE`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371460647
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/slowlog/DisruptorExceptionHandler.java
 ##
 @@ -0,0 +1,53 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.slowlog;
+
+import com.lmax.disruptor.ExceptionHandler;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Exception Handler for Online Slow Log Ring Buffer
+ */
+@InterfaceAudience.Private
+class DisruptorExceptionHandler implements ExceptionHandler {
+
+  private static final Logger LOG = 
LoggerFactory.getLogger(DisruptorExceptionHandler.class);
+
+  @Override
+  public void handleEventException(Throwable ex, long sequence, 
RingBufferTruck event) {
+LOG.error("Sequence={}, event={}", sequence, event, ex);
+throw new RuntimeException(ex);
+  }
+
+  @Override
+  public void handleOnStartException(Throwable ex) {
+LOG.error("Disruptor onStartException: {}", ex.toString(), ex);
 
 Review comment:
   1. Don't log the exception message and then log the exception, that's 
redundant.
   1. Re-throwing this exception means a higher lays is going to deal with the 
failure, so this log message provides no value.
   1. Re-throwing means the RS will halt under this condition. Is this what you 
want?
   1. The same questions apply to all of these handlers.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371439669
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 ##
 @@ -590,6 +596,10 @@ public HRegionServer(final Configuration conf) throws 
IOException {
   this.abortRequested = false;
   this.stopped = false;
 
+  // initiate online slowlog ringbuffer only for RegionServers
 
 Review comment:
   nit: this comment isn't helpful.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371438102
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
 ##
 @@ -496,11 +522,44 @@ void logResponse(Message param, String methodName, 
String call, String tag,
   }
 }
   }
-  responseInfo.put("multi.gets", numGets);
-  responseInfo.put("multi.mutations", numMutations);
-  responseInfo.put("multi.servicecalls", numServiceCalls);
+  responseInfo.put(MULTI_GETS, numGets);
+  responseInfo.put(MULTI_MUTATIONS, numMutations);
+  responseInfo.put(MULTI_SERVICECALLS, numServiceCalls);
 }
+final String tag = tooLarge ? "TooLarge" : "TooSlow";
 LOG.warn("(response" + tag + "): " + GSON.toJson(responseInfo));
+if (isOnlineSlowLogProviderEnabled && !tooLarge && 
this.onlineSlowLogProvider != null
+&& !call.startsWith("GetSlowLogResponses")
+&& !call.startsWith("ClearSlowLogsResponses")) {
+  // add too slow log to ringbuffer for retrieval of latest n slow logs
+
+  try {
+final SlowLogParams slowLogParams = 
ProtobufUtil.getSlowLogParams(param);
+
+final SlowLogPayload slowLogPayload = SlowLogPayload.newBuilder()
+  .setStartTime(startTime)
+  .setProcessingTime(processingTime)
+  .setQueueTime(qTime)
+  .setResponseSize(responseSize)
+  .setClientAddress(clientAddress)
+  .setServerClass(className)
+  .setMethodName(methodName)
+  .setCallDetails(call)
+  .setUserName(userName)
+  .setRegionName(slowLogParams != null ? slowLogParams.getRegionName() 
: StringUtils.EMPTY)
 
 Review comment:
   What's the point of making these `required` in the protobuf definition if 
you're just going to shove empty strings in? When a value isn't available, make 
them `optional` and leave them unset.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371436457
 
 

 ##
 File path: 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
 ##
 @@ -496,11 +522,44 @@ void logResponse(Message param, String methodName, 
String call, String tag,
   }
 }
   }
-  responseInfo.put("multi.gets", numGets);
-  responseInfo.put("multi.mutations", numMutations);
-  responseInfo.put("multi.servicecalls", numServiceCalls);
+  responseInfo.put(MULTI_GETS, numGets);
+  responseInfo.put(MULTI_MUTATIONS, numMutations);
+  responseInfo.put(MULTI_SERVICECALLS, numServiceCalls);
 }
+final String tag = tooLarge ? "TooLarge" : "TooSlow";
 LOG.warn("(response" + tag + "): " + GSON.toJson(responseInfo));
+if (isOnlineSlowLogProviderEnabled && !tooLarge && 
this.onlineSlowLogProvider != null
+&& !call.startsWith("GetSlowLogResponses")
+&& !call.startsWith("ClearSlowLogsResponses")) {
+  // add too slow log to ringbuffer for retrieval of latest n slow logs
+
+  try {
+final SlowLogParams slowLogParams = 
ProtobufUtil.getSlowLogParams(param);
+
+final SlowLogPayload slowLogPayload = SlowLogPayload.newBuilder()
+  .setStartTime(startTime)
+  .setProcessingTime(processingTime)
+  .setQueueTime(qTime)
+  .setResponseSize(responseSize)
+  .setClientAddress(clientAddress)
+  .setServerClass(className)
+  .setMethodName(methodName)
+  .setCallDetails(call)
+  .setUserName(userName)
+  .setRegionName(slowLogParams != null ? slowLogParams.getRegionName() 
: StringUtils.EMPTY)
+  .setParam(slowLogParams != null ? slowLogParams.getParams() : 
StringUtils.EMPTY)
+  .setMultiGets(responseInfo.containsKey(MULTI_GETS)
+? (int) responseInfo.get(MULTI_GETS) : 0)
+  .setMultiMutations(responseInfo.containsKey(MULTI_MUTATIONS)
+? (int) responseInfo.get(MULTI_MUTATIONS) : 0)
+  .setMultiServiceCalls(responseInfo.containsKey(MULTI_SERVICECALLS)
+? (int) responseInfo.get(MULTI_SERVICECALLS) : 0)
+  .build();
+this.onlineSlowLogProvider.addSlowLogPayload(slowLogPayload);
+  } catch (Exception e) {
+LOG.error("Error while adding slowlog response to ringbuffer..", e);
 
 Review comment:
   Why is this logging instead of emitting a metric? When something breaks, is 
this going to log 1000's of times per second? A recording a message in the ring 
buffer is a nice-to-have, not a critical operation; when it fails, it is not a 
server ERROR scenario. Please log at debug instead of error.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow response log

[GitBox]

ndimiduk commented on a change in pull request #754: HBASE-22978 : Online slow 
response log
URL: https://github.com/apache/hbase/pull/754#discussion_r371443217
 
 

 ##
 File path: 
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
 ##
 @@ -426,4 +437,55 @@ public void testAsyncTimeout() throws IOException {
   rpcServer.stop();
 }
   }
+
+  @Test
 
 Review comment:
   This feature isn't related to the IPC server, it's an internal detail of the 
Region Server. Thus the tests should be on the region server, not the IPC tests.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services