[GitHub] [hbase] cuibo01 commented on pull request #2690: HBASE-24395 backport HBASE-20589
cuibo01 commented on pull request #2690: URL: https://github.com/apache/hbase/pull/2690#issuecomment-731706228  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
Apache-HBase commented on pull request #2630: URL: https://github.com/apache/hbase/pull/2630#issuecomment-731705982 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 3s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 59s | master passed | | +1 :green_heart: | compile | 2m 18s | master passed | | +1 :green_heart: | shadedjars | 7m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 49s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | compile | 2m 18s | the patch passed | | +1 :green_heart: | javac | 2m 18s | the patch passed | | +1 :green_heart: | shadedjars | 7m 15s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 49s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 7s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 203m 42s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 42s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 4m 2s | hbase-rest in the patch passed. | | | | 248m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2630 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c9540b7d6ab0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c9156e7891 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/4/testReport/ | | Max. process+thread count | 3482 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-24395) ServerName#getHostname() is case sensitive
[ https://issues.apache.org/jira/browse/HBASE-24395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui reassigned HBASE-24395: -- Assignee: Bo Cui > ServerName#getHostname() is case sensitive > -- > > Key: HBASE-24395 > URL: https://issues.apache.org/jira/browse/HBASE-24395 > Project: HBase > Issue Type: Sub-task > Components: Balancer >Affects Versions: 1.3.1 >Reporter: Bo Cui >Assignee: Bo Cui >Priority: Major > Attachments: HBase-24395.patch, image-2020-05-18-17-42-57-119.png > > > ServerName calss，the getServerName(String hostName, int port, long > startcode),equals and compareTo are case insensitive, but getHostname() is > case sensitive. > if hostName is HOSTNAME1, ServerName is hostname1,1,1589615319931, and > getHostname() returns HOSTNAME1. > and then BaseLoadBalancer#retainAssignment() uses ServerName#getHostname(), > all keys of serversByHostname are > upperCase(HOSTNAME1,HOSTNAME2,HOSTNAME3,HOSTNAME4...) from > ServerManager#createDestinationServersList, but oldServerName.getHostname() > is lowerCase(hostname1,hostname2,hostname3...) from walLog dir. > !image-2020-05-18-17-42-57-119.png! > and finally...all region of old ServerName will be assigned to random hosts -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
Apache-HBase commented on pull request #2630: URL: https://github.com/apache/hbase/pull/2630#issuecomment-731702162 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 43s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 29s | master passed | | +1 :green_heart: | compile | 3m 34s | master passed | | +1 :green_heart: | shadedjars | 9m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 47s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 41s | the patch passed | | +1 :green_heart: | compile | 3m 17s | the patch passed | | +1 :green_heart: | javac | 3m 17s | the patch passed | | +1 :green_heart: | shadedjars | 9m 23s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 23s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 137m 48s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 5s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 3m 18s | hbase-rest in the patch passed. | | | | 193m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2630 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5a892764f1f1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c9156e7891 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/4/testReport/ | | Max. process+thread count | 3807 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2688: HBASE-25311 UI throws NPE
Apache-HBase commented on pull request #2688: URL: https://github.com/apache/hbase/pull/2688#issuecomment-731694716 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 16s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 6m 49s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 45s | the patch passed | | +1 :green_heart: | compile | 1m 10s | the patch passed | | +1 :green_heart: | javac | 1m 10s | the patch passed | | +1 :green_heart: | shadedjars | 7m 27s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 139m 24s | hbase-server in the patch passed. | | | | 168m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2688 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6e6bb7217f96 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c9156e7891 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/3/testReport/ | | Max. process+thread count | 4247 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
Apache-HBase commented on pull request #2630: URL: https://github.com/apache/hbase/pull/2630#issuecomment-731692206 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 44s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 21s | master passed | | +1 :green_heart: | checkstyle | 3m 4s | master passed | | +1 :green_heart: | spotbugs | 5m 56s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 15s | the patch passed | | +1 :green_heart: | checkstyle | 0m 35s | hbase-client: The patch generated 0 new + 276 unchanged - 1 fixed = 276 total (was 277) | | +1 :green_heart: | checkstyle | 1m 25s | The patch passed checkstyle in hbase-server | | +1 :green_heart: | checkstyle | 0m 46s | The patch passed checkstyle in hbase-thrift | | +1 :green_heart: | checkstyle | 0m 17s | The patch passed checkstyle in hbase-rest | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 21m 1s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 7m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 50s | The patch does not generate ASF License warnings. | | | | 61m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2630 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 6852841e4b90 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c9156e7891 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-23035) Retain region to the last RegionServer make the failover slower
[ https://issues.apache.org/jira/browse/HBASE-23035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-23035: Component/s: amv2 > Retain region to the last RegionServer make the failover slower > --- > > Key: HBASE-23035 > URL: https://issues.apache.org/jira/browse/HBASE-23035 > Project: HBase > Issue Type: Improvement > Components: amv2 >Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.2.1, 2.1.6 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0, 2.1.7, 2.2.2 > > > Now if one RS crashed, the regions will try to use the old location for the > region deploy. But one RS only have 3 threads to open region by default. If a > RS have hundreds of regions, the failover is very slower. Assign to same RS > may have good locality if the Datanode is deploied on same host. But slower > failover make the availability worse. And the locality is not big deal when > deploy HBase on cloud. > This was introduced by HBASE-18946. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-21565) Delete dead server from dead server list too early leads to concurrent Server Crash Procedures(SCP) for a same server
[
https://issues.apache.org/jira/browse/HBASE-21565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21565:
Component/s: amv2
> Delete dead server from dead server list too early leads to concurrent Server
> Crash Procedures(SCP) for a same server
> -
>
> Key: HBASE-21565
> URL: https://issues.apache.org/jira/browse/HBASE-21565
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 3.0.0-alpha-1
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.2.0
>
> Attachments: HBASE-21565.branch-2.001.patch,
> HBASE-21565.branch-2.002.patch, HBASE-21565.master.001.patch,
> HBASE-21565.master.002.patch, HBASE-21565.master.003.patch,
> HBASE-21565.master.004.patch, HBASE-21565.master.005.patch,
> HBASE-21565.master.006.patch, HBASE-21565.master.007.patch,
> HBASE-21565.master.008.patch, HBASE-21565.master.009.patch,
> HBASE-21565.master.010.patch
>
>
> There are 2 kinds of SCP for a same server will be scheduled during cluster
> restart, one is ZK session timeout, the other one is new server report in
> will cause the stale one do fail over. The only barrier for these 2 kinds of
> SCP is check if the server is in the dead server list.
> {code}
> if (this.deadservers.isDeadServer(serverName)) {
> LOG.warn("Expiration called on {} but crash processing already in
> progress", serverName);
> return false;
> }
> {code}
> But the problem is when master finish initialization, it will delete all
> stale servers from dead server list. Thus when the SCP for ZK session timeout
> come in, the barrier is already removed.
> Here is the logs that how this problem occur.
> {code}
> 2018-12-07,11:42:37,589 INFO
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure: Start pid=9,
> state=RUNNABLE:SERVER_CRASH_START, hasLock=true; ServerCrashProcedure
> server=c4-hadoop-tst-st27.bj,29100,1544153846859, splitWal=true, meta=false
> 2018-12-07,11:42:58,007 INFO
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure: Start pid=444,
> state=RUNNABLE:SERVER_CRASH_START, hasLock=true; ServerCrashProcedure
> server=c4-hadoop-tst-st27.bj,29100,1544153846859, splitWal=true, meta=false
> {code}
> Now we can see two SCP are scheduled for the same server.
> But the first procedure is finished after the second SCP starts.
> {code}
> 2018-12-07,11:43:08,038 INFO
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Finished pid=9,
> state=SUCCESS, hasLock=false; ServerCrashProcedure
> server=c4-hadoop-tst-st27.bj,29100,1544153846859, splitWal=true, meta=false
> in 30.5340sec
> {code}
> Thus it will leads the problem that regions will be assigned twice.
> {code}
> 2018-12-07,12:16:33,039 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: rit=OPEN,
> location=c4-hadoop-tst-st28.bj,29100,1544154149607, table=test_failover,
> region=459b3130b40caf3b8f3e1421766f4089 reported OPEN on
> server=c4-hadoop-tst-st29.bj,29100,1544154149615 but state has otherwise
> {code}
> And here we can see the server is removed from dead server list before the
> second SCP starts.
> {code}
> 2018-12-07,11:42:44,938 DEBUG org.apache.hadoop.hbase.master.DeadServer:
> Removed c4-hadoop-tst-st27.bj,29100,1544153846859 ; numProcessing=3
> {code}
> Thus we should not delete dead server from dead server list immediately.
> Patch to fix this problem will be upload later.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #2690: HBASE-24395 backport HBASE-20589
saintstack commented on a change in pull request #2690:
URL: https://github.com/apache/hbase/pull/2690#discussion_r528271857
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
##
@@ -1251,9 +1251,7 @@ protected void tryRegionServerReport(long
reportStartTime, long reportEndTime)
ClusterStatusProtos.ServerLoad sl = buildServerLoad(reportStartTime,
reportEndTime);
try {
RegionServerReportRequest.Builder request =
RegionServerReportRequest.newBuilder();
- ServerName sn = ServerName.parseVersionedServerName(
-this.serverName.getVersionedBytes());
- request.setServer(ProtobufUtil.toServerName(sn));
+ request.setServer(ProtobufUtil.toServerName(this.serverName));
Review comment:
Good
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
##
@@ -2288,9 +2286,7 @@ public void abort(final String reason, Throwable cause) {
if (rssStub != null && this.serverName != null) {
ReportRSFatalErrorRequest.Builder builder =
ReportRSFatalErrorRequest.newBuilder();
-ServerName sn =
-
ServerName.parseVersionedServerName(this.serverName.getVersionedBytes());
-builder.setServer(ProtobufUtil.toServerName(sn));
+builder.setServer(ProtobufUtil.toServerName(this.serverName));
Review comment:
Good
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
brfrn169 commented on pull request #2630: URL: https://github.com/apache/hbase/pull/2630#issuecomment-731673410 I made a mistake and the compile in the last QA failed.. Just fixed it and pushed the latest patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25284) Check-in "Enable memstore replication..." design
[
https://issues.apache.org/jira/browse/HBASE-25284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236830#comment-17236830
]
Hudson commented on HBASE-25284:
Results for branch branch-2
[build #107 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/107/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/107/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/107/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/107/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/107/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Check-in "Enable memstore replication..." design
>
>
> Key: HBASE-25284
> URL: https://issues.apache.org/jira/browse/HBASE-25284
> Project: HBase
> Issue Type: Sub-task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Add the design doc under dev-support/design-docs
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2688: HBASE-25311 UI throws NPE
Apache-HBase commented on pull request #2688: URL: https://github.com/apache/hbase/pull/2688#issuecomment-731664744 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 27s | master passed | | +1 :green_heart: | checkstyle | 1m 7s | master passed | | +1 :green_heart: | spotbugs | 1m 58s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | checkstyle | 1m 6s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 3s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 38m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2688 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 281f4b11d66a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c9156e7891 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
Apache-HBase commented on pull request #2630: URL: https://github.com/apache/hbase/pull/2630#issuecomment-731662752 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 4s | master passed | | +1 :green_heart: | compile | 2m 20s | master passed | | +1 :green_heart: | shadedjars | 7m 12s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 49s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 52s | root in the patch failed. | | -1 :x: | compile | 0m 27s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 27s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 5m 8s | patch has 42 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 8s | hbase-client in the patch passed. | | -1 :x: | unit | 0m 26s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 4m 36s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 3m 48s | hbase-rest in the patch passed. | | | | 38m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2630 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a8f55497933a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c9156e7891 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk8-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/testReport/ | | Max. process+thread count | 1904 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
Apache-HBase commented on pull request #2630: URL: https://github.com/apache/hbase/pull/2630#issuecomment-731662701 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 4s | master passed | | +1 :green_heart: | compile | 2m 35s | master passed | | +1 :green_heart: | shadedjars | 6m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 25s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 2m 2s | root in the patch failed. | | -1 :x: | compile | 0m 27s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 27s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 4m 51s | patch has 14 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 18s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 19s | hbase-client in the patch passed. | | -1 :x: | unit | 0m 29s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 3m 48s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 3m 6s | hbase-rest in the patch passed. | | | | 38m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2630 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 56aac97fd528 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c9156e7891 | | Default Java | AdoptOpenJDK-11.0.6+10 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/testReport/ | | Max. process+thread count | 2538 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
Apache-HBase commented on pull request #2630: URL: https://github.com/apache/hbase/pull/2630#issuecomment-731661920 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 32s | master passed | | +1 :green_heart: | checkstyle | 2m 42s | master passed | | +1 :green_heart: | spotbugs | 4m 56s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 37s | root in the patch failed. | | +1 :green_heart: | checkstyle | 0m 32s | hbase-client: The patch generated 0 new + 276 unchanged - 1 fixed = 276 total (was 277) | | -0 :warning: | checkstyle | 0m 29s | The patch fails to run checkstyle in hbase-server | | +1 :green_heart: | checkstyle | 0m 37s | The patch passed checkstyle in hbase-thrift | | +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-rest | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | -1 :x: | hadoopcheck | 1m 49s | The patch causes 42 errors with Hadoop v3.1.2. | | -1 :x: | hadoopcheck | 3m 38s | The patch causes 42 errors with Hadoop v3.2.1. | | -1 :x: | hadoopcheck | 5m 27s | The patch causes 42 errors with Hadoop v3.3.0. | | -1 :x: | spotbugs | 0m 25s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 42s | The patch does not generate ASF License warnings. | | | | 28m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2630 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux cf002ae4a2ba 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c9156e7891 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-general-check/output/patch-mvninstall-root.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-general-check/output/buildtool-patch-checkstyle-hbase-server.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-general-check/output/patch-javac-3.1.2.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-general-check/output/patch-javac-3.2.1.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-general-check/output/patch-javac-3.3.0.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/artifact/yetus-general-check/output/patch-spotbugs-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2630/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] cuibo01 closed pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
cuibo01 closed pull request #2689: URL: https://github.com/apache/hbase/pull/2689 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] cuibo01 closed pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
cuibo01 closed pull request #2689: URL: https://github.com/apache/hbase/pull/2689 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] cuibo01 commented on a change in pull request #2688: HBASE-25311 UI throws NPE
cuibo01 commented on a change in pull request #2688:
URL: https://github.com/apache/hbase/pull/2688#discussion_r528264879
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -3621,8 +3621,15 @@ public ReplicationPeerManager
getReplicationPeerManager() {
List replicationLoadSources =
getServerManager().getLoad(serverName).getReplicationLoadSourceList();
for (ReplicationLoadSource replicationLoadSource :
replicationLoadSources) {
-replicationLoadSourceMap.get(replicationLoadSource.getPeerID())
-.add(new Pair<>(serverName, replicationLoadSource));
+List>
replicationLoadSourceList =
Review comment:
yes, but i think log help understand the current code logic
If lambda is necessary, I can update the code
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
brfrn169 commented on pull request #2630: URL: https://github.com/apache/hbase/pull/2630#issuecomment-731659869 I just changed the patch regarding the following: - Add testing for Append and keep the last else branch throwing DoNotRetryIOException - Use DoNotRetryIOException instead of AssertionError - Use ArrayList instead of LinkedList Can you please take a look at this? @Apache9 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2689: URL: https://github.com/apache/hbase/pull/2689#issuecomment-731637535 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 26s | master passed | | +1 :green_heart: | compile | 1m 28s | master passed | | +1 :green_heart: | shadedjars | 7m 20s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | +1 :green_heart: | compile | 1m 29s | the patch passed | | +1 :green_heart: | javac | 1m 29s | the patch passed | | +1 :green_heart: | shadedjars | 7m 20s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 56s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 41s | hbase-replication in the patch passed. | | -1 :x: | unit | 199m 56s | hbase-server in the patch failed. | | | | 232m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2689 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b34b228f0a2c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c9156e7891 | | Default Java | AdoptOpenJDK-11.0.6+10 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/3/testReport/ | | Max. process+thread count | 3366 (vs. ulimit of 3) | | modules | C: hbase-replication hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2678: HBASE-25083 further HBase 1.y releases should have Hadoop 2.10 as a minimum version.
Apache-HBase commented on pull request #2678: URL: https://github.com/apache/hbase/pull/2678#issuecomment-731636614 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 1s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 54s | branch-1 passed | | +1 :green_heart: | compile | 1m 41s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 1m 43s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | refguide | 4m 7s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 3m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 18s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 3m 45s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 46s | the patch passed | | +1 :green_heart: | compile | 1m 43s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 1m 43s | the patch passed | | +1 :green_heart: | compile | 1m 50s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 1m 50s | the patch passed | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | xml | 0m 0s | The patch has 1 ill-formed XML file(s). | | +0 :ok: | refguide | 3m 0s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | -1 :x: | shadedjars | 3m 1s | patch has 13 errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 23s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 2m 13s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 3m 49s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | ||| _ Other Tests _ | | -1 :x: | unit | 46m 34s | root in the patch failed. | | +1 :green_heart: | asflicense | 0m 33s | The patch does not generate ASF License warnings. | | | | 101m 55s | | | Reason | Tests | |---:|:--| | XML | Parsing Error(s): | | | pom.xml | | Failed junit tests | hadoop.hbase.regionserver.TestWideScanner | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2678/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2678 | | Optional Tests | dupname asflicense shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile refguide | | uname | Linux 3a7d1f29d7de 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2678/out/precommit/personality/provided.sh | | git revision | branch-1 / edccfe4 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2678/2/artifact/out/branch-site/book.html | | xml | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2678/2/artifact/out/xml.txt | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2678/2/artifact/out/patch-site/book.html | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2678/2/artifact/out/patch-shadedjars.txt | | unit |
[GitHub] [hbase] Apache-HBase commented on pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2689: URL: https://github.com/apache/hbase/pull/2689#issuecomment-731630451 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 42s | master passed | | +1 :green_heart: | compile | 1m 14s | master passed | | +1 :green_heart: | shadedjars | 6m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 27s | the patch passed | | +1 :green_heart: | compile | 1m 15s | the patch passed | | +1 :green_heart: | javac | 1m 15s | the patch passed | | +1 :green_heart: | shadedjars | 6m 32s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 26s | hbase-replication in the patch passed. | | -1 :x: | unit | 141m 43s | hbase-server in the patch failed. | | | | 170m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2689 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux be2a65962341 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c9156e7891 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/3/testReport/ | | Max. process+thread count | 4847 (vs. ulimit of 3) | | modules | C: hbase-replication hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2689: URL: https://github.com/apache/hbase/pull/2689#issuecomment-731614097 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 40s | master passed | | +1 :green_heart: | checkstyle | 1m 17s | master passed | | +1 :green_heart: | spotbugs | 2m 28s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 23s | the patch passed | | +1 :green_heart: | checkstyle | 1m 15s | the patch passed | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 9s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 41m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2689 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle shellcheck shelldocs | | uname | Linux cbae5f208368 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c9156e7891 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-replication hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/3/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2688: HBASE-25311 UI throws NPE
virajjasani commented on a change in pull request #2688:
URL: https://github.com/apache/hbase/pull/2688#discussion_r528223952
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -3621,8 +3621,15 @@ public ReplicationPeerManager
getReplicationPeerManager() {
List replicationLoadSources =
getServerManager().getLoad(serverName).getReplicationLoadSourceList();
for (ReplicationLoadSource replicationLoadSource :
replicationLoadSources) {
-replicationLoadSourceMap.get(replicationLoadSource.getPeerID())
-.add(new Pair<>(serverName, replicationLoadSource));
+List>
replicationLoadSourceList =
Review comment:
nit: if debug log is not that imp, this could be replaced with few lines
of code using for-each and lambda:
```
replicationLoadSources.forEach(replicationLoadSource -> Optional
.ofNullable(replicationLoadSourceMap.get(replicationLoadSource.getPeerID()))
.ifPresent(pairs -> pairs.add(new Pair<>(serverName,
replicationLoadSource;
```
However, if debug log is imp, feel free to ignore this nit.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -3621,8 +3621,15 @@ public ReplicationPeerManager
getReplicationPeerManager() {
List replicationLoadSources =
getServerManager().getLoad(serverName).getReplicationLoadSourceList();
for (ReplicationLoadSource replicationLoadSource :
replicationLoadSources) {
-replicationLoadSourceMap.get(replicationLoadSource.getPeerID())
-.add(new Pair<>(serverName, replicationLoadSource));
+List>
replicationLoadSourceList =
+ replicationLoadSourceMap.get(replicationLoadSource.getPeerID());
+if (replicationLoadSourceList == null) {
+ LOG.debug(replicationLoadSource.getPeerID() + " does not exist, but
it exists "
++ "in znode(/hbase/replication/rs). when the rs restarts, peerId
is deleted, so "
++ "we just need to ignore it");
Review comment:
Please use placeholder in log with argument: `LOG.debug("{} does not
exist,.", replicationLoadSource.getPeerID());`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25280) [meta replicas] ArrayIndexOutOfBoundsException in ZKConnectionRegistry
[
https://issues.apache.org/jira/browse/HBASE-25280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236748#comment-17236748
]
Hudson commented on HBASE-25280:
Results for branch branch-2
[build #106 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [meta replicas] ArrayIndexOutOfBoundsException in ZKConnectionRegistry
> --
>
> Key: HBASE-25280
> URL: https://issues.apache.org/jira/browse/HBASE-25280
> Project: HBase
> Issue Type: Bug
> Components: meta replicas
>Affects Versions: HBASE-18070
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: HBASE-18070, HBASE-18070.branch-2
>
>
> ITBLL Testing HBASE-18070 feature, [~huaxiangsun] found this:
> {code:java}
> 2020-11-12 19:48:12,358 ERROR org.apache.hadoop.hbase.util.FutureUtils:
> Unexpected error caught when processing
> CompletableFuturejava.lang.ArrayIndexOutOfBoundsException: Index 3 out of
> bounds for length 3 at
> org.apache.hadoop.hbase.client.ZKConnectionRegistry.lambda$getMetaRegionLocation$2(ZKConnectionRegistry.java:180)
> at
> org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68)
> at
> java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
> at
> java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
> at
> java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
> at
> java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
> at
> org.apache.hadoop.hbase.client.ZKConnectionRegistry.lambda$getAndConvert$0(ZKConnectionRegistry.java:78)
> at
> org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68)
> at
> java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
> at
> java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
> at
> java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
> at
> java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:174)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:342)
> at java.base/java.lang.Thread.run(Thread.java:834) {code}
> The code has been this way a long time but his running with four replicas
> seems to have revealed a race exposed by using replicaid as index.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25055) Add ReplicationSource for meta WALs; add enable/disable when hbase:meta assigned to RS
[
https://issues.apache.org/jira/browse/HBASE-25055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236746#comment-17236746
]
Hudson commented on HBASE-25055:
Results for branch branch-2
[build #106 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add ReplicationSource for meta WALs; add enable/disable when hbase:meta
> assigned to RS
> --
>
> Key: HBASE-25055
> URL: https://issues.apache.org/jira/browse/HBASE-25055
> Project: HBase
> Issue Type: Sub-task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Add ReplicationSource that feeds on hbase:meta WAL files. Add enabling this
> source when hbase:meta is opened and hbase:meta region replicas are
> configured ON. Disable the source when the hbase:meta Region moves away.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25151) warmupRegion frustrates registering WALs on the catalog replicationsource
[
https://issues.apache.org/jira/browse/HBASE-25151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236747#comment-17236747
]
Hudson commented on HBASE-25151:
Results for branch branch-2
[build #106 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> warmupRegion frustrates registering WALs on the catalog replicationsource
> -
>
> Key: HBASE-25151
> URL: https://issues.apache.org/jira/browse/HBASE-25151
> Project: HBase
> Issue Type: Sub-task
> Components: read replicas
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Writing a test for HBASE-25145
> I noticed that the warmupRegion call triggered by the Master on Region move
> mess-up registering hbase:meta ReplicationSource. Add accommodation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25291) Document how to enable the meta replica load balance mode for the client and clean up around hbase:meta read replicas
[
https://issues.apache.org/jira/browse/HBASE-25291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236750#comment-17236750
]
Hudson commented on HBASE-25291:
Results for branch branch-2
[build #106 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Document how to enable the meta replica load balance mode for the client and
> clean up around hbase:meta read replicas
> -
>
> Key: HBASE-25291
> URL: https://issues.apache.org/jira/browse/HBASE-25291
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.4.0
>Reporter: Huaxiang Sun
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Need to document how to enable meta replica Load Balance mode for clients.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25126) Add load balance logic in hbase-client to distribute read load over meta replica regions.
[
https://issues.apache.org/jira/browse/HBASE-25126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236749#comment-17236749
]
Hudson commented on HBASE-25126:
Results for branch branch-2
[build #106 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add load balance logic in hbase-client to distribute read load over meta
> replica regions.
> -
>
> Key: HBASE-25126
> URL: https://issues.apache.org/jira/browse/HBASE-25126
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha-1
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25068) Pass WALFactory to Replication so it knows of all WALProviders, not just default/user-space
[
https://issues.apache.org/jira/browse/HBASE-25068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236745#comment-17236745
]
Hudson commented on HBASE-25068:
Results for branch branch-2
[build #106 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/106/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Pass WALFactory to Replication so it knows of all WALProviders, not just
> default/user-space
> ---
>
> Key: HBASE-25068
> URL: https://issues.apache.org/jira/browse/HBASE-25068
> Project: HBase
> Issue Type: Sub-task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
> Attachments:
> 0001-HBASE-25068-Pass-WALFactory-to-Replication-so-it-kno.patch.2,
> 0001-HBASE-25068-Pass-WALFactory-to-Replication-so-it-kno.patch.master
>
>
> Small change that passes all WALProviders to ReplicationService rather than
> just the default/user-space WALProvider. It does this using the WALFactory
> vessel since it holds all Providers. This change is to be exploited by
> adjacent sub-task HBASE-25055 in follow-on. This sub-task also exists to make
> the HBASE-25055 patch smaller and more focused, easier to review.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2511: HBASE-24960 reduce invalid subprocedure task
Apache-HBase commented on pull request #2511: URL: https://github.com/apache/hbase/pull/2511#issuecomment-731608764 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 3s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 51s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 7m 28s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 33s | the patch passed | | +1 :green_heart: | compile | 1m 12s | the patch passed | | +1 :green_heart: | javac | 1m 12s | the patch passed | | +1 :green_heart: | shadedjars | 7m 19s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 213m 1s | hbase-server in the patch failed. | | | | 243m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2511 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 00d6f9b0b3f1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-11.0.6+10 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/4/testReport/ | | Max. process+thread count | 3566 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2689: URL: https://github.com/apache/hbase/pull/2689#issuecomment-731607935 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 45s | master passed | | +1 :green_heart: | compile | 1m 30s | master passed | | +1 :green_heart: | shadedjars | 7m 18s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 35s | the patch passed | | +1 :green_heart: | compile | 1m 30s | the patch passed | | +1 :green_heart: | javac | 1m 30s | the patch passed | | +1 :green_heart: | shadedjars | 7m 19s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 56s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 42s | hbase-replication in the patch passed. | | +1 :green_heart: | unit | 197m 10s | hbase-server in the patch passed. | | | | 230m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2689 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8b369715e309 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/2/testReport/ | | Max. process+thread count | 3509 (vs. ulimit of 3) | | modules | C: hbase-replication hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-18070) Enable memstore replication for meta replica
[ https://issues.apache.org/jira/browse/HBASE-18070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-18070. --- Fix Version/s: (was: HBASE-18070.branch-2) (was: HBASE-18070) Resolution: Fixed Resolving. All subtasks done. Thanks to the army that helped dev and land this feature. > Enable memstore replication for meta replica > > > Key: HBASE-18070 > URL: https://issues.apache.org/jira/browse/HBASE-18070 > Project: HBase > Issue Type: New Feature >Reporter: Hua Xiang >Assignee: Huaxiang Sun >Priority: Critical > Labels: read-replicas > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Based on the current doc, memstore replication is not enabled for meta > replica. Memstore replication will be a good improvement for meta replica. > Create jira to track this effort (feasibility, design, implementation, etc). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-18070) Enable memstore replication for meta replica
[ https://issues.apache.org/jira/browse/HBASE-18070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236729#comment-17236729 ] Michael Stack commented on HBASE-18070: --- The last subtask has just been committed – updating the checked in design on master and branch-2. Last night's nightly on master failed but all the backup tests passed [https://ci-hadoop.apache.org/view/HBase/job/HBase/job/HBase%20Nightly/job/master/133/testReport/org.apache.hadoop.hbase.backup/] So, I'll let go of my concern that the merge of this feature to master broke backup. Last nights branch-2 #106 is not yet complete but it looks like a failure in the jdk11 runs but unrelated. [Test Result|https://ci-hadoop.apache.org/view/HBase/job/HBase/job/HBase%20Nightly/job/branch-2/106/testReport/] (1 failure / +1) * [org.apache.hadoop.hbase.master.TestSplitRegionWhileRSCrash.|https://ci-hadoop.apache.org/view/HBase/job/HBase/job/HBase%20Nightly/job/branch-2/106/testReport/junit/org.apache.hadoop.hbase.master/TestSplitRegionWhileRSCrash//] Let me queue up a new nightly now. I think we are done here. Let me resolve. > Enable memstore replication for meta replica > > > Key: HBASE-18070 > URL: https://issues.apache.org/jira/browse/HBASE-18070 > Project: HBase > Issue Type: New Feature >Reporter: Hua Xiang >Assignee: Huaxiang Sun >Priority: Critical > Labels: read-replicas > Fix For: 3.0.0-alpha-1, 2.4.0, HBASE-18070, HBASE-18070.branch-2 > > > Based on the current doc, memstore replication is not enabled for meta > replica. Memstore replication will be a good improvement for meta replica. > Create jira to track this effort (feasibility, design, implementation, etc). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25284) Check-in "Enable memstore replication..." design
[ https://issues.apache.org/jira/browse/HBASE-25284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25284. --- Hadoop Flags: Reviewed Resolution: Fixed Re-resolving after updating the already checked-in HBASE-18070 design doc. Applied to branch-2+. Resolving. > Check-in "Enable memstore replication..." design > > > Key: HBASE-25284 > URL: https://issues.apache.org/jira/browse/HBASE-25284 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Add the design doc under dev-support/design-docs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25284) Check-in "Enable memstore replication..." design
[ https://issues.apache.org/jira/browse/HBASE-25284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236725#comment-17236725 ] Michael Stack commented on HBASE-25284: --- I updated again to pick up the new fixes and then pushed. Thanks for the review [~zhangduo] > Check-in "Enable memstore replication..." design > > > Key: HBASE-25284 > URL: https://issues.apache.org/jira/browse/HBASE-25284 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Add the design doc under dev-support/design-docs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #2680: HBASE-25284 Check-in "Enable memstore replication..." design; ADDENDUM
saintstack merged pull request #2680: URL: https://github.com/apache/hbase/pull/2680 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2680: HBASE-25284 Check-in "Enable memstore replication..." design; ADDENDUM
Apache-HBase commented on pull request #2680: URL: https://github.com/apache/hbase/pull/2680#issuecomment-731602828 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 14s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2680/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2680 | | Optional Tests | dupname asflicense | | uname | Linux e5346d2a9b87 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Max. process+thread count | 42 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2680/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2680: HBASE-25284 Check-in "Enable memstore replication..." design; ADDENDUM
Apache-HBase commented on pull request #2680: URL: https://github.com/apache/hbase/pull/2680#issuecomment-731602788 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2680/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2680 | | Optional Tests | | | uname | Linux f9a2ab85e9a8 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Max. process+thread count | 48 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2680/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2680: HBASE-25284 Check-in "Enable memstore replication..." design; ADDENDUM
Apache-HBase commented on pull request #2680: URL: https://github.com/apache/hbase/pull/2680#issuecomment-731602724 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 8s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 1m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2680/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2680 | | Optional Tests | | | uname | Linux bd6a689b5c18 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Max. process+thread count | 48 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2680/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on a change in pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
brfrn169 commented on a change in pull request #2630:
URL: https://github.com/apache/hbase/pull/2630#discussion_r528213068
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CheckAndMutate.java
##
@@ -60,7 +53,7 @@
*/
@InterfaceAudience.Public
@InterfaceStability.Evolving
-public final class CheckAndMutate extends Mutation {
+public final class CheckAndMutate implements Row {
Review comment:
We haven't released the CheckAndMutate class yet. It's introduced in
2.4.0. We need to commit this change before releasing 2.4.0.
##
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java
##
@@ -459,9 +459,10 @@ default CheckAndMutateResult checkAndMutate(CheckAndMutate
checkAndMutate) throw
* {@link Put} and {@link Delete} are supported.
*
* @param rm object that specifies the set of mutations to perform atomically
+ * @return results of Increment/Append operations
* @throws IOException
Review comment:
Sure. I will do that.
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
##
@@ -3657,9 +3657,10 @@ public static CheckAndMutate
toCheckAndMutate(ClientProtos.Condition condition,
return builder.build((Put) m);
} else if (m instanceof Delete) {
return builder.build((Delete) m);
+} else if (m instanceof Increment) {
+ return builder.build((Increment) m);
} else {
- throw new DoNotRetryIOException("Unsupported mutate type: " +
mutations.get(0)
-.getClass().getSimpleName().toUpperCase());
+ return builder.build((Append) m);
Review comment:
Sure. I will add testing for Append and keep the last else branch
throwing DoNotRetryIOException.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
##
@@ -620,8 +622,22 @@ private CheckAndMutateResult checkAndMutate(HRegion
region, List 0) {
-ResultOrException roe = actionResult.getResultOrException(0);
-if (roe.hasResult()) {
- Result r = ProtobufUtil.toResult(roe.getResult(), cells);
- if (!r.isEmpty()) {
-result = r;
- }
-}
- }
- responseValue = new
CheckAndMutateResult(actionResult.getProcessed(), result);
+ results.add(regionName, index, getCheckAndMutateResult(actionResult,
cells));
} else {
- responseValue = actionResult.getProcessed() ?
-ProtobufUtil.EMPTY_RESULT_EXISTS_TRUE :
-ProtobufUtil.EMPTY_RESULT_EXISTS_FALSE;
+ results.add(regionName, index, getMutateRowResult(actionResult,
cells));
}
-results.add(regionName, index, responseValue);
continue;
}
if (actions.hasCondition()) {
-Result result = null;
-if (actionResult.getResultOrExceptionCount() > 0) {
- ResultOrException roe = actionResult.getResultOrException(0);
- Result r = ProtobufUtil.toResult(roe.getResult(), cells);
- if (!r.isEmpty()) {
-result = r;
- }
-}
-responseValue = new CheckAndMutateResult(actionResult.getProcessed(),
result);
-results.add(regionName, 0, responseValue);
+results.add(regionName, 0, getCheckAndMutateResult(actionResult,
cells));
} else {
+if (actionResult.hasProcessed()) {
Review comment:
The case where actions.hasCondition() is false and
actionResult.hasProcessed() is true is the case for a result of mutateRow().
After this change, mutateRow() can return results of Increment/Append
operations. That's why we need to add this logic here.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
##
@@ -972,11 +1002,44 @@ private void doBatchOp(final RegionActionResult.Builder
builder, final HRegion r
OperationStatus[] codes = region.batchMutate(mArray, atomic,
HConstants.NO_NONCE,
HConstants.NO_NONCE);
+ if (atomic) {
+LinkedList resultOrExceptions = new LinkedList<>();
Review comment:
The reason why I use LinkedList here is that I need to add a element at
the beginning of the list as follows:
https://github.com/apache/hbase/pull/2630/files#diff-e4052cd5a1f1c93375e3fbc931dc4df220deebc78c0d53fd2435b47fd04cd807R1019-R1020
https://github.com/apache/hbase/pull/2630/files#diff-e4052cd5a1f1c93375e3fbc931dc4df220deebc78c0d53fd2435b47fd04cd807R1031
Still ArrayList is faster in that case?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2680: HBASE-25284 Check-in "Enable memstore replication..." design; ADDENDUM
saintstack commented on pull request #2680: URL: https://github.com/apache/hbase/pull/2680#issuecomment-731602570 Updated the PR w/ latest edits. Let me merge. Thanks for the reviews lads. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2511: HBASE-24960 reduce invalid subprocedure task
Apache-HBase commented on pull request #2511: URL: https://github.com/apache/hbase/pull/2511#issuecomment-731600915 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 26s | master passed | | +1 :green_heart: | compile | 0m 58s | master passed | | +1 :green_heart: | shadedjars | 6m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | shadedjars | 6m 32s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 155m 23s | hbase-server in the patch failed. | | | | 181m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2511 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0781802c11a8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/4/testReport/ | | Max. process+thread count | 4597 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2689: URL: https://github.com/apache/hbase/pull/2689#issuecomment-731600233 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 29s | master passed | | +1 :green_heart: | compile | 1m 15s | master passed | | +1 :green_heart: | shadedjars | 6m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 27s | the patch passed | | +1 :green_heart: | compile | 1m 15s | the patch passed | | +1 :green_heart: | javac | 1m 15s | the patch passed | | +1 :green_heart: | shadedjars | 6m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 26s | hbase-replication in the patch passed. | | +1 :green_heart: | unit | 140m 39s | hbase-server in the patch passed. | | | | 168m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2689 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 69609f94e16a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/2/testReport/ | | Max. process+thread count | 4651 (vs. ulimit of 3) | | modules | C: hbase-replication hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
Apache9 commented on a change in pull request #2630:
URL: https://github.com/apache/hbase/pull/2630#discussion_r528207034
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ResponseConverter.java
##
@@ -158,52 +156,30 @@ public static SingleResponse getResult(final
ClientProtos.MutateRequest request,
// If there is an exception from the server, the exception is set at
// the RegionActionResult level, which has been handled above.
if (actions.hasCondition()) {
- Result result = null;
- if (actionResult.getResultOrExceptionCount() > 0) {
-ResultOrException roe = actionResult.getResultOrException(0);
-if (roe.hasResult()) {
- Result r = ProtobufUtil.toResult(roe.getResult(), cells);
- if (!r.isEmpty()) {
-result = r;
- }
-}
- }
- responseValue = new
CheckAndMutateResult(actionResult.getProcessed(), result);
+ results.add(regionName, index, getCheckAndMutateResult(actionResult,
cells));
} else {
- responseValue = actionResult.getProcessed() ?
-ProtobufUtil.EMPTY_RESULT_EXISTS_TRUE :
-ProtobufUtil.EMPTY_RESULT_EXISTS_FALSE;
+ results.add(regionName, index, getMutateRowResult(actionResult,
cells));
}
-results.add(regionName, index, responseValue);
continue;
}
if (actions.hasCondition()) {
-Result result = null;
-if (actionResult.getResultOrExceptionCount() > 0) {
- ResultOrException roe = actionResult.getResultOrException(0);
- Result r = ProtobufUtil.toResult(roe.getResult(), cells);
- if (!r.isEmpty()) {
-result = r;
- }
-}
-responseValue = new CheckAndMutateResult(actionResult.getProcessed(),
result);
-results.add(regionName, 0, responseValue);
+results.add(regionName, 0, getCheckAndMutateResult(actionResult,
cells));
} else {
+if (actionResult.hasProcessed()) {
Review comment:
Mind explaing a bit about this change here? Seems the logic is changed?
##
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java
##
@@ -459,9 +459,10 @@ default CheckAndMutateResult checkAndMutate(CheckAndMutate
checkAndMutate) throw
* {@link Put} and {@link Delete} are supported.
*
* @param rm object that specifies the set of mutations to perform atomically
+ * @return results of Increment/Append operations
* @throws IOException
Review comment:
nits: could remove the empty throws so we could fix on checkstyle
warning.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
##
@@ -972,11 +1003,41 @@ private void doBatchOp(final RegionActionResult.Builder
builder, final HRegion r
OperationStatus[] codes = region.batchMutate(mArray, atomic,
HConstants.NO_NONCE,
HConstants.NO_NONCE);
+ if (atomic) {
+LinkedList resultOrExceptions = new LinkedList<>();
+List results = new ArrayList<>();
+for (i = 0; i < codes.length; i++) {
+ if (codes[i].getResult() != null) {
+results.add(codes[i].getResult());
+ }
+ if (i != 0) {
+resultOrExceptions.add(getResultOrException(
+ ClientProtos.Result.getDefaultInstance(), i));
+ }
+}
+
+if (results.isEmpty()) {
+ resultOrExceptions.addFirst(getResultOrException(
+ClientProtos.Result.getDefaultInstance(), 0));
+} else {
+ // Set the result of the Increment/Append operations to the first
element of the
+ // ResultOrException list
+ Result result = Result.create(results.stream()
+.filter(r -> r.rawCells() != null)
+.flatMap(r -> Arrays.stream(r.rawCells()))
+.collect(Collectors.toList()));
+
resultOrExceptions.addFirst(getResultOrException(ProtobufUtil.toResult(result),
0));
+}
+
+builder.addAllResultOrException(resultOrExceptions);
+return;
+ }
+
for (i = 0; i < codes.length; i++) {
Mutation currentMutation = mArray[i];
ClientProtos.Action currentAction =
mutationActionMap.get(currentMutation);
-int index = currentAction.hasIndex() || !atomic ?
currentAction.getIndex() : i;
-Exception e = null;
+int index = currentAction.hasIndex() ? currentAction.getIndex() : i;
Review comment:
OK, just saw that there is a return above.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
##
@@ -972,11 +1002,44 @@ private void doBatchOp(final RegionActionResult.Builder
builder, final HRegion r
OperationStatus[] codes = region.batchMutate(mArray, atomic,
HConstants.NO_NONCE,
[GitHub] [hbase] Apache9 commented on a change in pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
Apache9 commented on a change in pull request #2630: URL: https://github.com/apache/hbase/pull/2630#discussion_r528205873 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java ## @@ -620,8 +622,22 @@ private CheckAndMutateResult checkAndMutate(HRegion region, List
[jira] [Commented] (HBASE-25313) [branch-1] Fix the broken pre-commit
[
https://issues.apache.org/jira/browse/HBASE-25313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236683#comment-17236683
]
Hudson commented on HBASE-25313:
Results for branch branch-1
[build #56 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/56/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/56//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/56//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/56//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> [branch-1] Fix the broken pre-commit
>
>
> Key: HBASE-25313
> URL: https://issues.apache.org/jira/browse/HBASE-25313
> Project: HBase
> Issue Type: Bug
> Components: jenkins
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Fix For: 1.7.0
>
>
> {code}
> Archive JUnit-formatted test results
> [2020-11-20T02:58:50.501Z] Recording test results
> [2020-11-20T02:58:50.641Z] No test report files were found. Configuration
> error?
> {code}
> branch-1's pre-commit keeps failing with error above, we need to fix it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on a change in pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
Apache9 commented on a change in pull request #2630:
URL: https://github.com/apache/hbase/pull/2630#discussion_r528205625
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/CheckAndMutate.java
##
@@ -60,7 +53,7 @@
*/
@InterfaceAudience.Public
@InterfaceStability.Evolving
-public final class CheckAndMutate extends Mutation {
+public final class CheckAndMutate implements Row {
Review comment:
OK, I'm fine with it, but since this is IA.Public, we need to make sure
that we haven't release this class out yet? There will be a 2.4.0 release soon.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #2630: HBASE-25242 Add Increment/Append support to RowMutations
Apache9 commented on a change in pull request #2630:
URL: https://github.com/apache/hbase/pull/2630#discussion_r528205391
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
##
@@ -3657,9 +3657,10 @@ public static CheckAndMutate
toCheckAndMutate(ClientProtos.Condition condition,
return builder.build((Put) m);
} else if (m instanceof Delete) {
return builder.build((Delete) m);
+} else if (m instanceof Increment) {
+ return builder.build((Increment) m);
} else {
- throw new DoNotRetryIOException("Unsupported mutate type: " +
mutations.get(0)
-.getClass().getSimpleName().toUpperCase());
+ return builder.build((Append) m);
Review comment:
This is for safety. Maybe in the future someone other than us two will
implement a new type of Mutation and we will throw a DoNotRetryIOException
directly here which will be easy to know what's wrong here, otherwise the
behavior will be confusing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2689: URL: https://github.com/apache/hbase/pull/2689#issuecomment-731585302 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 5s | master passed | | +1 :green_heart: | checkstyle | 1m 25s | master passed | | +1 :green_heart: | spotbugs | 2m 38s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 45s | the patch passed | | +1 :green_heart: | checkstyle | 1m 21s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 52s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 44m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2689 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux b8d595b1c16b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-replication hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2511: HBASE-24960 reduce invalid subprocedure task
Apache-HBase commented on pull request #2511: URL: https://github.com/apache/hbase/pull/2511#issuecomment-731584179 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | checkstyle | 1m 14s | master passed | | +1 :green_heart: | spotbugs | 2m 9s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 42s | the patch passed | | -0 :warning: | checkstyle | 1m 12s | hbase-server: The patch generated 17 new + 32 unchanged - 0 fixed = 49 total (was 32) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 56s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 42m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2511 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 4a98a9901c2e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2511: HBASE-24960 reduce invalid subprocedure task
Apache-HBase commented on pull request #2511: URL: https://github.com/apache/hbase/pull/2511#issuecomment-731578669 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 14s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 53s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 7m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 39s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | shadedjars | 7m 23s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 253m 46s | hbase-server in the patch failed. | | | | 285m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2511 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fff429c4c6bf 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-11.0.6+10 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/3/testReport/ | | Max. process+thread count | 3139 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2688: HBASE-25311 UI throws NPE
Apache-HBase commented on pull request #2688: URL: https://github.com/apache/hbase/pull/2688#issuecomment-731577441 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 24s | master passed | | +1 :green_heart: | compile | 1m 29s | master passed | | +1 :green_heart: | shadedjars | 9m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 49s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 7s | the patch passed | | +1 :green_heart: | compile | 1m 23s | the patch passed | | +1 :green_heart: | javac | 1m 23s | the patch passed | | +1 :green_heart: | shadedjars | 9m 1s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 54s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 136m 27s | hbase-server in the patch passed. | | | | 172m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2688 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9aba5c912b23 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/2/testReport/ | | Max. process+thread count | 3828 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2688: HBASE-25311 UI throws NPE
Apache-HBase commented on pull request #2688: URL: https://github.com/apache/hbase/pull/2688#issuecomment-731576832 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 24s | master passed | | +1 :green_heart: | compile | 0m 58s | master passed | | +1 :green_heart: | shadedjars | 6m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 31s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | shadedjars | 6m 31s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 142m 6s | hbase-server in the patch passed. | | | | 167m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2688 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 94550e5c9bc4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/2/testReport/ | | Max. process+thread count | 5433 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2689: URL: https://github.com/apache/hbase/pull/2689#issuecomment-731572210 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 5s | master passed | | +1 :green_heart: | compile | 1m 42s | master passed | | +1 :green_heart: | shadedjars | 7m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 6s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 39s | the patch passed | | +1 :green_heart: | compile | 1m 51s | the patch passed | | +1 :green_heart: | javac | 1m 51s | the patch passed | | +1 :green_heart: | shadedjars | 8m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 1s | hbase-replication in the patch passed. | | -1 :x: | unit | 212m 3s | hbase-server in the patch failed. | | | | 249m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2689 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4a2d234db343 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-11.0.6+10 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/1/testReport/ | | Max. process+thread count | 2728 (vs. ulimit of 3) | | modules | C: hbase-replication hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2690: HBASE-24395 backport HBASE-20589
Apache-HBase commented on pull request #2690: URL: https://github.com/apache/hbase/pull/2690#issuecomment-731571751 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 47s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 2m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 8m 16s | branch-1 passed | | +1 :green_heart: | compile | 1m 1s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 1m 10s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 2m 15s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 1m 7s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 2m 43s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 25s | branch-1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 1s | the patch passed | | +1 :green_heart: | compile | 0m 59s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 59s | the patch passed | | +1 :green_heart: | compile | 1m 9s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 1m 9s | the patch passed | | -1 :x: | checkstyle | 1m 32s | hbase-server: The patch generated 1 new + 142 unchanged - 0 fixed = 143 total (was 142) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 49s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 49s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 53s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 1m 7s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 4m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 42s | hbase-client in the patch passed. | | -1 :x: | unit | 99m 41s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 54s | The patch does not generate ASF License warnings. | | | | 159m 48s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.TestMetaAssignmentWithStopMaster | | | hadoop.hbase.TestClusterPortAssignment | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2690/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2690 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 49369da16469 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2690/out/precommit/personality/provided.sh | | git revision | branch-1 / edccfe4 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2690/1/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2690/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2690/1/testReport/ | | Max. process+thread count | 4347 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . |
[jira] [Updated] (HBASE-25281) Bulkload split hfile too many times due to unreasonable split point
[
https://issues.apache.org/jira/browse/HBASE-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
niuyulin updated HBASE-25281:
-
Description:
[https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java#L688]
if hfile span multi regions, for example A,B,C,D,E,F(the start key of these
regions are in ascending order), we should use region C endkey to split, not
region A. In this way, we can get equal .top and .bottom hfiles, reduce time
complexity of split from O( n ) to O(logn),decrease invoke of {{bulkLoad}} rpc
to regionserver ,and also avoid write amplification during {{copyHFileHalf}}
was:
[https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java#L688]
if hfile span multi regions, for example A,B,C,D,E,F(the start key of these
regions are in ascending order), we should use region C endkey to split, not
region A. In this way, we can get equal .top and .bottom hfiles, reduce time
complexity of split from O(n) to O(logn),decrease invoke of {{bulkLoad}} rpc to
regionserver ,and also avoid write amplification during {{copyHFileHalf}}
> Bulkload split hfile too many times due to unreasonable split point
> ---
>
> Key: HBASE-25281
> URL: https://issues.apache.org/jira/browse/HBASE-25281
> Project: HBase
> Issue Type: Improvement
> Components: tooling
>Reporter: niuyulin
>Assignee: niuyulin
>Priority: Minor
> Attachments: bulkload.png
>
>
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java#L688]
> if hfile span multi regions, for example A,B,C,D,E,F(the start key of these
> regions are in ascending order), we should use region C endkey to split, not
> region A. In this way, we can get equal .top and .bottom hfiles, reduce time
> complexity of split from O( n ) to O(logn),decrease invoke of {{bulkLoad}}
> rpc to regionserver ,and also avoid write amplification during
> {{copyHFileHalf}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2511: HBASE-24960 reduce invalid subprocedure task
Apache-HBase commented on pull request #2511: URL: https://github.com/apache/hbase/pull/2511#issuecomment-731567556 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 55s | master passed | | +1 :green_heart: | compile | 0m 55s | master passed | | +1 :green_heart: | shadedjars | 6m 50s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 31s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | shadedjars | 6m 47s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 159m 28s | hbase-server in the patch failed. | | | | 186m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2511 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9ac460f9acd0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/3/testReport/ | | Max. process+thread count | 4271 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25281) Bulkload split hfile too many times due to unreasonable split point
[
https://issues.apache.org/jira/browse/HBASE-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
niuyulin updated HBASE-25281:
-
Description:
[https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java#L688]
if hfile span multi regions, for example A,B,C,D,E,F(the start key of these
regions are in ascending order), we should use region C endkey to split, not
region A. In this way, we can get equal .top and .bottom hfiles, reduce time
complexity of split from O(n) to O(logn),decrease invoke of {{bulkLoad}} rpc to
regionserver ,and also avoid write amplification during {{copyHFileHalf}}
was:
[https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java#L688]
the split point is not in middle, If the hfile span N regions, we need split
N-1 times now
> Bulkload split hfile too many times due to unreasonable split point
> ---
>
> Key: HBASE-25281
> URL: https://issues.apache.org/jira/browse/HBASE-25281
> Project: HBase
> Issue Type: Improvement
> Components: tooling
>Reporter: niuyulin
>Assignee: niuyulin
>Priority: Minor
> Attachments: bulkload.png
>
>
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java#L688]
> if hfile span multi regions, for example A,B,C,D,E,F(the start key of these
> regions are in ascending order), we should use region C endkey to split, not
> region A. In this way, we can get equal .top and .bottom hfiles, reduce time
> complexity of split from O(n) to O(logn),decrease invoke of {{bulkLoad}} rpc
> to regionserver ,and also avoid write amplification during {{copyHFileHalf}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25281) Bulkload split hfile too many times due to unreasonable split point
[
https://issues.apache.org/jira/browse/HBASE-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
niuyulin updated HBASE-25281:
-
Description:
[https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java#L688]
if hfile span multi regions, for example A,B,C,D,E,F(the start key of these
regions are in ascending order), we should use region C endkey to split, not
region A. In this way, we can get equal .top and .bottom hfiles, reduce time
complexity of split from O(n) to O(logn),decrease invoke of {{bulkLoad}} rpc to
regionserver ,and also avoid write amplification during {{copyHFileHalf}}
was:
[https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java#L688]
if hfile span multi regions, for example A,B,C,D,E,F(the start key of these
regions are in ascending order), we should use region C endkey to split, not
region A. In this way, we can get equal .top and .bottom hfiles, reduce time
complexity of split from O(n) to O(logn),decrease invoke of {{bulkLoad}} rpc to
regionserver ,and also avoid write amplification during {{copyHFileHalf}}
> Bulkload split hfile too many times due to unreasonable split point
> ---
>
> Key: HBASE-25281
> URL: https://issues.apache.org/jira/browse/HBASE-25281
> Project: HBase
> Issue Type: Improvement
> Components: tooling
>Reporter: niuyulin
>Assignee: niuyulin
>Priority: Minor
> Attachments: bulkload.png
>
>
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java#L688]
> if hfile span multi regions, for example A,B,C,D,E,F(the start key of these
> regions are in ascending order), we should use region C endkey to split, not
> region A. In this way, we can get equal .top and .bottom hfiles, reduce time
> complexity of split from O(n) to O(logn),decrease invoke of {{bulkLoad}} rpc
> to regionserver ,and also avoid write amplification during {{copyHFileHalf}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2689: URL: https://github.com/apache/hbase/pull/2689#issuecomment-731566085 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 38s | master passed | | +1 :green_heart: | compile | 1m 31s | master passed | | +1 :green_heart: | shadedjars | 8m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 6s | the patch passed | | +1 :green_heart: | compile | 1m 45s | the patch passed | | +1 :green_heart: | javac | 1m 45s | the patch passed | | +1 :green_heart: | shadedjars | 8m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 31s | hbase-replication in the patch passed. | | -1 :x: | unit | 154m 55s | hbase-server in the patch failed. | | | | 191m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2689 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 43cf8c119d01 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/1/testReport/ | | Max. process+thread count | 3828 (vs. ulimit of 3) | | modules | C: hbase-replication hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25213) Should request Compaction when bulkLoadHFiles is done
[ https://issues.apache.org/jira/browse/HBASE-25213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236642#comment-17236642 ] niuyulin commented on HBASE-25213: -- of course, I have submitted another PR for branch-2 [~zghao] > Should request Compaction when bulkLoadHFiles is done > - > > Key: HBASE-25213 > URL: https://issues.apache.org/jira/browse/HBASE-25213 > Project: HBase > Issue Type: Improvement >Reporter: niuyulin >Assignee: niuyulin >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] nyl3532016 commented on pull request #2684: Backport HBASE-25213 Should request Compaction after bulkLoadHFiles is done
nyl3532016 commented on pull request #2684: URL: https://github.com/apache/hbase/pull/2684#issuecomment-731565809 test failed not related. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #2667: HBASE-25281 Bulkload split hfile too many times due to unreasonable split point
wchevreuil commented on a change in pull request #2667:
URL: https://github.com/apache/hbase/pull/2667#discussion_r528182565
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java
##
@@ -647,51 +684,24 @@ private String getUniqueName() {
return null;
}
if (Bytes.compareTo(first.get(), last.get()) > 0) {
- throw new IllegalArgumentException("Invalid range: " +
Bytes.toStringBinary(first.get()) +
-" > " + Bytes.toStringBinary(last.get()));
-}
-int idx =
- Collections.binarySearch(startEndKeys, Pair.newPair(first.get(),
HConstants.EMPTY_END_ROW),
-(p1, p2) -> Bytes.compareTo(p1.getFirst(), p2.getFirst()));
-if (idx < 0) {
- // not on boundary, returns -(insertion index). Calculate region it
- // would be in.
- idx = -(idx + 1) - 1;
-}
-int indexForCallable = idx;
-
-/*
- * we can consider there is a region hole in following conditions. 1) if
idx < 0,then first
- * region info is lost. 2) if the endkey of a region is not equal to the
startkey of the next
- * region. 3) if the endkey of the last region is not empty.
- */
-if (indexForCallable < 0) {
- throw new IOException("The first region info for table " + tableName +
-" can't be found in hbase:meta.Please use hbck tool to fix it first.");
-} else if ((indexForCallable == startEndKeys.size() - 1) &&
- !Bytes.equals(startEndKeys.get(indexForCallable).getSecond(),
HConstants.EMPTY_BYTE_ARRAY)) {
- throw new IOException("The last region info for table " + tableName +
-" can't be found in hbase:meta.Please use hbck tool to fix it first.");
-} else if (indexForCallable + 1 < startEndKeys.size() &&
- !(Bytes.compareTo(startEndKeys.get(indexForCallable).getSecond(),
-startEndKeys.get(indexForCallable + 1).getFirst()) == 0)) {
- throw new IOException("The endkey of one region for table " + tableName +
-" is not equal to the startkey of the next region in hbase:meta." +
-"Please use hbck tool to fix it first.");
+ throw new IllegalArgumentException("Invalid range: " +
Bytes.toStringBinary(first.get())
+ + " > " + Bytes.toStringBinary(last.get()));
}
-
-boolean lastKeyInRange = Bytes.compareTo(last.get(),
startEndKeys.get(idx).getSecond()) < 0 ||
- Bytes.equals(startEndKeys.get(idx).getSecond(),
HConstants.EMPTY_BYTE_ARRAY);
+int firstKeyRegionIdx = getVaildRegionIndex(startEndKeys, first.get(),
tableName);
+boolean lastKeyInRange =
+Bytes.compareTo(last.get(),
startEndKeys.get(firstKeyRegionIdx).getSecond()) < 0 || Bytes
+.equals(startEndKeys.get(firstKeyRegionIdx).getSecond(),
HConstants.EMPTY_BYTE_ARRAY);
if (!lastKeyInRange) {
- Pair startEndKey = startEndKeys.get(indexForCallable);
- List lqis =
-splitStoreFile(item,
FutureUtils.get(conn.getAdmin().getDescriptor(tableName)),
-startEndKey.getSecond());
+ int lastKeyRegionIdx = getVaildRegionIndex(startEndKeys, last.get(),
tableName);
+ int splitIdx = (firstKeyRegionIdx + lastKeyRegionIdx) / 2;
Review comment:
Makes sense, thanks for explaining!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2688: HBASE-25311 UI throws NPE
Apache-HBase commented on pull request #2688: URL: https://github.com/apache/hbase/pull/2688#issuecomment-731562787 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 26s | master passed | | +1 :green_heart: | checkstyle | 1m 6s | master passed | | +1 :green_heart: | spotbugs | 1m 57s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 25s | the patch passed | | +1 :green_heart: | checkstyle | 1m 6s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 11s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 38m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2688 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 61ccabd09d45 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2688: HBASE-25311 UI throws NPE
Apache-HBase commented on pull request #2688: URL: https://github.com/apache/hbase/pull/2688#issuecomment-731558509 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | master passed | | +1 :green_heart: | compile | 1m 5s | master passed | | +1 :green_heart: | shadedjars | 6m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 3s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 6m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 144m 50s | hbase-server in the patch failed. | | | | 172m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2688 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 16211d5507f1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-11.0.6+10 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/1/testReport/ | | Max. process+thread count | 4142 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2688: HBASE-25311 UI throws NPE
Apache-HBase commented on pull request #2688: URL: https://github.com/apache/hbase/pull/2688#issuecomment-731558378 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 49s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 6m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 27s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | shadedjars | 6m 31s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 144m 50s | hbase-server in the patch failed. | | | | 171m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2688 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f1627ad6189c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/1/testReport/ | | Max. process+thread count | 4885 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] cuibo01 opened a new pull request #2511: HBASE-24960 reduce invalid subprocedure task
cuibo01 opened a new pull request #2511: URL: https://github.com/apache/hbase/pull/2511 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] cuibo01 removed a comment on pull request #2511: HBASE-24960 reduce invalid subprocedure task
cuibo01 removed a comment on pull request #2511: URL: https://github.com/apache/hbase/pull/2511#issuecomment-731556733 > 我的评论从+1更改为修正检查样式并测试投诉。谢谢。 > Changed my review from +1 to fix the checkstyle and test complaints. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] cuibo01 closed pull request #2511: HBASE-24960 reduce invalid subprocedure task
cuibo01 closed pull request #2511: URL: https://github.com/apache/hbase/pull/2511 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] cuibo01 commented on pull request #2511: HBASE-24960 reduce invalid subprocedure task
cuibo01 commented on pull request #2511: URL: https://github.com/apache/hbase/pull/2511#issuecomment-731556733 > 我的评论从+1更改为修正检查样式并测试投诉。谢谢。 > Changed my review from +1 to fix the checkstyle and test complaints. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24395) ServerName#getHostname() is case sensitive
[ https://issues.apache.org/jira/browse/HBASE-24395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236622#comment-17236622 ] Bo Cui commented on HBASE-24395: [https://github.com/apache/hbase/pull/2690] update > ServerName#getHostname() is case sensitive > -- > > Key: HBASE-24395 > URL: https://issues.apache.org/jira/browse/HBASE-24395 > Project: HBase > Issue Type: Sub-task > Components: Balancer >Affects Versions: 1.3.1 >Reporter: Bo Cui >Priority: Major > Attachments: HBase-24395.patch, image-2020-05-18-17-42-57-119.png > > > ServerName calss，the getServerName(String hostName, int port, long > startcode),equals and compareTo are case insensitive, but getHostname() is > case sensitive. > if hostName is HOSTNAME1, ServerName is hostname1,1,1589615319931, and > getHostname() returns HOSTNAME1. > and then BaseLoadBalancer#retainAssignment() uses ServerName#getHostname(), > all keys of serversByHostname are > upperCase(HOSTNAME1,HOSTNAME2,HOSTNAME3,HOSTNAME4...) from > ServerManager#createDestinationServersList, but oldServerName.getHostname() > is lowerCase(hostname1,hostname2,hostname3...) from walLog dir. > !image-2020-05-18-17-42-57-119.png! > and finally...all region of old ServerName will be assigned to random hosts -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] cuibo01 opened a new pull request #2690: HBASE-24395 backport HBASE-20589
cuibo01 opened a new pull request #2690: URL: https://github.com/apache/hbase/pull/2690 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2511: HBASE-24960 reduce invalid subprocedure task
Apache-HBase commented on pull request #2511: URL: https://github.com/apache/hbase/pull/2511#issuecomment-731552920 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 18s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | checkstyle | 1m 13s | master passed | | +1 :green_heart: | spotbugs | 2m 10s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | the patch passed | | -0 :warning: | checkstyle | 1m 10s | hbase-server: The patch generated 17 new + 32 unchanged - 0 fixed = 49 total (was 32) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 3s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 43m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2511 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 5d45ef009d8a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2511/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
Apache-HBase commented on pull request #2689: URL: https://github.com/apache/hbase/pull/2689#issuecomment-731550915 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 43s | master passed | | +1 :green_heart: | checkstyle | 1m 19s | master passed | | +1 :green_heart: | spotbugs | 2m 29s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 24s | the patch passed | | +1 :green_heart: | checkstyle | 1m 16s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 12s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 46s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 41m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2689 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 66a433b04fe4 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-replication hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2689/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24924) stuck InitMetaProcedure in finishActiveMasterInitialization
[
https://issues.apache.org/jira/browse/HBASE-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236614#comment-17236614
]
Bo Cui commented on HBASE-24924:
{quote}
You still do not answer my question...
How could this happen without data corruption from outside?
{quote}
If the data was not corrupted, the problem would not have occurred.
But the data corruption exists，and we don't know how it is damaged.
we have a lot of cluster, and should consider how to automatically recover.
> stuck InitMetaProcedure in finishActiveMasterInitialization
> ---
>
> Key: HBASE-24924
> URL: https://issues.apache.org/jira/browse/HBASE-24924
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 3.0.0-alpha-1, 2.2.3
>Reporter: Bo Cui
>Assignee: Bo Cui
>Priority: Major
> Attachments: image-2020-08-21-09-12-33-894.png, masterLog.gif
>
>
> if procWAL has InitMetaProcedure, and InitMetaProcedure state is finished,
> and meta znode does not exist， finishActiveMasterInitialization will stuck
> because during startup，If InitMetaProcedure exists, InitMetaProcedure
> recreates a new CountDownLatch.
> master jstack
> !image-2020-08-21-09-12-33-894.png!
> master log
> !masterLog.gif!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] cuibo01 commented on a change in pull request #2352: HBASE-24925 SCP reduce unnecessary get requests
cuibo01 commented on a change in pull request #2352:
URL: https://github.com/apache/hbase/pull/2352#discussion_r528169267
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
##
@@ -318,4 +326,39 @@ protected void deleteZooKeeper(TableName tableName) {
LOG.warn("Failed deleting table state from zookeeper", e);
}
}
+
+ public void asyncLoadTableState() {
+new Thread(() -> {
+ LOG.info("load table state...");
+ ThreadFactoryBuilder builder = new
ThreadFactoryBuilder().setDaemon(true).setNameFormat("TableStateFetcher" +
"-%1$d");
+ ExecutorService
+executorService =
Executors.newFixedThreadPool(master.getConfiguration().getInt("hbase.table.state.fetcher.threads",10),
builder.build());
+
+ try {
+List results =
MetaTableAccessor.fullScan(master.getConnection(),
ClientMetaTableAccessor.QueryType.TABLE);
+List> futures = new ArrayList<>();
+results.forEach(r -> futures.add(executorService.submit(new
Callable() {
Review comment:
By add debug log and observation, the processing result takes a lot of
time.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] cuibo01 commented on a change in pull request #2352: HBASE-24925 SCP reduce unnecessary get requests
cuibo01 commented on a change in pull request #2352:
URL: https://github.com/apache/hbase/pull/2352#discussion_r528168990
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -1556,6 +1556,7 @@ public RegionInfo loadRegionFromMeta(String
regionEncodedName) throws UnknownReg
}
private void loadMeta() throws IOException {
+master.getTableStateManager().asyncLoadTableState();
Review comment:
yes, scan region and scan table will be the next work. they will be
processed together in async
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-23340) hmaster /hbase/replication/rs session expired (hbase replication default value is true, we don't use ) causes logcleaner can not clean oldWALs, which resulits in oldWA
[ https://issues.apache.org/jira/browse/HBASE-23340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui updated HBASE-23340: --- Status: Patch Available (was: Open) > hmaster /hbase/replication/rs session expired (hbase replication default > value is true, we don't use ) causes logcleaner can not clean oldWALs, which > resulits in oldWALs too large (more than 2TB) > - > > Key: HBASE-23340 > URL: https://issues.apache.org/jira/browse/HBASE-23340 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 2.2.3, 3.0.0-alpha-1 >Reporter: jackylau >Assignee: Bo Cui >Priority: Major > Attachments: Snipaste_2019-11-21_10-39-25.png, > Snipaste_2019-11-21_14-10-36.png > > > hmaster /hbase/replication/rs session expired (hbase replication default > value is true, we don't use ) causes logcleaner can not clean oldWALs, which > resulits in oldWALs too large (more than 2TB). > !Snipaste_2019-11-21_10-39-25.png! > > !Snipaste_2019-11-21_14-10-36.png! > > we can solve it by following : > 1) increase the session timeout(but i think it is not a good idea. because we > do not know how long to set is suitable) > 2) close the hbase replication. It is not a good idea too, when our user uses > this feature > 3) we need add retry times, for example when it has already happened three > times, we set the ReplicationLogCleaner and SnapShotCleaner stop > that is all my ideas, i do not konw it is suitable, If it is suitable, could > i commit a PR? > Does anynode have a good idea. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25311) ui throws NPE
[ https://issues.apache.org/jira/browse/HBASE-25311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui updated HBASE-25311: --- Status: Patch Available (was: Open) > ui throws NPE > - > > Key: HBASE-25311 > URL: https://issues.apache.org/jira/browse/HBASE-25311 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.3, 3.0.0-alpha-1 >Reporter: Bo Cui >Assignee: Bo Cui >Priority: Major > > https://github.com/apache/hbase/blob/eca904e0fb438461a8da3f37cea3eaf496988be9/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L3624 > if rs has invalid znode, and restart master, ui will throw NPE. > i encountered this problem during the upgrade. > workaround: restart HBase. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] cuibo01 opened a new pull request #2689: HBASE-23340 hmaster /hbase/replication/rs session expired (hbase repl…
cuibo01 opened a new pull request #2689: URL: https://github.com/apache/hbase/pull/2689 …ication default value is true, we don't use ) causes logcleaner can not clean oldWALs, which resulits in oldWALs too large (more than 2TB) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2688: HBASE-25311 UI throws NPE
Apache-HBase commented on pull request #2688: URL: https://github.com/apache/hbase/pull/2688#issuecomment-731525655 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | master passed | | +1 :green_heart: | checkstyle | 1m 8s | master passed | | +1 :green_heart: | spotbugs | 2m 1s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | -0 :warning: | checkstyle | 1m 5s | hbase-server: The patch generated 1 new + 89 unchanged - 0 fixed = 90 total (was 89) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 21s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 39m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2688 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux cc1732d809a0 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eca904e0fb | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2688/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
