[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-705982842 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 15s | master passed | | +1 :green_heart: | compile | 1m 28s | master passed | | +1 :green_heart: | shadedjars | 6m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 15s | the patch passed | | +1 :green_heart: | compile | 1m 31s | the patch passed | | +1 :green_heart: | javac | 1m 31s | the patch passed | | +1 :green_heart: | shadedjars | 6m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 40s | hbase-procedure in the patch passed. | | -1 :x: | unit | 135m 44s | hbase-server in the patch failed. | | | | 168m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d9f368ffaef2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 78ae1f176d | | Default Java | 2020-01-14 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/8/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/8/testReport/ | | Max. process+thread count | 3941 (vs. ulimit of 3) | | modules | C: hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/8/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
Apache-HBase commented on pull request #1531: URL: https://github.com/apache/hbase/pull/1531#issuecomment-705970633 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 58s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 58s | master passed | | +1 :green_heart: | compile | 3m 4s | master passed | | +1 :green_heart: | shadedjars | 7m 24s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 5s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 37s | the patch passed | | +1 :green_heart: | compile | 3m 14s | the patch passed | | +1 :green_heart: | javac | 3m 14s | the patch passed | | +1 :green_heart: | shadedjars | 7m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 245m 41s | root in the patch passed. | | | | 288m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1531 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8e09d31b7d3b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 78ae1f176d | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/4/testReport/ | | Max. process+thread count | 4957 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ramkrish86 commented on a change in pull request #2501: HBASE-25065 WAL archival can be batched/throttled and also done by a …
ramkrish86 commented on a change in pull request #2501:
URL: https://github.com/apache/hbase/pull/2501#discussion_r502191423
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
##
@@ -482,6 +505,8 @@ protected SyncFuture initialValue() {
this.walTooOldNs = TimeUnit.SECONDS.toNanos(conf.getInt(
SURVIVED_TOO_LONG_SEC_KEY, SURVIVED_TOO_LONG_SEC_DEFAULT));
this.useHsync = conf.getBoolean(HRegion.WAL_HSYNC_CONF_KEY,
HRegion.DEFAULT_WAL_HSYNC);
+archiveRetries =
this.conf.getInt("hbase.regionserver.walroll.archive.retries", 0);
Review comment:
Ideally if the WAL FS is having an issue then this archive may also
fail. By default we will try it (mandatory) once (so retries are 0) - then if
this new config is set to a non-zero value we will try to repeat it for the
configured times. Generally we are not going to configure it to a non-zero
value.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-25034) Table regions details on master GUI display slowly.
[ https://issues.apache.org/jira/browse/HBASE-25034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DingSheng Li updated HBASE-25034: - Attachment: The table display after pagination.html > Table regions details on master GUI display slowly. > --- > > Key: HBASE-25034 > URL: https://issues.apache.org/jira/browse/HBASE-25034 > Project: HBase > Issue Type: Improvement >Reporter: DingSheng Li >Priority: Major > Labels: newbie > Attachments: The table display after pagination.html > > > When a table has a large number of regions (e.g.,a single table contains more > than 100,000 regions), it takes about 20 to 30 minutes to display the table > regions on the master GUI, which is unacceptable to users. After testing, we > find that web page rendering takes up the most time,and this can be solved by > pagination query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2515: HBASE-14067 bundle ruby files for hbase shell into a jar.
Apache-HBase commented on pull request #2515: URL: https://github.com/apache/hbase/pull/2515#issuecomment-705957976 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 1s | Shelldocs was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 36s | master passed | | +1 :green_heart: | checkstyle | 1m 57s | master passed | | +1 :green_heart: | spotbugs | 8m 50s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 24s | the patch passed | | -0 :warning: | checkstyle | 1m 57s | root: The patch generated 21 new + 0 unchanged - 0 fixed = 21 total (was 0) | | -0 :warning: | rubocop | 1m 0s | The patch generated 4 new + 1107 unchanged - 5 fixed = total (was 1112) | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 4s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 17m 5s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 8m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 41s | The patch does not generate ASF License warnings. | | | | 56m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2515 | | Optional Tests | dupname asflicense shellcheck shelldocs rubocop xml hadoopcheck spotbugs hbaseanti checkstyle | | uname | Linux c402ffef26e3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 78ae1f176d | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/2/artifact/yetus-general-check/output/diff-checkstyle-root.txt | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 137 (vs. ulimit of 3) | | modules | C: hbase-shell hbase-assembly . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 spotbugs=3.1.12 rubocop=0.80.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
Apache-HBase commented on pull request #2514: URL: https://github.com/apache/hbase/pull/2514#issuecomment-705950160 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 50s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | master passed | | +1 :green_heart: | compile | 1m 10s | master passed | | +1 :green_heart: | shadedjars | 6m 53s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 11s | the patch passed | | +1 :green_heart: | compile | 1m 9s | the patch passed | | +1 :green_heart: | javac | 1m 9s | the patch passed | | +1 :green_heart: | shadedjars | 7m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 142m 3s | hbase-server in the patch failed. | | | | 172m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2514 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5c7d4e674778 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 78ae1f176d | | Default Java | 2020-01-14 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/2/testReport/ | | Max. process+thread count | 4031 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-705949938 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 32s | master passed | | +1 :green_heart: | checkstyle | 1m 19s | master passed | | +1 :green_heart: | spotbugs | 2m 28s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 25s | the patch passed | | +1 :green_heart: | checkstyle | 1m 17s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 16m 51s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 24s | The patch does not generate ASF License warnings. | | | | 40m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/8/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux a2bacc901f65 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 78ae1f176d | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/8/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2515: HBASE-14067 bundle ruby files for hbase shell into a jar.
Apache-HBase commented on pull request #2515: URL: https://github.com/apache/hbase/pull/2515#issuecomment-705943792 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 2s | master passed | | +1 :green_heart: | compile | 2m 34s | master passed | | +1 :green_heart: | shadedjars | 7m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 35s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 47s | the patch passed | | +1 :green_heart: | compile | 2m 36s | the patch passed | | +1 :green_heart: | javac | 2m 36s | the patch passed | | +1 :green_heart: | shadedjars | 7m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 374m 53s | root in the patch passed. | | | | 411m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2515 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3e8aeaa91358 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bdcafa895c | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/1/testReport/ | | Max. process+thread count | 3952 (vs. ulimit of 3) | | modules | C: hbase-shell hbase-assembly . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24813) ReplicationSource should clear buffer usage on ReplicationSourceManager upon termination
[ https://issues.apache.org/jira/browse/HBASE-24813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Xin updated HBASE-24813: Attachment: image-2020-10-09-10-50-00-372.png > ReplicationSource should clear buffer usage on ReplicationSourceManager upon > termination > > > Key: HBASE-24813 > URL: https://issues.apache.org/jira/browse/HBASE-24813 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0-alpha-1, 2.3.1, 2.2.6 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7 > > Attachments: TestReplicationSyncUpTool.log, > image-2020-10-09-10-50-00-372.png > > > Following investigations on the issue described by [~elserj] on HBASE-24779, > we found out that once a peer is removed, thus killing peers related > *ReplicationSource* instance, it may leave > *ReplicationSourceManager.totalBufferUsed* inconsistent. This can happen if > *ReplicationSourceWALReader* had put some entries on its queue to be > processed by *ReplicationSourceShipper,* but the peer removal killed the > shipper before it could process the pending entries. When > *ReplicationSourceWALReader* thread add entries to the queue, it increments > *ReplicationSourceManager.totalBufferUsed* with the sum of the entries sizes. > When those entries are read by *ReplicationSourceShipper,* > *ReplicationSourceManager.totalBufferUsed* is then decreased. We should also > decrease *ReplicationSourceManager.totalBufferUsed* when *ReplicationSource* > is terminated, otherwise those unprocessed entries size would be consuming > *ReplicationSourceManager.totalBufferUsed __*indefinitely, unless the RS gets > restarted. This may be a problem for deployments with multiple peers, or if > new peers are added.** -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24813) ReplicationSource should clear buffer usage on ReplicationSourceManager upon termination
[ https://issues.apache.org/jira/browse/HBASE-24813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210572#comment-17210572 ] Sun Xin commented on HBASE-24813: - Using isActive() instead of isAlive() in this [PR|https://github.com/apache/hbase/pull/2191/files#], that may work? !image-2020-10-09-10-50-00-372.png! > ReplicationSource should clear buffer usage on ReplicationSourceManager upon > termination > > > Key: HBASE-24813 > URL: https://issues.apache.org/jira/browse/HBASE-24813 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0-alpha-1, 2.3.1, 2.2.6 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7 > > Attachments: TestReplicationSyncUpTool.log, > image-2020-10-09-10-50-00-372.png > > > Following investigations on the issue described by [~elserj] on HBASE-24779, > we found out that once a peer is removed, thus killing peers related > *ReplicationSource* instance, it may leave > *ReplicationSourceManager.totalBufferUsed* inconsistent. This can happen if > *ReplicationSourceWALReader* had put some entries on its queue to be > processed by *ReplicationSourceShipper,* but the peer removal killed the > shipper before it could process the pending entries. When > *ReplicationSourceWALReader* thread add entries to the queue, it increments > *ReplicationSourceManager.totalBufferUsed* with the sum of the entries sizes. > When those entries are read by *ReplicationSourceShipper,* > *ReplicationSourceManager.totalBufferUsed* is then decreased. We should also > decrease *ReplicationSourceManager.totalBufferUsed* when *ReplicationSource* > is terminated, otherwise those unprocessed entries size would be consuming > *ReplicationSourceManager.totalBufferUsed __*indefinitely, unless the RS gets > restarted. This may be a problem for deployments with multiple peers, or if > new peers are added.** -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2520: Backport "HBASE-24628 Region normalizer now respects a rate limit" to branch-2
Apache-HBase commented on pull request #2520: URL: https://github.com/apache/hbase/pull/2520#issuecomment-705931960 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 16s | branch-2 passed | | +1 :green_heart: | compile | 1m 5s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 52s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 6m 45s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 137m 23s | hbase-server in the patch failed. | | | | 166m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2520 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cffe61ba22da 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 627002d279 | | Default Java | 2020-01-14 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/testReport/ | | Max. process+thread count | 3959 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705930883 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | master passed | | +1 :green_heart: | compile | 2m 30s | master passed | | +1 :green_heart: | shadedjars | 7m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 10s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 43s | the patch passed | | +1 :green_heart: | compile | 2m 32s | the patch passed | | +1 :green_heart: | javac | 2m 32s | the patch passed | | +1 :green_heart: | shadedjars | 7m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 379m 18s | root in the patch passed. | | | | 413m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/10/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 785cf22eb395 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bdcafa895c | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/10/testReport/ | | Max. process+thread count | 3894 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/10/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2520: Backport "HBASE-24628 Region normalizer now respects a rate limit" to branch-2
Apache-HBase commented on pull request #2520: URL: https://github.com/apache/hbase/pull/2520#issuecomment-705931072 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 38s | branch-2 passed | | +1 :green_heart: | compile | 0m 57s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 56s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 15s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | shadedjars | 5m 55s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 36s | hbase-server generated 11 new + 2 unchanged - 25 fixed = 13 total (was 27) | ||| _ Other Tests _ | | -1 :x: | unit | 137m 11s | hbase-server in the patch failed. | | | | 162m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2520 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3733501d6de2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 627002d279 | | Default Java | 1.8.0_232 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-jdk8-hadoop2-check/output/diff-javadoc-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/testReport/ | | Max. process+thread count | 4441 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] VictorZeng commented on pull request #77: HBASE-25161 [hbase-meta-repair] Add tool for repairing metadata.
VictorZeng commented on pull request #77: URL: https://github.com/apache/hbase-operator-tools/pull/77#issuecomment-705925637 > > > > There is no `HbckService` in versions of hbase _before 2.0.3 and 2.1.1_. HBCK2 will not work against these versions. If users use Hbase 2.1.0 version, you can use this tool to repair metadata. > > > > > > > > > Have you tried use these functions and got some error? If so, we should fix that, as neither `addFsRegionsMissingInMeta` nor `extraRegionsInMeta` rely on HbckService, they perform fixes on their own, thus, there should be no hbase version validation when using these two functions, and they should be compatible with the above versions. > > > > > > Hi wchevreuil, although both `addFsRegionsMissingInMeta` and `extraRegionsInMeta` do not rely on HbckService, but `assigns` command rely on HbckService... So I think this tool is very useful. > > > Hi wchevreuil, although both `addFsRegionsMissingInMeta` and `extraRegionsInMeta` do not rely on HbckService, but `assigns` command rely on HbckService... So I think this tool is very useful. > > Then the only extra bit is a bulk assignment? Could it reuse `addFsRegionsMissingInMeta` and `extraRegionsInMeta`, instead of providing extra lines of code that do same as the already existing methods? I do agree the command proposed here is very convenient, but we should re-use already existing code as much as possible. This tool function is more similar to the hbck1 `fixMeta` command, it can repair the metadata when users are deploying a 2.x version without HBCK2 support more quickly and conveniently. So it's necessary to provide extra codes. I think it's better to be independent. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
Apache-HBase commented on pull request #2514: URL: https://github.com/apache/hbase/pull/2514#issuecomment-705915243 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | master passed | | +1 :green_heart: | checkstyle | 1m 4s | master passed | | +1 :green_heart: | spotbugs | 2m 3s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 30s | the patch passed | | -0 :warning: | checkstyle | 1m 4s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 4s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 39m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2514 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux e5f4322b8197 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 78ae1f176d | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ankitsinghal commented on a change in pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
ankitsinghal commented on a change in pull request #2514:
URL: https://github.com/apache/hbase/pull/2514#discussion_r502112732
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileCompactionChore.java
##
@@ -84,10 +84,10 @@ public MobFileCompactionChore(Configuration conf, int
batchSize) {
protected void chore() {
boolean reported = false;
-
-try (Connection conn = master.getConnection();
-Admin admin = conn.getAdmin();) {
-
+Admin admin = null;
+try {
+ Connection conn = master.getConnection();
Review comment:
yes, we should guard-rail the internal connection (in master or RS) from
being closed by any shared services by probably wrapping it with
SharedConnection which throws an exception when close() is called out (the same
way we share it with users through coprocessor ). I'll raise another JIRA for
it.
Thanks guys for the ideas and review.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileCompactionChore.java
##
@@ -84,10 +84,10 @@ public MobFileCompactionChore(Configuration conf, int
batchSize) {
protected void chore() {
boolean reported = false;
-
-try (Connection conn = master.getConnection();
-Admin admin = conn.getAdmin();) {
-
+Admin admin = null;
+try {
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2515: HBASE-14067 bundle ruby files for hbase shell into a jar.
Apache-HBase commented on pull request #2515: URL: https://github.com/apache/hbase/pull/2515#issuecomment-705906411 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 46s | master passed | | +1 :green_heart: | compile | 3m 13s | master passed | | +1 :green_heart: | shadedjars | 7m 28s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 35s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 37s | the patch passed | | +1 :green_heart: | compile | 3m 11s | the patch passed | | +1 :green_heart: | javac | 3m 11s | the patch passed | | +1 :green_heart: | shadedjars | 7m 23s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 234m 26s | root in the patch passed. | | | | 278m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2515 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 008c8e023c27 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bdcafa895c | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/1/testReport/ | | Max. process+thread count | 4489 (vs. ulimit of 3) | | modules | C: hbase-shell hbase-assembly . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
Apache-HBase commented on pull request #1531: URL: https://github.com/apache/hbase/pull/1531#issuecomment-705902959 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 20m 59s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 31m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1531 | | Optional Tests | dupname asflicense shellcheck shelldocs hadoopcheck xml | | uname | Linux 10f7dd424018 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 78ae1f176d | | Max. process+thread count | 63 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/4/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2520: Backport "HBASE-24628 Region normalizer now respects a rate limit" to branch-2
Apache-HBase commented on pull request #2520: URL: https://github.com/apache/hbase/pull/2520#issuecomment-705896238 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 40s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 11s | branch-2 passed | | +1 :green_heart: | spotbugs | 2m 0s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 13s | the patch passed | | -0 :warning: | checkstyle | 1m 10s | hbase-server: The patch generated 13 new + 152 unchanged - 7 fixed = 165 total (was 159) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 29s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 33m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2520 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux ad9f2ea95dc7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 627002d279 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2520/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #2520: Backport "HBASE-24628 Region normalizer now respects a rate limit" to branch-2
ndimiduk opened a new pull request #2520: URL: https://github.com/apache/hbase/pull/2520 Implement a rate limiter for the normalizer. Implemented in terms of MB/sec of affacted region size (the same metrics used to make normalization decisions). Uses Guava `RateLimiter` to perform the resource accounting. `RateLimiter` works by blocking (uninterruptible 😖) the calling thread. Thus, the whole construction of the normalizer subsystem needed refactoring. See the provided `package-info.java` for an overview of this new structure. Introduces a new configuration, `hbase.normalizer.throughput.max_bytes_per_sec`, for specifying a limit on the throughput of actions executed by the normalizer. Note that while this configuration value is in bytes, the minimum honored valued `1_000_000`. Supports values configured using the human-readable suffixes honored by `Configuration.getLongBytes` Signed-off-by: Viraj Jasani Signed-off-by: Huaxiang Sun Signed-off-by: Michael Stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #2454: HBASE-24628 Region normalizer now respects a rate limit
ndimiduk commented on pull request #2454: URL: https://github.com/apache/hbase/pull/2454#issuecomment-705861790 Thanks for the detailed reviews everyone! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #2454: HBASE-24628 Region normalizer now respects a rate limit
ndimiduk merged pull request #2454: URL: https://github.com/apache/hbase/pull/2454 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705859621 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 56s | master passed | | +1 :green_heart: | compile | 3m 22s | master passed | | +1 :green_heart: | shadedjars | 8m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 13s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 0s | the patch passed | | +1 :green_heart: | compile | 3m 34s | the patch passed | | +1 :green_heart: | javac | 3m 34s | the patch passed | | +1 :green_heart: | shadedjars | 8m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 20s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 136m 34s | root in the patch failed. | | | | 180m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/10/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux eb0be4275ecd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bdcafa895c | | Default Java | 2020-01-14 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/10/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/10/testReport/ | | Max. process+thread count | 3732 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/10/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25156) TestMasterFailover.testSimpleMasterFailover is flaky
[
https://issues.apache.org/jira/browse/HBASE-25156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-25156:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
Backported to everywhere relevant. Fun fact: branch-1.4 has a wait-for loop in
place that I suspect is effectively smoothing over this issue, so this is a
regression since that release line.
Thanks [~stack] for the review.
> TestMasterFailover.testSimpleMasterFailover is flaky
> -
>
> Key: HBASE-25156
> URL: https://issues.apache.org/jira/browse/HBASE-25156
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 2.2.7
>
>
> {noformat}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 32.996 s <<< FAILURE! - in org.apache.hadoop.hbase.master.TestMasterFailover
> [ERROR]
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover
> Time elapsed: 12.317 s <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<2>
> at
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover(TestMasterFailover.java:133)
> {noformat}
> Looks like this test depends on metrics to be updated as a source of
> side-effect used to verify the test. Seems like it should retry the check a
> few times, or maybe we need a last-updated monotonic value that the test can
> check before and after it expects a change to be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2519: Backport "HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)" to branch-1
Apache-HBase commented on pull request #2519: URL: https://github.com/apache/hbase/pull/2519#issuecomment-705843667 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | patch | 0m 7s | https://github.com/apache/hbase/pull/2519 does not apply to branch-1. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/2519 | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2519/1/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #2519: Backport "HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)" to branch-1
ndimiduk merged pull request #2519: URL: https://github.com/apache/hbase/pull/2519 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25156) TestMasterFailover.testSimpleMasterFailover is flaky
[
https://issues.apache.org/jira/browse/HBASE-25156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-25156:
-
Fix Version/s: 1.7.0
> TestMasterFailover.testSimpleMasterFailover is flaky
> -
>
> Key: HBASE-25156
> URL: https://issues.apache.org/jira/browse/HBASE-25156
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 2.2.7
>
>
> {noformat}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 32.996 s <<< FAILURE! - in org.apache.hadoop.hbase.master.TestMasterFailover
> [ERROR]
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover
> Time elapsed: 12.317 s <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<2>
> at
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover(TestMasterFailover.java:133)
> {noformat}
> Looks like this test depends on metrics to be updated as a source of
> side-effect used to verify the test. Seems like it should retry the check a
> few times, or maybe we need a last-updated monotonic value that the test can
> check before and after it expects a change to be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #2519: Backport "HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)" to branch-1
ndimiduk opened a new pull request #2519: URL: https://github.com/apache/hbase/pull/2519 Change the test to wait for evidence that the active master has seen that the backup master killed by the test has gone away. This is done before proceeding to validate that the dead backup is correctly omitted from the ClusterStatus report. Also, minor fixup to several assertions, using `assertEquals` instead of `assertTrue(...equals(...))` and correcting expected vs. actual ordering of assertion arguments. Signed-off-by: Michael Stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #2518: Backport "HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)" to branch-2.2
ndimiduk merged pull request #2518: URL: https://github.com/apache/hbase/pull/2518 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25156) TestMasterFailover.testSimpleMasterFailover is flaky
[
https://issues.apache.org/jira/browse/HBASE-25156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-25156:
-
Fix Version/s: 2.2.7
> TestMasterFailover.testSimpleMasterFailover is flaky
> -
>
> Key: HBASE-25156
> URL: https://issues.apache.org/jira/browse/HBASE-25156
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7
>
>
> {noformat}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 32.996 s <<< FAILURE! - in org.apache.hadoop.hbase.master.TestMasterFailover
> [ERROR]
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover
> Time elapsed: 12.317 s <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<2>
> at
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover(TestMasterFailover.java:133)
> {noformat}
> Looks like this test depends on metrics to be updated as a source of
> side-effect used to verify the test. Seems like it should retry the check a
> few times, or maybe we need a last-updated monotonic value that the test can
> check before and after it expects a change to be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #2518: Backport "HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)" to branch-2.2
ndimiduk opened a new pull request #2518: URL: https://github.com/apache/hbase/pull/2518 Change the test to wait for evidence that the active master has seen that the backup master killed by the test has gone away. This is done before proceeding to validate that the dead backup is correctly omitted from the ClusterStatus report. Also, minor fixup to several assertions, using `assertEquals` instead of `assertTrue(...equals(...))` and correcting expected vs. actual ordering of assertion arguments. Signed-off-by: Michael Stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25156) TestMasterFailover.testSimpleMasterFailover is flaky
[
https://issues.apache.org/jira/browse/HBASE-25156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-25156:
-
Fix Version/s: 2.3.3
> TestMasterFailover.testSimpleMasterFailover is flaky
> -
>
> Key: HBASE-25156
> URL: https://issues.apache.org/jira/browse/HBASE-25156
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 2.4.0
>
>
> {noformat}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 32.996 s <<< FAILURE! - in org.apache.hadoop.hbase.master.TestMasterFailover
> [ERROR]
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover
> Time elapsed: 12.317 s <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<2>
> at
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover(TestMasterFailover.java:133)
> {noformat}
> Looks like this test depends on metrics to be updated as a source of
> side-effect used to verify the test. Seems like it should retry the check a
> few times, or maybe we need a last-updated monotonic value that the test can
> check before and after it expects a change to be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25156) TestMasterFailover.testSimpleMasterFailover is flaky
[
https://issues.apache.org/jira/browse/HBASE-25156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-25156:
-
Fix Version/s: 2.4.0
> TestMasterFailover.testSimpleMasterFailover is flaky
> -
>
> Key: HBASE-25156
> URL: https://issues.apache.org/jira/browse/HBASE-25156
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> {noformat}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 32.996 s <<< FAILURE! - in org.apache.hadoop.hbase.master.TestMasterFailover
> [ERROR]
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover
> Time elapsed: 12.317 s <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<2>
> at
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover(TestMasterFailover.java:133)
> {noformat}
> Looks like this test depends on metrics to be updated as a source of
> side-effect used to verify the test. Seems like it should retry the check a
> few times, or maybe we need a last-updated monotonic value that the test can
> check before and after it expects a change to be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk merged pull request #2517: Backport "HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)" to branch-2.3
ndimiduk merged pull request #2517: URL: https://github.com/apache/hbase/pull/2517 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #2517: Backport "HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)" to branch-2.3
ndimiduk opened a new pull request #2517: URL: https://github.com/apache/hbase/pull/2517 Change the test to wait for evidence that the active master has seen that the backup master killed by the test has gone away. This is done before proceeding to validate that the dead backup is correctly omitted from the ClusterStatus report. Also, minor fixup to several assertions, using `assertEquals` instead of `assertTrue(...equals(...))` and correcting expected vs. actual ordering of assertion arguments. Signed-off-by: Michael Stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2516: Backport "HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)" to branch-2
Apache-HBase commented on pull request #2516: URL: https://github.com/apache/hbase/pull/2516#issuecomment-705833478 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #2516: Backport "HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)" to branch-2
ndimiduk merged pull request #2516: URL: https://github.com/apache/hbase/pull/2516 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #2516: Backport "HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)" to branch-2
ndimiduk opened a new pull request #2516: URL: https://github.com/apache/hbase/pull/2516 Change the test to wait for evidence that the active master has seen that the backup master killed by the test has gone away. This is done before proceeding to validate that the dead backup is correctly omitted from the ClusterStatus report. Also, minor fixup to several assertions, using `assertEquals` instead of `assertTrue(...equals(...))` and correcting expected vs. actual ordering of assertion arguments. Signed-off-by: Michael Stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25156) TestMasterFailover.testSimpleMasterFailover is flaky
[
https://issues.apache.org/jira/browse/HBASE-25156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-25156:
-
Fix Version/s: 3.0.0-alpha-1
> TestMasterFailover.testSimpleMasterFailover is flaky
> -
>
> Key: HBASE-25156
> URL: https://issues.apache.org/jira/browse/HBASE-25156
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 3.0.0-alpha-1
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-1
>
>
> {noformat}
> [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 32.996 s <<< FAILURE! - in org.apache.hadoop.hbase.master.TestMasterFailover
> [ERROR]
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover
> Time elapsed: 12.317 s <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<2>
> at
> org.apache.hadoop.hbase.master.TestMasterFailover.testSimpleMasterFailover(TestMasterFailover.java:133)
> {noformat}
> Looks like this test depends on metrics to be updated as a source of
> side-effect used to verify the test. Seems like it should retry the check a
> few times, or maybe we need a last-updated monotonic value that the test can
> check before and after it expects a change to be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2515: HBASE-14067 bundle ruby files for hbase shell into a jar.
Apache-HBase commented on pull request #2515: URL: https://github.com/apache/hbase/pull/2515#issuecomment-705831620 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 39s | master passed | | +1 :green_heart: | checkstyle | 1m 57s | master passed | | +1 :green_heart: | spotbugs | 8m 55s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | -0 :warning: | checkstyle | 1m 55s | root: The patch generated 21 new + 0 unchanged - 0 fixed = 21 total (was 0) | | -0 :warning: | rubocop | 0m 59s | The patch generated 4 new + 1107 unchanged - 5 fixed = total (was 1112) | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 4s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 17m 0s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 8m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 40s | The patch does not generate ASF License warnings. | | | | 57m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2515 | | Optional Tests | dupname asflicense shellcheck shelldocs rubocop xml hadoopcheck spotbugs hbaseanti checkstyle | | uname | Linux 7246dc7e8d45 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bdcafa895c | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/1/artifact/yetus-general-check/output/diff-checkstyle-root.txt | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/1/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 137 (vs. ulimit of 3) | | modules | C: hbase-shell hbase-assembly . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2515/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 spotbugs=3.1.12 rubocop=0.80.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #2507: HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky
ndimiduk merged pull request #2507: URL: https://github.com/apache/hbase/pull/2507 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #2507: HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky
ndimiduk commented on pull request #2507: URL: https://github.com/apache/hbase/pull/2507#issuecomment-705831185 Failure in TestBucketCache is a known flaky. Thanks for the review @saintstack . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
Apache-HBase commented on pull request #2514: URL: https://github.com/apache/hbase/pull/2514#issuecomment-705828453 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 50s | master passed | | +1 :green_heart: | compile | 0m 58s | master passed | | +1 :green_heart: | shadedjars | 6m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 31s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | shadedjars | 6m 31s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 142m 41s | hbase-server in the patch passed. | | | | 170m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2514 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e872b1cbf88e 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bdcafa895c | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/1/testReport/ | | Max. process+thread count | 3809 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
Apache-HBase commented on pull request #2514: URL: https://github.com/apache/hbase/pull/2514#issuecomment-705820837 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 16s | master passed | | +1 :green_heart: | compile | 1m 5s | master passed | | +1 :green_heart: | shadedjars | 6m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 4s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 6m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 126m 7s | hbase-server in the patch passed. | | | | 153m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2514 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 24c0fa00d795 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bdcafa895c | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/1/testReport/ | | Max. process+thread count | 4197 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] busbey commented on pull request #2515: HBASE-14067 bundle ruby files for hbase shell into a jar.
busbey commented on pull request #2515: URL: https://github.com/apache/hbase/pull/2515#issuecomment-705820578 pushed an addendum after testing running the shell directly from a source build instead of a binary tarball. had a stale reference from when I was first testing running from source instead of build. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-14067) bundle ruby files for hbase shell into a jar.
[
https://issues.apache.org/jira/browse/HBASE-14067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14067:
Hadoop Flags: Incompatible change
Release Note:
The `hbase-shell` artifact now contains the ruby files that implement the hbase
shell. There should be no downstream impact for users of the shell that rely on
the `hbase shell` command.
Folks that wish to include the HBase ruby classes defined for the shell in
their own JRuby scripts should add the `hbase-shell.jar` file to their
classpath rather than add `${HBASE_HOME}/lib/ruby` to their load paths.
Status: Patch Available (was: In Progress)
> bundle ruby files for hbase shell into a jar.
> -
>
> Key: HBASE-14067
> URL: https://issues.apache.org/jira/browse/HBASE-14067
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> We currently package all the ruby scripts for the hbase shell by placing them
> in a directory within lib/. We should be able to put these in a jar file
> since we rely on jruby.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-14067) bundle ruby files for hbase shell into a jar.
[ https://issues.apache.org/jira/browse/HBASE-14067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14067: Fix Version/s: 2.4.0 3.0.0-alpha-1 > bundle ruby files for hbase shell into a jar. > - > > Key: HBASE-14067 > URL: https://issues.apache.org/jira/browse/HBASE-14067 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > We currently package all the ruby scripts for the hbase shell by placing them > in a directory within lib/. We should be able to put these in a jar file > since we rely on jruby. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-14067) bundle ruby files for hbase shell into a jar.
[ https://issues.apache.org/jira/browse/HBASE-14067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14067: Status: In Progress (was: Patch Available) > bundle ruby files for hbase shell into a jar. > - > > Key: HBASE-14067 > URL: https://issues.apache.org/jira/browse/HBASE-14067 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > We currently package all the ruby scripts for the hbase shell by placing them > in a directory within lib/. We should be able to put these in a jar file > since we rely on jruby. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-14067) bundle ruby files for hbase shell into a jar.
[ https://issues.apache.org/jira/browse/HBASE-14067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14067: Status: Patch Available (was: In Progress) > bundle ruby files for hbase shell into a jar. > - > > Key: HBASE-14067 > URL: https://issues.apache.org/jira/browse/HBASE-14067 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > We currently package all the ruby scripts for the hbase shell by placing them > in a directory within lib/. We should be able to put these in a jar file > since we rely on jruby. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] busbey opened a new pull request #2515: HBASE-14067 bundle ruby files for hbase shell into a jar.
busbey opened a new pull request #2515: URL: https://github.com/apache/hbase/pull/2515 * removes some cruft from the hbase-shell pom that appears to be from coping the hbase-server pom long ago * puts the ruby scripts into the hbase-shell jar following the guide from jruby for packaging * removes hard coding the location of the implementation scripts from our runtime * removes hard coding the load path for the implementation from the test code (leaves hard coding the test code location) * provides a work around for a name conflict between our shell and the ruby stdlib shell. tested via unit tests: ``` mvn -Dit.test=NoItTest -Dtest=NoUnitTests install && mvn --no-snapshot-updates -PrunAllTests -pl hbase-shell package ``` and via making an assembly: ``` mvn --no-snapshot-updates -Dtest=NoUnitTests -Dit.test=NoItTests package assembly:single ``` then testing with a stand alone cluster and using the shell with it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705794917 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | | -0 :warning: | rubocop | 0m 3s | The patch generated 8 new + 22 unchanged - 1 fixed = 30 total (was 23) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 19m 0s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 27m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/10/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | dupname asflicense rubocop hadoopcheck xml | | uname | Linux 7a3269922b02 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bdcafa895c | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/10/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 63 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/10/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) rubocop=0.80.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2504: HBASE-22976 [HBCK2] Add RecoveredEditsPlayer
Apache-HBase commented on pull request #2504: URL: https://github.com/apache/hbase/pull/2504#issuecomment-705785314 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 35s | master passed | | +1 :green_heart: | compile | 2m 57s | master passed | | +1 :green_heart: | shadedjars | 7m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 14s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 15s | the patch passed | | +1 :green_heart: | compile | 2m 54s | the patch passed | | +1 :green_heart: | javac | 2m 54s | the patch passed | | +1 :green_heart: | shadedjars | 6m 44s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 163m 14s | root in the patch passed. | | | | 204m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2504/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2504 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9276ae9761ae 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 8a995ae81b | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2504/4/testReport/ | | Max. process+thread count | 6861 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server hbase-mapreduce . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2504/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705780603 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 37s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 49s | master passed | | +1 :green_heart: | compile | 2m 37s | master passed | | +1 :green_heart: | shadedjars | 7m 4s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 24s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 52s | the patch passed | | +1 :green_heart: | compile | 2m 35s | the patch passed | | +1 :green_heart: | javac | 2m 35s | the patch passed | | +1 :green_heart: | shadedjars | 7m 6s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 23s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 314m 22s | root in the patch failed. | | | | 350m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/9/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 43c232fc498a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0d63318f10 | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/9/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/9/testReport/ | | Max. process+thread count | 3557 (vs. ulimit of 3) | | modules | C: hbase-shell . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/9/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] joshelser commented on a change in pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
joshelser commented on a change in pull request #2514:
URL: https://github.com/apache/hbase/pull/2514#discussion_r501958247
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileCompactionChore.java
##
@@ -84,10 +84,10 @@ public MobFileCompactionChore(Configuration conf, int
batchSize) {
protected void chore() {
boolean reported = false;
-
-try (Connection conn = master.getConnection();
-Admin admin = conn.getAdmin();) {
-
+Admin admin = null;
+try {
+ Connection conn = master.getConnection();
Review comment:
> that sounds like a good idea for a follow-on?
Fine to think about as follow-on. Just wanted to ask the question while it
was fresh :D
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-11747) ClusterStatus (heartbeat) is too bulky
[
https://issues.apache.org/jira/browse/HBASE-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210408#comment-17210408
]
Michael Stack commented on HBASE-11747:
---
Looking at a running cluster where servers are carrying about 500 Regions each,
heartbeat size is not too bad:
{code:java}
2020-10-08 19:09:47,344 TRACE
[RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=16000] ipc.RpcServer:
callId: 396871 service: RegionServerStatusService methodName:
RegionServerReport size: 84.5 K connection: 192.192.118.146:50556 deadline:
1602184197326
2020-10-08 19:09:47,344 TRACE
[RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=16000] ipc.RpcServer:
callId: 396218 service: RegionServerStatusService methodName:
RegionServerReport size: 83.7 K connection: 192.192.113.230:17968 deadline:
1602184197326 {code}
> ClusterStatus (heartbeat) is too bulky
> ---
>
> Key: HBASE-11747
> URL: https://issues.apache.org/jira/browse/HBASE-11747
> Project: HBase
> Issue Type: Sub-task
> Components: master, Operability, scaling
>Reporter: Virag Kothari
>Priority: Critical
> Attachments: exceptiontrace
>
>
> Following exception on 0.98 with 1M regions on cluster with 160 region servers
> {code}
> Caused by: java.io.IOException: Call to regionserverhost:port failed on local
> exception: com.google.protobuf.InvalidProtocolBufferException: Protocol
> message was too large. May be malicious. Use
> CodedInputStream.setSizeLimit() to increase the size limit.
> at
> org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1482)
> at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1454)
> at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.getClusterStatus(MasterProtos.java:42555)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.getClusterStatus(HConnectionManager.java:2132)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2166)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$16.call(HBaseAdmin.java:2162)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)
> ... 43 more
> Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol
> message was too large. May be malicious. Use
> CodedInputStream.setSizeLimit() to increase the size limit.
> at
> com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] busbey commented on a change in pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
busbey commented on a change in pull request #2514:
URL: https://github.com/apache/hbase/pull/2514#discussion_r501955660
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileCompactionChore.java
##
@@ -84,10 +84,10 @@ public MobFileCompactionChore(Configuration conf, int
batchSize) {
protected void chore() {
boolean reported = false;
-
-try (Connection conn = master.getConnection();
-Admin admin = conn.getAdmin();) {
-
+Admin admin = null;
+try {
Review comment:
Can we just use try with resources on the `Admin` object? Do we actually
use the Connection at all?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
Apache-HBase commented on pull request #2514: URL: https://github.com/apache/hbase/pull/2514#issuecomment-705767061 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 53s | master passed | | +1 :green_heart: | checkstyle | 1m 5s | master passed | | +1 :green_heart: | spotbugs | 2m 3s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 25s | the patch passed | | +1 :green_heart: | checkstyle | 1m 2s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 54s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 43m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2514 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux cd80b53ff684 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bdcafa895c | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2514/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] busbey commented on a change in pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
busbey commented on a change in pull request #2514:
URL: https://github.com/apache/hbase/pull/2514#discussion_r501949568
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileCompactionChore.java
##
@@ -84,10 +84,10 @@ public MobFileCompactionChore(Configuration conf, int
batchSize) {
protected void chore() {
boolean reported = false;
-
-try (Connection conn = master.getConnection();
-Admin admin = conn.getAdmin();) {
-
+Admin admin = null;
+try {
+ Connection conn = master.getConnection();
Review comment:
like have it log a debug/info for someone trying to call the normal
close but don't do it, then have `inShutdownClose` to actually close? that
sounds like a good idea for a follow-on?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] joshelser commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
joshelser commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705757460 > If 'main' or the session name are of no value, just exclude? Yeah, if we're consciously making that change, I can get behind it, too. Something as simple as: `hbase:%03n>`? (I don't know what the `%i` is actually doing, seems to always be zero when i'm playing around). I think Norbert addressed the licensing stuff in https://issues.apache.org/jira/browse/HBASE-20598?focusedCommentId=17197067&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17197067. IMO it's good to go (but would let Busbey come back and confirm as the one who's been here before). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] joshelser commented on a change in pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
joshelser commented on a change in pull request #2514:
URL: https://github.com/apache/hbase/pull/2514#discussion_r501930509
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileCompactionChore.java
##
@@ -84,10 +84,10 @@ public MobFileCompactionChore(Configuration conf, int
batchSize) {
protected void chore() {
boolean reported = false;
-
-try (Connection conn = master.getConnection();
-Admin admin = conn.getAdmin();) {
-
+Admin admin = null;
+try {
+ Connection conn = master.getConnection();
Review comment:
I feel like we've run into this a few times recently.
Any thoughts about whether it's a good idea to make the
ShortCircuitingClusterConnection _not_ close itself (unless we can know that
we're in the process of shutting down)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ankitsinghal opened a new pull request #2514: HBASE-25166 MobFileCompactionChore is closing the master's shared clu…
ankitsinghal opened a new pull request #2514: URL: https://github.com/apache/hbase/pull/2514 …ster connection This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #1727: HBASE-17756 We should have better introspection of HFiles
saintstack commented on pull request #1727: URL: https://github.com/apache/hbase/pull/1727#issuecomment-705743040 Putting aside for now until more interest. There is the reporter tool over in hbase-operator-tools that does table/region basis. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack closed pull request #1727: HBASE-17756 We should have better introspection of HFiles
saintstack closed pull request #1727: URL: https://github.com/apache/hbase/pull/1727 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2504: HBASE-22976 [HBCK2] Add RecoveredEditsPlayer
Apache-HBase commented on pull request #2504: URL: https://github.com/apache/hbase/pull/2504#issuecomment-705727349 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 33s | master passed | | +1 :green_heart: | checkstyle | 2m 24s | master passed | | +0 :ok: | refguide | 7m 7s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 16m 30s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 29s | the patch passed | | -0 :warning: | checkstyle | 2m 38s | root: The patch generated 1 new + 9 unchanged - 22 fixed = 10 total (was 31) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 7m 7s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 20m 18s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 12m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 52s | The patch does not generate ASF License warnings. | | | | 88m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2504/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2504 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide | | uname | Linux 8a9b6146fb16 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 8a995ae81b | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2504/4/artifact/yetus-general-check/output/branch-site/book.html | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2504/4/artifact/yetus-general-check/output/diff-checkstyle-root.txt | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2504/4/artifact/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 139 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server hbase-mapreduce . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2504/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-23959) Fix javadoc for JDK11
[
https://issues.apache.org/jira/browse/HBASE-23959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210353#comment-17210353
]
Hudson commented on HBASE-23959:
Results for branch branch-2
[build #72 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/72/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/72/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/72/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/72/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/72/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix javadoc for JDK11
> -
>
> Key: HBASE-23959
> URL: https://issues.apache.org/jira/browse/HBASE-23959
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha-1, 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Semen Komissarov
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Javadoc build fails with JDK11. See if this can be fixed to pass on both 8
> and 11.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25160) Refactor AccessController and VisibilityController
[
https://issues.apache.org/jira/browse/HBASE-25160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210354#comment-17210354
]
Hudson commented on HBASE-25160:
Results for branch branch-2
[build #72 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/72/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/72/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/72/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/72/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/72/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Refactor AccessController and VisibilityController
> --
>
> Key: HBASE-25160
> URL: https://issues.apache.org/jira/browse/HBASE-25160
> Project: HBase
> Issue Type: Improvement
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> After HBASE-24602, the batchMutate() related coprocessor methods of
> RegionObserver are called when increment/append operations are performed. We
> can refactor AccessController and VisibilityController to make use of that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #1549: HBASE-24025: Improve performance of move_servers_rsgroup by using async region move API
saintstack merged pull request #1549: URL: https://github.com/apache/hbase/pull/1549 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-24025) Improve performance of move_servers_rsgroup and move_tables_rsgroup by using async region move API
[ https://issues.apache.org/jira/browse/HBASE-24025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-24025. --- Fix Version/s: 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks for patch [~arshad.mohammad] > Improve performance of move_servers_rsgroup and move_tables_rsgroup by using > async region move API > -- > > Key: HBASE-24025 > URL: https://issues.apache.org/jira/browse/HBASE-24025 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Reporter: Mohammad Arshad >Assignee: Mohammad Arshad >Priority: Major > Fix For: 3.0.0-alpha-1 > > > Currently move_servers_rsgroup and move_tables_rsgroup commands and APIs are > taking lot of time. > In my test environment, to move a server with 100 regions it takes around 137 > seconds. > Similarly it takes around same time to move a table with 100 regions to other > group. > The time taken in rsgroup meta update is negligible. Almost all the time is > taken in region moment. This is happening because region is moved serially > using getAssignmentManager().move(region) API > API getAssignmentManager().moveAsync(regionplan) can be used to move the > regions in parallel to improve the performance of region group move servers > and tables commands and APIs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on pull request #1549: HBASE-24025: Improve performance of move_servers_rsgroup by using async region move API
saintstack commented on pull request #1549: URL: https://github.com/apache/hbase/pull/1549#issuecomment-705716803 Hmm... the build passed: `03:46:34 +1 overall 03:46:34 03:46:34 | Vote | Subsystem | Runtime | Comment 03:46:34 03:46:34 | 0 |reexec | 0m 27s | Docker mode activated. 03:46:34 | -0 | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file 03:46:34 | ||| --spotbugs-strict-precheck 03:46:34 | ||| --whitespace-eol-ignore-list 03:46:34 | ||| --whitespace-tabs-ignore-list 03:46:34 | ||| --quick-hadoopcheck 03:46:34 | ||| Prechecks 03:46:34 | ||| master Compile Tests 03:46:34 | +1 |mvninstall | 3m 50s | master passed 03:46:34 | +1 | compile | 0m 56s | master passed 03:46:34 | +1 |shadedjars | 6m 36s | branch has no errors when building our 03:46:34 | ||| shaded downstream artifacts. 03:46:34 | +1 | javadoc | 0m 40s | master passed 03:46:34 | ||| Patch Compile Tests 03:46:34 | +1 |mvninstall | 3m 31s | the patch passed 03:46:34 | +1 | compile | 0m 56s | the patch passed 03:46:34 | +1 | javac | 0m 56s | the patch passed 03:46:34 | +1 |shadedjars | 6m 32s | patch has no errors when building our 03:46:34 | ||| shaded downstream artifacts. 03:46:34 | +1 | javadoc | 0m 37s | the patch passed 03:46:34 | ||| Other Tests 03:46:34 | +1 | unit | 135m 18s | hbase-server in the patch passed. 03:46:34 | || 161m 28s | 03:46:34 ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25034) Table regions details on master GUI display slowly.
[ https://issues.apache.org/jira/browse/HBASE-25034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DingSheng Li updated HBASE-25034: - Attachment: 1.txt > Table regions details on master GUI display slowly. > --- > > Key: HBASE-25034 > URL: https://issues.apache.org/jira/browse/HBASE-25034 > Project: HBase > Issue Type: Improvement >Reporter: DingSheng Li >Priority: Major > Labels: newbie > > When a table has a large number of regions (e.g.,a single table contains more > than 100,000 regions), it takes about 20 to 30 minutes to display the table > regions on the master GUI, which is unacceptable to users. After testing, we > find that web page rendering takes up the most time,and this can be solved by > pagination query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25034) Table regions details on master GUI display slowly.
[ https://issues.apache.org/jira/browse/HBASE-25034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DingSheng Li updated HBASE-25034: - Attachment: (was: 1.txt) > Table regions details on master GUI display slowly. > --- > > Key: HBASE-25034 > URL: https://issues.apache.org/jira/browse/HBASE-25034 > Project: HBase > Issue Type: Improvement >Reporter: DingSheng Li >Priority: Major > Labels: newbie > > When a table has a large number of regions (e.g.,a single table contains more > than 100,000 regions), it takes about 20 to 30 minutes to display the table > regions on the master GUI, which is unacceptable to users. After testing, we > find that web page rendering takes up the most time,and this can be solved by > pagination query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2513: HBASE-25164 Make ModifyTableProcedure support changing meta replica c…
Apache-HBase commented on pull request #2513: URL: https://github.com/apache/hbase/pull/2513#issuecomment-705714983 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 39s | master passed | | +1 :green_heart: | compile | 1m 43s | master passed | | +1 :green_heart: | shadedjars | 6m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 15s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 1m 46s | the patch passed | | +1 :green_heart: | javac | 1m 46s | the patch passed | | +1 :green_heart: | shadedjars | 6m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 22s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 138m 2s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 1m 11s | hbase-it in the patch passed. | | | | 170m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2513/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2513 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b371031d44f5 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0d63318f10 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2513/1/testReport/ | | Max. process+thread count | 4477 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server hbase-it U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2513/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
saintstack commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705714198 If 'main' or the session name are of no value, just exclude? Any progress on the @busbey list of items from JIRA on doc'ing diffs, license changes, etc.? Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2513: HBASE-25164 Make ModifyTableProcedure support changing meta replica c…
Apache-HBase commented on pull request #2513: URL: https://github.com/apache/hbase/pull/2513#issuecomment-705711328 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 15s | master passed | | +1 :green_heart: | compile | 2m 1s | master passed | | +1 :green_heart: | shadedjars | 7m 4s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 24s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 23s | the patch passed | | +1 :green_heart: | compile | 2m 16s | the patch passed | | +1 :green_heart: | javac | 2m 16s | the patch passed | | +1 :green_heart: | shadedjars | 7m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 48s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 125m 52s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 1m 6s | hbase-it in the patch passed. | | | | 163m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2513/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2513 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f1ceaadb3526 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0d63318f10 | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2513/1/testReport/ | | Max. process+thread count | 3958 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server hbase-it U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2513/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2501: HBASE-25065 WAL archival can be batched/throttled and also done by a …
saintstack commented on a change in pull request #2501:
URL: https://github.com/apache/hbase/pull/2501#discussion_r501871589
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
##
@@ -715,11 +740,39 @@ private void cleanOldLogs() throws IOException {
regionsBlockingThisWal.clear();
}
}
+
if (logsToArchive != null) {
- for (Pair logAndSize : logsToArchive) {
-this.totalLogSize.addAndGet(-logAndSize.getSecond());
-archiveLogFile(logAndSize.getFirst());
-this.walFile2Props.remove(logAndSize.getFirst());
+ final List> localLogsToArchive = logsToArchive;
+ // make it async
+ for (Pair log : localLogsToArchive) {
+logArchiveExecutor.execute(() -> {
+ archiveRetryable(log);
+});
+this.walFile2Props.remove(log.getFirst());
+ }
+}
+ }
+
+ protected void archiveRetryable(final Pair log) {
Review comment:
nit: just name it archive ? That it retries is an internal affair.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
##
@@ -482,6 +505,8 @@ protected SyncFuture initialValue() {
this.walTooOldNs = TimeUnit.SECONDS.toNanos(conf.getInt(
SURVIVED_TOO_LONG_SEC_KEY, SURVIVED_TOO_LONG_SEC_DEFAULT));
this.useHsync = conf.getBoolean(HRegion.WAL_HSYNC_CONF_KEY,
HRegion.DEFAULT_WAL_HSYNC);
+archiveRetries =
this.conf.getInt("hbase.regionserver.walroll.archive.retries", 0);
Review comment:
Default is no retries?
Will the retry ever work? Will it be regular occurence?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705702661 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 16s | master passed | | +1 :green_heart: | compile | 2m 53s | master passed | | +1 :green_heart: | shadedjars | 6m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 55s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 57s | the patch passed | | +1 :green_heart: | compile | 2m 53s | the patch passed | | +1 :green_heart: | javac | 2m 53s | the patch passed | | +1 :green_heart: | shadedjars | 6m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 56s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 160m 37s | root in the patch failed. | | | | 198m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/9/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f647a2d68de4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0d63318f10 | | Default Java | 2020-01-14 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/9/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/9/testReport/ | | Max. process+thread count | 7463 (vs. ulimit of 3) | | modules | C: hbase-shell . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/9/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2513: HBASE-25164 Make ModifyTableProcedure support changing meta replica c…
saintstack commented on a change in pull request #2513:
URL: https://github.com/apache/hbase/pull/2513#discussion_r501856397
##
File path: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
##
@@ -1131,7 +1131,20 @@
/** Conf key for enabling meta replication */
public static final String USE_META_REPLICAS = "hbase.meta.replicas.use";
public static final boolean DEFAULT_USE_META_REPLICAS = false;
+
+ /**
+ * @deprecated Since 2.4.0, will be removed in 4.0.0. Please change the meta
replicas number by
+ * altering meta table, i.e, set a new 'region replication'
number and call
+ * modifyTable.
+ */
+ @Deprecated
public static final String META_REPLICAS_NUM = "hbase.meta.replica.count";
+ /**
+ * @deprecated Since 2.4.0, will be removed in 4.0.0. Please change the meta
replicas number by
+ * altering meta table, i.e, set a new 'region replication'
number and call
+ * modifyTable.
+ */
+ @Deprecated
Review comment:
Doc needs an edit. It has section on meta replicas how-to.
##
File path: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
##
@@ -1131,7 +1131,20 @@
/** Conf key for enabling meta replication */
public static final String USE_META_REPLICAS = "hbase.meta.replicas.use";
public static final boolean DEFAULT_USE_META_REPLICAS = false;
+
+ /**
+ * @deprecated Since 2.4.0, will be removed in 4.0.0. Please change the meta
replicas number by
+ * altering meta table, i.e, set a new 'region replication'
number and call
+ * modifyTable.
+ */
+ @Deprecated
public static final String META_REPLICAS_NUM = "hbase.meta.replica.count";
+ /**
+ * @deprecated Since 2.4.0, will be removed in 4.0.0. Please change the meta
replicas number by
+ * altering meta table, i.e, set a new 'region replication'
number and call
+ * modifyTable.
+ */
+ @Deprecated
Review comment:
Grep hbase.meta.replica.count in the doc.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -1093,6 +1091,39 @@ private void
finishActiveMasterInitialization(MonitoredTask status) throws IOExc
this.assignmentManager.joinCluster();
// The below depends on hbase:meta being online.
this.tableStateManager.start();
+
+// for migrating from a version without HBASE-25099, and also for honoring
the configuration
+// first.
+if (conf.get(HConstants.META_REPLICAS_NUM) != null) {
+ int replicasNumInConf =
+conf.getInt(HConstants.META_REPLICAS_NUM,
HConstants.DEFAULT_META_REPLICA_NUM);
+ TableDescriptor metaDesc =
tableDescriptors.get(TableName.META_TABLE_NAME);
+ if (metaDesc.getRegionReplication() != replicasNumInConf) {
+// it is possible that we already have some replicas before upgrading,
so we must set the
+// region replication number in meta TableDescriptor directly first,
without creating a
+// ModifyTableProcedure, otherwise it may cause a double assign for
the meta replicas.
+int existingReplicasCount =
+
assignmentManager.getRegionStates().getRegionsOfTable(TableName.META_TABLE_NAME).size();
+if (existingReplicasCount > metaDesc.getRegionReplication()) {
+ LOG.info("Update replica count of hbase:meta from {}(in
TableDescriptor)" +
+" to {}(existing ZNodes)", metaDesc.getRegionReplication(),
existingReplicasCount);
+ metaDesc = TableDescriptorBuilder.newBuilder(metaDesc)
+.setRegionReplication(existingReplicasCount).build();
+ tableDescriptors.update(metaDesc);
+}
+// check again, and issue a ModifyTableProcedure if needed
+if (metaDesc.getRegionReplication() != replicasNumInConf) {
+ LOG.info(
+"The {} config is {} while the replica count in TableDescriptor is
{}" +
+ " for hbase:meta, altering...",
+HConstants.META_REPLICAS_NUM, replicasNumInConf,
metaDesc.getRegionReplication());
+ procedureExecutor.submitProcedure(new ModifyTableProcedure(
+procedureExecutor.getEnvironment(),
TableDescriptorBuilder.newBuilder(metaDesc)
+ .setRegionReplication(replicasNumInConf).build(),
+null, metaDesc, false));
+}
+ }
+}
Review comment:
Good
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -1157,13 +1188,6 @@ private void
finishActiveMasterInitialization(MonitoredTask status) throws IOExc
}
assignmentManager.checkIfShouldMoveSystemRegionAsync();
-status.setStatus("Assign meta replicas");
-MasterMetaBootstrap metaBootstrap = createMetaBootstrap();
-try {
- metaBootstrap.assignMetaReplicas();
-} catch (IOException | KeeperException e){
- LOG.error("Assigning meta replica
[jira] [Created] (HBASE-25166) MobFileCompactionChore is closing the master's shared cluster connection
Ankit Singhal created HBASE-25166:
-
Summary: MobFileCompactionChore is closing the master's shared
cluster connection
Key: HBASE-25166
URL: https://issues.apache.org/jira/browse/HBASE-25166
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 3.0.0-alpha-1
Reporter: Ankit Singhal
Assignee: Ankit Singhal
Code which is doing so in MobFileCompactionChore
{code:java}
try (Connection conn = master.getConnection();
Admin admin = conn.getAdmin();) { {code}
As master uses this connection to read the meta or other system tables, so none
of the meta operations through master will work.
Symptoms in master logs:-
{code:java}
s, events=841, succcessCount=123, totalEvents=12824192,
totalSuccessCount=1891300
2020-10-05 16:34:25,062 INFO
org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor: Unable to
get remote Address
2020-10-05 16:34:25,062 ERROR
org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Failed to
normalize regions.
java.io.IOException: connection is closed
at
org.apache.hadoop.hbase.MetaTableAccessor.getMetaHTable(MetaTableAccessor.java:241)
at
org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:797)
at
org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:768)
at
org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:727)
at
org.apache.hadoop.hbase.MetaTableAccessor.fullScanTables(MetaTableAccessor.java:215)
at
org.apache.hadoop.hbase.master.TableStateManager.getTablesInStates(TableStateManager.java:189)
at
org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1821)
at
org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:48)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:188)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) {code}
Symptoms at the client:-
{code:java}
RpcRetryingCaller{globalStartTime=1602099132430, pause=100, maxAttempts=11},
java.io.IOException: java.io.IOException: connection is closed
at
org.apache.hadoop.hbase.MetaTableAccessor.getMetaHTable(MetaTableAccessor.java:241)
at
org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1116)
at
org.apache.hadoop.hbase.master.TableStateManager.readMetaState(TableStateManager.java:258)
at
org.apache.hadoop.hbase.master.TableStateManager.isTablePresent(TableStateManager.java:175)
at
org.apache.hadoop.hbase.master.HMaster.getTableDescriptors(HMaster.java:3277)
at
org.apache.hadoop.hbase.master.HMaster.listTableDescriptors(HMaster.java:3221)
at
org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1064)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:418)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack closed pull request #2496: HBASE-25151 warmupRegion frustrates registering WALs on the catalog r…
saintstack closed pull request #2496: URL: https://github.com/apache/hbase/pull/2496 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2496: HBASE-25151 warmupRegion frustrates registering WALs on the catalog r…
saintstack commented on pull request #2496: URL: https://github.com/apache/hbase/pull/2496#issuecomment-705682156 Thanks @Apache9 Closing. Merged manually to HBASE-18070 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2504: HBASE-22976 [HBCK2] Add RecoveredEditsPlayer
saintstack commented on pull request #2504: URL: https://github.com/apache/hbase/pull/2504#issuecomment-705680049 Addressed @wchevreuil review. No plans to move WALPlayer to hbck2. I'd added the hbck2 prefix since it tooling to do fixup (though it core). Thanks @wchevreuil This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2504: HBASE-22976 [HBCK2] Add RecoveredEditsPlayer
saintstack commented on a change in pull request #2504:
URL: https://github.com/apache/hbase/pull/2504#discussion_r501849460
##
File path:
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java
##
@@ -301,40 +303,56 @@ public WALKey getCurrentKey() throws IOException,
InterruptedException {
inpDirs.split(conf.get(WALPlayer.INPUT_FILES_SEPARATOR_KEY, ",")));
}
+ /**
+ * @param startTime If file looks like it has a timestamp in its name, we'll
check if newer
+ * or equal to this value else we will filter out the file.
If name does not
+ * seem to have a timestamp, we will just return it w/o
filtering.
+ * @param endTime If file looks like it has a timestamp in its name, we'll
check if older or equal
+ *to this value else we will filter out the file. If name
does not seem to
+ *have a timestamp, we will just return it w/o filtering.
+ */
private List getFiles(FileSystem fs, Path dir, long startTime,
long endTime)
throws IOException {
List result = new ArrayList<>();
LOG.debug("Scanning " + dir.toString() + " for WAL files");
-
RemoteIterator iter = fs.listLocatedStatus(dir);
-if (!iter.hasNext()) return Collections.emptyList();
+if (!iter.hasNext()) {
+ return Collections.emptyList();
+}
while (iter.hasNext()) {
LocatedFileStatus file = iter.next();
if (file.isDirectory()) {
-// recurse into sub directories
+// Recurse into sub directories
result.addAll(getFiles(fs, file.getPath(), startTime, endTime));
} else {
-String name = file.getPath().toString();
-int idx = name.lastIndexOf('.');
-if (idx > 0) {
- try {
-long fileStartTime = Long.parseLong(name.substring(idx+1));
-if (fileStartTime <= endTime) {
- LOG.info("Found: " + file);
- result.add(file);
-}
- } catch (NumberFormatException x) {
-idx = 0;
- }
-}
-if (idx == 0) {
- LOG.warn("File " + name + " does not appear to be an WAL file.
Skipping...");
-}
+addFile(result, file, startTime, endTime);
}
}
+// TODO: These results should be sorted? Results could be content of
recovered.edits directory
+// -- null padded increasing numeric -- or a WAL file w/ timestamp suffix
or timestamp and
+// then meta suffix. See AbstractFSWALProvider#WALStartTimeComparator
return result;
}
+ static void addFile(List result, LocatedFileStatus lfs, long
startTime,
+ long endTime) {
+long timestamp = WAL.getTimestamp(lfs.getPath().getName());
+if (timestamp > 0) {
+ // Looks like a valid timestamp.
+ if (timestamp <= endTime && timestamp >= startTime) {
+LOG.info("Found {}", lfs.getPath());
+result.add(lfs);
+ } else {
+LOG.debug("Skipped {}, outside range [{}/{} - {}/{}]", lfs.getPath(),
Review comment:
Made it info.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2504: HBASE-22976 [HBCK2] Add RecoveredEditsPlayer
saintstack commented on a change in pull request #2504:
URL: https://github.com/apache/hbase/pull/2504#discussion_r501849236
##
File path:
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java
##
@@ -301,40 +303,56 @@ public WALKey getCurrentKey() throws IOException,
InterruptedException {
inpDirs.split(conf.get(WALPlayer.INPUT_FILES_SEPARATOR_KEY, ",")));
}
+ /**
+ * @param startTime If file looks like it has a timestamp in its name, we'll
check if newer
+ * or equal to this value else we will filter out the file.
If name does not
+ * seem to have a timestamp, we will just return it w/o
filtering.
+ * @param endTime If file looks like it has a timestamp in its name, we'll
check if older or equal
+ *to this value else we will filter out the file. If name
does not seem to
+ *have a timestamp, we will just return it w/o filtering.
+ */
private List getFiles(FileSystem fs, Path dir, long startTime,
long endTime)
throws IOException {
List result = new ArrayList<>();
LOG.debug("Scanning " + dir.toString() + " for WAL files");
-
RemoteIterator iter = fs.listLocatedStatus(dir);
-if (!iter.hasNext()) return Collections.emptyList();
+if (!iter.hasNext()) {
+ return Collections.emptyList();
+}
while (iter.hasNext()) {
LocatedFileStatus file = iter.next();
if (file.isDirectory()) {
-// recurse into sub directories
+// Recurse into sub directories
result.addAll(getFiles(fs, file.getPath(), startTime, endTime));
} else {
-String name = file.getPath().toString();
-int idx = name.lastIndexOf('.');
-if (idx > 0) {
- try {
-long fileStartTime = Long.parseLong(name.substring(idx+1));
-if (fileStartTime <= endTime) {
- LOG.info("Found: " + file);
- result.add(file);
-}
- } catch (NumberFormatException x) {
-idx = 0;
- }
-}
-if (idx == 0) {
- LOG.warn("File " + name + " does not appear to be an WAL file.
Skipping...");
-}
+addFile(result, file, startTime, endTime);
}
}
+// TODO: These results should be sorted? Results could be content of
recovered.edits directory
+// -- null padded increasing numeric -- or a WAL file w/ timestamp suffix
or timestamp and
+// then meta suffix. See AbstractFSWALProvider#WALStartTimeComparator
return result;
}
+ static void addFile(List result, LocatedFileStatus lfs, long
startTime,
+ long endTime) {
+long timestamp = WAL.getTimestamp(lfs.getPath().getName());
+if (timestamp > 0) {
+ // Looks like a valid timestamp.
+ if (timestamp <= endTime && timestamp >= startTime) {
+LOG.info("Found {}", lfs.getPath());
+result.add(lfs);
+ } else {
+LOG.debug("Skipped {}, outside range [{}/{} - {}/{}]", lfs.getPath(),
+ startTime, Instant.ofEpochMilli(startTime), endTime,
Instant.ofEpochMilli(endTime));
+ }
+} else {
+ // If no timestamp, add it regardless.
+ LOG.info("Found {}", lfs);
Review comment:
Makes sense.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (HBASE-25165) Change 'State time' in UI so sorts
[ https://issues.apache.org/jira/browse/HBASE-25165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25165. --- Fix Version/s: 2.4.0 3.0.0-alpha-1 Hadoop Flags: Reviewed Release Note: Start time on the Master UI is now displayed using ISO8601 format instead of java Date#toString(). Resolution: Fixed Merged to master and backported to branch-2. Thanks for review [~ndimiduk] > Change 'State time' in UI so sorts > -- > > Key: HBASE-25165 > URL: https://issues.apache.org/jira/browse/HBASE-25165 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > Attachments: Screen Shot 2020-10-07 at 4.15.32 PM.png, Screen Shot > 2020-10-07 at 4.15.42 PM.png > > > Here is a minor issue. > I had an issue w/ crashing servers. The servers were auto-restarted on crash. > To find the crashing servers, I was sorting on the 'Start time' column in the > Master UI. This basically worked but the sort is unreliable as the date we > display starts with days-of-the-week. > This issue is about moving to display start time in iso8601 which is sortable > (and occupies less real estate). Let me add some images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #2508: HBASE-25165 Change 'State time' in UI so sorts
saintstack merged pull request #2508: URL: https://github.com/apache/hbase/pull/2508 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] wchevreuil commented on pull request #77: HBASE-25161 [hbase-meta-repair] Add tool for repairing metadata.
wchevreuil commented on pull request #77: URL: https://github.com/apache/hbase-operator-tools/pull/77#issuecomment-705666452 > > > There is no `HbckService` in versions of hbase _before 2.0.3 and 2.1.1_. HBCK2 will not work against these versions. If users use Hbase 2.1.0 version, you can use this tool to repair metadata. > > > > > > Have you tried use these functions and got some error? If so, we should fix that, as neither `addFsRegionsMissingInMeta` nor `extraRegionsInMeta` rely on HbckService, they perform fixes on their own, thus, there should be no hbase version validation when using these two functions, and they should be compatible with the above versions. > > Hi wchevreuil, although both `addFsRegionsMissingInMeta` and `extraRegionsInMeta` do not rely on HbckService, but `assigns` command rely on HbckService... So I think this tool is very useful. > Hi wchevreuil, although both `addFsRegionsMissingInMeta` and `extraRegionsInMeta` do not rely on HbckService, but `assigns` command rely on HbckService... So I think this tool is very useful. Then the only extra bit is a bulk assignment? Could it reuse `addFsRegionsMissingInMeta` and `extraRegionsInMeta`, instead of providing extra lines of code that do same as the already existing methods? I do agree the command proposed here is very convenient, but we should re-use already existing code as much as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2513: HBASE-25164 Make ModifyTableProcedure support changing meta replica c…
Apache-HBase commented on pull request #2513: URL: https://github.com/apache/hbase/pull/2513#issuecomment-705641388 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 41s | master passed | | +1 :green_heart: | checkstyle | 1m 46s | master passed | | +1 :green_heart: | spotbugs | 2m 39s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 24s | the patch passed | | -0 :warning: | checkstyle | 1m 6s | hbase-server: The patch generated 5 new + 110 unchanged - 0 fixed = 115 total (was 110) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 10s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 42m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2513/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2513 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux cb2c1d3ceab2 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0d63318f10 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2513/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server hbase-it U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2513/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nkalmar commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
nkalmar commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705638394 Okay, I set the session name back to main. But I cheated. Basically I hardcoded the session name to be main. Is this ok? with the patch it changed how it gets the session name. How I understand is it works by getting the top level class. It used to be main, but now it's the object we use to send command to hbase. Here's the hardcoded change: `:PROMPT_I => "%N(main):%03n:%i> ",` And it should look like something like this, if we don't want to hardcode "main" into it: `%N(%m):%03n:%i> ` So %m is getting the session name, but not the one we want. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nkalmar commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
nkalmar commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705630401 I only found a way to change the format, I didn't see any way to give a name myself. It gets it's workspace name from the underlying object. I don't know why this changed from main to the hbase instantiated command object. For example, I can set it to simple which will not print any session name: `@CONF[:PROMPT_MODE] = :SIMPLE` With this prompt will be only a '>>' I will try if I can set it up with a custom prompt, so far I didn't figure out how. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2501: HBASE-25065 WAL archival can be batched/throttled and also done by a …
Apache-HBase commented on pull request #2501: URL: https://github.com/apache/hbase/pull/2501#issuecomment-705629909 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | compile | 1m 0s | master passed | | +1 :green_heart: | shadedjars | 7m 18s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 1s | the patch passed | | +1 :green_heart: | compile | 0m 59s | the patch passed | | +1 :green_heart: | javac | 0m 59s | the patch passed | | +1 :green_heart: | shadedjars | 7m 13s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 201m 58s | hbase-server in the patch passed. | | | | 230m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2501 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 368f7bf1508a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / aff8bbf0cb | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/3/testReport/ | | Max. process+thread count | 2834 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2501: HBASE-25065 WAL archival can be batched/throttled and also done by a …
Apache-HBase commented on pull request #2501: URL: https://github.com/apache/hbase/pull/2501#issuecomment-705624687 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 49s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 7m 23s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 29s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | shadedjars | 7m 20s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 191m 14s | hbase-server in the patch passed. | | | | 222m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2501 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 06ca08fe1bf7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / aff8bbf0cb | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/3/testReport/ | | Max. process+thread count | 3461 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #2513: HBASE-25164 Make ModifyTableProcedure support changing meta replica c…
Apache9 commented on pull request #2513: URL: https://github.com/apache/hbase/pull/2513#issuecomment-705614132 This is the final piece of HBASE-25099, after this PR gets merged, we could use alter table to change the meta replica count. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 opened a new pull request #2513: HBASE-25164 Make ModifyTableProcedure support changing meta replica c…
Apache9 opened a new pull request #2513: URL: https://github.com/apache/hbase/pull/2513 …ount This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705605412 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 17s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | rubocop | 0m 7s | There were no new rubocop issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 23m 49s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 36m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/9/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | dupname asflicense rubocop hadoopcheck xml | | uname | Linux f1c7e0e16c9d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0d63318f10 | | Max. process+thread count | 63 (vs. ulimit of 3) | | modules | C: . hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/9/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) rubocop=0.80.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25124) Support changing region replica count without disabling table
[ https://issues.apache.org/jira/browse/HBASE-25124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25124: -- Fix Version/s: 2.4.0 3.0.0-alpha-1 > Support changing region replica count without disabling table > - > > Key: HBASE-25124 > URL: https://issues.apache.org/jira/browse/HBASE-25124 > Project: HBase > Issue Type: Sub-task > Components: meta, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25124) Support changing region replica count without disabling table
[ https://issues.apache.org/jira/browse/HBASE-25124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25124. --- Hadoop Flags: Reviewed Resolution: Fixed Pushed to master and branch-2. Thanks [~stack] for reviewing. > Support changing region replica count without disabling table > - > > Key: HBASE-25124 > URL: https://issues.apache.org/jira/browse/HBASE-25124 > Project: HBase > Issue Type: Sub-task > Components: meta, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] nkalmar edited a comment on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
nkalmar edited a comment on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705580405 Cancel that, this is not working, this config not only changes session name but the object as well, it's no longer hbase but basically a "main" string... reverting This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nkalmar edited a comment on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
nkalmar edited a comment on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705580405 I found a config that sets session name. It should have no impact on the prompt, I tested with my usual process (create/list/put/scan commands). I didn't see any issue. I'm doing a fresh new build from this PR and test again to do my due diligence. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nkalmar commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
nkalmar commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-705580405 I found a config that sets session name. It should have no impact on the prompt, I tested with my usual process (create/list/put/scan commands). I didn't see any issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-14067) bundle ruby files for hbase shell into a jar.
[ https://issues.apache.org/jira/browse/HBASE-14067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned HBASE-14067: --- Assignee: Sean Busbey > bundle ruby files for hbase shell into a jar. > - > > Key: HBASE-14067 > URL: https://issues.apache.org/jira/browse/HBASE-14067 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > We currently package all the ruby scripts for the hbase shell by placing them > in a directory within lib/. We should be able to put these in a jar file > since we rely on jruby. -- This message was sent by Atlassian Jira (v8.3.4#803005)
