[GitHub] [hbase] Apache-HBase commented on pull request #5335: HBASE-27981 Add connection and request attributes to slow log
Apache-HBase commented on PR #5335: URL: https://github.com/apache/hbase/pull/5335#issuecomment-1720708544 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 15s | master passed | | +1 :green_heart: | compile | 1m 31s | master passed | | +1 :green_heart: | shadedjars | 4m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 3s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | compile | 1m 30s | the patch passed | | +1 :green_heart: | javac | 1m 30s | the patch passed | | -1 :x: | shadedjars | 0m 6s | patch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 407m 53s | root in the patch passed. | | | | 431m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/8/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5335 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 346f426cb9f0 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Temurin-1.8.0_352-b08 | | shadedjars | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/8/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/8/testReport/ | | Max. process+thread count | 8441 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/8/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5335: HBASE-27981 Add connection and request attributes to slow log
Apache-HBase commented on PR #5335: URL: https://github.com/apache/hbase/pull/5335#issuecomment-1720434655 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 37s | master passed | | +1 :green_heart: | compile | 1m 55s | master passed | | +1 :green_heart: | shadedjars | 4m 37s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 42s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 31s | the patch passed | | +1 :green_heart: | compile | 1m 54s | the patch passed | | +1 :green_heart: | javac | 1m 54s | the patch passed | | -1 :x: | shadedjars | 0m 8s | patch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 275m 32s | root in the patch passed. | | | | 301m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5335 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 04de998935ca 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | shadedjars | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/8/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/8/testReport/ | | Max. process+thread count | 8380 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/8/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28086) Debian hbase package
[ https://issues.apache.org/jira/browse/HBASE-28086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765406#comment-17765406 ] Duo Zhang commented on HBASE-28086: --- Thanks for the offer. Do we have any documentation about how to generate the apt-get packages? And do we need to set up a website to host these packages? Or we can upload them to an exist debian source? > Debian hbase package > > > Key: HBASE-28086 > URL: https://issues.apache.org/jira/browse/HBASE-28086 > Project: HBase > Issue Type: Improvement > Components: Usability > Environment: Google Cloud Dataproc 2.1+ > https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-2.1 >Reporter: C.J. Collier >Priority: Major > Original Estimate: 4,032h > Remaining Estimate: 4,032h > > Hi team, > There doesn't seem to be an hbase package in debian any longer. Our > instructions on how to use bigtable on our product depend on hbase. I am > build-from-source logic into the installer, but would prefer to install a > package with apt-get instead. Can assist with generating debian/rules > C.J. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache9 commented on pull request #5390: HBASE-27993. AbstractFSWAL causes ArithmeticException due to improper logRollSize value checking
Apache9 commented on PR #5390: URL: https://github.com/apache/hbase/pull/5390#issuecomment-1720345639 Thanks for providing lots of PRs to make HBase more robust. Could you please open an umbrella issue first, and then convert all the related issues as sub tasks of this big issue, so it is easier to track these issues? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28076) NPE on initialization error in RecoveredReplicationSourceShipper
[
https://issues.apache.org/jira/browse/HBASE-28076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765405#comment-17765405
]
Duo Zhang commented on HBASE-28076:
---
IIRC the big refactoring had also been done on branch-2 as well. Will take a
look at the code for master and branch-3 today.
Thanks for providing the patch and the reminding.
> NPE on initialization error in RecoveredReplicationSourceShipper
>
>
> Key: HBASE-28076
> URL: https://issues.apache.org/jira/browse/HBASE-28076
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.6.0, 2.4.17, 2.5.5
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 2.5.6
>
>
> When we run into problems starting RecoveredReplicationSourceShipper, we try
> to stop the reader thread which we haven't initialized yet, resulting in an
> NPE.
> {noformat}
> ERROR org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Unexpected exception in redacted currentPath=hdfs://redacted
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.replication.regionserver.RecoveredReplicationSourceShipper.terminate(RecoveredReplicationSourceShipper.java:100)
> at
> org.apache.hadoop.hbase.replication.regionserver.RecoveredReplicationSourceShipper.getRecoveredQueueStartPos(RecoveredReplicationSourceShipper.java:87)
> at
> org.apache.hadoop.hbase.replication.regionserver.RecoveredReplicationSourceShipper.getStartPosition(RecoveredReplicationSourceShipper.java:62)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.lambda$tryStartNewShipper$3(ReplicationSource.java:349)
> at
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.tryStartNewShipper(ReplicationSource.java:341)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:601)
> at java.lang.Thread.run(Thread.java:750)
> {noformat}
> A simple null check should fix this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5402: HBASE-28085 Configurably use scanner timeout as rpc timeout for scanner next calls
Apache-HBase commented on PR #5402: URL: https://github.com/apache/hbase/pull/5402#issuecomment-1720318497 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 58s | branch-2 passed | | +1 :green_heart: | compile | 1m 1s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 35s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 5m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 8m 4s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 202m 58s | hbase-server in the patch passed. | | | | 235m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5402 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5b2a84c32d3c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3b34bc2aad | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/3/testReport/ | | Max. process+thread count | 4908 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5402: HBASE-28085 Configurably use scanner timeout as rpc timeout for scanner next calls
Apache-HBase commented on PR #5402: URL: https://github.com/apache/hbase/pull/5402#issuecomment-1720318380 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 34s | branch-2 passed | | +1 :green_heart: | compile | 0m 52s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 47s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 19s | the patch passed | | +1 :green_heart: | compile | 0m 53s | the patch passed | | +1 :green_heart: | javac | 0m 53s | the patch passed | | +1 :green_heart: | shadedjars | 4m 44s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 7m 54s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 204m 51s | hbase-server in the patch passed. | | | | 235m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5402 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bd6860a7919b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3b34bc2aad | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/3/testReport/ | | Max. process+thread count | 4714 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-28076) NPE on initialization error in RecoveredReplicationSourceShipper
[
https://issues.apache.org/jira/browse/HBASE-28076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani resolved HBASE-28076.
--
Fix Version/s: 2.6.0
2.4.18
2.5.6
Hadoop Flags: Reviewed
Resolution: Fixed
> NPE on initialization error in RecoveredReplicationSourceShipper
>
>
> Key: HBASE-28076
> URL: https://issues.apache.org/jira/browse/HBASE-28076
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.6.0, 2.4.17, 2.5.5
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 2.5.6
>
>
> When we run into problems starting RecoveredReplicationSourceShipper, we try
> to stop the reader thread which we haven't initialized yet, resulting in an
> NPE.
> {noformat}
> ERROR org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Unexpected exception in redacted currentPath=hdfs://redacted
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.replication.regionserver.RecoveredReplicationSourceShipper.terminate(RecoveredReplicationSourceShipper.java:100)
> at
> org.apache.hadoop.hbase.replication.regionserver.RecoveredReplicationSourceShipper.getRecoveredQueueStartPos(RecoveredReplicationSourceShipper.java:87)
> at
> org.apache.hadoop.hbase.replication.regionserver.RecoveredReplicationSourceShipper.getStartPosition(RecoveredReplicationSourceShipper.java:62)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.lambda$tryStartNewShipper$3(ReplicationSource.java:349)
> at
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.tryStartNewShipper(ReplicationSource.java:341)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:601)
> at java.lang.Thread.run(Thread.java:750)
> {noformat}
> A simple null check should fix this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] virajjasani commented on pull request #5397: HBASE-28076 NPE on initialization error in RecoveredReplicationSource…
virajjasani commented on PR #5397: URL: https://github.com/apache/hbase/pull/5397#issuecomment-1720300433 Thank you @stoty! merged to all active 2.x branches -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani merged pull request #5397: HBASE-28076 NPE on initialization error in RecoveredReplicationSource…
virajjasani merged PR #5397: URL: https://github.com/apache/hbase/pull/5397 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5335: HBASE-27981 Add connection and request attributes to slow log
Apache-HBase commented on PR #5335: URL: https://github.com/apache/hbase/pull/5335#issuecomment-1720284179 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 13s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 35s | master passed | | +1 :green_heart: | compile | 4m 41s | master passed | | +1 :green_heart: | checkstyle | 1m 3s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 11m 35s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 32s | the patch passed | | +1 :green_heart: | compile | 4m 41s | the patch passed | | +1 :green_heart: | cc | 4m 41s | the patch passed | | +1 :green_heart: | javac | 4m 41s | the patch passed | | +1 :green_heart: | checkstyle | 1m 4s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 8m 51s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | hbaseprotoc | 3m 26s | the patch passed | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 12m 10s | the patch passed | ||| _ Other Tests _ | | -1 :x: | asflicense | 0m 40s | The patch generated 1 ASF License warnings. | | | | 61m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/8/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5335 | | Optional Tests | dupname asflicense spotless javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux 6503aa0aca10 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | asflicense | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/8/artifact/yetus-general-check/output/patch-asflicense-problems.txt | | Max. process+thread count | 177 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/8/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5335: HBASE-27981 Add connection and request attributes to slow log
Apache-HBase commented on PR #5335: URL: https://github.com/apache/hbase/pull/5335#issuecomment-1720236921 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 2s | master passed | | +1 :green_heart: | compile | 1m 52s | master passed | | +1 :green_heart: | shadedjars | 5m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 31s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 48s | the patch passed | | +1 :green_heart: | compile | 1m 55s | the patch passed | | +1 :green_heart: | javac | 1m 55s | the patch passed | | -1 :x: | shadedjars | 0m 8s | patch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 43s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 385m 54s | root in the patch failed. | | | | 412m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5335 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4ce9317b1817 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Temurin-1.8.0_352-b08 | | shadedjars | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/testReport/ | | Max. process+thread count | 5206 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5402: HBASE-28085 Configurably use scanner timeout as rpc timeout for scanner next calls
Apache-HBase commented on PR #5402: URL: https://github.com/apache/hbase/pull/5402#issuecomment-1720149502 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 49s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 58s | branch-2 passed | | +1 :green_heart: | compile | 3m 12s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 55s | branch-2 passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 17s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 36s | the patch passed | | +1 :green_heart: | compile | 3m 10s | the patch passed | | +1 :green_heart: | javac | 3m 10s | the patch passed | | +1 :green_heart: | checkstyle | 0m 54s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 14s | Patch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 34s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. | | | | 36m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5402 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 23a6eae5c60a 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3b34bc2aad | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/3/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28079) Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
[
https://issues.apache.org/jira/browse/HBASE-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765360#comment-17765360
]
Hudson commented on HBASE-28079:
Results for branch master
[build #904 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/904/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/904/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/904/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/904/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Unhandled TableExistsException and NamespaceExistException in
> BackupSystemTable
> ---
>
> Key: HBASE-28079
> URL: https://issues.apache.org/jira/browse/HBASE-28079
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> Most backup-related operations instantiate a BackupSystemTable object. The
> first thing it does is check that the {{backup}} namespace and backup-related
> tables exist, and create them if not. One way in which this can be triggered
> is via LoadIncrementalHFiles, which can load files with a configurable number
> of threads. When bulkLoadHFile is called, a preCommitHFile hook is triggered
> which does the above. With a certain level of concurrency, one can easily
> trigger a race condition on creating the namespace or tables. This exception
> bubbles up and fails the bulkload.
> We should catch TableExistsException and NamespaceExistException in
> BackupSystemTable initialization so that these do not cause client pain.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28058) HMaster snapshot file clean thread and the snapshot request handler thread encountered a deadlock
[
https://issues.apache.org/jira/browse/HBASE-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765359#comment-17765359
]
Hudson commented on HBASE-28058:
Results for branch master
[build #904 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/904/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/904/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/904/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/904/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HMaster snapshot file clean thread and the snapshot request handler thread
> encountered a deadlock
> -
>
> Key: HBASE-28058
> URL: https://issues.apache.org/jira/browse/HBASE-28058
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-beta-1
>Reporter: Haiping lv
>Assignee: Haiping lv
>Priority: Major
> Fix For: 3.0.0
>
>
> HMaster encountered a deadlock when runing ITBLL test.
> The handler thread of HMaster holds the synchronized lock of SnapshotManager
> and competes for the readLock of takingSnapshotLock.
> {code:java}
> Thread 57 (RpcServer.default.FPBQ.Fifo.handler=22,queue=1,port=16000):
> State: WAITING
> Blocked count: 14
> Waited count: 206
> Waiting on
> java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@a7a58d4
> Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
>
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:675)
>
> org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1765)
>
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437)
> org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) {code}
> The Master Snapshot cleaning thread holds the writeLock of takingSnapshotLock
> to compete for the synchronized lock of SnapshotManager.
> {code:java}
> Thread 5405 (hfile_cleaner-dir-scan-pool-2):
> State: BLOCKED
> Blocked count: 50535
> Waited count: 92200
> Blocked on org.apache.hadoop.hbase.master.snapshot.SnapshotManager@21a16019
> Blocked by 57 (RpcServer.default.FPBQ.Fifo.handler=22,queue=1,port=16000)
> Stack:
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isTakingAnySnapshot(SnapshotManager.java:655)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache.getUnreferencedFiles(SnapshotFileCache.java:197)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner.getDeletableFiles(SnapshotHFileCleaner.java:78)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:375)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$1(CleanerChore.java:483)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$$Lambda$788/1121007598.act(Unknown
> Source)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:565)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:483)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$2(CleanerChore.java:495)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$$Lambda$794/1250580839.run(Unknown
> Source)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
>
[GitHub] [hbase] Apache-HBase commented on pull request #5402: HBASE-28085 Configurably use scanner timeout as rpc timeout for scanner next calls
Apache-HBase commented on PR #5402: URL: https://github.com/apache/hbase/pull/5402#issuecomment-1720095773 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 43s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 9s | branch-2 passed | | +1 :green_heart: | compile | 1m 7s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 48s | the patch passed | | +1 :green_heart: | compile | 1m 7s | the patch passed | | +1 :green_heart: | javac | 1m 7s | the patch passed | | +1 :green_heart: | shadedjars | 5m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 8m 12s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 209m 57s | hbase-server in the patch passed. | | | | 244m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5402 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux dd0c73b0fc90 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3b34bc2aad | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/2/testReport/ | | Max. process+thread count | 4923 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5402: HBASE-28085 Configurably use scanner timeout as rpc timeout for scanner next calls
Apache-HBase commented on PR #5402: URL: https://github.com/apache/hbase/pull/5402#issuecomment-1720083564 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 8s | branch-2 passed | | +1 :green_heart: | compile | 0m 53s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 45s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 12s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | shadedjars | 4m 43s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 7m 51s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 205m 30s | hbase-server in the patch passed. | | | | 235m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5402 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 136f98a8d376 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3b34bc2aad | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/2/testReport/ | | Max. process+thread count | 4499 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5335: HBASE-27981 Add connection and request attributes to slow log
Apache-HBase commented on PR #5335: URL: https://github.com/apache/hbase/pull/5335#issuecomment-1720062446 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 45s | master passed | | +1 :green_heart: | compile | 1m 54s | master passed | | +1 :green_heart: | shadedjars | 4m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 40s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 38s | the patch passed | | +1 :green_heart: | compile | 1m 57s | the patch passed | | +1 :green_heart: | javac | 1m 57s | the patch passed | | -1 :x: | shadedjars | 0m 8s | patch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 40s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 237m 32s | root in the patch failed. | | | | 264m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5335 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 32a6e2f6d434 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | shadedjars | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/testReport/ | | Max. process+thread count | 5547 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28079) Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
[
https://issues.apache.org/jira/browse/HBASE-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765347#comment-17765347
]
Hudson commented on HBASE-28079:
Results for branch branch-3
[build #45 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/45/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/45/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/45/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/45/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Unhandled TableExistsException and NamespaceExistException in
> BackupSystemTable
> ---
>
> Key: HBASE-28079
> URL: https://issues.apache.org/jira/browse/HBASE-28079
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> Most backup-related operations instantiate a BackupSystemTable object. The
> first thing it does is check that the {{backup}} namespace and backup-related
> tables exist, and create them if not. One way in which this can be triggered
> is via LoadIncrementalHFiles, which can load files with a configurable number
> of threads. When bulkLoadHFile is called, a preCommitHFile hook is triggered
> which does the above. With a certain level of concurrency, one can easily
> trigger a race condition on creating the namespace or tables. This exception
> bubbles up and fails the bulkload.
> We should catch TableExistsException and NamespaceExistException in
> BackupSystemTable initialization so that these do not cause client pain.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28058) HMaster snapshot file clean thread and the snapshot request handler thread encountered a deadlock
[
https://issues.apache.org/jira/browse/HBASE-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765346#comment-17765346
]
Hudson commented on HBASE-28058:
Results for branch branch-3
[build #45 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/45/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/45/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/45/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/45/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HMaster snapshot file clean thread and the snapshot request handler thread
> encountered a deadlock
> -
>
> Key: HBASE-28058
> URL: https://issues.apache.org/jira/browse/HBASE-28058
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-beta-1
>Reporter: Haiping lv
>Assignee: Haiping lv
>Priority: Major
> Fix For: 3.0.0
>
>
> HMaster encountered a deadlock when runing ITBLL test.
> The handler thread of HMaster holds the synchronized lock of SnapshotManager
> and competes for the readLock of takingSnapshotLock.
> {code:java}
> Thread 57 (RpcServer.default.FPBQ.Fifo.handler=22,queue=1,port=16000):
> State: WAITING
> Blocked count: 14
> Waited count: 206
> Waiting on
> java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@a7a58d4
> Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
>
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:675)
>
> org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1765)
>
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437)
> org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) {code}
> The Master Snapshot cleaning thread holds the writeLock of takingSnapshotLock
> to compete for the synchronized lock of SnapshotManager.
> {code:java}
> Thread 5405 (hfile_cleaner-dir-scan-pool-2):
> State: BLOCKED
> Blocked count: 50535
> Waited count: 92200
> Blocked on org.apache.hadoop.hbase.master.snapshot.SnapshotManager@21a16019
> Blocked by 57 (RpcServer.default.FPBQ.Fifo.handler=22,queue=1,port=16000)
> Stack:
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isTakingAnySnapshot(SnapshotManager.java:655)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache.getUnreferencedFiles(SnapshotFileCache.java:197)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner.getDeletableFiles(SnapshotHFileCleaner.java:78)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:375)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$1(CleanerChore.java:483)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$$Lambda$788/1121007598.act(Unknown
> Source)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:565)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:483)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$2(CleanerChore.java:495)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$$Lambda$794/1250580839.run(Unknown
> Source)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
>
[jira] [Created] (HBASE-28086) Debian hbase package
C.J. Collier created HBASE-28086: Summary: Debian hbase package Key: HBASE-28086 URL: https://issues.apache.org/jira/browse/HBASE-28086 Project: HBase Issue Type: Improvement Components: Usability Environment: Google Cloud Dataproc 2.1+ https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-2.1 Reporter: C.J. Collier Hi team, There doesn't seem to be an hbase package in debian any longer. Our instructions on how to use bigtable on our product depend on hbase. I am build-from-source logic into the installer, but would prefer to install a package with apt-get instead. Can assist with generating debian/rules C.J. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27987) HCommon get local file system fails with a casting error
[
https://issues.apache.org/jira/browse/HBASE-27987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ConfX resolved HBASE-27987.
---
Resolution: Duplicate
> HCommon get local file system fails with a casting error
>
>
> Key: HBASE-27987
> URL: https://issues.apache.org/jira/browse/HBASE-27987
> Project: HBase
> Issue Type: Bug
>Reporter: ConfX
>Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened
> After setting {{{}fs.file.impl=org.apache.hadoop.fs.RawLocalFileSystem{}}},
> trying to acquire local file system using the {{getLocal}} in
> {{org.apache.hadoop.fs}} fails with {{java.lang.ClassCastException}}
> h2. Where's the bug
> In the function {{getLocal}} of {{FileSystem}} in HCommon:
> {code:java}
> public static LocalFileSystem getLocal(Configuration conf)
> throws IOException {
> return (LocalFileSystem)get(LocalFileSystem.NAME, conf);
> } {code}
> the returned file system is directly casted to LocalFileSystem without
> checking. If the user set the implementation of the local filesystem to be
> Raw rather than Checksum, this type cast would fail.
> h2. How to reproduce
> # Set {{fs.file.impl=org.apache.hadoop.fs.RawLocalFileSystem}}
> # Run {{org.apache.hadoop.hbase.TestHBaseTestingUtility#testMiniDFSCluster}}
> and the following exception should be observed:
> {code:java}
> java.lang.ClassCastException: class org.apache.hadoop.fs.RawLocalFileSystem
> cannot be cast to class org.apache.hadoop.fs.LocalFileSystem
> (org.apache.hadoop.fs.RawLocalFileSystem and
> org.apache.hadoop.fs.LocalFileSystem are in unnamed module of loader 'app')
> at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:441)at
> org.apache.hadoop.hbase.HBaseTestingUtility.getNewDataTestDirOnTestFS(HBaseTestingUtility.java:550)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.setupDataTestDirOnTestFS(HBaseTestingUtility.java:536)
>at
> org.apache.hadoop.hbase.HBaseTestingUtility.getDataTestDirOnTestFS(HBaseTestingUtility.java:510)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.getDataTestDirOnTestFS(HBaseTestingUtility.java:524)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.createDirsAndSetProperties(HBaseTestingUtility.java:737)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:682)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:665)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:653)
> at
> org.apache.hadoop.hbase.TestHBaseTestingUtility.testMiniDFSCluster(TestHBaseTestingUtility.java:377)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28075) [hbase-thirdparty] Add shaded netty tcnative module
[ https://issues.apache.org/jira/browse/HBASE-28075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765307#comment-17765307 ] Bryan Beaudreault commented on HBASE-28075: --- I've already done this internally at my company and it works. We need to do a similar process as [this|https://github.com/apache/hbase-thirdparty/blob/master/hbase-shaded-netty/pom.xml#L94] that we do for epoll. Otherwise, system properties do change like existing system properties for netty. So you need to configure with correctly prefixed system properties, like -Dorg.apache.hbase.thirdparty.io.netty.maxDirectMemory. > [hbase-thirdparty] Add shaded netty tcnative module > --- > > Key: HBASE-28075 > URL: https://issues.apache.org/jira/browse/HBASE-28075 > Project: HBase > Issue Type: Sub-task >Reporter: Bryan Beaudreault >Priority: Major > > From HBASE-28008, we need to provide some default module that includes shaded > netty-tcnative so that users can easily get deployed with it if they want to. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5402: HBASE-28085 Configurably use scanner timeout as rpc timeout for scanner next calls
Apache-HBase commented on PR #5402: URL: https://github.com/apache/hbase/pull/5402#issuecomment-1719814190 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 50s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 9s | branch-2 passed | | +1 :green_heart: | compile | 3m 33s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 56s | branch-2 passed | | -1 :x: | spotless | 0m 44s | branch has 1 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 2m 40s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 35s | the patch passed | | +1 :green_heart: | compile | 3m 8s | the patch passed | | +1 :green_heart: | javac | 3m 7s | the patch passed | | +1 :green_heart: | checkstyle | 0m 54s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 25s | Patch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 38m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5402 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 21c5dee73c48 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3b34bc2aad | | Default Java | Eclipse Adoptium-11.0.17+8 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/2/artifact/yetus-general-check/output/branch-spotless.txt | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5335: HBASE-27981 Add connection and request attributes to slow log
Apache-HBase commented on PR #5335: URL: https://github.com/apache/hbase/pull/5335#issuecomment-1719787735 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 12s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 44s | master passed | | +1 :green_heart: | compile | 4m 44s | master passed | | +1 :green_heart: | checkstyle | 1m 6s | master passed | | +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 11m 38s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 36s | the patch passed | | +1 :green_heart: | compile | 4m 46s | the patch passed | | +1 :green_heart: | cc | 4m 46s | the patch passed | | +1 :green_heart: | javac | 4m 46s | the patch passed | | +1 :green_heart: | checkstyle | 1m 7s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 2s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | hbaseprotoc | 3m 22s | the patch passed | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 12m 10s | the patch passed | ||| _ Other Tests _ | | -1 :x: | asflicense | 0m 41s | The patch generated 1 ASF License warnings. | | | | 62m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5335 | | Optional Tests | dupname asflicense spotless javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux 6462e4a87948 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | asflicense | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/artifact/yetus-general-check/output/patch-asflicense-problems.txt | | Max. process+thread count | 176 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5335/7/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28085) Configurably use scanner timeout as rpc timeout for scanner next calls
[
https://issues.apache.org/jira/browse/HBASE-28085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault updated HBASE-28085:
--
Assignee: Bryan Beaudreault
Labels: patch-available (was: )
Status: Patch Available (was: Open)
> Configurably use scanner timeout as rpc timeout for scanner next calls
> --
>
> Key: HBASE-28085
> URL: https://issues.apache.org/jira/browse/HBASE-28085
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
>
> In the AsyncTable implementation, scanner next() calls use
> "hbase.client.scanner.timeout.period" as the rpc timeout. The reason is
> described in comments, and makes a lot of sense:
> {code:java}
> // As we have a call sequence for scan, it is useless to have a different rpc
> timeout which is
> // less than the scan timeout. If the server does not respond in time(usually
> this will not
> // happen as we have heartbeat now), we will get an
> OutOfOrderScannerNextException when
> // resending the next request and the only way to fix this is to close the
> scanner and open a
> // new one. {code}
> The branch-2 HTable implementation still uses the old behavior – the next()
> call passes the read rpc timeout as the rpc timeout, and uses the scanner
> timeout period as the operation timeout. This can lead to the above behavior.
> It would be nice to provide users a migration path to AsyncTable's behavior,
> in the form of a config flag which causes HTable to use
> "hbase.client.scanner.timeout.period" as rpc timeout for next() calls like
> AsyncTable does.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] bbeaudreault commented on a diff in pull request #5402: HBASE-28085 Configurably use scanner timeout as rpc timeout for scanner next calls
bbeaudreault commented on code in PR #5402: URL: https://github.com/apache/hbase/pull/5402#discussion_r1326202957 ## hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java: ## @@ -116,16 +117,15 @@ public ClientScanner(final Configuration conf, final Scan scan, final TableName this.connection = connection; this.pool = pool; this.primaryOperationTimeout = primaryOperationTimeout; -this.retries = conf.getInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, Review Comment: While I was here and needed a new config, I cleaned these up. Configuration getter calls introduce synchronization and parse cost, so it's better to use what we already parsed in ConnectionConfiguration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28056) [HBoss] add support for AWS v2 SDK
[ https://issues.apache.org/jira/browse/HBASE-28056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765230#comment-17765230 ] Steve Loughran commented on HBASE-28056: FYI, i have got things compiling, but mock tests are failing as the api is being used differently. HADOOP-1 makes things slightly easier, but HADOOP-18877 should provide an easier plugin point to pub the stub s3fs behind, as all calls to s3 will be behind an internal interface: what you need to implement becomes a lot clearer. > [HBoss] add support for AWS v2 SDK > -- > > Key: HBASE-28056 > URL: https://issues.apache.org/jira/browse/HBASE-28056 > Project: HBase > Issue Type: Bug > Components: hboss >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > > HBoss doesn't compile against a version of hadoop built with the AWS v2 SDK, > which HADOOP-18703 will do on hadoop trunk within a few days. > I think the solution here is probably some profile to build against different > sdk/hadoop versions -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1719674048 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 39s | master passed | | +1 :green_heart: | compile | 1m 3s | master passed | | +1 :green_heart: | shadedjars | 4m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 37s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 4m 42s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 39s | hbase-asyncfs in the patch passed. | | +1 :green_heart: | unit | 229m 11s | hbase-server in the patch passed. | | | | 255m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0aa6b07c0b52 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/4/testReport/ | | Max. process+thread count | 4454 (vs. ulimit of 3) | | modules | C: hbase-asyncfs hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/4/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5402: HBASE-28085 Configurably use scanner timeout as rpc timeout for scanner next calls
Apache-HBase commented on PR #5402: URL: https://github.com/apache/hbase/pull/5402#issuecomment-1719669984 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 54s | branch-2 passed | | +1 :green_heart: | compile | 1m 12s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 29s | branch-2 passed | | +1 :green_heart: | spotless | 1m 7s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 29s | branch-2 passed | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 1m 17s | root in the patch failed. | | -1 :x: | compile | 0m 54s | hbase-client in the patch failed. | | -0 :warning: | javac | 0m 54s | hbase-client in the patch failed. | | +1 :green_heart: | checkstyle | 0m 26s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | hadoopcheck | 1m 32s | The patch causes 17 errors with Hadoop v2.10.2. | | -1 :x: | hadoopcheck | 3m 5s | The patch causes 17 errors with Hadoop v3.2.4. | | -1 :x: | hadoopcheck | 4m 19s | The patch causes 17 errors with Hadoop v3.3.5. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | -1 :x: | spotbugs | 0m 13s | hbase-client in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 18m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5402 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux ac81b526db00 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3b34bc2aad | | Default Java | Eclipse Adoptium-11.0.17+8 | | mvninstall | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-general-check/output/patch-mvninstall-root.txt | | compile | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-general-check/output/patch-compile-hbase-client.txt | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-general-check/output/patch-compile-hbase-client.txt | | hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-general-check/output/patch-javac-2.10.2.txt | | hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-general-check/output/patch-javac-3.2.4.txt | | hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-general-check/output/patch-javac-3.3.5.txt | | spotbugs | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-general-check/output/patch-spotbugs-hbase-client.txt | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-client U: hbase-client | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5402: HBASE-28085 Configurably use scanner timeout as rpc timeout for scanner next calls
Apache-HBase commented on PR #5402: URL: https://github.com/apache/hbase/pull/5402#issuecomment-1719662806 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 34s | branch-2 passed | | +1 :green_heart: | compile | 0m 25s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | branch-2 passed | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 51s | root in the patch failed. | | -1 :x: | compile | 0m 17s | hbase-client in the patch failed. | | -0 :warning: | javac | 0m 17s | hbase-client in the patch failed. | | -1 :x: | shadedjars | 2m 12s | patch has 17 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 14s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 0m 17s | hbase-client in the patch failed. | | | | 14m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5402 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 47af5a1b4e7f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3b34bc2aad | | Default Java | Eclipse Adoptium-11.0.17+8 | | mvninstall | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-client.txt | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-client.txt | | shadedjars | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-client.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/testReport/ | | Max. process+thread count | 77 (vs. ulimit of 3) | | modules | C: hbase-client U: hbase-client | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5402: HBASE-28085 Configurably use scanner timeout as rpc timeout for scanner next calls
Apache-HBase commented on PR #5402: URL: https://github.com/apache/hbase/pull/5402#issuecomment-1719658864 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 28s | branch-2 passed | | +1 :green_heart: | compile | 0m 17s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 48s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 16s | branch-2 passed | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 40s | root in the patch failed. | | -1 :x: | compile | 0m 17s | hbase-client in the patch failed. | | -0 :warning: | javac | 0m 17s | hbase-client in the patch failed. | | -1 :x: | shadedjars | 2m 7s | patch has 17 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 14s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 0m 17s | hbase-client in the patch failed. | | | | 12m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5402 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux edbe8ef139b6 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3b34bc2aad | | Default Java | Temurin-1.8.0_352-b08 | | mvninstall | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk8-hadoop2-check/output/patch-mvninstall-root.txt | | compile | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk8-hadoop2-check/output/patch-compile-hbase-client.txt | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk8-hadoop2-check/output/patch-compile-hbase-client.txt | | shadedjars | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk8-hadoop2-check/output/patch-shadedjars.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-client.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/testReport/ | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-client U: hbase-client | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5402/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache-HBase commented on PR #5391: URL: https://github.com/apache/hbase/pull/5391#issuecomment-1719653331 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 21s | master passed | | +1 :green_heart: | compile | 1m 56s | master passed | | +1 :green_heart: | shadedjars | 5m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 7s | the patch passed | | +1 :green_heart: | compile | 1m 32s | the patch passed | | +1 :green_heart: | javac | 1m 32s | the patch passed | | +1 :green_heart: | shadedjars | 5m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 39s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 51s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 250m 54s | hbase-server in the patch passed. | | | | 285m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5391 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 95b16f51651b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/testReport/ | | Max. process+thread count | 4657 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1719650602 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 23s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 11s | master passed | | +1 :green_heart: | compile | 0m 48s | master passed | | +1 :green_heart: | shadedjars | 4m 52s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 32s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 16s | the patch passed | | +1 :green_heart: | compile | 0m 45s | the patch passed | | +1 :green_heart: | javac | 0m 45s | the patch passed | | +1 :green_heart: | shadedjars | 4m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 1s | hbase-asyncfs in the patch passed. | | +1 :green_heart: | unit | 220m 2s | hbase-server in the patch passed. | | | | 242m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8ce3ea27a8c1 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/4/testReport/ | | Max. process+thread count | 4231 (vs. ulimit of 3) | | modules | C: hbase-asyncfs hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/4/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache-HBase commented on PR #5391: URL: https://github.com/apache/hbase/pull/5391#issuecomment-1719578931 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 12s | master passed | | +1 :green_heart: | compile | 1m 4s | master passed | | +1 :green_heart: | shadedjars | 4m 53s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 46s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | compile | 1m 6s | the patch passed | | +1 :green_heart: | javac | 1m 6s | the patch passed | | +1 :green_heart: | shadedjars | 4m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 50s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 51s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 20s | hbase-client in the patch passed. | | -1 :x: | unit | 219m 36s | hbase-server in the patch failed. | | | | 245m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5391 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a736ce46156f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/testReport/ | | Max. process+thread count | 4624 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-28085) Configurably use scanner timeout as rpc timeout for scanner next calls
Bryan Beaudreault created HBASE-28085:
-
Summary: Configurably use scanner timeout as rpc timeout for
scanner next calls
Key: HBASE-28085
URL: https://issues.apache.org/jira/browse/HBASE-28085
Project: HBase
Issue Type: Improvement
Reporter: Bryan Beaudreault
In the AsyncTable implementation, scanner next() calls use
"hbase.client.scanner.timeout.period" as the rpc timeout. The reason is
described in comments, and makes a lot of sense:
{code:java}
// As we have a call sequence for scan, it is useless to have a different rpc
timeout which is
// less than the scan timeout. If the server does not respond in time(usually
this will not
// happen as we have heartbeat now), we will get an
OutOfOrderScannerNextException when
// resending the next request and the only way to fix this is to close the
scanner and open a
// new one. {code}
The branch-2 HTable implementation still uses the old behavior – the next()
call passes the read rpc timeout as the rpc timeout, and uses the scanner
timeout period as the operation timeout. This can lead to the above behavior.
It would be nice to provide users a migration path to AsyncTable's behavior, in
the form of a config flag which causes HTable to use
"hbase.client.scanner.timeout.period" as rpc timeout for next() calls like
AsyncTable does.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1719293483 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 32s | master passed | | +1 :green_heart: | compile | 2m 43s | master passed | | +1 :green_heart: | checkstyle | 0m 44s | master passed | | +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 49s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 37s | the patch passed | | +1 :green_heart: | compile | 2m 42s | the patch passed | | +1 :green_heart: | javac | 2m 42s | the patch passed | | +1 :green_heart: | checkstyle | 0m 9s | The patch passed checkstyle in hbase-asyncfs | | +1 :green_heart: | checkstyle | 0m 33s | hbase-server: The patch generated 0 new + 8 unchanged - 1 fixed = 8 total (was 9) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 0s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. | | | | 33m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 5034a2523341 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 80 (vs. ulimit of 3) | | modules | C: hbase-asyncfs hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/4/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-28084) Incremental backups should be forbidden after deleting backups
Dieter De Paepe created HBASE-28084: --- Summary: Incremental backups should be forbidden after deleting backups Key: HBASE-28084 URL: https://issues.apache.org/jira/browse/HBASE-28084 Project: HBase Issue Type: Bug Components: backuprestore Reporter: Dieter De Paepe Imagine I have a set of backups (Full1, Incr2, Incr3), delete the last backup (Incr3), and then create a new incremental backup (Incr4). This backup history will now show: Full1, Incr2, Incr4. However, restoring Incr4 will not contain the data that was captured in Incr3, effectively leading to data loss. This will certainly surprise some users. I suggest to add some internal bookkeeping to prevent incremental backups in case the most recent backup was deleted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache-HBase commented on PR #5391: URL: https://github.com/apache/hbase/pull/5391#issuecomment-1719245986 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 34s | master passed | | +1 :green_heart: | compile | 3m 37s | master passed | | +1 :green_heart: | checkstyle | 1m 8s | master passed | | +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 43s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 35s | the patch passed | | +1 :green_heart: | compile | 3m 38s | the patch passed | | +1 :green_heart: | javac | 3m 38s | the patch passed | | +1 :green_heart: | checkstyle | 1m 10s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 10s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 3m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. | | | | 38m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5391 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux e6ed2d77e7c9 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/6/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache-HBase commented on PR #5391: URL: https://github.com/apache/hbase/pull/5391#issuecomment-1719189028 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 54s | master passed | | +1 :green_heart: | compile | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 5m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 44s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | shadedjars | 5m 14s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 15s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 32s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 234m 48s | hbase-server in the patch passed. | | | | 264m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5391 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 971044ec98cb 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/testReport/ | | Max. process+thread count | 4670 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache-HBase commented on PR #5391: URL: https://github.com/apache/hbase/pull/5391#issuecomment-1719163063 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 18s | master passed | | +1 :green_heart: | compile | 1m 5s | master passed | | +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 4m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 50s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 21s | hbase-client in the patch passed. | | -1 :x: | unit | 220m 29s | hbase-server in the patch failed. | | | | 247m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5391 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5548c6912d1d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/testReport/ | | Max. process+thread count | 4609 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-28083) Failed backups block backup merges
Dieter De Paepe created HBASE-28083:
---
Summary: Failed backups block backup merges
Key: HBASE-28083
URL: https://issues.apache.org/jira/browse/HBASE-28083
Project: HBase
Issue Type: Bug
Components: backuprestore
Environment: Encountered on HBase
a2e7d2015e9f603e46339d0582e29a86843b9324 (branch-2), running in Kubernetes.
Reporter: Dieter De Paepe
I was doing some experiments with HBase backup, and came to the following
situation:
{noformat}
user@hadoop-client-769bc9946-g6p59:/$ hbase backup history
{ID=backup_1694682979468,Type=INCREMENTAL,Tables={foo:bar,foo:baz},State=COMPLETE,Start
time=Thu Sep 14 09:16:20 UTC 2023,End time=Thu Sep 14 09:16:45 UTC
2023,Progress=100%}
{ID=backup_1694682942306,Type=INCREMENTAL,Tables={foo:bar,foo:baz},State=COMPLETE,Start
time=Thu Sep 14 09:15:43 UTC 2023,End time=Thu Sep 14 09:16:11 UTC
2023,Progress=100%}
{ID=backup_1694682906267,Type=INCREMENTAL,Tables={foo:bar,foo:baz},State=COMPLETE,Start
time=Thu Sep 14 09:15:08 UTC 2023,End time=Thu Sep 14 09:15:34 UTC
2023,Progress=100%}
{ID=backup_1694682869991,Type=INCREMENTAL,Tables={foo:bar,foo:baz},State=COMPLETE,Start
time=Thu Sep 14 09:14:31 UTC 2023,End time=Thu Sep 14 09:14:58 UTC
2023,Progress=100%}
{ID=backup_1694681054166,Type=INCREMENTAL,Tables={foo:bar,foo:baz},State=FAILED,Start
time=Thu Sep 14 08:44:16 UTC 2023,Failed message=REPAIR status: repaired after
failure:
backup_1694681054166,Progress=0%}
{ID=backup_1694681006360,Type=FULL,Tables={foo:bar,foo:baz},State=COMPLETE,Start
time=Thu Sep 14 08:43:27 UTC 2023,End time=Thu Sep 14 08:43:56 UTC
2023,Progress=100%}
{noformat}
Even though I was able to create new incremental backups, I am not able to
merge the old backups, due to the failed (& repaired) backup:
{code:java}
# Failed incremental + completed incremental
user@hadoop-client-769bc9946-g6p59:/$ hbase backup merge
backup_1694681054166,backup_1694682869991
23/09/14 09:17:57 ERROR org.apache.hadoop.hbase.backup.BackupDriver: Error
running command-line tool
java.io.IOException: Backup image backup_1694681054166 can not be merged
becuase of its state: FAILED
at
org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.checkIfValidForMerge(BackupAdminImpl.java:664)
at
org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.mergeBackups(BackupAdminImpl.java:619)
at
org.apache.hadoop.hbase.backup.impl.BackupCommands$MergeCommand.execute(BackupCommands.java:834)
at
org.apache.hadoop.hbase.backup.BackupDriver.parseAndRun(BackupDriver.java:134)
at org.apache.hadoop.hbase.backup.BackupDriver.doWork(BackupDriver.java:169)
at org.apache.hadoop.hbase.backup.BackupDriver.run(BackupDriver.java:199)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82)
at org.apache.hadoop.hbase.backup.BackupDriver.main(BackupDriver.java:177)
{code}
{code:java}
# Completed full backup + failed incremental
user@hadoop-client-769bc9946-g6p59:/$ hbase backup merge
backup_1694681006360,backup_1694681054166
23/09/14 09:19:40 ERROR org.apache.hadoop.hbase.backup.BackupDriver: Error
running command-line tool
java.io.IOException: FULL backup image can not be merged for:
backup_1694681006360
at
org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.checkIfValidForMerge(BackupAdminImpl.java:659)
at
org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.mergeBackups(BackupAdminImpl.java:619)
at
org.apache.hadoop.hbase.backup.impl.BackupCommands$MergeCommand.execute(BackupCommands.java:834)
at
org.apache.hadoop.hbase.backup.BackupDriver.parseAndRun(BackupDriver.java:134)
at org.apache.hadoop.hbase.backup.BackupDriver.doWork(BackupDriver.java:169)
at org.apache.hadoop.hbase.backup.BackupDriver.run(BackupDriver.java:199)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82)
at org.apache.hadoop.hbase.backup.BackupDriver.main(BackupDriver.java:177)
{code}
I'm not sure why merging a failed backup is not possible, given that it is
possible to create new incremental backups on top of it. Either the data in
that failed backup is reliable (meaning it should be merge-able), or not
reliable (meaning it should not be possible to execute incremental backups on
top of it).
If I were to delete the failed backup, all other incremental backups would also
be deleted. This made me assume failed backups still keep track of reliable
data.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28068) Normalizer should batch merging 0 sized/empty regions
[
https://issues.apache.org/jira/browse/HBASE-28068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765070#comment-17765070
]
Rahul Kumar commented on HBASE-28068:
-
Thanks [~vjasani] Let me get a PR rolling, and we can discuss further criteria
over same.
> Normalizer should batch merging 0 sized/empty regions
> -
>
> Key: HBASE-28068
> URL: https://issues.apache.org/jira/browse/HBASE-28068
> Project: HBase
> Issue Type: Improvement
> Components: Normalizer
>Affects Versions: 2.5.5
>Reporter: Ravi Kishore Valeti
>Assignee: Rahul Kumar
>Priority: Minor
> Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1
>
>
> In our production environment, while investigating an issue, we observed that
> the Noramlizer had scheduled one single merge procedure to an RS providing
> 27K+ empty regions of a table (this was a result of a failed copy table job
> that left 27K+ empty regions of the table) to merge.
> This action led the procedure to go to stuck state and eventually the
> procedure framework bailed out after ~40mins. This was happening with each
> normalizer run until we deleted the table manually.
> Logs
> Normalizer triggers a merge procedure
> normalizer.RegionNormalizerWorker - NormalizationTarget[regionInfo=\{ENCODED
> => 6e8606335a62f6bafceb017dc7edfdf5, NAME => 'TEST.TEST_TABLE,.',
> STARTKEY => '', ENDKEY => ''},{*}regionSizeMb=0{*}],
> NormalizationTarget[regionInfo=\{ENCODED => 79607df308d7618e632abe8a12c1bf6b,
> NAME => 'TEST.TEST_TABLE,', STARTKEY => 'XXYY', ENDKEY =>
> 'YYZZ'},{*}regionSizeMb=0]{*}]] resulting in *pid 21968356*
> procedure immediately gets stuck
> procedure2.ProcedureExecutor - Worker *stuck* PEWorker-56(pid=21968356), run
> time 12.4850 sec
> Finally fails after ~40 mins
> procedure2.ProcedureExecutor - Worker *stuck* PEWorker-56(pid=21968356), run
> time *40 mins, 58.055 sec*
> Bails out with RuntimeException
> procedure2.ProcedureExecutor - force=false
> java.lang.UnsupportedOperationException: pid=21968356,
> state=FAILED:MERGE_TABLE_REGIONS_UPDATE_META, locked=true,
> exception=java.lang.{*}RuntimeException via CODE-BUG: Uncaught runtime
> exception{*}: pid=21968356, state=RUNNABLE:MERGE_TABLE_REGIONS_UPDATE_META,
> locked=true; MergeTableRegionsProcedure table=TEST.TEST_TABLE,
> {*}regions={*}{*}[269a1b168af497cce9ba6d3d581568f2{*}
> .
> .
> .
> .
> *27K+ regions printed here]*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1718953707 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 33s | master passed | | +1 :green_heart: | compile | 2m 39s | master passed | | +1 :green_heart: | checkstyle | 0m 44s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 50s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 30s | the patch passed | | +1 :green_heart: | compile | 2m 42s | the patch passed | | +1 :green_heart: | javac | 2m 42s | the patch passed | | -0 :warning: | checkstyle | 0m 8s | hbase-asyncfs: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | -0 :warning: | checkstyle | 0m 37s | hbase-server: The patch generated 2 new + 8 unchanged - 1 fixed = 10 total (was 9) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 8m 57s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. | | | | 33m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 54ebbd19993d 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-asyncfs.txt | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-asyncfs hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/3/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765040#comment-17765040
]
zhangbutao commented on HBASE-28061:
I applied your PR [https://github.com/apache/hbase/pull/5401] into Hbase2.5.5,
and recomplied code with default Hadoop dependency 3.2.4:
{code:java}
mvn clean install -DskipTests -Dhadoop.profile=3.0
-Dhadoop-three.version=3.2.4{code}
And then I ran my test case in both Hadoop3.2.4 and Hadoop3.3.1, and the
MiniHbaseCluster started successfully. Thanks for your fix!
BTW, do we have any plan & time to make the new 2.5.x release? And could we
release a new version against Hadoop3.3x? Because i am not sure if
Hbase-hadoop3.2x version can work well with all the interface of Hadoop3.3.x.
It would be nice if we could do this.
Thanks.
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HbaseMiniClusterTest.java,
> fix_LocatedBlock__getLocations.patch, full_output.txt, pom.xml, root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache9 commented on a diff in pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache9 commented on code in PR #5401: URL: https://github.com/apache/hbase/pull/5401#discussion_r1325503532 ## hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java: ## @@ -17,6 +17,8 @@ */ package org.apache.hadoop.hbase.fs; +import static org.apache.hadoop.hbase.util.LocatedBlockHelper.*; Review Comment: Seems the eclipse formatter spotless plugin will not expand the star imports automatically... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1718904985 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | master passed | | +1 :green_heart: | compile | 0m 18s | master passed | | +1 :green_heart: | checkstyle | 0m 8s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 32s | the patch passed | | +1 :green_heart: | compile | 0m 18s | the patch passed | | +1 :green_heart: | javac | 0m 18s | the patch passed | | +1 :green_heart: | checkstyle | 0m 9s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 8m 59s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. | | | | 23m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 9577a9307081 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 76 (vs. ulimit of 3) | | modules | C: hbase-asyncfs U: hbase-asyncfs | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] zhangbutao commented on a diff in pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
zhangbutao commented on code in PR #5401: URL: https://github.com/apache/hbase/pull/5401#discussion_r1325485997 ## hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java: ## @@ -17,6 +17,8 @@ */ package org.apache.hadoop.hbase.fs; +import static org.apache.hadoop.hbase.util.LocatedBlockHelper.*; Review Comment: ```suggestion import static org.apache.hadoop.hbase.util.LocatedBlockHelper.getLocatedBlockLocations;*; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1718898007 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 53s | master passed | | +1 :green_heart: | compile | 0m 16s | master passed | | +1 :green_heart: | shadedjars | 4m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 14s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 36s | the patch passed | | +1 :green_heart: | compile | 0m 15s | the patch passed | | +1 :green_heart: | javac | 0m 15s | the patch passed | | +1 :green_heart: | shadedjars | 4m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 37s | hbase-asyncfs in the patch passed. | | | | 19m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 788618e63e08 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/2/testReport/ | | Max. process+thread count | 522 (vs. ulimit of 3) | | modules | C: hbase-asyncfs U: hbase-asyncfs | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1718895434 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 13s | master passed | | +1 :green_heart: | compile | 0m 11s | master passed | | +1 :green_heart: | shadedjars | 4m 53s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 9s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 16s | the patch passed | | +1 :green_heart: | compile | 0m 11s | the patch passed | | +1 :green_heart: | javac | 0m 11s | the patch passed | | +1 :green_heart: | shadedjars | 4m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 57s | hbase-asyncfs in the patch passed. | | | | 17m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9ac9b17be77e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/2/testReport/ | | Max. process+thread count | 588 (vs. ulimit of 3) | | modules | C: hbase-asyncfs U: hbase-asyncfs | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache9 commented on code in PR #5401:
URL: https://github.com/apache/hbase/pull/5401#discussion_r1325466758
##
hbase-asyncfs/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputHelper.java:
##
@@ -383,7 +400,7 @@ private static List>
connectToDataNodes(Configuration conf, DFSC
BlockConstructionStage stage, DataChecksum summer, EventLoopGroup
eventLoopGroup,
Class channelClass) {
StorageType[] storageTypes = locatedBlock.getStorageTypes();
-DatanodeInfo[] datanodeInfos = locatedBlock.getLocations();
+DatanodeInfo[] datanodeInfos = getLocatedBlockLocations(locatedBlock);
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache9 commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1718886724 I've searched and removed all the calls to LocatedBlock.getLocations in hbase code base. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1718872112 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 50s | master passed | | +1 :green_heart: | compile | 0m 20s | master passed | | +1 :green_heart: | checkstyle | 0m 10s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | the patch passed | | +1 :green_heart: | compile | 0m 18s | the patch passed | | +1 :green_heart: | javac | 0m 18s | the patch passed | | +1 :green_heart: | checkstyle | 0m 9s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 2s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 24m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux a0dd96e0c8f8 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-asyncfs U: hbase-asyncfs | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1718864359 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | master passed | | +1 :green_heart: | compile | 0m 11s | master passed | | +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | compile | 0m 11s | the patch passed | | +1 :green_heart: | javac | 0m 11s | the patch passed | | +1 :green_heart: | shadedjars | 4m 55s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 0s | hbase-asyncfs in the patch passed. | | | | 17m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 171dfd1e5090 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/testReport/ | | Max. process+thread count | 636 (vs. ulimit of 3) | | modules | C: hbase-asyncfs U: hbase-asyncfs | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1718864306 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 10s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 51s | master passed | | +1 :green_heart: | compile | 0m 15s | master passed | | +1 :green_heart: | shadedjars | 4m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 14s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 36s | the patch passed | | +1 :green_heart: | compile | 0m 14s | the patch passed | | +1 :green_heart: | javac | 0m 14s | the patch passed | | +1 :green_heart: | shadedjars | 4m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 58s | hbase-asyncfs in the patch passed. | | | | 18m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1b4196b913eb 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/testReport/ | | Max. process+thread count | 578 (vs. ulimit of 3) | | modules | C: hbase-asyncfs U: hbase-asyncfs | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] zhangbutao commented on a diff in pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
zhangbutao commented on code in PR #5401:
URL: https://github.com/apache/hbase/pull/5401#discussion_r1325439479
##
hbase-asyncfs/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputHelper.java:
##
@@ -383,7 +400,7 @@ private static List>
connectToDataNodes(Configuration conf, DFSC
BlockConstructionStage stage, DataChecksum summer, EventLoopGroup
eventLoopGroup,
Class channelClass) {
StorageType[] storageTypes = locatedBlock.getStorageTypes();
-DatanodeInfo[] datanodeInfos = locatedBlock.getLocations();
+DatanodeInfo[] datanodeInfos = getLocatedBlockLocations(locatedBlock);
Review Comment:
There's another place to be fixed. :)
https://github.com/apache/hbase/blob/84ccae368f6ed94a751d576c59fc695c683f1c8b/hbase-asyncfs/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputHelper.java#L498
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache-HBase commented on PR #5391: URL: https://github.com/apache/hbase/pull/5391#issuecomment-1718850992 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 38s | master passed | | +1 :green_heart: | compile | 3m 37s | master passed | | +1 :green_heart: | checkstyle | 1m 6s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 45s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 35s | the patch passed | | +1 :green_heart: | compile | 3m 39s | the patch passed | | +1 :green_heart: | javac | 3m 39s | the patch passed | | -0 :warning: | checkstyle | 0m 15s | hbase-client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 10s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 3m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. | | | | 38m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5391 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 28a19edbaab4 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765019#comment-17765019
]
Duo Zhang commented on HBASE-28061:
---
Oh, just saw your patch, and there is another place need to handle.
Let me update the PR.
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HbaseMiniClusterTest.java,
> fix_LocatedBlock__getLocations.patch, full_output.txt, pom.xml, root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765016#comment-17765016
]
Duo Zhang commented on HBASE-28061:
---
I've posted a PR to fix the problem.
https://github.com/apache/hbase/pull/5401
Please have a try. If this could solve the problem, then we can include this
and make a new 2.5.x release.
Thanks.
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HbaseMiniClusterTest.java,
> fix_LocatedBlock__getLocations.patch, full_output.txt, pom.xml, root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28058) HMaster snapshot file clean thread and the snapshot request handler thread encountered a deadlock
[
https://issues.apache.org/jira/browse/HBASE-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765014#comment-17765014
]
Hudson commented on HBASE-28058:
Results for branch branch-2
[build #879 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HMaster snapshot file clean thread and the snapshot request handler thread
> encountered a deadlock
> -
>
> Key: HBASE-28058
> URL: https://issues.apache.org/jira/browse/HBASE-28058
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-beta-1
>Reporter: Haiping lv
>Assignee: Haiping lv
>Priority: Major
> Fix For: 3.0.0
>
>
> HMaster encountered a deadlock when runing ITBLL test.
> The handler thread of HMaster holds the synchronized lock of SnapshotManager
> and competes for the readLock of takingSnapshotLock.
> {code:java}
> Thread 57 (RpcServer.default.FPBQ.Fifo.handler=22,queue=1,port=16000):
> State: WAITING
> Blocked count: 14
> Waited count: 206
> Waiting on
> java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@a7a58d4
> Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
>
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:675)
>
> org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1765)
>
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437)
> org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) {code}
> The Master Snapshot cleaning thread holds the writeLock of takingSnapshotLock
> to compete for the synchronized lock of SnapshotManager.
> {code:java}
> Thread 5405 (hfile_cleaner-dir-scan-pool-2):
> State: BLOCKED
> Blocked count: 50535
> Waited count: 92200
> Blocked on org.apache.hadoop.hbase.master.snapshot.SnapshotManager@21a16019
> Blocked by 57 (RpcServer.default.FPBQ.Fifo.handler=22,queue=1,port=16000)
> Stack:
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isTakingAnySnapshot(SnapshotManager.java:655)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache.getUnreferencedFiles(SnapshotFileCache.java:197)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner.getDeletableFiles(SnapshotHFileCleaner.java:78)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:375)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$1(CleanerChore.java:483)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$$Lambda$788/1121007598.act(Unknown
> Source)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:565)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:483)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$2(CleanerChore.java:495)
>
>
[GitHub] [hbase] Apache9 opened a new pull request, #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache9 opened a new pull request, #5401: URL: https://github.com/apache/hbase/pull/5401 …ase of Hadoop3.3.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28079) Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
[
https://issues.apache.org/jira/browse/HBASE-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765015#comment-17765015
]
Hudson commented on HBASE-28079:
Results for branch branch-2
[build #879 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Unhandled TableExistsException and NamespaceExistException in
> BackupSystemTable
> ---
>
> Key: HBASE-28079
> URL: https://issues.apache.org/jira/browse/HBASE-28079
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> Most backup-related operations instantiate a BackupSystemTable object. The
> first thing it does is check that the {{backup}} namespace and backup-related
> tables exist, and create them if not. One way in which this can be triggered
> is via LoadIncrementalHFiles, which can load files with a configurable number
> of threads. When bulkLoadHFile is called, a preCommitHFile hook is triggered
> which does the above. With a certain level of concurrency, one can easily
> trigger a race condition on creating the namespace or tables. This exception
> bubbles up and fails the bulkload.
> We should catch TableExistsException and NamespaceExistException in
> BackupSystemTable initialization so that these do not cause client pain.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1325395461
##
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws
Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws
IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in
progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
ok, this is now taken care of
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-10278) Provide better write predictability
[ https://issues.apache.org/jira/browse/HBASE-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764997#comment-17764997 ] Ranganath Govardhanagiri commented on HBASE-10278: -- Hello [~apurtell] - Can you please provide a pointer to the more recent work (JIRA) > Provide better write predictability > --- > > Key: HBASE-10278 > URL: https://issues.apache.org/jira/browse/HBASE-10278 > Project: HBase > Issue Type: New Feature > Components: wal >Reporter: Himanshu Vashishtha >Priority: Major > Attachments: 10278-trunk-v2.1.patch, 10278-trunk-v2.1.patch, > 10278-wip-1.1.patch, Multiwaldesigndoc.pdf, SwitchWriterFlow.pptx > > > Currently, HBase has one WAL per region server. > Whenever there is any latency in the write pipeline (due to whatever reasons > such as n/w blip, a node in the pipeline having a bad disk, etc), the overall > write latency suffers. > Jonathan Hsieh and I analyzed various approaches to tackle this issue. We > also looked at HBASE-5699, which talks about adding concurrent multi WALs. > Along with performance numbers, we also focussed on design simplicity, > minimum impact on MTTR & Replication, and compatibility with 0.96 and 0.98. > Considering all these parameters, we propose a new HLog implementation with > WAL Switching functionality. > Please find attached the design doc for the same. It introduces the WAL > Switching feature, and experiments/results of a prototype implementation, > showing the benefits of this feature. > The second goal of this work is to serve as a building block for concurrent > multiple WALs feature. > Please review the doc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
