[jira] [Resolved] (HBASE-24546) CloneSnapshotProcedure unlimited retry
[
https://issues.apache.org/jira/browse/HBASE-24546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-24546.
Resolution: Fixed
Pushed to branch-2.2+. Thanks [~wenfeiyi666] for contributing.
> CloneSnapshotProcedure unlimited retry
> --
>
> Key: HBASE-24546
> URL: https://issues.apache.org/jira/browse/HBASE-24546
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 2.3.0, master, 2.2.5
>Reporter: wenfeiyi666
>Assignee: wenfeiyi666
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.2.6
>
>
> since regions dir was not remove in the previous execution created, need to
> be remove when retrying, resulting in exception, unlimited retry
> {code:java}
> procedure.CloneSnapshotProcedure: Retriable error trying to clone
> snapshot=snapshot_test to table=test:backup
> state=CLONE_SNAPSHOT_WRITE_FS_LAYOUT
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotException: clone snapshot={
> ss=snapshot_test table=test:backup type=FLUSH } failed because A clone should
> not have regions to remove
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:434)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFsLayout(CloneSnapshotProcedure.java:465)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFilesystemLayout(CloneSnapshotProcedure.java:392)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:67)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:194)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:962)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1662)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1409)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1979)
> Caused by: java.lang.IllegalArgumentException: A clone should not have
> regions to remove
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:418)
> ... 10 more
> {code}
> and the cloned regions name are unchanged, resulting in new created regions
> be removed when retrying
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2024: HBASE-24615 MutableRangeHistogram#updateSnapshotRangeMetrics doesn't …
Apache-HBase commented on pull request #2024: URL: https://github.com/apache/hbase/pull/2024#issuecomment-654036158 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | branch-2.3 passed | | +1 :green_heart: | checkstyle | 0m 15s | branch-2.3 passed | | +1 :green_heart: | spotbugs | 0m 32s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 7s | the patch passed | | +1 :green_heart: | checkstyle | 0m 12s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 16m 43s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 32m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2024 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 4e9e65afadce 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 9af91d4e1f | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-hadoop2-compat U: hbase-hadoop2-compat | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24665) all wal of RegionGroupingProvider together roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-24665: --- Fix Version/s: (was: 2.2.6) 2.2.7 > all wal of RegionGroupingProvider together roll > --- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.3.0, master, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0, 2.1.10, 1.4.14, 2.2.7 > > > when use RegionGroupingProvider, any a wal request roll, all wal will be > together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24671) Add excludefile and designatedfile options to graceful_stop.sh
[ https://issues.apache.org/jira/browse/HBASE-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baiqiang Zhao updated HBASE-24671: -- Release Note: Add excludefile and designatedfile options to graceful_stop.sh. Designated file with per line as unload targets. Exclude file should have per line. We do not unload regions to hostnames given in exclude file. > Add excludefile and designatedfile options to graceful_stop.sh > -- > > Key: HBASE-24671 > URL: https://issues.apache.org/jira/browse/HBASE-24671 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Baiqiang Zhao >Assignee: Baiqiang Zhao >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RegionMover is support excludefile and designatedfile options now. Integrate > these two options into graceful_stop.sh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24102) RegionMover should exclude draining/decommissioning nodes from target RSs
[
https://issues.apache.org/jira/browse/HBASE-24102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-24102:
---
Fix Version/s: (was: 2.2.5)
2.2.6
> RegionMover should exclude draining/decommissioning nodes from target RSs
> -
>
> Key: HBASE-24102
> URL: https://issues.apache.org/jira/browse/HBASE-24102
> Project: HBase
> Issue Type: Improvement
>Reporter: Anoop Sam John
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.1.10, 2.2.6
>
>
> When using RegionMover tool to unload the regions from a given RS, it decides
> the list of destination RSs by
> {code}
> List regionServers = new ArrayList<>();
> regionServers.addAll(admin.getRegionServers());
> // Remove the host Region server from target Region Servers list
> ServerName server = stripServer(regionServers, hostname, port);
> .
> // Remove RS present in the exclude file
> stripExcludes(regionServers);
> stripMaster(regionServers);
> {code}
> Ya it is removing the RSs mentioned in the exclude file.
> Better when the RegionMover user is NOT mentioning any exclude list, we can
> exclude the draining/decommissioning RSs
> Admin#listDecommissionedRegionServers()
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24666) Offload the replication source/sink job to independent Replication Server
[ https://issues.apache.org/jira/browse/HBASE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151785#comment-17151785 ] Zheng Wang commented on HBASE-24666: Maybe it is better to use a generic name(eg: ChoreServer), so we can move some other works such as compaction and log split to it in future. > Offload the replication source/sink job to independent Replication Server > - > > Key: HBASE-24666 > URL: https://issues.apache.org/jira/browse/HBASE-24666 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Priority: Major > > The basic idea is add a role "ReplicationServer" to take the replication > source/sink job. HMaster is responsible for scheduling the replication job to > different ReplicationServer. > [link Design > doc|https://docs.google.com/document/d/16kRPVGctFSf__nC3yaVZmAm3GTxIbHefekKC_rMmTw8/edit?usp=sharing] > Suggestions are welcomed. Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24671) Add excludefile and designatedfile options to graceful_stop.sh
[ https://issues.apache.org/jira/browse/HBASE-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151844#comment-17151844 ] Anoop Sam John commented on HBASE-24671: Can u pls add details on how a user can pass these files while calling graceful_stop.sh? That will complete the ReleaseNotes > Add excludefile and designatedfile options to graceful_stop.sh > -- > > Key: HBASE-24671 > URL: https://issues.apache.org/jira/browse/HBASE-24671 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Baiqiang Zhao >Assignee: Baiqiang Zhao >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RegionMover is support excludefile and designatedfile options now. Integrate > these two options into graceful_stop.sh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24671) Add excludefile and designatedfile options to graceful_stop.sh
[ https://issues.apache.org/jira/browse/HBASE-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baiqiang Zhao updated HBASE-24671: -- Release Note: Add excludefile and designatedfile options to graceful_stop.sh. Designated file with per line as unload targets. Exclude file should have per line. We do not unload regions to hostnames given in exclude file. Here is a simple example using graceful_stop.sh with designatedfile option: ./bin/graceful_stop.sh --maxthreads 4 --designatedfile /path/designatedfile hostname The usage of the excludefile option is the same as the above. If excludefile and designedfile are used at the same time. First filter out the list of RSs based on the designedfile, and then exclude the RS contained in the excludefile from the RSs. Finally, the rest of the RSs are the targets of unload. was: Add excludefile and designatedfile options to graceful_stop.sh. Designated file with per line as unload targets. Exclude file should have per line. We do not unload regions to hostnames given in exclude file. > Add excludefile and designatedfile options to graceful_stop.sh > -- > > Key: HBASE-24671 > URL: https://issues.apache.org/jira/browse/HBASE-24671 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Baiqiang Zhao >Assignee: Baiqiang Zhao >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RegionMover is support excludefile and designatedfile options now. Integrate > these two options into graceful_stop.sh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24666) Offload the replication source/sink job to independent Replication Server
[
https://issues.apache.org/jira/browse/HBASE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151929#comment-17151929
]
Guanghao Zhang commented on HBASE-24666:
{quote}you don execute any CP hooks as part of compaction? May be you don have
such a need ?
{quote}
Yes. For internal usage, we didn't consider this problem at all..
> Offload the replication source/sink job to independent Replication Server
> -
>
> Key: HBASE-24666
> URL: https://issues.apache.org/jira/browse/HBASE-24666
> Project: HBase
> Issue Type: Umbrella
>Reporter: Guanghao Zhang
>Priority: Major
>
> The basic idea is add a role "ReplicationServer" to take the replication
> source/sink job. HMaster is responsible for scheduling the replication job to
> different ReplicationServer.
> [link Design
> doc|https://docs.google.com/document/d/16kRPVGctFSf__nC3yaVZmAm3GTxIbHefekKC_rMmTw8/edit?usp=sharing]
> Suggestions are welcomed. Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HBASE-24685) MultiAction and FailureInfo should be removed
[ https://issues.apache.org/jira/browse/HBASE-24685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani reassigned HBASE-24685: Assignee: Viraj Jasani > MultiAction and FailureInfo should be removed > - > > Key: HBASE-24685 > URL: https://issues.apache.org/jira/browse/HBASE-24685 > Project: HBase > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > > Just came across MultiAction and FailureInfo which are IA.Private and not > being used anywhere on trunk. Both of them are being used on branch-2 though. > We should remove them on trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24685) MultiAction and FailureInfo should be removed
Viraj Jasani created HBASE-24685: Summary: MultiAction and FailureInfo should be removed Key: HBASE-24685 URL: https://issues.apache.org/jira/browse/HBASE-24685 Project: HBase Issue Type: Task Reporter: Viraj Jasani Just came across MultiAction and FailureInfo which are IA.Private and not being used anywhere on trunk. Both of them are being used on branch-2 though. We should remove them on trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2024: HBASE-24615 MutableRangeHistogram#updateSnapshotRangeMetrics doesn't …
Apache-HBase commented on pull request #2024: URL: https://github.com/apache/hbase/pull/2024#issuecomment-654038613 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 48s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 6s | branch-2.3 passed | | +1 :green_heart: | compile | 0m 23s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 7m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 19s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 22s | the patch passed | | +1 :green_heart: | compile | 0m 22s | the patch passed | | +1 :green_heart: | javac | 0m 22s | the patch passed | | +1 :green_heart: | shadedjars | 6m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 31s | hbase-hadoop2-compat in the patch passed. | | | | 27m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2024 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bce949b16934 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 9af91d4e1f | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/testReport/ | | Max. process+thread count | 332 (vs. ulimit of 12500) | | modules | C: hbase-hadoop2-compat U: hbase-hadoop2-compat | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24022) Set version as 2.2.5-SNAPSHOT in branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-24022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-24022: --- Fix Version/s: (was: 2.2.5) 2.2.6 > Set version as 2.2.5-SNAPSHOT in branch-2.2 > --- > > Key: HBASE-24022 > URL: https://issues.apache.org/jira/browse/HBASE-24022 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24280) Hadoop2 and Hadoop3 profiles being activated simultaneously causing test failures
[
https://issues.apache.org/jira/browse/HBASE-24280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-24280:
---
Fix Version/s: 2.2.6
> Hadoop2 and Hadoop3 profiles being activated simultaneously causing test
> failures
> -
>
> Key: HBASE-24280
> URL: https://issues.apache.org/jira/browse/HBASE-24280
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 2.3.0, 2.2.6
>
> Attachments: HBASE-24280.master.001.patch,
> TEST-org.apache.hadoop.hbase.rest.TestSecureRESTServer.xml
>
>
> [~ndimiduk] pointed out that, after this change went in, TestSecureRESTServer
> started failing with Hadoop3 on branch-2.3
> https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/56/
> Of course, I ran this with 1.8.0_241 and Maven 3.6.33 and it passed :) {{mvn
> clean package -Dtest=TestSecureRESTServer -Dhadoop.profile=3.0
> -DfailIfNoTests=false}}
> FYI [~stoty] in case you can repro a failure and want to dig in. Feel free to
> re-assign.
> It looks like we didn't have a nightly run of branch-2.2 due to docker
> container build issues. Will be interesting to see if it fails there. It did
> not fail the master nightly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24671) Add excludefile and designatedfile options to graceful_stop.sh
[ https://issues.apache.org/jira/browse/HBASE-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151777#comment-17151777 ] Baiqiang Zhao commented on HBASE-24671: --- Release notes have been added [~anoop.hbase] > Add excludefile and designatedfile options to graceful_stop.sh > -- > > Key: HBASE-24671 > URL: https://issues.apache.org/jira/browse/HBASE-24671 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Baiqiang Zhao >Assignee: Baiqiang Zhao >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RegionMover is support excludefile and designatedfile options now. Integrate > these two options into graceful_stop.sh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-21905) TestFIFOCompactionPolicy is flaky
[ https://issues.apache.org/jira/browse/HBASE-21905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21905: --- Fix Version/s: 2.2.6 > TestFIFOCompactionPolicy is flaky > - > > Key: HBASE-21905 > URL: https://issues.apache.org/jira/browse/HBASE-21905 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 3.0.0-alpha-1, 1.5.0, 2.3.0 >Reporter: Andrew Kyle Purtell >Assignee: Bharath Vissapragada >Priority: Major > Labels: branch-1 > Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0, 2.2.6 > > Attachments: > org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt, > testFIFOCompactionPolicyExpiredEmptyHFiles-failure-log.txt > > > java.lang.IllegalArgumentException , overlaps with > For example: > [ERROR] > testFIFOCompactionPolicyExpiredEmptyHFiles(org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy) > Time elapsed: 3.321 s <<< ERROR! > java.io.IOException: > java.io.IOException: > [hdfs://localhost:41525/user/apurtell/test-data/734de07d-1f22-46a9-a1f5-96ad4578450b/data/default/testFIFOCompactionPolicyExpiredEmptyHFiles/c4f673438e09d7ef5a9b79b363639cde/f/c0c5836c1f714f78847cf00326586b69, > > hdfs://localhost:41525/user/apurtell/test-data/734de07d-1f22-46a9-a1f5-96ad4578450b/data/default/testFIFOCompactionPolicyExpiredEmptyHFiles/c4f673438e09d7ef5a9b79b363639cde/f/c65648691f614b2d8dd4b586c5923bfe] > overlaps with > [hdfs://localhost:41525/user/apurtell/test-data/734de07d-1f22-46a9-a1f5-96ad4578450b/data/default/testFIFOCompactionPolicyExpiredEmptyHFiles/c4f673438e09d7ef5a9b79b363639cde/f/c0c5836c1f714f78847cf00326586b69] > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2438) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277) > Caused by: java.lang.IllegalArgumentException: > [hdfs://localhost:41525/user/apurtell/test-data/734de07d-1f22-46a9-a1f5-96ad4578450b/data/default/testFIFOCompactionPolicyExpiredEmptyHFiles/c4f673438e09d7ef5a9b79b363639cde/f/c0c5836c1f714f78847cf00326586b69, > > hdfs://localhost:41525/user/apurtell/test-data/734de07d-1f22-46a9-a1f5-96ad4578450b/data/default/testFIFOCompactionPolicyExpiredEmptyHFiles/c4f673438e09d7ef5a9b79b363639cde/f/c65648691f614b2d8dd4b586c5923bfe] > overlaps with > [hdfs://localhost:41525/user/apurtell/test-data/734de07d-1f22-46a9-a1f5-96ad4578450b/data/default/testFIFOCompactionPolicyExpiredEmptyHFiles/c4f673438e09d7ef5a9b79b363639cde/f/c0c5836c1f714f78847cf00326586b69] > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:119) > at > org.apache.hadoop.hbase.regionserver.HStore.addToCompactingFiles(HStore.java:1824) > at > org.apache.hadoop.hbase.regionserver.HStore.requestCompaction(HStore.java:1798) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread.selectCompaction(CompactSplitThread.java:415) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestCompactionInternal(CompactSplitThread.java:388) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestCompactionInternal(CompactSplitThread.java:317) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestCompaction(CompactSplitThread.java:306) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.compactRegion(RSRpcServices.java:1513) > at > org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:23649) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2380) > ... 3 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24615) MutableRangeHistogram#updateSnapshotRangeMetrics doesn't calculate the distribution for last bucket.
[
https://issues.apache.org/jira/browse/HBASE-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151779#comment-17151779
]
wenfeiyi666 commented on HBASE-24615:
-
ping [~dmanning] and [~shahrs87]
> MutableRangeHistogram#updateSnapshotRangeMetrics doesn't calculate the
> distribution for last bucket.
>
>
> Key: HBASE-24615
> URL: https://issues.apache.org/jira/browse/HBASE-24615
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.3.0, master, 1.3.7, 2.2.6
>Reporter: Rushabh Shah
>Assignee: wenfeiyi666
>Priority: Major
>
> We are not processing the distribution for last bucket.
> https://github.com/apache/hbase/blob/master/hbase-hadoop-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableRangeHistogram.java#L70
> {code:java}
> public void updateSnapshotRangeMetrics(MetricsRecordBuilder
> metricsRecordBuilder,
> Snapshot snapshot) {
> long priorRange = 0;
> long cumNum = 0;
> final long[] ranges = getRanges();
> final String rangeType = getRangeType();
> for (int i = 0; i < ranges.length - 1; i++) { -> The bug lies
> here. We are not processing last bucket.
> long val = snapshot.getCountAtOrBelow(ranges[i]);
> if (val - cumNum > 0) {
> metricsRecordBuilder.addCounter(
> Interns.info(name + "_" + rangeType + "_" + priorRange + "-" +
> ranges[i], desc),
> val - cumNum);
> }
> priorRange = ranges[i];
> cumNum = val;
> }
> long val = snapshot.getCount();
> if (val - cumNum > 0) {
> metricsRecordBuilder.addCounter(
> Interns.info(name + "_" + rangeType + "_" + ranges[ranges.length -
> 1] + "-inf", desc),
> val - cumNum);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24230) Support user-defined version timestamp when bulk load data
[ https://issues.apache.org/jira/browse/HBASE-24230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Zhang updated HBASE-24230: --- Priority: Minor (was: Trivial) > Support user-defined version timestamp when bulk load data > -- > > Key: HBASE-24230 > URL: https://issues.apache.org/jira/browse/HBASE-24230 > Project: HBase > Issue Type: Improvement > Components: hbase-connectors >Affects Versions: 1.0.0 >Reporter: Xiao Zhang >Assignee: Xiao Zhang >Priority: Minor > > In hbase-connectors-1.0.0, loading data in bulk, only the current system time > can be used as the KeyValue version timestamp. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2021: HBASE-24665 all wal of RegionGroupingProvider together roll
Apache-HBase commented on pull request #2021: URL: https://github.com/apache/hbase/pull/2021#issuecomment-654045181 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | master passed | | +1 :green_heart: | compile | 0m 57s | master passed | | +1 :green_heart: | shadedjars | 5m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 29s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | shadedjars | 5m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 141m 24s | hbase-server in the patch passed. | | | | 165m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2021/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2021 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d7e490663dbe 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 44691ce23c | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2021/1/testReport/ | | Max. process+thread count | 4178 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2021/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2022: HBASE-24665 all wal of RegionGroupingProvider together roll
Apache-HBase commented on pull request #2022: URL: https://github.com/apache/hbase/pull/2022#issuecomment-654059307 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 51s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 56s | branch-2.3 passed | | +1 :green_heart: | compile | 1m 27s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 8m 17s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 51s | hbase-server in branch-2.3 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 20s | the patch passed | | +1 :green_heart: | compile | 1m 25s | the patch passed | | +1 :green_heart: | javac | 1m 25s | the patch passed | | +1 :green_heart: | shadedjars | 8m 6s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 54s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 133m 41s | hbase-server in the patch passed. | | | | 169m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2022/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2022 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f38076aaf57c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 9af91d4e1f | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2022/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2022/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2022/1/testReport/ | | Max. process+thread count | 4380 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2022/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2022: HBASE-24665 all wal of RegionGroupingProvider together roll
Apache-HBase commented on pull request #2022: URL: https://github.com/apache/hbase/pull/2022#issuecomment-654059767 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 54s | branch-2.3 passed | | +1 :green_heart: | compile | 1m 11s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 6m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | the patch passed | | +1 :green_heart: | compile | 1m 9s | the patch passed | | +1 :green_heart: | javac | 1m 9s | the patch passed | | +1 :green_heart: | shadedjars | 6m 15s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 144m 27s | hbase-server in the patch passed. | | | | 171m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2022/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2022 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cc4040d9ae62 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 9af91d4e1f | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2022/1/testReport/ | | Max. process+thread count | 3538 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2022/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24671) Add excludefile and designatedfile options to graceful_stop.sh
[ https://issues.apache.org/jira/browse/HBASE-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151876#comment-17151876 ] Baiqiang Zhao commented on HBASE-24671: --- Please check the new Release Notes [~anoop.hbase] > Add excludefile and designatedfile options to graceful_stop.sh > -- > > Key: HBASE-24671 > URL: https://issues.apache.org/jira/browse/HBASE-24671 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Baiqiang Zhao >Assignee: Baiqiang Zhao >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RegionMover is support excludefile and designatedfile options now. Integrate > these two options into graceful_stop.sh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ddupg opened a new pull request #2026: HBASE-22738 Fallback to default group to choose RS when there are no …
ddupg opened a new pull request #2026: URL: https://github.com/apache/hbase/pull/2026 …RS in current group backport HBASE-22738 to branch2 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24427) HStore.add log format error
[ https://issues.apache.org/jira/browse/HBASE-24427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-24427: --- Fix Version/s: (was: 2.2.5) 2.2.6 > HStore.add log format error > --- > > Key: HBASE-24427 > URL: https://issues.apache.org/jira/browse/HBASE-24427 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.3.0, master, 2.2.4 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.0, 2.1.10, 2.2.6 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24426) Missing regionName while logging warning in HBCKServerCrashProcedure
[ https://issues.apache.org/jira/browse/HBASE-24426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-24426: --- Fix Version/s: (was: 2.2.5) 2.2.6 > Missing regionName while logging warning in HBCKServerCrashProcedure > > > Key: HBASE-24426 > URL: https://issues.apache.org/jira/browse/HBASE-24426 > Project: HBase > Issue Type: Bug >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.0, 2.2.6 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] HorizonNet commented on a change in pull request #2016: HBASE-24653 Show snapshot owner on Master WebUI
HorizonNet commented on a change in pull request #2016: URL: https://github.com/apache/hbase/pull/2016#discussion_r450037665 ## File path: hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon ## @@ -673,6 +675,8 @@ AssignmentManager assignmentManager = master.getAssignmentManager(); <% snapshotTable.getNameAsString() %> <% new Date(snapshotDesc.getCreationTime()) %> +<% snapshotDesc.getOwner() %> +<% snapshotDesc.getTtl() %> Review comment: I think human readable would be better. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24666) Offload the replication source/sink job to independent Replication Server
[ https://issues.apache.org/jira/browse/HBASE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151809#comment-17151809 ] ramkrishna.s.vasudevan commented on HBASE-24666: As per your doc -the external compaction based process (currently MR based) you don execute any CP hooks as part of compaction? May be you don have such a need ? > Offload the replication source/sink job to independent Replication Server > - > > Key: HBASE-24666 > URL: https://issues.apache.org/jira/browse/HBASE-24666 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Priority: Major > > The basic idea is add a role "ReplicationServer" to take the replication > source/sink job. HMaster is responsible for scheduling the replication job to > different ReplicationServer. > [link Design > doc|https://docs.google.com/document/d/16kRPVGctFSf__nC3yaVZmAm3GTxIbHefekKC_rMmTw8/edit?usp=sharing] > Suggestions are welcomed. Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
Apache-HBase commented on pull request #1916: URL: https://github.com/apache/hbase/pull/1916#issuecomment-654051757 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 15s | branch-2.2 passed | | +1 :green_heart: | compile | 0m 56s | branch-2.2 passed | | +1 :green_heart: | checkstyle | 1m 19s | branch-2.2 passed | | +1 :green_heart: | shadedjars | 4m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | branch-2.2 passed | | +0 :ok: | spotbugs | 3m 18s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 16s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 41s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | checkstyle | 1m 18s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 0s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 25m 12s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | | +1 :green_heart: | findbugs | 3m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 179m 25s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 33s | The patch does not generate ASF License warnings. | | | | 243m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1916 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux ebc0fd97eb30 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1916/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 4be5fe3154 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/3/testReport/ | | Max. process+thread count | 4531 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1916/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1900: HBASE-24546 CloneSnapshotProcedure unlimited retry
Apache-HBase commented on pull request #1900: URL: https://github.com/apache/hbase/pull/1900#issuecomment-654061353 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 47s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 10s | master passed | | +1 :green_heart: | compile | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 6m 34s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 48s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 49s | the patch passed | | +1 :green_heart: | compile | 1m 15s | the patch passed | | +1 :green_heart: | javac | 1m 15s | the patch passed | | +1 :green_heart: | shadedjars | 6m 47s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 49s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 223m 49s | hbase-server in the patch passed. | | | | 254m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1900/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1900 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6a2aef1b59ff 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fe2ae809d1 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1900/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1900/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1900/2/testReport/ | | Max. process+thread count | 3264 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1900/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2025: HBASE-24489 Rewrite TestClusterRestartFailover.test since namespace t…
Apache-HBase commented on pull request #2025: URL: https://github.com/apache/hbase/pull/2025#issuecomment-654130992 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 14s | master passed | | +1 :green_heart: | checkstyle | 1m 24s | master passed | | +1 :green_heart: | spotbugs | 2m 37s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 12s | the patch passed | | +1 :green_heart: | checkstyle | 1m 13s | hbase-server: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 22s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 39m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2025/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2025 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 998c6891a46b 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 287f29818f | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2025/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2024: HBASE-24615 MutableRangeHistogram#updateSnapshotRangeMetrics doesn't …
Apache-HBase commented on pull request #2024: URL: https://github.com/apache/hbase/pull/2024#issuecomment-654040039 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 14s | branch-2.3 passed | | +1 :green_heart: | compile | 0m 20s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 5m 48s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 21s | hbase-hadoop2-compat in branch-2.3 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 55s | the patch passed | | +1 :green_heart: | compile | 0m 20s | the patch passed | | +1 :green_heart: | javac | 0m 20s | the patch passed | | +1 :green_heart: | shadedjars | 5m 46s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 18s | hbase-hadoop2-compat in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 28s | hbase-hadoop2-compat in the patch failed. | | | | 23m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2024 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3f575e3b6264 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 9af91d4e1f | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop2-compat.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop2-compat.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-hadoop2-compat.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/testReport/ | | Max. process+thread count | 212 (vs. ulimit of 12500) | | modules | C: hbase-hadoop2-compat U: hbase-hadoop2-compat | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2024/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2021: HBASE-24665 all wal of RegionGroupingProvider together roll
Apache-HBase commented on pull request #2021: URL: https://github.com/apache/hbase/pull/2021#issuecomment-654041836 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 17s | master passed | | +1 :green_heart: | compile | 1m 3s | master passed | | +1 :green_heart: | shadedjars | 5m 57s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 41s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 5m 42s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 129m 11s | hbase-server in the patch passed. | | | | 155m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2021/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2021 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1c2b3648a0ba 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 44691ce23c | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2021/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2021/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2021/1/testReport/ | | Max. process+thread count | 4462 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2021/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on pull request #1909: HBASE-24569 Get hostAndWeights in addition using localhost if it is n…
bsglz commented on pull request #1909: URL: https://github.com/apache/hbase/pull/1909#issuecomment-654043899 @Apache9 Any more comments? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on pull request #1926: HBASE-24586 Add table level locality in table.jsp
bsglz commented on pull request #1926: URL: https://github.com/apache/hbase/pull/1926#issuecomment-654044030 @virajjasani Ping. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on pull request #2011: HBASE-24664 Some changing of split region by overall region size rath…
bsglz commented on pull request #2011: URL: https://github.com/apache/hbase/pull/2011#issuecomment-654043581 @wchevreuil @Apache9 Ping. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24666) Offload the replication source/sink job to independent Replication Server
[
https://issues.apache.org/jira/browse/HBASE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151793#comment-17151793
]
Guanghao Zhang commented on HBASE-24666:
{quote} I found that the CP hooks is becoming a blocker in moving things out of
RS.
{quote}
Yes. CP is a blocker if it need access RS during Replication/Compaction. Need
to check when the CP was called.
> Offload the replication source/sink job to independent Replication Server
> -
>
> Key: HBASE-24666
> URL: https://issues.apache.org/jira/browse/HBASE-24666
> Project: HBase
> Issue Type: Umbrella
>Reporter: Guanghao Zhang
>Priority: Major
>
> The basic idea is add a role "ReplicationServer" to take the replication
> source/sink job. HMaster is responsible for scheduling the replication job to
> different ReplicationServer.
> [link Design
> doc|https://docs.google.com/document/d/16kRPVGctFSf__nC3yaVZmAm3GTxIbHefekKC_rMmTw8/edit?usp=sharing]
> Suggestions are welcomed. Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani opened a new pull request #2027: HBASE-24685 : Removing MultiAction and FailureInfo
virajjasani opened a new pull request #2027: URL: https://github.com/apache/hbase/pull/2027 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2019: HBASE-24681 Remove the cache walsById/walsByIdRecoveredQueues from Re…
Apache-HBase commented on pull request #2019: URL: https://github.com/apache/hbase/pull/2019#issuecomment-654043469 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-24666 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 2s | HBASE-24666 passed | | +1 :green_heart: | compile | 0m 57s | HBASE-24666 passed | | +1 :green_heart: | shadedjars | 5m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | HBASE-24666 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 42s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | shadedjars | 6m 1s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 36s | hbase-server generated 7 new + 28 unchanged - 0 fixed = 35 total (was 28) | ||| _ Other Tests _ | | -1 :x: | unit | 205m 21s | hbase-server in the patch failed. | | | | 230m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2019 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5464af8faf74 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / fe2ae809d1 | | Default Java | 1.8.0_232 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/testReport/ | | Max. process+thread count | 3916 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-24684) Fetch ReplicationSink servers list from HMaster instead of ZooKeeper
Guanghao Zhang created HBASE-24684: -- Summary: Fetch ReplicationSink servers list from HMaster instead of ZooKeeper Key: HBASE-24684 URL: https://issues.apache.org/jira/browse/HBASE-24684 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24683) Add a basic ReplicationServer which only implement ReplicationSink Service
Guanghao Zhang created HBASE-24683: -- Summary: Add a basic ReplicationServer which only implement ReplicationSink Service Key: HBASE-24683 URL: https://issues.apache.org/jira/browse/HBASE-24683 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2026: HBASE-22738 Fallback to default group to choose RS when there are no …
Apache-HBase commented on pull request #2026: URL: https://github.com/apache/hbase/pull/2026#issuecomment-654156908 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 58s | branch-2.3 passed | | +1 :green_heart: | compile | 0m 23s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 5m 24s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 36s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed | | +1 :green_heart: | javac | 0m 23s | the patch passed | | +1 :green_heart: | shadedjars | 5m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 4m 49s | hbase-rsgroup in the patch passed. | | | | 27m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2026/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2026 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 01eba63619d5 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 5d5b156ec3 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2026/1/testReport/ | | Max. process+thread count | 2326 (vs. ulimit of 12500) | | modules | C: hbase-rsgroup U: hbase-rsgroup | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2026/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-24593) [branch-2.2] Fix the maven compilation failure for nightly build
[ https://issues.apache.org/jira/browse/HBASE-24593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-24593. Fix Version/s: 2.2.6 Resolution: Fixed > [branch-2.2] Fix the maven compilation failure for nightly build > > > Key: HBASE-24593 > URL: https://issues.apache.org/jira/browse/HBASE-24593 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.6 > > > [https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/896/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24562) Stabilize master startup with meta replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-24562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-24562: --- Fix Version/s: 2.2.6 > Stabilize master startup with meta replicas enabled > --- > > Key: HBASE-24562 > URL: https://issues.apache.org/jira/browse/HBASE-24562 > Project: HBase > Issue Type: Improvement > Components: meta, read replicas >Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.4.0, 2.2.5 >Reporter: Szabolcs Bukros >Assignee: Szabolcs Bukros >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0, 2.2.6 > > > This is related to HBASE-21624 . > I created a separate ticket because in the original one a "complete solution > for meta replicas" was requested and this is not one. I'm just trying to make > master startup more stable by making assigning meta replicas asynchronous and > preventing a potential assignment failure from crashing master. > The idea is that starting master with less or even no meta replicas assigned > is preferable to not having a running master. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24428) Priority compaction for recently split daughter regions
[
https://issues.apache.org/jira/browse/HBASE-24428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-24428:
---
Fix Version/s: (was: 2.2.5)
2.2.6
> Priority compaction for recently split daughter regions
> ---
>
> Key: HBASE-24428
> URL: https://issues.apache.org/jira/browse/HBASE-24428
> Project: HBase
> Issue Type: Improvement
> Components: Compaction
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0, 1.7.0, 2.2.6
>
>
> We observe that under hotspotting conditions that splitting will proceed very
> slowly and the "_Cannot split region due to reference files being there_" log
> line will be logged excessively. (branch-1 based production.) This is because
> after a region is split it must be compacted before it can be split again.
> Reference files must be replaced by real HFiles, normal housekeeping
> performed during compaction. However if the regionserver is under excessive
> load, its compaction queues may become deep. The daughters of a recently
> split hotspotting region may themselves continue to hotspot and will rapidly
> need to split again. If the scheduled compaction work to remove/replace
> reference files is queued hundreds or thousands of compaction queue elements
> behind current, the recently split daughter regions will not be able to split
> again for a long time and may grow very large, producing additional
> complications (very large regions, very deep replication queues).
> To help avoid this condition we should prioritize the compaction of recently
> split daughter regions. Compaction requests include a {{priority}} field and
> CompactionRequest implements a comparator that sorts by this field. We
> already detect when a compaction request involves a region that has reference
> files, to ensure that it gets selected to be eligible for compaction, but we
> do not seem to prioritize the requests for post-split housekeeping. Split
> work should be placed at the top of the queue. Ensure that this is happening.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24666) Offload the replication source/sink job to independent Replication Server
[ https://issues.apache.org/jira/browse/HBASE-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151790#comment-17151790 ] Anoop Sam John commented on HBASE-24666: [~zghao] Even around Replication also we will have CP hooks. I did not check in detail.. But will it need access to the RSs during the replication? I found that the CP hooks is becoming a blocker in moving things out of RS. > Offload the replication source/sink job to independent Replication Server > - > > Key: HBASE-24666 > URL: https://issues.apache.org/jira/browse/HBASE-24666 > Project: HBase > Issue Type: Umbrella >Reporter: Guanghao Zhang >Priority: Major > > The basic idea is add a role "ReplicationServer" to take the replication > source/sink job. HMaster is responsible for scheduling the replication job to > different ReplicationServer. > [link Design > doc|https://docs.google.com/document/d/16kRPVGctFSf__nC3yaVZmAm3GTxIbHefekKC_rMmTw8/edit?usp=sharing] > Suggestions are welcomed. Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1900: HBASE-24546 CloneSnapshotProcedure unlimited retry
Apache-HBase commented on pull request #1900: URL: https://github.com/apache/hbase/pull/1900#issuecomment-654066327 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 47s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 17s | master passed | | +1 :green_heart: | compile | 1m 2s | master passed | | +1 :green_heart: | shadedjars | 6m 14s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 6m 13s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 238m 6s | hbase-server in the patch passed. | | | | 265m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1900/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1900 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c986f5a8a36c 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fe2ae809d1 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1900/2/testReport/ | | Max. process+thread count | 3705 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1900/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ddupg opened a new pull request #2025: HBASE-24489 Rewrite TestClusterRestartFailover.test since namespace t…
ddupg opened a new pull request #2025: URL: https://github.com/apache/hbase/pull/2025 …able is gone on on master This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on a change in pull request #1991: HBASE-24650 Change the return types of the new checkAndMutate methods…
brfrn169 commented on a change in pull request #1991:
URL: https://github.com/apache/hbase/pull/1991#discussion_r450112215
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncTableImpl.java
##
@@ -497,7 +497,7 @@ public void run(MultiResponse resp) {
"Failed to mutate row: " +
Bytes.toStringBinary(mutation.getRow()), ex));
} else {
future.complete(respConverter
- .apply((Result)
multiResp.getResults().get(regionName).result.get(0)));
+ .apply((RES)
multiResp.getResults().get(regionName).result.get(0)));
Review comment:
Is there any problem here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2020: HBASE-24682 Refactor ReplicationSource#addHFileRefs method: move it t…
Apache-HBase commented on pull request #2020: URL: https://github.com/apache/hbase/pull/2020#issuecomment-654050902 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 41s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-24666 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 20s | HBASE-24666 passed | | +1 :green_heart: | compile | 1m 2s | HBASE-24666 passed | | +1 :green_heart: | shadedjars | 6m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | HBASE-24666 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | shadedjars | 6m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 204m 11s | hbase-server in the patch passed. | | | | 231m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2020/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2020 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cfe499f5932b 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / fe2ae809d1 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2020/1/testReport/ | | Max. process+thread count | 3285 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2020/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2023: HBASE-24665 all wal of RegionGroupingProvider together roll
Apache-HBase commented on pull request #2023: URL: https://github.com/apache/hbase/pull/2023#issuecomment-654063612 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 52s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1.4 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 34s | branch-1.4 passed | | +1 :green_heart: | compile | 0m 50s | branch-1.4 passed with JDK v1.8.0_252 | | +1 :green_heart: | compile | 1m 2s | branch-1.4 passed with JDK v1.7.0_262 | | +1 :green_heart: | checkstyle | 2m 11s | branch-1.4 passed | | +1 :green_heart: | shadedjars | 3m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 49s | branch-1.4 passed with JDK v1.8.0_252 | | +1 :green_heart: | javadoc | 0m 49s | branch-1.4 passed with JDK v1.7.0_262 | | +0 :ok: | spotbugs | 3m 46s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 43s | branch-1.4 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 29s | the patch passed | | +1 :green_heart: | compile | 0m 51s | the patch passed with JDK v1.8.0_252 | | +1 :green_heart: | javac | 0m 51s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed with JDK v1.7.0_262 | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | checkstyle | 2m 3s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 2m 49s | Patch does not cause any errors with Hadoop 2.7.7. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed with JDK v1.8.0_252 | | +1 :green_heart: | javadoc | 0m 49s | the patch passed with JDK v1.7.0_262 | | +1 :green_heart: | findbugs | 3m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 124m 13s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 170m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2023/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2023 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 81107a638d46 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-2023/out/precommit/personality/provided.sh | | git revision | branch-1.4 / 1e6594f | | Default Java | 1.7.0_262 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_252 /usr/lib/jvm/zulu-7-amd64:1.7.0_262 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2023/1/testReport/ | | Max. process+thread count | 4169 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2023/1/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24671) Add excludefile and designatedfile options to graceful_stop.sh
[ https://issues.apache.org/jira/browse/HBASE-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baiqiang Zhao updated HBASE-24671: -- Release Note: Add excludefile and designatedfile options to graceful_stop.sh. Designated file with per line as unload targets. Exclude file should have per line. We do not unload regions to hostnames given in exclude file. Here is a simple example using graceful_stop.sh with designatedfile option: ./bin/graceful_stop.sh --maxthreads 4 --designatedfile /path/designatedfile hostname The usage of the excludefile option is the same as the above. was: Add excludefile and designatedfile options to graceful_stop.sh. Designated file with per line as unload targets. Exclude file should have per line. We do not unload regions to hostnames given in exclude file. Here is a simple example using graceful_stop.sh with designatedfile option: ./bin/graceful_stop.sh --maxthreads 4 --designatedfile /path/designatedfile hostname The usage of the excludefile option is the same as the above. If excludefile and designedfile are used at the same time. First filter out the list of RSs based on the designedfile, and then exclude the RS contained in the excludefile from the RSs. Finally, the rest of the RSs are the targets of unload. > Add excludefile and designatedfile options to graceful_stop.sh > -- > > Key: HBASE-24671 > URL: https://issues.apache.org/jira/browse/HBASE-24671 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Baiqiang Zhao >Assignee: Baiqiang Zhao >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RegionMover is support excludefile and designatedfile options now. Integrate > these two options into graceful_stop.sh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-24685) MultiAction and FailureInfo should be removed
[ https://issues.apache.org/jira/browse/HBASE-24685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-24685 started by Viraj Jasani. > MultiAction and FailureInfo should be removed > - > > Key: HBASE-24685 > URL: https://issues.apache.org/jira/browse/HBASE-24685 > Project: HBase > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > > Just came across MultiAction and FailureInfo which are IA.Private and not > being used anywhere on trunk. Both of them are being used on branch-2 though. > We should remove them on trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2026: HBASE-22738 Fallback to default group to choose RS when there are no …
Apache-HBase commented on pull request #2026: URL: https://github.com/apache/hbase/pull/2026#issuecomment-654158184 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 45s | branch-2.3 passed | | +1 :green_heart: | compile | 0m 28s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 6m 25s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 23s | hbase-rsgroup in branch-2.3 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 22s | the patch passed | | +1 :green_heart: | compile | 0m 26s | the patch passed | | +1 :green_heart: | javac | 0m 26s | the patch passed | | +1 :green_heart: | shadedjars | 6m 25s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 21s | hbase-rsgroup in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 4m 1s | hbase-rsgroup in the patch passed. | | | | 30m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2026/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2026 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2fe1afe53aab 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 5d5b156ec3 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2026/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-rsgroup.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2026/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-rsgroup.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2026/1/testReport/ | | Max. process+thread count | 2606 (vs. ulimit of 12500) | | modules | C: hbase-rsgroup U: hbase-rsgroup | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2026/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24670) KeyPrefix related split policies should extends SteppingSplitPolicy
[ https://issues.apache.org/jira/browse/HBASE-24670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang updated HBASE-24670: --- Description: Currently the KeyPrefixRegionSplitPolicy and DelimitedKeyPrefixRegionSplitPolicy are extends IncreasingToUpperBoundRegionSplitPolicy, since we had changed the default split policy as SteppingSplitPolicy, I think it is better to change them to extends SteppingSplitPolicy too. (was: Currently the KeyPrefixRegionSplitPolicy and DelimitedKeyPrefixRegionSplitPolicy are extends IncreasingToUpperBoundRegionSplitPolicy, since we had changed the default split policy as SteppingSplitPolicy, I thind it is better to change them to extends SteppingSplitPolicy too.) > KeyPrefix related split policies should extends SteppingSplitPolicy > --- > > Key: HBASE-24670 > URL: https://issues.apache.org/jira/browse/HBASE-24670 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > > Currently the KeyPrefixRegionSplitPolicy and > DelimitedKeyPrefixRegionSplitPolicy are extends > IncreasingToUpperBoundRegionSplitPolicy, since we had changed the default > split policy as SteppingSplitPolicy, I think it is better to change them to > extends SteppingSplitPolicy too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2030: HBASE-24431 RSGroupInfo add configuration map to store something extra
Apache-HBase commented on pull request #2030: URL: https://github.com/apache/hbase/pull/2030#issuecomment-654600556 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.3 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 34s | branch-2.3 passed | | +1 :green_heart: | checkstyle | 2m 7s | branch-2.3 passed | | +1 :green_heart: | spotbugs | 4m 35s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 14s | the patch passed | | -0 :warning: | checkstyle | 0m 13s | hbase-rsgroup: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) | | -0 :warning: | rubocop | 0m 13s | The patch generated 21 new + 46 unchanged - 1 fixed = 67 total (was 47) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 16m 38s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | hbaseprotoc | 2m 18s | the patch passed | | +1 :green_heart: | spotbugs | 5m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 1m 2s | The patch does not generate ASF License warnings. | | | | 53m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2030/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2030 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle cc hbaseprotoc prototool rubocop | | uname | Linux e3bf7bd6e730 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 53690d01e2 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2030/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-rsgroup.txt | | rubocop | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2030/1/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-protocol hbase-server hbase-rsgroup hbase-shell U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2030/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on pull request #2012: HBASE-24670 KeyPrefix related split policies should exends SteppingSp…
bsglz commented on pull request #2012: URL: https://github.com/apache/hbase/pull/2012#issuecomment-654619572 Please take a look if you have time, thanks. @wchevreuil @virajjasani This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2016: HBASE-24653 Show snapshot owner on Master WebUI
Apache-HBase commented on pull request #2016: URL: https://github.com/apache/hbase/pull/2016#issuecomment-654605519 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 45s | master passed | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 136m 59s | hbase-server in the patch passed. | | | | 148m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2016/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2016 | | Optional Tests | javac javadoc unit | | uname | Linux 8484d6ecbfb5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a1d7e6e253 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2016/2/testReport/ | | Max. process+thread count | 4328 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2016/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2027: HBASE-24685 : Removing MultiAction and FailureInfo
Apache-HBase commented on pull request #2027: URL: https://github.com/apache/hbase/pull/2027#issuecomment-654164352 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 45s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 35s | master passed | | +1 :green_heart: | compile | 0m 41s | master passed | | +1 :green_heart: | shadedjars | 6m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 0m 26s | the patch passed | | +1 :green_heart: | javac | 0m 26s | the patch passed | | +1 :green_heart: | shadedjars | 5m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 4s | hbase-client in the patch passed. | | | | 24m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2027/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2027 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 37504157889a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 287f29818f | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2027/1/testReport/ | | Max. process+thread count | 343 (vs. ulimit of 12500) | | modules | C: hbase-client U: hbase-client | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2027/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #2011: HBASE-24664 Some changing of split region by overall region size rath…
wchevreuil commented on a change in pull request #2011:
URL: https://github.com/apache/hbase/pull/2011#discussion_r450139888
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java
##
@@ -68,22 +76,14 @@ protected void configureForRegion(HRegion region) {
@Override
protected boolean shouldSplit() {
-boolean foundABigStore = false;
-
+// If any of the stores is unable to split (eg they contain reference
files)
+// then don't split
for (HStore store : region.getStores()) {
- // If any of the stores are unable to split (eg they contain reference
files)
- // then don't split
- if ((!store.canSplit())) {
+ if (!store.canSplit()) {
Review comment:
Move this check to the for loops inside _isExceedSize_, so that we don't
have to do do an extra iteration for all stores again in case none returns
false for _canSplit_.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java
##
@@ -94,4 +94,33 @@ long getDesiredMaxFileSize() {
public boolean positiveJitterRate() {
return this.jitterRate > 0;
}
+
+ /**
+ * @return true if region size exceed the sizeToCheck
+ */
+ protected boolean isExceedSize(long sizeToCheck, String extraLogStr) {
+if (overallHregionFiles) {
+ long sumSize = 0;
+ for (HStore store : region.getStores()) {
+sumSize += store.getSize();
+ }
+ if (sumSize > sizeToCheck) {
Review comment:
We should just return this comparison and let each caller decide how to
log it? That would discard the need for having an extra param just for the sake
of logging.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (HBASE-24686) [LOG] Log improvement in Connection#close
mokai created HBASE-24686: - Summary: [LOG] Log improvement in Connection#close Key: HBASE-24686 URL: https://issues.apache.org/jira/browse/HBASE-24686 Project: HBase Issue Type: Improvement Components: Client, logging Affects Versions: 2.2.3 Reporter: mokai We met some customers used hbase connection improperly, some threads call failed since the shared connection closed by one of the threads. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2025: HBASE-24489 Rewrite TestClusterRestartFailover.test since namespace t…
Apache-HBase commented on pull request #2025: URL: https://github.com/apache/hbase/pull/2025#issuecomment-654230489 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 1s | master passed | | +1 :green_heart: | compile | 1m 18s | master passed | | +1 :green_heart: | shadedjars | 6m 36s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 45s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 43s | the patch passed | | +1 :green_heart: | compile | 1m 15s | the patch passed | | +1 :green_heart: | javac | 1m 15s | the patch passed | | +1 :green_heart: | shadedjars | 6m 48s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 50s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 220m 49s | hbase-server in the patch passed. | | | | 251m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2025/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2025 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 38fd49ed1b6a 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 287f29818f | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2025/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2025/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2025/1/testReport/ | | Max. process+thread count | 2716 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2025/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24376) MergeNormalizer is merging non-adjacent regions and causing region overlaps/holes.
[
https://issues.apache.org/jira/browse/HBASE-24376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152065#comment-17152065
]
Ruslan Sabitov commented on HBASE-24376:
I faced with the issue in branch-1 (HBase 1.2.0-cdh5.16.2):
2020-07-02 16:41:18,337 INFO
org.apache.hadoop.hbase.master.normalizer.MergeNormalizationPlan: Executing
merging normalization plan: MergeNormalizationPlan{firstRegion={ENC
ODED => feba09265266f1c3c090bc42cc90becc, NAME =>
'tableName,Aw-BEZ0JD4M3HrvA4Yks,1593695478557.feba09265266f1c3c090bc42cc90becc.',
STARTKEY => 'Aw-BEZ0JD4M3HrvA4Yks', EN
DKEY => 'D0sYMT716R0tyHPGk8ii'}, secondRegion={ENCODED =>
8003ecbf849c4f5e27bf5956ec0729cc, NAME =>
'TableName,B_zjT044PCvwQ4I53Q5m,1593695479990.8003ecbf849c4f5e27bf5956
ec0729cc.', STARTKEY => 'B_zjT044PCvwQ4I53Q5m', ENDKEY =>
'FCzjFZ4Vhb0hpVtn6VxP'}}
> MergeNormalizer is merging non-adjacent regions and causing region
> overlaps/holes.
> --
>
> Key: HBASE-24376
> URL: https://issues.apache.org/jira/browse/HBASE-24376
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.3.0
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.4.0
>
>
> Currently, we found normalizer was merging regions which are non-adjacent, it
> will cause inconsistencies in the cluster.
> {code:java}
> 439055 2020-05-08 17:47:09,814 INFO
> org.apache.hadoop.hbase.master.normalizer.MergeNormalizationPlan: Executing
> merging normalization plan: MergeNormalizationPlan{firstRegion={ENCODED =>
> 47fe236a5e3649ded95cb64ad0c08492, NAME =>
> 'TABLE,\x03\x01\x05\x01\x04\x02,1554838974870.47fe236a5e3649ded95cb64ad
> 0c08492.', STARTKEY => '\x03\x01\x05\x01\x04\x02', ENDKEY =>
> '\x03\x01\x05\x01\x04\x02\x01\x02\x02201904082200\x00\x00\x03Mac\x00\x00\x00\x00\x00\x00\x00\x00\x00iMac13,1\x00\x00\x00\x00\x00\x049.3-14E260\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x05'},
> secondRegion={ENCODED => 0c0f2aa67f4329d5c4 8ba0320f173d31, NAME =>
> 'TABLE,\x03\x01\x05\x02\x01\x01,1554830735526.0c0f2aa67f4329d5c48ba0320f173d31.',
> STARTKEY => '\x03\x01\x05\x02\x01\x01', ENDKEY =>
> '\x03\x01\x05\x02\x01\x02'}}
> 439056 2020-05-08 17:47:11,438 INFO org.apache.hadoop.hbase.ScheduledChore:
> CatalogJanitor-*:16000 average execution time: 1676219193 ns.
> 439057 2020-05-08 17:47:11,730 INFO org.apache.hadoop.hbase.master.HMaster:
> Client=null/null merge regions [47fe236a5e3649ded95cb64ad0c08492],
> [0c0f2aa67f4329d5c48ba0320f173d31]
> {code}
>
> The root cause is that getMergeNormalizationPlan() uses a list of regionInfo
> which is ordered by regionName. regionName does not necessary guarantee the
> order of STARTKEY (let's say 'aa1', 'aa1!', in order of regionName, it will
> be 'aa1!' followed by 'aa1'. This will result in normalizer merging
> non-adjacent regions into one and creates overlaps. This is not an issue in
> branch-1 as the list is already ordered by RegionInfo.COMPARATOR in
> normalizer.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani merged pull request #2027: HBASE-24685 : Removing MultiAction and FailureInfo
virajjasani merged pull request #2027: URL: https://github.com/apache/hbase/pull/2027 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #1909: HBASE-24569 Get hostAndWeights in addition using localhost if it is n…
Apache9 commented on pull request #1909:
URL: https://github.com/apache/hbase/pull/1909#issuecomment-654218536
> > I do not fully understand the logic here, why it is OK to use localhost
if the returned hostAndWeight is null? We will only use the related methods to
get hostAndWeight for the local machine?
>
> Good question, the input host might be other machine in distributed mode,
but in that case the hostAndWeights will not use localhost as host name(get
from BlockLocation.hosts), so it is ok.
I do not get your point...
```
private float getBlockLocalityIndexInternal(String host, Visitor visitor) {
float localityIndex = 0;
HostAndWeight hostAndWeight = this.hostAndWeights.get(host);
if (hostAndWeight == null) {
hostAndWeight = this.hostAndWeights.get(HConstants.LOCALHOST);
}
if (hostAndWeight != null && uniqueBlocksTotalWeight != 0) {
localityIndex = visitor.visit(hostAndWeight);
}
return localityIndex;
}
```
The modified code is like this, no matter what is the host passed in, you
will always use localhost to get the hostAndWeight and again? What do you mean
by 'but in that case the hostAndWeights will not use localhost as host name'?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 edited a comment on pull request #1909: HBASE-24569 Get hostAndWeights in addition using localhost if it is n…
Apache9 edited a comment on pull request #1909:
URL: https://github.com/apache/hbase/pull/1909#issuecomment-654218536
> > I do not fully understand the logic here, why it is OK to use localhost
if the returned hostAndWeight is null? We will only use the related methods to
get hostAndWeight for the local machine?
>
> Good question, the input host might be other machine in distributed mode,
but in that case the hostAndWeights will not use localhost as host name(get
from BlockLocation.hosts), so it is ok.
I do not get your point...
```
private float getBlockLocalityIndexInternal(String host, Visitor visitor) {
float localityIndex = 0;
HostAndWeight hostAndWeight = this.hostAndWeights.get(host);
if (hostAndWeight == null) {
hostAndWeight = this.hostAndWeights.get(HConstants.LOCALHOST);
}
if (hostAndWeight != null && uniqueBlocksTotalWeight != 0) {
localityIndex = visitor.visit(hostAndWeight);
}
return localityIndex;
}
```
The modified code is like this, no matter what is the host passed in, you
will always use localhost to get the hostAndWeight again if the first get
returns null? What do you mean by 'but in that case the hostAndWeights will not
use localhost as host name'?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2027: HBASE-24685 : Removing MultiAction and FailureInfo
Apache-HBase commented on pull request #2027: URL: https://github.com/apache/hbase/pull/2027#issuecomment-654164189 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 17s | master passed | | +1 :green_heart: | compile | 0m 29s | master passed | | +1 :green_heart: | shadedjars | 5m 47s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 28s | hbase-client in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 0s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | shadedjars | 5m 47s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 26s | hbase-client in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 11s | hbase-client in the patch passed. | | | | 24m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2027/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2027 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 098b4ddabdf8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 287f29818f | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2027/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2027/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2027/1/testReport/ | | Max. process+thread count | 272 (vs. ulimit of 12500) | | modules | C: hbase-client U: hbase-client | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2027/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24663) Add procedure process time statistics UI
[ https://issues.apache.org/jira/browse/HBASE-24663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152002#comment-17152002 ] Junhong Xu commented on HBASE-24663: May I take this issue, sir? [~zghao] > Add procedure process time statistics UI > > > Key: HBASE-24663 > URL: https://issues.apache.org/jira/browse/HBASE-24663 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Priority: Major > > Added in "Procedures & Locks" jsp. > For the first version UI, we care about the process time of > ServerCrashProcedure, TRSP, OpenRegionProcedure and CloseRegionProcedure. > Plan to show the avg/P50/P90/min/max process time of these procedures. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2026: HBASE-22738 Fallback to default group to choose RS when there are no …
Apache-HBase commented on pull request #2026: URL: https://github.com/apache/hbase/pull/2026#issuecomment-654159689 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 34s | branch-2.3 passed | | +1 :green_heart: | checkstyle | 0m 15s | branch-2.3 passed | | +1 :green_heart: | spotbugs | 0m 42s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 12s | the patch passed | | +1 :green_heart: | checkstyle | 0m 13s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 16m 44s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 0m 48s | the patch passed | ||| _ Other Tests _ | | -1 :x: | asflicense | 0m 14s | The patch generated 1 ASF License warnings. | | | | 33m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2026/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2026 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 03a040459beb 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 5d5b156ec3 | | asflicense | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2026/1/artifact/yetus-general-check/output/patch-asflicense-problems.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-rsgroup U: hbase-rsgroup | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2026/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2025: HBASE-24489 Rewrite TestClusterRestartFailover.test since namespace t…
Apache-HBase commented on pull request #2025: URL: https://github.com/apache/hbase/pull/2025#issuecomment-654236150 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 15s | master passed | | +1 :green_heart: | compile | 0m 59s | master passed | | +1 :green_heart: | shadedjars | 6m 16s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 6m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 232m 44s | hbase-server in the patch passed. | | | | 260m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2025/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2025 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7629df3c7b48 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 287f29818f | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2025/1/testReport/ | | Max. process+thread count | 2612 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2025/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] timoha commented on pull request #1826: HBASE-24438 Don't update TaskMonitor when deserializing ServerCrashProcedure
timoha commented on pull request #1826: URL: https://github.com/apache/hbase/pull/1826#issuecomment-654381936 Looks like operator intervention is needed for this issue :) > I for one do not look at TaskMonitor figuring state of Procedures. Do others? Thanks. Just from my perspective, I find it useful to see the procedure progress as looking plainly at procedure list isn't as helpful. In ideal world, I wouldn't need this information at all, as it would just do its job (and would only show something when it's broken). However, since this task exists, it should not have false positives. To make it clear, I'm against "improving" this side-effect as I wouldn't find it helpful to me as operator (I just don't care that something is de-serializing), that was just a suggestion that I now regret bringing up. I'm ok with closing this PR if you decide to go that way. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24376) MergeNormalizer is merging non-adjacent regions and causing region overlaps/holes.
[
https://issues.apache.org/jira/browse/HBASE-24376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-24376:
-
Fix Version/s: (was: 2.4.0)
> MergeNormalizer is merging non-adjacent regions and causing region
> overlaps/holes.
> --
>
> Key: HBASE-24376
> URL: https://issues.apache.org/jira/browse/HBASE-24376
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.3.0
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.0
>
>
> Currently, we found normalizer was merging regions which are non-adjacent, it
> will cause inconsistencies in the cluster.
> {code:java}
> 439055 2020-05-08 17:47:09,814 INFO
> org.apache.hadoop.hbase.master.normalizer.MergeNormalizationPlan: Executing
> merging normalization plan: MergeNormalizationPlan{firstRegion={ENCODED =>
> 47fe236a5e3649ded95cb64ad0c08492, NAME =>
> 'TABLE,\x03\x01\x05\x01\x04\x02,1554838974870.47fe236a5e3649ded95cb64ad
> 0c08492.', STARTKEY => '\x03\x01\x05\x01\x04\x02', ENDKEY =>
> '\x03\x01\x05\x01\x04\x02\x01\x02\x02201904082200\x00\x00\x03Mac\x00\x00\x00\x00\x00\x00\x00\x00\x00iMac13,1\x00\x00\x00\x00\x00\x049.3-14E260\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x05'},
> secondRegion={ENCODED => 0c0f2aa67f4329d5c4 8ba0320f173d31, NAME =>
> 'TABLE,\x03\x01\x05\x02\x01\x01,1554830735526.0c0f2aa67f4329d5c48ba0320f173d31.',
> STARTKEY => '\x03\x01\x05\x02\x01\x01', ENDKEY =>
> '\x03\x01\x05\x02\x01\x02'}}
> 439056 2020-05-08 17:47:11,438 INFO org.apache.hadoop.hbase.ScheduledChore:
> CatalogJanitor-*:16000 average execution time: 1676219193 ns.
> 439057 2020-05-08 17:47:11,730 INFO org.apache.hadoop.hbase.master.HMaster:
> Client=null/null merge regions [47fe236a5e3649ded95cb64ad0c08492],
> [0c0f2aa67f4329d5c48ba0320f173d31]
> {code}
>
> The root cause is that getMergeNormalizationPlan() uses a list of regionInfo
> which is ordered by regionName. regionName does not necessary guarantee the
> order of STARTKEY (let's say 'aa1', 'aa1!', in order of regionName, it will
> be 'aa1!' followed by 'aa1'. This will result in normalizer merging
> non-adjacent regions into one and creates overlaps. This is not an issue in
> branch-1 as the list is already ordered by RegionInfo.COMPARATOR in
> normalizer.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24665) all wal of RegionGroupingProvider together roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-24665: - Fix Version/s: (was: 2.3.0) 2.3.1 > all wal of RegionGroupingProvider together roll > --- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.3.0, master, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 2.1.10, 1.4.14, 2.2.7 > > > when use RegionGroupingProvider, any a wal request roll, all wal will be > together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2006: HBASE-24632 Enable procedure-based log splitting as default in hbase3
Apache-HBase commented on pull request #2006: URL: https://github.com/apache/hbase/pull/2006#issuecomment-654404214 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 43s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 34s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 34s | branch-2 passed | | +1 :green_heart: | spotbugs | 2m 44s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 23s | the patch passed | | -0 :warning: | checkstyle | 1m 3s | hbase-server: The patch generated 1 new + 22 unchanged - 4 fixed = 23 total (was 26) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 20s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 39m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2006/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2006 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 05bf0e4a77e2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 5416cef27f | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2006/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2006/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24625) AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length.
[
https://issues.apache.org/jira/browse/HBASE-24625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152250#comment-17152250
]
Michael Stack commented on HBASE-24625:
---
I'd have reopened it because it is failing branch-2. See
[https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html]
See the bottom half of the screen where
replication.regionserver.TestWALEntryStream fails since #6494.
Here is what happens when I try test locally:
{code:java}
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] org.apache.hadoop.hbase.replication.regionserver.TestWALEntryStream.null
[ERROR] Run 1: TestWALEntryStream.testReplicationSourceWALReaderRecovered:442
» TestTimedOut ...
[ERROR] Run 2: TestWALEntryStream » Appears to be stuck in thread
AsyncFSWAL-1-1
[INFO]
[ERROR] TestWALEntryStream.testReplicationSourceWALReaderRecovered:442 »
Interrupted
[INFO]
[ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0
{code}
Will try and take a look later...
> AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced
> file length.
>
>
> Key: HBASE-24625
> URL: https://issues.apache.org/jira/browse/HBASE-24625
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 2.1.0, 2.0.0, 2.2.0, 2.3.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.2.6
>
>
> By HBASE-14004, we introduce {{WALFileLengthProvider}} interface to keep the
> current writing wal file length by ourselves, {{WALEntryStream}} used by
> {{ReplicationSourceWALReader}} could only read WAL file byte size <=
> {{WALFileLengthProvider.getLogFileSizeIfBeingWritten}} if the WAL file is
> current been writing on the same RegionServer .
> {{AsyncFSWAL}} implements {{WALFileLengthProvider}} by
> {{AbstractFSWAL.getLogFileSizeIfBeingWritten}}, just as folllows :
> {code:java}
>public OptionalLong getLogFileSizeIfBeingWritten(Path path) {
> rollWriterLock.lock();
> try {
> Path currentPath = getOldPath();
> if (path.equals(currentPath)) {
> W writer = this.writer;
> return writer != null ? OptionalLong.of(writer.getLength()) :
> OptionalLong.empty();
> } else {
> return OptionalLong.empty();
> }
> } finally {
> rollWriterLock.unlock();
> }
> }
> {code}
> For {{AsyncFSWAL}}, above {{AsyncFSWAL.writer}} is
> {{AsyncProtobufLogWriter}} ，and {{AsyncProtobufLogWriter.getLength}} is as
> follows:
> {code:java}
> public long getLength() {
> return length.get();
> }
> {code}
> But for {{AsyncProtobufLogWriter}}, any append method may increase the above
> {{AsyncProtobufLogWriter.length}}, especially for following
> {{AsyncFSWAL.append}}
> method just appending the {{WALEntry}} to
> {{FanOutOneBlockAsyncDFSOutput.buf}}:
> {code:java}
> public void append(Entry entry) {
> int buffered = output.buffered();
> try {
> entry.getKey().
>
> getBuilder(compressor).setFollowingKvCount(entry.getEdit().size()).build()
> .writeDelimitedTo(asyncOutputWrapper);
> } catch (IOException e) {
> throw new AssertionError("should not happen", e);
> }
>
> try {
>for (Cell cell : entry.getEdit().getCells()) {
> cellEncoder.write(cell);
>}
> } catch (IOException e) {
>throw new AssertionError("should not happen", e);
> }
> length.addAndGet(output.buffered() - buffered);
> }
> {code}
> That is to say, {{AsyncFSWAL.getLogFileSizeIfBeingWritten}} could not reflect
> the file length which successfully synced to underlying HDFS, which is not
> as expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2028: Backport "HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length. (#1970)" to branch-2.3
Apache-HBase commented on pull request #2028: URL: https://github.com/apache/hbase/pull/2028#issuecomment-654378957 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.3 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 55s | branch-2.3 passed | | +1 :green_heart: | checkstyle | 1m 30s | branch-2.3 passed | | +1 :green_heart: | spotbugs | 2m 37s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 37s | the patch passed | | +1 :green_heart: | checkstyle | 0m 11s | The patch passed checkstyle in hbase-asyncfs | | +1 :green_heart: | checkstyle | 1m 14s | hbase-server: The patch generated 0 new + 44 unchanged - 3 fixed = 44 total (was 47) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 41s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 54s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 45m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2028/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2028 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux b5d4ed50dfbb 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 5d5b156ec3 | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-asyncfs hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2028/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2028: Backport "HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length. (#1970)" to branch-2.3
Apache-HBase commented on pull request #2028: URL: https://github.com/apache/hbase/pull/2028#issuecomment-654373837 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 44s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 39s | branch-2.3 passed | | +1 :green_heart: | compile | 1m 17s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 5m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 50s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 23s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | shadedjars | 4m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 49s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 46s | hbase-asyncfs in the patch passed. | | -1 :x: | unit | 7m 52s | hbase-server in the patch failed. | | | | 33m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2028/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2028 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f0de9667dbd2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 5d5b156ec3 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2028/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2028/1/testReport/ | | Max. process+thread count | 767 (vs. ulimit of 12500) | | modules | C: hbase-asyncfs hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2028/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2017: HBASE-24669 Logging of ppid should be consistent across all occurrences
ndimiduk commented on a change in pull request #2017:
URL: https://github.com/apache/hbase/pull/2017#discussion_r450379754
##
File path:
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
##
@@ -1842,7 +1842,7 @@ private void
countDownChildren(RootProcedureState procStack,
store.update(parent);
scheduler.addFront(parent);
LOG.info("Finished subprocedure pid={}, resume processing parent {}",
- procedure.getProcId(), parent);
+ procedure.getProcId(), parent.toString().replace("pid=","ppid="));
Review comment:
Oh, I see. Good find. How about `"Finished subprocedure (pid={}), resume
processing of parent (ppid={})`, and use `parent.getProcId()` instead of the
string replace.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #2006: HBASE-24632 Enable procedure-based log splitting as default in hbase3
saintstack commented on pull request #2006: URL: https://github.com/apache/hbase/pull/2006#issuecomment-654389336 Test faillures are because we try to delete a non-empty directory. The left-over WALs are meta WALs but for meta regions that have since moved to another server (after a successful close); i.e. they WALs are no longer needed... They are for archive. The old zk-based WAL splitter specifically handled this case archiving remaining meta files if the crashed server was NOT carrying meta. Added this special handling to the new procedure-based WAL split which was missing it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24546) CloneSnapshotProcedure unlimited retry
[
https://issues.apache.org/jira/browse/HBASE-24546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-24546:
-
Fix Version/s: (was: 2.3.1)
1.3.0
> CloneSnapshotProcedure unlimited retry
> --
>
> Key: HBASE-24546
> URL: https://issues.apache.org/jira/browse/HBASE-24546
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 2.3.0, master, 2.2.5
>Reporter: wenfeiyi666
>Assignee: wenfeiyi666
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.3.0, 2.2.6
>
>
> since regions dir was not remove in the previous execution created, need to
> be remove when retrying, resulting in exception, unlimited retry
> {code:java}
> procedure.CloneSnapshotProcedure: Retriable error trying to clone
> snapshot=snapshot_test to table=test:backup
> state=CLONE_SNAPSHOT_WRITE_FS_LAYOUT
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotException: clone snapshot={
> ss=snapshot_test table=test:backup type=FLUSH } failed because A clone should
> not have regions to remove
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:434)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFsLayout(CloneSnapshotProcedure.java:465)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFilesystemLayout(CloneSnapshotProcedure.java:392)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:67)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:194)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:962)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1662)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1409)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1979)
> Caused by: java.lang.IllegalArgumentException: A clone should not have
> regions to remove
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:418)
> ... 10 more
> {code}
> and the cloned regions name are unchanged, resulting in new created regions
> be removed when retrying
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-11288) Splittable Meta
[ https://issues.apache.org/jira/browse/HBASE-11288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152090#comment-17152090 ] Duo Zhang commented on HBASE-11288: --- Any updates here? Thanks. > Splittable Meta > --- > > Key: HBASE-11288 > URL: https://issues.apache.org/jira/browse/HBASE-11288 > Project: HBase > Issue Type: Umbrella > Components: meta >Reporter: Francis Christopher Liu >Assignee: Francis Christopher Liu >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24625) AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length.
[
https://issues.apache.org/jira/browse/HBASE-24625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reopened HBASE-24625:
--
Reopening to apply patch to branch-2.3
> AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced
> file length.
>
>
> Key: HBASE-24625
> URL: https://issues.apache.org/jira/browse/HBASE-24625
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 2.1.0, 2.0.0, 2.2.0, 2.3.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0, 2.2.6
>
>
> By HBASE-14004, we introduce {{WALFileLengthProvider}} interface to keep the
> current writing wal file length by ourselves, {{WALEntryStream}} used by
> {{ReplicationSourceWALReader}} could only read WAL file byte size <=
> {{WALFileLengthProvider.getLogFileSizeIfBeingWritten}} if the WAL file is
> current been writing on the same RegionServer .
> {{AsyncFSWAL}} implements {{WALFileLengthProvider}} by
> {{AbstractFSWAL.getLogFileSizeIfBeingWritten}}, just as folllows :
> {code:java}
>public OptionalLong getLogFileSizeIfBeingWritten(Path path) {
> rollWriterLock.lock();
> try {
> Path currentPath = getOldPath();
> if (path.equals(currentPath)) {
> W writer = this.writer;
> return writer != null ? OptionalLong.of(writer.getLength()) :
> OptionalLong.empty();
> } else {
> return OptionalLong.empty();
> }
> } finally {
> rollWriterLock.unlock();
> }
> }
> {code}
> For {{AsyncFSWAL}}, above {{AsyncFSWAL.writer}} is
> {{AsyncProtobufLogWriter}} ，and {{AsyncProtobufLogWriter.getLength}} is as
> follows:
> {code:java}
> public long getLength() {
> return length.get();
> }
> {code}
> But for {{AsyncProtobufLogWriter}}, any append method may increase the above
> {{AsyncProtobufLogWriter.length}}, especially for following
> {{AsyncFSWAL.append}}
> method just appending the {{WALEntry}} to
> {{FanOutOneBlockAsyncDFSOutput.buf}}:
> {code:java}
> public void append(Entry entry) {
> int buffered = output.buffered();
> try {
> entry.getKey().
>
> getBuilder(compressor).setFollowingKvCount(entry.getEdit().size()).build()
> .writeDelimitedTo(asyncOutputWrapper);
> } catch (IOException e) {
> throw new AssertionError("should not happen", e);
> }
>
> try {
>for (Cell cell : entry.getEdit().getCells()) {
> cellEncoder.write(cell);
>}
> } catch (IOException e) {
>throw new AssertionError("should not happen", e);
> }
> length.addAndGet(output.buffered() - buffered);
> }
> {code}
> That is to say, {{AsyncFSWAL.getLogFileSizeIfBeingWritten}} could not reflect
> the file length which successfully synced to underlying HDFS, which is not
> as expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24625) AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length.
[
https://issues.apache.org/jira/browse/HBASE-24625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-24625:
-
Fix Version/s: (was: 2.4.0)
(was: 2.3.1)
2.3.0
> AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced
> file length.
>
>
> Key: HBASE-24625
> URL: https://issues.apache.org/jira/browse/HBASE-24625
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 2.1.0, 2.0.0, 2.2.0, 2.3.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.2.6
>
>
> By HBASE-14004, we introduce {{WALFileLengthProvider}} interface to keep the
> current writing wal file length by ourselves, {{WALEntryStream}} used by
> {{ReplicationSourceWALReader}} could only read WAL file byte size <=
> {{WALFileLengthProvider.getLogFileSizeIfBeingWritten}} if the WAL file is
> current been writing on the same RegionServer .
> {{AsyncFSWAL}} implements {{WALFileLengthProvider}} by
> {{AbstractFSWAL.getLogFileSizeIfBeingWritten}}, just as folllows :
> {code:java}
>public OptionalLong getLogFileSizeIfBeingWritten(Path path) {
> rollWriterLock.lock();
> try {
> Path currentPath = getOldPath();
> if (path.equals(currentPath)) {
> W writer = this.writer;
> return writer != null ? OptionalLong.of(writer.getLength()) :
> OptionalLong.empty();
> } else {
> return OptionalLong.empty();
> }
> } finally {
> rollWriterLock.unlock();
> }
> }
> {code}
> For {{AsyncFSWAL}}, above {{AsyncFSWAL.writer}} is
> {{AsyncProtobufLogWriter}} ，and {{AsyncProtobufLogWriter.getLength}} is as
> follows:
> {code:java}
> public long getLength() {
> return length.get();
> }
> {code}
> But for {{AsyncProtobufLogWriter}}, any append method may increase the above
> {{AsyncProtobufLogWriter.length}}, especially for following
> {{AsyncFSWAL.append}}
> method just appending the {{WALEntry}} to
> {{FanOutOneBlockAsyncDFSOutput.buf}}:
> {code:java}
> public void append(Entry entry) {
> int buffered = output.buffered();
> try {
> entry.getKey().
>
> getBuilder(compressor).setFollowingKvCount(entry.getEdit().size()).build()
> .writeDelimitedTo(asyncOutputWrapper);
> } catch (IOException e) {
> throw new AssertionError("should not happen", e);
> }
>
> try {
>for (Cell cell : entry.getEdit().getCells()) {
> cellEncoder.write(cell);
>}
> } catch (IOException e) {
>throw new AssertionError("should not happen", e);
> }
> length.addAndGet(output.buffered() - buffered);
> }
> {code}
> That is to say, {{AsyncFSWAL.getLogFileSizeIfBeingWritten}} could not reflect
> the file length which successfully synced to underlying HDFS, which is not
> as expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24546) CloneSnapshotProcedure unlimited retry
[
https://issues.apache.org/jira/browse/HBASE-24546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152079#comment-17152079
]
Hudson commented on HBASE-24546:
Results for branch branch-2.3
[build #173 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/173/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/173/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/173/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/173/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/173/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> CloneSnapshotProcedure unlimited retry
> --
>
> Key: HBASE-24546
> URL: https://issues.apache.org/jira/browse/HBASE-24546
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 2.3.0, master, 2.2.5
>Reporter: wenfeiyi666
>Assignee: wenfeiyi666
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.2.6
>
>
> since regions dir was not remove in the previous execution created, need to
> be remove when retrying, resulting in exception, unlimited retry
> {code:java}
> procedure.CloneSnapshotProcedure: Retriable error trying to clone
> snapshot=snapshot_test to table=test:backup
> state=CLONE_SNAPSHOT_WRITE_FS_LAYOUT
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotException: clone snapshot={
> ss=snapshot_test table=test:backup type=FLUSH } failed because A clone should
> not have regions to remove
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:434)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFsLayout(CloneSnapshotProcedure.java:465)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFilesystemLayout(CloneSnapshotProcedure.java:392)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:67)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:194)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:962)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1662)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1409)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1979)
> Caused by: java.lang.IllegalArgumentException: A clone should not have
> regions to remove
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:418)
> ... 10 more
> {code}
> and the cloned regions name are unchanged, resulting in new created regions
> be removed when retrying
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24635) Split TestMetaWithReplicas
[ https://issues.apache.org/jira/browse/HBASE-24635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-24635: - Fix Version/s: (was: 2.4.0) (was: 2.3.1) 2.3.0 > Split TestMetaWithReplicas > -- > > Key: HBASE-24635 > URL: https://issues.apache.org/jira/browse/HBASE-24635 > Project: HBase > Issue Type: Task > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0, 2.2.6 > > > It will stop and then start a mini cluster every time after each test method, > so let's just split them into individual test files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24615) MutableRangeHistogram#updateSnapshotRangeMetrics doesn't calculate the distribution for last bucket.
[
https://issues.apache.org/jira/browse/HBASE-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152157#comment-17152157
]
Rushabh Shah commented on HBASE-24615:
--
[~wenfeiyi666] just fyi you dont have to create a separate PR for branch-2. You
just need to have a PR for master branch and the committer will try to backport
to all the other branches. If the rebase work is more then he/she will let you
know to create another PR for those branch. Thank you !
> MutableRangeHistogram#updateSnapshotRangeMetrics doesn't calculate the
> distribution for last bucket.
>
>
> Key: HBASE-24615
> URL: https://issues.apache.org/jira/browse/HBASE-24615
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.3.0, master, 1.3.7, 2.2.6
>Reporter: Rushabh Shah
>Assignee: wenfeiyi666
>Priority: Major
>
> We are not processing the distribution for last bucket.
> https://github.com/apache/hbase/blob/master/hbase-hadoop-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableRangeHistogram.java#L70
> {code:java}
> public void updateSnapshotRangeMetrics(MetricsRecordBuilder
> metricsRecordBuilder,
> Snapshot snapshot) {
> long priorRange = 0;
> long cumNum = 0;
> final long[] ranges = getRanges();
> final String rangeType = getRangeType();
> for (int i = 0; i < ranges.length - 1; i++) { -> The bug lies
> here. We are not processing last bucket.
> long val = snapshot.getCountAtOrBelow(ranges[i]);
> if (val - cumNum > 0) {
> metricsRecordBuilder.addCounter(
> Interns.info(name + "_" + rangeType + "_" + priorRange + "-" +
> ranges[i], desc),
> val - cumNum);
> }
> priorRange = ranges[i];
> cumNum = val;
> }
> long val = snapshot.getCount();
> if (val - cumNum > 0) {
> metricsRecordBuilder.addCounter(
> Interns.info(name + "_" + rangeType + "_" + ranges[ranges.length -
> 1] + "-inf", desc),
> val - cumNum);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] huaxiangsun commented on pull request #1986: HBASE-24581 Skip compaction request/check for replica regions at the …
huaxiangsun commented on pull request #1986: URL: https://github.com/apache/hbase/pull/1986#issuecomment-654339877 @busbey @infraio Ping for comments, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #2028: Backport "HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length. (#1970)" to branch-2.3
ndimiduk opened a new pull request #2028: URL: https://github.com/apache/hbase/pull/2028 Signed-off-by: Duo Zhang This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on pull request #1962: HBASE-24615 MutableRangeHistogram#updateSnapshotRangeMetrics doesn't calculate the distribution for last bucket.
shahrs87 commented on pull request #1962: URL: https://github.com/apache/hbase/pull/1962#issuecomment-654348836 @WenFeiYi Thank you for the PR. Mind writing a small test case for this. Overall the code looks good to me. Thank you ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #2028: Backport "HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length. (#1970)" to branch-2.3
ndimiduk commented on pull request #2028: URL: https://github.com/apache/hbase/pull/2028#issuecomment-654356936 Patch from branch-2 applies cleanly to branch-2.3. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #2009: HBASE-21596 Delete for a specific cell version can bring back version…
wchevreuil commented on a change in pull request #2009:
URL: https://github.com/apache/hbase/pull/2009#discussion_r450329196
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -3170,37 +3171,87 @@ public void prepareDeleteTimestamps(Mutation mutation,
Map> f
count = kvCount.get(qual);
Get get = new Get(CellUtil.cloneRow(cell));
- get.readVersions(count);
- get.addColumn(family, qual);
+ get.readVersions(Integer.MAX_VALUE);
if (coprocessorHost != null) {
if (!coprocessorHost.prePrepareTimeStampForDeleteVersion(mutation,
cell,
byteNow, get)) {
- updateDeleteLatestVersionTimestamp(cell, get, count, byteNow);
+ updateDeleteLatestVersionTimestamp(cell, get, count,
+
this.htableDescriptor.getColumnFamily(family).getMaxVersions(),
+byteNow, deleteCells);
+
}
} else {
-updateDeleteLatestVersionTimestamp(cell, get, count, byteNow);
+updateDeleteLatestVersionTimestamp(cell, get, count,
+this.htableDescriptor.getColumnFamily(family).getMaxVersions(),
+ byteNow, deleteCells);
}
} else {
PrivateCellUtil.updateLatestStamp(cell, byteNow);
+ deleteCells.add(cell);
}
}
+ e.setValue(deleteCells);
}
}
- void updateDeleteLatestVersionTimestamp(Cell cell, Get get, int count,
byte[] byteNow)
- throws IOException {
-List result = get(get, false);
-
+ private void updateDeleteLatestVersionTimestamp(Cell cell, Get get, int
count, int maxVersions,
+ byte[] byteNow, List deleteCells) throws IOException {
+List result = new ArrayList<>(deleteCells);
+Scan scan = new Scan(get);
+scan.setRaw(true);
+this.getScanner(scan).next(result);
+List cells = new ArrayList<>();
if (result.size() < count) {
// Nothing to delete
PrivateCellUtil.updateLatestStamp(cell, byteNow);
- return;
-}
-if (result.size() > count) {
- throw new RuntimeException("Unexpected size: " + result.size());
+ cells.add(cell);
+ deleteCells.addAll(cells);
+} else if (result.size() > count) {
+ int currentVersion = 0;
+ long latestCellTS = Long.MAX_VALUE;
+ result.sort((cell1, cell2) -> {
+if(cell1.getTimestamp()>cell2.getTimestamp()){
+ return -1;
+} else if(cell1.getTimestamp()= maxVersions) {
+Cell tempCell = null;
+try {
+ tempCell = PrivateCellUtil.deepClone(cell);
+} catch (CloneNotSupportedException e) {
+ throw new IOException(e);
+}
+PrivateCellUtil.setTimestamp(tempCell, getCell.getTimestamp());
+cells.add(tempCell);
+ } else if (currentVersion == 0) {
+PrivateCellUtil.setTimestamp(cell, getCell.getTimestamp());
+cells.add(cell);
+ }
+ currentVersion++;
+}
+latestCellTS = getCell.getTimestamp();
+ }
+
+} else {
+ Cell getCell = result.get(0);
Review comment:
It's not needed, because we don't have to worry about additional
versions, we only need to put a single marker for current TS.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2018: HBASE-24659 Calculate FIXED_OVERHEAD automatically
saintstack commented on a change in pull request #2018:
URL: https://github.com/apache/hbase/pull/2018#discussion_r449899043
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -8405,12 +8405,7 @@ private static long getLongValue(final Cell cell) throws
DoNotRetryIOException {
return cells;
}
- public static final long FIXED_OVERHEAD = ClassSize.align(
- ClassSize.OBJECT +
- 56 * ClassSize.REFERENCE +
- 3 * Bytes.SIZEOF_INT +
- 14 * Bytes.SIZEOF_LONG +
- 3 * Bytes.SIZEOF_BOOLEAN);
+ public static final long FIXED_OVERHEAD =
ClassSize.estimateBase(HRegion.class, false);
Review comment:
Does ClassSize come up w/ same general numbers as old manual technique.
It does deep size rather than shallow?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2027: HBASE-24685 : Removing MultiAction and FailureInfo
Apache-HBase commented on pull request #2027: URL: https://github.com/apache/hbase/pull/2027#issuecomment-654167129 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 1s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 6s | master passed | | +1 :green_heart: | checkstyle | 0m 29s | master passed | | +1 :green_heart: | spotbugs | 1m 1s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 45s | the patch passed | | +1 :green_heart: | checkstyle | 0m 27s | hbase-client: The patch generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 26s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 1m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 32m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2027/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2027 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 58bd482677b1 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 287f29818f | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-client U: hbase-client | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2027/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-24685) MultiAction and FailureInfo should be removed
[ https://issues.apache.org/jira/browse/HBASE-24685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani resolved HBASE-24685. -- Fix Version/s: 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed > MultiAction and FailureInfo should be removed > - > > Key: HBASE-24685 > URL: https://issues.apache.org/jira/browse/HBASE-24685 > Project: HBase > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > Just came across MultiAction and FailureInfo which are IA.Private and not > being used anywhere on trunk. Both of them are being used on branch-2 though. > We should remove them on trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24546) CloneSnapshotProcedure unlimited retry
[
https://issues.apache.org/jira/browse/HBASE-24546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151973#comment-17151973
]
Hudson commented on HBASE-24546:
Results for branch master
[build #1778 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1663//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> CloneSnapshotProcedure unlimited retry
> --
>
> Key: HBASE-24546
> URL: https://issues.apache.org/jira/browse/HBASE-24546
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 2.3.0, master, 2.2.5
>Reporter: wenfeiyi666
>Assignee: wenfeiyi666
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.2.6
>
>
> since regions dir was not remove in the previous execution created, need to
> be remove when retrying, resulting in exception, unlimited retry
> {code:java}
> procedure.CloneSnapshotProcedure: Retriable error trying to clone
> snapshot=snapshot_test to table=test:backup
> state=CLONE_SNAPSHOT_WRITE_FS_LAYOUT
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotException: clone snapshot={
> ss=snapshot_test table=test:backup type=FLUSH } failed because A clone should
> not have regions to remove
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:434)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFsLayout(CloneSnapshotProcedure.java:465)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFilesystemLayout(CloneSnapshotProcedure.java:392)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:67)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:194)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:962)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1662)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1409)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1979)
> Caused by: java.lang.IllegalArgumentException: A clone should not have
> regions to remove
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:418)
> ... 10 more
> {code}
> and the cloned regions name are unchanged, resulting in new created regions
> be removed when retrying
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24671) Add excludefile and designatedfile options to graceful_stop.sh
[
https://issues.apache.org/jira/browse/HBASE-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151971#comment-17151971
]
Hudson commented on HBASE-24671:
Results for branch master
[build #1778 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1663//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add excludefile and designatedfile options to graceful_stop.sh
> --
>
> Key: HBASE-24671
> URL: https://issues.apache.org/jira/browse/HBASE-24671
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1, 2.4.0
>Reporter: Baiqiang Zhao
>Assignee: Baiqiang Zhao
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> RegionMover is support excludefile and designatedfile options now. Integrate
> these two options into graceful_stop.sh.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24578) [WAL] Add a parameter to config RingBufferEventHandler's SyncFuture count
[
https://issues.apache.org/jira/browse/HBASE-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151972#comment-17151972
]
Hudson commented on HBASE-24578:
Results for branch master
[build #1778 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1663//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1778/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [WAL] Add a parameter to config RingBufferEventHandler's SyncFuture count
> -
>
> Key: HBASE-24578
> URL: https://issues.apache.org/jira/browse/HBASE-24578
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Affects Versions: 1.4.13, 2.2.5
>Reporter: Reid Chan
>Assignee: wenfeiyi666
>Priority: Major
>
> The current value of RingBufferEventHandler's handler is the value of
> {{hbase.regionserver.handler.count}}, which works good in default wal
> provider --- one WAL per regionserver.
> When trying to use WAL group provider, either by group or wal per region, the
> default value is bad. If rs has 100 regions and wal per region strategy is
> used, then rs will allocate 100 *
> SyncFuture[$hbase.regionserver.handler.count] array
> {code}
> int maxHandlersCount = conf.getInt(HConstants.REGION_SERVER_HANDLER_COUNT,
> 200);
> this.ringBufferEventHandler = new RingBufferEventHandler(
> conf.getInt("hbase.regionserver.hlog.syncer.count", 5),
> maxHandlersCount);
> ...
>
> RingBufferEventHandler(final int syncRunnerCount, final int maxHandlersCount)
> {
> this.syncFutures = new SyncFuture[maxHandlersCount];
> ...
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24686) [LOG] Log improvement in Connection#close
[ https://issues.apache.org/jira/browse/HBASE-24686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mokai updated HBASE-24686: -- Description: We met some customers used hbase connection improperly, some threads call failed since the shared connection closed by one of the threads. It's better to print the details when connection closing. was:We met some customers used hbase connection improperly, some threads call failed since the shared connection closed by one of the threads. > [LOG] Log improvement in Connection#close > - > > Key: HBASE-24686 > URL: https://issues.apache.org/jira/browse/HBASE-24686 > Project: HBase > Issue Type: Improvement > Components: Client, logging >Affects Versions: 2.2.3 >Reporter: mokai >Priority: Major > > We met some customers used hbase connection improperly, some threads call > failed since the shared connection closed by one of the threads. > It's better to print the details when connection closing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on pull request #1926: HBASE-24586 Add table level locality in table.jsp
virajjasani commented on pull request #1926: URL: https://github.com/apache/hbase/pull/1926#issuecomment-654214933 @bsglz Thanks for the reminder. Will take a look in some time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24625) AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length.
[
https://issues.apache.org/jira/browse/HBASE-24625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152383#comment-17152383
]
Nick Dimiduk commented on HBASE-24625:
--
>From a thread dump of {{TestWALEntryStream}} when the test gets killed
{noformat}
"Time-limited test"
java.lang.Thread.State: TIMED_WAITING
at java.base@11.0.4/java.lang.Object.wait(Native Method)
at
app//org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:142)
at
app//org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:752)
at
app//org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:645)
at
app//org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:604)
at
app//org.apache.hadoop.hbase.replication.regionserver.TestWALEntryStream.appendToLogAndSync(TestWALEntryStream.java:581)
at
app//org.apache.hadoop.hbase.replication.regionserver.TestWALEntryStream.testDifferentCounts(TestWALEntryStream.java:161)
{noformat}
> AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced
> file length.
>
>
> Key: HBASE-24625
> URL: https://issues.apache.org/jira/browse/HBASE-24625
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 2.1.0, 2.0.0, 2.2.0, 2.3.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.2.6
>
>
> By HBASE-14004, we introduce {{WALFileLengthProvider}} interface to keep the
> current writing wal file length by ourselves, {{WALEntryStream}} used by
> {{ReplicationSourceWALReader}} could only read WAL file byte size <=
> {{WALFileLengthProvider.getLogFileSizeIfBeingWritten}} if the WAL file is
> current been writing on the same RegionServer .
> {{AsyncFSWAL}} implements {{WALFileLengthProvider}} by
> {{AbstractFSWAL.getLogFileSizeIfBeingWritten}}, just as folllows :
> {code:java}
>public OptionalLong getLogFileSizeIfBeingWritten(Path path) {
> rollWriterLock.lock();
> try {
> Path currentPath = getOldPath();
> if (path.equals(currentPath)) {
> W writer = this.writer;
> return writer != null ? OptionalLong.of(writer.getLength()) :
> OptionalLong.empty();
> } else {
> return OptionalLong.empty();
> }
> } finally {
> rollWriterLock.unlock();
> }
> }
> {code}
> For {{AsyncFSWAL}}, above {{AsyncFSWAL.writer}} is
> {{AsyncProtobufLogWriter}} ，and {{AsyncProtobufLogWriter.getLength}} is as
> follows:
> {code:java}
> public long getLength() {
> return length.get();
> }
> {code}
> But for {{AsyncProtobufLogWriter}}, any append method may increase the above
> {{AsyncProtobufLogWriter.length}}, especially for following
> {{AsyncFSWAL.append}}
> method just appending the {{WALEntry}} to
> {{FanOutOneBlockAsyncDFSOutput.buf}}:
> {code:java}
> public void append(Entry entry) {
> int buffered = output.buffered();
> try {
> entry.getKey().
>
> getBuilder(compressor).setFollowingKvCount(entry.getEdit().size()).build()
> .writeDelimitedTo(asyncOutputWrapper);
> } catch (IOException e) {
> throw new AssertionError("should not happen", e);
> }
>
> try {
>for (Cell cell : entry.getEdit().getCells()) {
> cellEncoder.write(cell);
>}
> } catch (IOException e) {
>throw new AssertionError("should not happen", e);
> }
> length.addAndGet(output.buffered() - buffered);
> }
> {code}
> That is to say, {{AsyncFSWAL.getLogFileSizeIfBeingWritten}} could not reflect
> the file length which successfully synced to underlying HDFS, which is not
> as expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24688) AssignRegionHandler uses EventType.M_RS_CLOSE_META instead of EventType.M_RS_OPEN_META
Huaxiang Sun created HBASE-24688: Summary: AssignRegionHandler uses EventType.M_RS_CLOSE_META instead of EventType.M_RS_OPEN_META Key: HBASE-24688 URL: https://issues.apache.org/jira/browse/HBASE-24688 Project: HBase Issue Type: Bug Reporter: Huaxiang Sun This results in openMetaRegion always be executed in closeMetaExecutor. -- This message was sent by Atlassian Jira (v8.3.4#803005)
