[GitHub] [hbase] infraio commented on pull request #2008: HBASE-24665 all wal of RegionGroupingProvider together roll
infraio commented on pull request #2008: URL: https://github.com/apache/hbase/pull/2008#issuecomment-654031282 Can you add more comments to explain "why the fix worked"? And add a UT for this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-22146) SpaceQuotaViolationPolicy Disable is not working in Namespace level
[ https://issues.apache.org/jira/browse/HBASE-22146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22146: --- Fix Version/s: (was: 2.2.6) 2.2.7 > SpaceQuotaViolationPolicy Disable is not working in Namespace level > --- > > Key: HBASE-22146 > URL: https://issues.apache.org/jira/browse/HBASE-22146 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.0.0 >Reporter: Uma Maheswari >Assignee: Surbhi Kochhar >Priority: Major > Labels: Quota, space > Fix For: 3.0.0-alpha-1, 2.2.7 > > > SpaceQuotaViolationPolicy Disable is not working in Namespace level > PFB the steps: > * Create Namespace and set Quota violation policy as Disable > * Create tables under namespace and violate Quota > Expected result: Tables to get disabled > Actual Result: Tables are not getting disabled > Note: mutation operation is not allowed on the table -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2020: HBASE-24682 Refactor ReplicationSource#addHFileRefs method: move it t…
Apache-HBase commented on pull request #2020: URL: https://github.com/apache/hbase/pull/2020#issuecomment-654028817 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-24666 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 18s | HBASE-24666 passed | | +1 :green_heart: | compile | 1m 3s | HBASE-24666 passed | | +1 :green_heart: | shadedjars | 5m 44s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 42s | hbase-server in HBASE-24666 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 3s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 5m 47s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 139m 31s | hbase-server in the patch passed. | | | | 165m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2020/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2020 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f82d59bc50d2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / fe2ae809d1 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2020/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2020/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2020/1/testReport/ | | Max. process+thread count | 4408 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2020/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio commented on pull request #1935: HBASE-22146 SpaceQuotaViolationPolicy Disable is not working in Names…
infraio commented on pull request #1935: URL: https://github.com/apache/hbase/pull/1935#issuecomment-654028642 > HBASE-22146 SpaceQuotaViolationPolicy Disable is not working in Namespace level > > When a namespace space quota is deleted, the namespace quota (q:s) row from hbase:quota table is removed, however the related u:p rows from tables in the namespace still stay. Removing the u:p rows for tables in the namespace to fix this issue. How this fix related to the issue description? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio merged pull request #1900: HBASE-24546 CloneSnapshotProcedure unlimited retry
infraio merged pull request #1900: URL: https://github.com/apache/hbase/pull/1900 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio merged pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
infraio merged pull request #1916: URL: https://github.com/apache/hbase/pull/1916 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2019: HBASE-24681 Remove the cache walsById/walsByIdRecoveredQueues from Re…
Apache-HBase commented on pull request #2019: URL: https://github.com/apache/hbase/pull/2019#issuecomment-654022149 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-24666 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | HBASE-24666 passed | | +1 :green_heart: | compile | 1m 4s | HBASE-24666 passed | | +1 :green_heart: | shadedjars | 5m 51s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 44s | hbase-server in HBASE-24666 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 59s | the patch passed | | +1 :green_heart: | compile | 1m 3s | the patch passed | | +1 :green_heart: | javac | 1m 3s | the patch passed | | +1 :green_heart: | shadedjars | 5m 44s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 136m 11s | hbase-server in the patch failed. | | | | 162m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2019 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 24549fb1185f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / fe2ae809d1 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/testReport/ | | Max. process+thread count | 4761 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2022: HBASE-24665 all wal of RegionGroupingProvider together roll
Apache-HBase commented on pull request #2022: URL: https://github.com/apache/hbase/pull/2022#issuecomment-654019300 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | branch-2.3 passed | | +1 :green_heart: | checkstyle | 1m 32s | branch-2.3 passed | | +1 :green_heart: | spotbugs | 2m 30s | branch-2.3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 22s | the patch passed | | +1 :green_heart: | checkstyle | 1m 21s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 24m 0s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 51m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2022/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2022 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 653fdd3f6056 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 9af91d4e1f | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2022/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] WenFeiYi opened a new pull request #2024: HBASE-24615 MutableRangeHistogram#updateSnapshotRangeMetrics doesn't …
WenFeiYi opened a new pull request #2024: URL: https://github.com/apache/hbase/pull/2024 …calculate the distribution for last bucket. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2021: HBASE-24665 all wal of RegionGroupingProvider together roll
Apache-HBase commented on pull request #2021: URL: https://github.com/apache/hbase/pull/2021#issuecomment-654008844 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 46s | master passed | | +1 :green_heart: | checkstyle | 1m 4s | master passed | | +1 :green_heart: | spotbugs | 2m 4s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 27s | the patch passed | | +1 :green_heart: | checkstyle | 1m 7s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 7s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 35m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2021/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2021 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux dfbdc80e3ca0 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 44691ce23c | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2021/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] WenFeiYi opened a new pull request #2023: HBASE-24665 all wal of RegionGroupingProvider together roll
WenFeiYi opened a new pull request #2023: URL: https://github.com/apache/hbase/pull/2023 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] WenFeiYi opened a new pull request #2022: HBASE-24665 all wal of RegionGroupingProvider together roll
WenFeiYi opened a new pull request #2022: URL: https://github.com/apache/hbase/pull/2022 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-24665) all wal of RegionGroupingProvider together roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151714#comment-17151714 ] wenfeiyi666 edited comment on HBASE-24665 at 7/6/20, 3:53 AM: -- also 1.x ,2.3,master was (Author: wenfeiyi666): 1.x, 2.3, master > all wal of RegionGroupingProvider together roll > --- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.3.0, master, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0, 2.1.10, 1.4.14, 2.2.6 > > > when use RegionGroupingProvider, any a wal request roll, all wal will be > together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-24665) all wal of RegionGroupingProvider together roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151714#comment-17151714 ] wenfeiyi666 edited comment on HBASE-24665 at 7/6/20, 3:51 AM: -- 1.x, 2.3, master was (Author: wenfeiyi666): and branch-1.x > all wal of RegionGroupingProvider together roll > --- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.3.0, master, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0, 2.1.10, 1.4.14, 2.2.6 > > > when use RegionGroupingProvider, any a wal request roll, all wal will be > together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] WenFeiYi opened a new pull request #2021: HBASE-24665 all wal of RegionGroupingProvider together roll
WenFeiYi opened a new pull request #2021: URL: https://github.com/apache/hbase/pull/2021 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24503) Backport HBASE-24492 to all 2.x branch
[ https://issues.apache.org/jira/browse/HBASE-24503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-24503: --- Fix Version/s: (was: 2.2.6) 2.2.7 > Backport HBASE-24492 to all 2.x branch > -- > > Key: HBASE-24503 > URL: https://issues.apache.org/jira/browse/HBASE-24503 > Project: HBase > Issue Type: Sub-task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Fix For: 2.3.1, 2.1.10, 2.2.7 > > > After release 2.3.0 is out, we need to backport HBASE-24492 to all 2.x > branches. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22348) allow one to actually disable replication svc
[ https://issues.apache.org/jira/browse/HBASE-22348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22348: --- Fix Version/s: (was: 2.2.6) 2.2.7 > allow one to actually disable replication svc > - > > Key: HBASE-22348 > URL: https://issues.apache.org/jira/browse/HBASE-22348 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.2.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Labels: replication > Fix For: 2.2.7 > > Attachments: HBASE-22348.patch > > > Minor, but it does create extra ZK traffic for no reason and there's no way > to disable that it appears. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22917) Proc-WAL roll fails always saying someone else has already created log
[
https://issues.apache.org/jira/browse/HBASE-22917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-22917:
---
Fix Version/s: (was: 2.2.6)
2.2.7
> Proc-WAL roll fails always saying someone else has already created log
> --
>
> Key: HBASE-22917
> URL: https://issues.apache.org/jira/browse/HBASE-22917
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, wal
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.2.7
>
>
> Recently we met a weird scenario where Procedure WAL roll fails as it is
> already created by someone else.
> Later while going through the logs and code, observed that during Proc-WAL
> roll it failed to write the header. On failure file stream is just closed,
> {code}
> try {
> ProcedureWALFormat.writeHeader(newStream, header);
> startPos = newStream.getPos();
> } catch (IOException ioe) {
> LOG.warn("Encountered exception writing header", ioe);
> newStream.close();
> return false;
> }
> {code}
> Since we don't delete the corrupted file or increment the *flushLogId*, so on
> each retry it is trying to create the same *flushLogId* file. However Hmaster
> failover will resolve this issue, but we should handle it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24403) FsDelegationToken should cache Token
[
https://issues.apache.org/jira/browse/HBASE-24403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-24403:
---
Fix Version/s: (was: 2.2.6)
2.2.7
> FsDelegationToken should cache Token
>
>
> Key: HBASE-24403
> URL: https://issues.apache.org/jira/browse/HBASE-24403
> Project: HBase
> Issue Type: Bug
>Reporter: wuchang
>Assignee: wuchang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.2.7
>
> Attachments: 24403.patch
>
>
> When doing Bulkload, we find that FsDelegationToken will acquire token of
> NameNode everytime for a single file, although, from the comment of
> acquireDelegationToken(), it claims that it is trying to find token in cache
> firstly, but the newly requested token is never put to cache and thus the
> cache is still empty for the following request;
> In cases there are many files to do the bulk load, the token request will
> cause big short to NameNode.
>
> {code:java}
> public void acquireDelegationToken(final FileSystem fs)
> throws IOException {
> if (userProvider.isHadoopSecurityEnabled()) {
> this.fs = fs;
> userToken = userProvider.getCurrent().getToken("HDFS_DELEGATION_TOKEN",
> fs.getCanonicalServiceName());
> if (userToken == null) {
> hasForwardedToken = false;
> try {
> userToken = fs.getDelegationToken(renewer);
> } catch (NullPointerException npe) {
> // we need to handle NullPointerException in case HADOOP-10009 is missing
> LOG.error("Failed to get token for " + renewer);
> }
> } else {
> hasForwardedToken = true;
> LOG.info("Use the existing token: " + userToken);
> }
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2020: HBASE-24682 Refactor ReplicationSource#addHFileRefs method: move it t…
Apache-HBase commented on pull request #2020: URL: https://github.com/apache/hbase/pull/2020#issuecomment-653999426 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 39s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-24666 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 22s | HBASE-24666 passed | | +1 :green_heart: | checkstyle | 1m 19s | HBASE-24666 passed | | +1 :green_heart: | spotbugs | 2m 15s | HBASE-24666 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 55s | the patch passed | | -0 :warning: | checkstyle | 1m 17s | hbase-server: The patch generated 10 new + 4 unchanged - 0 fixed = 14 total (was 4) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 37s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 39m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2020/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2020 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux e6d9d63ca70d 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / fe2ae809d1 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2020/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2020/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1900: HBASE-24546 CloneSnapshotProcedure unlimited retry
Apache-HBase commented on pull request #1900: URL: https://github.com/apache/hbase/pull/1900#issuecomment-653999308 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 13s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | master passed | | +1 :green_heart: | checkstyle | 1m 18s | master passed | | +1 :green_heart: | spotbugs | 2m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 4s | the patch passed | | +1 :green_heart: | checkstyle | 1m 14s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 11s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 38m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1900/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1900 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 299d25a45407 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fe2ae809d1 | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1900/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24671) Add excludefile and designatedfile options to graceful_stop.sh
[ https://issues.apache.org/jira/browse/HBASE-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151740#comment-17151740 ] Anoop Sam John commented on HBASE-24671: Release notes pls. Thanks for the patch [~DeanZ] > Add excludefile and designatedfile options to graceful_stop.sh > -- > > Key: HBASE-24671 > URL: https://issues.apache.org/jira/browse/HBASE-24671 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Baiqiang Zhao >Assignee: Baiqiang Zhao >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RegionMover is support excludefile and designatedfile options now. Integrate > these two options into graceful_stop.sh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24665) all wal of RegionGroupingProvider together roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenfeiyi666 updated HBASE-24665: Fix Version/s: 1.4.14 > all wal of RegionGroupingProvider together roll > --- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.3.0, master, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0, 2.1.10, 1.4.14, 2.2.6 > > > when use RegionGroupingProvider, any a wal request roll, all wal will be > together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24665) all wal of RegionGroupingProvider together roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenfeiyi666 updated HBASE-24665: Fix Version/s: 2.1.10 2.3.0 3.0.0-alpha-1 > all wal of RegionGroupingProvider together roll > --- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.3.0, master, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0, 2.1.10, 2.2.6 > > > when use RegionGroupingProvider, any a wal request roll, all wal will be > together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24665) all wal of RegionGroupingProvider together roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenfeiyi666 updated HBASE-24665: Affects Version/s: 2.2.6 1.4.14 2.1.10 2.3.0 > all wal of RegionGroupingProvider together roll > --- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.3.0, master, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 2.2.6 > > > when use RegionGroupingProvider, any a wal request roll, all wal will be > together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24665) all wal of RegionGroupingProvider together roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wenfeiyi666 updated HBASE-24665: Affects Version/s: (was: 2.2.6) master > all wal of RegionGroupingProvider together roll > --- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug >Affects Versions: master >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 2.2.6 > > > when use RegionGroupingProvider, any a wal request roll, all wal will be > together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Reidddddd merged pull request #1923: HBASE-24578 [WAL] Add a parameter to config RingBufferEventHandler's SyncFuture count
Reidd merged pull request #1923: URL: https://github.com/apache/hbase/pull/1923 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-24682) Refactor ReplicationSource#addHFileRefs method: move it to ReplicationSourceManager
[ https://issues.apache.org/jira/browse/HBASE-24682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-24682: -- Assignee: Guanghao Zhang > Refactor ReplicationSource#addHFileRefs method: move it to > ReplicationSourceManager > --- > > Key: HBASE-24682 > URL: https://issues.apache.org/jira/browse/HBASE-24682 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2019: HBASE-24681 Remove the cache walsById/walsByIdRecoveredQueues from Re…
Apache-HBase commented on pull request #2019: URL: https://github.com/apache/hbase/pull/2019#issuecomment-653993464 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-24666 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | HBASE-24666 passed | | +1 :green_heart: | checkstyle | 1m 8s | HBASE-24666 passed | | +1 :green_heart: | spotbugs | 1m 59s | HBASE-24666 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 21s | the patch passed | | -0 :warning: | checkstyle | 1m 2s | hbase-server: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 10s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 32m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2019 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 91cf0d917179 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / fe2ae809d1 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2019/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio opened a new pull request #2020: HBASE-24682 Refactor ReplicationSource#addHFileRefs method: move it t…
infraio opened a new pull request #2020: URL: https://github.com/apache/hbase/pull/2020 …o ReplicationSourceManager This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24546) CloneSnapshotProcedure unlimited retry
[
https://issues.apache.org/jira/browse/HBASE-24546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151724#comment-17151724
]
wenfeiyi666 commented on HBASE-24546:
-
ping [~zghao]
> CloneSnapshotProcedure unlimited retry
> --
>
> Key: HBASE-24546
> URL: https://issues.apache.org/jira/browse/HBASE-24546
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 2.3.0, master, 2.2.5
>Reporter: wenfeiyi666
>Assignee: wenfeiyi666
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.2.6
>
>
> since regions dir was not remove in the previous execution created, need to
> be remove when retrying, resulting in exception, unlimited retry
> {code:java}
> procedure.CloneSnapshotProcedure: Retriable error trying to clone
> snapshot=snapshot_test to table=test:backup
> state=CLONE_SNAPSHOT_WRITE_FS_LAYOUT
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotException: clone snapshot={
> ss=snapshot_test table=test:backup type=FLUSH } failed because A clone should
> not have regions to remove
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:434)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFsLayout(CloneSnapshotProcedure.java:465)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.createFilesystemLayout(CloneSnapshotProcedure.java:392)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure.executeFromState(CloneSnapshotProcedure.java:67)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:194)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:962)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1662)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1409)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1979)
> Caused by: java.lang.IllegalArgumentException: A clone should not have
> regions to remove
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
> at
> org.apache.hadoop.hbase.master.procedure.CloneSnapshotProcedure$1.createHdfsRegions(CloneSnapshotProcedure.java:418)
> ... 10 more
> {code}
> and the cloned regions name are unchanged, resulting in new created regions
> be removed when retrying
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] WenFeiYi commented on a change in pull request #1916: HBASE-24546 CloneSnapshotProcedure unlimited retry
WenFeiYi commented on a change in pull request #1916:
URL: https://github.com/apache/hbase/pull/1916#discussion_r442743034
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CloneSnapshotProcedure.java
##
@@ -457,6 +457,10 @@ private void postCloneSnapshot(final MasterProcedureEnv
env)
// 1. Create Table Descriptor
// using a copy of descriptor, table will be created enabling first
final Path tempTableDir = FSUtils.getTableDir(tempdir,
tableDescriptor.getTableName());
+if (FSUtils.isExists(mfs.getFileSystem(), tempTableDir)) {
+ LOG.warn("temp table dir already exists on disk: {}, will be deleted.",
tempTableDir);
+ FSUtils.deleteDirectory(mfs.getFileSystem(), tempTableDir);
+}
Review comment:
if ok, I will update other commit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (HBASE-24681) Remove the "cache" walsById/walsByIdRecoveredQueues from ReplicationSourceManager
[ https://issues.apache.org/jira/browse/HBASE-24681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-24681: -- Assignee: Guanghao Zhang > Remove the "cache" walsById/walsByIdRecoveredQueues from > ReplicationSourceManager > - > > Key: HBASE-24681 > URL: https://issues.apache.org/jira/browse/HBASE-24681 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24671) Add excludefile and designatedfile options to graceful_stop.sh
[
https://issues.apache.org/jira/browse/HBASE-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151720#comment-17151720
]
Hudson commented on HBASE-24671:
Results for branch branch-2
[build #2734 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2734/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2734/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2734/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2734/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2734/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add excludefile and designatedfile options to graceful_stop.sh
> --
>
> Key: HBASE-24671
> URL: https://issues.apache.org/jira/browse/HBASE-24671
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1, 2.4.0
>Reporter: Baiqiang Zhao
>Assignee: Baiqiang Zhao
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> RegionMover is support excludefile and designatedfile options now. Integrate
> these two options into graceful_stop.sh.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] infraio commented on a change in pull request #2016: HBASE-24653 Show snapshot owner on Master WebUI
infraio commented on a change in pull request #2016: URL: https://github.com/apache/hbase/pull/2016#discussion_r449958060 ## File path: hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon ## @@ -673,6 +675,8 @@ AssignmentManager assignmentManager = master.getAssignmentManager(); <% snapshotTable.getNameAsString() %> <% new Date(snapshotDesc.getCreationTime()) %> +<% snapshotDesc.getOwner() %> +<% snapshotDesc.getTtl() %> Review comment: human readable TTL? or use TTL(Seconds)? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio opened a new pull request #2019: HBASE-24681 Remove the cache walsById/walsByIdRecoveredQueues from Re…
infraio opened a new pull request #2019: URL: https://github.com/apache/hbase/pull/2019 …plicationSourceManager This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24665) all wal of RegionGroupingProvider together roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151714#comment-17151714 ] wenfeiyi666 commented on HBASE-24665: - and branch-1.x > all wal of RegionGroupingProvider together roll > --- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug >Affects Versions: 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 2.2.6 > > > when use RegionGroupingProvider, any a wal request roll, all wal will be > together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24671) Add excludefile and designatedfile options to graceful_stop.sh
[ https://issues.apache.org/jira/browse/HBASE-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151700#comment-17151700 ] Baiqiang Zhao commented on HBASE-24671: --- Thank you [~vjasani] > Add excludefile and designatedfile options to graceful_stop.sh > -- > > Key: HBASE-24671 > URL: https://issues.apache.org/jira/browse/HBASE-24671 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Baiqiang Zhao >Assignee: Baiqiang Zhao >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RegionMover is support excludefile and designatedfile options now. Integrate > these two options into graceful_stop.sh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24682) Refactor ReplicationSource#addHFileRefs method: move it to ReplicationSourceManager
Guanghao Zhang created HBASE-24682: -- Summary: Refactor ReplicationSource#addHFileRefs method: move it to ReplicationSourceManager Key: HBASE-24682 URL: https://issues.apache.org/jira/browse/HBASE-24682 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24681) Remove the "cache" walsById/walsByIdRecoveredQueues from ReplicationSourceManager
Guanghao Zhang created HBASE-24681: -- Summary: Remove the "cache" walsById/walsByIdRecoveredQueues from ReplicationSourceManager Key: HBASE-24681 URL: https://issues.apache.org/jira/browse/HBASE-24681 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23634) Enable "Split WAL to HFile" by default
[
https://issues.apache.org/jira/browse/HBASE-23634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151688#comment-17151688
]
Guanghao Zhang commented on HBASE-23634:
{quote}Now to make these file are official HFiles (move to under the region/cf
directory) is the responsibility of primary regions. Only the primary region
should do this. This will happen when the primary region is opened.
{quote}
Right.
> Enable "Split WAL to HFile" by default
> --
>
> Key: HBASE-23634
> URL: https://issues.apache.org/jira/browse/HBASE-23634
> Project: HBase
> Issue Type: Task
>Affects Versions: 3.0.0-alpha-1, 2.3.0
>Reporter: Guanghao Zhang
>Priority: Blocker
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2018: HBASE-24659 Calculate FIXED_OVERHEAD automatically
Apache-HBase commented on pull request #2018: URL: https://github.com/apache/hbase/pull/2018#issuecomment-653948591 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 2s | master passed | | +1 :green_heart: | compile | 1m 51s | master passed | | +1 :green_heart: | shadedjars | 7m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 13s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 13s | the patch passed | | +1 :green_heart: | compile | 2m 2s | the patch passed | | +1 :green_heart: | javac | 2m 2s | the patch passed | | +1 :green_heart: | shadedjars | 7m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 49s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 223m 31s | hbase-server in the patch passed. | | | | 262m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2018 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3516ce982d3c 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fe2ae809d1 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/testReport/ | | Max. process+thread count | 3502 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2018: HBASE-24659 Calculate FIXED_OVERHEAD automatically
Apache-HBase commented on pull request #2018: URL: https://github.com/apache/hbase/pull/2018#issuecomment-653939159 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 11s | master passed | | +1 :green_heart: | compile | 1m 28s | master passed | | +1 :green_heart: | shadedjars | 5m 51s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 20s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 1s | the patch passed | | +1 :green_heart: | compile | 1m 29s | the patch passed | | +1 :green_heart: | javac | 1m 29s | the patch passed | | +1 :green_heart: | shadedjars | 5m 44s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 18s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 38s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 24s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 129m 46s | hbase-server in the patch passed. | | | | 159m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2018 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 449b58a7ac6e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fe2ae809d1 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/testReport/ | | Max. process+thread count | 4275 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2018: HBASE-24659 Calculate FIXED_OVERHEAD automatically
Apache-HBase commented on pull request #2018: URL: https://github.com/apache/hbase/pull/2018#issuecomment-653926487 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 46s | master passed | | +1 :green_heart: | checkstyle | 1m 42s | master passed | | +1 :green_heart: | spotbugs | 2m 44s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 46s | the patch passed | | +1 :green_heart: | checkstyle | 0m 23s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 1m 17s | hbase-server: The patch generated 0 new + 197 unchanged - 9 fixed = 197 total (was 206) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 18s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 38m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2018 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 025b92fe8378 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / fe2ae809d1 | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1991: HBASE-24650 Change the return types of the new checkAndMutate methods…
Apache-HBase commented on pull request #1991: URL: https://github.com/apache/hbase/pull/1991#issuecomment-653911439 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 56s | master passed | | +1 :green_heart: | compile | 2m 18s | master passed | | +1 :green_heart: | shadedjars | 6m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 48s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 47s | the patch passed | | +1 :green_heart: | compile | 2m 16s | the patch passed | | +1 :green_heart: | javac | 2m 16s | the patch passed | | +1 :green_heart: | shadedjars | 6m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 46s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 14s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 202m 13s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 5m 18s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 4m 2s | hbase-rest in the patch passed. | | | | 244m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1991 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 069beffa18fd 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a7a0e1a596 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/testReport/ | | Max. process+thread count | 2833 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1991: HBASE-24650 Change the return types of the new checkAndMutate methods…
Apache-HBase commented on pull request #1991: URL: https://github.com/apache/hbase/pull/1991#issuecomment-653910812 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 42s | master passed | | +1 :green_heart: | compile | 2m 45s | master passed | | +1 :green_heart: | shadedjars | 6m 29s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 27s | hbase-client in master failed. | | -0 :warning: | javadoc | 0m 19s | hbase-rest in master failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in master failed. | | -0 :warning: | javadoc | 1m 0s | hbase-thrift in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 32s | the patch passed | | +1 :green_heart: | compile | 2m 43s | the patch passed | | +1 :green_heart: | javac | 2m 43s | the patch passed | | +1 :green_heart: | shadedjars | 6m 22s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 26s | hbase-client in the patch failed. | | -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. | | -0 :warning: | javadoc | 0m 57s | hbase-thrift in the patch failed. | | -0 :warning: | javadoc | 0m 19s | hbase-rest in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 20s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 191m 51s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 5m 15s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 3m 26s | hbase-rest in the patch passed. | | | | 238m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1991 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4aef4cce48bd 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a7a0e1a596 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-rest.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-rest.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/testReport/ | | Max. process+thread count | 3458 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To
[jira] [Commented] (HBASE-24625) AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length.
[
https://issues.apache.org/jira/browse/HBASE-24625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151601#comment-17151601
]
Hudson commented on HBASE-24625:
Results for branch master
[build #1777 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1777/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1777/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1663//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1777/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1777/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced
> file length.
>
>
> Key: HBASE-24625
> URL: https://issues.apache.org/jira/browse/HBASE-24625
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 2.1.0, 2.0.0, 2.2.0, 2.3.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0, 2.2.6
>
>
> By HBASE-14004, we introduce {{WALFileLengthProvider}} interface to keep the
> current writing wal file length by ourselves, {{WALEntryStream}} used by
> {{ReplicationSourceWALReader}} could only read WAL file byte size <=
> {{WALFileLengthProvider.getLogFileSizeIfBeingWritten}} if the WAL file is
> current been writing on the same RegionServer .
> {{AsyncFSWAL}} implements {{WALFileLengthProvider}} by
> {{AbstractFSWAL.getLogFileSizeIfBeingWritten}}, just as folllows :
> {code:java}
>public OptionalLong getLogFileSizeIfBeingWritten(Path path) {
> rollWriterLock.lock();
> try {
> Path currentPath = getOldPath();
> if (path.equals(currentPath)) {
> W writer = this.writer;
> return writer != null ? OptionalLong.of(writer.getLength()) :
> OptionalLong.empty();
> } else {
> return OptionalLong.empty();
> }
> } finally {
> rollWriterLock.unlock();
> }
> }
> {code}
> For {{AsyncFSWAL}}, above {{AsyncFSWAL.writer}} is
> {{AsyncProtobufLogWriter}} ，and {{AsyncProtobufLogWriter.getLength}} is as
> follows:
> {code:java}
> public long getLength() {
> return length.get();
> }
> {code}
> But for {{AsyncProtobufLogWriter}}, any append method may increase the above
> {{AsyncProtobufLogWriter.length}}, especially for following
> {{AsyncFSWAL.append}}
> method just appending the {{WALEntry}} to
> {{FanOutOneBlockAsyncDFSOutput.buf}}:
> {code:java}
> public void append(Entry entry) {
> int buffered = output.buffered();
> try {
> entry.getKey().
>
> getBuilder(compressor).setFollowingKvCount(entry.getEdit().size()).build()
> .writeDelimitedTo(asyncOutputWrapper);
> } catch (IOException e) {
> throw new AssertionError("should not happen", e);
> }
>
> try {
>for (Cell cell : entry.getEdit().getCells()) {
> cellEncoder.write(cell);
>}
> } catch (IOException e) {
>throw new AssertionError("should not happen", e);
> }
> length.addAndGet(output.buffered() - buffered);
> }
> {code}
> That is to say, {{AsyncFSWAL.getLogFileSizeIfBeingWritten}} could not reflect
> the file length which successfully synced to underlying HDFS, which is not
> as expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] HorizonNet commented on a change in pull request #2016: HBASE-24653 Show snapshot owner on Master WebUI
HorizonNet commented on a change in pull request #2016: URL: https://github.com/apache/hbase/pull/2016#discussion_r449887464 ## File path: hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon ## @@ -673,6 +675,8 @@ AssignmentManager assignmentManager = master.getAssignmentManager(); <% snapshotTable.getNameAsString() %> <% new Date(snapshotDesc.getCreationTime()) %> +<% snapshotDesc.getOwner() %> +<% snapshotDesc.getTtl() %> Review comment: Does it make sense to also check if it is 0 and display `FOREVER` as done in `snapshotsStats.jsp`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24678) Add Bulk load param details into its responseTooSlow log
[
https://issues.apache.org/jira/browse/HBASE-24678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151584#comment-17151584
]
Viraj Jasani commented on HBASE-24678:
--
{quote}At least for bulk load kind of ops, it might need comparably more time?
{quote}
+1, we should have this, else bulk ops are always treated as time consuming and
warn log could be misleading.
Also, for BulkLoad param details, we might want to add new param to
OnlineLogRecord (client side) as well as SlowLogPayload (For Rpc call payload).
This will help for online ring buffer, but anyways, it can be sub-task also.
> Add Bulk load param details into its responseTooSlow log
>
>
> Key: HBASE-24678
> URL: https://issues.apache.org/jira/browse/HBASE-24678
> Project: HBase
> Issue Type: Improvement
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Major
>
> Right now the log will come like
> {code}
> (responseTooSlow):
> {"call":"BulkLoadHFile(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$BulkLoadHFileRequest)","starttimems":1593820455043,"responsesize":2,"method":"BulkLoadHFile","param":"TODO:
> class
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$BulkLoadHFileRequest",..}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24671) Add excludefile and designatedfile options to graceful_stop.sh
[ https://issues.apache.org/jira/browse/HBASE-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani resolved HBASE-24671. -- Fix Version/s: 2.4.0 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Merged changes to master and branch-2. Thanks for the patch [~DeanZ]. > Add excludefile and designatedfile options to graceful_stop.sh > -- > > Key: HBASE-24671 > URL: https://issues.apache.org/jira/browse/HBASE-24671 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Baiqiang Zhao >Assignee: Baiqiang Zhao >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > RegionMover is support excludefile and designatedfile options now. Integrate > these two options into graceful_stop.sh. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2018: HBASE-24659 Calculate FIXED_OVERHEAD automatically
Apache-HBase commented on pull request #2018: URL: https://github.com/apache/hbase/pull/2018#issuecomment-653893247 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | compile | 1m 28s | master passed | | +1 :green_heart: | shadedjars | 6m 42s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 22s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 49s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 8s | the patch passed | | +1 :green_heart: | compile | 1m 30s | the patch passed | | +1 :green_heart: | javac | 1m 30s | the patch passed | | +1 :green_heart: | shadedjars | 5m 53s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 43s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 26s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 129m 50s | hbase-server in the patch passed. | | | | 161m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2018 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4b16bbdbf512 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a7a0e1a596 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/1/testReport/ | | Max. process+thread count | 4519 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani closed pull request #2013: HBASE-24671 Add excludefile and designatedfile options to graceful_stop.sh
virajjasani closed pull request #2013: URL: https://github.com/apache/hbase/pull/2013 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1991: HBASE-24650 Change the return types of the new checkAndMutate methods…
Apache-HBase commented on pull request #1991: URL: https://github.com/apache/hbase/pull/1991#issuecomment-653889258 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 37s | master passed | | +1 :green_heart: | checkstyle | 2m 47s | master passed | | +1 :green_heart: | spotbugs | 4m 51s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 21s | the patch passed | | +1 :green_heart: | checkstyle | 2m 29s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 22s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 5m 27s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 49s | The patch does not generate ASF License warnings. | | | | 43m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1991 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux bd0126a7ef78 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a7a0e1a596 | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1991/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24526) Deadlock executing assign meta procedure
[
https://issues.apache.org/jira/browse/HBASE-24526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151574#comment-17151574
]
Duo Zhang commented on HBASE-24526:
---
In general there are two problems.
1. The updating meta is a blocking operation, which will cause the PEWorker to
wait. So if meta is temporarily unavailable, it is possible that all PEWokers
are stuck and we can not make any progress for a long time. This can not be
fully solved by checking meta availability before executing the procedure, as
we need sometime to find out the meta is unavailable, and the state of meta is
reset by TRSP, which is also executed by a PEWorker, which could make things
worse. You can see HBASE-19976 and its sub tasks on how we plan to partially
solve it by introducing priority for procedures and also add more workers if
all workers are stuck. There is a UT called TestProcedurePriority which is to
confirm that the mechanism can finally make progress.
IIRC there is still a problem that, we always poll SCP priority to TRSP, so if
a bunch of servers crash at the same time, we have to finish the all the wal
splitting before we actually assigning meta...
In general, on master branch, since we have builtin async connection support
for master service, we could introduce a sub procedure for updating meta in
TRSP and SCP, so the parent procedure will release the PEWorker, and in the
update meta procedure, we could make use of the async connection to release the
PEWorker when updating meta. On branch-2.x, this is not possible, but I think a
possible way, is to reduce the retry count and timeout to release the PEWorker
soon so we have a chance to schedule the procedure with highest priority.
2. All meta updating operations are under the region node lock, together with
modification to the in memory state, which implies that, the in memory state of
a region is always the same with the record in meta. This is a very strong
assumption which makes the logic much easier to implement, but also makes the
optimization in #1 impossible, as when meta updating fails, if we still want to
keep the state in sync, the only way is to abort... Maybe in some places we
have retry on meta updating failure, but I'm not sure whether the logic is
correct. We should revisit the related code to remove this assumption, so we
can apply the optimization in #1.
Thanks.
> Deadlock executing assign meta procedure
>
>
> Key: HBASE-24526
> URL: https://issues.apache.org/jira/browse/HBASE-24526
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, Region Assignment
>Affects Versions: 2.3.0
>Reporter: Nick Dimiduk
>Priority: Critical
>
> I have what appears to be a deadlock while assigning meta. During recovery,
> master creates the assign procedure for meta, and immediately marks meta as
> assigned in zookeeper. It then creates the subprocedure to open meta on the
> target region. However, the PEWorker pool is full of procedures that are
> stuck, I think because their calls to update meta are going nowhere. For what
> it's worth, the balancer is running concurrently, and has calculated a plan
> size of 41.
> From the master log,
> {noformat}
> 2020-06-06 00:34:07,314 INFO
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure:
> Starting pid=17802, ppid=17801,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true;
> TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN;
> state=OPEN, location=null; forceNewPlan=true, retain=false
> 2020-06-06 00:34:07,465 INFO
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator: Setting hbase:meta
> (replicaId=0) location in ZooKeeper as
> hbasedn139.example.com,16020,1591403576247
> 2020-06-06 00:34:07,466 INFO
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Initialized
> subprocedures=[{pid=17803, ppid=17802, state=RUNNABLE;
> org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure}]
> {noformat}
> {{pid=17803}} is not mentioned again. hbasedn139 never receives an
> {{openRegion}} RPC.
> Meanwhile, additional procedures are scheduled and picked up by workers, each
> getting "stuck". I see log lines for all 16 PEWorker threads, saying that
> they are stuck.
> {noformat}
> 2020-06-06 00:34:07,961 INFO
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler: Took xlock
> for pid=17804, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE;
> TransitRegionStateProcedure table=IntegrationTestBigLinkedList,
> region=54f4f6c0e921e6d25e6043cba79c09aa, REOPEN/MOVE
> 2020-06-06 00:34:07,961 INFO
> org.apache.hadoop.hbase.master.assignment.RegionStateStore: pid=17804
> updating hbase:meta row=54f4f6c0e921e6d25e6043cba79c09aa,
> regionState=CLOSING, regionLocation=hbasedn046.example.com,16020,1591402383956
>
[GitHub] [hbase] Apache-HBase commented on pull request #2018: HBASE-24659 Calculate FIXED_OVERHEAD automatically
Apache-HBase commented on pull request #2018: URL: https://github.com/apache/hbase/pull/2018#issuecomment-653880419 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 19s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 58s | master passed | | +1 :green_heart: | checkstyle | 1m 39s | master passed | | +1 :green_heart: | spotbugs | 2m 50s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | checkstyle | 0m 26s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 1m 16s | hbase-server: The patch generated 0 new + 197 unchanged - 9 fixed = 197 total (was 206) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 23s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. | | | | 39m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2018 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 60fd4891054b 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a7a0e1a596 | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2018/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24669) Logging of ppid should be consistent across all occurrences
[
https://issues.apache.org/jira/browse/HBASE-24669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17151548#comment-17151548
]
niuyulin commented on HBASE-24669:
--
hi, [~ndimiduk] , I submit a PR, could you help review it
> Logging of ppid should be consistent across all occurrences
> ---
>
> Key: HBASE-24669
> URL: https://issues.apache.org/jira/browse/HBASE-24669
> Project: HBase
> Issue Type: Improvement
> Components: Operability, proc-v2
>Reporter: Nick Dimiduk
>Assignee: niuyulin
>Priority: Minor
>
> Most places we log {{pid}} and {{ppid}} I noticed at least one occurrence of
> {{parent pid}} instead of {{ppid}}. All this logging should be uniform so
> that it's easy to grep the logs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] nyl3532016 opened a new pull request #2018: HBASE-24659 Calculate FIXED_OVERHEAD automatically
nyl3532016 opened a new pull request #2018: URL: https://github.com/apache/hbase/pull/2018 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1814: HBASE-20904 Prometheus /metrics http endpoint for monitoring
Apache-HBase commented on pull request #1814: URL: https://github.com/apache/hbase/pull/1814#issuecomment-653875553 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 37s | master passed | | +1 :green_heart: | compile | 2m 19s | master passed | | +1 :green_heart: | shadedjars | 5m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 52s | master passed | | -0 :warning: | patch | 9m 1s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 21s | the patch passed | | +1 :green_heart: | compile | 2m 17s | the patch passed | | +1 :green_heart: | javac | 2m 17s | the patch passed | | +1 :green_heart: | shadedjars | 5m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 49s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 298m 31s | root in the patch failed. | | | | 330m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1814 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 86bf582967f0 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a7a0e1a596 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/testReport/ | | Max. process+thread count | 4225 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-hadoop-compat hbase-http . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2009: HBASE-21596 Delete for a specific cell version can bring back version…
virajjasani commented on a change in pull request #2009:
URL: https://github.com/apache/hbase/pull/2009#discussion_r449863611
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -3170,37 +3171,87 @@ public void prepareDeleteTimestamps(Mutation mutation,
Map> f
count = kvCount.get(qual);
Get get = new Get(CellUtil.cloneRow(cell));
- get.readVersions(count);
- get.addColumn(family, qual);
+ get.readVersions(Integer.MAX_VALUE);
if (coprocessorHost != null) {
if (!coprocessorHost.prePrepareTimeStampForDeleteVersion(mutation,
cell,
byteNow, get)) {
- updateDeleteLatestVersionTimestamp(cell, get, count, byteNow);
+ updateDeleteLatestVersionTimestamp(cell, get, count,
+
this.htableDescriptor.getColumnFamily(family).getMaxVersions(),
+byteNow, deleteCells);
+
}
} else {
-updateDeleteLatestVersionTimestamp(cell, get, count, byteNow);
+updateDeleteLatestVersionTimestamp(cell, get, count,
+this.htableDescriptor.getColumnFamily(family).getMaxVersions(),
+ byteNow, deleteCells);
}
} else {
PrivateCellUtil.updateLatestStamp(cell, byteNow);
+ deleteCells.add(cell);
}
}
+ e.setValue(deleteCells);
}
}
- void updateDeleteLatestVersionTimestamp(Cell cell, Get get, int count,
byte[] byteNow)
- throws IOException {
-List result = get(get, false);
-
+ private void updateDeleteLatestVersionTimestamp(Cell cell, Get get, int
count, int maxVersions,
+ byte[] byteNow, List deleteCells) throws IOException {
+List result = new ArrayList<>(deleteCells);
+Scan scan = new Scan(get);
+scan.setRaw(true);
+this.getScanner(scan).next(result);
+List cells = new ArrayList<>();
if (result.size() < count) {
// Nothing to delete
PrivateCellUtil.updateLatestStamp(cell, byteNow);
- return;
-}
-if (result.size() > count) {
- throw new RuntimeException("Unexpected size: " + result.size());
+ cells.add(cell);
+ deleteCells.addAll(cells);
+} else if (result.size() > count) {
+ int currentVersion = 0;
+ long latestCellTS = Long.MAX_VALUE;
+ result.sort((cell1, cell2) -> {
+if(cell1.getTimestamp()>cell2.getTimestamp()){
+ return -1;
+} else if(cell1.getTimestamp()= maxVersions) {
+Cell tempCell = null;
+try {
+ tempCell = PrivateCellUtil.deepClone(cell);
+} catch (CloneNotSupportedException e) {
+ throw new IOException(e);
+}
+PrivateCellUtil.setTimestamp(tempCell, getCell.getTimestamp());
+cells.add(tempCell);
+ } else if (currentVersion == 0) {
+PrivateCellUtil.setTimestamp(cell, getCell.getTimestamp());
+cells.add(cell);
+ }
+ currentVersion++;
+}
+latestCellTS = getCell.getTimestamp();
+ }
+
+} else {
+ Cell getCell = result.get(0);
Review comment:
I am not sure if sorting of result is required here. If not required,
rest looks good.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1814: HBASE-20904 Prometheus /metrics http endpoint for monitoring
Apache-HBase commented on pull request #1814: URL: https://github.com/apache/hbase/pull/1814#issuecomment-653869842 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 40s | master passed | | +1 :green_heart: | compile | 3m 3s | master passed | | +1 :green_heart: | shadedjars | 6m 26s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 15s | root in master failed. | | -0 :warning: | javadoc | 0m 15s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 17s | hbase-hadoop-compat in master failed. | | -0 :warning: | javadoc | 0m 17s | hbase-http in master failed. | | -0 :warning: | patch | 8m 2s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 28s | the patch passed | | +1 :green_heart: | compile | 3m 3s | the patch passed | | +1 :green_heart: | javac | 3m 3s | the patch passed | | +1 :green_heart: | shadedjars | 6m 21s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 15s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 18s | hbase-hadoop-compat in the patch failed. | | -0 :warning: | javadoc | 0m 16s | hbase-http in the patch failed. | | -0 :warning: | javadoc | 0m 13s | root in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 240m 18s | root in the patch passed. | | | | 274m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1814 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b180c78c89f4 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a7a0e1a596 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-http.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-http.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/testReport/ | | Max. process+thread count | 4471 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-hadoop-compat hbase-http . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL
[GitHub] [hbase] brfrn169 commented on a change in pull request #1991: HBASE-24650 Change the return types of the new checkAndMutate methods…
brfrn169 commented on a change in pull request #1991:
URL: https://github.com/apache/hbase/pull/1991#discussion_r449854459
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncTableImpl.java
##
@@ -497,7 +497,7 @@ public void run(MultiResponse resp) {
"Failed to mutate row: " +
Bytes.toStringBinary(mutation.getRow()), ex));
} else {
future.complete(respConverter
- .apply((Result)
multiResp.getResults().get(regionName).result.get(0)));
+ .apply((RES)
multiResp.getResults().get(regionName).result.get(0)));
Review comment:
Yes, correct. Previously, we returned only Result type but after this
fix, we will return Result (for normal mutateRow) and CheckAndMutateResult (for
checkAndMutate). So I used a generic type here.
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableOverAsyncTable.java
##
@@ -300,13 +300,16 @@ public boolean thenMutate(RowMutations mutation) throws
IOException {
}
@Override
- public boolean checkAndMutate(CheckAndMutate checkAndMutate) throws
IOException {
+ public CheckAndMutateResult checkAndMutate(CheckAndMutate checkAndMutate)
throws IOException {
return FutureUtils.get(table.checkAndMutate(checkAndMutate));
}
@Override
- public boolean[] checkAndMutate(List checkAndMutates) throws
IOException {
-return
Booleans.toArray(FutureUtils.get(table.checkAndMutateAll(checkAndMutates)));
+ public CheckAndMutateResult[] checkAndMutate(List
checkAndMutates)
Review comment:
Yeah that's better. I will change it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1814: HBASE-20904 Prometheus /metrics http endpoint for monitoring
Apache-HBase commented on pull request #1814: URL: https://github.com/apache/hbase/pull/1814#issuecomment-653849751 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 25s | master passed | | +1 :green_heart: | checkstyle | 1m 57s | master passed | | +0 :ok: | refguide | 4m 58s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 10m 9s | master passed | | -0 :warning: | patch | 9m 1s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 23s | the patch passed | | -0 :warning: | checkstyle | 1m 55s | root: The patch generated 29 new + 7 unchanged - 0 fixed = 36 total (was 7) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 3s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 5m 2s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 10m 59s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 10m 53s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 52s | The patch does not generate ASF License warnings. | | | | 62m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1814 | | Optional Tests | dupname asflicense refguide xml spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 563604ed26d0 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a7a0e1a596 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-general-check/output/branch-site/book.html | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-general-check/output/diff-checkstyle-root.txt | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/artifact/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 137 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-hadoop-compat hbase-http . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1814/11/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
