[jira] [Commented] (HBASE-23702) `hbase.hstore.flusher.count` setting to 0 breaks HMaster
[
https://issues.apache.org/jira/browse/HBASE-23702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018528#comment-17018528
]
HBase QA commented on HBASE-23702:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
58s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
12s{color} | {color:red} hbase-server: The patch generated 3 new + 21 unchanged
- 0 fixed = 24 total (was 21) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 4 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
3s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 17s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}164m 51s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}226m 27s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.TestMasterShutdown |
| | hadoop.hbase.master.assignment.TestRegionMoveAndAbandon |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/1091/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-23702 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12991282/HBASE-23702-000.patch
|
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d2f807e70001 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[GitHub] [hbase] saintstack commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters…
saintstack commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters… URL: https://github.com/apache/hbase/pull/1012#issuecomment-575870648 Fix dumb compile issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #1062: HBASE-23705 Add CellComparator to HFileContext
saintstack commented on issue #1062: HBASE-23705 Add CellComparator to HFileContext URL: https://github.com/apache/hbase/pull/1062#issuecomment-575869743 Fix complaints and UTs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23702) `hbase.hstore.flusher.count` setting to 0 breaks HMaster
[ https://issues.apache.org/jira/browse/HBASE-23702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ctest updated HBASE-23702: -- Attachment: HBASE-23702-000.patch > `hbase.hstore.flusher.count` setting to 0 breaks HMaster > > > Key: HBASE-23702 > URL: https://issues.apache.org/jira/browse/HBASE-23702 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.2.3 > Environment: HBase-2.2.3 > Java openjdk 1.8.0_222 > Ubuntu 18.04 >Reporter: Ctest >Priority: Major > Attachments: HBASE-23702-000.patch, HBASE-ISSUEID-000.patch.txt, > hbase-src-patch.txt > > > Summary > The configuration parameter `hbase.hstore.flusher.count` allows `0` (which > turns out to be a bad value after I understand the source code). > When we set `hbase.hstore.flusher.count` to 0, HMaster will fail in 200,000 > ms without any error message, but > ``` > java.lang.RuntimeException: Master not initialized after 20ms > ``` > > There’s no messages pinpointing `hbase.hstore.flusher.count` being the root > cause. > > 0 means `MemStoreFlusher` is not alive. During the initialization time of > HMaster, `isHealthy()` will be called and it will try to check if > `MemStoreFlusher` is alive or not. However, since we set > `hbase.hstore.flusher.count` to 0, there will be no flusher created at the > first place, which cause`isHealthy()` to return false. As a consequence, > HRegionServer does not enter its main loop. > > *How To Reproduce* > > The version of HBase: 2.2.3 > > # Set the value of configuration parameter `hbase.hstore.flusher.count` to > `0` in “conf/hbase-site.xml” > # Start hbase by running “bin/start-hbase.sh” > # Wait for 3-4 minutes > # Open the log file “logs/hbase-- master -.log” -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23702) `hbase.hstore.flusher.count` setting to 0 breaks HMaster
[ https://issues.apache.org/jira/browse/HBASE-23702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ctest updated HBASE-23702: -- Attachment: HBASE-ISSUEID-000.patch.txt > `hbase.hstore.flusher.count` setting to 0 breaks HMaster > > > Key: HBASE-23702 > URL: https://issues.apache.org/jira/browse/HBASE-23702 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.2.3 > Environment: HBase-2.2.3 > Java openjdk 1.8.0_222 > Ubuntu 18.04 >Reporter: Ctest >Priority: Major > Attachments: HBASE-ISSUEID-000.patch.txt, hbase-src-patch.txt > > > Summary > The configuration parameter `hbase.hstore.flusher.count` allows `0` (which > turns out to be a bad value after I understand the source code). > When we set `hbase.hstore.flusher.count` to 0, HMaster will fail in 200,000 > ms without any error message, but > ``` > java.lang.RuntimeException: Master not initialized after 20ms > ``` > > There’s no messages pinpointing `hbase.hstore.flusher.count` being the root > cause. > > 0 means `MemStoreFlusher` is not alive. During the initialization time of > HMaster, `isHealthy()` will be called and it will try to check if > `MemStoreFlusher` is alive or not. However, since we set > `hbase.hstore.flusher.count` to 0, there will be no flusher created at the > first place, which cause`isHealthy()` to return false. As a consequence, > HRegionServer does not enter its main loop. > > *How To Reproduce* > > The version of HBase: 2.2.3 > > # Set the value of configuration parameter `hbase.hstore.flusher.count` to > `0` in “conf/hbase-site.xml” > # Start hbase by running “bin/start-hbase.sh” > # Wait for 3-4 minutes > # Open the log file “logs/hbase-- master -.log” -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23701) Make sure HBaseClassTestRule doesn't suffer same issue as HBaseClassTestRuleChecker
[
https://issues.apache.org/jira/browse/HBASE-23701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018485#comment-17018485
]
Hudson commented on HBASE-23701:
Results for branch branch-2.1
[build #1776 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1776/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1776//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1776//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1776//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Make sure HBaseClassTestRule doesn't suffer same issue as
> HBaseClassTestRuleChecker
> ---
>
> Key: HBASE-23701
> URL: https://issues.apache.org/jira/browse/HBASE-23701
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> [~bharathv] pointed out on HBASE-23695
> ([https://github.com/apache/hbase/pull/1052]) that HBaseClassTestRule suffers
> the same potential bug that I fixed in HBASE-23695 for
> HBaseClassTestRuleChecker. Make sure the fix is in both places.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1043: HBASE-23055 Alter hbase:meta
Apache-HBase commented on issue #1043: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/1043#issuecomment-575856451 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 6 new or modified test files. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 19s | branch-2 passed | | +1 :green_heart: | compile | 2m 31s | branch-2 passed | | +1 :green_heart: | checkstyle | 3m 3s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 18s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 55s | branch-2 passed | | +0 :ok: | spotbugs | 0m 36s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 6m 5s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 1s | the patch passed | | +1 :green_heart: | compile | 2m 32s | the patch passed | | +1 :green_heart: | javac | 2m 32s | the patch passed | | +1 :green_heart: | checkstyle | 0m 26s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 38s | hbase-client: The patch generated 0 new + 83 unchanged - 4 fixed = 83 total (was 87) | | +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-zookeeper | | +1 :green_heart: | checkstyle | 1m 29s | hbase-server: The patch generated 0 new + 404 unchanged - 17 fixed = 404 total (was 421) | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-shell | | +1 :green_heart: | rubocop | 0m 3s | There were no new rubocop issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 12s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 22s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 55s | the patch passed | | +1 :green_heart: | findbugs | 6m 30s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 10s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 3m 42s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 53s | hbase-zookeeper in the patch passed. | | +1 :green_heart: | unit | 150m 45s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 9m 0s | hbase-shell in the patch passed. | | +1 :green_heart: | asflicense | 2m 45s | The patch does not generate ASF License warnings. | | | | 240m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1043/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1043 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile rubocop | | uname | Linux 9ff8e0a84da1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1043/out/precommit/personality/provided.sh | | git revision | branch-2 / 1ea389a834 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1043/4/testReport/ | | Max. process+thread count | 5024 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-zookeeper hbase-server hbase-shell U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1043/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 rubocop=0.79.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on
[GitHub] [hbase] bharathv commented on a change in pull request #1064: HBASE-23707 Add IntelliJ check style plugin configuration
bharathv commented on a change in pull request #1064: HBASE-23707 Add IntelliJ check style plugin configuration URL: https://github.com/apache/hbase/pull/1064#discussion_r368198213 ## File path: .idea/checkstyle-idea.xml ## @@ -0,0 +1,16 @@ + + + + + + Review comment: Shouldn't we include the path for active configuration? Something like below, otherwise how does it hook up with our check style configuration? Something like below.. ``` ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1064: HBASE-23707 Add IntelliJ check style plugin configuration
bharathv commented on a change in pull request #1064: HBASE-23707 Add IntelliJ check style plugin configuration URL: https://github.com/apache/hbase/pull/1064#discussion_r368198213 ## File path: .idea/checkstyle-idea.xml ## @@ -0,0 +1,16 @@ + + + + + + Review comment: Shouldn't we include the path for active configuration? Something like below, otherwise how does it hook up with our check style plugin? Something like below.. ``` ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1064: HBASE-23707 Add IntelliJ check style plugin configuration
bharathv commented on a change in pull request #1064: HBASE-23707 Add IntelliJ check style plugin configuration URL: https://github.com/apache/hbase/pull/1064#discussion_r368198141 ## File path: .idea/checkstyle-idea.xml ## @@ -0,0 +1,16 @@ + + + + + + + + + + + Review comment: I think this should be "JavaOnlyWithTests".. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1064: HBASE-23707 Add IntelliJ check style plugin configuration
Apache-HBase commented on issue #1064: HBASE-23707 Add IntelliJ check style plugin configuration URL: https://github.com/apache/hbase/pull/1064#issuecomment-575855194 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 28s | master passed | | +1 :green_heart: | checkstyle | 2m 13s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 54s | the patch passed | | +1 :green_heart: | checkstyle | 2m 10s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. | | | | 16m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1064/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1064 | | Optional Tests | dupname asflicense checkstyle xml | | uname | Linux 49ff86982143 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1064/out/precommit/personality/provided.sh | | git revision | master / 70c8a5d939 | | Max. process+thread count | 96 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1064/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23707) Add IntelliJ check style plugin configuration
[ https://issues.apache.org/jira/browse/HBASE-23707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018463#comment-17018463 ] Nick Dimiduk commented on HBASE-23707: -- FYI [~stack] [~bharathv] > Add IntelliJ check style plugin configuration > - > > Key: HBASE-23707 > URL: https://issues.apache.org/jira/browse/HBASE-23707 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > IntelliJ defines project configuration across a number of files, specifically > so that some configurations can be committed with the source repository. The > checkstyle plugin configuration is one such config file; add it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-23707) Add IntelliJ check style plugin configuration
[ https://issues.apache.org/jira/browse/HBASE-23707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-23707 started by Nick Dimiduk. > Add IntelliJ check style plugin configuration > - > > Key: HBASE-23707 > URL: https://issues.apache.org/jira/browse/HBASE-23707 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > IntelliJ defines project configuration across a number of files, specifically > so that some configurations can be committed with the source repository. The > checkstyle plugin configuration is one such config file; add it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23707) Add IntelliJ check style plugin configuration
[ https://issues.apache.org/jira/browse/HBASE-23707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reassigned HBASE-23707: Assignee: Nick Dimiduk > Add IntelliJ check style plugin configuration > - > > Key: HBASE-23707 > URL: https://issues.apache.org/jira/browse/HBASE-23707 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > IntelliJ defines project configuration across a number of files, specifically > so that some configurations can be committed with the source repository. The > checkstyle plugin configuration is one such config file; add it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on issue #1064: HBASE-23707 Add IntelliJ check style plugin configuration
ndimiduk commented on issue #1064: HBASE-23707 Add IntelliJ check style plugin configuration URL: https://github.com/apache/hbase/pull/1064#issuecomment-575853591 There's no way that I know of for keeping the checkstyle version specified here in sync with the version defined in our pom. Any thoughts here @jshiell ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk opened a new pull request #1064: HBASE-23707 Add IntelliJ check style plugin configuration
ndimiduk opened a new pull request #1064: HBASE-23707 Add IntelliJ check style plugin configuration URL: https://github.com/apache/hbase/pull/1064 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23707) Add IntelliJ check style plugin configuration
Nick Dimiduk created HBASE-23707: Summary: Add IntelliJ check style plugin configuration Key: HBASE-23707 URL: https://issues.apache.org/jira/browse/HBASE-23707 Project: HBase Issue Type: Sub-task Components: build Reporter: Nick Dimiduk IntelliJ defines project configuration across a number of files, specifically so that some configurations can be committed with the source repository. The checkstyle plugin configuration is one such config file; add it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on a change in pull request #1057: HBASE-23701 Try to converge automated checks around Category
ndimiduk commented on a change in pull request #1057: HBASE-23701 Try to
converge automated checks around Category
URL: https://github.com/apache/hbase/pull/1057#discussion_r368194720
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseClassTestRule.java
##
@@ -59,13 +65,17 @@ private HBaseClassTestRule(Class clazz, Timeout
timeout) {
private static long getTimeoutInSeconds(Class clazz) {
Category[] categories = clazz.getAnnotationsByType(Category.class);
-for (Class c : categories[0].value()) {
- if (c == SmallTests.class || c == MediumTests.class || c ==
LargeTests.class) {
-// All tests have a 13 minutes timeout.
-return TimeUnit.MINUTES.toSeconds(13);
- }
- if (c == IntegrationTests.class) {
-return TimeUnit.MINUTES.toSeconds(Long.MAX_VALUE);
+
+// @Category is not repeatable -- it is only possible to get an array of
length zero or one.
+if (categories.length == 1) {
Review comment:
nit: using `ArrayUtils.notEmpty` protects against an NPE.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (HBASE-23700) Upgrade checkstyle and plugin versions
[
https://issues.apache.org/jira/browse/HBASE-23700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-23700.
--
Resolution: Fixed
Applied to master and branch-2. Tried branch-1, but the new version of the
plugin doesn't work on java7.
{{noformat}}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-checkstyle-plugin:3.1.0:check (default-cli) on
project hbase: Execution default-cli of goal
org.apache.maven.plugins:maven-checkstyle-plugin:3
.1.0:check failed: Unable to load the mojo 'check' in the plugin
'org.apache.maven.plugins:maven-checkstyle-plugin:3.1.0' due to an API
incompatibility: org.codehaus.plexus.component.repository.exception.
ComponentLookupException:
org/apache/maven/plugins/checkstyle/CheckstyleViolationCheckMojo : Unsupported
major.minor version 52.0
{{noformat}}
> Upgrade checkstyle and plugin versions
> --
>
> Key: HBASE-23700
> URL: https://issues.apache.org/jira/browse/HBASE-23700
> Project: HBase
> Issue Type: Task
> Components: build
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
>
> Our checkstyle version is pretty antiquated. Update it, and pull in the
> latest version of the maven plugin as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk merged pull request #1063: HBASE-23700 Upgrade checkstyle and plugin versions (#1056)
ndimiduk merged pull request #1063: HBASE-23700 Upgrade checkstyle and plugin versions (#1056) URL: https://github.com/apache/hbase/pull/1063 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23700) Upgrade checkstyle and plugin versions
[ https://issues.apache.org/jira/browse/HBASE-23700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23700: - Fix Version/s: 2.3.0 3.0.0 > Upgrade checkstyle and plugin versions > -- > > Key: HBASE-23700 > URL: https://issues.apache.org/jira/browse/HBASE-23700 > Project: HBase > Issue Type: Task > Components: build >Reporter: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0, 2.3.0 > > > Our checkstyle version is pretty antiquated. Update it, and pull in the > latest version of the maven plugin as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23700) Upgrade checkstyle and plugin versions
[ https://issues.apache.org/jira/browse/HBASE-23700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reassigned HBASE-23700: Assignee: Nick Dimiduk > Upgrade checkstyle and plugin versions > -- > > Key: HBASE-23700 > URL: https://issues.apache.org/jira/browse/HBASE-23700 > Project: HBase > Issue Type: Task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0, 2.3.0 > > > Our checkstyle version is pretty antiquated. Update it, and pull in the > latest version of the maven plugin as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #1063: HBASE-23700 Upgrade checkstyle and plugin versions (#1056)
ndimiduk opened a new pull request #1063: HBASE-23700 Upgrade checkstyle and plugin versions (#1056) URL: https://github.com/apache/hbase/pull/1063 Bump checkstyle version to 8.28, maven-checkstyle-plugin to 3.1.0. As per HBASE-23242 and the updated checkstyle docs[1], the LineLength check should be placed under an instance of Checker. [1] https://checkstyle.sourceforge.io/config_sizes.html#LineLength Co-authored-by: Bharath Vissapragada Signed-off-by: Jan Hentschel Signed-off-by: Viraj Jasani This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk merged pull request #1056: HBASE-23700 Upgrade checkstyle and plugin versions
ndimiduk merged pull request #1056: HBASE-23700 Upgrade checkstyle and plugin versions URL: https://github.com/apache/hbase/pull/1056 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23701) Make sure HBaseClassTestRule doesn't suffer same issue as HBaseClassTestRuleChecker
[
https://issues.apache.org/jira/browse/HBASE-23701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018435#comment-17018435
]
Hudson commented on HBASE-23701:
Results for branch branch-2.2
[build #757 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/757/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/757//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/757//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/757//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Make sure HBaseClassTestRule doesn't suffer same issue as
> HBaseClassTestRuleChecker
> ---
>
> Key: HBASE-23701
> URL: https://issues.apache.org/jira/browse/HBASE-23701
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> [~bharathv] pointed out on HBASE-23695
> ([https://github.com/apache/hbase/pull/1052]) that HBaseClassTestRule suffers
> the same potential bug that I fixed in HBASE-23695 for
> HBaseClassTestRuleChecker. Make sure the fix is in both places.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23569) Validate that the log cleaner actually cleans oldWALs as expected
[
https://issues.apache.org/jira/browse/HBASE-23569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018434#comment-17018434
]
Hudson commented on HBASE-23569:
Results for branch branch-2.2
[build #757 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/757/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/757//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/757//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/757//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Validate that the log cleaner actually cleans oldWALs as expected
> -
>
> Key: HBASE-23569
> URL: https://issues.apache.org/jira/browse/HBASE-23569
> Project: HBase
> Issue Type: Test
> Components: integration tests, master, test
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0, 2.2.3
>
> Attachments: HBASE-23569.master.000.patch,
> HBASE-23569.master.002.patch, HBASE-23569.master.002.patch
>
>
> The fix for HBASE-23287 (LogCleaner is not added to choreService) is in but
> we are lacking test coverage that validates that the log cleaner actually
> cleans oldWALs as expected. Add the test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-21969) Improve the update of destination rsgroup of RSGroupInfoManagerImpl#moveTables()
[
https://issues.apache.org/jira/browse/HBASE-21969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-21969:
-
Fix Version/s: (was: 2.3.1)
2.3.0
> Improve the update of destination rsgroup of
> RSGroupInfoManagerImpl#moveTables()
> -
>
> Key: HBASE-21969
> URL: https://issues.apache.org/jira/browse/HBASE-21969
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.3.0
>
> Attachments: HBASE-21969.branch-1.000.patch,
> HBASE-21969.branch-2.000.patch, HBASE-21969.master.000.patch,
> HBASE-21969.master.001.patch
>
>
> RSGroupInfoManagerImpl#moveTables() has the following code:
> {code}
> for (TableName tableName : tableNames) {
>
> if (groupName != null) {
> RSGroupInfo dst = new RSGroupInfo(newGroupMap.get(groupName));
> dst.addTable(tableName);
> newGroupMap.put(dst.getName(), dst);
> }
> }
> {code}
> It is to add the tables to "dst" (the destination rsgroup) and put it back to
> newGroupMap to replace the old RSGroupInfo instance in the map.
> The current logic allocates a new RSGroupInfo instance and calls put
> operation of map for each table to move, which is not needed and causes a lot
> of objects to be allocated (the constructor of RSGroupInfo allocates 2
> TreeSets).
> The logic could be improved as: allocate a new RSGroupInfo only once, add all
> tables, and put it into the map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-22804) Provide an API to get list of successful regions and total expected regions in Canary
[ https://issues.apache.org/jira/browse/HBASE-22804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-22804: - Fix Version/s: (was: 2.3.1) 2.3.0 > Provide an API to get list of successful regions and total expected regions > in Canary > - > > Key: HBASE-22804 > URL: https://issues.apache.org/jira/browse/HBASE-22804 > Project: HBase > Issue Type: Improvement > Components: canary >Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0, 2.1.5, 2.2.1 >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Labels: Canary > Fix For: 3.0.0, 1.5.0, 2.3.0, 1.3.6, 1.4.11, 2.1.7, 2.2.2 > > Attachments: HBASE-22804.branch-1.001.patch, > HBASE-22804.branch-1.002.patch, HBASE-22804.branch-1.003.patch, > HBASE-22804.branch-1.004.patch, HBASE-22804.branch-1.005.patch, > HBASE-22804.branch-1.006.patch, HBASE-22804.branch-1.007.patch, > HBASE-22804.branch-1.008.patch, HBASE-22804.branch-1.009.patch, > HBASE-22804.branch-1.009.patch, HBASE-22804.branch-1.010.patch, > HBASE-22804.branch-2.001.patch, HBASE-22804.branch-2.002.patch, > HBASE-22804.branch-2.003.patch, HBASE-22804.branch-2.004.patch, > HBASE-22804.branch-2.005.patch, HBASE-22804.branch-2.006.patch, > HBASE-22804.master.001.patch, HBASE-22804.master.002.patch, > HBASE-22804.master.003.patch, HBASE-22804.master.004.patch, > HBASE-22804.master.005.patch, HBASE-22804.master.006.patch > > > At present HBase Canary tool only prints the successes as part of logs. > Providing an API to get the list of successes, as well as total number of > expected regions, will make it easier to get a more accurate availability > estimate. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23706) Update checkstyle plugin (and update checkstyle.xml to match)
[ https://issues.apache.org/jira/browse/HBASE-23706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-23706. --- Resolution: Duplicate Resolving as duplicate of HBASE-23700 (Thanks [~bharathv]) > Update checkstyle plugin (and update checkstyle.xml to match) > - > > Key: HBASE-23706 > URL: https://issues.apache.org/jira/browse/HBASE-23706 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Priority: Major > > In parent issue, its suggested we update our checkstyle plugin to match of > intellij plugin default at least. Will need checkstyle.xml changes else it > fails parse (See notes in parent by [~bharathv] on what needs doing). > For extra points, do the [~ndimiduk] suggestion: "...It would be nice also if > we could commit the .idea/checkstyle-idea.xml file with the checkstyle > version used buy the plugin pinned to the same version as we're using in > maven." -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1062: HBASE-23705 Add CellComparator to HFileContext
Apache-HBase commented on issue #1062: HBASE-23705 Add CellComparator to HFileContext URL: https://github.com/apache/hbase/pull/1062#issuecomment-575841816 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 18 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 46s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 7m 51s | master passed | | +1 :green_heart: | compile | 1m 56s | master passed | | +1 :green_heart: | checkstyle | 2m 28s | master passed | | +1 :green_heart: | shadedjars | 5m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 15s | master passed | | +0 :ok: | spotbugs | 4m 57s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 6m 35s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 33s | the patch passed | | +1 :green_heart: | compile | 1m 48s | the patch passed | | +1 :green_heart: | javac | 1m 48s | the patch passed | | -1 :x: | checkstyle | 0m 26s | hbase-common: The patch generated 2 new + 14 unchanged - 18 fixed = 16 total (was 32) | | -1 :x: | checkstyle | 1m 38s | hbase-server: The patch generated 3 new + 443 unchanged - 5 fixed = 446 total (was 448) | | +1 :green_heart: | checkstyle | 0m 20s | hbase-mapreduce: The patch generated 0 new + 17 unchanged - 1 fixed = 17 total (was 18) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 22s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 16s | the patch passed | | -1 :x: | findbugs | 0m 58s | hbase-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 9s | hbase-common in the patch passed. | | -1 :x: | unit | 32m 36s | hbase-server in the patch failed. | | -1 :x: | unit | 16m 4s | hbase-mapreduce in the patch failed. | | +1 :green_heart: | asflicense | 0m 58s | The patch does not generate ASF License warnings. | | | | 129m 51s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-common | | | Using .equals to compare two byte[]'s, (equivalent to ==) in org.apache.hadoop.hbase.CellComparatorImpl.getCellComparator(byte[]) At CellComparatorImpl.java:byte[]'s, (equivalent to ==) in org.apache.hadoop.hbase.CellComparatorImpl.getCellComparator(byte[]) At CellComparatorImpl.java:[line 428] | | Failed junit tests | hadoop.hbase.regionserver.TestBulkLoad | | | hadoop.hbase.io.hfile.TestReseekTo | | | hadoop.hbase.io.hfile.TestFixedFileTrailer | | | hadoop.hbase.regionserver.TestHStoreFile | | | hadoop.hbase.mapreduce.TestHFileOutputFormat2 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1062/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1062 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux ec3cc2104878 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1062/out/precommit/personality/provided.sh | | git revision | master / df8f80a819 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1062/1/artifact/out/diff-checkstyle-hbase-common.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1062/1/artifact/out/diff-checkstyle-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1062/1/artifact/out/new-findbugs-hbase-common.html | | unit |
[jira] [Resolved] (HBASE-15648) Reduce number of concurrent region location lookups when MetaCache entry is cleared
[ https://issues.apache.org/jira/browse/HBASE-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-15648. --- Resolution: Later No progress. Old. Resolving as 'Later'. > Reduce number of concurrent region location lookups when MetaCache entry is > cleared > --- > > Key: HBASE-15648 > URL: https://issues.apache.org/jira/browse/HBASE-15648 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 1.2.0, 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.7.0 > > Attachments: HBASE-15648-branch-1.3.v1.patch > > > It seems in HConnectionImplementation#locateRegionInMeta if region location > is removed from the cache, with large number of client threads we could have > many of them getting cache miss and doing meta scan, which looks unnecessary > - we could empty mechanism similar to what we have in IdLock in HFileReader > to fetch the block to cache, do ensure that if one thread is already looking > up location for region R1, other threads who need it's location wait until > first thread finishes his work. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-15539) HBase Client region location is expensive
[ https://issues.apache.org/jira/browse/HBASE-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-15539. --- Resolution: Later No progress. Resolving as 'Later'. > HBase Client region location is expensive > -- > > Key: HBASE-15539 > URL: https://issues.apache.org/jira/browse/HBASE-15539 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Vladimir Rodionov >Assignee: Mikhail Antonov >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > ConnectionImplementation.locateRegion and MetaCache.getTableLocations are hot > spots in a client. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-19452) Turn ON off heap Bucket Cache by default
[
https://issues.apache.org/jira/browse/HBASE-19452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018420#comment-17018420
]
HBase QA commented on HBASE-19452:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} HBASE-19452 does not apply to master. Rebase required? Wrong
Branch? See
https://yetus.apache.org/documentation/in-progress/precommit-patchnames for
help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19452 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12901671/HBASE-19452_V3.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1090/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
> Turn ON off heap Bucket Cache by default
>
>
> Key: HBASE-19452
> URL: https://issues.apache.org/jira/browse/HBASE-19452
> Project: HBase
> Issue Type: Sub-task
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Major
> Attachments: HBASE-19452.patch, HBASE-19452_V2.patch,
> HBASE-19452_V3.patch
>
>
> BC's hbase.bucketcache.ioengine by default is empty now. Means now BC.
> Make this default to be 'offheap'. And the default off heap size for the BC
> also to be provided. This can be 8 GB? Better to make it also a % of the
> Xmx. Lets continue to be 40% of Xmx as LRU cache default size.
> When user has to disable BC, configure size as 0. An empty value of this
> config will be treated as default size.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-23676) Address feedback on HBASE-23055 Alter hbase:meta.
[ https://issues.apache.org/jira/browse/HBASE-23676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-23676. --- Resolution: Won't Fix The parent HBASE-23055 was recast. This issue no longer relevant (The feedback was addressed up in new patch attached on HBASE-23055) > Address feedback on HBASE-23055 Alter hbase:meta. > - > > Key: HBASE-23676 > URL: https://issues.apache.org/jira/browse/HBASE-23676 > Project: HBase > Issue Type: Bug > Components: meta >Affects Versions: 2.3.0 >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Good feedback on HBASE-23055 came in after merge from [~zhangduo]. Opening > this issue to address it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-20618) Skip large rows instead of throwing an exception to client
[
https://issues.apache.org/jira/browse/HBASE-20618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-20618:
--
Fix Version/s: (was: 1.7.0)
(was: 2.3.0)
(was: 3.0.0)
> Skip large rows instead of throwing an exception to client
> --
>
> Key: HBASE-20618
> URL: https://issues.apache.org/jira/browse/HBASE-20618
> Project: HBase
> Issue Type: New Feature
>Reporter: Swapna
>Priority: Minor
> Attachments: HBASE-20618.hbasemaster.v01.patch,
> HBASE-20618.hbasemaster.v02.patch, HBASE-20618.v1.branch-1.patch,
> HBASE-20618.v1.branch-1.patch
>
>
> Currently HBase supports throwing RowTooBigException incase there is a row
> with one of the column family data exceeds the configured maximum
> https://issues.apache.org/jira/browse/HBASE-10925?attachmentOrder=desc
> We have some bad rows growing very large. We need a way to skip these rows
> for most of our jobs.
> Some of the options we considered:
> Option 1:
> Hbase client handle the exception and restart the scanner past bad row by
> capturing the row key where it failed. Can be by adding the rowkey to the
> exception stack trace, which seems brittle. Client would ignore the setting
> if its upgraded before server.
> Option 2:
> Skip through big rows on Server.Go with server level config similar to
> "hbase.table.max.rowsize" or request based by changing the scan request api.
> If allowed to do per request, based on the scan request config, Client will
> have to ignore the setting if its upgraded before server.
> {code}
> try {
> populateResult(results, this.storeHeap, scannerContext, current);
> } catch(RowTooBigException e) {
> LOG.info("Row exceeded the limit in storeheap. Skipping row with
> key:"+Bytes.toString(current.getRowArray()));
> this.storeHeap.reseek(PrivateCellUtil.createLastOnRow(current));
> results.clear();
> scannerContext.clearProgress();
> continue;
> }
> {code}
> Prefer the option 2 with server level config. Please share your inputs
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-19452) Turn ON off heap Bucket Cache by default
[ https://issues.apache.org/jira/browse/HBASE-19452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-19452: -- Fix Version/s: (was: 2.3.0) > Turn ON off heap Bucket Cache by default > > > Key: HBASE-19452 > URL: https://issues.apache.org/jira/browse/HBASE-19452 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > Attachments: HBASE-19452.patch, HBASE-19452_V2.patch, > HBASE-19452_V3.patch > > > BC's hbase.bucketcache.ioengine by default is empty now. Means now BC. > Make this default to be 'offheap'. And the default off heap size for the BC > also to be provided. This can be 8 GB? Better to make it also a % of the > Xmx. Lets continue to be 40% of Xmx as LRU cache default size. > When user has to disable BC, configure size as 0. An empty value of this > config will be treated as default size. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22699) refactor isMetaClearingException
[ https://issues.apache.org/jira/browse/HBASE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018416#comment-17018416 ] Michael Stack commented on HBASE-22699: --- To be clear, this has been reverted from all branches. PRs ready for commit do you think [~Joseph295]? > refactor isMetaClearingException > > > Key: HBASE-22699 > URL: https://issues.apache.org/jira/browse/HBASE-22699 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Junhong Xu >Assignee: Junhong Xu >Priority: Minor > Fix For: 3.0.0, 2.3.0, 2.2.4 > > > It is not so readable -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-11996) Add "Table Creator" to the HTD
[ https://issues.apache.org/jira/browse/HBASE-11996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-11996: -- Fix Version/s: (was: 2.3.0) > Add "Table Creator" to the HTD > -- > > Key: HBASE-11996 > URL: https://issues.apache.org/jira/browse/HBASE-11996 > Project: HBase > Issue Type: New Feature > Components: Admin, master, Operability >Reporter: Matteo Bertozzi >Assignee: Chia-Ping Tsai >Priority: Trivial > > It will be nice storing the user who created the table. It is useful in > situations where you want to remove a table but you don't know who asking to. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-11148) Provide a distributed procedure to globally roll logs
[ https://issues.apache.org/jira/browse/HBASE-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-11148: -- Fix Version/s: (was: 2.3.0) > Provide a distributed procedure to globally roll logs > - > > Key: HBASE-11148 > URL: https://issues.apache.org/jira/browse/HBASE-11148 > Project: HBase > Issue Type: New Feature >Reporter: Jerry He >Assignee: Jerry He >Priority: Major > Attachments: HBASE-11148-trunk-v1.patch, HBASE-11148-trunk-v2.patch > > > Propose a distributed procedure here to globally roll logs. > Currently HBaseAdmin and HBase shell provides a way to roll the WAL on a > single RS. > Some use cases may require that all the RS roll the logs at the same time and > in a coordinated way. Also there may be requirement that some tasks be done > together with the roll log on each region server. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22917) Proc-WAL roll fails always saying someone else has already created log
[
https://issues.apache.org/jira/browse/HBASE-22917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018411#comment-17018411
]
Michael Stack commented on HBASE-22917:
---
Removed 2.3.0 and master as target. ProcedureStore implementation has changed
in these branches.
> Proc-WAL roll fails always saying someone else has already created log
> --
>
> Key: HBASE-22917
> URL: https://issues.apache.org/jira/browse/HBASE-22917
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, wal
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.2.4
>
>
> Recently we met a weird scenario where Procedure WAL roll fails as it is
> already created by someone else.
> Later while going through the logs and code, observed that during Proc-WAL
> roll it failed to write the header. On failure file stream is just closed,
> {code}
> try {
> ProcedureWALFormat.writeHeader(newStream, header);
> startPos = newStream.getPos();
> } catch (IOException ioe) {
> LOG.warn("Encountered exception writing header", ioe);
> newStream.close();
> return false;
> }
> {code}
> Since we don't delete the corrupted file or increment the *flushLogId*, so on
> each retry it is trying to create the same *flushLogId* file. However Hmaster
> failover will resolve this issue, but we should handle it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-22917) Proc-WAL roll fails always saying someone else has already created log
[
https://issues.apache.org/jira/browse/HBASE-22917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-22917:
--
Fix Version/s: (was: 2.3.0)
(was: 3.0.0)
> Proc-WAL roll fails always saying someone else has already created log
> --
>
> Key: HBASE-22917
> URL: https://issues.apache.org/jira/browse/HBASE-22917
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, wal
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.2.4
>
>
> Recently we met a weird scenario where Procedure WAL roll fails as it is
> already created by someone else.
> Later while going through the logs and code, observed that during Proc-WAL
> roll it failed to write the header. On failure file stream is just closed,
> {code}
> try {
> ProcedureWALFormat.writeHeader(newStream, header);
> startPos = newStream.getPos();
> } catch (IOException ioe) {
> LOG.warn("Encountered exception writing header", ioe);
> newStream.close();
> return false;
> }
> {code}
> Since we don't delete the corrupted file or increment the *flushLogId*, so on
> each retry it is trying to create the same *flushLogId* file. However Hmaster
> failover will resolve this issue, but we should handle it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-19562) Purge mirror writing of region and table info into fs at .tableinfo and .regioninfo
[ https://issues.apache.org/jira/browse/HBASE-19562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-19562: -- Fix Version/s: (was: 2.3.0) 3.0.0 > Purge mirror writing of region and table info into fs at .tableinfo and > .regioninfo > --- > > Key: HBASE-19562 > URL: https://issues.apache.org/jira/browse/HBASE-19562 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Critical > Fix For: 3.0.0 > > Attachments: > 0002-HBASE-19562-Purge-mirror-writing-of-region-and-table.patch > > > We don't use these files in hbase2 yet we keep writing them when we create a > table or region. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-18561) Blog post w/intro to SyncTable
[ https://issues.apache.org/jira/browse/HBASE-18561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-18561: -- Fix Version/s: (was: 2.3.0) > Blog post w/intro to SyncTable > -- > > Key: HBASE-18561 > URL: https://issues.apache.org/jira/browse/HBASE-18561 > Project: HBase > Issue Type: Task > Components: community, documentation, Replication >Affects Versions: 1.2.0 >Reporter: Sean Busbey >Priority: Critical > > In addition to HBASE-15557, we should have a blog post that goes over > SyncTable / HashTable . They're a great addition that needs more surfacing to > operators. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23569) Validate that the log cleaner actually cleans oldWALs as expected
[
https://issues.apache.org/jira/browse/HBASE-23569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018408#comment-17018408
]
Hudson commented on HBASE-23569:
Results for branch branch-2
[build #2419 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2419/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2419//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2419//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2419//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Validate that the log cleaner actually cleans oldWALs as expected
> -
>
> Key: HBASE-23569
> URL: https://issues.apache.org/jira/browse/HBASE-23569
> Project: HBase
> Issue Type: Test
> Components: integration tests, master, test
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0, 2.2.3
>
> Attachments: HBASE-23569.master.000.patch,
> HBASE-23569.master.002.patch, HBASE-23569.master.002.patch
>
>
> The fix for HBASE-23287 (LogCleaner is not added to choreService) is in but
> we are lacking test coverage that validates that the log cleaner actually
> cleans oldWALs as expected. Add the test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23701) Make sure HBaseClassTestRule doesn't suffer same issue as HBaseClassTestRuleChecker
[
https://issues.apache.org/jira/browse/HBASE-23701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018409#comment-17018409
]
Hudson commented on HBASE-23701:
Results for branch branch-2
[build #2419 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2419/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2419//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2419//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2419//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Make sure HBaseClassTestRule doesn't suffer same issue as
> HBaseClassTestRuleChecker
> ---
>
> Key: HBASE-23701
> URL: https://issues.apache.org/jira/browse/HBASE-23701
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> [~bharathv] pointed out on HBASE-23695
> ([https://github.com/apache/hbase/pull/1052]) that HBaseClassTestRule suffers
> the same potential bug that I fixed in HBASE-23695 for
> HBaseClassTestRuleChecker. Make sure the fix is in both places.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-18326) Fix and reenable TestMasterProcedureWalLease
[ https://issues.apache.org/jira/browse/HBASE-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-18326. --- Assignee: Szabolcs Bukros Resolution: Fixed Resolving because test was removed. Thanks [~bszabolcs]. Assigned ticket to you as you did research. > Fix and reenable TestMasterProcedureWalLease > > > Key: HBASE-18326 > URL: https://issues.apache.org/jira/browse/HBASE-18326 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Michael Stack >Assignee: Szabolcs Bukros >Priority: Blocker > Fix For: 3.0.0, 2.3.0 > > > Fix and reenable flakey important test. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23612) Update pom.xml to use another 2.5.0 protoc as external protobuf
[ https://issues.apache.org/jira/browse/HBASE-23612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-23612. --- Fix Version/s: 3.0.0 Hadoop Flags: Reviewed Assignee: zhao bo Resolution: Fixed Merged. Thanks for the patch [~bzhaoopenstack] > Update pom.xml to use another 2.5.0 protoc as external protobuf > --- > > Key: HBASE-23612 > URL: https://issues.apache.org/jira/browse/HBASE-23612 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: zhao bo >Assignee: zhao bo >Priority: Major > Fix For: 3.0.0 > > > Currently, there is no protoc 2.5.0 for release [1]. So we can make a new one > for ARM specific. For make sure that could work on ARM. > We will introduce a new ARM artifact for protoc, group_id is > org.openlabtesting.protobuf .. This is just used for protobuf-maven-plugin to > compile .proto files. As the 3.X version of protoc support ARM already. So > this won't affect the internal protoc usage, which is 3.5.1-1 now. > > [1][https://github.com/protocolbuffers/protobuf/issues/3844#issuecomment-343355946] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #959: HBASE-23612: Add new profile to make hbase build success on ARM
saintstack merged pull request #959: HBASE-23612: Add new profile to make hbase build success on ARM URL: https://github.com/apache/hbase/pull/959 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern
saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern URL: https://github.com/apache/hbase/pull/1016#discussion_r368174853 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/util/OnlineMergeTool.java ## @@ -0,0 +1,528 @@ +/** + * Copyright 2010 The Apache Software Foundation Review comment: Fix year. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern
saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL]
HBASE Support Merge region by pattern
URL: https://github.com/apache/hbase/pull/1016#discussion_r368176220
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/util/OnlineMergeTool.java
##
@@ -0,0 +1,528 @@
+/**
+ * Copyright 2010 The Apache Software Foundation
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.util;
+
+import java.io.IOException;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.MasterNotRunningException;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.classification.InterfaceAudience;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.HBaseAdmin;
+import org.apache.hadoop.hbase.client.HConnectionManager;
+import org.apache.hadoop.hbase.client.RegionLocator;
+import org.apache.hadoop.io.WritableComparator;
+import org.apache.hadoop.util.Tool;
+import org.apache.hadoop.util.ToolRunner;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Utility that can merge any two regions in the same table: adjacent,
+ * overlapping or disjoint. It can also merge every regions, two by two.
+ */
+@InterfaceAudience.Private
+public class OnlineMergeTool extends Configured implements Tool {
+ static final Logger LOG = LoggerFactory.getLogger(OnlineMergeTool.class);
+ private final int COMPACTPAUSETIME = 180 * 1000;
+ private final int DEFAULTMERGEPAUSETIME = 120 * 1000;
+ private final String COMPACTIONATTRIBUTE = "MAJOR";
+ private final long GB = 1024L * 1024L * 1024L;
+ private final SimpleDateFormat DATE_FORMAT
+ = new SimpleDateFormat("/MM/dd HH:mm:ss");
+ private final HBaseConfiguration conf;
+ private volatile MetaUtils utils;
+ private volatile boolean isMetaTable;
+ private volatile Connection connection;
+ private volatile Admin admin;
+ // Name of table
+ private String tableName = null;
+ // Name of region 1
+ private String startRegion = null;
+ // Name of region 2
+ private String stopRegion = null;
+ // Name of maxRegionSize
+ private Long maxRegionSize = 0L;
+ // Name of maxRegionCreateTime
+ private String maxRegionCreateTime = null;
+ // Name of numMaxMergePlans
+ private String numMaxMergePlans = null;
+ // Name of targetRegionCount
+ private Long targetRegionCount = 0L;
+ /**
+ * print Execution Plan information
+ */
+ private boolean printExecutionPlan = true;
+ /**
+ * config merge pause time
+ */
+ private int mergePauseTime = 0;
+
+ /**
+ * default constructor
+ */
+ public OnlineMergeTool() throws IOException {
+this(new HBaseConfiguration());
+ }
+
+ /**
+ * @param conf The current configuration.
+ * @throws IOException If IO problem encountered
+ */
+ public OnlineMergeTool(HBaseConfiguration conf) throws IOException {
+super(conf);
+this.conf = conf;
+this.conf.setInt("hbase.client.retries.number", 3);
+this.conf.setInt("hbase.client.pause", 1000);
+this.connection = HConnectionManager.createConnection(this.conf);
+this.admin = connection.getAdmin();
+ }
+
+ /**
+ * Main program
+ *
+ * @param args The command line parameters.
+ */
+ public static void main(String[] args) {
+int status = 0;
+try {
+ status = ToolRunner.run(new OnlineMergeTool(), args);
+} catch (Exception e) {
+ LOG.error("exiting due to error", e);
+ status = -1;
+}
+System.exit(status);
+ }
+
+ @Override
+ public int run(String[] args) throws Exception {
+
+if (!doCommandLine(args)) {
+ return -1;
+}
+
+isMetaTable = Bytes.compareTo(Bytes.toBytes(tableName),
HConstants.META_TABLE_NAME) == 0;
+// Verify file system is up.
+FileSystem fs = FileSystem.get(this.conf); // get DFS handle
+
[GitHub] [hbase] saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern
saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL]
HBASE Support Merge region by pattern
URL: https://github.com/apache/hbase/pull/1016#discussion_r368175421
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/util/OnlineMergeTool.java
##
@@ -0,0 +1,528 @@
+/**
+ * Copyright 2010 The Apache Software Foundation
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.util;
+
+import java.io.IOException;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.MasterNotRunningException;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.classification.InterfaceAudience;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.HBaseAdmin;
+import org.apache.hadoop.hbase.client.HConnectionManager;
+import org.apache.hadoop.hbase.client.RegionLocator;
+import org.apache.hadoop.io.WritableComparator;
+import org.apache.hadoop.util.Tool;
+import org.apache.hadoop.util.ToolRunner;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Utility that can merge any two regions in the same table: adjacent,
+ * overlapping or disjoint. It can also merge every regions, two by two.
+ */
+@InterfaceAudience.Private
+public class OnlineMergeTool extends Configured implements Tool {
+ static final Logger LOG = LoggerFactory.getLogger(OnlineMergeTool.class);
+ private final int COMPACTPAUSETIME = 180 * 1000;
+ private final int DEFAULTMERGEPAUSETIME = 120 * 1000;
+ private final String COMPACTIONATTRIBUTE = "MAJOR";
+ private final long GB = 1024L * 1024L * 1024L;
+ private final SimpleDateFormat DATE_FORMAT
+ = new SimpleDateFormat("/MM/dd HH:mm:ss");
+ private final HBaseConfiguration conf;
+ private volatile MetaUtils utils;
+ private volatile boolean isMetaTable;
+ private volatile Connection connection;
+ private volatile Admin admin;
+ // Name of table
+ private String tableName = null;
+ // Name of region 1
+ private String startRegion = null;
+ // Name of region 2
+ private String stopRegion = null;
+ // Name of maxRegionSize
+ private Long maxRegionSize = 0L;
+ // Name of maxRegionCreateTime
+ private String maxRegionCreateTime = null;
+ // Name of numMaxMergePlans
+ private String numMaxMergePlans = null;
+ // Name of targetRegionCount
+ private Long targetRegionCount = 0L;
+ /**
+ * print Execution Plan information
+ */
+ private boolean printExecutionPlan = true;
+ /**
+ * config merge pause time
+ */
+ private int mergePauseTime = 0;
+
+ /**
+ * default constructor
+ */
+ public OnlineMergeTool() throws IOException {
+this(new HBaseConfiguration());
+ }
+
+ /**
+ * @param conf The current configuration.
+ * @throws IOException If IO problem encountered
+ */
+ public OnlineMergeTool(HBaseConfiguration conf) throws IOException {
Review comment:
Yeah, arent' all merges online? Why not just MergeTool?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern
saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern URL: https://github.com/apache/hbase/pull/1016#discussion_r368175257 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/util/OnlineMergeTool.java ## @@ -0,0 +1,528 @@ +/** + * Copyright 2010 The Apache Software Foundation + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * http://www.apache.org/licenses/LICENSE-2.0 + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hbase.util; + +import java.io.IOException; +import java.text.ParseException; +import java.text.SimpleDateFormat; +import java.util.ArrayList; +import java.util.List; + +import org.apache.hadoop.conf.Configured; +import org.apache.hadoop.fs.FileSystem; +import org.apache.hadoop.hbase.HBaseConfiguration; +import org.apache.hadoop.hbase.HConstants; +import org.apache.hadoop.hbase.HRegionInfo; +import org.apache.hadoop.hbase.MasterNotRunningException; +import org.apache.hadoop.hbase.TableName; +import org.apache.hadoop.hbase.classification.InterfaceAudience; +import org.apache.hadoop.hbase.client.Admin; +import org.apache.hadoop.hbase.client.Connection; +import org.apache.hadoop.hbase.client.HBaseAdmin; +import org.apache.hadoop.hbase.client.HConnectionManager; +import org.apache.hadoop.hbase.client.RegionLocator; +import org.apache.hadoop.io.WritableComparator; +import org.apache.hadoop.util.Tool; +import org.apache.hadoop.util.ToolRunner; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Utility that can merge any two regions in the same table: adjacent, + * overlapping or disjoint. It can also merge every regions, two by two. Review comment: This is for branch-1? In branch-2, we have a multimerge procedure that can take many (adjacent) regions and squash them up together. We could have this in branch-1. What would you do for branch-2? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern
saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern URL: https://github.com/apache/hbase/pull/1016#discussion_r368174999 ## File path: bin/hbase.cmd ## @@ -436,4 +436,5 @@ goto :eof echo mapredcpDump CLASSPATH entries required by mapreduce echo version Print the version echo CLASSNAME Run the class named CLASSNAME + echo onlinemerge Run the merge tool Review comment: What is difference between a merge and an online merge? Why does this have to be here and not in the shell? Shell already has a merge region facility. Add this as an option to the shell command? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern
saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL]
HBASE Support Merge region by pattern
URL: https://github.com/apache/hbase/pull/1016#discussion_r368175342
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/util/OnlineMergeTool.java
##
@@ -0,0 +1,528 @@
+/**
+ * Copyright 2010 The Apache Software Foundation
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.util;
+
+import java.io.IOException;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.MasterNotRunningException;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.classification.InterfaceAudience;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.HBaseAdmin;
+import org.apache.hadoop.hbase.client.HConnectionManager;
+import org.apache.hadoop.hbase.client.RegionLocator;
+import org.apache.hadoop.io.WritableComparator;
+import org.apache.hadoop.util.Tool;
+import org.apache.hadoop.util.ToolRunner;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Utility that can merge any two regions in the same table: adjacent,
+ * overlapping or disjoint. It can also merge every regions, two by two.
+ */
+@InterfaceAudience.Private
+public class OnlineMergeTool extends Configured implements Tool {
+ static final Logger LOG = LoggerFactory.getLogger(OnlineMergeTool.class);
+ private final int COMPACTPAUSETIME = 180 * 1000;
+ private final int DEFAULTMERGEPAUSETIME = 120 * 1000;
+ private final String COMPACTIONATTRIBUTE = "MAJOR";
+ private final long GB = 1024L * 1024L * 1024L;
+ private final SimpleDateFormat DATE_FORMAT
+ = new SimpleDateFormat("/MM/dd HH:mm:ss");
Review comment:
Do ISO8601?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern
saintstack commented on a change in pull request #1016: HBASE-23656 [MERGETOOL]
HBASE Support Merge region by pattern
URL: https://github.com/apache/hbase/pull/1016#discussion_r368176192
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/util/OnlineMergeTool.java
##
@@ -0,0 +1,528 @@
+/**
+ * Copyright 2010 The Apache Software Foundation
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.util;
+
+import java.io.IOException;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionInfo;
+import org.apache.hadoop.hbase.MasterNotRunningException;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.classification.InterfaceAudience;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.HBaseAdmin;
+import org.apache.hadoop.hbase.client.HConnectionManager;
+import org.apache.hadoop.hbase.client.RegionLocator;
+import org.apache.hadoop.io.WritableComparator;
+import org.apache.hadoop.util.Tool;
+import org.apache.hadoop.util.ToolRunner;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Utility that can merge any two regions in the same table: adjacent,
+ * overlapping or disjoint. It can also merge every regions, two by two.
+ */
+@InterfaceAudience.Private
+public class OnlineMergeTool extends Configured implements Tool {
+ static final Logger LOG = LoggerFactory.getLogger(OnlineMergeTool.class);
+ private final int COMPACTPAUSETIME = 180 * 1000;
+ private final int DEFAULTMERGEPAUSETIME = 120 * 1000;
+ private final String COMPACTIONATTRIBUTE = "MAJOR";
+ private final long GB = 1024L * 1024L * 1024L;
+ private final SimpleDateFormat DATE_FORMAT
+ = new SimpleDateFormat("/MM/dd HH:mm:ss");
+ private final HBaseConfiguration conf;
+ private volatile MetaUtils utils;
+ private volatile boolean isMetaTable;
+ private volatile Connection connection;
+ private volatile Admin admin;
+ // Name of table
+ private String tableName = null;
+ // Name of region 1
+ private String startRegion = null;
+ // Name of region 2
+ private String stopRegion = null;
+ // Name of maxRegionSize
+ private Long maxRegionSize = 0L;
+ // Name of maxRegionCreateTime
+ private String maxRegionCreateTime = null;
+ // Name of numMaxMergePlans
+ private String numMaxMergePlans = null;
+ // Name of targetRegionCount
+ private Long targetRegionCount = 0L;
+ /**
+ * print Execution Plan information
+ */
+ private boolean printExecutionPlan = true;
+ /**
+ * config merge pause time
+ */
+ private int mergePauseTime = 0;
+
+ /**
+ * default constructor
+ */
+ public OnlineMergeTool() throws IOException {
+this(new HBaseConfiguration());
+ }
+
+ /**
+ * @param conf The current configuration.
+ * @throws IOException If IO problem encountered
+ */
+ public OnlineMergeTool(HBaseConfiguration conf) throws IOException {
+super(conf);
+this.conf = conf;
+this.conf.setInt("hbase.client.retries.number", 3);
+this.conf.setInt("hbase.client.pause", 1000);
+this.connection = HConnectionManager.createConnection(this.conf);
+this.admin = connection.getAdmin();
+ }
+
+ /**
+ * Main program
+ *
+ * @param args The command line parameters.
+ */
+ public static void main(String[] args) {
+int status = 0;
+try {
+ status = ToolRunner.run(new OnlineMergeTool(), args);
+} catch (Exception e) {
+ LOG.error("exiting due to error", e);
+ status = -1;
+}
+System.exit(status);
+ }
+
+ @Override
+ public int run(String[] args) throws Exception {
+
+if (!doCommandLine(args)) {
+ return -1;
+}
+
+isMetaTable = Bytes.compareTo(Bytes.toBytes(tableName),
HConstants.META_TABLE_NAME) == 0;
+// Verify file system is up.
+FileSystem fs = FileSystem.get(this.conf); // get DFS handle
+
[GitHub] [hbase] saintstack commented on a change in pull request #1022: HBASE-23680 RegionProcedureStore missing cleaning of hfile archive
saintstack commented on a change in pull request #1022: HBASE-23680
RegionProcedureStore missing cleaning of hfile archive
URL: https://github.com/apache/hbase/pull/1022#discussion_r368174200
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -1520,8 +1518,10 @@ protected void stopServiceThreads() {
private void createProcedureExecutor() throws IOException {
MasterProcedureEnv procEnv = new MasterProcedureEnv(this);
-procedureStore =
- new RegionProcedureStore(this, new
MasterProcedureEnv.FsUtilsLeaseRecovery(this));
+// Create cleaner thread pool
+cleanerPool = new DirScanPool(conf);
Review comment:
Strange place to start cleanerPool down here in the procedure startup given
it is used cleaning WALs and hfile...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23706) Update checkstyle plugin (and update checkstyle.xml to match)
[ https://issues.apache.org/jira/browse/HBASE-23706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018385#comment-17018385 ] Bharath Vissapragada commented on HBASE-23706: -- Same as HBASE-23700? > Update checkstyle plugin (and update checkstyle.xml to match) > - > > Key: HBASE-23706 > URL: https://issues.apache.org/jira/browse/HBASE-23706 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Priority: Major > > In parent issue, its suggested we update our checkstyle plugin to match of > intellij plugin default at least. Will need checkstyle.xml changes else it > fails parse (See notes in parent by [~bharathv] on what needs doing). > For extra points, do the [~ndimiduk] suggestion: "...It would be nice also if > we could commit the .idea/checkstyle-idea.xml file with the checkstyle > version used buy the plugin pinned to the same version as we're using in > maven." -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23690) Checkstyle plugin complains about our checkstyle.xml format; doc how to resolve mismatched version
[
https://issues.apache.org/jira/browse/HBASE-23690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-23690.
---
Fix Version/s: 3.0.0
Assignee: Michael Stack
Resolution: Fixed
Resolving. Made subissue to address the update of the checkstyle plugin and to
do the Nick nice suggestion above.
Thanks for reviews.
> Checkstyle plugin complains about our checkstyle.xml format; doc how to
> resolve mismatched version
> --
>
> Key: HBASE-23690
> URL: https://issues.apache.org/jira/browse/HBASE-23690
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0
>
>
> Trying to add the checkstyle.xml to the intellij checkstyle plugin after
> reading HBASE-23688, it complains with the following when it reads in the
> config file:
> {code}
> com.puppycrawl.tools.checkstyle.api.CheckstyleException: cannot initialize
> module TreeWalker - TreeWalker is not allowed as a parent of LineLength
> Please review 'Parent Module' section for this Check in web documentation if
> Check is standard.
> at com.puppycrawl.tools.checkstyle.Checker.setupChild(Checker.java:473)
> at
> com.puppycrawl.tools.checkstyle.api.AutomaticBean.configure(AutomaticBean.java:198)
> at
> org.infernus.idea.checkstyle.service.cmd.OpCreateChecker.execute(OpCreateChecker.java:61)
> at
> org.infernus.idea.checkstyle.service.cmd.OpCreateChecker.execute(OpCreateChecker.java:26)
> at
> org.infernus.idea.checkstyle.service.CheckstyleActionsImpl.executeCommand(CheckstyleActionsImpl.java:130)
> at
> org.infernus.idea.checkstyle.service.CheckstyleActionsImpl.createChecker(CheckstyleActionsImpl.java:60)
> at
> org.infernus.idea.checkstyle.service.CheckstyleActionsImpl.createChecker(CheckstyleActionsImpl.java:51)
> at
> org.infernus.idea.checkstyle.checker.CheckerFactoryWorker.run(CheckerFactoryWorker.java:46)
> Caused by: com.puppycrawl.tools.checkstyle.api.CheckstyleException:
> TreeWalker is not allowed as a parent of LineLength Please review 'Parent
> Module' section for this Check in web documentation if Check is standard.
> at
> com.puppycrawl.tools.checkstyle.TreeWalker.setupChild(TreeWalker.java:147)
> at
> com.puppycrawl.tools.checkstyle.api.AutomaticBean.configure(AutomaticBean.java:198)
> at com.puppycrawl.tools.checkstyle.Checker.setupChild(Checker.java:468)
> ... 7 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23706) Update checkstyle plugin (and update checkstyle.xml to match)
Michael Stack created HBASE-23706: - Summary: Update checkstyle plugin (and update checkstyle.xml to match) Key: HBASE-23706 URL: https://issues.apache.org/jira/browse/HBASE-23706 Project: HBase Issue Type: Sub-task Reporter: Michael Stack In parent issue, its suggested we update our checkstyle plugin to match of intellij plugin default at least. Will need checkstyle.xml changes else it fails parse (See notes in parent by [~bharathv] on what needs doing). For extra points, do the [~ndimiduk] suggestion: "...It would be nice also if we could commit the .idea/checkstyle-idea.xml file with the checkstyle version used buy the plugin pinned to the same version as we're using in maven." -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #1044: HBASE-23690 Checkstyle plugin complains about our checkstyle.xml format
saintstack merged pull request #1044: HBASE-23690 Checkstyle plugin complains about our checkstyle.xml format URL: https://github.com/apache/hbase/pull/1044 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1055: HBASE-23699 ByteBuff may not recycle correctly when NettyRpcServer enabled and client disconnect
saintstack commented on a change in pull request #1055: HBASE-23699 ByteBuff
may not recycle correctly when NettyRpcServer enabled and client disconnect
URL: https://github.com/apache/hbase/pull/1055#discussion_r368166340
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServerResponseEncoder.java
##
@@ -44,7 +44,6 @@ public void write(ChannelHandlerContext ctx, Object msg,
ChannelPromise promise)
RpcResponse resp = (RpcResponse) msg;
BufferChain buf = resp.getResponse();
ctx.write(Unpooled.wrappedBuffer(buf.getBuffers()),
promise).addListener(f -> {
-resp.done();
Review comment:
Any chance of an explanation on how this addresses the issue? Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1055: HBASE-23699 ByteBuff may not recycle correctly when NettyRpcServer enabled and client disconnect
saintstack commented on a change in pull request #1055: HBASE-23699 ByteBuff
may not recycle correctly when NettyRpcServer enabled and client disconnect
URL: https://github.com/apache/hbase/pull/1055#discussion_r368166024
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServerResponseEncoder.java
##
@@ -44,7 +44,6 @@ public void write(ChannelHandlerContext ctx, Object msg,
ChannelPromise promise)
RpcResponse resp = (RpcResponse) msg;
BufferChain buf = resp.getResponse();
ctx.write(Unpooled.wrappedBuffer(buf.getBuffers()),
promise).addListener(f -> {
-resp.done();
Review comment:
When is 'done' called or is it that we now defer to cleanup?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters…
Apache-HBase commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters… URL: https://github.com/apache/hbase/pull/1012#issuecomment-575822517 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 0s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 31s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | checkstyle | 1m 25s | master passed | | +1 :green_heart: | shadedjars | 4m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | | +0 :ok: | spotbugs | 4m 31s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 29s | master passed | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 2m 38s | root in the patch failed. | | -1 :x: | compile | 0m 52s | hbase-server in the patch failed. | | -1 :x: | javac | 0m 52s | hbase-server in the patch failed. | | -1 :x: | checkstyle | 1m 21s | hbase-server: The patch generated 1 new + 259 unchanged - 1 fixed = 260 total (was 260) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | shadedjars | 3m 37s | patch has 16 errors when building our shaded downstream artifacts. | | -1 :x: | hadoopcheck | 2m 4s | The patch causes 16 errors with Hadoop v2.8.5. | | -1 :x: | hadoopcheck | 4m 13s | The patch causes 16 errors with Hadoop v2.9.2. | | -1 :x: | hadoopcheck | 6m 24s | The patch causes 16 errors with Hadoop v3.1.2. | | +1 :green_heart: | javadoc | 0m 32s | the patch passed | | -1 :x: | findbugs | 0m 50s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 52s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 38m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1012 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 9516bd0457ab 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1012/out/precommit/personality/provided.sh | | git revision | master / df8f80a819 | | Default Java | 1.8.0_181 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/artifact/out/patch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/artifact/out/patch-compile-hbase-server.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/artifact/out/patch-compile-hbase-server.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/artifact/out/diff-checkstyle-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/artifact/out/patch-shadedjars.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/artifact/out/patch-javac-2.8.5.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/artifact/out/patch-javac-2.9.2.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/artifact/out/patch-javac-3.1.2.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/artifact/out/patch-findbugs-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1012/5/testReport/ | | Max. process+thread count | 96 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output |
[GitHub] [hbase] petersomogyi commented on issue #1061: HBASE-23653 Expose content of meta table in web ui; addendum
petersomogyi commented on issue #1061: HBASE-23653 Expose content of meta table in web ui; addendum URL: https://github.com/apache/hbase/pull/1061#issuecomment-575821829 > What runs error prone? I haven't seen this on any of the precommit reports. It does not run on precommit. I filed HBASE-23704 for this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21345) [hbck2] Allow version check to proceed even though master is 'initializing'.
[ https://issues.apache.org/jira/browse/HBASE-21345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018353#comment-17018353 ] Sakthi commented on HBASE-21345: This looks like a good addition candidate for branch-2.2+ as well. Any objections if create a Jira for porting this to 2.2+ [~stack]? > [hbck2] Allow version check to proceed even though master is 'initializing'. > > > Key: HBASE-21345 > URL: https://issues.apache.org/jira/browse/HBASE-21345 > Project: HBase > Issue Type: Bug > Components: hbck2 >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 2.1.1, 2.0.3 > > Attachments: > 0001-HBASE-21345-hbck2-Allow-version-check-to-proceed-eve.patch, > 0001-HBASE-21345-hbck2-Allow-version-check-to-proceed-eve.patch > > > We recently added to hbck2 a check of the cluster version it is to go > against. This means a getClusterMetrics call with the option HBASE_VERSION > set. > In testing, trying to fix a failed namespace assign on startup, hbck2 was > shut out with a "PleaseHoldException". > Let the getClusterMetrics call happen even during startup... so tooling can > probe Master state externally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on issue #1062: HBASE-23705 Add CellComparator to HFileContext
saintstack commented on issue #1062: HBASE-23705 Add CellComparator to HFileContext URL: https://github.com/apache/hbase/pull/1062#issuecomment-575812631 Patch looks big but core changes are small just having cellcomparator in context only and not all over the place. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #1043: HBASE-23055 Alter hbase:meta
saintstack commented on issue #1043: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/1043#issuecomment-575812360 Fix checkstyle. Any chance of review? Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters…
saintstack commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters… URL: https://github.com/apache/hbase/pull/1012#issuecomment-575810487 Updated patch but same two tests will fail until HBASE-23705 goes in. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23698) Switch central repo to https on branch-1
[
https://issues.apache.org/jira/browse/HBASE-23698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018347#comment-17018347
]
Hudson commented on HBASE-23698:
Results for branch branch-1.3
[build #1103 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/1103/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/1103//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/1103//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/1103//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Switch central repo to https on branch-1
>
>
> Key: HBASE-23698
> URL: https://issues.apache.org/jira/browse/HBASE-23698
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Affects Versions: 1.6.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Blocker
> Labels: broken-build
> Fix For: 1.6.0, 1.3.7, 1.4.13
>
>
> Maven central started enforcing https [1] today and branch-1 builds are
> failing presumably due to an older maven version.
> Maven 3.2.3 [2] and later already switched the defaults, so not every one
> might run into this problem, depending on what mvn they are using. Jenkins
> jobs are hitting this, so they are likely using an older version.
> [1] https://blog.sonatype.com/central-repository-moving-to-https
> [2] https://issues.apache.org/jira/browse/MNG-5672
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack opened a new pull request #1062: HBASE-23705 Add CellComparator to HFileContext
saintstack opened a new pull request #1062: HBASE-23705 Add CellComparator to HFileContext URL: https://github.com/apache/hbase/pull/1062 Codecs don't have access to what CellComparator to use. Backfill. M hbase-common/src/main/java/org/apache/hadoop/hbase/CellComparator.java Adds a new compareRows with default implementation that takes a ByteBuffer. Needed by the index in a block encoder implementation. M hbase-common/src/main/java/org/apache/hadoop/hbase/CellComparatorImpl.java Adds implementation for meta of new compareRows method. Adds utility method for figuring comparator based off tablename. M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/AbstractDataBlockEncoder.java M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/RowIndexCodecV1.java M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/RowIndexSeekerV1.java Comparator is in context. Remove redundant handling. M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java Comparator is in context. Remove redundant handling. Clean javadoc. M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDecodingContext.java Clean javadoc. M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/RowIndexEncoderV1.java Cache context so can use it to get comparator to use later. M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileContext.java Cache cellcomparator to use. Javdoc on diff between HFileContext and HFileInfo. M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileContextBuilder.java Add CellComparator M hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterImpl.java M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileWriter.java Remove comparator caching. Get from context instead. M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java Skip a reflection if we can. M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileInfo.java Javadoc. Removed unused filed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018333#comment-17018333
]
Hudson commented on HBASE-18095:
Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #42 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/42/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/42//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/42//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/42//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide an option for clients to find the server hosting META that does not
> involve the ZooKeeper client
>
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
> Issue Type: New Feature
> Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the
> regionserver hosting the meta table region. Site configuration provides the
> client a list of ZK quorum peers and the client uses an embedded ZK client to
> query meta location. Timeouts and retry behavior of this embedded ZK client
> are managed orthogonally to HBase layer settings and in some cases the ZK
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage
> or network partition.
> We should consider new configuration settings that provide a list of
> well-known master and backup master locations, and with this information the
> client can contact any of the master processes directly. Any master in either
> active or passive state will track meta location and respond to requests for
> it with its cached last known location. If this location is stale, the client
> can ask again with a flag set that requests the master refresh its location
> cache and return the up-to-date location. Every client interaction with the
> cluster thus uses only HBase RPC as transport, with appropriate settings
> applied to the connection. The configuration toggle that enables this
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and
> contact the ZK service directly at the beginning of the connection lifecycle.
> This has several benefits. ZK service need not be exposed to clients, and
> their potential abuse, yet no benefit ZK provides the HBase server cluster is
> compromised. Normalizing HBase client and ZK client timeout settings and
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer
> necessary.
> And, from [~ghelmling]: There is an additional complication here for
> token-based authentication. When a delegation token is used for SASL
> authentication, the client uses the cluster ID obtained from Zookeeper to
> select the token identifier to use. So there would also need to be some
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] joshelser commented on issue #936: HBASE-17115 Define UI admins via an ACL
joshelser commented on issue #936: HBASE-17115 Define UI admins via an ACL URL: https://github.com/apache/hbase/pull/936#issuecomment-575793728 > I don't see the logs, ... pages listed. Those should be limited to admins too? https://github.com/apache/hbase/blob/978546b2f247b29dd63bad55b17fdc2e7a31e55d/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L681-L683 The logs servlet already had the AdminAuthorizedServlet in the chain; it was just ineffective because we weren't configuring/setting an `ADMIN_ACL`. > I don't see the ... debug, ... pages listed. Those should be limited to admins too? Good catch! Was missing this, will fix. > I don't see the ... zk dump pages listed. Those should be limited to admins too? I'm not finding this, @busbey . I need to go looking to see if this is a branch-specific feature. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23698) Switch central repo to https on branch-1
[
https://issues.apache.org/jira/browse/HBASE-23698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018302#comment-17018302
]
Hudson commented on HBASE-23698:
Results for branch branch-1.4
[build #1150 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1150/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1150//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1150//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1150//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Switch central repo to https on branch-1
>
>
> Key: HBASE-23698
> URL: https://issues.apache.org/jira/browse/HBASE-23698
> Project: HBase
> Issue Type: Bug
> Components: tooling
>Affects Versions: 1.6.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Blocker
> Labels: broken-build
> Fix For: 1.6.0, 1.3.7, 1.4.13
>
>
> Maven central started enforcing https [1] today and branch-1 builds are
> failing presumably due to an older maven version.
> Maven 3.2.3 [2] and later already switched the defaults, so not every one
> might run into this problem, depending on what mvn they are using. Jenkins
> jobs are hitting this, so they are likely using an older version.
> [1] https://blog.sonatype.com/central-repository-moving-to-https
> [2] https://issues.apache.org/jira/browse/MNG-5672
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23705) Add CellComparator to HFileContext
Michael Stack created HBASE-23705: - Summary: Add CellComparator to HFileContext Key: HBASE-23705 URL: https://issues.apache.org/jira/browse/HBASE-23705 Project: HBase Issue Type: Sub-task Components: io Reporter: Michael Stack Assignee: Michael Stack Fix For: 3.0.0, 2.3.0 The HFileContext is present when reading and writing files. It is populated at read time using HFile trailer content and file metadata. At write time, we create it up front. Interesting is that though CellComparator is written to the HFile trailer, and parse of the Trailer creates an HFileInfo which builds the HFileContext at read time, the HFileContext does not expose what CellComparator to use decoding and seeking. Around the codebase there are various compensations made for this lack with decoders that actually have a decoding context (with a reference to the hfilecontext), hard-coding use of the default CellComparator. StoreFileInfo will use default if not passed a comparator (even though we'd just read the trailer) and HFile itself is similar. Let me fix this situation removing ambiguity. It will also fix bugs in parent issue where UTs are failing because wrong CellComparator is being used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-21065) Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters on meta while we are at it)
[
https://issues.apache.org/jira/browse/HBASE-21065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018270#comment-17018270
]
Michael Stack commented on HBASE-21065:
---
Changing the encoding on meta exposes the fact that the ROW_INDEX_V1 encoder
does not work on the hbase:meta table; it has hard-coded the user-space
CellComparator. Reviewing how CellComparators are instantiated around the
codebase, we are inconsistent and encoding context does not have what
CellComparator is appropriate. Let me fix this first. Will fix the UT failures
we're seeing in the PR here.
> Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters on meta while we
> are at it)
> -
>
> Key: HBASE-21065
> URL: https://issues.apache.org/jira/browse/HBASE-21065
> Project: HBase
> Issue Type: Improvement
> Components: meta, Performance
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
>
> Some users end up hitting meta hard. Bulk is probably because our client goes
> to meta too often, and the real 'fix' for a saturated meta is splitting it,
> but the encoding that came in with HBASE-16213, ROW_INDEX_V1, could help in
> the near term. It adds an index on hfile blocks and helped improve random
> reads against user-space tables (less compares as we used index to go direct
> to requested Cells rather than look at each Cell in turn until we found what
> we wanted -- see RN on HBASE-16213 for citation).
> I also noticed code-reading that we don't enable blooms on hbase:meta tables;
> that could save some CPU and speed things up a bit too:
> {code}
> // Disable blooms for meta. Needs work. Seems to mess w/
> getClosestOrBefore.
> .setBloomFilterType(BloomType.NONE)
> {code}
> This issue is about doing a bit of perf compare of encoding *on* vs current
> default (and will check diff in size of indexed blocks).
> Meta access is mostly random-read I believe (A review of a user's access
> showed this so at least for their workload). The nice addition, HBASE-19722
> Meta query statistics metrics source, would help verify if it saw some usage
> on a prod cluster.
> If all is good, I'd like to make a small patch, one that could be easily
> backported, with minimal changes in it.
> As is, its all a little awkward as the meta table schema is hard-coded and
> meta is immutable -- stuff we'll have to fix if we want to split meta -- so
> in the meantime it requires a code change to enable (and a backport of
> HBASE-16213 -- this patch is in 1.4.0 only currently, perhaps that is
> enough). Code change to enable is small:
> {code}
> diff --git
> a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
>
> b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> index 28c7ec3c2f..8f08f94dc1 100644
> ---
> a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> +++
> b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
> @@ -160,6 +160,7 @@ public class FSTableDescriptors implements
> TableDescriptors {
> .setScope(HConstants.REPLICATION_SCOPE_LOCAL)
> // Disable blooms for meta. Needs work. Seems to mess w/
> getClosestOrBefore.
> .setBloomFilterType(BloomType.NONE)
> +
> .setDataBlockEncoding(org.apache.hadoop.hbase.io.encoding.DataBlockEncoding.ROW_INDEX_V1)
> .build())
>
> .setColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(HConstants.TABLE_FAMILY)
> .setMaxVersions(conf.getInt(HConstants.HBASE_META_VERSIONS,
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters…
saintstack commented on issue #1012: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters… URL: https://github.com/apache/hbase/pull/1012#issuecomment-575769016 Let me break out the CellComparator work. It has come to overwhelm this patch. It needs to land before this one else UTs keep failing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1060: HBASE-23347 Allow custom authentication methods for RPCs; addendum
Apache-HBase commented on issue #1060: HBASE-23347 Allow custom authentication methods for RPCs; addendum URL: https://github.com/apache/hbase/pull/1060#issuecomment-575765857 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 1s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 37s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 36s | master passed | | +1 :green_heart: | compile | 1m 31s | master passed | | +1 :green_heart: | checkstyle | 1m 56s | master passed | | +1 :green_heart: | shadedjars | 4m 49s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 7s | master passed | | +0 :ok: | spotbugs | 5m 4s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 6m 15s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 9s | the patch passed | | +1 :green_heart: | compile | 1m 35s | the patch passed | | +1 :green_heart: | javac | 1m 35s | the patch passed | | +1 :green_heart: | checkstyle | 1m 55s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 19s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 6s | the patch passed | | +1 :green_heart: | findbugs | 6m 34s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 1m 59s | hbase-client in the patch failed. | | -1 :x: | unit | 172m 18s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 57s | The patch does not generate ASF License warnings. | | | | 242m 17s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.security.provider.TestSaslClientAuthenticationProviders | | | hadoop.hbase.master.assignment.TestRegionMoveAndAbandon | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1060/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1060 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux b3ad105a52a9 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1060/out/precommit/personality/provided.sh | | git revision | master / c4395b52c9 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1060/2/artifact/out/patch-unit-hbase-client.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1060/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1060/2/testReport/ | | Max. process+thread count | 5216 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1060/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1030: HBASE-23621 Reduced the number of Checkstyle violations in tests of hbase-common
Apache-HBase commented on issue #1030: HBASE-23621 Reduced the number of Checkstyle violations in tests of hbase-common URL: https://github.com/apache/hbase/pull/1030#issuecomment-575743560 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 11 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 1m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 7m 30s | branch-1 passed | | +1 :green_heart: | compile | 0m 59s | branch-1 passed with JDK v1.8.0_242 | | +1 :green_heart: | compile | 1m 9s | branch-1 passed with JDK v1.7.0_252 | | +1 :green_heart: | checkstyle | 2m 21s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 55s | branch-1 passed with JDK v1.8.0_242 | | +1 :green_heart: | javadoc | 1m 6s | branch-1 passed with JDK v1.7.0_252 | | +0 :ok: | spotbugs | 2m 51s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 2s | branch-1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 5s | the patch passed | | +1 :green_heart: | compile | 0m 59s | the patch passed with JDK v1.8.0_242 | | +1 :green_heart: | javac | 0m 59s | the patch passed | | +1 :green_heart: | compile | 1m 10s | the patch passed with JDK v1.7.0_252 | | +1 :green_heart: | javac | 1m 10s | the patch passed | | +1 :green_heart: | checkstyle | 0m 30s | hbase-common: The patch generated 0 new + 20 unchanged - 32 fixed = 20 total (was 52) | | +1 :green_heart: | checkstyle | 1m 46s | hbase-server: The patch generated 0 new + 330 unchanged - 1 fixed = 330 total (was 331) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 14s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 48s | the patch passed with JDK v1.8.0_242 | | +1 :green_heart: | javadoc | 1m 4s | the patch passed with JDK v1.7.0_252 | | +1 :green_heart: | findbugs | 4m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 31s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 141m 5s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 44s | The patch does not generate ASF License warnings. | | | | 193m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1030/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1030 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux a610891b9b7b 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1030/out/precommit/personality/provided.sh | | git revision | branch-1 / d466f77 | | Default Java | 1.7.0_252 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_242 /usr/lib/jvm/zulu-7-amd64:1.7.0_252 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1030/7/testReport/ | | Max. process+thread count | 4996 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1030/7/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific
[GitHub] [hbase] Apache-HBase commented on issue #1061: HBASE-23653 Expose content of meta table in web ui; addendum
Apache-HBase commented on issue #1061: HBASE-23653 Expose content of meta table in web ui; addendum URL: https://github.com/apache/hbase/pull/1061#issuecomment-575743224 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 20s | master passed | | +1 :green_heart: | compile | 0m 57s | master passed | | +1 :green_heart: | checkstyle | 1m 18s | master passed | | +1 :green_heart: | shadedjars | 4m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | | +0 :ok: | spotbugs | 4m 23s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 20s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 55s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | checkstyle | 1m 18s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 33s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 48s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | | +1 :green_heart: | findbugs | 4m 41s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 155m 17s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. | | | | 212m 52s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.assignment.TestRegionMoveAndAbandon | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1061/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1061 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 14d6ec3278ac 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1061/out/precommit/personality/provided.sh | | git revision | master / 3b64ea50ec | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1061/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1061/1/testReport/ | | Max. process+thread count | 4810 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1061/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] joshelser commented on a change in pull request #936: HBASE-17115 Define UI admins via an ACL
joshelser commented on a change in pull request #936: HBASE-17115 Define UI admins via an ACL URL: https://github.com/apache/hbase/pull/936#discussion_r368071622 ## File path: hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java ## @@ -770,30 +777,28 @@ public void addJerseyResourcePackage(final String packageName, } /** - * Add a servlet in the server. + * Adds a servlet in the server that any user can access. * @param name The name of the servlet (can be passed as null) * @param pathSpec The path spec for the servlet * @param clazz The servlet class */ - public void addServlet(String name, String pathSpec, + public void addUnprivilegedServlet(String name, String pathSpec, Review comment: > Adds a servlet in the server that any user can access. I expanded this to the Javadoc already, but I suppose I can expand some more. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] joshelser commented on a change in pull request #936: HBASE-17115 Define UI admins via an ACL
joshelser commented on a change in pull request #936: HBASE-17115 Define UI admins via an ACL URL: https://github.com/apache/hbase/pull/936#discussion_r368068359 ## File path: hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java ## @@ -131,6 +131,10 @@ "signature.secret.file"; public static final String HTTP_AUTHENTICATION_SIGNATURE_SECRET_FILE_KEY = HTTP_AUTHENTICATION_PREFIX + HTTP_AUTHENTICATION_SIGNATURE_SECRET_FILE_SUFFIX; + public static final String HTTP_SPNEGO_AUTHENTICATION_ADMIN_USERS_KEY = + HTTP_SPNEGO_AUTHENTICATION_PREFIX + "admin.users"; Review comment: Yes, likely. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23701) Make sure HBaseClassTestRule doesn't suffer same issue as HBaseClassTestRuleChecker
[ https://issues.apache.org/jira/browse/HBASE-23701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-23701: --- Fix Version/s: 2.1.9 2.2.3 2.3.0 3.0.0 > Make sure HBaseClassTestRule doesn't suffer same issue as > HBaseClassTestRuleChecker > --- > > Key: HBASE-23701 > URL: https://issues.apache.org/jira/browse/HBASE-23701 > Project: HBase > Issue Type: Bug >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Minor > Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9 > > > [~bharathv] pointed out on HBASE-23695 > ([https://github.com/apache/hbase/pull/1052]) that HBaseClassTestRule suffers > the same potential bug that I fixed in HBASE-23695 for > HBaseClassTestRuleChecker. Make sure the fix is in both places. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23701) Make sure HBaseClassTestRule doesn't suffer same issue as HBaseClassTestRuleChecker
[ https://issues.apache.org/jira/browse/HBASE-23701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser resolved HBASE-23701. Hadoop Flags: Reviewed Resolution: Fixed Thanks for the reviews, Bharath and Viraj! > Make sure HBaseClassTestRule doesn't suffer same issue as > HBaseClassTestRuleChecker > --- > > Key: HBASE-23701 > URL: https://issues.apache.org/jira/browse/HBASE-23701 > Project: HBase > Issue Type: Bug >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Minor > Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9 > > > [~bharathv] pointed out on HBASE-23695 > ([https://github.com/apache/hbase/pull/1052]) that HBaseClassTestRule suffers > the same potential bug that I fixed in HBASE-23695 for > HBaseClassTestRuleChecker. Make sure the fix is in both places. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] asfgit closed pull request #1057: HBASE-23701 Try to converge automated checks around Category
asfgit closed pull request #1057: HBASE-23701 Try to converge automated checks around Category URL: https://github.com/apache/hbase/pull/1057 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-23665) Split unit tests from TestTableName into a separate test only class
[
https://issues.apache.org/jira/browse/HBASE-23665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-23665.
--
Resolution: Fixed
Thanks for the cleanup [~bharathv].
> Split unit tests from TestTableName into a separate test only class
> ---
>
> Key: HBASE-23665
> URL: https://issues.apache.org/jira/browse/HBASE-23665
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Nick Dimiduk
>Assignee: Bharath Vissapragada
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
>
> The class {{TestTableName}} in hbase-common is both a unit test for the
> {{TableName}} class, and a junit {{TestRule}}. This double-duty is strange;
> the Rule functionality is not discoverable at all. Split the Rule out into a
> separate class, maybe {{TableNameRule}} or some such.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk merged pull request #1051: HBASE-23665: Split unit tests from TestTableName into a separate test…
ndimiduk merged pull request #1051: HBASE-23665: Split unit tests from TestTableName into a separate test… URL: https://github.com/apache/hbase/pull/1051 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1057: HBASE-23701 Try to converge automated checks around Category
Apache-HBase commented on issue #1057: HBASE-23701 Try to converge automated checks around Category URL: https://github.com/apache/hbase/pull/1057#issuecomment-575725247 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 30s | master passed | | +1 :green_heart: | compile | 0m 25s | master passed | | +1 :green_heart: | checkstyle | 0m 29s | master passed | | +1 :green_heart: | shadedjars | 5m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | master passed | | +0 :ok: | spotbugs | 1m 0s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 56s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 54s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | checkstyle | 0m 27s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 19m 45s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 28s | the patch passed | | +1 :green_heart: | findbugs | 1m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 56s | hbase-common in the patch passed. | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 61m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1057/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1057 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 5f9ca075e942 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1057/out/precommit/personality/provided.sh | | git revision | master / c4395b52c9 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1057/2/testReport/ | | Max. process+thread count | 286 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1057/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] joshelser commented on issue #1057: HBASE-23701 Try to converge automated checks around Category
joshelser commented on issue #1057: HBASE-23701 Try to converge automated checks around Category URL: https://github.com/apache/hbase/pull/1057#issuecomment-575724261 Thanks folks! Let me merge before I forget :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1048: HBASE-23694 After RegionProcedureStore completes migration of WALProcedureStore, …
saintstack commented on a change in pull request #1048: HBASE-23694 After
RegionProcedureStore completes migration of WALProcedureStore, …
URL: https://github.com/apache/hbase/pull/1048#discussion_r368051343
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStore.java
##
@@ -364,6 +364,7 @@ public void handleCorrupted(ProcedureIterator procIter)
throws IOException {
throw new IOException("Failed to delete the WALProcedureStore migrated
proc wal directory " +
procWALDir);
}
+store.stop(true);
Review comment:
Ok. Good enough for now. Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] joshelser commented on a change in pull request #1057: HBASE-23701 Try to converge automated checks around Category
joshelser commented on a change in pull request #1057: HBASE-23701 Try to
converge automated checks around Category
URL: https://github.com/apache/hbase/pull/1057#discussion_r368024179
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseClassTestRule.java
##
@@ -59,13 +59,20 @@ private HBaseClassTestRule(Class clazz, Timeout
timeout) {
private static long getTimeoutInSeconds(Class clazz) {
Category[] categories = clazz.getAnnotationsByType(Category.class);
-for (Class c : categories[0].value()) {
- if (c == SmallTests.class || c == MediumTests.class || c ==
LargeTests.class) {
-// All tests have a 13 minutes timeout.
-return TimeUnit.MINUTES.toSeconds(13);
- }
- if (c == IntegrationTests.class) {
-return TimeUnit.MINUTES.toSeconds(Long.MAX_VALUE);
+// This should never happen
+if (categories.length > 1) {
+ throw new IllegalArgumentException("Code-bug: unsure how to handle more
than one Category");
+}
+// Fail gracefully if there is no Category defined
+if (categories.length == 1) {
+ for (Class c : categories[0].value()) {
+if (c == SmallTests.class || c == MediumTests.class || c ==
LargeTests.class) {
Review comment:
I like it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] joshelser commented on a change in pull request #1057: HBASE-23701 Try to converge automated checks around Category
joshelser commented on a change in pull request #1057: HBASE-23701 Try to
converge automated checks around Category
URL: https://github.com/apache/hbase/pull/1057#discussion_r368021245
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseClassTestRule.java
##
@@ -59,13 +59,20 @@ private HBaseClassTestRule(Class clazz, Timeout
timeout) {
private static long getTimeoutInSeconds(Class clazz) {
Category[] categories = clazz.getAnnotationsByType(Category.class);
-for (Class c : categories[0].value()) {
- if (c == SmallTests.class || c == MediumTests.class || c ==
LargeTests.class) {
-// All tests have a 13 minutes timeout.
-return TimeUnit.MINUTES.toSeconds(13);
- }
- if (c == IntegrationTests.class) {
-return TimeUnit.MINUTES.toSeconds(Long.MAX_VALUE);
+// This should never happen
+if (categories.length > 1) {
+ throw new IllegalArgumentException("Code-bug: unsure how to handle more
than one Category");
Review comment:
Yeah, my comment on Jira about there never being more than one was from me
seeing this was not a Repeatable annotation. Thanks for linking the docs.
I don't like the `Preconditions.checkArgument` though, as this is
duplicating the same exception (a test category is not defined).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (HBASE-23703) Add HBase 2.2.3 documentation to website
[ https://issues.apache.org/jira/browse/HBASE-23703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-23703. --- Fix Version/s: 3.0.0 Resolution: Fixed Pushed to master. > Add HBase 2.2.3 documentation to website > > > Key: HBASE-23703 > URL: https://issues.apache.org/jira/browse/HBASE-23703 > Project: HBase > Issue Type: Improvement > Components: documentation, website >Affects Versions: 2.2.3 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0 > > > The website does not have documentation of HBase 2.2 release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] petersomogyi merged pull request #1059: HBASE-23703 Add HBase 2.2.3 documentation to website
petersomogyi merged pull request #1059: HBASE-23703 Add HBase 2.2.3 documentation to website URL: https://github.com/apache/hbase/pull/1059 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1030: HBASE-23621 Reduced the number of Checkstyle violations in tests of hbase-common
Apache-HBase commented on issue #1030: HBASE-23621 Reduced the number of Checkstyle violations in tests of hbase-common URL: https://github.com/apache/hbase/pull/1030#issuecomment-575664031 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 48s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 11 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 1m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 8m 54s | branch-1 passed | | +1 :green_heart: | compile | 1m 22s | branch-1 passed with JDK v1.8.0_242 | | +1 :green_heart: | compile | 1m 30s | branch-1 passed with JDK v1.7.0_252 | | +1 :green_heart: | checkstyle | 2m 43s | branch-1 passed | | +1 :green_heart: | shadedjars | 4m 4s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | branch-1 passed with JDK v1.8.0_242 | | +1 :green_heart: | javadoc | 1m 17s | branch-1 passed with JDK v1.7.0_252 | | +0 :ok: | spotbugs | 3m 46s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 5m 13s | branch-1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 58s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed with JDK v1.8.0_242 | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | compile | 1m 33s | the patch passed with JDK v1.7.0_252 | | +1 :green_heart: | javac | 1m 33s | the patch passed | | +1 :green_heart: | checkstyle | 0m 35s | hbase-common: The patch generated 0 new + 20 unchanged - 32 fixed = 20 total (was 52) | | +1 :green_heart: | checkstyle | 1m 59s | hbase-server: The patch generated 0 new + 330 unchanged - 1 fixed = 330 total (was 331) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 44s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 6m 23s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 1m 1s | the patch passed with JDK v1.8.0_242 | | +1 :green_heart: | javadoc | 1m 16s | the patch passed with JDK v1.7.0_252 | | +1 :green_heart: | findbugs | 4m 48s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 52s | hbase-common in the patch passed. | | -1 :x: | unit | 157m 26s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 47s | The patch does not generate ASF License warnings. | | | | 220m 57s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles | | | hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint | | | hadoop.hbase.master.TestWarmupRegion | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1030/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1030 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 40729bbdb814 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1030/out/precommit/personality/provided.sh | | git revision | branch-1 / d466f77 | | Default Java | 1.7.0_252 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_242 /usr/lib/jvm/zulu-7-amd64:1.7.0_252 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1030/6/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1030/6/testReport/ | | Max. process+thread count | 5071 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1030/6/console | | versions |
[GitHub] [hbase] Apache9 commented on issue #1022: HBASE-23680 RegionProcedureStore missing cleaning of hfile archive
Apache9 commented on issue #1022: HBASE-23680 RegionProcedureStore missing cleaning of hfile archive URL: https://github.com/apache/hbase/pull/1022#issuecomment-575662440 Ping @busbey @saintstack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23704) Use errorProne profile for pre commit verification
[ https://issues.apache.org/jira/browse/HBASE-23704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018100#comment-17018100 ] Peter Somogyi commented on HBASE-23704: --- I remember something but don't know the details. Nonetheless, it would be great to do something against this because we only notice these kinds of issues when we have a broken build. > Use errorProne profile for pre commit verification > -- > > Key: HBASE-23704 > URL: https://issues.apache.org/jira/browse/HBASE-23704 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > > Many times we ran into the problem that the pre-commit verification was > successful but after the commit landed on the branch errorProne failed in the > Nightly builds. > We should already use errorProne profile during the pre-commit builds to > prevent similar build failures. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-23653) Expose content of meta table in web ui
[
https://issues.apache.org/jira/browse/HBASE-23653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi reopened HBASE-23653:
---
Reopen for error prone fix.
> Expose content of meta table in web ui
> --
>
> Key: HBASE-23653
> URL: https://issues.apache.org/jira/browse/HBASE-23653
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, UI
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 23653.0.png, 23653.1.png
>
>
> While testing improvements to hbck2, breaking and fixing a cluster's region
> assignments, we've found that quite a few questions can only be answered by
> dropping into a shell and scanning meta. This is in contrast to the nice web
> view we have on procedures and their locks. It would be nice to have a
> similar view into the content of meta. I've put together and extension to the
> existing {{table.jsp}} that does this. It's had a couple rounds of operator
> feedback. Will post up some screenshots along with the PR.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] petersomogyi opened a new pull request #1061: HBASE-23653 Expose content of meta table in web ui; addendum
petersomogyi opened a new pull request #1061: HBASE-23653 Expose content of meta table in web ui; addendum URL: https://github.com/apache/hbase/pull/1061 Fix error prone problem This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] petersomogyi commented on issue #1060: HBASE-23347 Allow custom authentication methods for RPCs; addendum
petersomogyi commented on issue #1060: HBASE-23347 Allow custom authentication methods for RPCs; addendum URL: https://github.com/apache/hbase/pull/1060#issuecomment-575655000 > One thing - MetaBrowser wasn't changed by my commit. Looks like that might have come in via @ndimiduk 's HBASE-23653? I haven't noticed it was from a different commit. Let me remove that modification here and create an addendum for HBASE-23653 for easier tracking. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] joshelser commented on issue #1060: HBASE-23347 Allow custom authentication methods for RPCs; addendum
joshelser commented on issue #1060: HBASE-23347 Allow custom authentication methods for RPCs; addendum URL: https://github.com/apache/hbase/pull/1060#issuecomment-575653996 Looks great! Thanks for fixing, @petersomogyi . I didn't realize that errorprone wasn't run during QA! One thing - MetaBrowser wasn't changed by my commit. Looks like that might have come in via @ndimiduk 's HBASE-23653? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
