[jira] [Commented] (HBASE-20483) [PERFORMANCE] Flushing is 2x slower in hbase2.
[ https://issues.apache.org/jira/browse/HBASE-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504311#comment-16504311 ] Anoop Sam John commented on HBASE-20483: [~stack] The other jira abt SegmentScanner fixed this also right? > [PERFORMANCE] Flushing is 2x slower in hbase2. > -- > > Key: HBASE-20483 > URL: https://issues.apache.org/jira/browse/HBASE-20483 > Project: HBase > Issue Type: Sub-task > Components: Performance >Reporter: stack >Assignee: stack >Priority: Major > Attachments: > 0001-HBASE-20483-perpertual-flush-and-memstore-fill-branch-2.0.patch, > 0001-HBASE-20483-perpetual-flush.branch-1.4.patch, > 0001-HBASE-20483-perpetual-flush.branch-1.4.patch, > 0001-HBASE-20483-perpetual.tests.patch, > 0001-HBASE-20483-perpetual.tests.patch, > 0001-Perpetual-flush-memstore-fill-etc.-tools.patch, > PerpetualFlushingProgram.patch, PerpetualFlushingProgram1.2.patch, > SnapshotSegmentScanner_2.0.0.patch > > > When flush is slow, memstores backup and go slower. See "1.2.7 vs. 2.0.0 > Flush History" > https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826 > for compare. First noted by [~anoop.hbase] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18118) Default storage policy if not configured cannot be "NONE"
[ https://issues.apache.org/jira/browse/HBASE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504286#comment-16504286 ] Yu Li commented on HBASE-18118: --- Sorry but somehow missed the ping... I think the description of HBASE-20691 tells the story clearly, thanks for putting it up. [~busbey] > Default storage policy if not configured cannot be "NONE" > - > > Key: HBASE-18118 > URL: https://issues.apache.org/jira/browse/HBASE-18118 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-18118.patch > > > HBase can't use 'NONE' as default storage policy if not configured because > HDFS supports no such policy. This policy name was probably available in a > precommit or early version of the HDFS side support for heterogeneous > storage. Now the best default is 'HOT'. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20689) Docker fails to install rubocop for precommit
[
https://issues.apache.org/jira/browse/HBASE-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504260#comment-16504260
]
Hudson commented on HBASE-20689:
Results for branch branch-2
[build #832 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/832/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/832//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/832//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/832//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Docker fails to install rubocop for precommit
> -
>
> Key: HBASE-20689
> URL: https://issues.apache.org/jira/browse/HBASE-20689
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20689.master.001.patch
>
>
> Precommit started to fail with rubocop install in 'Building base image' phase.
> The used maven:3.5-jdk8 image was updated ~10h ago, probably related.
> [https://hub.docker.com/r/library/maven/]
> One of the failed jobs:
> [https://builds.apache.org/job/PreCommit-HBASE-Build/13105/console]
> {noformat}
> 14:42:23 Setting up ruby (1:2.3.3) ...
> 14:42:23 Processing triggers for libc-bin (2.24-11+deb9u3) ...
> 14:42:23 Successfully installed rake-12.3.1
> 14:42:23 Parsing documentation for rake-12.3.1
> 14:42:23 Installing ri documentation for rake-12.3.1
> 14:42:23 Done installing documentation for rake after 1 seconds
> 14:42:23 Building native extensions. This could take a while...
> 14:42:23 [91mERROR: Error installing rubocop:
> 14:42:23 ERROR: Failed to build gem native extension.
> 14:42:23
> 14:42:23 current directory:
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0/ext/jaro_winkler
> 14:42:23 /usr/bin/ruby2.3 -r ./siteconf20180606-949-99wdpz.rb extconf.rb
> 14:42:23 mkmf.rb can't find header files for ruby at
> /usr/lib/ruby/include/ruby.h
> 14:42:23
> 14:42:23 extconf failed, exit code 1
> 14:42:23
> 14:42:23 Gem files will remain installed in
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0 for inspection.
> 14:42:23 Results logged to
> /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/jaro_winkler-1.4.0/gem_make.out
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20665) "Already cached block XXX" message should be DEBUG
[
https://issues.apache.org/jira/browse/HBASE-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504242#comment-16504242
]
Hadoop QA commented on HBASE-20665:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
37s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
19s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 51s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}112m
38s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}156m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20665 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926825/HBASE-20665.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 9bba1f105544 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b002b0b8b0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13126/testReport/ |
| Max. process+thread count | 4370 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13126/console |
|
[jira] [Commented] (HBASE-20686) Asyncfs should retry upon RetryStartFileException
[
https://issues.apache.org/jira/browse/HBASE-20686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504239#comment-16504239
]
Chia-Ping Tsai commented on HBASE-20686:
{quote}IMHO async client should be consistent with DFSOutputStream behavior.
I.e. if DFSOutputStream unwraps AccessControlException, async client should
unwrap it too.
{quote}
It is good if the consistent code can bring any benefit. Or add comment to
explain why we have to look up other exceptions.
> Asyncfs should retry upon RetryStartFileException
> -
>
> Key: HBASE-20686
> URL: https://issues.apache.org/jira/browse/HBASE-20686
> Project: HBase
> Issue Type: Bug
> Components: asyncclient
>Affects Versions: 2.0.0-beta-1
> Environment: HBase 2.0, Hadoop 3 with at-rest encryption
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-20686.master.001.patch
>
>
> In Hadoop-2.6 and above, HDFS client retries on RetryStartFileException when
> NameNode experience encryption zone related issue. The code exists in
> DFSOutputStream#newStreamForCreate(). (HDFS-6970)
> In HBase-2's asyncfs implementation,
> FanOutOneBlockAsyncDFSOutputHelper#createOutput() is somewhat an imitation of
> HDFS's DFSOutputStream#newStreamForCreate(). However it does not retry upon
> RetryStartFileException. So it is less resilient to such issues.
> Also, DFSOutputStream#newStreamForCreate() upwraps RemoteExceptions, but
> asyncfs does not. Therefore, hbase gets different exceptions than before.
> File this jira to get this corrected.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20695) Implement table level RegionServer replication metrics
[
https://issues.apache.org/jira/browse/HBASE-20695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504236#comment-16504236
]
Hadoop QA commented on HBASE-20695:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 52s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}149m
33s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}189m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20695 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926821/HBASE-20695.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 742adbb0bd9c 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11
12:16:42 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b002b0b8b0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13123/testReport/ |
| Max. process+thread count | 4544 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13123/console |
[jira] [Commented] (HBASE-20611) UnsupportedOperationException may thrown when calling getCallQueueInfo()
[
https://issues.apache.org/jira/browse/HBASE-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504220#comment-16504220
]
Hadoop QA commented on HBASE-20611:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
7s{color} | {color:red} hbase-server: The patch generated 2 new + 24 unchanged
- 0 fixed = 26 total (was 24) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 49s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}107m
34s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}147m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20611 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926822/HBASE-20611.master.007.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 905162f59694 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b002b0b8b0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/13124/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13124/testReport/ |
| Max. process+thread count | 4064 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
ht
[jira] [Commented] (HBASE-20686) Asyncfs should retry upon RetryStartFileException
[ https://issues.apache.org/jira/browse/HBASE-20686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504212#comment-16504212 ] Wei-Chiu Chuang commented on HBASE-20686: - Thanks for reviewing the patch, [~chia7712]. The jira summary was misleading. The patch unwraps these exceptions because IMHO async client should be consistent with DFSOutputStream behavior. I.e. if DFSOutputStream unwraps AccessControlException, async client should unwrap it too. If you think I should descope the patch and focus on handling RetryStartFileException, I can attach a patch as well. Thank you. > Asyncfs should retry upon RetryStartFileException > - > > Key: HBASE-20686 > URL: https://issues.apache.org/jira/browse/HBASE-20686 > Project: HBase > Issue Type: Bug > Components: asyncclient >Affects Versions: 2.0.0-beta-1 > Environment: HBase 2.0, Hadoop 3 with at-rest encryption >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Attachments: HBASE-20686.master.001.patch > > > In Hadoop-2.6 and above, HDFS client retries on RetryStartFileException when > NameNode experience encryption zone related issue. The code exists in > DFSOutputStream#newStreamForCreate(). (HDFS-6970) > In HBase-2's asyncfs implementation, > FanOutOneBlockAsyncDFSOutputHelper#createOutput() is somewhat an imitation of > HDFS's DFSOutputStream#newStreamForCreate(). However it does not retry upon > RetryStartFileException. So it is less resilient to such issues. > Also, DFSOutputStream#newStreamForCreate() upwraps RemoteExceptions, but > asyncfs does not. Therefore, hbase gets different exceptions than before. > File this jira to get this corrected. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20689) Docker fails to install rubocop for precommit
[
https://issues.apache.org/jira/browse/HBASE-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504210#comment-16504210
]
Hudson commented on HBASE-20689:
Results for branch branch-2.0
[build #397 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/397/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/397//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/397//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/397//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Docker fails to install rubocop for precommit
> -
>
> Key: HBASE-20689
> URL: https://issues.apache.org/jira/browse/HBASE-20689
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20689.master.001.patch
>
>
> Precommit started to fail with rubocop install in 'Building base image' phase.
> The used maven:3.5-jdk8 image was updated ~10h ago, probably related.
> [https://hub.docker.com/r/library/maven/]
> One of the failed jobs:
> [https://builds.apache.org/job/PreCommit-HBASE-Build/13105/console]
> {noformat}
> 14:42:23 Setting up ruby (1:2.3.3) ...
> 14:42:23 Processing triggers for libc-bin (2.24-11+deb9u3) ...
> 14:42:23 Successfully installed rake-12.3.1
> 14:42:23 Parsing documentation for rake-12.3.1
> 14:42:23 Installing ri documentation for rake-12.3.1
> 14:42:23 Done installing documentation for rake after 1 seconds
> 14:42:23 Building native extensions. This could take a while...
> 14:42:23 [91mERROR: Error installing rubocop:
> 14:42:23 ERROR: Failed to build gem native extension.
> 14:42:23
> 14:42:23 current directory:
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0/ext/jaro_winkler
> 14:42:23 /usr/bin/ruby2.3 -r ./siteconf20180606-949-99wdpz.rb extconf.rb
> 14:42:23 mkmf.rb can't find header files for ruby at
> /usr/lib/ruby/include/ruby.h
> 14:42:23
> 14:42:23 extconf failed, exit code 1
> 14:42:23
> 14:42:23 Gem files will remain installed in
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0 for inspection.
> 14:42:23 Results logged to
> /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/jaro_winkler-1.4.0/gem_make.out
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504193#comment-16504193
]
stack commented on HBASE-20671:
---
Will I commit this in a subissue as a workaround?
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments: 0001-Test-for-HBASE-20671.patch,
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip,
> workaround.txt
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19453) Use maven filtering instead of ant replace task for project version in hbase-defaults.xml
[
https://issues.apache.org/jira/browse/HBASE-19453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504188#comment-16504188
]
Hadoop QA commented on HBASE-19453:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
37s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
31s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 4m
58s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 57s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
23s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-19453 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926763/HBASE-19453.master.003.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile refguide |
| uname | Linux a4723c5df66e 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / b002b0b8b0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/13127/artifact/patchprocess/branch-site/book.html
|
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/13127/artifact/patchprocess/patch-site/book.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13127/testReport/ |
| Max. process+thread count | 284 (vs. ulimit of 1) |
| modules
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504182#comment-16504182
]
Hadoop QA commented on HBASE-20671:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
41s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 53s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}109m
11s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}149m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20671 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926808/workaround.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux bd079b4d7ac9 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / b002b0b8b0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
htt
[jira] [Commented] (HBASE-20611) UnsupportedOperationException may thrown when calling getCallQueueInfo()
[
https://issues.apache.org/jira/browse/HBASE-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504180#comment-16504180
]
Hadoop QA commented on HBASE-20611:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
12s{color} | {color:red} hbase-server: The patch generated 1 new + 24 unchanged
- 0 fixed = 25 total (was 24) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 56s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}113m 30s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}155m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20611 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926817/HBASE-20611.master.006.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux a81f37383589 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b002b0b8b0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/13121/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/13121/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13121/testReport/ |
| Max. process
[jira] [Commented] (HBASE-19064) Synchronous replication for HBase
[
https://issues.apache.org/jira/browse/HBASE-19064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504179#comment-16504179
]
Hudson commented on HBASE-19064:
Results for branch HBASE-19064
[build #154 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/154/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/154//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/154//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/154//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Synchronous replication for HBase
> -
>
> Key: HBASE-19064
> URL: https://issues.apache.org/jira/browse/HBASE-19064
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> The guys from Alibaba made a presentation on HBaseCon Asia about the
> synchronous replication for HBase. We(Xiaomi) think this is a very useful
> feature for HBase so we want to bring it into the community version.
> This is a big feature so we plan to do it in a feature branch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20611) UnsupportedOperationException may thrown when calling getCallQueueInfo()
[
https://issues.apache.org/jira/browse/HBASE-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504162#comment-16504162
]
Chia-Ping Tsai edited comment on HBASE-20611 at 6/7/18 2:24 AM:
{code:java}
+ if (lastModifyCount != modifyCount) { // structure has been modified
since #iterator called
+throw new ConcurrentModificationException();
+ }{code}
The queue is changed frequently so the error may happen frequently also. Could
we do the snapshot of objects? If it is impossible to do the snapshot, we have
to handle the error on RSDumpServlet#dumpCallQueues. Otherwise, user will be
scared by our web UI.
{code:java}
+ @Override
+ public Iterator iterator(){
+return new AdaptiveLifoCoDelCallQueue.Itr();
+ }
+
+ private final class Itr implements Iterator {
+private int index = 0;
+private Object[] crArray = AdaptiveLifoCoDelCallQueue.this.queue.toArray();
+int size = crArray.length;
+public boolean hasNext() {
+ return index < size;
+}
+
+@SuppressWarnings("unchecked")
+public CallRunner next() {
+ if (index < size) {
+return (CallRunner) crArray[index++];
+ }
+ throw new NoSuchElementException();
+}
+ }{code}
BTW, if the Itr is not fat, we can use anonymous class instead. Also, please
make the fields final as much as possible.
was (Author: chia7712):
{code:java}
+ if (lastModifyCount != modifyCount) { // structure has been modified
since #iterator called
+throw new ConcurrentModificationException();
+ }{code}
The queue is changed frequently so the error may happen frequently also. Could
we do the snapshot of objects? If it is impossible to do the snapshot, we have
to handle the error on RSDumpServlet#dumpCallQueues. Otherwise, user will be
scared by our web UI.
{code:java}
+ @Override
+ public Iterator iterator(){
+return new AdaptiveLifoCoDelCallQueue.Itr();
+ }
+
+ private final class Itr implements Iterator {
+private int index = 0;
+private Object[] crArray = AdaptiveLifoCoDelCallQueue.this.queue.toArray();
+int size = crArray.length;
+public boolean hasNext() {
+ return index < size;
+}
+
+@SuppressWarnings("unchecked")
+public CallRunner next() {
+ if (index < size) {
+return (CallRunner) crArray[index++];
+ }
+ throw new NoSuchElementException();
+}
+ }{code}
BTW, if the Itr is not fat, we can use inner class instead. Also, please make
the fields final as much as possible.
> UnsupportedOperationException may thrown when calling getCallQueueInfo()
>
>
> Key: HBASE-20611
> URL: https://issues.apache.org/jira/browse/HBASE-20611
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20611.master.002.patch,
> HBASE-20611.master.003.patch, HBASE-20611.master.004.patch,
> HBASE-20611.master.005.patch, HBASE-20611.master.006.patch,
> HBASE-20611.master.007.patch
>
>
> HBASE-16290 added a new feature to dump queue info, the method
> getCallQueueInfo() need to iterate the queue to get the elements in the
> queue. But, except the Java's LinkedBlockingQueue, the other queue
> implementations like BoundedPriorityBlockingQueue and
> AdaptiveLifoCoDelCallQueue don't implement the method iterator(). If those
> queues are used, a UnsupportedOperationException will be thrown.
> This can be easily be reproduced by the UT testCallQueueInfo while adding a
> conf: conf.set("hbase.ipc.server.callqueue.type", "deadline")
> {code}
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.iterator(BoundedPriorityBlockingQueue.java:285)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.getCallQueueCountsSummary(RpcExecutor.java:166)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.getCallQueueInfo(SimpleRpcScheduler.java:241)
> at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testCallQueueInfo(TestSimpleRpcScheduler.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20611) UnsupportedOperationException may thrown when calling getCallQueueInfo()
[
https://issues.apache.org/jira/browse/HBASE-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504162#comment-16504162
]
Chia-Ping Tsai commented on HBASE-20611:
{code:java}
+ if (lastModifyCount != modifyCount) { // structure has been modified
since #iterator called
+throw new ConcurrentModificationException();
+ }{code}
The queue is changed frequently so the error may happen frequently also. Could
we do the snapshot of objects? If it is impossible to do the snapshot, we have
to handle the error on RSDumpServlet#dumpCallQueues. Otherwise, user will be
scared by our web UI.
{code:java}
+ @Override
+ public Iterator iterator(){
+return new AdaptiveLifoCoDelCallQueue.Itr();
+ }
+
+ private final class Itr implements Iterator {
+private int index = 0;
+private Object[] crArray = AdaptiveLifoCoDelCallQueue.this.queue.toArray();
+int size = crArray.length;
+public boolean hasNext() {
+ return index < size;
+}
+
+@SuppressWarnings("unchecked")
+public CallRunner next() {
+ if (index < size) {
+return (CallRunner) crArray[index++];
+ }
+ throw new NoSuchElementException();
+}
+ }{code}
BTW, if the Itr is not fat, we can use inner class instead. Also, please make
the fields final as much as possible.
> UnsupportedOperationException may thrown when calling getCallQueueInfo()
>
>
> Key: HBASE-20611
> URL: https://issues.apache.org/jira/browse/HBASE-20611
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20611.master.002.patch,
> HBASE-20611.master.003.patch, HBASE-20611.master.004.patch,
> HBASE-20611.master.005.patch, HBASE-20611.master.006.patch,
> HBASE-20611.master.007.patch
>
>
> HBASE-16290 added a new feature to dump queue info, the method
> getCallQueueInfo() need to iterate the queue to get the elements in the
> queue. But, except the Java's LinkedBlockingQueue, the other queue
> implementations like BoundedPriorityBlockingQueue and
> AdaptiveLifoCoDelCallQueue don't implement the method iterator(). If those
> queues are used, a UnsupportedOperationException will be thrown.
> This can be easily be reproduced by the UT testCallQueueInfo while adding a
> conf: conf.set("hbase.ipc.server.callqueue.type", "deadline")
> {code}
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.iterator(BoundedPriorityBlockingQueue.java:285)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.getCallQueueCountsSummary(RpcExecutor.java:166)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.getCallQueueInfo(SimpleRpcScheduler.java:241)
> at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testCallQueueInfo(TestSimpleRpcScheduler.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20631) B&R: Merge command enhancements
[
https://issues.apache.org/jira/browse/HBASE-20631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504161#comment-16504161
]
Hadoop QA commented on HBASE-20631:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-20631 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20631 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926824/HBASE-20631-v1.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13125/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> B&R: Merge command enhancements
>
>
> Key: HBASE-20631
> URL: https://issues.apache.org/jira/browse/HBASE-20631
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-20631-v1.patch
>
>
> Currently, merge supports only list of backup ids, which users must provide.
> Date range merges seem more convenient for users.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20665) "Already cached block XXX" message should be DEBUG
[
https://issues.apache.org/jira/browse/HBASE-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504159#comment-16504159
]
Josh Elser commented on HBASE-20665:
Reattaching – QA should be good now.
> "Already cached block XXX" message should be DEBUG
> --
>
> Key: HBASE-20665
> URL: https://issues.apache.org/jira/browse/HBASE-20665
> Project: HBase
> Issue Type: Task
> Components: BlockCache
>Affects Versions: 1.2.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Eric Maynard
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.5
>
> Attachments: HBASE-20665.master.001.patch,
> HBASE-20665.master.001.patch
>
>
> Testing a local cluster that relies on the LruBlockCache for a scan-heavy
> workload and I'm getting a bunch of log entries at WARN
> {code}
> 2018-05-30 12:28:47,192 WARN org.apache.hadoop.hbase.io.hfile.LruBlockCache:
> Cached an already cached block: df01f5bf6a244f6bb1a626b927377fff_54780812
> cb:df01f5bf6a244f6bb1a626b927377fff_54780812. This is harmless and can happen
> in rare cases (see HBASE-8547)
> {code}
> As the log message notes (and the code confirms) this is a harmless result of
> contention for getting a block into the CHM. the message should be at DEBUG.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20665) "Already cached block XXX" message should be DEBUG
[
https://issues.apache.org/jira/browse/HBASE-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20665:
---
Attachment: HBASE-20665.master.001.patch
> "Already cached block XXX" message should be DEBUG
> --
>
> Key: HBASE-20665
> URL: https://issues.apache.org/jira/browse/HBASE-20665
> Project: HBase
> Issue Type: Task
> Components: BlockCache
>Affects Versions: 1.2.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Eric Maynard
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.5
>
> Attachments: HBASE-20665.master.001.patch,
> HBASE-20665.master.001.patch
>
>
> Testing a local cluster that relies on the LruBlockCache for a scan-heavy
> workload and I'm getting a bunch of log entries at WARN
> {code}
> 2018-05-30 12:28:47,192 WARN org.apache.hadoop.hbase.io.hfile.LruBlockCache:
> Cached an already cached block: df01f5bf6a244f6bb1a626b927377fff_54780812
> cb:df01f5bf6a244f6bb1a626b927377fff_54780812. This is harmless and can happen
> in rare cases (see HBASE-8547)
> {code}
> As the log message notes (and the code confirms) this is a harmless result of
> contention for getting a block into the CHM. the message should be at DEBUG.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504155#comment-16504155
]
Josh Elser commented on HBASE-20671:
+1 to your extra debugging. That will pay for itself in spades!
I was going to suggest that you include the {{regionInfo}} in the log message,
but I see that it will already be there from RegionStateStore.
Let me get QA going.
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments: 0001-Test-for-HBASE-20671.patch,
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip,
> workaround.txt
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20631) B&R: Merge command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-20631: -- Status: Patch Available (was: Open) Patch v1. cc: [~te...@apache.org] > B&R: Merge command enhancements > > > Key: HBASE-20631 > URL: https://issues.apache.org/jira/browse/HBASE-20631 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-20631-v1.patch > > > Currently, merge supports only list of backup ids, which users must provide. > Date range merges seem more convenient for users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20631) B&R: Merge command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-20631: -- Attachment: HBASE-20631-v1.patch > B&R: Merge command enhancements > > > Key: HBASE-20631 > URL: https://issues.apache.org/jira/browse/HBASE-20631 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-20631-v1.patch > > > Currently, merge supports only list of backup ids, which users must provide. > Date range merges seem more convenient for users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20611) UnsupportedOperationException may thrown when calling getCallQueueInfo()
[
https://issues.apache.org/jira/browse/HBASE-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20611:
Attachment: HBASE-20611.master.007.patch
> UnsupportedOperationException may thrown when calling getCallQueueInfo()
>
>
> Key: HBASE-20611
> URL: https://issues.apache.org/jira/browse/HBASE-20611
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20611.master.002.patch,
> HBASE-20611.master.003.patch, HBASE-20611.master.004.patch,
> HBASE-20611.master.005.patch, HBASE-20611.master.006.patch,
> HBASE-20611.master.007.patch
>
>
> HBASE-16290 added a new feature to dump queue info, the method
> getCallQueueInfo() need to iterate the queue to get the elements in the
> queue. But, except the Java's LinkedBlockingQueue, the other queue
> implementations like BoundedPriorityBlockingQueue and
> AdaptiveLifoCoDelCallQueue don't implement the method iterator(). If those
> queues are used, a UnsupportedOperationException will be thrown.
> This can be easily be reproduced by the UT testCallQueueInfo while adding a
> conf: conf.set("hbase.ipc.server.callqueue.type", "deadline")
> {code}
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.iterator(BoundedPriorityBlockingQueue.java:285)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.getCallQueueCountsSummary(RpcExecutor.java:166)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.getCallQueueInfo(SimpleRpcScheduler.java:241)
> at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testCallQueueInfo(TestSimpleRpcScheduler.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20611) UnsupportedOperationException may thrown when calling getCallQueueInfo()
[
https://issues.apache.org/jira/browse/HBASE-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504137#comment-16504137
]
Xu Cang commented on HBASE-20611:
-
You are right, Ted.
Let me remove those.
> UnsupportedOperationException may thrown when calling getCallQueueInfo()
>
>
> Key: HBASE-20611
> URL: https://issues.apache.org/jira/browse/HBASE-20611
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20611.master.002.patch,
> HBASE-20611.master.003.patch, HBASE-20611.master.004.patch,
> HBASE-20611.master.005.patch, HBASE-20611.master.006.patch
>
>
> HBASE-16290 added a new feature to dump queue info, the method
> getCallQueueInfo() need to iterate the queue to get the elements in the
> queue. But, except the Java's LinkedBlockingQueue, the other queue
> implementations like BoundedPriorityBlockingQueue and
> AdaptiveLifoCoDelCallQueue don't implement the method iterator(). If those
> queues are used, a UnsupportedOperationException will be thrown.
> This can be easily be reproduced by the UT testCallQueueInfo while adding a
> conf: conf.set("hbase.ipc.server.callqueue.type", "deadline")
> {code}
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.iterator(BoundedPriorityBlockingQueue.java:285)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.getCallQueueCountsSummary(RpcExecutor.java:166)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.getCallQueueInfo(SimpleRpcScheduler.java:241)
> at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testCallQueueInfo(TestSimpleRpcScheduler.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20695) Implement table level RegionServer replication metrics
[ https://issues.apache.org/jira/browse/HBASE-20695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-20695: Attachment: HBASE-20695.master.002.patch > Implement table level RegionServer replication metrics > --- > > Key: HBASE-20695 > URL: https://issues.apache.org/jira/browse/HBASE-20695 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Xu Cang >Assignee: Xu Cang >Priority: Minor > Attachments: HBASE-20695.master.001.patch, > HBASE-20695.master.002.patch > > > Region server metrics now are mainly global metrics. It would be nice to have > table level metrics such as table level source.AgeOfLastShippedOp to indicate > operators which table's replication is lagging behind. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20695) Implement table level RegionServer replication metrics
[
https://issues.apache.org/jira/browse/HBASE-20695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504132#comment-16504132
]
Hadoop QA commented on HBASE-20695:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
6s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
57s{color} | {color:red} hbase-server: The patch generated 1 new + 2 unchanged
- 0 fixed = 3 total (was 2) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
18s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 56s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}109m
16s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}148m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20695 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926807/HBASE-20695.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7241328628de 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b002b0b8b0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/13118/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13118/testReport/ |
| Ma
[jira] [Commented] (HBASE-19967) Add Major Compaction Tool options for off-peak / on-peak hours
[ https://issues.apache.org/jira/browse/HBASE-19967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504131#comment-16504131 ] Xu Cang commented on HBASE-19967: - [~apurtell] I am trying to understand your description better. Are you suggesting by defining off-peak range, this command line program can take w sets of arguments, and it makes the decision which set to use based on time? Or are you suggesting to hook this configuration to PressureAwareCompactionController to control compaction's throughput? Thanks, Xu > Add Major Compaction Tool options for off-peak / on-peak hours > -- > > Key: HBASE-19967 > URL: https://issues.apache.org/jira/browse/HBASE-19967 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Priority: Major > > After HBASE-19528 an operator can disable automatic major compaction and more > intelligently manage major compaction impact on cluster operations with an > external tool that drives the compaction activity. This tool can be invoked > at whatever interval is desirable, and can restrict activity by table and > column family, with given concurrency of regionservers compacting at a given > time. > Add Major Compaction Tool options for off-peak / on-peak hours. Allow for > definition of "off-peak" as a time range bounded by two points on a 24 hour > clock, and for two concurrency target for global compaction activity, one for > the off-peak interval, the other for the remainder. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20692) Help Phoenix better identify itself when something goes wrong on a cluster
[ https://issues.apache.org/jira/browse/HBASE-20692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504100#comment-16504100 ] Sean Busbey commented on HBASE-20692: - Putting "Phoenix" into every class name would allow for circumventing package-protected access while making it at least a little more likely folks realize a CNFE is a phoenix issue. Maybe not an unreasonable request? It would mean in the above the class would be named something like "org.apache.hadoop.hbase.regionserver.wal.PhoenixIndexedWALEditCodec" > Help Phoenix better identify itself when something goes wrong on a cluster > -- > > Key: HBASE-20692 > URL: https://issues.apache.org/jira/browse/HBASE-20692 > Project: HBase > Issue Type: Wish > Components: community >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Priority: Major > > User list question about a failing 2.0.0 test cluster came in[1], turned out > to be Phoenix installed but missing from the classpath. Problem area wasn't > immediately obvious because the ClassNotFoundException appeared to be for an > HBase class: org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec. > Checking on recent-ish phoenix, they have a bunch of classes in > org.apache.hadoop.hbase[2]. Presumably to circumvent package-protected access > restrictions, but I didn't check. > Should come up with some ways to make it more obvious when our shared users > need to talk with them. this jira's a place to collect some suggestions once > folks have a chance to see why phoenix is in our package space in the first > place. Once we have something concrete to propose we can take the suggestions > over to dev@phoenix. > [1]: https://s.apache.org/oqfr > [2]: > https://github.com/apache/phoenix/tree/v5.0.0-alpha-HBase-2.0/phoenix-core/src/main/java/org/apache/hadoop/hbase -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[
https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504096#comment-16504096
]
Hadoop QA commented on HBASE-20630:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} hbase-backup: The patch generated 2 new + 0 unchanged
- 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 52s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
34s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20630 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926815/HBASE-20630-v4.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c6aaf4c2b950 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b002b0b8b0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/13120/artifact/patchprocess/diff-checkstyle-hbase-backup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13120/testReport/ |
| Max. process+thread count | 4008 (vs. ulimit of 1) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://builds
[jira] [Commented] (HBASE-20611) UnsupportedOperationException may thrown when calling getCallQueueInfo()
[
https://issues.apache.org/jira/browse/HBASE-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504094#comment-16504094
]
Ted Yu commented on HBASE-20611:
[~chia7712]:
Can you take a look too ?
> UnsupportedOperationException may thrown when calling getCallQueueInfo()
>
>
> Key: HBASE-20611
> URL: https://issues.apache.org/jira/browse/HBASE-20611
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20611.master.002.patch,
> HBASE-20611.master.003.patch, HBASE-20611.master.004.patch,
> HBASE-20611.master.005.patch, HBASE-20611.master.006.patch
>
>
> HBASE-16290 added a new feature to dump queue info, the method
> getCallQueueInfo() need to iterate the queue to get the elements in the
> queue. But, except the Java's LinkedBlockingQueue, the other queue
> implementations like BoundedPriorityBlockingQueue and
> AdaptiveLifoCoDelCallQueue don't implement the method iterator(). If those
> queues are used, a UnsupportedOperationException will be thrown.
> This can be easily be reproduced by the UT testCallQueueInfo while adding a
> conf: conf.set("hbase.ipc.server.callqueue.type", "deadline")
> {code}
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.iterator(BoundedPriorityBlockingQueue.java:285)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.getCallQueueCountsSummary(RpcExecutor.java:166)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.getCallQueueInfo(SimpleRpcScheduler.java:241)
> at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testCallQueueInfo(TestSimpleRpcScheduler.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20611) UnsupportedOperationException may thrown when calling getCallQueueInfo()
[
https://issues.apache.org/jira/browse/HBASE-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504093#comment-16504093
]
Ted Yu commented on HBASE-20611:
One previous comment which you didn't answer:
{code}
+assertEquals(false, match);
+assertNotEquals(testList.size(), testIteratorList.size());
{code}
It doesn't seem the above assertions would trigger
ConcurrentModificationException. If ConcurrentModificationException is thrown
earlier in the test, what's the meaning of adding the assertions ?
> UnsupportedOperationException may thrown when calling getCallQueueInfo()
>
>
> Key: HBASE-20611
> URL: https://issues.apache.org/jira/browse/HBASE-20611
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20611.master.002.patch,
> HBASE-20611.master.003.patch, HBASE-20611.master.004.patch,
> HBASE-20611.master.005.patch, HBASE-20611.master.006.patch
>
>
> HBASE-16290 added a new feature to dump queue info, the method
> getCallQueueInfo() need to iterate the queue to get the elements in the
> queue. But, except the Java's LinkedBlockingQueue, the other queue
> implementations like BoundedPriorityBlockingQueue and
> AdaptiveLifoCoDelCallQueue don't implement the method iterator(). If those
> queues are used, a UnsupportedOperationException will be thrown.
> This can be easily be reproduced by the UT testCallQueueInfo while adding a
> conf: conf.set("hbase.ipc.server.callqueue.type", "deadline")
> {code}
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.iterator(BoundedPriorityBlockingQueue.java:285)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.getCallQueueCountsSummary(RpcExecutor.java:166)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.getCallQueueInfo(SimpleRpcScheduler.java:241)
> at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testCallQueueInfo(TestSimpleRpcScheduler.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20671:
--
Status: Patch Available (was: Open)
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments: 0001-Test-for-HBASE-20671.patch,
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip,
> workaround.txt
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20611) UnsupportedOperationException may thrown when calling getCallQueueInfo()
[
https://issues.apache.org/jira/browse/HBASE-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20611:
Attachment: HBASE-20611.master.006.patch
> UnsupportedOperationException may thrown when calling getCallQueueInfo()
>
>
> Key: HBASE-20611
> URL: https://issues.apache.org/jira/browse/HBASE-20611
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20611.master.002.patch,
> HBASE-20611.master.003.patch, HBASE-20611.master.004.patch,
> HBASE-20611.master.005.patch, HBASE-20611.master.006.patch
>
>
> HBASE-16290 added a new feature to dump queue info, the method
> getCallQueueInfo() need to iterate the queue to get the elements in the
> queue. But, except the Java's LinkedBlockingQueue, the other queue
> implementations like BoundedPriorityBlockingQueue and
> AdaptiveLifoCoDelCallQueue don't implement the method iterator(). If those
> queues are used, a UnsupportedOperationException will be thrown.
> This can be easily be reproduced by the UT testCallQueueInfo while adding a
> conf: conf.set("hbase.ipc.server.callqueue.type", "deadline")
> {code}
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.iterator(BoundedPriorityBlockingQueue.java:285)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.getCallQueueCountsSummary(RpcExecutor.java:166)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.getCallQueueInfo(SimpleRpcScheduler.java:241)
> at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testCallQueueInfo(TestSimpleRpcScheduler.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20611) UnsupportedOperationException may thrown when calling getCallQueueInfo()
[
https://issues.apache.org/jira/browse/HBASE-20611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504069#comment-16504069
]
Xu Cang commented on HBASE-20611:
-
{code:java}
+ * Use poll() whenever possible.
{code}
Can you clarify the above - poll is not part of Iterator?
Xu: I removed this line and added some other explanations.
This 'AdaptiveLifoCoDelCallQueue" uses LinkedBlockingQueue inside. From
this class, the code cannot access LinkedBlockingQueue's internal data
structure. So I cannot implement the same algorithm as I did for
BoundedPriorityBlockingQueue.
Whish results to high space complexity for this iterator(). So I just
want to alert user using this iterator with cautions.
{code:java}
+ if (index < size())
+return (E) BoundedPriorityBlockingQueue.this.queue.objects[index++];
{code}
Enclose return in pair of curlies.
{code:java}
+ private final class Itr implements Iterator {
+private int index = 0;
+private Object[] crArray = AdaptiveLifoCoDelCallQueue.this.queue.toArray();
{code}
Why modification count is not involved in the above Iterator ?
Xu: Same reason as above. The iterator logic for AdaptiveCoDel queue is
different than the one I implemented for BOundedPriorityBlockingQueue. Because
the latter one I have access to its internal data structure (a fake
PriorityQueue defined as an inner class).
I uploaded a new patch which addresses your code reviews. Thanks again.
> UnsupportedOperationException may thrown when calling getCallQueueInfo()
>
>
> Key: HBASE-20611
> URL: https://issues.apache.org/jira/browse/HBASE-20611
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Xu Cang
>Priority: Major
> Attachments: HBASE-20611.master.002.patch,
> HBASE-20611.master.003.patch, HBASE-20611.master.004.patch,
> HBASE-20611.master.005.patch
>
>
> HBASE-16290 added a new feature to dump queue info, the method
> getCallQueueInfo() need to iterate the queue to get the elements in the
> queue. But, except the Java's LinkedBlockingQueue, the other queue
> implementations like BoundedPriorityBlockingQueue and
> AdaptiveLifoCoDelCallQueue don't implement the method iterator(). If those
> queues are used, a UnsupportedOperationException will be thrown.
> This can be easily be reproduced by the UT testCallQueueInfo while adding a
> conf: conf.set("hbase.ipc.server.callqueue.type", "deadline")
> {code}
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.iterator(BoundedPriorityBlockingQueue.java:285)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.getCallQueueCountsSummary(RpcExecutor.java:166)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.getCallQueueInfo(SimpleRpcScheduler.java:241)
> at
> org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testCallQueueInfo(TestSimpleRpcScheduler.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504072#comment-16504072 ] Ted Yu commented on HBASE-20630: Should be good, after checkstyle warnings are fixed. > B&R: Delete command enhancements > > > Key: HBASE-20630 > URL: https://issues.apache.org/jira/browse/HBASE-20630 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch, > HBASE-20630-v3.patch, HBASE-20630-v4.patch > > > Make the command more useable. Currently, user needs to provide list of > backup ids to delete. It would be nice to have more convenient options, such > as: deleting all backups which are older than XXX days, etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20340) potential null pointer exception in org.apache.hadoop.hbase.rest.TableScanResource given empty resource
[
https://issues.apache.org/jira/browse/HBASE-20340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504068#comment-16504068
]
Hadoop QA commented on HBASE-20340:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
55s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 56s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
0s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20340 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926810/HBASE-20340-master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux afda75987f4f 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b002b0b8b0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13119/testReport/ |
| Max. process+thread count | 2440 (vs. ulimit of 1) |
| modules | C: hbase-rest U: hbase-rest |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/13119/console |
| Pow
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504061#comment-16504061 ] Vladimir Rodionov commented on HBASE-20630: --- Patch v4. > B&R: Delete command enhancements > > > Key: HBASE-20630 > URL: https://issues.apache.org/jira/browse/HBASE-20630 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch, > HBASE-20630-v3.patch, HBASE-20630-v4.patch > > > Make the command more useable. Currently, user needs to provide list of > backup ids to delete. It would be nice to have more convenient options, such > as: deleting all backups which are older than XXX days, etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20630) B&R: Delete command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-20630: -- Attachment: HBASE-20630-v4.patch > B&R: Delete command enhancements > > > Key: HBASE-20630 > URL: https://issues.apache.org/jira/browse/HBASE-20630 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch, > HBASE-20630-v3.patch, HBASE-20630-v4.patch > > > Make the command more useable. Currently, user needs to provide list of > backup ids to delete. It would be nice to have more convenient options, such > as: deleting all backups which are older than XXX days, etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20689) Docker fails to install rubocop for precommit
[
https://issues.apache.org/jira/browse/HBASE-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504050#comment-16504050
]
Josh Elser commented on HBASE-20689:
Yay, thanks!!
> Docker fails to install rubocop for precommit
> -
>
> Key: HBASE-20689
> URL: https://issues.apache.org/jira/browse/HBASE-20689
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20689.master.001.patch
>
>
> Precommit started to fail with rubocop install in 'Building base image' phase.
> The used maven:3.5-jdk8 image was updated ~10h ago, probably related.
> [https://hub.docker.com/r/library/maven/]
> One of the failed jobs:
> [https://builds.apache.org/job/PreCommit-HBASE-Build/13105/console]
> {noformat}
> 14:42:23 Setting up ruby (1:2.3.3) ...
> 14:42:23 Processing triggers for libc-bin (2.24-11+deb9u3) ...
> 14:42:23 Successfully installed rake-12.3.1
> 14:42:23 Parsing documentation for rake-12.3.1
> 14:42:23 Installing ri documentation for rake-12.3.1
> 14:42:23 Done installing documentation for rake after 1 seconds
> 14:42:23 Building native extensions. This could take a while...
> 14:42:23 [91mERROR: Error installing rubocop:
> 14:42:23 ERROR: Failed to build gem native extension.
> 14:42:23
> 14:42:23 current directory:
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0/ext/jaro_winkler
> 14:42:23 /usr/bin/ruby2.3 -r ./siteconf20180606-949-99wdpz.rb extconf.rb
> 14:42:23 mkmf.rb can't find header files for ruby at
> /usr/lib/ruby/include/ruby.h
> 14:42:23
> 14:42:23 extconf failed, exit code 1
> 14:42:23
> 14:42:23 Gem files will remain installed in
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0 for inspection.
> 14:42:23 Results logged to
> /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/jaro_winkler-1.4.0/gem_make.out
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[
https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504046#comment-16504046
]
Hadoop QA commented on HBASE-20630:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} hbase-backup: The patch generated 3 new + 0 unchanged
- 0 fixed = 3 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
54s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 18s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
21s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20630 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926801/HBASE-20630-v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c6b13ca27cac 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b002b0b8b0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/13117/artifact/patchprocess/diff-checkstyle-hbase-backup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13117/testReport/ |
| Max. process+thread count | 4237 (vs. ulimit of 1) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://builds
[jira] [Commented] (HBASE-20340) potential null pointer exception in org.apache.hadoop.hbase.rest.TableScanResource given empty resource
[
https://issues.apache.org/jira/browse/HBASE-20340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504041#comment-16504041
]
Xu Cang commented on HBASE-20340:
-
[~nihaljain.cs] thanks for the revuew!
I modified it as v2.
> potential null pointer exception in
> org.apache.hadoop.hbase.rest.TableScanResource given empty resource
> ---
>
> Key: HBASE-20340
> URL: https://issues.apache.org/jira/browse/HBASE-20340
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 2.0.0-beta-2
>Reporter: andy zhou
>Assignee: Xu Cang
>Priority: Minor
> Attachments: HBASE-20340-master.001.patch,
> HBASE-20340-master.002.patch
>
>
> Our code analyzer has detected a potential null pointer issue in
> org.apache.hadoop.hbase.rest.TableScanResource as follows:
> {code:java}
> // org.apache.hadoop.hbase.client.Result.java Line #244
> public List listCells() {
> return isEmpty()? null: Arrays.asList(rawCells());
> }
> {code}
> {code:java}
> // org.apache.hadoop.hbase.rest.TableScanResource Line #95
> List kvs = rs.listCells();
> for (Cell kv : kvs) { ... }
> {code}
>
> Given empty rs, kvs shall be null instead of an empty list, leading to
> potential null pointer exception
>
> (org.apache.hadoop.hbase.client.Result.java Line #244 and
> org.apache.hadoop.hbase.rest.TableScanResource Line #95
> Linkage to the code is here:
> https://github.com/apache/hbase/blob/9e9b347d667e1fc6165c9f8ae5ae7052147e8895/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java#L244
> SourceBrella Inc.
> https://github.com/apache/hbase/blob/9e9b347d667e1fc6165c9f8ae5ae7052147e8895/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableScanResource.java#L95
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20340) potential null pointer exception in org.apache.hadoop.hbase.rest.TableScanResource given empty resource
[
https://issues.apache.org/jira/browse/HBASE-20340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20340:
Attachment: HBASE-20340-master.002.patch
> potential null pointer exception in
> org.apache.hadoop.hbase.rest.TableScanResource given empty resource
> ---
>
> Key: HBASE-20340
> URL: https://issues.apache.org/jira/browse/HBASE-20340
> Project: HBase
> Issue Type: Bug
> Components: hbase
>Affects Versions: 2.0.0-beta-2
>Reporter: andy zhou
>Assignee: Xu Cang
>Priority: Minor
> Attachments: HBASE-20340-master.001.patch,
> HBASE-20340-master.002.patch
>
>
> Our code analyzer has detected a potential null pointer issue in
> org.apache.hadoop.hbase.rest.TableScanResource as follows:
> {code:java}
> // org.apache.hadoop.hbase.client.Result.java Line #244
> public List listCells() {
> return isEmpty()? null: Arrays.asList(rawCells());
> }
> {code}
> {code:java}
> // org.apache.hadoop.hbase.rest.TableScanResource Line #95
> List kvs = rs.listCells();
> for (Cell kv : kvs) { ... }
> {code}
>
> Given empty rs, kvs shall be null instead of an empty list, leading to
> potential null pointer exception
>
> (org.apache.hadoop.hbase.client.Result.java Line #244 and
> org.apache.hadoop.hbase.rest.TableScanResource Line #95
> Linkage to the code is here:
> https://github.com/apache/hbase/blob/9e9b347d667e1fc6165c9f8ae5ae7052147e8895/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java#L244
> SourceBrella Inc.
> https://github.com/apache/hbase/blob/9e9b347d667e1fc6165c9f8ae5ae7052147e8895/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableScanResource.java#L95
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504033#comment-16504033
]
stack commented on HBASE-20671:
---
[~elserj] I think we should commit workaround.txt. It will not load an empty
row as is happening here. It will also dump out all about the row should this
scenario ever arise again which hopefully will give us more clue as to what is
going on.
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments: 0001-Test-for-HBASE-20671.patch,
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip,
> workaround.txt
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20671:
--
Attachment: workaround.txt
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments: 0001-Test-for-HBASE-20671.patch,
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip,
> workaround.txt
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[
https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504031#comment-16504031
]
Hadoop QA commented on HBASE-20630:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
54s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} hbase-backup: The patch generated 2 new + 0 unchanged
- 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 59s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
42s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20630 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926793/HBASE-20630-v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 700f6765c896 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b002b0b8b0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/13116/artifact/patchprocess/diff-checkstyle-hbase-backup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/13116/testReport/ |
| Max. process+thread count | 4036 (vs. ulimit of 1) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://builds
[jira] [Updated] (HBASE-20695) Implement table level RegionServer replication metrics
[ https://issues.apache.org/jira/browse/HBASE-20695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-20695: Attachment: (was: HBASE-20695.master.001.patch) > Implement table level RegionServer replication metrics > --- > > Key: HBASE-20695 > URL: https://issues.apache.org/jira/browse/HBASE-20695 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Xu Cang >Assignee: Xu Cang >Priority: Minor > Attachments: HBASE-20695.master.001.patch > > > Region server metrics now are mainly global metrics. It would be nice to have > table level metrics such as table level source.AgeOfLastShippedOp to indicate > operators which table's replication is lagging behind. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20695) Implement table level RegionServer replication metrics
[ https://issues.apache.org/jira/browse/HBASE-20695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-20695: Attachment: HBASE-20695.master.001.patch Status: Patch Available (was: Open) > Implement table level RegionServer replication metrics > --- > > Key: HBASE-20695 > URL: https://issues.apache.org/jira/browse/HBASE-20695 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Xu Cang >Assignee: Xu Cang >Priority: Minor > Attachments: HBASE-20695.master.001.patch, > HBASE-20695.master.001.patch > > > Region server metrics now are mainly global metrics. It would be nice to have > table level metrics such as table level source.AgeOfLastShippedOp to indicate > operators which table's replication is lagging behind. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20695) Implement table level RegionServer replication metrics
[ https://issues.apache.org/jira/browse/HBASE-20695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-20695: Attachment: HBASE-20695.master.001.patch > Implement table level RegionServer replication metrics > --- > > Key: HBASE-20695 > URL: https://issues.apache.org/jira/browse/HBASE-20695 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Xu Cang >Assignee: Xu Cang >Priority: Minor > Attachments: HBASE-20695.master.001.patch > > > Region server metrics now are mainly global metrics. It would be nice to have > table level metrics such as table level source.AgeOfLastShippedOp to indicate > operators which table's replication is lagging behind. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[
https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504026#comment-16504026
]
Ted Yu commented on HBASE-20630:
{code}
+private void executeDeleteOlderThan(CommandLine cmdline) throws
IOException {
+ String value = cmdline.getOptionValue(OPTION_PATH);
{code}
The deletion operation shouldn't depend on Path option, right ?
It seems you use Path option for deletion. I think you meant to use Keep option.
{code}
import static
org.apache.hadoop.hbase.backup.BackupRestoreConstants.OPTION_PATH_DESC;
+import static
org.apache.hadoop.hbase.backup.BackupRestoreConstants.OPTION_PATH;
{code}
Order should be reversed.
> B&R: Delete command enhancements
>
>
> Key: HBASE-20630
> URL: https://issues.apache.org/jira/browse/HBASE-20630
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch,
> HBASE-20630-v3.patch
>
>
> Make the command more useable. Currently, user needs to provide list of
> backup ids to delete. It would be nice to have more convenient options, such
> as: deleting all backups which are older than XXX days, etc
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20695) Implement table level RegionServer replication metrics
[
https://issues.apache.org/jira/browse/HBASE-20695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504025#comment-16504025
]
Xu Cang commented on HBASE-20695:
-
Newly create metrics are something like this
}, { "name" : "Hadoop:service=HBase,name=RegionServer,sub=Replication",
"modelerType" : "RegionServer,sub=Replication", "tag.Context" : "regionserver",
"tag.Hostname" : "ip-10-1-207-149", "source.1.logEditsRead" : 4,
"source.1.logReadInBytes" : 1206, "source.MyTable.completedLogs" : 0,
"Source.MyTable.ageOfLastShippedOp_num_ops" : 1,
"Source.MyTable.ageOfLastShippedOp_min" : 170,
"Source.MyTable.ageOfLastShippedOp_max" : 170,
"Source.MyTable.ageOfLastShippedOp_mean" : 170,
"Source.MyTable.ageOfLastShippedOp_25th_percentile" : 170,
"Source.MyTable.ageOfLastShippedOp_median" : 170,
"Source.MyTable.ageOfLastShippedOp_75th_percentile" : 170,
"Source.MyTable.ageOfLastShippedOp_90th_percentile" : 170,
"Source.MyTable.ageOfLastShippedOp_95th_percentile" : 170,
"Source.MyTable.ageOfLastShippedOp_98th_percentile" : 170,
"Source.MyTable.ageOfLastShippedOp_99th_percentile" : 170,
"Source.MyTable.ageOfLastShippedOp_99.9th_percentile" : 170,
> Implement table level RegionServer replication metrics
> ---
>
> Key: HBASE-20695
> URL: https://issues.apache.org/jira/browse/HBASE-20695
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Xu Cang
>Assignee: Xu Cang
>Priority: Minor
>
> Region server metrics now are mainly global metrics. It would be nice to have
> table level metrics such as table level source.AgeOfLastShippedOp to indicate
> operators which table's replication is lagging behind.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20695) Implement table level RegionServer replication metrics
Xu Cang created HBASE-20695: --- Summary: Implement table level RegionServer replication metrics Key: HBASE-20695 URL: https://issues.apache.org/jira/browse/HBASE-20695 Project: HBase Issue Type: Improvement Components: metrics Reporter: Xu Cang Assignee: Xu Cang Region server metrics now are mainly global metrics. It would be nice to have table level metrics such as table level source.AgeOfLastShippedOp to indicate operators which table's replication is lagging behind. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504007#comment-16504007 ] Vladimir Rodionov commented on HBASE-20630: --- Patch v3. Changed -p to -k (keep last N days) > B&R: Delete command enhancements > > > Key: HBASE-20630 > URL: https://issues.apache.org/jira/browse/HBASE-20630 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch, > HBASE-20630-v3.patch > > > Make the command more useable. Currently, user needs to provide list of > backup ids to delete. It would be nice to have more convenient options, such > as: deleting all backups which are older than XXX days, etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20630) B&R: Delete command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-20630: -- Attachment: HBASE-20630-v3.patch > B&R: Delete command enhancements > > > Key: HBASE-20630 > URL: https://issues.apache.org/jira/browse/HBASE-20630 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch, > HBASE-20630-v3.patch > > > Make the command more useable. Currently, user needs to provide list of > backup ids to delete. It would be nice to have more convenient options, such > as: deleting all backups which are older than XXX days, etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20630) B&R: Delete command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-20630: -- Release Note: Added two options to Delete command: -l - list of backup ids, comma-separeted -k N - keep all backups newer than N days They are mutually exclusive > B&R: Delete command enhancements > > > Key: HBASE-20630 > URL: https://issues.apache.org/jira/browse/HBASE-20630 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch > > > Make the command more useable. Currently, user needs to provide list of > backup ids to delete. It would be nice to have more convenient options, such > as: deleting all backups which are older than XXX days, etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20631) B&R: Merge command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-20631: -- Release Note: (was: Added two options to Delete command: -l - list of backup ids, comma-separeted -p N - purge all backups older than N days They are mutually exclusive) > B&R: Merge command enhancements > > > Key: HBASE-20631 > URL: https://issues.apache.org/jira/browse/HBASE-20631 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > > Currently, merge supports only list of backup ids, which users must provide. > Date range merges seem more convenient for users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20689) Docker fails to install rubocop for precommit
[
https://issues.apache.org/jira/browse/HBASE-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503971#comment-16503971
]
Sean Busbey commented on HBASE-20689:
-
oh sweet. okay then hopefully no need to change the other branches. we can wait
for something to fail since this should be recognizable.
> Docker fails to install rubocop for precommit
> -
>
> Key: HBASE-20689
> URL: https://issues.apache.org/jira/browse/HBASE-20689
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20689.master.001.patch
>
>
> Precommit started to fail with rubocop install in 'Building base image' phase.
> The used maven:3.5-jdk8 image was updated ~10h ago, probably related.
> [https://hub.docker.com/r/library/maven/]
> One of the failed jobs:
> [https://builds.apache.org/job/PreCommit-HBASE-Build/13105/console]
> {noformat}
> 14:42:23 Setting up ruby (1:2.3.3) ...
> 14:42:23 Processing triggers for libc-bin (2.24-11+deb9u3) ...
> 14:42:23 Successfully installed rake-12.3.1
> 14:42:23 Parsing documentation for rake-12.3.1
> 14:42:23 Installing ri documentation for rake-12.3.1
> 14:42:23 Done installing documentation for rake after 1 seconds
> 14:42:23 Building native extensions. This could take a while...
> 14:42:23 [91mERROR: Error installing rubocop:
> 14:42:23 ERROR: Failed to build gem native extension.
> 14:42:23
> 14:42:23 current directory:
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0/ext/jaro_winkler
> 14:42:23 /usr/bin/ruby2.3 -r ./siteconf20180606-949-99wdpz.rb extconf.rb
> 14:42:23 mkmf.rb can't find header files for ruby at
> /usr/lib/ruby/include/ruby.h
> 14:42:23
> 14:42:23 extconf failed, exit code 1
> 14:42:23
> 14:42:23 Gem files will remain installed in
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0 for inspection.
> 14:42:23 Results logged to
> /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/jaro_winkler-1.4.0/gem_make.out
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20689) Docker fails to install rubocop for precommit
[
https://issues.apache.org/jira/browse/HBASE-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503966#comment-16503966
]
Peter Somogyi commented on HBASE-20689:
---
I just pushed it to branch-2.0+. Other branches use ubuntu:trusty base image.
Thanks for the reviews!
> Docker fails to install rubocop for precommit
> -
>
> Key: HBASE-20689
> URL: https://issues.apache.org/jira/browse/HBASE-20689
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20689.master.001.patch
>
>
> Precommit started to fail with rubocop install in 'Building base image' phase.
> The used maven:3.5-jdk8 image was updated ~10h ago, probably related.
> [https://hub.docker.com/r/library/maven/]
> One of the failed jobs:
> [https://builds.apache.org/job/PreCommit-HBASE-Build/13105/console]
> {noformat}
> 14:42:23 Setting up ruby (1:2.3.3) ...
> 14:42:23 Processing triggers for libc-bin (2.24-11+deb9u3) ...
> 14:42:23 Successfully installed rake-12.3.1
> 14:42:23 Parsing documentation for rake-12.3.1
> 14:42:23 Installing ri documentation for rake-12.3.1
> 14:42:23 Done installing documentation for rake after 1 seconds
> 14:42:23 Building native extensions. This could take a while...
> 14:42:23 [91mERROR: Error installing rubocop:
> 14:42:23 ERROR: Failed to build gem native extension.
> 14:42:23
> 14:42:23 current directory:
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0/ext/jaro_winkler
> 14:42:23 /usr/bin/ruby2.3 -r ./siteconf20180606-949-99wdpz.rb extconf.rb
> 14:42:23 mkmf.rb can't find header files for ruby at
> /usr/lib/ruby/include/ruby.h
> 14:42:23
> 14:42:23 extconf failed, exit code 1
> 14:42:23
> 14:42:23 Gem files will remain installed in
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0 for inspection.
> 14:42:23 Results logged to
> /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/jaro_winkler-1.4.0/gem_make.out
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20689) Docker fails to install rubocop for precommit
[
https://issues.apache.org/jira/browse/HBASE-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-20689:
--
Resolution: Fixed
Fix Version/s: 2.0.1
2.1.0
3.0.0
Status: Resolved (was: Patch Available)
Committed to branch-2.0+.
> Docker fails to install rubocop for precommit
> -
>
> Key: HBASE-20689
> URL: https://issues.apache.org/jira/browse/HBASE-20689
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Blocker
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20689.master.001.patch
>
>
> Precommit started to fail with rubocop install in 'Building base image' phase.
> The used maven:3.5-jdk8 image was updated ~10h ago, probably related.
> [https://hub.docker.com/r/library/maven/]
> One of the failed jobs:
> [https://builds.apache.org/job/PreCommit-HBASE-Build/13105/console]
> {noformat}
> 14:42:23 Setting up ruby (1:2.3.3) ...
> 14:42:23 Processing triggers for libc-bin (2.24-11+deb9u3) ...
> 14:42:23 Successfully installed rake-12.3.1
> 14:42:23 Parsing documentation for rake-12.3.1
> 14:42:23 Installing ri documentation for rake-12.3.1
> 14:42:23 Done installing documentation for rake after 1 seconds
> 14:42:23 Building native extensions. This could take a while...
> 14:42:23 [91mERROR: Error installing rubocop:
> 14:42:23 ERROR: Failed to build gem native extension.
> 14:42:23
> 14:42:23 current directory:
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0/ext/jaro_winkler
> 14:42:23 /usr/bin/ruby2.3 -r ./siteconf20180606-949-99wdpz.rb extconf.rb
> 14:42:23 mkmf.rb can't find header files for ruby at
> /usr/lib/ruby/include/ruby.h
> 14:42:23
> 14:42:23 extconf failed, exit code 1
> 14:42:23
> 14:42:23 Gem files will remain installed in
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0 for inspection.
> 14:42:23 Results logged to
> /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/jaro_winkler-1.4.0/gem_make.out
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20689) Docker fails to install rubocop for precommit
[
https://issues.apache.org/jira/browse/HBASE-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503964#comment-16503964
]
Sean Busbey commented on HBASE-20689:
-
lemme see if I can unwind my working directory to push this now.
> Docker fails to install rubocop for precommit
> -
>
> Key: HBASE-20689
> URL: https://issues.apache.org/jira/browse/HBASE-20689
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Blocker
> Attachments: HBASE-20689.master.001.patch
>
>
> Precommit started to fail with rubocop install in 'Building base image' phase.
> The used maven:3.5-jdk8 image was updated ~10h ago, probably related.
> [https://hub.docker.com/r/library/maven/]
> One of the failed jobs:
> [https://builds.apache.org/job/PreCommit-HBASE-Build/13105/console]
> {noformat}
> 14:42:23 Setting up ruby (1:2.3.3) ...
> 14:42:23 Processing triggers for libc-bin (2.24-11+deb9u3) ...
> 14:42:23 Successfully installed rake-12.3.1
> 14:42:23 Parsing documentation for rake-12.3.1
> 14:42:23 Installing ri documentation for rake-12.3.1
> 14:42:23 Done installing documentation for rake after 1 seconds
> 14:42:23 Building native extensions. This could take a while...
> 14:42:23 [91mERROR: Error installing rubocop:
> 14:42:23 ERROR: Failed to build gem native extension.
> 14:42:23
> 14:42:23 current directory:
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0/ext/jaro_winkler
> 14:42:23 /usr/bin/ruby2.3 -r ./siteconf20180606-949-99wdpz.rb extconf.rb
> 14:42:23 mkmf.rb can't find header files for ruby at
> /usr/lib/ruby/include/ruby.h
> 14:42:23
> 14:42:23 extconf failed, exit code 1
> 14:42:23
> 14:42:23 Gem files will remain installed in
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0 for inspection.
> 14:42:23 Results logged to
> /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/jaro_winkler-1.4.0/gem_make.out
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20689) Docker fails to install rubocop for precommit
[
https://issues.apache.org/jira/browse/HBASE-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503965#comment-16503965
]
Sean Busbey commented on HBASE-20689:
-
and yes, all branches.
> Docker fails to install rubocop for precommit
> -
>
> Key: HBASE-20689
> URL: https://issues.apache.org/jira/browse/HBASE-20689
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Blocker
> Attachments: HBASE-20689.master.001.patch
>
>
> Precommit started to fail with rubocop install in 'Building base image' phase.
> The used maven:3.5-jdk8 image was updated ~10h ago, probably related.
> [https://hub.docker.com/r/library/maven/]
> One of the failed jobs:
> [https://builds.apache.org/job/PreCommit-HBASE-Build/13105/console]
> {noformat}
> 14:42:23 Setting up ruby (1:2.3.3) ...
> 14:42:23 Processing triggers for libc-bin (2.24-11+deb9u3) ...
> 14:42:23 Successfully installed rake-12.3.1
> 14:42:23 Parsing documentation for rake-12.3.1
> 14:42:23 Installing ri documentation for rake-12.3.1
> 14:42:23 Done installing documentation for rake after 1 seconds
> 14:42:23 Building native extensions. This could take a while...
> 14:42:23 [91mERROR: Error installing rubocop:
> 14:42:23 ERROR: Failed to build gem native extension.
> 14:42:23
> 14:42:23 current directory:
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0/ext/jaro_winkler
> 14:42:23 /usr/bin/ruby2.3 -r ./siteconf20180606-949-99wdpz.rb extconf.rb
> 14:42:23 mkmf.rb can't find header files for ruby at
> /usr/lib/ruby/include/ruby.h
> 14:42:23
> 14:42:23 extconf failed, exit code 1
> 14:42:23
> 14:42:23 Gem files will remain installed in
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0 for inspection.
> 14:42:23 Results logged to
> /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/jaro_winkler-1.4.0/gem_make.out
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503962#comment-16503962
]
stack commented on HBASE-20671:
---
Ok. Assigning to you then. I'm interested. This is a good one so shout if more
info to figure. Thanks.
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments: 0001-Test-for-HBASE-20671.patch,
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack reassigned HBASE-20671:
-
Assignee: Josh Elser (was: stack)
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments: 0001-Test-for-HBASE-20671.patch,
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[
https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503961#comment-16503961
]
Ted Yu commented on HBASE-20630:
{code}
+} catch (Exception e) {
+ LOG.error("failed", e);
{code}
Please add assertion in the catch block. Current formation wouldn't fail the
test.
> B&R: Delete command enhancements
>
>
> Key: HBASE-20630
> URL: https://issues.apache.org/jira/browse/HBASE-20630
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch
>
>
> Make the command more useable. Currently, user needs to provide list of
> backup ids to delete. It would be nice to have more convenient options, such
> as: deleting all backups which are older than XXX days, etc
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20689) Docker fails to install rubocop for precommit
[
https://issues.apache.org/jira/browse/HBASE-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503959#comment-16503959
]
Josh Elser commented on HBASE-20689:
I'll commit this later tonight if no one else gets to it. Seems to be blocking
us universally.
I assume it should hit all branches?
> Docker fails to install rubocop for precommit
> -
>
> Key: HBASE-20689
> URL: https://issues.apache.org/jira/browse/HBASE-20689
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Blocker
> Attachments: HBASE-20689.master.001.patch
>
>
> Precommit started to fail with rubocop install in 'Building base image' phase.
> The used maven:3.5-jdk8 image was updated ~10h ago, probably related.
> [https://hub.docker.com/r/library/maven/]
> One of the failed jobs:
> [https://builds.apache.org/job/PreCommit-HBASE-Build/13105/console]
> {noformat}
> 14:42:23 Setting up ruby (1:2.3.3) ...
> 14:42:23 Processing triggers for libc-bin (2.24-11+deb9u3) ...
> 14:42:23 Successfully installed rake-12.3.1
> 14:42:23 Parsing documentation for rake-12.3.1
> 14:42:23 Installing ri documentation for rake-12.3.1
> 14:42:23 Done installing documentation for rake after 1 seconds
> 14:42:23 Building native extensions. This could take a while...
> 14:42:23 [91mERROR: Error installing rubocop:
> 14:42:23 ERROR: Failed to build gem native extension.
> 14:42:23
> 14:42:23 current directory:
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0/ext/jaro_winkler
> 14:42:23 /usr/bin/ruby2.3 -r ./siteconf20180606-949-99wdpz.rb extconf.rb
> 14:42:23 mkmf.rb can't find header files for ruby at
> /usr/lib/ruby/include/ruby.h
> 14:42:23
> 14:42:23 extconf failed, exit code 1
> 14:42:23
> 14:42:23 Gem files will remain installed in
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0 for inspection.
> 14:42:23 Results logged to
> /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/jaro_winkler-1.4.0/gem_make.out
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20605) Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission check
[ https://issues.apache.org/jira/browse/HBASE-20605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503956#comment-16503956 ] Hudson commented on HBASE-20605: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1120 (See [https://builds.apache.org/job/HBase-1.2-IT/1120/]) HBASE-20605 Excludes Azure's new filesystem from the (elserj: rev 96e8ab4af9e0e8c49165d6f2d1bb6e82f554ef5a) * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSecureBulkLoadEndpoint.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java > Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission > check > - > > Key: HBASE-20605 > URL: https://issues.apache.org/jira/browse/HBASE-20605 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 1.5.0, 1.2.7, 1.3.3, 1.4.5 > > Attachments: HBASE-20605.001.branch-1.patch, > HBASE-20605.002.branch-1.patch > > > Some folks in Hadoop are working on landing a new FileSystem from the Azure > team: HADOOP-15407 > At present, this FileSystem doesn't support permissions which causes the > SecureBulkLoadEndpoint to balk because it the staging directory doesn't have > the proper 711 permissions. > We have a static list of FileSystem schemes which we ignore this check on. I > have a patch on an HBase 1.1ish which: > # Adds the new FileSystem scheme > # Makes this list configurable for the future -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503955#comment-16503955 ] Ted Yu commented on HBASE-20630: Synonyms of purge: cleanse, clear, remove, depose Can you choose one from the above (first letter is different from 'p') ? > B&R: Delete command enhancements > > > Key: HBASE-20630 > URL: https://issues.apache.org/jira/browse/HBASE-20630 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch > > > Make the command more useable. Currently, user needs to provide list of > backup ids to delete. It would be nice to have more convenient options, such > as: deleting all backups which are older than XXX days, etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20630) B&R: Delete command enhancements
[
https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503947#comment-16503947
]
Vladimir Rodionov edited comment on HBASE-20630 at 6/6/18 9:43 PM:
---
{quote}
The above is confusing : getting root path has nothing to do with purge.
{quote}
Yes, but both Path and Purge starts with *p*. I use *-p* to define both: path
and purge days, but in different commands
{quote}
Did you mean OPTION_PATH_PURGE ?
{quote}
Yes, it was mispelling
was (Author: vrodionov):
{quote}
The above is confusing : getting root path has nothing to do with purge.
{quote}
Yes, but both Path and Purge starts with *p*. I use *-p* to define both: path
and purge days, but in different commands
{quote}
Did you mean OPTION_PATH_PURGE ?
{quote}
Yes, it was misspelling
> B&R: Delete command enhancements
>
>
> Key: HBASE-20630
> URL: https://issues.apache.org/jira/browse/HBASE-20630
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch
>
>
> Make the command more useable. Currently, user needs to provide list of
> backup ids to delete. It would be nice to have more convenient options, such
> as: deleting all backups which are older than XXX days, etc
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20605) Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission check
[ https://issues.apache.org/jira/browse/HBASE-20605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503950#comment-16503950 ] Hudson commented on HBASE-20605: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #419 (See [https://builds.apache.org/job/HBase-1.3-IT/419/]) HBASE-20605 Excludes Azure's new filesystem from the (elserj: rev 685906c8f6510d1bf3727742a6844ccfd310c555) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSecureBulkLoadEndpoint.java > Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission > check > - > > Key: HBASE-20605 > URL: https://issues.apache.org/jira/browse/HBASE-20605 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 1.5.0, 1.2.7, 1.3.3, 1.4.5 > > Attachments: HBASE-20605.001.branch-1.patch, > HBASE-20605.002.branch-1.patch > > > Some folks in Hadoop are working on landing a new FileSystem from the Azure > team: HADOOP-15407 > At present, this FileSystem doesn't support permissions which causes the > SecureBulkLoadEndpoint to balk because it the staging directory doesn't have > the proper 711 permissions. > We have a static list of FileSystem schemes which we ignore this check on. I > have a patch on an HBase 1.1ish which: > # Adds the new FileSystem scheme > # Makes this list configurable for the future -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20630) B&R: Delete command enhancements
[
https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503947#comment-16503947
]
Vladimir Rodionov edited comment on HBASE-20630 at 6/6/18 9:42 PM:
---
{quote}
The above is confusing : getting root path has nothing to do with purge.
{quote}
Yes, but both Path and Purge starts with *p*. I use *-p* to define both: path
and purge days, but in different commands
{quote}
Did you mean OPTION_PATH_PURGE ?
{quote}
Yes, it was misspelling
was (Author: vrodionov):
{quote}
The above is confusing : getting root path has nothing to do with purge.
{quote}
Yes, but both Path and Purge starts with *p*. I use *-p* to define both: path
and purge days, but in different commands
> B&R: Delete command enhancements
>
>
> Key: HBASE-20630
> URL: https://issues.apache.org/jira/browse/HBASE-20630
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch
>
>
> Make the command more useable. Currently, user needs to provide list of
> backup ids to delete. It would be nice to have more convenient options, such
> as: deleting all backups which are older than XXX days, etc
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503949#comment-16503949 ] Vladimir Rodionov commented on HBASE-20630: --- Patch v2. > B&R: Delete command enhancements > > > Key: HBASE-20630 > URL: https://issues.apache.org/jira/browse/HBASE-20630 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch > > > Make the command more useable. Currently, user needs to provide list of > backup ids to delete. It would be nice to have more convenient options, such > as: deleting all backups which are older than XXX days, etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[
https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503947#comment-16503947
]
Vladimir Rodionov commented on HBASE-20630:
---
{quote}
The above is confusing : getting root path has nothing to do with purge.
{quote}
Yes, but both Path and Purge starts with *p*. I use *-p* to define both: path
and purge days, but in different commands
> B&R: Delete command enhancements
>
>
> Key: HBASE-20630
> URL: https://issues.apache.org/jira/browse/HBASE-20630
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch
>
>
> Make the command more useable. Currently, user needs to provide list of
> backup ids to delete. It would be nice to have more convenient options, such
> as: deleting all backups which are older than XXX days, etc
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20630) B&R: Delete command enhancements
[ https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-20630: -- Attachment: HBASE-20630-v2.patch > B&R: Delete command enhancements > > > Key: HBASE-20630 > URL: https://issues.apache.org/jira/browse/HBASE-20630 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-20630-v1.patch, HBASE-20630-v2.patch > > > Make the command more useable. Currently, user needs to provide list of > backup ids to delete. It would be nice to have more convenient options, such > as: deleting all backups which are older than XXX days, etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[
https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503932#comment-16503932
]
Ted Yu commented on HBASE-20630:
I searched OPTION_PURGE_PATH in the patch but didn't find it.
Did you mean OPTION_PATH_PURGE ?
In getBackupRootPath :
{code}
-value = cmdline.getOptionValue(OPTION_PATH);
+value = cmdline.getOptionValue(OPTION_PATH_PURGE);
{code}
The above is confusing : getting root path has nothing to do with purge.
> B&R: Delete command enhancements
>
>
> Key: HBASE-20630
> URL: https://issues.apache.org/jira/browse/HBASE-20630
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-20630-v1.patch
>
>
> Make the command more useable. Currently, user needs to provide list of
> backup ids to delete. It would be nice to have more convenient options, such
> as: deleting all backups which are older than XXX days, etc
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20605) Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission check
[ https://issues.apache.org/jira/browse/HBASE-20605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-20605: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks Ted and Sean! > Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission > check > - > > Key: HBASE-20605 > URL: https://issues.apache.org/jira/browse/HBASE-20605 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 1.5.0, 1.2.7, 1.3.3, 1.4.5 > > Attachments: HBASE-20605.001.branch-1.patch, > HBASE-20605.002.branch-1.patch > > > Some folks in Hadoop are working on landing a new FileSystem from the Azure > team: HADOOP-15407 > At present, this FileSystem doesn't support permissions which causes the > SecureBulkLoadEndpoint to balk because it the staging directory doesn't have > the proper 711 permissions. > We have a static list of FileSystem schemes which we ignore this check on. I > have a patch on an HBase 1.1ish which: > # Adds the new FileSystem scheme > # Makes this list configurable for the future -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20605) Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission check
[ https://issues.apache.org/jira/browse/HBASE-20605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-20605: --- Fix Version/s: 1.2.7 > Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission > check > - > > Key: HBASE-20605 > URL: https://issues.apache.org/jira/browse/HBASE-20605 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 1.5.0, 1.2.7, 1.3.3, 1.4.5 > > Attachments: HBASE-20605.001.branch-1.patch, > HBASE-20605.002.branch-1.patch > > > Some folks in Hadoop are working on landing a new FileSystem from the Azure > team: HADOOP-15407 > At present, this FileSystem doesn't support permissions which causes the > SecureBulkLoadEndpoint to balk because it the staging directory doesn't have > the proper 711 permissions. > We have a static list of FileSystem schemes which we ignore this check on. I > have a patch on an HBase 1.1ish which: > # Adds the new FileSystem scheme > # Makes this list configurable for the future -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503926#comment-16503926
]
Josh Elser commented on HBASE-20671:
Thanks, sir. I can try to take this back and do more digging on my end with the
great insight you've given already.
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments: 0001-Test-for-HBASE-20671.patch,
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20630) B&R: Delete command enhancements
[
https://issues.apache.org/jira/browse/HBASE-20630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503927#comment-16503927
]
Vladimir Rodionov commented on HBASE-20630:
---
{quote}
Option is no longer supported ?
{quote}
No, it was renamed.
{quote}
The constant name, OPTION_PATH_DESC, doesn't reflect purging.
{quote}
The option OPTION_PURGE_PATH has two different descriptions: OPTION_PATH_DESC
and OPTION_PURGE_DESC. It depends on command.
> B&R: Delete command enhancements
>
>
> Key: HBASE-20630
> URL: https://issues.apache.org/jira/browse/HBASE-20630
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-20630-v1.patch
>
>
> Make the command more useable. Currently, user needs to provide list of
> backup ids to delete. It would be nice to have more convenient options, such
> as: deleting all backups which are older than XXX days, etc
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503922#comment-16503922
]
stack commented on HBASE-20671:
---
[~elserj attached my test if it helps...
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments: 0001-Test-for-HBASE-20671.patch,
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20671:
--
Attachment: 0001-Test-for-HBASE-20671.patch
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments: 0001-Test-for-HBASE-20671.patch,
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503919#comment-16503919
]
stack commented on HBASE-20671:
---
bq. I'm very suspect of something in the region replication codepath (if that's
what you are thinking by "errant writer"). I want to try to repro this in a
junit test now (if you haven't already).
I made a test where I turned off the hbase:meta GC'er, merged, and then killed
master. We don't assign out any merge parent. We don't find a ghost line in the
hbase:meta when we load hbase:meta when new Master comes up.
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments:
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503917#comment-16503917
]
Josh Elser commented on HBASE-20671:
{quote}Let me make sure there weren't any other tests that were being run
against the same cluster.. (they should all be disabled internally).
{quote}
Confirmed. No rogue hbck'ing happening.
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments:
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503916#comment-16503916
]
stack commented on HBASE-20671:
---
07 crashed before the master did... and we had started a recovery.
2018-05-30 04:27:10,012 INFO [main-EventThread] master.RegionServerTracker:
RegionServer ephemeral node deleted, processing expiration
[ctr-e138-1518143905142-336066-01-07.hwx.site,16020,1527652019629]
Claims no WALs for this server:
2018-05-30 04:27:10,131 INFO [PEWorker-15] master.SplitLogManager:
hdfs://mycluster/apps/hbase/data/WALs/ctr-e138-1518143905142-336066-01-07.hwx.site,16020,1527652019629-splitting
is empty dir, no logs to split
... which is hard to believe. Would be good to check the 07 side if this
true.
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments:
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20693: --- Priority: Minor (was: Major) > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Improvement >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0 > > > Log Level page design was changed to include header and footers in > HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the > log level page will be as it were before HBASE-20577 i.e without the > navigation bar. This JIRA will refactor rest and thrift and extract > 'header.jsp' and 'footer.jsp' form them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20693: --- Description: Log Level page design was changed to include header and footers in HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the log level page will be as it were before HBASE-20577 i.e without the navigation bar. This JIRA will refactor rest and thrift and extract 'header.jsp' and 'footer.jsp' from them. (was: Log Level page design was changed to include header and footers in HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the log level page will be as it were before HBASE-20577 i.e without the navigation bar. This JIRA will refactor rest and thrift and extract 'header.jsp' and 'footer.jsp' form them.) > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Improvement >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0 > > > Log Level page design was changed to include header and footers in > HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the > log level page will be as it were before HBASE-20577 i.e without the > navigation bar. This JIRA will refactor rest and thrift and extract > 'header.jsp' and 'footer.jsp' from them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20693: --- Issue Type: Improvement (was: Bug) > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Improvement >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > > Log Level page design was changed to include header and footers in > HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the > log level page will be as it were before HBASE-20577 i.e without the > navigation bar. This JIRA will refactor rest and thrift and extract > 'header.jsp' and 'footer.jsp' form them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503904#comment-16503904 ] Nihal Jain commented on HBASE-20693: Hi I have removed the patch and moved it to HBASE-20577 as addendum. In this JIRA I will refactor the jsp's of rest and thrift. > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Bug >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > > Log Level page design was changed to include header and footers in > HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the > log level page will be as it were before HBASE-20577 i.e without the > navigation bar. This JIRA will refactor rest and thrift and extract > 'header.jsp' and 'footer.jsp' form them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20693: --- Attachment: (was: rest_after_patch.png) > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Bug >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > > Log Level page design was changed to include header and footers in > HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the > log level page will be as it were before HBASE-20577 i.e without the > navigation bar. This JIRA will refactor rest and thrift and extract > 'header.jsp' and 'footer.jsp' form them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20693: --- Status: Open (was: Patch Available) > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Bug >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > > Log Level page design was changed to include header and footers in > HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the > log level page will be as it were before HBASE-20577 i.e without the > navigation bar. This JIRA will refactor rest and thrift and extract > 'header.jsp' and 'footer.jsp' form them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20693: --- Description: Log Level page design was changed to include header and footers in HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the log level page will be as it were before HBASE-20577 i.e without the navigation bar. This JIRA will refactor rest and thrift and extract 'header.jsp' and 'footer.jsp' form them. (was: Log Level page design was changed to include header and footers in HBASE-20577. Not all web UIs in hbase have header/footer jsp. Also, it would be wrong to assume some new project would be having its jsp refactored into header/footer. To mitigate, we can do the following: 1) fall back to original log level design if 'header.jsp'/'footer.jsp' is not present 2) Next, refactor jsp file of other projects (like rest, thrift) to have 'header.jsp' and 'footer.jsp') > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Bug >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > > Log Level page design was changed to include header and footers in > HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the > log level page will be as it were before HBASE-20577 i.e without the > navigation bar. This JIRA will refactor rest and thrift and extract > 'header.jsp' and 'footer.jsp' form them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20693: --- Attachment: (was: HBASE-20469.master.001.patch) > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Bug >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > > Log Level page design was changed to include header and footers in > HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the > log level page will be as it were before HBASE-20577 i.e without the > navigation bar. This JIRA will refactor rest and thrift and extract > 'header.jsp' and 'footer.jsp' form them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20577) Make Log Level page design consistent with the design of other pages in UI
[ https://issues.apache.org/jira/browse/HBASE-20577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-20577. Resolution: Fixed Thanks for the addendum > Make Log Level page design consistent with the design of other pages in UI > -- > > Key: HBASE-20577 > URL: https://issues.apache.org/jira/browse/HBASE-20577 > Project: HBase > Issue Type: Improvement > Components: UI, Usability >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20577.master.001.patch, > HBASE-20577.master.002.patch, HBASE-20577.master.ADDENDUM.patch, > after_patch_LogLevel_CLI.png, after_patch_get_log_level.png, > after_patch_require_field_validation.png, after_patch_set_log_level_bad.png, > after_patch_set_log_level_success.png, > before_patch_no_validation_required_field.png, rest_after_addendum_patch.png > > > The Log Level page in web UI seems out of the place. I think we should make > it look consistent with design of other pages in HBase web UI. > Also, validation of required fields should be done, otherwise user should not > be allowed to click submit button. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20693: --- Summary: Refactor rest and thrift jsp's and extract header and footer (was: Log Level page may throw FNFE if no header.jsp or footer.jsp is present) > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Bug >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20469.master.001.patch, rest_after_patch.png > > > Log Level page design was changed to include header and footers in > HBASE-20577. Not all web UIs in hbase have header/footer jsp. Also, it would > be wrong to assume some new project would be having its jsp refactored into > header/footer. > To mitigate, we can do the following: > 1) fall back to original log level design if 'header.jsp'/'footer.jsp' is > not present > 2) Next, refactor jsp file of other projects (like rest, thrift) to have > 'header.jsp' and 'footer.jsp' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503900#comment-16503900
]
Josh Elser commented on HBASE-20671:
{quote}or an errant writer? An hbck or something?
{quote}
I know this exact test doesn't do any HBCK'ing. Let me make sure there weren't
any other tests that were being run against the same cluster.. (they should all
be disabled internally).
I'm very suspect of something in the region replication codepath (if that's
what you are thinking by "errant writer"). I want to try to repro this in a
junit test now (if you haven't already).
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments:
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20577) Make Log Level page design consistent with the design of other pages in UI
[ https://issues.apache.org/jira/browse/HBASE-20577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503898#comment-16503898 ] Nihal Jain commented on HBASE-20577: Ok understood. I have attached [^HBASE-20577.master.ADDENDUM.patch] patch which will fall back to original log level design if 'header.jsp'/'footer.jsp' is not present. Will take care of point 2 in [HBASE-20693|https://issues.apache.org/jira/browse/HBASE-20693] > Make Log Level page design consistent with the design of other pages in UI > -- > > Key: HBASE-20577 > URL: https://issues.apache.org/jira/browse/HBASE-20577 > Project: HBase > Issue Type: Improvement > Components: UI, Usability >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20577.master.001.patch, > HBASE-20577.master.002.patch, HBASE-20577.master.ADDENDUM.patch, > after_patch_LogLevel_CLI.png, after_patch_get_log_level.png, > after_patch_require_field_validation.png, after_patch_set_log_level_bad.png, > after_patch_set_log_level_success.png, > before_patch_no_validation_required_field.png, rest_after_addendum_patch.png > > > The Log Level page in web UI seems out of the place. I think we should make > it look consistent with design of other pages in HBase web UI. > Also, validation of required fields should be done, otherwise user should not > be allowed to click submit button. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20689) Docker fails to install rubocop for precommit
[
https://issues.apache.org/jira/browse/HBASE-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503896#comment-16503896
]
Josh Elser commented on HBASE-20689:
+1 too!
> Docker fails to install rubocop for precommit
> -
>
> Key: HBASE-20689
> URL: https://issues.apache.org/jira/browse/HBASE-20689
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Blocker
> Attachments: HBASE-20689.master.001.patch
>
>
> Precommit started to fail with rubocop install in 'Building base image' phase.
> The used maven:3.5-jdk8 image was updated ~10h ago, probably related.
> [https://hub.docker.com/r/library/maven/]
> One of the failed jobs:
> [https://builds.apache.org/job/PreCommit-HBASE-Build/13105/console]
> {noformat}
> 14:42:23 Setting up ruby (1:2.3.3) ...
> 14:42:23 Processing triggers for libc-bin (2.24-11+deb9u3) ...
> 14:42:23 Successfully installed rake-12.3.1
> 14:42:23 Parsing documentation for rake-12.3.1
> 14:42:23 Installing ri documentation for rake-12.3.1
> 14:42:23 Done installing documentation for rake after 1 seconds
> 14:42:23 Building native extensions. This could take a while...
> 14:42:23 [91mERROR: Error installing rubocop:
> 14:42:23 ERROR: Failed to build gem native extension.
> 14:42:23
> 14:42:23 current directory:
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0/ext/jaro_winkler
> 14:42:23 /usr/bin/ruby2.3 -r ./siteconf20180606-949-99wdpz.rb extconf.rb
> 14:42:23 mkmf.rb can't find header files for ruby at
> /usr/lib/ruby/include/ruby.h
> 14:42:23
> 14:42:23 extconf failed, exit code 1
> 14:42:23
> 14:42:23 Gem files will remain installed in
> /var/lib/gems/2.3.0/gems/jaro_winkler-1.4.0 for inspection.
> 14:42:23 Results logged to
> /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/jaro_winkler-1.4.0/gem_make.out
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20681) IntegrationTestDriver fails after HADOOP-15406 due to missing hamcrest-core
[
https://issues.apache.org/jira/browse/HBASE-20681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503894#comment-16503894
]
Josh Elser commented on HBASE-20681:
Need to get HBASE-20689 landed.
> IntegrationTestDriver fails after HADOOP-15406 due to missing hamcrest-core
> ---
>
> Key: HBASE-20681
> URL: https://issues.apache.org/jira/browse/HBASE-20681
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20681.001.patch, HBASE-20681.002.patch,
> HBASE-20681.003.patch, HBASE-20681.004.patch
>
>
> HADOOP-15406 marked mockito and junit as test-only dependencies which, I
> believe, has stopped them from being included in a stock Hadoop classpath.
> Prior, you'd get hamcrest at {{share/hadoop/common/lib/hamcrest-core-1.3.jar}}
> However, we depend on it being there for our junit in hbase-it:
> {noformat}
> [INFO] --- maven-dependency-plugin:3.0.1:tree (default-cli) @ hbase-it ---
> [INFO] org.apache.hbase:hbase-it:jar:2.0.1-SNAPSHOT
> [INFO] +- junit:junit:jar:4.12:test
> [INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test
> {noformat}
> We need to make sure we include it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HBASE-20577) Make Log Level page design consistent with the design of other pages in UI
[ https://issues.apache.org/jira/browse/HBASE-20577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain reopened HBASE-20577: > Make Log Level page design consistent with the design of other pages in UI > -- > > Key: HBASE-20577 > URL: https://issues.apache.org/jira/browse/HBASE-20577 > Project: HBase > Issue Type: Improvement > Components: UI, Usability >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20577.master.001.patch, > HBASE-20577.master.002.patch, HBASE-20577.master.ADDENDUM.patch, > after_patch_LogLevel_CLI.png, after_patch_get_log_level.png, > after_patch_require_field_validation.png, after_patch_set_log_level_bad.png, > after_patch_set_log_level_success.png, > before_patch_no_validation_required_field.png, rest_after_addendum_patch.png > > > The Log Level page in web UI seems out of the place. I think we should make > it look consistent with design of other pages in HBase web UI. > Also, validation of required fields should be done, otherwise user should not > be allowed to click submit button. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20577) Make Log Level page design consistent with the design of other pages in UI
[ https://issues.apache.org/jira/browse/HBASE-20577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20577: --- Attachment: rest_after_addendum_patch.png > Make Log Level page design consistent with the design of other pages in UI > -- > > Key: HBASE-20577 > URL: https://issues.apache.org/jira/browse/HBASE-20577 > Project: HBase > Issue Type: Improvement > Components: UI, Usability >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20577.master.001.patch, > HBASE-20577.master.002.patch, HBASE-20577.master.ADDENDUM.patch, > after_patch_LogLevel_CLI.png, after_patch_get_log_level.png, > after_patch_require_field_validation.png, after_patch_set_log_level_bad.png, > after_patch_set_log_level_success.png, > before_patch_no_validation_required_field.png, rest_after_addendum_patch.png > > > The Log Level page in web UI seems out of the place. I think we should make > it look consistent with design of other pages in HBase web UI. > Also, validation of required fields should be done, otherwise user should not > be allowed to click submit button. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20671) Merged region brought back to life causing RS to be killed by Master
[
https://issues.apache.org/jira/browse/HBASE-20671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503890#comment-16503890
]
stack commented on HBASE-20671:
---
Thanks [~elserj]
The hbase:meta listing is interesting. The timestamps are behind the timestamp
of the delete of the family done by:
2018-05-30 04:26:00,634 DEBUG [PEWorker-9] hbase.META: Delete
{"totalColumns":1,"row":"tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.","families":{"info":[
So... unlikely it HBASE-20065. Skipped hbase:meta recovery (not too likely but
perhaps) or an errant writer? An hbck or something?
We assign meta to 07 at 2018-05-30 04:01:39,120 , about twenty minutes
before the fun starts.
> Merged region brought back to life causing RS to be killed by Master
>
>
> Key: HBASE-20671
> URL: https://issues.apache.org/jira/browse/HBASE-20671
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.0
>Reporter: Josh Elser
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.1
>
> Attachments:
> hbase-hbase-master-ctr-e138-1518143905142-336066-01-03.hwx.site.log.zip,
> hbase-hbase-regionserver-ctr-e138-1518143905142-336066-01-02.hwx.site.log.zip
>
>
> Another bug coming out of a master restart and replay of the pv2 logs.
> The master merged two regions into one successfully, was restarted, but then
> ended up assigning the children region back out to the cluster. There is a
> log message which appears to indicate that RegionStates acknowledges that it
> doesn't know what this region is as it's replaying the pv2 WAL; however, it
> incorrectly assumes that the region is just OFFLINE and needs to be assigned.
> {noformat}
> 2018-05-30 04:26:00,055 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=2] master.HMaster:
> Client=hrt_qa//172.27.85.11 Merge regions a7dd6606dcacc9daf085fc9fa2aecc0c
> and 4017a3c778551d4d258c785d455f9c0b
> 2018-05-30 04:28:27,525 DEBUG
> [master/ctr-e138-1518143905142-336066-01-03:2]
> procedure2.ProcedureExecutor: Completed pid=4368, state=SUCCESS;
> MergeTableRegionsProcedure table=tabletwo_merge,
> regions=[a7dd6606dcacc9daf085fc9fa2aecc0c, 4017a3c778551d4d258c785d455f9c0b],
> forcibly=false
> {noformat}
> {noformat}
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: a7dd6606dcacc9daf085fc9fa2aecc0c
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,263 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=a7dd6606dcacc9daf085fc9fa2aecc0c
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.AssignmentManager: 4017a3c778551d4d258c785d455f9c0b
> regionState=null; presuming OFFLINE
> 2018-05-30 04:29:20,266 INFO
> [master/ctr-e138-1518143905142-336066-01-03:2]
> assignment.RegionStates: Added to offline, CURRENTLY NEVER CLEARED!!!
> rit=OFFLINE, location=null, table=tabletwo_merge,
> region=4017a3c778551d4d258c785d455f9c0b
> {noformat}
> Eventually, the RS reports in its online regions, and the master tells it to
> kill itself:
> {noformat}
> 2018-05-30 04:29:24,272 WARN
> [RpcServer.default.FPBQ.Fifo.handler=26,queue=2,port=2]
> assignment.AssignmentManager: Killing
> ctr-e138-1518143905142-336066-01-02.hwx.site,16020,1527654546619: Not
> online: tabletwo_merge,,1527652130538.a7dd6606dcacc9daf085fc9fa2aecc0c.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20577) Make Log Level page design consistent with the design of other pages in UI
[ https://issues.apache.org/jira/browse/HBASE-20577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20577: --- Attachment: HBASE-20577.master.ADDENDUM.patch > Make Log Level page design consistent with the design of other pages in UI > -- > > Key: HBASE-20577 > URL: https://issues.apache.org/jira/browse/HBASE-20577 > Project: HBase > Issue Type: Improvement > Components: UI, Usability >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20577.master.001.patch, > HBASE-20577.master.002.patch, HBASE-20577.master.ADDENDUM.patch, > after_patch_LogLevel_CLI.png, after_patch_get_log_level.png, > after_patch_require_field_validation.png, after_patch_set_log_level_bad.png, > after_patch_set_log_level_success.png, > before_patch_no_validation_required_field.png > > > The Log Level page in web UI seems out of the place. I think we should make > it look consistent with design of other pages in HBase web UI. > Also, validation of required fields should be done, otherwise user should not > be allowed to click submit button. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
