[jira] [Commented] (HBASE-18641) Include block content verification logic used in lruCache in bucketCache
[
https://issues.apache.org/jira/browse/HBASE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168839#comment-16168839
]
Hadoop QA commented on HBASE-18641:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 56s{color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}407m 26s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 3m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}441m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
|
| | hadoop.hbase.regionserver.TestRecoveredEdits |
| | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint |
| | hadoop.hbase.client.TestAdmin2 |
| | hadoop.hbas
[jira] [Commented] (HBASE-18831) Add explicit dependency on javax.el
[
https://issues.apache.org/jira/browse/HBASE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168829#comment-16168829
]
Hadoop QA commented on HBASE-18831:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
17s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
55s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
11s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
6s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 53s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 89m
23s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
53s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
17s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}153m 46s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
3s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}320m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bd219c0 |
| JIRA Issue | HBASE-18831 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887462/HBASE-18831.branch-2.001.patch
|
| Optional Tests | asflicense javac javadoc unit xml compile |
| uname | Linux 7cd5574b12ad 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / a5c8461 |
| Default Java | 1.8.0_144 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/8652/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8652/testReport/ |
| modules | C: hbase-server hbase-thrift hbase-rest . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8652/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Add explicit dependency on javax.el
> ---
>
>
[jira] [Commented] (HBASE-18831) Add explicit dependency on javax.el
[ https://issues.apache.org/jira/browse/HBASE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168819#comment-16168819 ] Duo Zhang commented on HBASE-18831: --- [~stack] Verified. It works for me. Thanks. > Add explicit dependency on javax.el > --- > > Key: HBASE-18831 > URL: https://issues.apache.org/jira/browse/HBASE-18831 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18831.branch-2.001.patch > > > Previous build would search for it running up through all point version from > 3.0.1-b1 until it hit b8. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16478: -- Fix Version/s: (was: 2.0.0) 2.0.0-alpha-4 > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16478.master.001.patch, > HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-16478) Rename WALKey in PB to WALEdit
[ https://issues.apache.org/jira/browse/HBASE-16478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-16478: --- Reopening. This change might make more problems than it is worth given we did not carry the general project to completion; in short, this makes it so hbase-protocol no longer works if it is messing w/ our WAL stuff; no one should be doing it but downstreamer hbase-indexer from ngdata repo does Reopening for now. > Rename WALKey in PB to WALEdit > -- > > Key: HBASE-16478 > URL: https://issues.apache.org/jira/browse/HBASE-16478 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0 > > Attachments: HBASE-16478.master.001.patch, > HBASE-16478.master.001.patch, hbase-16478_v1.patch > > > As per title. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18831) Add explicit dependency on javax.el
[ https://issues.apache.org/jira/browse/HBASE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168798#comment-16168798 ] stack commented on HBASE-18831: --- Does this make it work then [~Apache9]? > Add explicit dependency on javax.el > --- > > Key: HBASE-18831 > URL: https://issues.apache.org/jira/browse/HBASE-18831 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18831.branch-2.001.patch > > > Previous build would search for it running up through all point version from > 3.0.1-b1 until it hit b8. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18601) Update Htrace to 4.2
[
https://issues.apache.org/jira/browse/HBASE-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168794#comment-16168794
]
Hadoop QA commented on HBASE-18601:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
29s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 6m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
32s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-testing-util . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 11m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
14s{color} | {color:red} hbase-rest in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
31s{color} | {color:red} hbase-spark in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 5m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} rubocop {color} | {color:green} 0m
3s{color} | {color:green} There were no new rubocop issues. {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
1s{color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
28s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
1s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
54s{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m
50s{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.2. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
43s{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.3. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m
37s{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
32s{color} | {color:red} The patch causes 16 errors with Hadoop v2.6.5. {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-testing-util . {color} |
| {color:red
[jira] [Commented] (HBASE-18831) Add explicit dependency on javax.el
[ https://issues.apache.org/jira/browse/HBASE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168764#comment-16168764 ] Duo Zhang commented on HBASE-18831: --- +1. When using our internal nexus server it ends with 3.0.1-b08-jbossxxx and then fails... > Add explicit dependency on javax.el > --- > > Key: HBASE-18831 > URL: https://issues.apache.org/jira/browse/HBASE-18831 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18831.branch-2.001.patch > > > Previous build would search for it running up through all point version from > 3.0.1-b1 until it hit b8. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18832) LTT fails with casting exception for HColumnDescriptor
Umesh Agashe created HBASE-18832:
Summary: LTT fails with casting exception for HColumnDescriptor
Key: HBASE-18832
URL: https://issues.apache.org/jira/browse/HBASE-18832
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0-alpha-4
Reporter: Umesh Agashe
Assignee: Umesh Agashe
Here is the stack trace:
{code}
2017-09-15 12:38:38,158 ERROR [main] util.AbstractHBaseTool: Error running
command-line tool
java.lang.ClassCastException:
org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder$ModifyableColumnFamilyDescriptor
cannot be cast to org.apache.hadoop.hbase.HColumnDescriptor
at
org.apache.hadoop.hbase.util.LoadTestTool.applyColumnFamilyOptions(LoadTestTool.java:265)
at
org.apache.hadoop.hbase.util.LoadTestTool.initTestTable(LoadTestTool.java:540)
at
org.apache.hadoop.hbase.util.LoadTestTool.loadTable(LoadTestTool.java:567)
at
org.apache.hadoop.hbase.util.LoadTestTool.doWork(LoadTestTool.java:548)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:154)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.doStaticMain(AbstractHBaseTool.java:262)
at org.apache.hadoop.hbase.util.LoadTestTool.main(LoadTestTool.java:793)
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18831) Add explicit dependency on javax.el
[ https://issues.apache.org/jira/browse/HBASE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18831: -- Assignee: stack Fix Version/s: 2.0.0-alpha-4 Status: Patch Available (was: Open) > Add explicit dependency on javax.el > --- > > Key: HBASE-18831 > URL: https://issues.apache.org/jira/browse/HBASE-18831 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18831.branch-2.001.patch > > > Previous build would search for it running up through all point version from > 3.0.1-b1 until it hit b8. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18831) Add explicit dependency on javax.el
[ https://issues.apache.org/jira/browse/HBASE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168731#comment-16168731 ] Umesh Agashe commented on HBASE-18831: -- +1 > Add explicit dependency on javax.el > --- > > Key: HBASE-18831 > URL: https://issues.apache.org/jira/browse/HBASE-18831 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: stack > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18831.branch-2.001.patch > > > Previous build would search for it running up through all point version from > 3.0.1-b1 until it hit b8. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18831) Add explicit dependency on javax.el
[ https://issues.apache.org/jira/browse/HBASE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18831: -- Attachment: HBASE-18831.branch-2.001.patch > Add explicit dependency on javax.el > --- > > Key: HBASE-18831 > URL: https://issues.apache.org/jira/browse/HBASE-18831 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: stack > Attachments: HBASE-18831.branch-2.001.patch > > > Previous build would search for it running up through all point version from > 3.0.1-b1 until it hit b8. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18831) Add explicit dependency on javax.el
stack created HBASE-18831: - Summary: Add explicit dependency on javax.el Key: HBASE-18831 URL: https://issues.apache.org/jira/browse/HBASE-18831 Project: HBase Issue Type: Bug Components: dependencies Reporter: stack Previous build would search for it running up through all point version from 3.0.1-b1 until it hit b8. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168722#comment-16168722
]
Hadoop QA commented on HBASE-13346:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
37s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 18 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 12m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 9m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
39m 20s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 47s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
37s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m
17s{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}209m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker |
| | org.apache.hadoop.hbase.regionserver.TestCompaction |
| | org.apache.hadoop.hbase.snapshot.TestSnapshotClientRetries |
| | org.apache.hadoop.hbase.TestHBaseTestingUtility |
| | org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook |
| | org.apache.hadoop.hbase.wal.TestWALFi
[jira] [Updated] (HBASE-18830) TestCanaryTool does not check Canary monitor's error code
[ https://issues.apache.org/jira/browse/HBASE-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated HBASE-18830: - Attachment: HBASE-18830.001.patch Added assertion checks to make sure that the error code for the ToolRunner run() method is used. Testing Done: Checked that TestCanaryTool unit tests fail when there is an error code in the current Canary run. [~apurtell] [~sukuna...@gmail.com] please review. > TestCanaryTool does not check Canary monitor's error code > - > > Key: HBASE-18830 > URL: https://issues.apache.org/jira/browse/HBASE-18830 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Attachments: HBASE-18830.001.patch > > > None of the tests inside TestCanaryTool check Canary monitor's error code. > Thus, it is possible that the monitor has registered an error and yet the > tests pass. We should check the value returned by the _ToolRunner.run()_ > method inside each unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18601) Update Htrace to 4.2
[ https://issues.apache.org/jira/browse/HBASE-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168711#comment-16168711 ] Tamas Penzes commented on HBASE-18601: -- Yeah, [~mdrob], we have also talked about using a wrapper class yesterday, but then my patch was already done. But this way the replacement of the tracing might be easy later too. > Update Htrace to 4.2 > > > Key: HBASE-18601 > URL: https://issues.apache.org/jira/browse/HBASE-18601 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0, 3.0.0 >Reporter: Tamas Penzes >Assignee: Tamas Penzes > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18601.master.001.patch, > HBASE-18601.master.002.patch, HBASE-18601.master.003.patch > > > HTrace is not perfectly integrated into HBase, the version 3.2.0 is buggy, > the upgrade to 4.x is not trivial and would take time. It might not worth to > keep it in this state, so would be better to remove it. > Of course it doesn't mean tracing would be useless, just that in this form > the use of HTrace 3.2 might not add any value to the project and fixing it > would be far too much effort. > - > Based on the decision of the community we keep htrace now and update version -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18601) Update Htrace to 4.2
[ https://issues.apache.org/jira/browse/HBASE-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamas Penzes updated HBASE-18601: - Attachment: HBASE-18601.master.003.patch > Update Htrace to 4.2 > > > Key: HBASE-18601 > URL: https://issues.apache.org/jira/browse/HBASE-18601 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0, 3.0.0 >Reporter: Tamas Penzes >Assignee: Tamas Penzes > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18601.master.001.patch, > HBASE-18601.master.002.patch, HBASE-18601.master.003.patch > > > HTrace is not perfectly integrated into HBase, the version 3.2.0 is buggy, > the upgrade to 4.x is not trivial and would take time. It might not worth to > keep it in this state, so would be better to remove it. > Of course it doesn't mean tracing would be useless, just that in this form > the use of HTrace 3.2 might not add any value to the project and fixing it > would be far too much effort. > - > Based on the decision of the community we keep htrace now and update version -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
[ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168703#comment-16168703 ] Ted Yu commented on HBASE-17852: Looks like a new table is introduced. How you thought about achieving the same purpose with additional column family in backup table ? Please summary design in description. > Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental > backup) > > > Key: HBASE-17852 > URL: https://issues.apache.org/jira/browse/HBASE-17852 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-17852-v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
[ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-17852: -- Attachment: HBASE-17852-v1.patch Patch v1 > Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental > backup) > > > Key: HBASE-17852 > URL: https://issues.apache.org/jira/browse/HBASE-17852 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-17852-v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18830) TestCanaryTool does not check Canary monitor's error code
Chinmay Kulkarni created HBASE-18830: Summary: TestCanaryTool does not check Canary monitor's error code Key: HBASE-18830 URL: https://issues.apache.org/jira/browse/HBASE-18830 Project: HBase Issue Type: Bug Reporter: Chinmay Kulkarni Assignee: Chinmay Kulkarni None of the tests inside TestCanaryTool check Canary monitor's error code. Thus, it is possible that the monitor has registered an error and yet the tests pass. We should check the value returned by the _ToolRunner.run()_ method inside each unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18762) Canary sink type cast error
[ https://issues.apache.org/jira/browse/HBASE-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated HBASE-18762: - Attachment: HBASE-18762.001.patch > Canary sink type cast error > --- > > Key: HBASE-18762 > URL: https://issues.apache.org/jira/browse/HBASE-18762 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Attachments: HBASE-18762.001.patch > > > When running the main method of Canary.java, we see the following error: > Exception in thread "main" java.lang.ClassCastException: > org.apache.hadoop.hbase.tool.Canary$RegionServerStdOutSink cannot be cast to > org.apache.hadoop.hbase.tool.Canary$RegionStdOutSink > at org.apache.hadoop.hbase.tool.Canary.newMonitor(Canary.java:911) > at org.apache.hadoop.hbase.tool.Canary.run(Canary.java:796) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.hbase.tool.Canary.main(Canary.java:1571) > This happens because we typecast the sink depending on the mode (zookeeper > mode/region server mode) that Canary is configured in. In case no mode is > specified, we typecast the sink into _RegionStdOutSink_. In general, it is > possible to provide inconsistent mode and sink types while running Canary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18829) Consider reverting HBASE-14893 Race between mutation on region and region closing operation
[ https://issues.apache.org/jira/browse/HBASE-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168680#comment-16168680 ] Vincent Poon commented on HBASE-18829: -- PHOENIX-3111 introduced a preClose() hook which stalls the region closing until all scanners are done, which sidesteps the race condition in HBASE-14893. However, that is problematic because it might stall forever - PHOENIX-4214 Assuming PHOENIX-4214 can be fixed, then HBASE-14893 doesn't matter to Phoenix. The decision to revert / not-revert should be based on its correctness from HBase's perspective. > Consider reverting HBASE-14893 Race between mutation on region and region > closing operation > --- > > Key: HBASE-18829 > URL: https://issues.apache.org/jira/browse/HBASE-18829 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 18829.v1.txt > > > HBASE-14893 was brought to attention by [~rajeshbabu] over PHOENIX-3111. > This issue is to consider reverting the fix from HBASE-14893 based on the > following observations: > * The closing boolean was intended to be acquired before taking the lock > ([~enis]) > * Phoenix local index has evolved over the years, the situation leading to > NotServingRegionException may not exist from Phoenix side > * Even if the situation still exists, downstream project (Phoenix) should > properly handle NotServingRegionException without change in locking scheme in > hbase -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18762) Canary sink type cast error
[ https://issues.apache.org/jira/browse/HBASE-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated HBASE-18762: - Attachment: (was: HBASE-18762.001.patch) > Canary sink type cast error > --- > > Key: HBASE-18762 > URL: https://issues.apache.org/jira/browse/HBASE-18762 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > > When running the main method of Canary.java, we see the following error: > Exception in thread "main" java.lang.ClassCastException: > org.apache.hadoop.hbase.tool.Canary$RegionServerStdOutSink cannot be cast to > org.apache.hadoop.hbase.tool.Canary$RegionStdOutSink > at org.apache.hadoop.hbase.tool.Canary.newMonitor(Canary.java:911) > at org.apache.hadoop.hbase.tool.Canary.run(Canary.java:796) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.hbase.tool.Canary.main(Canary.java:1571) > This happens because we typecast the sink depending on the mode (zookeeper > mode/region server mode) that Canary is configured in. In case no mode is > specified, we typecast the sink into _RegionStdOutSink_. In general, it is > possible to provide inconsistent mode and sink types while running Canary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18829) Consider reverting HBASE-14893 Race between mutation on region and region closing operation
[ https://issues.apache.org/jira/browse/HBASE-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168671#comment-16168671 ] Sergey Soldatov commented on HBASE-18829: - [~lhofhansl] I have strong feeling that Phoenix code is not relying on this behavior anymore ([~jamestaylor] and [~rajeshbabu] may correct me). Initially HBASE-14893 was introduced to avoid failing batchMutate because of closing state. Right now Phoenix is waiting for the completeness of batch mutations in preClose(), so closing state would not be set until we complete (or fail) our operations. > Consider reverting HBASE-14893 Race between mutation on region and region > closing operation > --- > > Key: HBASE-18829 > URL: https://issues.apache.org/jira/browse/HBASE-18829 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 18829.v1.txt > > > HBASE-14893 was brought to attention by [~rajeshbabu] over PHOENIX-3111. > This issue is to consider reverting the fix from HBASE-14893 based on the > following observations: > * The closing boolean was intended to be acquired before taking the lock > ([~enis]) > * Phoenix local index has evolved over the years, the situation leading to > NotServingRegionException may not exist from Phoenix side > * Even if the situation still exists, downstream project (Phoenix) should > properly handle NotServingRegionException without change in locking scheme in > hbase -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18725) [C++] Install header files as well as library
[ https://issues.apache.org/jira/browse/HBASE-18725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-18725. --- Resolution: Fixed Assignee: Enis Soztutar I have pushed this. > [C++] Install header files as well as library > - > > Key: HBASE-18725 > URL: https://issues.apache.org/jira/browse/HBASE-18725 > Project: HBase > Issue Type: Sub-task >Reporter: Scott Hunt >Assignee: Enis Soztutar >Priority: Critical > Fix For: HBASE-14850 > > Attachments: hbase-18725-v0.patch, hbase-18725-v1.patch > > > Currently "make install" only installs libHbaseClient[_d].[so|a], but in > order for the library to be useful for building applications, we'll need to > install the header files also. > This, of course, brings up another problem: that the headers aren't in their > own directory structure, so that application includes could look something > like: #include > I suggest that we create an hbase sub-directory (under hbase-native-client), > move all the [public] headers into there (we can keep the current directory > structure under that, i.e. hbase/core/cell.h, etc.) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18641) Include block content verification logic used in lruCache in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18641: --- Status: Patch Available (was: Reopened) > Include block content verification logic used in lruCache in bucketCache > > > Key: HBASE-18641 > URL: https://issues.apache.org/jira/browse/HBASE-18641 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair >Priority: Minor > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18641-branch-1.PATCH, HBASE-18641.PATCH, > HBASE-18641-V1.0.PATCH, HBASE-18641-WIP.PATCH > > > With off-heap/bucketCache being used to cache data blocks without going > through on-heap cache, the logic used in lruCache to check the content of > already cached block need to be included in bucketCache. Please see this > [discussion|https://mail-archives.apache.org/mod_mbox/hbase-dev/201708.mbox/%3cCAO40JLCnXLw3=0bbUaXdDx=w2fklljefvgj6-uvj_2jhfvo...@mail.gmail.com%3e] > for details. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168662#comment-16168662 ] Ted Yu commented on HBASE-18652: I am good with patch v3. > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: 18652-branch-1-V3.0.PATCH, HBASE-18652-addendum.patch, > HBASE-18652-BRANCH-1.PATCH, HBASE-18652-branch-1-v1.0.patch, > HBASE-18652-branch-1-V2.0.PATCH, HBASE-18652-branch-1-V3.0.PATCH, > HBASE-18652-branch-1-V3.0.PATCH, HBASE-18652.PATCH, HBASE-18652-V1.0.PATCH, > HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18829) Consider reverting HBASE-14893 Race between mutation on region and region closing operation
[
https://issues.apache.org/jira/browse/HBASE-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168633#comment-16168633
]
Hadoop QA commented on HBASE-18829:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
44s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
59s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
56s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
39m 23s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}102m
37s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}163m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18829 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887412/18829.v1.txt |
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 4043d0eca903 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a6d8ced |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
htt
[jira] [Updated] (HBASE-18762) Canary sink type cast error
[ https://issues.apache.org/jira/browse/HBASE-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated HBASE-18762: - Attachment: HBASE-18762.001.patch > Canary sink type cast error > --- > > Key: HBASE-18762 > URL: https://issues.apache.org/jira/browse/HBASE-18762 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Attachments: HBASE-18762.001.patch > > > When running the main method of Canary.java, we see the following error: > Exception in thread "main" java.lang.ClassCastException: > org.apache.hadoop.hbase.tool.Canary$RegionServerStdOutSink cannot be cast to > org.apache.hadoop.hbase.tool.Canary$RegionStdOutSink > at org.apache.hadoop.hbase.tool.Canary.newMonitor(Canary.java:911) > at org.apache.hadoop.hbase.tool.Canary.run(Canary.java:796) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.hbase.tool.Canary.main(Canary.java:1571) > This happens because we typecast the sink depending on the mode (zookeeper > mode/region server mode) that Canary is configured in. In case no mode is > specified, we typecast the sink into _RegionStdOutSink_. In general, it is > possible to provide inconsistent mode and sink types while running Canary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18762) Canary sink type cast error
[ https://issues.apache.org/jira/browse/HBASE-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168526#comment-16168526 ] Chinmay Kulkarni edited comment on HBASE-18762 at 9/15/17 9:18 PM: --- [~apurtell] Okay thanks. I've attached a patch where I've refactored the sinks. Tested with the TestCanaryTool test suite. Please take a look and provide feedback. was (Author: ckulkarni): [~apurtell] Okay thanks. I've attached a patch where I've refactored the sinks. Tested with the TestCanaryTool test suite. Please take a look provide feedback. > Canary sink type cast error > --- > > Key: HBASE-18762 > URL: https://issues.apache.org/jira/browse/HBASE-18762 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Attachments: HBASE-18762.001.patch > > > When running the main method of Canary.java, we see the following error: > Exception in thread "main" java.lang.ClassCastException: > org.apache.hadoop.hbase.tool.Canary$RegionServerStdOutSink cannot be cast to > org.apache.hadoop.hbase.tool.Canary$RegionStdOutSink > at org.apache.hadoop.hbase.tool.Canary.newMonitor(Canary.java:911) > at org.apache.hadoop.hbase.tool.Canary.run(Canary.java:796) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.hbase.tool.Canary.main(Canary.java:1571) > This happens because we typecast the sink depending on the mode (zookeeper > mode/region server mode) that Canary is configured in. In case no mode is > specified, we typecast the sink into _RegionStdOutSink_. In general, it is > possible to provide inconsistent mode and sink types while running Canary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18762) Canary sink type cast error
[ https://issues.apache.org/jira/browse/HBASE-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated HBASE-18762: - Attachment: (was: HBASE-18762.001.patch) > Canary sink type cast error > --- > > Key: HBASE-18762 > URL: https://issues.apache.org/jira/browse/HBASE-18762 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > > When running the main method of Canary.java, we see the following error: > Exception in thread "main" java.lang.ClassCastException: > org.apache.hadoop.hbase.tool.Canary$RegionServerStdOutSink cannot be cast to > org.apache.hadoop.hbase.tool.Canary$RegionStdOutSink > at org.apache.hadoop.hbase.tool.Canary.newMonitor(Canary.java:911) > at org.apache.hadoop.hbase.tool.Canary.run(Canary.java:796) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.hbase.tool.Canary.main(Canary.java:1571) > This happens because we typecast the sink depending on the mode (zookeeper > mode/region server mode) that Canary is configured in. In case no mode is > specified, we typecast the sink into _RegionStdOutSink_. In general, it is > possible to provide inconsistent mode and sink types while running Canary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18762) Canary sink type cast error
[ https://issues.apache.org/jira/browse/HBASE-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated HBASE-18762: - Attachment: HBASE-18762.001.patch [~apurtell] Okay thanks. I've attached a patch where I've refactored the sinks. Tested with the TestCanaryTool test suite. Please take a look provide feedback. > Canary sink type cast error > --- > > Key: HBASE-18762 > URL: https://issues.apache.org/jira/browse/HBASE-18762 > Project: HBase > Issue Type: Bug >Reporter: Chinmay Kulkarni >Assignee: Chinmay Kulkarni > Attachments: HBASE-18762.001.patch > > > When running the main method of Canary.java, we see the following error: > Exception in thread "main" java.lang.ClassCastException: > org.apache.hadoop.hbase.tool.Canary$RegionServerStdOutSink cannot be cast to > org.apache.hadoop.hbase.tool.Canary$RegionStdOutSink > at org.apache.hadoop.hbase.tool.Canary.newMonitor(Canary.java:911) > at org.apache.hadoop.hbase.tool.Canary.run(Canary.java:796) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.hbase.tool.Canary.main(Canary.java:1571) > This happens because we typecast the sink depending on the mode (zookeeper > mode/region server mode) that Canary is configured in. In case no mode is > specified, we typecast the sink into _RegionStdOutSink_. In general, it is > possible to provide inconsistent mode and sink types while running Canary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18829) Consider reverting HBASE-14893 Race between mutation on region and region closing operation
[ https://issues.apache.org/jira/browse/HBASE-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168505#comment-16168505 ] Josh Elser commented on HBASE-18829: bq. If Phoenix has solution in place dealing with the NotServingRegionException, it would be safer to revert. It seems like this would be the better long-term solution to make (we're not under the gun right now). Admittedly, I haven't traced through the HRegion code to appreciate the concurrency. Unless I've missed it, there's no other reason that HBASE-14893 was done than letting the Phoenix code work without a more substantial change downstream. In that case, I think the revert makes sense and we can do a "non-lazy" fix in Phoenix land. I think this is what Lars is saying too. This means we'd improve this dodgy code via PHOENIX-3177? Or just PHOENIX-4214? What's your take, [~rajeshbabu]? > Consider reverting HBASE-14893 Race between mutation on region and region > closing operation > --- > > Key: HBASE-18829 > URL: https://issues.apache.org/jira/browse/HBASE-18829 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 18829.v1.txt > > > HBASE-14893 was brought to attention by [~rajeshbabu] over PHOENIX-3111. > This issue is to consider reverting the fix from HBASE-14893 based on the > following observations: > * The closing boolean was intended to be acquired before taking the lock > ([~enis]) > * Phoenix local index has evolved over the years, the situation leading to > NotServingRegionException may not exist from Phoenix side > * Even if the situation still exists, downstream project (Phoenix) should > properly handle NotServingRegionException without change in locking scheme in > hbase -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18813) TestCanaryTool fails on branch-1 / branch-1.4
[
https://issues.apache.org/jira/browse/HBASE-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168484#comment-16168484
]
Hudson commented on HBASE-18813:
SUCCESS: Integrated in Jenkins build HBase-1.4 #919 (See
[https://builds.apache.org/job/HBase-1.4/919/])
Revert "Amend HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4"
(apurtell: rev 31b9096034e19171989fd5b76313e7e0f1a9a12a)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
Amend HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4 (apurtell: rev
f54c06f492c791882e65e10f7c6f9f995027357e)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
> TestCanaryTool fails on branch-1 / branch-1.4
> --
>
> Key: HBASE-18813
> URL: https://issues.apache.org/jira/browse/HBASE-18813
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 1.4.0, 1.5.0
>
> Attachments: HBASE-18813-branch-1-addendum.patch,
> HBASE-18813-branch-1-addendum.patch, HBASE-18813-branch-1.patch
>
>
> Mocking error
> {noformat}
> Running org.apache.hadoop.hbase.tool.TestCanaryTool
> Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 77.873 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.tool.TestCanaryTool
> testReadTableTimeouts(org.apache.hadoop.hbase.tool.TestCanaryTool) Time
> elapsed: 13.845 sec <<< FAILURE!
> org.mockito.exceptions.verification.junit.ArgumentsAreDifferent:
> Argument(s) are different! Wanted:
> mockAppender.doAppend(
>
> );
> -> at
> org.apache.hadoop.hbase.tool.TestCanaryTool.testReadTableTimeouts(TestCanaryTool.java:150)
> Actual invocation has different arguments:
> mockAppender.doAppend(
> org.apache.log4j.spi.LoggingEvent@7f5fcfe9
> );
> -> at
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.tool.TestCanaryTool.testReadTableTimeouts(TestCanaryTool.java:150)
> Results :
> Failed tests:
> TestCanaryTool.testReadTableTimeouts:150
> Argument(s) are different! Wanted:
> mockAppender.doAppend(
>
> );
> -> at
> org.apache.hadoop.hbase.tool.TestCanaryTool.testReadTableTimeouts(TestCanaryTool.java:150)
> Actual invocation has different arguments:
> mockAppender.doAppend(
> org.apache.log4j.spi.LoggingEvent@7f5fcfe9
> );
> -> at
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168476#comment-16168476
]
Tamas Penzes commented on HBASE-13346:
--
Second patch fixes backward compatibility issues. Should work properly.
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tamas Penzes updated HBASE-13346:
-
Attachment: HBASE-13346.master.002.patch
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18829) Consider reverting HBASE-14893 Race between mutation on region and region closing operation
[ https://issues.apache.org/jira/browse/HBASE-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168469#comment-16168469 ] Ted Yu commented on HBASE-18829: If Phoenix has solution in place dealing with the NotServingRegionException, it would be safer to revert. > Consider reverting HBASE-14893 Race between mutation on region and region > closing operation > --- > > Key: HBASE-18829 > URL: https://issues.apache.org/jira/browse/HBASE-18829 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 18829.v1.txt > > > HBASE-14893 was brought to attention by [~rajeshbabu] over PHOENIX-3111. > This issue is to consider reverting the fix from HBASE-14893 based on the > following observations: > * The closing boolean was intended to be acquired before taking the lock > ([~enis]) > * Phoenix local index has evolved over the years, the situation leading to > NotServingRegionException may not exist from Phoenix side > * Even if the situation still exists, downstream project (Phoenix) should > properly handle NotServingRegionException without change in locking scheme in > hbase -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18829) Consider reverting HBASE-14893 Race between mutation on region and region closing operation
[ https://issues.apache.org/jira/browse/HBASE-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168459#comment-16168459 ] Lars Hofhansl commented on HBASE-18829: --- Then again, this is since 27/Nov/15. So now I'd be worried about code implemented since then relying on this behavior. > Consider reverting HBASE-14893 Race between mutation on region and region > closing operation > --- > > Key: HBASE-18829 > URL: https://issues.apache.org/jira/browse/HBASE-18829 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 18829.v1.txt > > > HBASE-14893 was brought to attention by [~rajeshbabu] over PHOENIX-3111. > This issue is to consider reverting the fix from HBASE-14893 based on the > following observations: > * The closing boolean was intended to be acquired before taking the lock > ([~enis]) > * Phoenix local index has evolved over the years, the situation leading to > NotServingRegionException may not exist from Phoenix side > * Even if the situation still exists, downstream project (Phoenix) should > properly handle NotServingRegionException without change in locking scheme in > hbase -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-18797) Deprecate Filter#filterKeyValue and add Filter#filterCell
[ https://issues.apache.org/jira/browse/HBASE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18797 started by Tamas Penzes. > Deprecate Filter#filterKeyValue and add Filter#filterCell > - > > Key: HBASE-18797 > URL: https://issues.apache.org/jira/browse/HBASE-18797 > Project: HBase > Issue Type: Sub-task > Components: API, Filters >Reporter: Abhishek Singh Chouhan >Assignee: Tamas Penzes >Priority: Critical > Fix For: 2.0.0-alpha-3 > > > Part of filter package cleanup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18797) Deprecate Filter#filterKeyValue and add Filter#filterCell
[ https://issues.apache.org/jira/browse/HBASE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tamas Penzes reassigned HBASE-18797: Assignee: Tamas Penzes (was: Abhishek Singh Chouhan) > Deprecate Filter#filterKeyValue and add Filter#filterCell > - > > Key: HBASE-18797 > URL: https://issues.apache.org/jira/browse/HBASE-18797 > Project: HBase > Issue Type: Sub-task > Components: API, Filters >Reporter: Abhishek Singh Chouhan >Assignee: Tamas Penzes >Priority: Critical > Fix For: 2.0.0-alpha-3 > > > Part of filter package cleanup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18813) TestCanaryTool fails on branch-1 / branch-1.4
[
https://issues.apache.org/jira/browse/HBASE-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168448#comment-16168448
]
Hudson commented on HBASE-18813:
FAILURE: Integrated in Jenkins build HBase-1.5 #63 (See
[https://builds.apache.org/job/HBase-1.5/63/])
Revert "Amend HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4"
(apurtell: rev 469d6bf457c2c4d8ebe10c1e39004a6b9d907112)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
Amend HBASE-18813 TestCanaryTool fails on branch-1 / branch-1.4 (apurtell: rev
e5f80f36e2f32c44422d7d84482d77759075088a)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
> TestCanaryTool fails on branch-1 / branch-1.4
> --
>
> Key: HBASE-18813
> URL: https://issues.apache.org/jira/browse/HBASE-18813
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 1.4.0, 1.5.0
>
> Attachments: HBASE-18813-branch-1-addendum.patch,
> HBASE-18813-branch-1-addendum.patch, HBASE-18813-branch-1.patch
>
>
> Mocking error
> {noformat}
> Running org.apache.hadoop.hbase.tool.TestCanaryTool
> Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 77.873 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.tool.TestCanaryTool
> testReadTableTimeouts(org.apache.hadoop.hbase.tool.TestCanaryTool) Time
> elapsed: 13.845 sec <<< FAILURE!
> org.mockito.exceptions.verification.junit.ArgumentsAreDifferent:
> Argument(s) are different! Wanted:
> mockAppender.doAppend(
>
> );
> -> at
> org.apache.hadoop.hbase.tool.TestCanaryTool.testReadTableTimeouts(TestCanaryTool.java:150)
> Actual invocation has different arguments:
> mockAppender.doAppend(
> org.apache.log4j.spi.LoggingEvent@7f5fcfe9
> );
> -> at
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.tool.TestCanaryTool.testReadTableTimeouts(TestCanaryTool.java:150)
> Results :
> Failed tests:
> TestCanaryTool.testReadTableTimeouts:150
> Argument(s) are different! Wanted:
> mockAppender.doAppend(
>
> );
> -> at
> org.apache.hadoop.hbase.tool.TestCanaryTool.testReadTableTimeouts(TestCanaryTool.java:150)
> Actual invocation has different arguments:
> mockAppender.doAppend(
> org.apache.log4j.spi.LoggingEvent@7f5fcfe9
> );
> -> at
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18797) Deprecate Filter#filterKeyValue and add Filter#filterCell
[ https://issues.apache.org/jira/browse/HBASE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168438#comment-16168438 ] Tamas Penzes commented on HBASE-18797: -- What about the following: * filterKeyValue stays abstract but becomes deprecated in Filter (this is the main goal) * filterCell method added to Filter with a default behaviour that includes all except when we filter all remaining * override filterKeyValue method (still deprecated) in FilterBase with calling filterCell This way all the classes which extend Filter or FilterBase and override filterKeyValue (e.g. custom filters) stay backward compatible, but are able to use filterCell with default implementation if they wish. All classes extending FilterBase get filterKeyValue deprecated but calling their filterCell method (or the inherited one, if not implemented). filterCell is implemented in each Filters (renamed filterKeyValue methods). What do you think? > Deprecate Filter#filterKeyValue and add Filter#filterCell > - > > Key: HBASE-18797 > URL: https://issues.apache.org/jira/browse/HBASE-18797 > Project: HBase > Issue Type: Sub-task > Components: API, Filters >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Critical > Fix For: 2.0.0-alpha-3 > > > Part of filter package cleanup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18829) Consider reverting HBASE-14893 Race between mutation on region and region closing operation
[ https://issues.apache.org/jira/browse/HBASE-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18829: --- Assignee: Ted Yu Status: Patch Available (was: Open) > Consider reverting HBASE-14893 Race between mutation on region and region > closing operation > --- > > Key: HBASE-18829 > URL: https://issues.apache.org/jira/browse/HBASE-18829 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 18829.v1.txt > > > HBASE-14893 was brought to attention by [~rajeshbabu] over PHOENIX-3111. > This issue is to consider reverting the fix from HBASE-14893 based on the > following observations: > * The closing boolean was intended to be acquired before taking the lock > ([~enis]) > * Phoenix local index has evolved over the years, the situation leading to > NotServingRegionException may not exist from Phoenix side > * Even if the situation still exists, downstream project (Phoenix) should > properly handle NotServingRegionException without change in locking scheme in > hbase -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18829) Consider reverting HBASE-14893 Race between mutation on region and region closing operation
[ https://issues.apache.org/jira/browse/HBASE-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18829: --- Attachment: 18829.v1.txt > Consider reverting HBASE-14893 Race between mutation on region and region > closing operation > --- > > Key: HBASE-18829 > URL: https://issues.apache.org/jira/browse/HBASE-18829 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu > Attachments: 18829.v1.txt > > > HBASE-14893 was brought to attention by [~rajeshbabu] over PHOENIX-3111. > This issue is to consider reverting the fix from HBASE-14893 based on the > following observations: > * The closing boolean was intended to be acquired before taking the lock > ([~enis]) > * Phoenix local index has evolved over the years, the situation leading to > NotServingRegionException may not exist from Phoenix side > * Even if the situation still exists, downstream project (Phoenix) should > properly handle NotServingRegionException without change in locking scheme in > hbase -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18829) Consider reverting HBASE-14893 Race between mutation on region and region closing operation
[ https://issues.apache.org/jira/browse/HBASE-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168411#comment-16168411 ] Lars Hofhansl commented on HBASE-18829: --- My comment from HBASE-14893: +1 on reverting. The intention here was good. At the same time I think that accommodating this way in HBase for an issue in Phoenix is too dangerous. > Consider reverting HBASE-14893 Race between mutation on region and region > closing operation > --- > > Key: HBASE-18829 > URL: https://issues.apache.org/jira/browse/HBASE-18829 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu > > HBASE-14893 was brought to attention by [~rajeshbabu] over PHOENIX-3111. > This issue is to consider reverting the fix from HBASE-14893 based on the > following observations: > * The closing boolean was intended to be acquired before taking the lock > ([~enis]) > * Phoenix local index has evolved over the years, the situation leading to > NotServingRegionException may not exist from Phoenix side > * Even if the situation still exists, downstream project (Phoenix) should > properly handle NotServingRegionException without change in locking scheme in > hbase -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14893) Race between mutation on region and region closing operation leads to NotServingRegionException
[
https://issues.apache.org/jira/browse/HBASE-14893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168409#comment-16168409
]
Lars Hofhansl commented on HBASE-14893:
---
+1 on reverting.
The intention here was good.
At the same time I think that accommodating this way in HBase for an issue in
Phoenix is too dangerous.
> Race between mutation on region and region closing operation leads to
> NotServingRegionException
> ---
>
> Key: HBASE-14893
> URL: https://issues.apache.org/jira/browse/HBASE-14893
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17
>
> Attachments: 14893-v1.txt
>
>
> During system test involving Phoenix local index, we observed the following
> in region server log:
> {code}
> 2015-11-25 08:20:03,258 DEBUG
> [B.defaultRpcServer.handler=38,queue=2,port=49524] ipc.RpcServer:
> B.defaultRpcServer.handler=38,queue=2,port=49524: callId: 28 service:
> ClientService methodName: Scan size: 277 connection: 100.75.224.9:64138^M
> org.apache.hadoop.hbase.NotServingRegionException:
> GIGANTIC_TABLE,,1448439565197.0dc568cba621f11fd848ef87241d8535. is closing^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:7649)^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2803)^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2760)^M
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:140)^M
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:417)^M
> at
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:177)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$52.call(RegionCoprocessorHost.java:1318)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1748)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperationWithResult(RegionCoprocessorHost.java:1712)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1313)^M
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2259)^M
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)^M
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)^M
> {code}
> Here is related code snippet from UngroupedAggregateRegionObserver:
> {code}
> region.startRegionOperation();
> try {
> ...
> // Commit in batches based on
> UPSERT_BATCH_SIZE_ATTRIB in config
> if (!indexMutations.isEmpty() && batchSize > 0 &&
> indexMutations.size() % batchSize == 0) {
> commitBatch(region, indexMutations, null);
> {code}
> In startRegionOperation(), read lock on region was obtained. So region close
> should not proceed until the operation completes.
> However, we still got region closing because region#closing is set to true
> before write lock is taken in region#doClose() :
> {code}
> this.closing.set(true);
> status.setStatus("Disabling writes for close");
> // block waiting for the lock for closing
> lock.writeLock().lock();
> {code}
> Proposed fix is to obtain write lock first.
> Thanks to Rajeshbabu for offline discussion.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18824) Default timestamp for Put and Delete is not System.currentTimeMillis(), but Long.MAX_VALUE
[
https://issues.apache.org/jira/browse/HBASE-18824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168406#comment-16168406
]
Jerry He commented on HBASE-18824:
--
One thing that can be improved is to add meaningful comment for the field
HConstants.LATEST_TIMESTAMP to make it more clear for people who are into 'how
it works'.
> Default timestamp for Put and Delete is not System.currentTimeMillis(), but
> Long.MAX_VALUE
> --
>
> Key: HBASE-18824
> URL: https://issues.apache.org/jira/browse/HBASE-18824
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In http://hbase.apache.org/book.html#versions,
> 1. In chapter 27.2.4 Put
> bq. Doing a put always creates a new version of a cell, at a certain
> timestamp. {color:#205081}By default the system uses the server’s
> currentTimeMillis{color}, ...
> 2. In chapter 27.2.5 Delete
> bq. Deletes work by creating tombstone markers. For example, let’s suppose we
> want to delete a row. For this you can specify a version, or else
> {color:#205081}by default the currentTimeMillis is used.{color}...
> Checking the code, when timestamp is not specified,
> HConstants.LATEST_TIMESTAMP is used, which is Long.MAX_VALUE, rather than
> System.currentTimeMillis()
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-14893) Race between mutation on region and region closing operation leads to NotServingRegionException
[
https://issues.apache.org/jira/browse/HBASE-14893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168392#comment-16168392
]
Ted Yu commented on HBASE-14893:
Logged HBASE-18829 since this was integrated long time ago.
> Race between mutation on region and region closing operation leads to
> NotServingRegionException
> ---
>
> Key: HBASE-14893
> URL: https://issues.apache.org/jira/browse/HBASE-14893
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17
>
> Attachments: 14893-v1.txt
>
>
> During system test involving Phoenix local index, we observed the following
> in region server log:
> {code}
> 2015-11-25 08:20:03,258 DEBUG
> [B.defaultRpcServer.handler=38,queue=2,port=49524] ipc.RpcServer:
> B.defaultRpcServer.handler=38,queue=2,port=49524: callId: 28 service:
> ClientService methodName: Scan size: 277 connection: 100.75.224.9:64138^M
> org.apache.hadoop.hbase.NotServingRegionException:
> GIGANTIC_TABLE,,1448439565197.0dc568cba621f11fd848ef87241d8535. is closing^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:7649)^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2803)^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2760)^M
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:140)^M
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:417)^M
> at
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:177)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$52.call(RegionCoprocessorHost.java:1318)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1748)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperationWithResult(RegionCoprocessorHost.java:1712)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1313)^M
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2259)^M
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)^M
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)^M
> {code}
> Here is related code snippet from UngroupedAggregateRegionObserver:
> {code}
> region.startRegionOperation();
> try {
> ...
> // Commit in batches based on
> UPSERT_BATCH_SIZE_ATTRIB in config
> if (!indexMutations.isEmpty() && batchSize > 0 &&
> indexMutations.size() % batchSize == 0) {
> commitBatch(region, indexMutations, null);
> {code}
> In startRegionOperation(), read lock on region was obtained. So region close
> should not proceed until the operation completes.
> However, we still got region closing because region#closing is set to true
> before write lock is taken in region#doClose() :
> {code}
> this.closing.set(true);
> status.setStatus("Disabling writes for close");
> // block waiting for the lock for closing
> lock.writeLock().lock();
> {code}
> Proposed fix is to obtain write lock first.
> Thanks to Rajeshbabu for offline discussion.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18829) Consider reverting HBASE-14893 Race between mutation on region and region closing operation
Ted Yu created HBASE-18829: -- Summary: Consider reverting HBASE-14893 Race between mutation on region and region closing operation Key: HBASE-18829 URL: https://issues.apache.org/jira/browse/HBASE-18829 Project: HBase Issue Type: Bug Reporter: Ted Yu HBASE-14893 was brought to attention by [~rajeshbabu] over PHOENIX-3111. This issue is to consider reverting the fix from HBASE-14893 based on the following observations: * The closing boolean was intended to be acquired before taking the lock ([~enis]) * Phoenix local index has evolved over the years, the situation leading to NotServingRegionException may not exist from Phoenix side * Even if the situation still exists, downstream project (Phoenix) should properly handle NotServingRegionException without change in locking scheme in hbase -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18712) Specify -X for precommit unit tests
[ https://issues.apache.org/jira/browse/HBASE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168377#comment-16168377 ] stack commented on HBASE-18712: --- We should resolve this? Summary changed radically and then the suggestion of running with -X has pushback? > Specify -X for precommit unit tests > --- > > Key: HBASE-18712 > URL: https://issues.apache.org/jira/browse/HBASE-18712 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 18712.v1.txt, 18712.v2.txt, 18712.v3.txt > > > Add -X in dev-support/hbase-personality.sh for precommit unit tests so that > we have more information when "The forked VM terminated without saying > properly goodbye" happens again. > The following (initial proposal) doesn't apply to jdk 1.8 and has limited > benefit: > Currently hbase-surefire.argLine doesn't specify MaxPermSize for the test > run(s). > This sometimes resulted in mvn build prematurely exiting, leaving some large > tests behind. > The tests would be deemed timed out. > As indicated by the following post: > https://stackoverflow.com/questions/23260057/the-forked-vm-terminated-without-saying-properly-goodbye-vm-crash-or-system-exi > We should specify large enough MaxPermSize so that mvn build doesn't end > prematurely. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18772) [JDK8] Replace AtomicLong with LongAdder
[ https://issues.apache.org/jira/browse/HBASE-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18772: -- Fix Version/s: (was: 2.0.0-beta-1) 2.0.0-alpha-3 > [JDK8] Replace AtomicLong with LongAdder > - > > Key: HBASE-18772 > URL: https://issues.apache.org/jira/browse/HBASE-18772 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0-alpha-2 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Trivial > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-18772.master.patch, HBASE-18772.master-v2.patch, > HBASE-18772.master-v3.patch, HBASE-18772.master-v4.patch, > HBASE-18772.v0.addendum.patch > > > Currently we use many AtomicLong in HBase Region Code ，such as BucketCache > calss realCacheSize,heapSize,blockNumber,accessCount and HRegion calss > compactionNumBytesCompacted etc . > In JDK8 LongAdder is faster than AtomicLong ,should use this replace > AtomicLong -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-17980) Any HRegionInfo we give out should be immutable
[ https://issues.apache.org/jira/browse/HBASE-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-17980: --- Reopening so the addendum can be applied. Resolve after addendum goes in. The addendum is not in alpha-3. We can live w/ that. Thanks [~chia7712]. +1 on commit. > Any HRegionInfo we give out should be immutable > --- > > Key: HBASE-17980 > URL: https://issues.apache.org/jira/browse/HBASE-17980 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng > Labels: beginner > Fix For: 2.0.0-alpha-3 > > Attachments: HBASE-17980.master.001.patch, > HBASE-17980.master.002.patch, HBASE-17980.master.003.patch, > HBASE-17980.master.004.patch, HBASE-17980.master.005.patch, > HBASE-17980.master.006.patch, HBASE-17980.master.007.patch, > HBASE-17980.master.v0.patch, HBASE-17980.master.v1.patch, > HBASE-17980-master.v2.patch, HBASE-17980-master.v2.patch, > HBASE-17980.master.v3.patch, HBASE-17980.master.v4.patch, > HBASE-17980.master.v5.patch, HBASE-17980.master.v6.patch, > HBASE-17980.v0.addendum.patch > > > This is similar to HBASE-15583. > # Introduce RegionInfo class. HRegionInfo will extend RegionInfo. > # Deprecate HRegionInfo to be removed in 3.0 > # RegionInfo contain all of the read-only methods of HRegionInfo > # Add "RegionInfo Builder" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[ https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168356#comment-16168356 ] Andrew Purtell commented on HBASE-18796: [~abhishek.chouhan] Are there similar issues with the master and branch-2 branches? We will need changes/patches for them and commits there before commit to branch-1, unless they are not impacted. I think we need a new unit in TestEndToEndSplitTransaction that confirms the presence of this issue or not, and this should be committed everywhere. > Admin#isTableAvailable returns incorrect result before daughter regions are > opened > -- > > Key: HBASE-18796 > URL: https://issues.apache.org/jira/browse/HBASE-18796 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Attachments: HBASE-18796.branch-1.001.patch, > HBASE-18796.branch-1.001.patch > > > Admin#isTableAvailable checks if it can getServerName for the meta entries it > reads. During the time of split server location are added to the meta entries > in MetaTableAccessor#splitRegion although the description of the method says > "Does not add the location information to the daughter regions since they are > not open yet.". At this point during the split daughter regions are not > actually open, so we can get to a state where parent is offline, daughters > are not yet open but isTableAvailable returns true. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18641) Include block content verification logic used in lruCache in bucketCache
[ https://issues.apache.org/jira/browse/HBASE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168354#comment-16168354 ] Biju Nair commented on HBASE-18641: --- Hi @stack, [~ted_yu] any other comments on the branch-1 patch? > Include block content verification logic used in lruCache in bucketCache > > > Key: HBASE-18641 > URL: https://issues.apache.org/jira/browse/HBASE-18641 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair >Priority: Minor > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: HBASE-18641-branch-1.PATCH, HBASE-18641.PATCH, > HBASE-18641-V1.0.PATCH, HBASE-18641-WIP.PATCH > > > With off-heap/bucketCache being used to cache data blocks without going > through on-heap cache, the logic used in lruCache to check the content of > already cached block need to be included in bucketCache. Please see this > [discussion|https://mail-archives.apache.org/mod_mbox/hbase-dev/201708.mbox/%3cCAO40JLCnXLw3=0bbUaXdDx=w2fklljefvgj6-uvj_2jhfvo...@mail.gmail.com%3e] > for details. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[ https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168353#comment-16168353 ] Tianying Chang commented on HBASE-15400: thanks [~davelatham] for confirming! We will port the other two also. > Use DateTieredCompactor for Date Tiered Compaction > -- > > Key: HBASE-15400 > URL: https://issues.apache.org/jira/browse/HBASE-15400 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: HBASE-15400-0.98.patch, HBASE-15400-15389-v12.patch, > HBASE-15400-branch-1.patch, HBASE-15400.patch, HBASE-15400-v1.pa, > HBASE-15400-v3.patch, HBASE-15400-v3-v3.patch, HBASE-15400-v3-v4.patch, > HBASE-15400-v3-v5.patch, HBASE-15400-v6.patch, HBASE-15400-v7.patch > > > When we compact, we can output multiple files along the current window > boundaries. There are two use cases: > 1. Major compaction: We want to output date tiered store files with data > older than max age archived in trunks of the window size on the higher tier. > Once a window is old enough, we don't combine the windows to promote to the > next tier any further. So files in these windows retain the same timespan as > they were minor-compacted last time, which is the window size of the highest > tier. Major compaction will touch these files and we want to maintain the > same layout. This way, TTL and archiving will be simpler and more efficient. > 2. Bulk load files and the old file generated by major compaction before > upgrading to DTCP. > Pros: > 1. Restore locality, process versioning, updates and deletes while > maintaining the tiered layout. > 2. The best way to fix a skewed layout. > > This work is based on a prototype of DateTieredCompactor from HBASE-15389 and > focused on the part to meet needs for these two use cases while supporting > others. I have to call out a few design decisions: > 1. We only want to output the files along all windows for major compaction. > And we want to output multiple files older than max age in the trunks of the > maximum tier window size determined by base window size, windows per tier and > max age. > 2. For minor compaction, we don't want to output too many files, which will > remain around because of current restriction of contiguous compaction by seq > id. I will only output two files if all the files in the windows are being > combined, one for the data within window and the other for the out-of-window > tail. If there is any file in the window excluded from compaction, only one > file will be output from compaction. When the windows are promoted, the > situation of out of order data will gradually improve. For the incoming > window, we need to accommodate the case with user-specified future data. > 3. We have to pass the boundaries with the list of store file as a complete > time snapshot instead of two separate calls because window layout is > determined by the time the computation is called. So we will need new type of > compaction request. > 4. Since we will assign the same seq id for all output files, we need to sort > by maxTimestamp subsequently. Right now all compaction policy gets the files > sorted for StoreFileManager which sorts by seq id and other criteria. I will > use this order for DTCP only, to avoid impacting other compaction policies. > 5. We need some cleanup of current design of StoreEngine and CompactionPolicy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18652) Expose individual cache stats in a CombinedCache through JMX
[ https://issues.apache.org/jira/browse/HBASE-18652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168349#comment-16168349 ] Biju Nair commented on HBASE-18652: --- Hi @stack, @ted_yu, any other comments on the branch-1 patch or are we good? Please let me know. > Expose individual cache stats in a CombinedCache through JMX > > > Key: HBASE-18652 > URL: https://issues.apache.org/jira/browse/HBASE-18652 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Biju Nair >Assignee: Biju Nair > Fix For: 1.4.0, 1.5.0, 2.0.0-alpha-3 > > Attachments: 18652-branch-1-V3.0.PATCH, HBASE-18652-addendum.patch, > HBASE-18652-BRANCH-1.PATCH, HBASE-18652-branch-1-v1.0.patch, > HBASE-18652-branch-1-V2.0.PATCH, HBASE-18652-branch-1-V3.0.PATCH, > HBASE-18652-branch-1-V3.0.PATCH, HBASE-18652.PATCH, HBASE-18652-V1.0.PATCH, > HBASE-18652-V2.0.PATCH, HBASE-18652-WIP.PATCH > > > With offHeap cache being used to store data blocks and on-heap for index and > bloom filters, exposing the stats from the individual caches through JMX will > help understand the cache usage trend. Currently the combined cache stats is > available through JMX which may not provide insight into the individual cache > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[
https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168326#comment-16168326
]
Hadoop QA commented on HBASE-18796:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
56s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
57s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
46s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
45s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} hbase-client in the patch passed with JDK
v1.8.0_144. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} hbase-server-jdk1.8.0_144 with JDK v1.8.0_144
generated 0 new + 5 unchanged - 5 fixed = 5 total (was 10) {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} hbase-client in the patch passed with JDK
v1.7.0_151. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} hbase-server-jdk1.7.0_151 with JDK v1.7.0_151
generated 0 new + 5 unchanged - 5 fixed = 5 total (was 10) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 58s{color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} hbase-client-jdk1.8.0_144 with JDK v1.8.0_144
generated 0 new + 13 unchanged - 13 fixed = 13 total (was 26) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} hbase-server-jdk1.8.0_144 with JDK v1.8.0_144
generated 0 new + 3 unchanged - 3 fixed = 3 total (was 6) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} hbase-client-jdk1.7.0_151 with JDK v1.7.0_151
generated 0 new
[jira] [Commented] (HBASE-14893) Race between mutation on region and region closing operation leads to NotServingRegionException
[
https://issues.apache.org/jira/browse/HBASE-14893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168323#comment-16168323
]
Enis Soztutar commented on HBASE-14893:
---
Indeed the intention for closing boolean seems to be that it is acquired before
the lock. If we are reverting this, let's do it in another issue, since this
has been released already in a lot of releases.
> Race between mutation on region and region closing operation leads to
> NotServingRegionException
> ---
>
> Key: HBASE-14893
> URL: https://issues.apache.org/jira/browse/HBASE-14893
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17
>
> Attachments: 14893-v1.txt
>
>
> During system test involving Phoenix local index, we observed the following
> in region server log:
> {code}
> 2015-11-25 08:20:03,258 DEBUG
> [B.defaultRpcServer.handler=38,queue=2,port=49524] ipc.RpcServer:
> B.defaultRpcServer.handler=38,queue=2,port=49524: callId: 28 service:
> ClientService methodName: Scan size: 277 connection: 100.75.224.9:64138^M
> org.apache.hadoop.hbase.NotServingRegionException:
> GIGANTIC_TABLE,,1448439565197.0dc568cba621f11fd848ef87241d8535. is closing^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:7649)^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2803)^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2760)^M
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:140)^M
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:417)^M
> at
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:177)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$52.call(RegionCoprocessorHost.java:1318)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1748)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperationWithResult(RegionCoprocessorHost.java:1712)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1313)^M
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2259)^M
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)^M
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)^M
> {code}
> Here is related code snippet from UngroupedAggregateRegionObserver:
> {code}
> region.startRegionOperation();
> try {
> ...
> // Commit in batches based on
> UPSERT_BATCH_SIZE_ATTRIB in config
> if (!indexMutations.isEmpty() && batchSize > 0 &&
> indexMutations.size() % batchSize == 0) {
> commitBatch(region, indexMutations, null);
> {code}
> In startRegionOperation(), read lock on region was obtained. So region close
> should not proceed until the operation completes.
> However, we still got region closing because region#closing is set to true
> before write lock is taken in region#doClose() :
> {code}
> this.closing.set(true);
> status.setStatus("Disabling writes for close");
> // block waiting for the lock for closing
> lock.writeLock().lock();
> {code}
> Proposed fix is to obtain write lock first.
> Thanks to Rajeshbabu for offline discussion.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18767) Release hbase-2.0.0-alpha-3; Theme "Scrubbed API"
[
https://issues.apache.org/jira/browse/HBASE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168321#comment-16168321
]
stack commented on HBASE-18767:
---
High-level stuff done over last day or so to generate RC (mostly out of
refguide)
{code}
735 mvn clean install -DskipTests assembly:single
-Dassembly.file=hbase-assembly/src/main/assembly/src.xml -Prelease
# Now check that can build from the src tgz And that src layout looks right.
736 cd hbase-assembly/target/
737 tar xfz hbase-2.0.0-alpha3-src.tar.gz
738 cd hbase-2.0.0-alpha3
739 mvn clean install -DskipTests assembly:single
-Dassembly.file=hbase-assembly/src/main/assembly/src.xml -Prelease
{code}
Then...
{code}
744 mvn clean install -DskipTests -Prelease
746 mvn install -DskipTests site assembly:single -Prelease
# Check tgz basically works and looks right.
747 cd hbase-assembly/target/
749 tar xfz hbase-2.0.0-alpha3-bin.tar.gz
750 ls -la hbase-2.0.0-alpha3-bin.tar.gz
751 cd hbase-2.0.0-alpha3
752 ./bin/start-hbase.sh
753 export
JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/
754 ./bin/start-hbase.sh
755 tail -f logs/hbase-stack-master-kalashnikov.att.net.log
756 ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation
sequentialWrite 1
757 ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation sequentiaRead 1
758 ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation sequentialRead
1
759 vi logs/hbase-stack-master-kalashnikov.att.net.log
{code}
Tag and push to mvn
{code}
git tag -s 2.0.0-alpha-3RC0.2 -m "Second attempt at a tag on the first
2.0.0-alpha-3RC0; ignore the former in favor of this one; i.e. 2.0.0-alpha-3RC0"
770 git push --tags
771 mvn deploy -DskipTests -Papache-release -Prelease
{code}
Close repository up in mvn.
Do md5s, and sign.
{code}
629 for i in *.tar.gz; do echo $i; gpg --print-md SHA512 $i > $i.sha ; done
630 for i in *.tar.gz; do echo $i; gpg --armor --output $i.asc --detach-sig
$i ; done
631 for i in *.tar.gz; do echo $i; gpg --default-key 8ACC93D2 --armor
--output $i.asc --detach-sig $i ; done
632 gpg --verify hbase-2.0.0-alpha3-bin.tar.gz
633 gpg --verify hbase-2.0.0-alpha3-bin.tar.gz.asc
{code}
svn commit it all in a hbase-2.0.0-alpha-3RC0 dir at this location
{code}
$ svn info
Path: .
Working Copy Root Path: /Users/stack/checkouts/hbase.dev
URL: https://dist.apache.org/repos/dist/dev/hbase
Relative URL: ^/dev/hbase
Repository Root: https://dist.apache.org/repos/dist
Repository UUID: 0d268c88-bc11-4956-87df-91683dc98e59
Revision: 21246
Node Kind: directory
Schedule: normal
Last Changed Author: stack
Last Changed Rev: 21196
Last Changed Date: 2017-08-16 14:00:31 -0700 (Wed, 16 Aug 2017)
{code}
> Release hbase-2.0.0-alpha-3; Theme "Scrubbed API"
> -
>
> Key: HBASE-18767
> URL: https://issues.apache.org/jira/browse/HBASE-18767
> Project: HBase
> Issue Type: Task
>Reporter: stack
> Fix For: 2.0.0-alpha-3
>
>
> From the dev mail thread: "Moving 2.0 forward", the theme is solidifying API.
> I listed a bunch of API JIRAs to address. [~mdrob] nicely tagged them all w/
> the 2.0.0-alpha-3 fix version. This issue is for pushing out alpha-3.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18828) [2.0] Generate CHANGES.txt
stack created HBASE-18828: - Summary: [2.0] Generate CHANGES.txt Key: HBASE-18828 URL: https://issues.apache.org/jira/browse/HBASE-18828 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Priority: Blocker See [~busbey] writeup on generating CHANGES.txt here HBASE-14025 (fold it into refguide while at it..) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17553) Make a 2.0.0 Release
[ https://issues.apache.org/jira/browse/HBASE-17553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168312#comment-16168312 ] stack commented on HBASE-17553: --- I was going to put up a JIRA that we remove the CHANGES.txt file but then came across this, HBASE-14025. TODO: Making subtask to update CHANGES.txt for 2.0. > Make a 2.0.0 Release > > > Key: HBASE-17553 > URL: https://issues.apache.org/jira/browse/HBASE-17553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > > Umbrella issue to keep account of tasks to make a 2.0.0 release. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18820) assembly is missing hbase-replication
[ https://issues.apache.org/jira/browse/HBASE-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18820: -- Summary: assembly is missing hbase-replication (was: assembly is missing hbase-permission) > assembly is missing hbase-replication > - > > Key: HBASE-18820 > URL: https://issues.apache.org/jira/browse/HBASE-18820 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-3 > > Attachments: 18820.txt > > > Means can't build from src tgz. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18767) Release hbase-2.0.0-alpha-3; Theme "Scrubbed API"
[ https://issues.apache.org/jira/browse/HBASE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168304#comment-16168304 ] stack commented on HBASE-18767: --- Pushed RC0 this morning. tgz is 180MB. Needs a diet. > Release hbase-2.0.0-alpha-3; Theme "Scrubbed API" > - > > Key: HBASE-18767 > URL: https://issues.apache.org/jira/browse/HBASE-18767 > Project: HBase > Issue Type: Task >Reporter: stack > Fix For: 2.0.0-alpha-3 > > > From the dev mail thread: "Moving 2.0 forward", the theme is solidifying API. > I listed a bunch of API JIRAs to address. [~mdrob] nicely tagged them all w/ > the 2.0.0-alpha-3 fix version. This issue is for pushing out alpha-3. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168299#comment-16168299 ] stack commented on HBASE-14004: --- This is a great fix. > [Replication] Inconsistency between Memstore and WAL may result in data in > remote cluster that is not in the origin > --- > > Key: HBASE-14004 > URL: https://issues.apache.org/jira/browse/HBASE-14004 > Project: HBase > Issue Type: Bug > Components: regionserver, Replication >Reporter: He Liangliang >Assignee: Duo Zhang >Priority: Critical > Labels: replication, wal > Fix For: 3.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-14004.patch, HBASE-14004-v1.patch, > HBASE-14004-v2.patch, HBASE-14004-v2.patch, HBASE-14004-v3.patch > > > Looks like the current write path can cause inconsistency between > memstore/hfile and WAL which cause the slave cluster has more data than the > master cluster. > The simplified write path looks like: > 1. insert record into Memstore > 2. write record to WAL > 3. sync WAL > 4. rollback Memstore if 3 fails > It's possible that the HDFS sync RPC call fails, but the data is already > (may partially) transported to the DNs which finally get persisted. As a > result, the handler will rollback the Memstore and the later flushed HFile > will also skip this record. > == > This is a long lived issue. The above problem is solved by write path > reorder, as now we will sync wal first before modifying memstore. But the > problem may still exists as replication thread may read the new data before > we return from hflush. See this document for more details: > https://docs.google.com/document/d/11AyWtGhItQs6vsLRIx32PwTxmBY3libXwGXI25obVEY/edit# > So we need to keep a sync length in WAL and tell replication wal reader this > is limit when you read this wal file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14893) Race between mutation on region and region closing operation leads to NotServingRegionException
[
https://issues.apache.org/jira/browse/HBASE-14893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168284#comment-16168284
]
stack commented on HBASE-14893:
---
Even with a test, this is crazy stuff. Seems like Phoenix now agrees this was a
mistake having suffered the consequence of taking our locks along vectors that
don't align w/ how we intended their use. There is a move to revert this
change. I'd be in favor.
> Race between mutation on region and region closing operation leads to
> NotServingRegionException
> ---
>
> Key: HBASE-14893
> URL: https://issues.apache.org/jira/browse/HBASE-14893
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17
>
> Attachments: 14893-v1.txt
>
>
> During system test involving Phoenix local index, we observed the following
> in region server log:
> {code}
> 2015-11-25 08:20:03,258 DEBUG
> [B.defaultRpcServer.handler=38,queue=2,port=49524] ipc.RpcServer:
> B.defaultRpcServer.handler=38,queue=2,port=49524: callId: 28 service:
> ClientService methodName: Scan size: 277 connection: 100.75.224.9:64138^M
> org.apache.hadoop.hbase.NotServingRegionException:
> GIGANTIC_TABLE,,1448439565197.0dc568cba621f11fd848ef87241d8535. is closing^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:7649)^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2803)^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2760)^M
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:140)^M
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:417)^M
> at
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:177)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$52.call(RegionCoprocessorHost.java:1318)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1748)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperationWithResult(RegionCoprocessorHost.java:1712)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1313)^M
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2259)^M
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)^M
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)^M
> {code}
> Here is related code snippet from UngroupedAggregateRegionObserver:
> {code}
> region.startRegionOperation();
> try {
> ...
> // Commit in batches based on
> UPSERT_BATCH_SIZE_ATTRIB in config
> if (!indexMutations.isEmpty() && batchSize > 0 &&
> indexMutations.size() % batchSize == 0) {
> commitBatch(region, indexMutations, null);
> {code}
> In startRegionOperation(), read lock on region was obtained. So region close
> should not proceed until the operation completes.
> However, we still got region closing because region#closing is set to true
> before write lock is taken in region#doClose() :
> {code}
> this.closing.set(true);
> status.setStatus("Disabling writes for close");
> // block waiting for the lock for closing
> lock.writeLock().lock();
> {code}
> Proposed fix is to obtain write lock first.
> Thanks to Rajeshbabu for offline discussion.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-14893) Race between mutation on region and region closing operation leads to NotServingRegionException
[
https://issues.apache.org/jira/browse/HBASE-14893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168266#comment-16168266
]
Ted Yu commented on HBASE-14893:
Sorry for late response.
bq. This is crazy stuff with coprocessor taking out internal region lock. It
should have a test
Agreed test should be added. Though it may be easier to come up with test from
Phoenix side.
Over in PHOENIX-4214, Vincent seems to have some test.
> Race between mutation on region and region closing operation leads to
> NotServingRegionException
> ---
>
> Key: HBASE-14893
> URL: https://issues.apache.org/jira/browse/HBASE-14893
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17
>
> Attachments: 14893-v1.txt
>
>
> During system test involving Phoenix local index, we observed the following
> in region server log:
> {code}
> 2015-11-25 08:20:03,258 DEBUG
> [B.defaultRpcServer.handler=38,queue=2,port=49524] ipc.RpcServer:
> B.defaultRpcServer.handler=38,queue=2,port=49524: callId: 28 service:
> ClientService methodName: Scan size: 277 connection: 100.75.224.9:64138^M
> org.apache.hadoop.hbase.NotServingRegionException:
> GIGANTIC_TABLE,,1448439565197.0dc568cba621f11fd848ef87241d8535. is closing^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:7649)^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2803)^M
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2760)^M
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:140)^M
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:417)^M
> at
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:177)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$52.call(RegionCoprocessorHost.java:1318)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1748)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperationWithResult(RegionCoprocessorHost.java:1712)^M
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1313)^M
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2259)^M
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)^M
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)^M
> {code}
> Here is related code snippet from UngroupedAggregateRegionObserver:
> {code}
> region.startRegionOperation();
> try {
> ...
> // Commit in batches based on
> UPSERT_BATCH_SIZE_ATTRIB in config
> if (!indexMutations.isEmpty() && batchSize > 0 &&
> indexMutations.size() % batchSize == 0) {
> commitBatch(region, indexMutations, null);
> {code}
> In startRegionOperation(), read lock on region was obtained. So region close
> should not proceed until the operation completes.
> However, we still got region closing because region#closing is set to true
> before write lock is taken in region#doClose() :
> {code}
> this.closing.set(true);
> status.setStatus("Disabling writes for close");
> // block waiting for the lock for closing
> lock.writeLock().lock();
> {code}
> Proposed fix is to obtain write lock first.
> Thanks to Rajeshbabu for offline discussion.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18813) TestCanaryTool fails on branch-1 / branch-1.4
[
https://issues.apache.org/jira/browse/HBASE-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-18813:
---
Attachment: HBASE-18813-branch-1-addendum.patch
Now intermittently failing still with JRE 8. Disable the offending units.
No failures now with 10 iterations.
> TestCanaryTool fails on branch-1 / branch-1.4
> --
>
> Key: HBASE-18813
> URL: https://issues.apache.org/jira/browse/HBASE-18813
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 1.4.0, 1.5.0
>
> Attachments: HBASE-18813-branch-1-addendum.patch,
> HBASE-18813-branch-1-addendum.patch, HBASE-18813-branch-1.patch
>
>
> Mocking error
> {noformat}
> Running org.apache.hadoop.hbase.tool.TestCanaryTool
> Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 77.873 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.tool.TestCanaryTool
> testReadTableTimeouts(org.apache.hadoop.hbase.tool.TestCanaryTool) Time
> elapsed: 13.845 sec <<< FAILURE!
> org.mockito.exceptions.verification.junit.ArgumentsAreDifferent:
> Argument(s) are different! Wanted:
> mockAppender.doAppend(
>
> );
> -> at
> org.apache.hadoop.hbase.tool.TestCanaryTool.testReadTableTimeouts(TestCanaryTool.java:150)
> Actual invocation has different arguments:
> mockAppender.doAppend(
> org.apache.log4j.spi.LoggingEvent@7f5fcfe9
> );
> -> at
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.tool.TestCanaryTool.testReadTableTimeouts(TestCanaryTool.java:150)
> Results :
> Failed tests:
> TestCanaryTool.testReadTableTimeouts:150
> Argument(s) are different! Wanted:
> mockAppender.doAppend(
>
> );
> -> at
> org.apache.hadoop.hbase.tool.TestCanaryTool.testReadTableTimeouts(TestCanaryTool.java:150)
> Actual invocation has different arguments:
> mockAppender.doAppend(
> org.apache.log4j.spi.LoggingEvent@7f5fcfe9
> );
> -> at
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18827) Site build takes 1.5 hours; it looks like it is stuck...
stack created HBASE-18827: - Summary: Site build takes 1.5 hours; it looks like it is stuck... Key: HBASE-18827 URL: https://issues.apache.org/jira/browse/HBASE-18827 Project: HBase Issue Type: Bug Components: site Reporter: stack Priority: Minor I'm trying to make a release. I'm in a tizzy as is usual around these times*. 1.5 hours seems totally over-the-top. I think [~misty]'s lovely automation has hidden this fact from us but needs digging on why we are taking so long. The cycle seems to be provoked by hbase-archetypes module but I got 'mvn log glaze disease' as soon as I tried digging in. Filing issue in case someone else wants to have a go at this before I. Also filing if only to note that the thing does eventually finish in case I forget * I go to build the doc/site and the build never seems to end. First there is the issue HBASE-18821 where we were NPE'ing after a bunch of time had passed. My fix for HBASE-18821 then got me further but after what seemed hours, it failed with a cryptic message (Thanks [~Apache9] for figuring that I'd made an incorrect fix over in HBASE-18821). Next up I'm looking at a cycle that never seems to end... only it eventually does after 90minutes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15400) Use DateTieredCompactor for Date Tiered Compaction
[ https://issues.apache.org/jira/browse/HBASE-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168113#comment-16168113 ] Dave Latham commented on HBASE-15400: - I believe that HBASE-15181 alone would work, but it would not support maintaining a date tiered layout along with major compactions. Major compactions would result in a single file per store, but new flushes from that point on would begin date tiering of files again. We ran it in production for a short time before moving to the version with both HBASE-15389 and HBASE-15400. I would recommend including those two as well - I would expect the branch-1 patches to be pretty easy to port to 1.2. > Use DateTieredCompactor for Date Tiered Compaction > -- > > Key: HBASE-15400 > URL: https://issues.apache.org/jira/browse/HBASE-15400 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: Clara Xiong >Assignee: Clara Xiong > Fix For: 2.0.0, 1.3.0, 0.98.19 > > Attachments: HBASE-15400-0.98.patch, HBASE-15400-15389-v12.patch, > HBASE-15400-branch-1.patch, HBASE-15400.patch, HBASE-15400-v1.pa, > HBASE-15400-v3.patch, HBASE-15400-v3-v3.patch, HBASE-15400-v3-v4.patch, > HBASE-15400-v3-v5.patch, HBASE-15400-v6.patch, HBASE-15400-v7.patch > > > When we compact, we can output multiple files along the current window > boundaries. There are two use cases: > 1. Major compaction: We want to output date tiered store files with data > older than max age archived in trunks of the window size on the higher tier. > Once a window is old enough, we don't combine the windows to promote to the > next tier any further. So files in these windows retain the same timespan as > they were minor-compacted last time, which is the window size of the highest > tier. Major compaction will touch these files and we want to maintain the > same layout. This way, TTL and archiving will be simpler and more efficient. > 2. Bulk load files and the old file generated by major compaction before > upgrading to DTCP. > Pros: > 1. Restore locality, process versioning, updates and deletes while > maintaining the tiered layout. > 2. The best way to fix a skewed layout. > > This work is based on a prototype of DateTieredCompactor from HBASE-15389 and > focused on the part to meet needs for these two use cases while supporting > others. I have to call out a few design decisions: > 1. We only want to output the files along all windows for major compaction. > And we want to output multiple files older than max age in the trunks of the > maximum tier window size determined by base window size, windows per tier and > max age. > 2. For minor compaction, we don't want to output too many files, which will > remain around because of current restriction of contiguous compaction by seq > id. I will only output two files if all the files in the windows are being > combined, one for the data within window and the other for the out-of-window > tail. If there is any file in the window excluded from compaction, only one > file will be output from compaction. When the windows are promoted, the > situation of out of order data will gradually improve. For the incoming > window, we need to accommodate the case with user-specified future data. > 3. We have to pass the boundaries with the list of store file as a complete > time snapshot instead of two separate calls because window layout is > determined by the time the computation is called. So we will need new type of > compaction request. > 4. Since we will assign the same seq id for all output files, we need to sort > by maxTimestamp subsequently. Right now all compaction policy gets the files > sorted for StoreFileManager which sorts by seq id and other criteria. I will > use this order for DTCP only, to avoid impacting other compaction policies. > 5. We need some cleanup of current design of StoreEngine and CompactionPolicy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18446) Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix)
[ https://issues.apache.org/jira/browse/HBASE-18446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168100#comment-16168100 ] stack commented on HBASE-18446: --- I added the comment " I think for most cases filtering on the final RegionScanner is enough, and it is dangerous to create a custom store file level scanner as our correctness highly depends on the complicated SQM which is evaluated after fetching kvs from store file level scanner." from [~Apache9] to the high-level on why CPs have to change in 2.0 in our 'hbase 2.0 book'. You say it well Duo. We should change your 'i think' to an assertion by us -- but first running it by the Phoenix crew > Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix) > --- > > Key: HBASE-18446 > URL: https://issues.apache.org/jira/browse/HBASE-18446 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18446.patch, HBASE-18446-v1.patch > > > Do not see any reason why it is marked as IA.LimitedPrivate. It is not > referenced in any CPs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[ https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168079#comment-16168079 ] Reid Chan commented on HBASE-18651: --- I can code another version to see if meets your needs. > Let ChaosMonkeyRunner expose the chaos monkey runner it creates > --- > > Key: HBASE-18651 > URL: https://issues.apache.org/jira/browse/HBASE-18651 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Reid Chan > Attachments: HBASE-18651.master.001.patch, > HBASE-18651.master.002.patch > > > Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without > keeping track of the instance. > This poses some challenge when ChaosMonkeyRunner is used programmatically > because the caller cannot get hold of the runner. > As [~mdrob] suggested, we should expose the chaos monkey runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[
https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168027#comment-16168027
]
Reid Chan edited comment on HBASE-18651 at 9/15/17 3:56 PM:
I have observed other classes in hbase-it, none of them has annotation for
audience.
{{ChaosMonkeyRunner}} in {{Monkeys}} already extends {{AbstractHBaseTool}} and
implements required methods.
{{IntegrationTestMonkeys}} doesn't have to extends {{IntegrationTestBase}}
again, this test, just proving it has controls on {{ChaosMonkeyRunner}}, is
enough in my opinions.
was (Author: reidchan):
I have observed other classes in hbase-it, none of them has annotation for
audience.
{{ChaosMonkeyRunner}} in {{Monkeys}} already extends {{AbstractHBaseTool}} and
implements required methods.
{{Monkeys}} don't have to extends {{IntegrationTestBase}} again, this test,
just proving it has controls on {{ChaosMonkeyRunner}}, is enough in my opinions.
> Let ChaosMonkeyRunner expose the chaos monkey runner it creates
> ---
>
> Key: HBASE-18651
> URL: https://issues.apache.org/jira/browse/HBASE-18651
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Reid Chan
> Attachments: HBASE-18651.master.001.patch,
> HBASE-18651.master.002.patch
>
>
> Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without
> keeping track of the instance.
> This poses some challenge when ChaosMonkeyRunner is used programmatically
> because the caller cannot get hold of the runner.
> As [~mdrob] suggested, we should expose the chaos monkey runner.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[
https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168001#comment-16168001
]
Ted Yu edited comment on HBASE-18651 at 9/15/17 3:54 PM:
-
For the IntegrationTestMonkeys.java, can you take a look at:
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestAcidGuarantees.java
{code}
public class IntegrationTestAcidGuarantees extends IntegrationTestBase {
{code}
was (Author: yuzhih...@gmail.com):
For the integration test, can you take a look at:
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestAcidGuarantees.java
{code}
public class IntegrationTestAcidGuarantees extends IntegrationTestBase {
{code}
> Let ChaosMonkeyRunner expose the chaos monkey runner it creates
> ---
>
> Key: HBASE-18651
> URL: https://issues.apache.org/jira/browse/HBASE-18651
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Reid Chan
> Attachments: HBASE-18651.master.001.patch,
> HBASE-18651.master.002.patch
>
>
> Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without
> keeping track of the instance.
> This poses some challenge when ChaosMonkeyRunner is used programmatically
> because the caller cannot get hold of the runner.
> As [~mdrob] suggested, we should expose the chaos monkey runner.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168034#comment-16168034 ] Hudson commented on HBASE-14004: FAILURE: Integrated in Jenkins build HBase-2.0 #518 (See [https://builds.apache.org/job/HBase-2.0/518/]) HBASE-14004 [Replication] Inconsistency between Memstore and WAL may (zhangduo: rev d90f77ab7dc46a19893786b6a740bc73470fd779) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALProvider.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestWALEntryStream.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/wal/IOTestProvider.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/DisabledWALProvider.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALFactory.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALEntryStream.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALFileLengthProvider.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/Threads.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSource.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReplicationService.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RegionGroupingProvider.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RecoveredReplicationSource.java > [Replication] Inconsistency between Memstore and WAL may result in data in > remote cluster that is not in the origin > --- > > Key: HBASE-14004 > URL: https://issues.apache.org/jira/browse/HBASE-14004 > Project: HBase > Issue Type: Bug > Components: regionserver, Replication >Reporter: He Liangliang >Assignee: Duo Zhang >Priority: Critical > Labels: replication, wal > Fix For: 3.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-14004.patch, HBASE-14004-v1.patch, > HBASE-14004-v2.patch, HBASE-14004-v2.patch, HBASE-14004-v3.patch > > > Looks like the current write path can cause inconsistency between > memstore/hfile and WAL which cause the slave cluster has more data than the > master cluster. > The simplified write path looks like: > 1. insert record into Memstore > 2. write record to WAL > 3. sync WAL > 4. rollback Memstore if 3 fails > It's possible that the HDFS sync RPC call fails, but the data is already > (may partially) transported to the DNs which finally get persisted. As a > result, the handler will rollback the Memstore and the later flushed HFile > will also skip this record. > == > This is a long lived issue. The above problem is solved by write path > reorder, as now we will sync wal first before modifying memstore. But the > problem may still exists as replication thread may read the new data before > we return from hflush. See this document for more details: > https://docs.google.com/document/d/11AyWtGhItQs6vsLRIx32PwTxmBY3libXwGXI25obVEY/edit# > So we need to keep a sync length in WAL and tell replication wal reader this > is limit when you read this wal file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18446) Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix)
[ https://issues.apache.org/jira/browse/HBASE-18446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168035#comment-16168035 ] Hudson commented on HBASE-18446: FAILURE: Integrated in Jenkins build HBase-2.0 #518 (See [https://builds.apache.org/job/HBase-2.0/518/]) HBASE-18446 Mark StoreFileScanner/StoreFileReader as (zhangduo: rev a5c8461ca87d6324f16ffd126b765146fdd5315a) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileReader.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java > Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix) > --- > > Key: HBASE-18446 > URL: https://issues.apache.org/jira/browse/HBASE-18446 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18446.patch, HBASE-18446-v1.patch > > > Do not see any reason why it is marked as IA.LimitedPrivate. It is not > referenced in any CPs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[
https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168027#comment-16168027
]
Reid Chan commented on HBASE-18651:
---
I have observed other classes in hbase-it, none of them has annotation for
audience.
{{ChaosMonkeyRunner}} in {{Monkeys}} already extends {{AbstractHBaseTool}} and
implements required methods.
{{Monkeys}} don't have to extends {{IntegrationTestBase}} again, this test,
just proving it has controls on {{ChaosMonkeyRunner}}, is enough in my opinions.
> Let ChaosMonkeyRunner expose the chaos monkey runner it creates
> ---
>
> Key: HBASE-18651
> URL: https://issues.apache.org/jira/browse/HBASE-18651
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Reid Chan
> Attachments: HBASE-18651.master.001.patch,
> HBASE-18651.master.002.patch
>
>
> Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without
> keeping track of the instance.
> This poses some challenge when ChaosMonkeyRunner is used programmatically
> because the caller cannot get hold of the runner.
> As [~mdrob] suggested, we should expose the chaos monkey runner.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18800) [Propaganda] Push shaded hbase-client as gateway to an hbase cluster going forward
[ https://issues.apache.org/jira/browse/HBASE-18800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168019#comment-16168019 ] stack commented on HBASE-18800: --- [~mantonov] Good question sir. I presumed we'd talked this out already but it has always been a side-show. Let me do a thread w/ this proposal as foreground. Thanks sir (Need to also mention Elliott's downside using shaded client...) > [Propaganda] Push shaded hbase-client as gateway to an hbase cluster going > forward > -- > > Key: HBASE-18800 > URL: https://issues.apache.org/jira/browse/HBASE-18800 > Project: HBase > Issue Type: Bug >Reporter: stack > > We've bandied this about for a while now that folks should consume hbase via > the shaded hbase-client; it should work if their needs are minimal (and if it > doesn't work, would be good to hear why). This issue is about evangelizing > the shaded hbase-client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18800) [Propaganda] Push shaded hbase-client as gateway to an hbase cluster going forward
[ https://issues.apache.org/jira/browse/HBASE-18800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168011#comment-16168011 ] Mikhail Antonov commented on HBASE-18800: - Hmm why is that a jira and not a discussion on a dev list, is there supposed to be patch or something? (I support the idea of pushing shaded client as the main client) > [Propaganda] Push shaded hbase-client as gateway to an hbase cluster going > forward > -- > > Key: HBASE-18800 > URL: https://issues.apache.org/jira/browse/HBASE-18800 > Project: HBase > Issue Type: Bug >Reporter: stack > > We've bandied this about for a while now that folks should consume hbase via > the shaded hbase-client; it should work if their needs are minimal (and if it > doesn't work, would be good to hear why). This issue is about evangelizing > the shaded hbase-client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18798) Remove the unused methods in RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-18798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18798: --- Status: Patch Available (was: Open) v1: rebase > Remove the unused methods in RegionServerObserver > - > > Key: HBASE-18798 > URL: https://issues.apache.org/jira/browse/HBASE-18798 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18798.v0.patch, HBASE-18798.v0.patch, > HBASE-18798.v0.patch, HBASE-18798.v1.patch > > > # preRollBackMerge > # postRollBackMerge > # preMergeCommit > # postMergeCommit > # postMerge > # preMerge > HBASE-17470 drop the rs-side merge code -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18798) Remove the unused methods in RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-18798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18798: --- Attachment: HBASE-18798.v1.patch > Remove the unused methods in RegionServerObserver > - > > Key: HBASE-18798 > URL: https://issues.apache.org/jira/browse/HBASE-18798 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18798.v0.patch, HBASE-18798.v0.patch, > HBASE-18798.v0.patch, HBASE-18798.v1.patch > > > # preRollBackMerge > # postRollBackMerge > # preMergeCommit > # postMergeCommit > # postMerge > # preMerge > HBASE-17470 drop the rs-side merge code -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18798) Remove the unused methods in RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-18798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18798: --- Status: Open (was: Patch Available) > Remove the unused methods in RegionServerObserver > - > > Key: HBASE-18798 > URL: https://issues.apache.org/jira/browse/HBASE-18798 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18798.v0.patch, HBASE-18798.v0.patch, > HBASE-18798.v0.patch > > > # preRollBackMerge > # postRollBackMerge > # preMergeCommit > # postMergeCommit > # postMerge > # preMerge > HBASE-17470 drop the rs-side merge code -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[
https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168001#comment-16168001
]
Ted Yu commented on HBASE-18651:
For the integration test, can you take a look at:
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestAcidGuarantees.java
{code}
public class IntegrationTestAcidGuarantees extends IntegrationTestBase {
{code}
> Let ChaosMonkeyRunner expose the chaos monkey runner it creates
> ---
>
> Key: HBASE-18651
> URL: https://issues.apache.org/jira/browse/HBASE-18651
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Reid Chan
> Attachments: HBASE-18651.master.001.patch,
> HBASE-18651.master.002.patch
>
>
> Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without
> keeping track of the instance.
> This poses some challenge when ChaosMonkeyRunner is used programmatically
> because the caller cannot get hold of the runner.
> As [~mdrob] suggested, we should expose the chaos monkey runner.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18446) Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix)
[ https://issues.apache.org/jira/browse/HBASE-18446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167999#comment-16167999 ] Hudson commented on HBASE-18446: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3719 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3719/]) HBASE-18446 Mark StoreFileScanner/StoreFileReader as (zhangduo: rev a6d8cedb06eecbdb1d15e8067d470f5b871187c1) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileReader.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java > Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix) > --- > > Key: HBASE-18446 > URL: https://issues.apache.org/jira/browse/HBASE-18446 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18446.patch, HBASE-18446-v1.patch > > > Do not see any reason why it is marked as IA.LimitedPrivate. It is not > referenced in any CPs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167998#comment-16167998 ] Hudson commented on HBASE-14004: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3719 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3719/]) HBASE-14004 [Replication] Inconsistency between Memstore and WAL may (zhangduo: rev 4341c3f554cf85e73d3bb536bdda33a83f463f16) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/DisabledWALProvider.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestWALEntryStream.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALEntryStream.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSource.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReplicationService.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/Threads.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALFactory.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALFileLengthProvider.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALProvider.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RegionGroupingProvider.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RecoveredReplicationSource.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/wal/IOTestProvider.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > [Replication] Inconsistency between Memstore and WAL may result in data in > remote cluster that is not in the origin > --- > > Key: HBASE-14004 > URL: https://issues.apache.org/jira/browse/HBASE-14004 > Project: HBase > Issue Type: Bug > Components: regionserver, Replication >Reporter: He Liangliang >Assignee: Duo Zhang >Priority: Critical > Labels: replication, wal > Fix For: 3.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-14004.patch, HBASE-14004-v1.patch, > HBASE-14004-v2.patch, HBASE-14004-v2.patch, HBASE-14004-v3.patch > > > Looks like the current write path can cause inconsistency between > memstore/hfile and WAL which cause the slave cluster has more data than the > master cluster. > The simplified write path looks like: > 1. insert record into Memstore > 2. write record to WAL > 3. sync WAL > 4. rollback Memstore if 3 fails > It's possible that the HDFS sync RPC call fails, but the data is already > (may partially) transported to the DNs which finally get persisted. As a > result, the handler will rollback the Memstore and the later flushed HFile > will also skip this record. > == > This is a long lived issue. The above problem is solved by write path > reorder, as now we will sync wal first before modifying memstore. But the > problem may still exists as replication thread may read the new data before > we return from hflush. See this document for more details: > https://docs.google.com/document/d/11AyWtGhItQs6vsLRIx32PwTxmBY3libXwGXI25obVEY/edit# > So we need to keep a sync length in WAL and tell replication wal reader this > is limit when you read this wal file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[ https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167984#comment-16167984 ] Mikhail Antonov commented on HBASE-18090: - Looks good to me! The only thing that would be good to do is probably to add some tests for the new methods in the RegionSplitter and the SplitAlgos. That looks actually backward compatible and self-contained enough that it can go to branch-1 (cc [~apurtell]). I'd consider it for 1.3.2 port - I think risk-benefit ratio is good. [~ghelmling] [~ashu210890] you may be interested too. > Improve TableSnapshotInputFormat to allow more multiple mappers per region > -- > > Key: HBASE-18090 > URL: https://issues.apache.org/jira/browse/HBASE-18090 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 1.4.0 >Reporter: Mikhail Antonov >Assignee: xinxin fan > Attachments: HBASE-18090-branch-1.3-v1.patch, > HBASE-18090-branch-1.3-v2.patch, HBASE-18090-V3-master.patch > > > TableSnapshotInputFormat runs one map task per region in the table snapshot. > This places unnecessary restriction that the region layout of the original > table needs to take the processing resources available to MR job into > consideration. Allowing to run multiple mappers per region (assuming > reasonably even key distribution) would be useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[
https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167982#comment-16167982
]
Ted Yu commented on HBASE-18651:
{code}
+public class Monkeys implements Closeable {
{code}
Add annotation for audience.
{code}
+} catch (InterruptedException e) {
+ LOG.warn("Interruption occured while stopping chaos monkeys " + e);
{code}
Restore interrupt status.
> Let ChaosMonkeyRunner expose the chaos monkey runner it creates
> ---
>
> Key: HBASE-18651
> URL: https://issues.apache.org/jira/browse/HBASE-18651
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Reid Chan
> Attachments: HBASE-18651.master.001.patch,
> HBASE-18651.master.002.patch
>
>
> Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without
> keeping track of the instance.
> This poses some challenge when ChaosMonkeyRunner is used programmatically
> because the caller cannot get hold of the runner.
> As [~mdrob] suggested, we should expose the chaos monkey runner.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-18816) Introduce pojo class for colulmn family name
[ https://issues.apache.org/jira/browse/HBASE-18816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned HBASE-18816: -- Assignee: Chia-Ping Tsai > Introduce pojo class for colulmn family name > > > Key: HBASE-18816 > URL: https://issues.apache.org/jira/browse/HBASE-18816 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > > User don't be allowed to use arbitrary name for column family, so we should > add an new class to wrap the cf name with precheck in order to assure that > any cf name we get are legal. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18690) Replace o.a.h.c.InterfaceAudience by o.a.h.h.c.InterfaceAudience
[ https://issues.apache.org/jira/browse/HBASE-18690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18690: --- Attachment: HBASE-18690.branch-1.v0.patch retry...In the meantime, would you please take a look? [~busbey] > Replace o.a.h.c.InterfaceAudience by o.a.h.h.c.InterfaceAudience > > > Key: HBASE-18690 > URL: https://issues.apache.org/jira/browse/HBASE-18690 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 1.4.0, 1.3.2, 1.5.0, 1.2.7 > > Attachments: HBASE-18690.branch-1.v0.patch, > HBASE-18690.branch-1.v0.patch, HBASE-18690.branch-1.v0.patch, > HBASE-18690.v0.patch, HBASE-18690.v0.patch > > > Some classes still import the hadoop's InterfaceAudience. > # Interns > # MetricsInfoImpl > # RestCsrfPreventionFilter > # AbstractFileStatusFilter > # FileStatusFilter -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18690) Replace o.a.h.c.InterfaceAudience by o.a.h.h.c.InterfaceAudience
[ https://issues.apache.org/jira/browse/HBASE-18690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18690: --- Status: Patch Available (was: Open) > Replace o.a.h.c.InterfaceAudience by o.a.h.h.c.InterfaceAudience > > > Key: HBASE-18690 > URL: https://issues.apache.org/jira/browse/HBASE-18690 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 1.4.0, 1.3.2, 1.5.0, 1.2.7 > > Attachments: HBASE-18690.branch-1.v0.patch, > HBASE-18690.branch-1.v0.patch, HBASE-18690.branch-1.v0.patch, > HBASE-18690.v0.patch, HBASE-18690.v0.patch > > > Some classes still import the hadoop's InterfaceAudience. > # Interns > # MetricsInfoImpl > # RestCsrfPreventionFilter > # AbstractFileStatusFilter > # FileStatusFilter -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18690) Replace o.a.h.c.InterfaceAudience by o.a.h.h.c.InterfaceAudience
[ https://issues.apache.org/jira/browse/HBASE-18690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18690: --- Description: Some classes still import the hadoop's InterfaceAudience. # Interns # MetricsInfoImpl # RestCsrfPreventionFilter # AbstractFileStatusFilter # FileStatusFilter was: Some classes still import the hadoop's InterfaceAudience. # RestCsrfPreventionFilter # MetricsInfoImpl # FileStatusFilter # AbstractFileStatusFilter # StartcodeAgnosticServerName # FavoredNodesPromoter # FavoredNodesManager # BackupClientFactory # Interns > Replace o.a.h.c.InterfaceAudience by o.a.h.h.c.InterfaceAudience > > > Key: HBASE-18690 > URL: https://issues.apache.org/jira/browse/HBASE-18690 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 1.4.0, 1.3.2, 1.5.0, 1.2.7 > > Attachments: HBASE-18690.branch-1.v0.patch, > HBASE-18690.branch-1.v0.patch, HBASE-18690.v0.patch, HBASE-18690.v0.patch > > > Some classes still import the hadoop's InterfaceAudience. > # Interns > # MetricsInfoImpl > # RestCsrfPreventionFilter > # AbstractFileStatusFilter > # FileStatusFilter -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18690) Replace o.a.h.c.InterfaceAudience by o.a.h.h.c.InterfaceAudience
[ https://issues.apache.org/jira/browse/HBASE-18690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18690: --- Status: Open (was: Patch Available) > Replace o.a.h.c.InterfaceAudience by o.a.h.h.c.InterfaceAudience > > > Key: HBASE-18690 > URL: https://issues.apache.org/jira/browse/HBASE-18690 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > Fix For: 1.4.0, 1.3.2, 1.5.0, 1.2.7 > > Attachments: HBASE-18690.branch-1.v0.patch, > HBASE-18690.branch-1.v0.patch, HBASE-18690.v0.patch, HBASE-18690.v0.patch > > > Some classes still import the hadoop's InterfaceAudience. > # Interns > # MetricsInfoImpl > # RestCsrfPreventionFilter > # AbstractFileStatusFilter > # FileStatusFilter -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18821) Enforcer plugin NPEs again when generating site
[ https://issues.apache.org/jira/browse/HBASE-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167920#comment-16167920 ] stack commented on HBASE-18821: --- Thanks [~Apache9] I just got this this morning (site seems stuck in a cycle -- takes for ever). Let me pull this in. 1.4.1 has the NPE issue. What you've done seems right. Thanks. > Enforcer plugin NPEs again when generating site > --- > > Key: HBASE-18821 > URL: https://issues.apache.org/jira/browse/HBASE-18821 > Project: HBase > Issue Type: Sub-task > Components: site >Affects Versions: 2.0.0-alpha-3 >Reporter: stack >Assignee: stack > Fix For: 3.0.0, 2.0.0-alpha-3 > > Attachments: HBASE-18821-addendum.patch > > > The parent issue came back, HBASE-17351. Using 1.4.1 version of plugin NPEs. > Specify 1.4. I played w/ the suggestions over in MENFORCER-248 but to no > avail. There is a jenkins repository with a 1.4.2 which probably fixes this > and then there is the new 3.0.0 stuff Need to come back here. For now, > let me get a fix in so can make progress on releases. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18142) Deletion of a cell deletes the previous versions too
[
https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167919#comment-16167919
]
Chia-Ping Tsai commented on HBASE-18142:
{code}
- @test_table.put("101", "x:a", "1")
- @test_table.put("101", "x:a", "2", Time.now.to_i)
-
- @test_table.put("102", "x:a", "1", 1212)
- @test_table.put("102", "x:a", "2", 1213)
-
- @test_table.put(103, "x:a", "3")
- @test_table.put(103, "x:a", "4")
-
+ @test_table.put("102", "x:a", "2", 1212)
+ @test_table.put(103, "x:a", "3", 1214)
+
@test_table.put("104", "x:a", 5)
@test_table.put("104", "x:b", 6)
-
+
@test_table.put(105, "x:a", "3")
@test_table.put(105, "x:a", "4")
{code}
Because of the bending time issue, you have to change the initial data in order
to run the test of deleting specified version. +1
> Deletion of a cell deletes the previous versions too
>
>
> Key: HBASE-18142
> URL: https://issues.apache.org/jira/browse/HBASE-18142
> Project: HBase
> Issue Type: Bug
> Components: API, shell
>Affects Versions: 3.0.0, 1.3.1, 1.2.6, 2.0.0-alpha-1
>Reporter: Karthick
>Assignee: ChunHao
> Labels: beginner
> Fix For: 2.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7
>
> Attachments: HBASE-18142.master.v0.patch,
> HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch,
> HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch,
> HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch,
> HBASE-18142.master.v7.patch, HBASE-18142.master.v8.patch
>
>
> When I tried to delete a cell using it's timestamp in the Hbase Shell, the
> previous versions of the same cell also got deleted. But when I tried the
> same using the Java API, then the previous versions are not deleted and I can
> retrive the previous values.
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java
> see this file to fix the issue. This method (public Delete addColumn(final
> byte [] family, final byte [] qualifier, final long timestamp)) only deletes
> the current version of the cell. The previous versions are not deleted.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[ https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167918#comment-16167918 ] Reid Chan commented on HBASE-18651: --- any suggestions. [~tedyu] [~mdrob] > Let ChaosMonkeyRunner expose the chaos monkey runner it creates > --- > > Key: HBASE-18651 > URL: https://issues.apache.org/jira/browse/HBASE-18651 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Reid Chan > Attachments: HBASE-18651.master.001.patch, > HBASE-18651.master.002.patch > > > Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without > keeping track of the instance. > This poses some challenge when ChaosMonkeyRunner is used programmatically > because the caller cannot get hold of the runner. > As [~mdrob] suggested, we should expose the chaos monkey runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167894#comment-16167894
]
Hadoop QA commented on HBASE-13346:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 18 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
40s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
1s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
38m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
54s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 90m
55s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
21s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
36s{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
59s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}198m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-13346 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887312/HBASE-13346.master.001.patch
|
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ee0cc5cdabaa 3.13.0-123
[jira] [Comment Edited] (HBASE-18797) Deprecate Filter#filterKeyValue and add Filter#filterCell
[ https://issues.apache.org/jira/browse/HBASE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167758#comment-16167758 ] Chia-Ping Tsai edited comment on HBASE-18797 at 9/15/17 1:26 PM: - bq. All the existing filters will have both filterCell and filterKeyValue (this will just call filterCell). -Can we cleanup the filterKV on server side if current clients still use filterKeyValue?- I get it. +1 was (Author: chia7712): bq. All the existing filters will have both filterCell and filterKeyValue (this will just call filterCell). Can we cleanup the filterKV on server side if current clients still use filterKeyValue? > Deprecate Filter#filterKeyValue and add Filter#filterCell > - > > Key: HBASE-18797 > URL: https://issues.apache.org/jira/browse/HBASE-18797 > Project: HBase > Issue Type: Sub-task > Components: API, Filters >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Critical > Fix For: 2.0.0-alpha-3 > > > Part of filter package cleanup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18797) Deprecate Filter#filterKeyValue and add Filter#filterCell
[ https://issues.apache.org/jira/browse/HBASE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167831#comment-16167831 ] Anoop Sam John commented on HBASE-18797: The core code has to call the new method now. ie. filterCell().. What if there is an old custom Filter where the impl is having only filterKV() been implemented. We have to have BC. That is why the filterCell() def impl in Filter has to call old filterKV method > Deprecate Filter#filterKeyValue and add Filter#filterCell > - > > Key: HBASE-18797 > URL: https://issues.apache.org/jira/browse/HBASE-18797 > Project: HBase > Issue Type: Sub-task > Components: API, Filters >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Critical > Fix For: 2.0.0-alpha-3 > > > Part of filter package cleanup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18797) Deprecate Filter#filterKeyValue and add Filter#filterCell
[
https://issues.apache.org/jira/browse/HBASE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167816#comment-16167816
]
Tamas Penzes edited comment on HBASE-18797 at 9/15/17 12:51 PM:
Why not having a much more simple solution?
* Rename filterKeyValue(Cell kv) to filterCell(Cell c) everywhere
* Create abstract filterCell(Cell c); in Filter.java
* Create non abstract @Deprecated filterKeyValue(Cell c){ return
filterCell(c);} in Filter.java.
This way filterKeyValue would be deprecated everywhere, as requested by the
tickets creator, but would still work.
The same functionality would be available as filterCell as well.
Simple, short solution, with minimal changes.
was (Author: tamaas):
Why not having a much more simple solution?
* Rename filterKeyValue(Cell kv) to filterCell(Cell c) everywhere
* Create non abstract @Deprecated filterKeyValue(Cell c){ return
filterCell(c);} in Filter.java.
This way filterKeyValue would be deprecated everywhere, as requested by the
tickets creator, but would still work.
The same functionality would be available as filterCell as well.
Simple, short solution, with minimal changes.
> Deprecate Filter#filterKeyValue and add Filter#filterCell
> -
>
> Key: HBASE-18797
> URL: https://issues.apache.org/jira/browse/HBASE-18797
> Project: HBase
> Issue Type: Sub-task
> Components: API, Filters
>Reporter: Abhishek Singh Chouhan
>Assignee: Abhishek Singh Chouhan
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
>
> Part of filter package cleanup.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167815#comment-16167815
]
Tamas Penzes commented on HBASE-13346:
--
[~abhishek.chouhan] You are right, my fault that I haven't assigned the main
ticket to myself when I started working on it a few days ago.
Jira was open on my machine and haven't checked for changes on the state of the
ticket. Will take care next time.
Can I take over the sub-task from you?
Thanks, Tamaas
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
> Attachments: HBASE-13346.master.001.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18797) Deprecate Filter#filterKeyValue and add Filter#filterCell
[
https://issues.apache.org/jira/browse/HBASE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167816#comment-16167816
]
Tamas Penzes commented on HBASE-18797:
--
Why not having a much more simple solution?
* Rename filterKeyValue(Cell kv) to filterCell(Cell c) everywhere
* Create non abstract @Deprecated filterKeyValue(Cell c){ return
filterCell(c);} in Filter.java.
This way filterKeyValue would be deprecated everywhere, as requested by the
tickets creator, but would still work.
The same functionality would be available as filterCell as well.
Simple, short solution, with minimal changes.
> Deprecate Filter#filterKeyValue and add Filter#filterCell
> -
>
> Key: HBASE-18797
> URL: https://issues.apache.org/jira/browse/HBASE-18797
> Project: HBase
> Issue Type: Sub-task
> Components: API, Filters
>Reporter: Abhishek Singh Chouhan
>Assignee: Abhishek Singh Chouhan
>Priority: Critical
> Fix For: 2.0.0-alpha-3
>
>
> Part of filter package cleanup.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
