[jira] [Created] (HBASE-21064) Support set region server quota and allow exceed user/table/ns quota when rs has available quota
Yi Mei created HBASE-21064: -- Summary: Support set region server quota and allow exceed user/table/ns quota when rs has available quota Key: HBASE-21064 URL: https://issues.apache.org/jira/browse/HBASE-21064 Project: HBase Issue Type: Task Affects Versions: 3.0.0, 2.2.0 Reporter: Yi Mei Assignee: Yi Mei Support set region server quota which represents the read/write ability of region servers in HBase shell. And add a switch named "ALLOW_EXCEED" to allow exceed user/table/ns quota when region server has available quota. Use the following command to set RS quota: set_quota TYPE => THROTTLE, REGIONSERVER => 'rs', LIMIT => '10req/sec', ALLOW_EXCEED => true -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21064) Support set region server quota and allow exceed user/table/ns quota when rs has available quota
[ https://issues.apache.org/jira/browse/HBASE-21064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21064: --- Issue Type: Sub-task (was: Task) Parent: HBASE-16707 > Support set region server quota and allow exceed user/table/ns quota when rs > has available quota > > > Key: HBASE-21064 > URL: https://issues.apache.org/jira/browse/HBASE-21064 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > > Support set region server quota which represents the read/write ability of > region servers in HBase shell. > And add a switch named "ALLOW_EXCEED" to allow exceed user/table/ns quota > when region server has available quota. > Use the following command to set RS quota: > set_quota TYPE => THROTTLE, REGIONSERVER => 'rs', LIMIT => '10req/sec', > ALLOW_EXCEED => true -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[
https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582070#comment-16582070
]
Hadoop QA commented on HBASE-21047:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
1s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
43s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
18s{color} | {color:red} hbase-server: The patch generated 1 new + 93 unchanged
- 1 fixed = 94 total (was 94) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 35s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 2s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}116m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21047 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12935796/HBASE-21047.branch-1.v2.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 21d5df64607a 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 x86_64 x8
[jira] [Updated] (HBASE-21064) Support set region server quota and allow exceed user/table/ns quota when rs has available quota
[ https://issues.apache.org/jira/browse/HBASE-21064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21064: --- Attachment: HBASE-21064.master.001.patch > Support set region server quota and allow exceed user/table/ns quota when rs > has available quota > > > Key: HBASE-21064 > URL: https://issues.apache.org/jira/browse/HBASE-21064 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21064.master.001.patch > > > Support set region server quota which represents the read/write ability of > region servers in HBase shell. > And add a switch named "ALLOW_EXCEED" to allow exceed user/table/ns quota > when region server has available quota. > Use the following command to set RS quota: > set_quota TYPE => THROTTLE, REGIONSERVER => 'rs', LIMIT => '10req/sec', > ALLOW_EXCEED => true -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20469) Directory used for sidelining old recovered edits files should be made configurable
[
https://issues.apache.org/jira/browse/HBASE-20469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582150#comment-16582150
]
Hudson commented on HBASE-20469:
Results for branch branch-2.1
[build #196 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/196/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/196//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/196//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/196//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Directory used for sidelining old recovered edits files should be made
> configurable
> ---
>
> Key: HBASE-20469
> URL: https://issues.apache.org/jira/browse/HBASE-20469
> Project: HBase
> Issue Type: Improvement
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.1, 1.4.7
>
> Attachments: HBASE-20469.master.001.patch
>
>
> Currently the directory used for sidelining of old recovered edit files is
> hardcoded to be "/tmp"
> {code:java}
> Path tmp = new Path("/tmp");
> {code}
> [See L484
> WALSplittter.java|https://github.com/apache/hbase/blob/273d252838e96c4b4af2401743d84e482c4ec565/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java#L484]
> Instead, we can use some configurable directory in the following manner:
>
> {code:java}
> String tmpDirName = conf.get(HConstants.TEMPORARY_FS_DIRECTORY_KEY,
> HConstants.DEFAULT_TEMPORARY_HDFS_DIRECTORY);
> .
> .
> Path tmp = new Path(tmpDirName);
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20881) Introduce a region transition procedure to handle all the state transition for a region
[
https://issues.apache.org/jira/browse/HBASE-20881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582264#comment-16582264
]
Hadoop QA commented on HBASE-20881:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 26 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch hbase-protocol-shaded passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} hbase-client: The patch generated 0 new + 2
unchanged - 84 fixed = 2 total (was 86) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} hbase-procedure: The patch generated 0 new + 19
unchanged - 1 fixed = 19 total (was 20) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} hbase-server: The patch generated 0 new + 246
unchanged - 73 fixed = 246 total (was 319) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hbase-rsgroup passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 34s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
31s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
5s{color} | {color:green} hbase-client in the patch passed. {color}
[jira] [Updated] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[ https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Khandelwal updated HBASE-21047: -- Attachment: HBASE-21047.v1.patch > Object creation of StoreFileScanner thru constructor and close may leave > refCount to -1 > --- > > Key: HBASE-21047 > URL: https://issues.apache.org/jira/browse/HBASE-21047 > Project: HBase > Issue Type: Bug >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21047.branch-1.v1.patch, > HBASE-21047.branch-1.v2.patch, HBASE-21047.v1.patch > > > During object creation "*StoreFileScanner*", it does not increase the > refCount whereas while close it decrements the reader refCount. This will > cause refCount to -1 and isReadReference method was returning true > (refCount.get() != 0 This is causing store file not to be deleted. This may > also cause issue in situation when some thread is holding a scanner but it > may actually be closed due to above bug. Impact of this would be really high. > Attatching patch for the fix which makes sure if reference is held either > thru getScanner method or constructor, ref is always updated. Patch also > contains a test which validates the issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[ https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582287#comment-16582287 ] Vishal Khandelwal commented on HBASE-21047: --- Thanks [~liuml07] [~apurtell]: PFA, patch for master branch > Object creation of StoreFileScanner thru constructor and close may leave > refCount to -1 > --- > > Key: HBASE-21047 > URL: https://issues.apache.org/jira/browse/HBASE-21047 > Project: HBase > Issue Type: Bug >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21047.branch-1.v1.patch, > HBASE-21047.branch-1.v2.patch, HBASE-21047.v1.patch > > > During object creation "*StoreFileScanner*", it does not increase the > refCount whereas while close it decrements the reader refCount. This will > cause refCount to -1 and isReadReference method was returning true > (refCount.get() != 0 This is causing store file not to be deleted. This may > also cause issue in situation when some thread is holding a scanner but it > may actually be closed due to above bug. Impact of this would be really high. > Attatching patch for the fix which makes sure if reference is held either > thru getScanner method or constructor, ref is always updated. Patch also > contains a test which validates the issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20881) Introduce a region transition procedure to handle all the state transition for a region
[ https://issues.apache.org/jira/browse/HBASE-20881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20881: -- Attachment: HBASE-20881-v13.patch > Introduce a region transition procedure to handle all the state transition > for a region > --- > > Key: HBASE-20881 > URL: https://issues.apache.org/jira/browse/HBASE-20881 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-20881-v1.patch, HBASE-20881-v10.patch, > HBASE-20881-v11.patch, HBASE-20881-v12.patch, HBASE-20881-v13.patch, > HBASE-20881-v2.patch, HBASE-20881-v3.patch, HBASE-20881-v4.patch, > HBASE-20881-v4.patch, HBASE-20881-v5.patch, HBASE-20881-v6.patch, > HBASE-20881-v7.patch, HBASE-20881-v7.patch, HBASE-20881-v8.patch, > HBASE-20881-v9.patch, HBASE-20881.patch > > > Now have an AssignProcedure, an UnssignProcedure, and also a > MoveRegionProcedure which schedules an AssignProcedure and an > UnssignProcedure to move a region. This makes the logic a bit complicated, as > MRP is not a RIT, so when SCP can not interrupt it directly... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21065) Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters on meta while we are at it)
stack created HBASE-21065:
-
Summary: Try ROW_INDEX_V1 encoding on meta table (fix bloomfilters
on meta while we are at it)
Key: HBASE-21065
URL: https://issues.apache.org/jira/browse/HBASE-21065
Project: HBase
Issue Type: Improvement
Components: meta, Performance
Reporter: stack
Assignee: stack
Some users end up hitting meta hard. Bulk is probably because our client goes
to meta too often, and the real 'fix' for a saturated meta is splitting it, but
the encoding that came in with HBASE-16213, ROW_INDEX_V1, could help in the
near term. It adds an index on hfile blocks and helped improve random reads
against user-space tables (less compares as we used index to go direct to
requested Cells rather than look at each Cell in turn until we found what we
wanted -- see RN on HBASE-16213 for citation).
I also noticed code-reading that we don't enable blooms on hbase:meta tables;
that could save some CPU and speed things up a bit too:
{code}
// Disable blooms for meta. Needs work. Seems to mess w/
getClosestOrBefore.
.setBloomFilterType(BloomType.NONE)
{code}
This issue is about doing a bit of perf compare of encoding *on* vs current
default (and will check diff in size of indexed blocks).
Meta access is mostly random-read I believe (A review of a user's access showed
this so at least for their workload). The nice addition, HBASE-19722 Meta query
statistics metrics source, would help verify if it saw some usage on a prod
cluster.
If all is good, I'd like to make a small patch, one that could be easily
backported, with minimal changes in it.
As is, its all a little awkward as the meta table schema is hard-coded and meta
is immutable -- stuff we'll have to fix if we want to split meta -- so in the
meantime it requires a code change to enable (and a backport of HBASE-16213 --
this patch is in 1.4.0 only currently, perhaps that is enough). Code change to
enable is small:
{code}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
index 28c7ec3c2f..8f08f94dc1 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
@@ -160,6 +160,7 @@ public class FSTableDescriptors implements TableDescriptors
{
.setScope(HConstants.REPLICATION_SCOPE_LOCAL)
// Disable blooms for meta. Needs work. Seems to mess w/
getClosestOrBefore.
.setBloomFilterType(BloomType.NONE)
+
.setDataBlockEncoding(org.apache.hadoop.hbase.io.encoding.DataBlockEncoding.ROW_INDEX_V1)
.build())
.setColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(HConstants.TABLE_FAMILY)
.setMaxVersions(conf.getInt(HConstants.HBASE_META_VERSIONS,
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582302#comment-16582302
]
Reid Chan commented on HBASE-20993:
---
bq. where is the best placement of the preamble reply on the RpcServer?
What i see, we are at the same place, unless you didn't read my code.
Server already does everything bad preamble reply, acl check failure response,
auth check failure response, fallback-simple reply etc, what you code is
redundant. The problem is client ignores them all, and i don't think it add any
complexity.
Looks like we won't have consensus on this problem.
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.2.001.patch,
> HBASE-20993.branch-1.wip.002.patch, HBASE-20993.branch-1.wip.patch
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin
[jira] [Commented] (HBASE-12233) Bring back root table
[ https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582343#comment-16582343 ] stack commented on HBASE-12233: --- Was looking at this anew today: * We'd bring back root because we need to be able to split meta to distribute loading, allow meta to get much bigger because more regions (scaling) and we want to add more meta info per region (files in a region, etc.) * All to do w/ meta is currently hardcoded; the meta table schema and the meta first regioninfo. This would have to change if we were to allow a split meta. The meta table descriptor is not kept anywhere but in code (user-space table descriptors are in the filesystem). Meta table is echoed out to the filesystem but never read from there. Ditto RegionInfo. * Another hardcoding is that the meta table regions are hardcoded OPEN. That would have to change too allowing offline of regions as they split. * New clients would look for a new root znode. It'd point to a root table, a table that would never split. It'd have the list of meta regions as it did in the past. We'd try and avoid mistakes of the past regards stuff like key scheme and wacky comparator to interpret it. We'd not disturb the current meta znodes. * We'd revamp codebase so for example new clients went via root to find meta table. * Old clients would go looking for the single hard-coded meta region. We'd point them at any RS or even at the Master. On each there'd be a facade that proxied requests for the hardcoded meta region forwarding them on to actual meta table. Might work. * Startup would assign root first, and so on. Might work now we have AMv2 and its startup has been cleaned up recently. Bit of work here. Will write up a page if do more on this. > Bring back root table > - > > Key: HBASE-12233 > URL: https://issues.apache.org/jira/browse/HBASE-12233 > Project: HBase > Issue Type: Sub-task >Reporter: Virag Kothari >Assignee: Virag Kothari >Priority: Major > Attachments: HBASE-12233.patch > > > First step towards splitting meta. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations
[
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582360#comment-16582360
]
Ted Yu commented on HBASE-19008:
[~reidchan]:
Can you take another look ?
> Add missing equals or hashCode method(s) to stock Filter implementations
>
>
> Key: HBASE-19008
> URL: https://issues.apache.org/jira/browse/HBASE-19008
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Major
> Labels: filter
> Attachments: Filters.png, HBASE-19008-1.patch, HBASE-19008-10.patch,
> HBASE-19008-11.patch, HBASE-19008-12.patch, HBASE-19008-13.patch,
> HBASE-19008-2.patch, HBASE-19008-3.patch, HBASE-19008-4.patch,
> HBASE-19008-5.patch, HBASE-19008-6.patch, HBASE-19008-7.patch,
> HBASE-19008-8.patch, HBASE-19008-9.patch, HBASE-19008.patch
>
>
> In HBASE-15410, [~mdrob] reminded me that Filter implementations may not
> write {{equals}} or {{hashCode}} method(s).
> This issue is to add missing {{equals}} or {{hashCode}} method(s) to stock
> Filter implementations such as KeyOnlyFilter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Khandelwal updated HBASE-20940: -- Attachment: HBASE-20940.branch-1.v2.patch > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20940.branch-1.3.v1.patch, > HBASE-20940.branch-1.3.v2.patch, HBASE-20940.branch-1.v1.patch, > HBASE-20940.branch-1.v2.patch, HBASE-20940.v1.patch, HBASE-20940.v2.patch, > HBASE-20940.v3.patch, HBASE-20940.v4.patch > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582365#comment-16582365 ] Vishal Khandelwal commented on HBASE-20940: --- [~apurtell]: updated the patch for branch-1 as well. > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20940.branch-1.3.v1.patch, > HBASE-20940.branch-1.3.v2.patch, HBASE-20940.branch-1.v1.patch, > HBASE-20940.branch-1.v2.patch, HBASE-20940.v1.patch, HBASE-20940.v2.patch, > HBASE-20940.v3.patch, HBASE-20940.v4.patch > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20469) Directory used for sidelining old recovered edits files should be made configurable
[
https://issues.apache.org/jira/browse/HBASE-20469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582366#comment-16582366
]
Hudson commented on HBASE-20469:
Results for branch branch-1
[build #415 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/415/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/415//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/415//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/415//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Directory used for sidelining old recovered edits files should be made
> configurable
> ---
>
> Key: HBASE-20469
> URL: https://issues.apache.org/jira/browse/HBASE-20469
> Project: HBase
> Issue Type: Improvement
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.1, 1.4.7
>
> Attachments: HBASE-20469.master.001.patch
>
>
> Currently the directory used for sidelining of old recovered edit files is
> hardcoded to be "/tmp"
> {code:java}
> Path tmp = new Path("/tmp");
> {code}
> [See L484
> WALSplittter.java|https://github.com/apache/hbase/blob/273d252838e96c4b4af2401743d84e482c4ec565/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java#L484]
> Instead, we can use some configurable directory in the following manner:
>
> {code:java}
> String tmpDirName = conf.get(HConstants.TEMPORARY_FS_DIRECTORY_KEY,
> HConstants.DEFAULT_TEMPORARY_HDFS_DIRECTORY);
> .
> .
> Path tmp = new Path(tmpDirName);
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20469) Directory used for sidelining old recovered edits files should be made configurable
[
https://issues.apache.org/jira/browse/HBASE-20469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582368#comment-16582368
]
Hudson commented on HBASE-20469:
Results for branch branch-2
[build #1117 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1117/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1117//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1117//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1117//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Directory used for sidelining old recovered edits files should be made
> configurable
> ---
>
> Key: HBASE-20469
> URL: https://issues.apache.org/jira/browse/HBASE-20469
> Project: HBase
> Issue Type: Improvement
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.1, 1.4.7
>
> Attachments: HBASE-20469.master.001.patch
>
>
> Currently the directory used for sidelining of old recovered edit files is
> hardcoded to be "/tmp"
> {code:java}
> Path tmp = new Path("/tmp");
> {code}
> [See L484
> WALSplittter.java|https://github.com/apache/hbase/blob/273d252838e96c4b4af2401743d84e482c4ec565/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java#L484]
> Instead, we can use some configurable directory in the following manner:
>
> {code:java}
> String tmpDirName = conf.get(HConstants.TEMPORARY_FS_DIRECTORY_KEY,
> HConstants.DEFAULT_TEMPORARY_HDFS_DIRECTORY);
> .
> .
> Path tmp = new Path(tmpDirName);
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations
[
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582374#comment-16582374
]
Reid Chan commented on HBASE-19008:
---
At least, we are near the end :)
{{checkstyle}}:
./hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueExcludeFilter.java:36:import
org.apache.hadoop.hbase.util.Bytes;: Wrong order for
'org.apache.hadoop.hbase.util.Bytes' import. [ImportOrder]
{{whitespace}}:
./hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueExcludeFilter.java:246:
./hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueFilter.java:464:
./hbase-spark/src/main/java/org/apache/hadoop/hbase/spark/SparkSQLPushDownFilter.java:279:
> Add missing equals or hashCode method(s) to stock Filter implementations
>
>
> Key: HBASE-19008
> URL: https://issues.apache.org/jira/browse/HBASE-19008
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Major
> Labels: filter
> Attachments: Filters.png, HBASE-19008-1.patch, HBASE-19008-10.patch,
> HBASE-19008-11.patch, HBASE-19008-12.patch, HBASE-19008-13.patch,
> HBASE-19008-2.patch, HBASE-19008-3.patch, HBASE-19008-4.patch,
> HBASE-19008-5.patch, HBASE-19008-6.patch, HBASE-19008-7.patch,
> HBASE-19008-8.patch, HBASE-19008-9.patch, HBASE-19008.patch
>
>
> In HBASE-15410, [~mdrob] reminded me that Filter implementations may not
> write {{equals}} or {{hashCode}} method(s).
> This issue is to add missing {{equals}} or {{hashCode}} method(s) to stock
> Filter implementations such as KeyOnlyFilter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20469) Directory used for sidelining old recovered edits files should be made configurable
[
https://issues.apache.org/jira/browse/HBASE-20469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582377#comment-16582377
]
Hudson commented on HBASE-20469:
Results for branch branch-1.4
[build #420 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/420/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/420//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/420//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/420//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Directory used for sidelining old recovered edits files should be made
> configurable
> ---
>
> Key: HBASE-20469
> URL: https://issues.apache.org/jira/browse/HBASE-20469
> Project: HBase
> Issue Type: Improvement
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.1, 1.4.7
>
> Attachments: HBASE-20469.master.001.patch
>
>
> Currently the directory used for sidelining of old recovered edit files is
> hardcoded to be "/tmp"
> {code:java}
> Path tmp = new Path("/tmp");
> {code}
> [See L484
> WALSplittter.java|https://github.com/apache/hbase/blob/273d252838e96c4b4af2401743d84e482c4ec565/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java#L484]
> Instead, we can use some configurable directory in the following manner:
>
> {code:java}
> String tmpDirName = conf.get(HConstants.TEMPORARY_FS_DIRECTORY_KEY,
> HConstants.DEFAULT_TEMPORARY_HDFS_DIRECTORY);
> .
> .
> Path tmp = new Path(tmpDirName);
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20175) hbase-spark needs scala dependency convergance
[
https://issues.apache.org/jira/browse/HBASE-20175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582435#comment-16582435
]
Artem Ervits commented on HBASE-20175:
--
[~mdrob] please go ahead with email to dev list. I can make the necessary
changes in a follow-up Jira when we get a go ahead.
> hbase-spark needs scala dependency convergance
> --
>
> Key: HBASE-20175
> URL: https://issues.apache.org/jira/browse/HBASE-20175
> Project: HBase
> Issue Type: Bug
> Components: dependencies, spark
>Reporter: Mike Drob
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-20175.v01.patch, HBASE-20175.v04.patch,
> HBASE-20175.v05.patch
>
>
> This is a follow-on to HBASE-16179 - I think we might need to specify an
> exclude in the dependency management.
> {noformat}
> [INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @
> hbase-spark ---
> [WARNING] Expected all dependencies to require Scala version: 2.11.8
> [WARNING] org.apache.hbase:hbase-spark:3.0.0-SNAPSHOT requires scala
> version: 2.11.8
> [WARNING] org.apache.spark:spark-streaming_2.11:2.1.1 requires scala
> version: 2.11.8
> [WARNING] org.apache.spark:spark-streaming_2.11:2.1.1 requires scala
> version: 2.11.8
> [WARNING] org.scalatest:scalatest_2.11:2.2.4 requires scala version: 2.11.2
> {noformat}
> [~tedyu] - since you're already fiddling in this area, do you want to take a
> look?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[
https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582437#comment-16582437
]
Hadoop QA commented on HBASE-21047:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
13s{color} | {color:red} hbase-server: The patch generated 1 new + 37 unchanged
- 1 fixed = 38 total (was 38) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
26s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 49s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}117m
42s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}156m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21047 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12935832/HBASE-21047.v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 49095d65e5e5 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 092efb4274 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/14061/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14061/testReport/ |
| Max. process+thread count | 4506 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://bui
[jira] [Commented] (HBASE-21040) Replace call to printStackTrace() with proper logger call
[
https://issues.apache.org/jira/browse/HBASE-21040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582443#comment-16582443
]
Hudson commented on HBASE-21040:
Results for branch master
[build #436 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/436/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/436//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/436//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/436//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Replace call to printStackTrace() with proper logger call
> -
>
> Key: HBASE-21040
> URL: https://issues.apache.org/jira/browse/HBASE-21040
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Subrat Mishra
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-21040.master.001.patch,
> HBASE-21040.master.002.patch
>
>
> Here is related code:
> {code}
> hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/RestoreDriver.java:
> e.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientAsyncPrefetchScanner.java:
> first.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java:
> t.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java:
>e.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java:
>e.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java:
>e.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java:
>e.printStackTrace();
> hbase-examples/src/main/java/org/apache/hadoop/hbase/mapreduce/SampleUploader.java:
> e.printStackTrace();
> hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/DemoClient.java:
> e.printStackTrace();
> hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/HttpDoAsClient.java:
> e.printStackTrace();
> hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/HttpDoAsClient.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java:
> ioe.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java:
> ie.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/CellCounter.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/CopyTable.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/HashTable.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/SyncTable.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TsvImporterMapper.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TsvImporterTextMapper.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java:
> e.printStackTrace();
> hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALPrettyPrinter.java:
> e.printStackTrace();
> hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java:
> e.printStackTrace();
> hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java:
[jira] [Commented] (HBASE-20469) Directory used for sidelining old recovered edits files should be made configurable
[
https://issues.apache.org/jira/browse/HBASE-20469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582445#comment-16582445
]
Hudson commented on HBASE-20469:
Results for branch master
[build #436 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/436/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/436//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/436//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/436//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Directory used for sidelining old recovered edits files should be made
> configurable
> ---
>
> Key: HBASE-20469
> URL: https://issues.apache.org/jira/browse/HBASE-20469
> Project: HBase
> Issue Type: Improvement
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.1, 1.4.7
>
> Attachments: HBASE-20469.master.001.patch
>
>
> Currently the directory used for sidelining of old recovered edit files is
> hardcoded to be "/tmp"
> {code:java}
> Path tmp = new Path("/tmp");
> {code}
> [See L484
> WALSplittter.java|https://github.com/apache/hbase/blob/273d252838e96c4b4af2401743d84e482c4ec565/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java#L484]
> Instead, we can use some configurable directory in the following manner:
>
> {code:java}
> String tmpDirName = conf.get(HConstants.TEMPORARY_FS_DIRECTORY_KEY,
> HConstants.DEFAULT_TEMPORARY_HDFS_DIRECTORY);
> .
> .
> Path tmp = new Path(tmpDirName);
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21050) Exclusive lock may be held by a SUCCESS state procedure forever
[
https://issues.apache.org/jira/browse/HBASE-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582444#comment-16582444
]
Hudson commented on HBASE-21050:
Results for branch master
[build #436 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/436/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/436//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/436//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/436//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Exclusive lock may be held by a SUCCESS state procedure forever
> ---
>
> Key: HBASE-21050
> URL: https://issues.apache.org/jira/browse/HBASE-21050
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 2.1.0, 2.0.1
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.1.1
>
> Attachments: HBASE-21050.branch-2.0.001.patch
>
>
> After HBASE-20846, we restore lock info for procedures. But, there is a case
> that the lock and be held by a already success procedure. Since the procedure
> won't execute again, the lock will held by the procedure forever.
> 1. All children for pid=1208 had been finished, but before procedure 1208
> awake, the master was killed
> {code}
> 2018-08-05 02:20:14,465 INFO [PEWorker-8]
> procedure2.ProcedureExecutor(1659): Finished subprocedure(s) of pid=1208,
> ppid=1206, state=RUNNABLE, hasLock=true; MoveRegionProcedure
> hri=c2a23a735f16df57299
> dba6fd4599f2f, source=e010125050127.bja,60020,1533403109034,
> destination=e010125050127.bja,60020,1533403109034; resume parent processing.
> 2018-08-05 02:20:14,466 INFO [PEWorker-8]
> procedure2.ProcedureExecutor(1296): Finished pid=1232, ppid=1208,
> state=SUCCESS, hasLock=false; AssignProcedure
> table=IntegrationTestBigLinkedList, region=c2a
> 23a735f16df57299dba6fd4599f2f, target=e010125050127.bja,60020,1533403109034
> in 1.5060sec
> {code}
> 2. Master restarts, since procedure 1208 held the lock before restart, so the
> lock was resotore for it
> {code}
> 2018-08-05 02:20:30,803 DEBUG [Thread-15] procedure2.ProcedureExecutor(456):
> Loading pid=1208, ppid=1206, state=SUCCESS, hasLock=false;
> MoveRegionProcedure hri=c2a23a735f16df57299dba6fd4599f2f, source=
> e010125050127.bja,60020,1533403109034,
> destination=e010125050127.bja,60020,1533403109034
> 2018-08-05 02:20:30,818 DEBUG [Thread-15] procedure2.Procedure(898):
> pid=1208, ppid=1206, state=SUCCESS, hasLock=false; MoveRegionProcedure
> hri=c2a23a735f16df57299dba6fd4599f2f, source=e010125050127.bj
> a,60020,1533403109034, destination=e010125050127.bja,60020,1533403109034 held
> the lock before restarting, call acquireLock to restore it.
> 2018-08-05 02:20:30,818 INFO [Thread-15]
> procedure.MasterProcedureScheduler(631): pid=1208, ppid=1206, state=SUCCESS,
> hasLock=false; MoveRegionProcedure hri=c2a23a735f16df57299dba6fd4599f2f,
> source=e0
> 10125050127.bja,60020,1533403109034,
> destination=e010125050127.bja,60020,1533403109034 checking lock on
> c2a23a735f16df57299dba6fd4599f2f
> {code}
> 3. Since procedure 1208 is success, it won't execute later, so the lock will
> be held by it forever
> We need to check the state of the procedure before restoring locks, if the
> procedure is already finished (success or rollback), we do not need to
> acquire lock for it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[
https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582500#comment-16582500
]
Hadoop QA commented on HBASE-20940:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
46s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
22s{color} | {color:red} hbase-server: The patch generated 23 new + 124
unchanged - 3 fixed = 147 total (was 127) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
45s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 36s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 8s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}118m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestModifyNamespaceProcedure |
| | hadoop.hbase.regionserver.TestHRegion |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-20940 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12935847/HBASE-20940.branch-1.v2.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedj
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582548#comment-16582548
]
Josh Elser commented on HBASE-21062:
{quote}Sorry I missed this point previously. It is right to keep the original
behavior.
{quote}
No worries! You and I, both :)
{quote}'us it' -> "use it"
{quote}
Thanks, Ted. Will fix on commit along with checkstyle.
Unit test failure seems unrelated based on error, but will rerun locally.
Ping [~Apache9], [~stack], for your branches.
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch,
> HBASE-21062.002.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20881) Introduce a region transition procedure to handle all the state transition for a region
[
https://issues.apache.org/jira/browse/HBASE-20881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582582#comment-16582582
]
Hadoop QA commented on HBASE-20881:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 26 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
30s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch hbase-protocol-shaded passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} hbase-client: The patch generated 0 new + 2
unchanged - 84 fixed = 2 total (was 86) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} hbase-procedure: The patch generated 0 new + 19
unchanged - 1 fixed = 19 total (was 20) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} hbase-server: The patch generated 0 new + 246
unchanged - 73 fixed = 246 total (was 319) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hbase-rsgroup passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
16s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 46s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
31s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
7s{color} | {color:green} hbase-client in the patch passed. {color}
[jira] [Commented] (HBASE-20951) Ratis LogService backed WALs
[
https://issues.apache.org/jira/browse/HBASE-20951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582640#comment-16582640
]
Josh Elser commented on HBASE-20951:
{quote}Can we make it generic? Then we can have following, it still provides
the flexibility to support FSBased system (maybe not only HDFS if on the
Cloud?), and interface is cleaner.
{quote}
Sounds like a good idea to me, [~reidchan]. Thanks for the suggestion!
> Ratis LogService backed WALs
>
>
> Key: HBASE-20951
> URL: https://issues.apache.org/jira/browse/HBASE-20951
> Project: HBase
> Issue Type: New Feature
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
>
> Umbrella issue for the Ratis+WAL work:
> Design doc:
> [https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit#|https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit]
> The (over-simplified) goal is to re-think the current WAL APIs we have now,
> ensure that they are de-coupled from the notion of being backed by HDFS, swap
> the current implementations over to the new API, and then wire up the Ratis
> LogService to the new WAL API.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20705) Having RPC Quota on a table prevents Space quota to be recreated/removed
[
https://issues.apache.org/jira/browse/HBASE-20705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582666#comment-16582666
]
Josh Elser commented on HBASE-20705:
Working on committing to 2.2 and master.
[~Apache9], [~stack], another for your braches, gents. Test-only.
> Having RPC Quota on a table prevents Space quota to be recreated/removed
>
>
> Key: HBASE-20705
> URL: https://issues.apache.org/jira/browse/HBASE-20705
> Project: HBase
> Issue Type: Bug
>Reporter: Biju Nair
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-20705.master.001.patch
>
>
> * Property {{hbase.quota.remove.on.table.delete}} is set to {{true}} by
> default
> * Create a table and set RPC and Space quota
> {noformat}
> hbase(main):022:0> create 't2','cf1'
> Created table t2
> Took 0.7420 seconds
> => Hbase::Table - t2
> hbase(main):023:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0105 seconds
> hbase(main):024:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0186 seconds
> hbase(main):025:0> list_quotas
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT =>
> 10M/sec, SCOPE => MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824, VIOLATION_POLICY
> => NO_WRITES{noformat}
> * Drop the table and the Space quota is set to {{REMOVE => true}}
> {noformat}
> hbase(main):026:0> disable 't2'
> Took 0.4363 seconds
> hbase(main):027:0> drop 't2'
> Took 0.2344 seconds
> hbase(main):028:0> list_quotas
> TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE => true
> USER => u1 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT => 10M/sec,
> SCOPE => MACHINE{noformat}
> * Recreate the table and set Space quota back. The Space quota on the table
> is still set to {{REMOVE => true}}
> {noformat}
> hbase(main):029:0> create 't2','cf1'
> Created table t2
> Took 0.7348 seconds
> => Hbase::Table - t2
> hbase(main):031:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0088 seconds
> hbase(main):032:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT =>
> 10M/sec, SCOPE => MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE => true{noformat}
> * Remove RPC quota and drop the table, the Space Quota is not removed
> {noformat}
> hbase(main):033:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT => NONE
> Took 0.0193 seconds
> hbase(main):036:0> disable 't2'
> Took 0.4305 seconds
> hbase(main):037:0> drop 't2'
> Took 0.2353 seconds
> hbase(main):038:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE
> => true{noformat}
> * Deleting the quota entry from {{hbase:quota}} seems to be the option to
> reset it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20705) Having RPC Quota on a table prevents Space quota to be recreated/removed
[
https://issues.apache.org/jira/browse/HBASE-20705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20705:
---
Fix Version/s: 2.2.0
3.0.0
> Having RPC Quota on a table prevents Space quota to be recreated/removed
>
>
> Key: HBASE-20705
> URL: https://issues.apache.org/jira/browse/HBASE-20705
> Project: HBase
> Issue Type: Bug
>Reporter: Biju Nair
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-20705.master.001.patch
>
>
> * Property {{hbase.quota.remove.on.table.delete}} is set to {{true}} by
> default
> * Create a table and set RPC and Space quota
> {noformat}
> hbase(main):022:0> create 't2','cf1'
> Created table t2
> Took 0.7420 seconds
> => Hbase::Table - t2
> hbase(main):023:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0105 seconds
> hbase(main):024:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0186 seconds
> hbase(main):025:0> list_quotas
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT =>
> 10M/sec, SCOPE => MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824, VIOLATION_POLICY
> => NO_WRITES{noformat}
> * Drop the table and the Space quota is set to {{REMOVE => true}}
> {noformat}
> hbase(main):026:0> disable 't2'
> Took 0.4363 seconds
> hbase(main):027:0> drop 't2'
> Took 0.2344 seconds
> hbase(main):028:0> list_quotas
> TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE => true
> USER => u1 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT => 10M/sec,
> SCOPE => MACHINE{noformat}
> * Recreate the table and set Space quota back. The Space quota on the table
> is still set to {{REMOVE => true}}
> {noformat}
> hbase(main):029:0> create 't2','cf1'
> Created table t2
> Took 0.7348 seconds
> => Hbase::Table - t2
> hbase(main):031:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0088 seconds
> hbase(main):032:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT =>
> 10M/sec, SCOPE => MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE => true{noformat}
> * Remove RPC quota and drop the table, the Space Quota is not removed
> {noformat}
> hbase(main):033:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT => NONE
> Took 0.0193 seconds
> hbase(main):036:0> disable 't2'
> Took 0.4305 seconds
> hbase(main):037:0> drop 't2'
> Took 0.2353 seconds
> hbase(main):038:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE
> => true{noformat}
> * Deleting the quota entry from {{hbase:quota}} seems to be the option to
> reset it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations
[
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liubangchen updated HBASE-19008:
Attachment: HBASE-19008-14.patch
> Add missing equals or hashCode method(s) to stock Filter implementations
>
>
> Key: HBASE-19008
> URL: https://issues.apache.org/jira/browse/HBASE-19008
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Major
> Labels: filter
> Attachments: Filters.png, HBASE-19008-1.patch, HBASE-19008-10.patch,
> HBASE-19008-11.patch, HBASE-19008-12.patch, HBASE-19008-13.patch,
> HBASE-19008-14.patch, HBASE-19008-2.patch, HBASE-19008-3.patch,
> HBASE-19008-4.patch, HBASE-19008-5.patch, HBASE-19008-6.patch,
> HBASE-19008-7.patch, HBASE-19008-8.patch, HBASE-19008-9.patch,
> HBASE-19008.patch
>
>
> In HBASE-15410, [~mdrob] reminded me that Filter implementations may not
> write {{equals}} or {{hashCode}} method(s).
> This issue is to add missing {{equals}} or {{hashCode}} method(s) to stock
> Filter implementations such as KeyOnlyFilter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-12233) Bring back root table
[ https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582845#comment-16582845 ] Andrew Purtell commented on HBASE-12233: bq. We'd try and avoid mistakes of the past regards stuff like key scheme and wacky comparator to interpret it. Do you mean just for new root or new root and new splittable meta both? Presume the latter. Seems difficult to have a single-region meta facade for older clients, if it points to a table with new rules they can't parse. It might be better in the long run to split out meta handling into separate code, keep the old stuff as "MetaPre3.0Compat" (but less ugly in the naming, of course), write up a new implementation, and then switch between them given runtime detection of whether a root region exists or not. Could be as simple as checking if a znode to root exists. This could mostly be contained to the client, but I am not saying refactor in the server too wouldn't be useful (it might) > Bring back root table > - > > Key: HBASE-12233 > URL: https://issues.apache.org/jira/browse/HBASE-12233 > Project: HBase > Issue Type: Sub-task >Reporter: Virag Kothari >Assignee: Virag Kothari >Priority: Major > Attachments: HBASE-12233.patch > > > First step towards splitting meta. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21014) Improve Stochastic Balancer to write HDFS favoured node hints for region primary blocks to avoid destroying data locality if needing to use HDFS Balancer
[
https://issues.apache.org/jira/browse/HBASE-21014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582857#comment-16582857
]
Toshihiro Suzuki commented on HBASE-21014:
--
[~harisekhon] Sounds like similar to HBASE-15531? However, it looks like this
is not complete yet.
> Improve Stochastic Balancer to write HDFS favoured node hints for region
> primary blocks to avoid destroying data locality if needing to use HDFS
> Balancer
> -
>
> Key: HBASE-21014
> URL: https://issues.apache.org/jira/browse/HBASE-21014
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Affects Versions: 1.1.2
>Reporter: Hari Sekhon
>Priority: Major
>
> Improve Stochastic Balancer to include the HDFS region location hints to
> avoid HDFS Balancer destroying data locality.
> Right now according to a mix of docs, jiras and mailing list info it appears
> that one must change
> {code:java}
> hbase.master.loadbalancer.class{code}
> to the org.apache.hadoop.hbase.favored.FavoredNodeLoadBalancer as it looks
> like this functionality is only within FavoredNodeBalancer and not the
> standard Stochastic Balancer.
> [http://hbase.apache.org/book.html#_hbase_and_hdfs]
> This is not ideal because we'd still like to use all the heuristics and work
> that has gone in the Stochastic Balancer which I believe right now is the
> best and most mature HBase balancer.
> See also the linked Jiras and this discussion:
> [http://apache-hbase.679495.n3.nabble.com/HDFS-Balancer-td4086607.html]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations
[
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582899#comment-16582899
]
Hadoop QA commented on HBASE-19008:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} hbase-client: The patch generated 1 new + 349
unchanged - 72 fixed = 350 total (was 421) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} hbase-server: The patch generated 0 new + 392
unchanged - 14 fixed = 392 total (was 406) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch hbase-spark passed checkstyle {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
16s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 55s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
1s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}118m 54s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
43s{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
57s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}174m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.security.token.TestZKSecretWatcher |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1
[jira] [Commented] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[ https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582918#comment-16582918 ] Andrew Purtell commented on HBASE-21047: bq. ./hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java:73:import org.junit.After;: Wrong order for 'org.junit.After' import. [ImportOrder] I'll fix this nit on commit, no problem. Let me do some local checks and then get this in. Thanks for the patches! > Object creation of StoreFileScanner thru constructor and close may leave > refCount to -1 > --- > > Key: HBASE-21047 > URL: https://issues.apache.org/jira/browse/HBASE-21047 > Project: HBase > Issue Type: Bug >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21047.branch-1.v1.patch, > HBASE-21047.branch-1.v2.patch, HBASE-21047.v1.patch > > > During object creation "*StoreFileScanner*", it does not increase the > refCount whereas while close it decrements the reader refCount. This will > cause refCount to -1 and isReadReference method was returning true > (refCount.get() != 0 This is causing store file not to be deleted. This may > also cause issue in situation when some thread is holding a scanner but it > may actually be closed due to above bug. Impact of this would be really high. > Attatching patch for the fix which makes sure if reference is held either > thru getScanner method or constructor, ref is always updated. Patch also > contains a test which validates the issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations
[
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582932#comment-16582932
]
Ted Yu commented on HBASE-19008:
Failure in TestZKSecretWatcher was not related to the patch.
> Add missing equals or hashCode method(s) to stock Filter implementations
>
>
> Key: HBASE-19008
> URL: https://issues.apache.org/jira/browse/HBASE-19008
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Major
> Labels: filter
> Attachments: Filters.png, HBASE-19008-1.patch, HBASE-19008-10.patch,
> HBASE-19008-11.patch, HBASE-19008-12.patch, HBASE-19008-13.patch,
> HBASE-19008-14.patch, HBASE-19008-2.patch, HBASE-19008-3.patch,
> HBASE-19008-4.patch, HBASE-19008-5.patch, HBASE-19008-6.patch,
> HBASE-19008-7.patch, HBASE-19008-8.patch, HBASE-19008-9.patch,
> HBASE-19008.patch
>
>
> In HBASE-15410, [~mdrob] reminded me that Filter implementations may not
> write {{equals}} or {{hashCode}} method(s).
> This issue is to add missing {{equals}} or {{hashCode}} method(s) to stock
> Filter implementations such as KeyOnlyFilter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20723) Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them
[
https://issues.apache.org/jira/browse/HBASE-20723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582974#comment-16582974
]
Jerry Chabot commented on HBASE-20723:
--
This also appears to be a problem in HBase 1.3.1. Does anyone know if there is
a patch available that can be applied to 1.3.1?
> Custom hbase.wal.dir results in data loss because we write recovered edits
> into a different place than where the recovering region server looks for them
>
>
> Key: HBASE-20723
> URL: https://issues.apache.org/jira/browse/HBASE-20723
> Project: HBase
> Issue Type: Bug
> Components: Recovery, wal
>Affects Versions: 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 2.0.0
>Reporter: Rohan Pednekar
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.1.0, 1.5.0, 1.4.6
>
> Attachments: 20723.branch-1.txt, 20723.branch-2.txt, 20723.v1.txt,
> 20723.v10.txt, 20723.v2.txt, 20723.v3.txt, 20723.v4.txt, 20723.v5.txt,
> 20723.v5.txt, 20723.v6.txt, 20723.v7.txt, 20723.v8.txt, 20723.v9.txt, logs.zip
>
>
> Description:
> When custom hbase.wal.dir is configured the recovery system uses it in place
> of the HBase root dir and thus constructs an incorrect path for recovered
> edits when splitting WALs. This causes the recovery code in Region Servers to
> believe there are no recovered edits to replay, which causes a loss of writes
> that had not flushed prior to loss of a server.
>
> Reproduction:
> This is an Azure HDInsight HBase cluster with HDP 2.6. and HBase
> 1.1.2.2.6.3.2-14
> By default the underlying data is going to wasb://x@y/hbase
> I tried to move WAL folders to HDFS, which is the SSD mounted on each VM at
> /mnt.
> hbase.wal.dir= hdfs://mycluster/walontest
> hbase.wal.dir.perms=700
> hbase.rootdir.perms=700
> hbase.rootdir=
> wasb://XYZ[@hbaseperf.core.net|mailto:duohbase5ds...@duohbaseperf.blob.core.windows.net]/hbase
> Procedure to reproduce this issue:
> 1. create a table in hbase shell
> 2. insert a row in hbase shell
> 3. reboot the VM which hosts that region
> 4. scan the table in hbase shell and it is empty
> Looking at the region server logs:
> {code:java}
> 2018-06-12 22:08:40,455 INFO [RS_LOG_REPLAY_OPS-wn2-duohba:16020-0-Writer-1]
> wal.WALSplitter: This region's directory doesn't exist:
> hdfs://mycluster/walontest/data/default/tb1/b7fd7db5694eb71190955292b3ff7648.
> It is very likely that it was already split so it's safe to discard those
> edits.
> {code}
> The log split/replay ignored actual WAL due to WALSplitter is looking for the
> region directory in the hbase.wal.dir we specified rather than the
> hbase.rootdir.
> Looking at the source code,
>
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java]
> it uses the rootDir, which is walDir, as the tableDir root path.
> So if we use HBASE-17437, waldir and hbase rootdir are in different path or
> even in different filesystem, then the #5 uses walDir as tableDir is
> apparently wrong.
> CC: [~zyork], [~yuzhih...@gmail.com] Attached the logs for quick review.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20723) Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them
[
https://issues.apache.org/jira/browse/HBASE-20723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582987#comment-16582987
]
Sean Busbey commented on HBASE-20723:
-
are you sure? HBASE-17437 is marked as 1.4.0+?
> Custom hbase.wal.dir results in data loss because we write recovered edits
> into a different place than where the recovering region server looks for them
>
>
> Key: HBASE-20723
> URL: https://issues.apache.org/jira/browse/HBASE-20723
> Project: HBase
> Issue Type: Bug
> Components: Recovery, wal
>Affects Versions: 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 2.0.0
>Reporter: Rohan Pednekar
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.1.0, 1.5.0, 1.4.6
>
> Attachments: 20723.branch-1.txt, 20723.branch-2.txt, 20723.v1.txt,
> 20723.v10.txt, 20723.v2.txt, 20723.v3.txt, 20723.v4.txt, 20723.v5.txt,
> 20723.v5.txt, 20723.v6.txt, 20723.v7.txt, 20723.v8.txt, 20723.v9.txt, logs.zip
>
>
> Description:
> When custom hbase.wal.dir is configured the recovery system uses it in place
> of the HBase root dir and thus constructs an incorrect path for recovered
> edits when splitting WALs. This causes the recovery code in Region Servers to
> believe there are no recovered edits to replay, which causes a loss of writes
> that had not flushed prior to loss of a server.
>
> Reproduction:
> This is an Azure HDInsight HBase cluster with HDP 2.6. and HBase
> 1.1.2.2.6.3.2-14
> By default the underlying data is going to wasb://x@y/hbase
> I tried to move WAL folders to HDFS, which is the SSD mounted on each VM at
> /mnt.
> hbase.wal.dir= hdfs://mycluster/walontest
> hbase.wal.dir.perms=700
> hbase.rootdir.perms=700
> hbase.rootdir=
> wasb://XYZ[@hbaseperf.core.net|mailto:duohbase5ds...@duohbaseperf.blob.core.windows.net]/hbase
> Procedure to reproduce this issue:
> 1. create a table in hbase shell
> 2. insert a row in hbase shell
> 3. reboot the VM which hosts that region
> 4. scan the table in hbase shell and it is empty
> Looking at the region server logs:
> {code:java}
> 2018-06-12 22:08:40,455 INFO [RS_LOG_REPLAY_OPS-wn2-duohba:16020-0-Writer-1]
> wal.WALSplitter: This region's directory doesn't exist:
> hdfs://mycluster/walontest/data/default/tb1/b7fd7db5694eb71190955292b3ff7648.
> It is very likely that it was already split so it's safe to discard those
> edits.
> {code}
> The log split/replay ignored actual WAL due to WALSplitter is looking for the
> region directory in the hbase.wal.dir we specified rather than the
> hbase.rootdir.
> Looking at the source code,
>
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java]
> it uses the rootDir, which is walDir, as the tableDir root path.
> So if we use HBASE-17437, waldir and hbase rootdir are in different path or
> even in different filesystem, then the #5 uses walDir as tableDir is
> apparently wrong.
> CC: [~zyork], [~yuzhih...@gmail.com] Attached the logs for quick review.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582991#comment-16582991
]
Andrew Purtell commented on HBASE-20993:
No commit without consensus, please.
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.2.001.patch,
> HBASE-20993.branch-1.wip.002.patch, HBASE-20993.branch-1.wip.patch
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFallback.java:55)
> Caused by: GSSException: No valid credentials provided (Mechanism level:
> Failed to find any Kerberos tgt)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5
[jira] [Commented] (HBASE-20723) Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them
[
https://issues.apache.org/jira/browse/HBASE-20723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583000#comment-16583000
]
Jerry Chabot commented on HBASE-20723:
--
I had someone confirm looking at the 1.3.1 branch. "If you look at
https://github.com/apache/hbase/blob/rel/1.3.1/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
you can see the "protected final Path rootDir;" data member. This was renamed
to "walDir" in the fix patch for that bug, so it does look like 1.3.1 has the
unpatched code"
We are currently trying to verify use the How to Reproduce steps described
above.
> Custom hbase.wal.dir results in data loss because we write recovered edits
> into a different place than where the recovering region server looks for them
>
>
> Key: HBASE-20723
> URL: https://issues.apache.org/jira/browse/HBASE-20723
> Project: HBase
> Issue Type: Bug
> Components: Recovery, wal
>Affects Versions: 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 2.0.0
>Reporter: Rohan Pednekar
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.1.0, 1.5.0, 1.4.6
>
> Attachments: 20723.branch-1.txt, 20723.branch-2.txt, 20723.v1.txt,
> 20723.v10.txt, 20723.v2.txt, 20723.v3.txt, 20723.v4.txt, 20723.v5.txt,
> 20723.v5.txt, 20723.v6.txt, 20723.v7.txt, 20723.v8.txt, 20723.v9.txt, logs.zip
>
>
> Description:
> When custom hbase.wal.dir is configured the recovery system uses it in place
> of the HBase root dir and thus constructs an incorrect path for recovered
> edits when splitting WALs. This causes the recovery code in Region Servers to
> believe there are no recovered edits to replay, which causes a loss of writes
> that had not flushed prior to loss of a server.
>
> Reproduction:
> This is an Azure HDInsight HBase cluster with HDP 2.6. and HBase
> 1.1.2.2.6.3.2-14
> By default the underlying data is going to wasb://x@y/hbase
> I tried to move WAL folders to HDFS, which is the SSD mounted on each VM at
> /mnt.
> hbase.wal.dir= hdfs://mycluster/walontest
> hbase.wal.dir.perms=700
> hbase.rootdir.perms=700
> hbase.rootdir=
> wasb://XYZ[@hbaseperf.core.net|mailto:duohbase5ds...@duohbaseperf.blob.core.windows.net]/hbase
> Procedure to reproduce this issue:
> 1. create a table in hbase shell
> 2. insert a row in hbase shell
> 3. reboot the VM which hosts that region
> 4. scan the table in hbase shell and it is empty
> Looking at the region server logs:
> {code:java}
> 2018-06-12 22:08:40,455 INFO [RS_LOG_REPLAY_OPS-wn2-duohba:16020-0-Writer-1]
> wal.WALSplitter: This region's directory doesn't exist:
> hdfs://mycluster/walontest/data/default/tb1/b7fd7db5694eb71190955292b3ff7648.
> It is very likely that it was already split so it's safe to discard those
> edits.
> {code}
> The log split/replay ignored actual WAL due to WALSplitter is looking for the
> region directory in the hbase.wal.dir we specified rather than the
> hbase.rootdir.
> Looking at the source code,
>
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java]
> it uses the rootDir, which is walDir, as the tableDir root path.
> So if we use HBASE-17437, waldir and hbase rootdir are in different path or
> even in different filesystem, then the #5 uses walDir as tableDir is
> apparently wrong.
> CC: [~zyork], [~yuzhih...@gmail.com] Attached the logs for quick review.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20723) Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them
[
https://issues.apache.org/jira/browse/HBASE-20723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583042#comment-16583042
]
Jerry Chabot commented on HBASE-20723:
--
BTW, we are using EMR 5.7 with EMRFS so hbase.rootdir and hbase.wal.dir are
different. The former points to S3. The latter to HDFS. This bug refers to
https://issues.apache.org/jira/browse/HBASE-17437 which is needed to allow
hbase.wal.dir to outside the root directory. Did I read that correctly? If so,
how could we specify different locations?
> Custom hbase.wal.dir results in data loss because we write recovered edits
> into a different place than where the recovering region server looks for them
>
>
> Key: HBASE-20723
> URL: https://issues.apache.org/jira/browse/HBASE-20723
> Project: HBase
> Issue Type: Bug
> Components: Recovery, wal
>Affects Versions: 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 2.0.0
>Reporter: Rohan Pednekar
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.1.0, 1.5.0, 1.4.6
>
> Attachments: 20723.branch-1.txt, 20723.branch-2.txt, 20723.v1.txt,
> 20723.v10.txt, 20723.v2.txt, 20723.v3.txt, 20723.v4.txt, 20723.v5.txt,
> 20723.v5.txt, 20723.v6.txt, 20723.v7.txt, 20723.v8.txt, 20723.v9.txt, logs.zip
>
>
> Description:
> When custom hbase.wal.dir is configured the recovery system uses it in place
> of the HBase root dir and thus constructs an incorrect path for recovered
> edits when splitting WALs. This causes the recovery code in Region Servers to
> believe there are no recovered edits to replay, which causes a loss of writes
> that had not flushed prior to loss of a server.
>
> Reproduction:
> This is an Azure HDInsight HBase cluster with HDP 2.6. and HBase
> 1.1.2.2.6.3.2-14
> By default the underlying data is going to wasb://x@y/hbase
> I tried to move WAL folders to HDFS, which is the SSD mounted on each VM at
> /mnt.
> hbase.wal.dir= hdfs://mycluster/walontest
> hbase.wal.dir.perms=700
> hbase.rootdir.perms=700
> hbase.rootdir=
> wasb://XYZ[@hbaseperf.core.net|mailto:duohbase5ds...@duohbaseperf.blob.core.windows.net]/hbase
> Procedure to reproduce this issue:
> 1. create a table in hbase shell
> 2. insert a row in hbase shell
> 3. reboot the VM which hosts that region
> 4. scan the table in hbase shell and it is empty
> Looking at the region server logs:
> {code:java}
> 2018-06-12 22:08:40,455 INFO [RS_LOG_REPLAY_OPS-wn2-duohba:16020-0-Writer-1]
> wal.WALSplitter: This region's directory doesn't exist:
> hdfs://mycluster/walontest/data/default/tb1/b7fd7db5694eb71190955292b3ff7648.
> It is very likely that it was already split so it's safe to discard those
> edits.
> {code}
> The log split/replay ignored actual WAL due to WALSplitter is looking for the
> region directory in the hbase.wal.dir we specified rather than the
> hbase.rootdir.
> Looking at the source code,
>
> [https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java]
> it uses the rootDir, which is walDir, as the tableDir root path.
> So if we use HBASE-17437, waldir and hbase rootdir are in different path or
> even in different filesystem, then the #5 uses walDir as tableDir is
> apparently wrong.
> CC: [~zyork], [~yuzhih...@gmail.com] Attached the logs for quick review.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18477) Umbrella JIRA for HBase Read Replica clusters
[
https://issues.apache.org/jira/browse/HBASE-18477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583063#comment-16583063
]
Hudson commented on HBASE-18477:
Results for branch HBASE-18477
[build #297 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/297/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/297//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/297//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/297//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/297//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Umbrella JIRA for HBase Read Replica clusters
> -
>
> Key: HBASE-18477
> URL: https://issues.apache.org/jira/browse/HBASE-18477
> Project: HBase
> Issue Type: New Feature
>Reporter: Zach York
>Assignee: Zach York
>Priority: Major
> Attachments: HBase Read-Replica Clusters Scope doc.docx, HBase
> Read-Replica Clusters Scope doc.pdf, HBase Read-Replica Clusters Scope
> doc_v2.docx, HBase Read-Replica Clusters Scope doc_v2.pdf
>
>
> Recently, changes (such as HBASE-17437) have unblocked HBase to run with a
> root directory external to the cluster (such as in Amazon S3). This means
> that the data is stored outside of the cluster and can be accessible after
> the cluster has been terminated. One use case that is often asked about is
> pointing multiple clusters to one root directory (sharing the data) to have
> read resiliency in the case of a cluster failure.
>
> This JIRA is an umbrella JIRA to contain all the tasks necessary to create a
> read-replica HBase cluster that is pointed at the same root directory.
>
> This requires making the Read-Replica cluster Read-Only (no metadata
> operation or data operations).
> Separating the hbase:meta table for each cluster (Otherwise HBase gets
> confused with multiple clusters trying to update the meta table with their ip
> addresses)
> Adding refresh functionality for the meta table to ensure new metadata is
> picked up on the read replica cluster.
> Adding refresh functionality for HFiles for a given table to ensure new data
> is picked up on the read replica cluster.
>
> This can be used with any existing cluster that is backed by an external
> filesystem.
>
> Please note that this feature is still quite manual (with the potential for
> automation later).
>
> More information on this particular feature can be found here:
> https://aws.amazon.com/blogs/big-data/setting-up-read-replica-clusters-with-hbase-on-amazon-s3/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20874) Sending compaction descriptions from all regionservers to master.
[ https://issues.apache.org/jira/browse/HBASE-20874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit Goel updated HBASE-20874: --- Attachment: HBASE-20874.master.006.patch > Sending compaction descriptions from all regionservers to master. > - > > Key: HBASE-20874 > URL: https://issues.apache.org/jira/browse/HBASE-20874 > Project: HBase > Issue Type: Sub-task >Reporter: Mohit Goel >Assignee: Mohit Goel >Priority: Minor > Attachments: HBASE-20874.master.004.patch, > HBASE-20874.master.005.patch, HBASE-20874.master.006.patch > > > Need to send the compaction description from region servers to Master , to > let master know of the entire compaction state of the cluster. Further need > to change the implementation of client Side API than like getCompactionState, > which will consult master for the result instead of sending individual > request to regionservers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[ https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21047: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.4.7 2.1.1 2.2.0 2.0.2 1.3.3 3.0.0 Status: Resolved (was: Patch Available) > Object creation of StoreFileScanner thru constructor and close may leave > refCount to -1 > --- > > Key: HBASE-21047 > URL: https://issues.apache.org/jira/browse/HBASE-21047 > Project: HBase > Issue Type: Bug >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.0.2, 2.2.0, 2.1.1, 1.4.7 > > Attachments: HBASE-21047.branch-1.v1.patch, > HBASE-21047.branch-1.v2.patch, HBASE-21047.v1.patch > > > During object creation "*StoreFileScanner*", it does not increase the > refCount whereas while close it decrements the reader refCount. This will > cause refCount to -1 and isReadReference method was returning true > (refCount.get() != 0 This is causing store file not to be deleted. This may > also cause issue in situation when some thread is holding a scanner but it > may actually be closed due to above bug. Impact of this would be really high. > Attatching patch for the fix which makes sure if reference is held either > thru getScanner method or constructor, ref is always updated. Patch also > contains a test which validates the issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[
https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583133#comment-16583133
]
Andrew Purtell commented on HBASE-20940:
Thanks for the latest patches.
This test failure looks germane:
{noformat}
[ERROR] Tests run: 107, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
81.343 s <<< FAILURE! - in org.apache.hadoop.hbase.regionserver.TestHRegion
[ERROR] testMerge(org.apache.hadoop.hbase.regionserver.TestHRegion) Time
elapsed: 0.592 s <<< ERROR!
java.io.IOException: Unable to merge regions
testMerge,,1534422923610.9cbd17d09ee3048c65721d90ee02248a. and
testMerge,mdo,1534422923610.3e0185086609af44c9da53f9f1b829bb.
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testMerge(TestHRegion.java:2703)
{noformat}
Let me see if I can reproduce it.
> HStore.cansplit should not allow split to happen if it has references
> -
>
> Key: HBASE-20940
> URL: https://issues.apache.org/jira/browse/HBASE-20940
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.2
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20940.branch-1.3.v1.patch,
> HBASE-20940.branch-1.3.v2.patch, HBASE-20940.branch-1.v1.patch,
> HBASE-20940.branch-1.v2.patch, HBASE-20940.v1.patch, HBASE-20940.v2.patch,
> HBASE-20940.v3.patch, HBASE-20940.v4.patch
>
>
> When split happens and immediately another split happens, it may result into
> a split of a region who still has references to its parent. More details
> about scenario can be found here HBASE-20933
> HStore.hasReferences should check from fs.storefile rather than in memory
> objects.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[ https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583134#comment-16583134 ] Hudson commented on HBASE-21047: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #451 (See [https://builds.apache.org/job/HBase-1.3-IT/451/]) HBASE-21047 Object creation of StoreFileScanner thru constructor and (apurtell: rev efafa4108f75819c27e999a6b9e00fe9a37879ee) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java > Object creation of StoreFileScanner thru constructor and close may leave > refCount to -1 > --- > > Key: HBASE-21047 > URL: https://issues.apache.org/jira/browse/HBASE-21047 > Project: HBase > Issue Type: Bug >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.0.2, 2.2.0, 2.1.1, 1.4.7 > > Attachments: HBASE-21047.branch-1.v1.patch, > HBASE-21047.branch-1.v2.patch, HBASE-21047.v1.patch > > > During object creation "*StoreFileScanner*", it does not increase the > refCount whereas while close it decrements the reader refCount. This will > cause refCount to -1 and isReadReference method was returning true > (refCount.get() != 0 This is causing store file not to be deleted. This may > also cause issue in situation when some thread is holding a scanner but it > may actually be closed due to above bug. Impact of this would be really high. > Attatching patch for the fix which makes sure if reference is held either > thru getScanner method or constructor, ref is always updated. Patch also > contains a test which validates the issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20705) Having RPC Quota on a table prevents Space quota to be recreated/removed
[
https://issues.apache.org/jira/browse/HBASE-20705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583141#comment-16583141
]
Hudson commented on HBASE-20705:
Results for branch branch-2
[build #1120 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1120/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1120//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1120//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1120//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Having RPC Quota on a table prevents Space quota to be recreated/removed
>
>
> Key: HBASE-20705
> URL: https://issues.apache.org/jira/browse/HBASE-20705
> Project: HBase
> Issue Type: Bug
>Reporter: Biju Nair
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-20705.master.001.patch
>
>
> * Property {{hbase.quota.remove.on.table.delete}} is set to {{true}} by
> default
> * Create a table and set RPC and Space quota
> {noformat}
> hbase(main):022:0> create 't2','cf1'
> Created table t2
> Took 0.7420 seconds
> => Hbase::Table - t2
> hbase(main):023:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0105 seconds
> hbase(main):024:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0186 seconds
> hbase(main):025:0> list_quotas
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT =>
> 10M/sec, SCOPE => MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824, VIOLATION_POLICY
> => NO_WRITES{noformat}
> * Drop the table and the Space quota is set to {{REMOVE => true}}
> {noformat}
> hbase(main):026:0> disable 't2'
> Took 0.4363 seconds
> hbase(main):027:0> drop 't2'
> Took 0.2344 seconds
> hbase(main):028:0> list_quotas
> TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE => true
> USER => u1 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT => 10M/sec,
> SCOPE => MACHINE{noformat}
> * Recreate the table and set Space quota back. The Space quota on the table
> is still set to {{REMOVE => true}}
> {noformat}
> hbase(main):029:0> create 't2','cf1'
> Created table t2
> Took 0.7348 seconds
> => Hbase::Table - t2
> hbase(main):031:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0088 seconds
> hbase(main):032:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT =>
> 10M/sec, SCOPE => MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE => true{noformat}
> * Remove RPC quota and drop the table, the Space Quota is not removed
> {noformat}
> hbase(main):033:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT => NONE
> Took 0.0193 seconds
> hbase(main):036:0> disable 't2'
> Took 0.4305 seconds
> hbase(main):037:0> drop 't2'
> Took 0.2353 seconds
> hbase(main):038:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE
> => true{noformat}
> * Deleting the quota entry from {{hbase:quota}} seems to be the option to
> reset it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583140#comment-16583140
]
Hudson commented on HBASE-21062:
Results for branch branch-2
[build #1120 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1120/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1120//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1120//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1120//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch,
> HBASE-21062.002.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20387) flaky infrastructure should work for all branches
[
https://issues.apache.org/jira/browse/HBASE-20387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583144#comment-16583144
]
Hudson commented on HBASE-20387:
Results for branch HBASE-20387
[build #14 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/14/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/14//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/14//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/14//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> flaky infrastructure should work for all branches
> -
>
> Key: HBASE-20387
> URL: https://issues.apache.org/jira/browse/HBASE-20387
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-20387.0.patch, HBASE-20387.1.patch
>
>
> We need a flaky list per-branch, since what does/does not work reliably on
> master isn't really relevant to our older maintenance release lines.
> We should just make the invocation a step in the current per-branch nightly
> jobs, prior to when we need the list in the stages that run unit tests. We
> can publish it in the nightly job as well so that precommit can still get it.
> (and can fetch it per-branch if needed)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20940: --- Status: Open (was: Patch Available) > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20940.branch-1.3.v1.patch, > HBASE-20940.branch-1.3.v2.patch, HBASE-20940.branch-1.v1.patch, > HBASE-20940.branch-1.v2.patch, HBASE-20940.v1.patch, HBASE-20940.v2.patch, > HBASE-20940.v3.patch, HBASE-20940.v4.patch > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583148#comment-16583148 ] Andrew Purtell commented on HBASE-20940: Yes, the test failure is reproducible. I tested with the branch-1.3 patch > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20940.branch-1.3.v1.patch, > HBASE-20940.branch-1.3.v2.patch, HBASE-20940.branch-1.v1.patch, > HBASE-20940.branch-1.v2.patch, HBASE-20940.v1.patch, HBASE-20940.v2.patch, > HBASE-20940.v3.patch, HBASE-20940.v4.patch > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583175#comment-16583175
]
stack commented on HBASE-21062:
---
+1 for branch-2.0. thanks
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch,
> HBASE-21062.002.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-12233) Bring back root table
[ https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583186#comment-16583186 ] stack commented on HBASE-12233: --- Thanks [~apurtell]. Was thinking root only. Redoing meta would require a rewrite of meta on upgrade? And rewrite of data in proxy on each access (we'd be doing the serializations anyways so maybe not too bad). Maybe keep meta as is, introduce root, and if proxy works, later could use it as facade rewriting meta (feel free to ignore crazy talk..) > Bring back root table > - > > Key: HBASE-12233 > URL: https://issues.apache.org/jira/browse/HBASE-12233 > Project: HBase > Issue Type: Sub-task >Reporter: Virag Kothari >Assignee: Virag Kothari >Priority: Major > Attachments: HBASE-12233.patch > > > First step towards splitting meta. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20881) Introduce a region transition procedure to handle all the state transition for a region
[ https://issues.apache.org/jira/browse/HBASE-20881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583190#comment-16583190 ] stack commented on HBASE-20881: --- One thought is that patch is big so if tests pass, we could commit and work through outstanding issues in followons? > Introduce a region transition procedure to handle all the state transition > for a region > --- > > Key: HBASE-20881 > URL: https://issues.apache.org/jira/browse/HBASE-20881 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-20881-v1.patch, HBASE-20881-v10.patch, > HBASE-20881-v11.patch, HBASE-20881-v12.patch, HBASE-20881-v13.patch, > HBASE-20881-v2.patch, HBASE-20881-v3.patch, HBASE-20881-v4.patch, > HBASE-20881-v4.patch, HBASE-20881-v5.patch, HBASE-20881-v6.patch, > HBASE-20881-v7.patch, HBASE-20881-v7.patch, HBASE-20881-v8.patch, > HBASE-20881-v9.patch, HBASE-20881.patch > > > Now have an AssignProcedure, an UnssignProcedure, and also a > MoveRegionProcedure which schedules an AssignProcedure and an > UnssignProcedure to move a region. This makes the logic a bit complicated, as > MRP is not a RIT, so when SCP can not interrupt it directly... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-12233) Bring back root table
[ https://issues.apache.org/jira/browse/HBASE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583201#comment-16583201 ] Andrew Purtell commented on HBASE-12233: Close, I meant rewrite meta on upgrade, don't bother with the proxy, do it all right. However, fork enough in the client so the client can talk to either an old or upgraded (post meta semi-singularity) cluster. > Bring back root table > - > > Key: HBASE-12233 > URL: https://issues.apache.org/jira/browse/HBASE-12233 > Project: HBase > Issue Type: Sub-task >Reporter: Virag Kothari >Assignee: Virag Kothari >Priority: Major > Attachments: HBASE-12233.patch > > > First step towards splitting meta. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-20734: -- Attachment: HBASE-20734.master.001.patch > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.master.001.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583202#comment-16583202 ] Zach York commented on HBASE-20734: --- Apparently I lost my reviewboard password... let me try to recover it so I can put one up, but updated patch attached. > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.master.001.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-20734: -- Attachment: HBASE-20734.master.002.patch > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.master.001.patch, HBASE-20734.master.002.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583205#comment-16583205 ] Zach York commented on HBASE-20734: --- Got it now, RB up! > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.master.001.patch, HBASE-20734.master.002.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20874) Sending compaction descriptions from all regionservers to master.
[
https://issues.apache.org/jira/browse/HBASE-20874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583222#comment-16583222
]
Hadoop QA commented on HBASE-20874:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 12 new + 413 unchanged - 0 fixed =
425 total (was 413) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 5s{color} | {color:orange} The patch generated 13 new + 749 unchanged - 0
fixed = 762 total (was 749) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
21s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 52s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
2m 3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
59s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}114m 27s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
45s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
15s{color} | {color:green} The patch does not generate ASF License
[jira] [Commented] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[
https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583291#comment-16583291
]
Hudson commented on HBASE-21047:
Results for branch branch-2.1
[build #200 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/200/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/200//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/200//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/200//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Object creation of StoreFileScanner thru constructor and close may leave
> refCount to -1
> ---
>
> Key: HBASE-21047
> URL: https://issues.apache.org/jira/browse/HBASE-21047
> Project: HBase
> Issue Type: Bug
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.0.2, 2.2.0, 2.1.1, 1.4.7
>
> Attachments: HBASE-21047.branch-1.v1.patch,
> HBASE-21047.branch-1.v2.patch, HBASE-21047.v1.patch
>
>
> During object creation "*StoreFileScanner*", it does not increase the
> refCount whereas while close it decrements the reader refCount. This will
> cause refCount to -1 and isReadReference method was returning true
> (refCount.get() != 0 This is causing store file not to be deleted. This may
> also cause issue in situation when some thread is holding a scanner but it
> may actually be closed due to above bug. Impact of this would be really high.
> Attatching patch for the fix which makes sure if reference is held either
> thru getScanner method or constructor, ref is always updated. Patch also
> contains a test which validates the issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[
https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583294#comment-16583294
]
Hudson commented on HBASE-21047:
Results for branch branch-2
[build #1121 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1121/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1121//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1121//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1121//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Object creation of StoreFileScanner thru constructor and close may leave
> refCount to -1
> ---
>
> Key: HBASE-21047
> URL: https://issues.apache.org/jira/browse/HBASE-21047
> Project: HBase
> Issue Type: Bug
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.0.2, 2.2.0, 2.1.1, 1.4.7
>
> Attachments: HBASE-21047.branch-1.v1.patch,
> HBASE-21047.branch-1.v2.patch, HBASE-21047.v1.patch
>
>
> During object creation "*StoreFileScanner*", it does not increase the
> refCount whereas while close it decrements the reader refCount. This will
> cause refCount to -1 and isReadReference method was returning true
> (refCount.get() != 0 This is causing store file not to be deleted. This may
> also cause issue in situation when some thread is holding a scanner but it
> may actually be closed due to above bug. Impact of this would be really high.
> Attatching patch for the fix which makes sure if reference is held either
> thru getScanner method or constructor, ref is always updated. Patch also
> contains a test which validates the issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[
https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583336#comment-16583336
]
Hudson commented on HBASE-21047:
Results for branch branch-2.0
[build #688 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/688/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/688//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/688//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/688//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Object creation of StoreFileScanner thru constructor and close may leave
> refCount to -1
> ---
>
> Key: HBASE-21047
> URL: https://issues.apache.org/jira/browse/HBASE-21047
> Project: HBase
> Issue Type: Bug
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.0.2, 2.2.0, 2.1.1, 1.4.7
>
> Attachments: HBASE-21047.branch-1.v1.patch,
> HBASE-21047.branch-1.v2.patch, HBASE-21047.v1.patch
>
>
> During object creation "*StoreFileScanner*", it does not increase the
> refCount whereas while close it decrements the reader refCount. This will
> cause refCount to -1 and isReadReference method was returning true
> (refCount.get() != 0 This is causing store file not to be deleted. This may
> also cause issue in situation when some thread is holding a scanner but it
> may actually be closed due to above bug. Impact of this would be really high.
> Attatching patch for the fix which makes sure if reference is held either
> thru getScanner method or constructor, ref is always updated. Patch also
> contains a test which validates the issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20387) flaky infrastructure should work for all branches
[ https://issues.apache.org/jira/browse/HBASE-20387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583345#comment-16583345 ] Hudson commented on HBASE-20387: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #452 (See [https://builds.apache.org/job/HBase-1.3-IT/452/]) HBASE-20387 turn flaky test tracking infra into per-branch pipeline. (busbey: rev 0c0c723e2559211fff73f9f876e2a582a60680a8) * (edit) dev-support/Jenkinsfile * (add) dev-support/flaky-tests/report-flakies.py * (add) dev-support/flaky-tests/flaky-reporting.Jenkinsfile * (add) dev-support/flaky-tests/findHangingTests.py * (edit) dev-support/hbase_nightly_yetus.sh * (delete) dev-support/findHangingTests.py * (add) dev-support/flaky-tests/run-flaky-tests.Jenkinsfile * (add) dev-support/Dockerfile * (add) dev-support/flaky-tests/flaky-dashboard-template.html > flaky infrastructure should work for all branches > - > > Key: HBASE-20387 > URL: https://issues.apache.org/jira/browse/HBASE-20387 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Attachments: HBASE-20387.0.patch, HBASE-20387.1.patch > > > We need a flaky list per-branch, since what does/does not work reliably on > master isn't really relevant to our older maintenance release lines. > We should just make the invocation a step in the current per-branch nightly > jobs, prior to when we need the list in the stages that run unit tests. We > can publish it in the nightly job as well so that precommit can still get it. > (and can fetch it per-branch if needed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20387) flaky infrastructure should work for all branches
[ https://issues.apache.org/jira/browse/HBASE-20387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583346#comment-16583346 ] Hudson commented on HBASE-20387: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1146 (See [https://builds.apache.org/job/HBase-1.2-IT/1146/]) HBASE-20387 turn flaky test tracking infra into per-branch pipeline. (busbey: rev 9037405d7a02a19166a1efe69c62c8ba26d5c01f) * (add) dev-support/flaky-tests/findHangingTests.py * (delete) dev-support/findHangingTests.py * (add) dev-support/flaky-tests/report-flakies.py * (add) dev-support/flaky-tests/flaky-reporting.Jenkinsfile * (edit) dev-support/Jenkinsfile * (add) dev-support/flaky-tests/flaky-dashboard-template.html * (add) dev-support/Dockerfile * (edit) dev-support/hbase_nightly_yetus.sh * (add) dev-support/flaky-tests/run-flaky-tests.Jenkinsfile > flaky infrastructure should work for all branches > - > > Key: HBASE-20387 > URL: https://issues.apache.org/jira/browse/HBASE-20387 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Attachments: HBASE-20387.0.patch, HBASE-20387.1.patch > > > We need a flaky list per-branch, since what does/does not work reliably on > master isn't really relevant to our older maintenance release lines. > We should just make the invocation a step in the current per-branch nightly > jobs, prior to when we need the list in the stages that run unit tests. We > can publish it in the nightly job as well so that precommit can still get it. > (and can fetch it per-branch if needed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20387) flaky infrastructure should work for all branches
[ https://issues.apache.org/jira/browse/HBASE-20387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20387: Resolution: Fixed Fix Version/s: 1.4.7 2.1.1 2.2.0 2.0.2 1.3.3 1.2.7 1.5.0 3.0.0 Status: Resolved (was: Patch Available) > flaky infrastructure should work for all branches > - > > Key: HBASE-20387 > URL: https://issues.apache.org/jira/browse/HBASE-20387 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.2.7, 1.3.3, 2.0.2, 2.2.0, 2.1.1, 1.4.7 > > Attachments: HBASE-20387.0.patch, HBASE-20387.1.patch > > > We need a flaky list per-branch, since what does/does not work reliably on > master isn't really relevant to our older maintenance release lines. > We should just make the invocation a step in the current per-branch nightly > jobs, prior to when we need the list in the stages that run unit tests. We > can publish it in the nightly job as well so that precommit can still get it. > (and can fetch it per-branch if needed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21058) Nightly tests for branches 1 fail to build ref guide
[
https://issues.apache.org/jira/browse/HBASE-21058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583364#comment-16583364
]
Sean Busbey commented on HBASE-21058:
-
bump. can I get a review here?
> Nightly tests for branches 1 fail to build ref guide
>
>
> Key: HBASE-21058
> URL: https://issues.apache.org/jira/browse/HBASE-21058
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-21058.0.patch
>
>
> Nightly on all branches 1 reports failure to get a PDF version of the ref
> guide
> {code}
> -1refguide2m 14s patch failed to produce the pdf version of the
> reference guide.
> {code}
> Actual build log looks clean
> {code}
> [INFO] --- asciidoctor-maven-plugin:1.5.2.1:process-asciidoc (output-pdf) @
> hbase ---
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for pass
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> [INFO] Rendered /testptch/hbase/src/main/asciidoc/book.adoc
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21056) Findbugs false positive: BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583363#comment-16583363
]
Sean Busbey commented on HBASE-21056:
-
bump. can I get a review here?
> Findbugs false positive: BucketCache.persistToFile may fail to clean up
> java.io.OutputStream
> -
>
> Key: HBASE-21056
> URL: https://issues.apache.org/jira/browse/HBASE-21056
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-21056.0.patch
>
>
> Found by the nightly job via FindBugs:
> {code}
> FindBugs module:hbase-server
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail
> to clean up java.io.OutputStream Obligation to clean up resource created at
> BucketCache.java:up java.io.OutputStream Obligation to clean up resource
> created at BucketCache.java:[line 1089] is not discharged
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20387) flaky infrastructure should work for all branches
[ https://issues.apache.org/jira/browse/HBASE-20387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583367#comment-16583367 ] Hudson commented on HBASE-20387: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #453 (See [https://builds.apache.org/job/HBase-1.3-IT/453/]) HBASE-20387 ADDENDUM backport findHangingTests.py changes from master to (busbey: rev d95e664245b886da97f8ea3d0fbd080c37ef9db1) * (edit) dev-support/flaky-tests/findHangingTests.py > flaky infrastructure should work for all branches > - > > Key: HBASE-20387 > URL: https://issues.apache.org/jira/browse/HBASE-20387 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.2.7, 1.3.3, 2.0.2, 2.2.0, 2.1.1, 1.4.7 > > Attachments: HBASE-20387.0.patch, HBASE-20387.1.patch > > > We need a flaky list per-branch, since what does/does not work reliably on > master isn't really relevant to our older maintenance release lines. > We should just make the invocation a step in the current per-branch nightly > jobs, prior to when we need the list in the stages that run unit tests. We > can publish it in the nightly job as well so that precommit can still get it. > (and can fetch it per-branch if needed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20387) flaky infrastructure should work for all branches
[ https://issues.apache.org/jira/browse/HBASE-20387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583370#comment-16583370 ] Hudson commented on HBASE-20387: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1147 (See [https://builds.apache.org/job/HBase-1.2-IT/1147/]) HBASE-20387 ADDENDUM backport findHangingTests.py changes from master to (busbey: rev 2676d498f5b515dfb7cb6b08d3458480b1b6bfbc) * (edit) dev-support/flaky-tests/findHangingTests.py > flaky infrastructure should work for all branches > - > > Key: HBASE-20387 > URL: https://issues.apache.org/jira/browse/HBASE-20387 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.2.7, 1.3.3, 2.0.2, 2.2.0, 2.1.1, 1.4.7 > > Attachments: HBASE-20387.0.patch, HBASE-20387.1.patch > > > We need a flaky list per-branch, since what does/does not work reliably on > master isn't really relevant to our older maintenance release lines. > We should just make the invocation a step in the current per-branch nightly > jobs, prior to when we need the list in the stages that run unit tests. We > can publish it in the nightly job as well so that precommit can still get it. > (and can fetch it per-branch if needed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Khandelwal updated HBASE-20940: -- Attachment: HBASE-20940.branch-1.v3.patch > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20940.branch-1.3.v1.patch, > HBASE-20940.branch-1.3.v2.patch, HBASE-20940.branch-1.v1.patch, > HBASE-20940.branch-1.v2.patch, HBASE-20940.branch-1.v3.patch, > HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch, > HBASE-20940.v4.patch > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Khandelwal updated HBASE-20940: -- Attachment: result_HBASE-20940.branch-1.v2.log > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20940.branch-1.3.v1.patch, > HBASE-20940.branch-1.3.v2.patch, HBASE-20940.branch-1.v1.patch, > HBASE-20940.branch-1.v2.patch, HBASE-20940.branch-1.v3.patch, > HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch, > HBASE-20940.v4.patch, result_HBASE-20940.branch-1.v2.log > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[
https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583430#comment-16583430
]
Vishal Khandelwal commented on HBASE-20940:
---
Thanks [~apurtell] for looking into it and running the test. I am not sure why
my previous run did not show "org.apache.hadoop.hbase.regionserver.TestHRegion"
at all :( [^result_HBASE-20940.branch-1.v2.log] .
I have attached the latest patch fixing the TestHRegion#testMerge and following
are the results. Full run also seems fine to me now.
{code:java}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running
org.apache.hadoop.hbase.master.procedure.TestModifyNamespaceProcedure
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.131 s
- in org.apache.hadoop.hbase.master.procedure.TestModifyNamespaceProcedure
[INFO] Running org.apache.hadoop.hbase.regionserver.TestHRegion
[INFO] Tests run: 105, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 70.823
s - in org.apache.hadoop.hbase.regionserver.TestHRegion
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 111, Failures: 0, Errors: 0, Skipped: 0
{code}
> HStore.cansplit should not allow split to happen if it has references
> -
>
> Key: HBASE-20940
> URL: https://issues.apache.org/jira/browse/HBASE-20940
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.2
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20940.branch-1.3.v1.patch,
> HBASE-20940.branch-1.3.v2.patch, HBASE-20940.branch-1.v1.patch,
> HBASE-20940.branch-1.v2.patch, HBASE-20940.branch-1.v3.patch,
> HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch,
> HBASE-20940.v4.patch, result_HBASE-20940.branch-1.v2.log
>
>
> When split happens and immediately another split happens, it may result into
> a split of a region who still has references to its parent. More details
> about scenario can be found here HBASE-20933
> HStore.hasReferences should check from fs.storefile rather than in memory
> objects.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20734: --- Status: Patch Available (was: Open) > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.master.001.patch, HBASE-20734.master.002.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
