date:20141127


[ 
https://issues.apache.org/jira/browse/HBASE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227397#comment-14227397
 ] 

Hudson commented on HBASE-12558:


SUCCESS: Integrated in HBase-1.0 #515 (See 
[https://builds.apache.org/job/HBase-1.0/515/])
HBASE-12558 TestHCM.testClusterStatus Unexpected exception, 
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException but 
wasjunit.framework.AssertionFailedError -- DISABLING FAILING TEST (stack: rev 
c3b99a5406d02cc29b7078aa1682241fa0eb0726)
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java


 TestHCM.testClusterStatus Unexpected exception, 
 expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException 
 but wasjunit.framework.AssertionFailedError
 -

 Key: HBASE-12558
 URL: https://issues.apache.org/jira/browse/HBASE-12558
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.2

 Attachments: 12558-master.patch, 12558.ignore.txt


 Happens for me reliably on mac os x. I looked at fixing it. The listener is 
 not noticing the publish for whatever reason.  Thats where I stopped.
 {code}
 java.lang.Exception: Unexpected exception, 
 expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException 
 but wasjunit.framework.AssertionFailedError
   at junit.framework.Assert.fail(Assert.java:57)
   at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3537)
   at 
 org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:273)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12558) TestHCM.testClusterStatus Unexpected exception, expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException but wasjunit.framework.AssertionFailedErro


[ 
https://issues.apache.org/jira/browse/HBASE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227399#comment-14227399
 ] 

Hudson commented on HBASE-12558:


SUCCESS: Integrated in HBase-TRUNK #5833 (See 
[https://builds.apache.org/job/HBase-TRUNK/5833/])
HBASE-12558 TestHCM.testClusterStatus Unexpected exception, 
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException but 
wasjunit.framework.AssertionFailedError -- DISABLING FAILING TEST (stack: rev 
aa0bd50fd40d8090c5a98cbde063621eadd988f8)
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java


 TestHCM.testClusterStatus Unexpected exception, 
 expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException 
 but wasjunit.framework.AssertionFailedError
 -

 Key: HBASE-12558
 URL: https://issues.apache.org/jira/browse/HBASE-12558
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.2

 Attachments: 12558-master.patch, 12558.ignore.txt


 Happens for me reliably on mac os x. I looked at fixing it. The listener is 
 not noticing the publish for whatever reason.  Thats where I stopped.
 {code}
 java.lang.Exception: Unexpected exception, 
 expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException 
 but wasjunit.framework.AssertionFailedError
   at junit.framework.Assert.fail(Assert.java:57)
   at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3537)
   at 
 org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:273)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11902) RegionServer was blocked while aborting

2014-11-27 Thread Qiang Tian (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qiang Tian updated HBASE-11902:
---
Status: Open  (was: Patch Available)

 RegionServer was blocked while aborting
 ---

 Key: HBASE-11902
 URL: https://issues.apache.org/jira/browse/HBASE-11902
 Project: HBase
  Issue Type: Bug
  Components: regionserver, wal
Affects Versions: 0.98.4
 Environment: hbase-0.98.4, hadoop-2.3.0-cdh5.1, jdk1.7
Reporter: Victor Xu
Assignee: Qiang Tian
 Attachments: hbase-hadoop-regionserver-hadoop461.cm6.log, 
 hbase11902-master.patch, hbase11902-master_v2.patch, jstack_hadoop461.cm6.log


 Generally, regionserver automatically aborts when isHealth() returns false. 
 But it sometimes got blocked while aborting. I saved the jstack and logs, and 
 found out that it was caused by datanodes failures. The regionserver60020 
 thread was blocked while closing WAL. 
 This issue doesn't happen so frequently, but if it happens, it always leads 
 to huge amount of requests failure. The only way to do is KILL -9.
 I think it's a bug, but I haven't found a decent solution. Does anyone have 
 the same problem?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11902) RegionServer was blocked while aborting

2014-11-27 Thread Qiang Tian (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qiang Tian updated HBASE-11902:
---
Attachment: hbase11902-master_v3.patch

Local run passed.
based on HBASE-8208(that is why wal.sync is introduced in memstore flush), we 
also need to fail the flush when getting HDFS failure.



 RegionServer was blocked while aborting
 ---

 Key: HBASE-11902
 URL: https://issues.apache.org/jira/browse/HBASE-11902
 Project: HBase
  Issue Type: Bug
  Components: regionserver, wal
Affects Versions: 0.98.4
 Environment: hbase-0.98.4, hadoop-2.3.0-cdh5.1, jdk1.7
Reporter: Victor Xu
Assignee: Qiang Tian
 Attachments: hbase-hadoop-regionserver-hadoop461.cm6.log, 
 hbase11902-master.patch, hbase11902-master_v2.patch, 
 hbase11902-master_v3.patch, jstack_hadoop461.cm6.log


 Generally, regionserver automatically aborts when isHealth() returns false. 
 But it sometimes got blocked while aborting. I saved the jstack and logs, and 
 found out that it was caused by datanodes failures. The regionserver60020 
 thread was blocked while closing WAL. 
 This issue doesn't happen so frequently, but if it happens, it always leads 
 to huge amount of requests failure. The only way to do is KILL -9.
 I think it's a bug, but I haven't found a decent solution. Does anyone have 
 the same problem?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11902) RegionServer was blocked while aborting

2014-11-27 Thread Qiang Tian (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qiang Tian updated HBASE-11902:
---
Status: Patch Available  (was: Open)

 RegionServer was blocked while aborting
 ---

 Key: HBASE-11902
 URL: https://issues.apache.org/jira/browse/HBASE-11902
 Project: HBase
  Issue Type: Bug
  Components: regionserver, wal
Affects Versions: 0.98.4
 Environment: hbase-0.98.4, hadoop-2.3.0-cdh5.1, jdk1.7
Reporter: Victor Xu
Assignee: Qiang Tian
 Attachments: hbase-hadoop-regionserver-hadoop461.cm6.log, 
 hbase11902-master.patch, hbase11902-master_v2.patch, 
 hbase11902-master_v3.patch, jstack_hadoop461.cm6.log


 Generally, regionserver automatically aborts when isHealth() returns false. 
 But it sometimes got blocked while aborting. I saved the jstack and logs, and 
 found out that it was caused by datanodes failures. The regionserver60020 
 thread was blocked while closing WAL. 
 This issue doesn't happen so frequently, but if it happens, it always leads 
 to huge amount of requests failure. The only way to do is KILL -9.
 I think it's a bug, but I haven't found a decent solution. Does anyone have 
 the same problem?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12581) TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 addendum)


[ 
https://issues.apache.org/jira/browse/HBASE-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227457#comment-14227457
 ] 

ramkrishna.s.vasudevan commented on HBASE-12581:


Ok I found the issue.  I don't think it is not a problem with the test or the 
way things work.  I still need to see whether the recent Connection related 
changes is slowing things in any way.
The problem is this

We create a table for every test and set OWNER as the owner of the table with 
RWXCA.  So this information of adding owner to the table is used in the 
ACL.postCreateTableHandler.  We create a HTable instance on the ACL table and 
do a put saying OWNER is the owner of the table.
The ACL.postPut() is the place where we add this information to the table ZK. 
But this tests just after creating the table tries to add a PUT to the user 
table with ACL permissions for the other user - but the operation is performed 
as part of OWNER.
The initial check whether a WRITE permission is associated with a user first 
happens in ACL.prePut
{code}
 User user = getActiveUser();
checkForReservedTagPresence(user, put);
RegionCoprocessorEnvironment env = c.getEnvironment();
Mapbyte[],? extends CollectionCell families = put.getFamilyCellMap();
AuthResult authResult = permissionGranted(OpType.PUT, user, env, families, 
Action.WRITE);
logResult(authResult);
if (!authResult.isAllowed()) {
  if (cellFeaturesEnabled  !compatibleEarlyTermination) {
put.setAttribute(CHECK_COVERING_PERM, TRUE);
  } else {
throw new AccessDeniedException(Insufficient permissions  + 
authResult.toContextString());
  }
}
{code}
Now as the postPut update for the updateACL in ZK has not yet completed by this 
time we see that the user does not have permission on the table directly.  
Hence the flow further tries to see if the permission is added as a Cell ACL 
for the user OWNER. Anyway that is not there and so the test fails in 
ACL.preBatchMutate
{code}
if (!authResult.isAllowed()) {
throw new AccessDeniedException(Insufficient permissions 
+ authResult.toContextString());
  }
{code}
I do think this could even be a real time problem also if just after creating a 
table with a owner, if a Put happens that put may be denied access as happens 
in the test.
Introducing a sleep after setUp() method of the test makes the test pass 
consistently.

 TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 
 addendum)
 ---

 Key: HBASE-12581
 URL: https://issues.apache.org/jira/browse/HBASE-12581
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.2

 Attachments: 12581.addendum.txt, 12581.txt, acls.txt


 TestCellACLWithMultipleVersions failed after HBASE-12404 went in (though it 
 passed twice on hadoopqa!).  Fails locally too.  Here, make a Connection when 
 we go to check perms.  That seems to fix it. Going to commit since patch is 
 just more of hbase-12404... this is in essence and addendum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12558) TestHCM.testClusterStatus Unexpected exception, expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException but wasjunit.framework.AssertionFailedErro


[ 
https://issues.apache.org/jira/browse/HBASE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227508#comment-14227508
 ] 

stack commented on HBASE-12558:
---

[~tianq] suggests trying a revert on HBASE-12359  Could see if that fixes it.  
I seem to remember this being an issue before HBASE-12359 but my history could 
be wrong.  Worth a try

 TestHCM.testClusterStatus Unexpected exception, 
 expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException 
 but wasjunit.framework.AssertionFailedError
 -

 Key: HBASE-12558
 URL: https://issues.apache.org/jira/browse/HBASE-12558
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.2

 Attachments: 12558-master.patch, 12558.ignore.txt


 Happens for me reliably on mac os x. I looked at fixing it. The listener is 
 not noticing the publish for whatever reason.  Thats where I stopped.
 {code}
 java.lang.Exception: Unexpected exception, 
 expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException 
 but wasjunit.framework.AssertionFailedError
   at junit.framework.Assert.fail(Assert.java:57)
   at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3537)
   at 
 org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:273)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12592) Fix the truncate feature or purge it.

stack created HBASE-12592:
-

 Summary: Fix the truncate feature or purge it.
 Key: HBASE-12592
 URL: https://issues.apache.org/jira/browse/HBASE-12592
 Project: HBase
  Issue Type: Umbrella
Reporter: stack
Assignee: stack


While working on our move to unmanaged connections, I ran into issues in 
TestAccessController.testTruncatePerms; the acl permissions were not being 
restored when the table was restored.  Digging, I ended up in the truncate 
'handler'. Its a subclass of the delete table handler.  For the create table 
part of trunctate, it does not call create table handler.  Instead it 
copy/pastes what is going on over there only the copy/paste has rotted and 
needs recalibrating so acl perms get restored at least; better would be 
removing the copy/paste and using the create table handler.  Or, we could just 
drop the truncate feature as just not appropriate to a store like this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12592) Fix the truncate feature or purge it.

2014-11-27 Thread Matteo Bertozzi (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-12592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227537#comment-14227537
]

Matteo Bertozzi commented on HBASE-12592:
-

once we will have a proper notification bus, the truncate can be implemented as
notification to the regions to delete all the files.
at the moment we just have a workaround drop/recreate. The reason we introduced
a truncate was to do a clear of the table but avoiding to delete every single
record and also to be able to perform the clear operation as a table admin
(drop create requires a global permission)

Fix the truncate feature or purge it.
-

Key: HBASE-12592
URL: https://issues.apache.org/jira/browse/HBASE-12592
Project: HBase
Issue Type: Umbrella
Reporter: stack
Assignee: stack

While working on our move to unmanaged connections, I ran into issues in
TestAccessController.testTruncatePerms; the acl permissions were not being
restored when the table was restored. Digging, I ended up in the truncate
'handler'. Its a subclass of the delete table handler. For the create table
part of trunctate, it does not call create table handler. Instead it
copy/pastes what is going on over there only the copy/paste has rotted and
needs recalibrating so acl perms get restored at least; better would be
removing the copy/paste and using the create table handler. Or, we could
just drop the truncate feature as just not appropriate to a store like this.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11902) RegionServer was blocked while aborting

2014-11-27 Thread Hadoop QA (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227542#comment-14227542
]

Hadoop QA commented on HBASE-11902:
---

{color:green}+1 overall{color}. Here are the results of testing the latest
attachment

http://issues.apache.org/jira/secure/attachment/12684027/hbase11902-master_v3.patch
against master branch at commit aa0bd50fd40d8090c5a98cbde063621eadd988f8.
ATTACHMENT ID: 12684027

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//artifact/patchprocess/checkstyle-aggregate.html

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11852//console

This message is automatically generated.

RegionServer was blocked while aborting
---

Key: HBASE-11902
URL: https://issues.apache.org/jira/browse/HBASE-11902
Project: HBase
Issue Type: Bug
Components: regionserver, wal
Affects Versions: 0.98.4
Environment: hbase-0.98.4, hadoop-2.3.0-cdh5.1, jdk1.7
Reporter: Victor Xu
Assignee: Qiang Tian
Attachments: hbase-hadoop-regionserver-hadoop461.cm6.log,
hbase11902-master.patch, hbase11902-master_v2.patch,
hbase11902-master_v3.patch, jstack_hadoop461.cm6.log

Generally, regionserver automatically aborts when isHealth() returns false.
But it sometimes got blocked while aborting. I saved the jstack and logs, and
found out that it was caused by datanodes failures. The regionserver60020
thread was blocked while closing WAL.
This issue doesn't happen so frequently, but if it happens, it always leads
to huge amount of requests failure. The only way to do is KILL -9.
I think it's a bug, but I haven't found a decent solution. Does anyone have
the same problem?

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12581) TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 addendum)

[
https://issues.apache.org/jira/browse/HBASE-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227545#comment-14227545
]

stack commented on HBASE-12581:
---

[~ram_krish] Thanks for taking a look!

bq. I still need to see whether the recent Connection related changes is
slowing things in any way.

I was thinking that because the Connection instance varies, we now may 'miss'
notification of the acl update.

I've seen versions of our missing notification of acl update in a few guises
during the move to unmanaged connections. One interesting one was over in
TestAccessController.testTruncatePerms. It does truncate. Truncate is kinda
broke in that it copy/pastes what create table handler does only its missing
pieces. I was seeing that after putting back the table, the acl perms were not
also put back for this 'new' table. I changed the test back to how it was
before just to get it passing again. Let me file an issue on truncate and this
test now (HBASE-12592).

bq. Introducing a sleep after setUp() method of the test makes the test pass
consistently.

We need to add something like the waitFor in HBaseTestingUtility where we wait
for the acl stuff to show up before we proceed with the test? Would need to add
it in to these tests here in this test suite but also to any test making use of
acl table.

For acl perms, we always read from zk, never from the table?

Thanks for helping [~ram_krish] That helped a bunch.

TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404
addendum)
---

Key: HBASE-12581
URL: https://issues.apache.org/jira/browse/HBASE-12581
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Critical
Fix For: 2.0.0, 0.99.2

Attachments: 12581.addendum.txt, 12581.txt, acls.txt

TestCellACLWithMultipleVersions failed after HBASE-12404 went in (though it
passed twice on hadoopqa!). Fails locally too. Here, make a Connection when
we go to check perms. That seems to fix it. Going to commit since patch is
just more of hbase-12404... this is in essence and addendum.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12558) TestHCM.testClusterStatus Unexpected exception, expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException but wasjunit.framework.AssertionFailedErro


[ 
https://issues.apache.org/jira/browse/HBASE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227548#comment-14227548
 ] 

stack commented on HBASE-12558:
---

I'll try in a day or two. We are getting blue builds again. Lets get a run of 
them first then come back here.

 TestHCM.testClusterStatus Unexpected exception, 
 expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException 
 but wasjunit.framework.AssertionFailedError
 -

 Key: HBASE-12558
 URL: https://issues.apache.org/jira/browse/HBASE-12558
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.2

 Attachments: 12558-master.patch, 12558.ignore.txt


 Happens for me reliably on mac os x. I looked at fixing it. The listener is 
 not noticing the publish for whatever reason.  Thats where I stopped.
 {code}
 java.lang.Exception: Unexpected exception, 
 expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException 
 but wasjunit.framework.AssertionFailedError
   at junit.framework.Assert.fail(Assert.java:57)
   at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
   at 
 org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3537)
   at 
 org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:273)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12581) TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 addendum)

[
https://issues.apache.org/jira/browse/HBASE-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227556#comment-14227556
]

ramkrishna.s.vasudevan commented on HBASE-12581:

bq.For acl perms, we always read from zk, never from the table?
Yes always from the ZK. We hope the ZK is always updated. Hence we always
wanted to have a notification bus so that all the zk related updates are in
sync.
bq.We need to add something like the waitFor in HBaseTestingUtility where we
wait for the acl stuff to show up before we proceed with the test? Would need
to add it in to these tests here in this test suite but also to any test making
use of acl table.
Yes. This is needed. But to know whether the required update is done in the
ACL table is done should be done by the super user. I can do that if you are
not going to work on it.
Anyway for this test to pass we can just add a sleep in the setup or at the
beginning of the failing test.

TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404
addendum)
---

Key: HBASE-12581
URL: https://issues.apache.org/jira/browse/HBASE-12581
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Critical
Fix For: 2.0.0, 0.99.2

Attachments: 12581.addendum.txt, 12581.txt, acls.txt

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12593) Tags and Tag dictionary to work with BB

ramkrishna.s.vasudevan created HBASE-12593:
--

 Summary: Tags and Tag dictionary to work with BB
 Key: HBASE-12593
 URL: https://issues.apache.org/jira/browse/HBASE-12593
 Project: HBase
  Issue Type: Sub-task
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan


Adding the subtask so that we don't forget it. Came up while reviewing the 
items required for this parent task.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12583) Allow creating reference files even the split row not lies in the storefile range if required


 [ 
https://issues.apache.org/jira/browse/HBASE-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

rajeshbabu updated HBASE-12583:
---
Attachment: HBASE-12583.patch

 Allow creating reference files even the split row not lies in the storefile 
 range if required
 -

 Key: HBASE-12583
 URL: https://issues.apache.org/jira/browse/HBASE-12583
 Project: HBase
  Issue Type: Improvement
Reporter: rajeshbabu
Assignee: rajeshbabu
  Labels: Phoenix
 Fix For: 2.0.0, 0.98.9, 0.99.2

 Attachments: HBASE-12583.patch


 Currently in HRegionFileSystem#splitStoreFile we are not creating reference 
 files if the split row not lies in the storefile range that means one of the 
 child region doesn't have any data.
 {code}
// Check whether the split row lies in the range of the store file
 // If it is outside the range, return directly.
 if (top) {
   //check if larger than last key.
   KeyValue splitKey = KeyValueUtil.createFirstOnRow(splitRow);
   byte[] lastKey = f.createReader().getLastKey();
   // If lastKey is null means storefile is empty.
   if (lastKey == null) return null;
   if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
   splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0, 
 lastKey.length)  0) {
 return null;
   }
 } else {
   //check if smaller than first key
   KeyValue splitKey = KeyValueUtil.createLastOnRow(splitRow);
   byte[] firstKey = f.createReader().getFirstKey();
   // If firstKey is null means storefile is empty.
   if (firstKey == null) return null;
   if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
   splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0, 
 firstKey.length)  0) {
 return null;
   }
 }
 {code}
 In some cases when split row should be compared with part of rowkey(in 
 composite rowkey) mainly for secondary index tables we need to create 
 reference files even when split row not lies in the storefile range so that 
 they can be rewritten to it's child regions by some custom half store file 
 reader which compare the part of row key with split row.
 The check of comparing split row with storefile range and returning directly 
 can be avoided by having special boolean attribute in table descriptor when 
 it set to true. Or else we can have coprocessor hooks so that in the hooks we 
 can create the references and bypass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12540) TestRegionServerMetrics#testMobMetrics test failure


[ 
https://issues.apache.org/jira/browse/HBASE-12540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227564#comment-14227564
 ] 

stack commented on HBASE-12540:
---

[~jingchengdu] I could try your patch but lets wait a few days. The build went 
unstable over these last few days after some big patches went in changing how 
we do connections. We seem to be almost stable again.  Once stability is back, 
I can introduce this change. Thanks.

 TestRegionServerMetrics#testMobMetrics test failure
 ---

 Key: HBASE-12540
 URL: https://issues.apache.org/jira/browse/HBASE-12540
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: Jingcheng Du
 Attachments: HBASE-12540.diff, log.txt


 Got this on an internal rig run.  Maybe you want to take a looksee 
 [~jingchengdu]?
 {code}
 Error Message
 Metrics Counters should be equal expected:5 but was:2
 Stacktrace
 java.lang.AssertionError: Metrics Counters should be equal expected:5 but 
 was:2
   at org.junit.Assert.fail(Assert.java:88)
   at org.junit.Assert.failNotEquals(Assert.java:743)
   at org.junit.Assert.assertEquals(Assert.java:118)
   at org.junit.Assert.assertEquals(Assert.java:555)
   at 
 org.apache.hadoop.hbase.test.MetricsAssertHelperImpl.assertCounter(MetricsAssertHelperImpl.java:185)
   at 
 org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics.testMobMetrics(TestRegionServerMetrics.java:448)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
   at org.junit.runners.Suite.runChild(Suite.java:127)
   at org.junit.runners.Suite.runChild(Suite.java:26)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12583) Allow creating reference files even the split row not lies in the storefile range if required


 [ 
https://issues.apache.org/jira/browse/HBASE-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

rajeshbabu updated HBASE-12583:
---
Status: Patch Available  (was: Open)

Here is the patch skip the storefile range check in 
HRegionFileSystem#splitStoreFile

[~stack]
Actually I thought of adding special attribute for table to skip the check.
But Ram suggested to this using RegionSplitPolicy so that it will be meaningful 
and clean. So I have done the same way.

Please review.


 Allow creating reference files even the split row not lies in the storefile 
 range if required
 -

 Key: HBASE-12583
 URL: https://issues.apache.org/jira/browse/HBASE-12583
 Project: HBase
  Issue Type: Improvement
Reporter: rajeshbabu
Assignee: rajeshbabu
  Labels: Phoenix
 Fix For: 2.0.0, 0.98.9, 0.99.2

 Attachments: HBASE-12583.patch


 Currently in HRegionFileSystem#splitStoreFile we are not creating reference 
 files if the split row not lies in the storefile range that means one of the 
 child region doesn't have any data.
 {code}
// Check whether the split row lies in the range of the store file
 // If it is outside the range, return directly.
 if (top) {
   //check if larger than last key.
   KeyValue splitKey = KeyValueUtil.createFirstOnRow(splitRow);
   byte[] lastKey = f.createReader().getLastKey();
   // If lastKey is null means storefile is empty.
   if (lastKey == null) return null;
   if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
   splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0, 
 lastKey.length)  0) {
 return null;
   }
 } else {
   //check if smaller than first key
   KeyValue splitKey = KeyValueUtil.createLastOnRow(splitRow);
   byte[] firstKey = f.createReader().getFirstKey();
   // If firstKey is null means storefile is empty.
   if (firstKey == null) return null;
   if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
   splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0, 
 firstKey.length)  0) {
 return null;
   }
 }
 {code}
 In some cases when split row should be compared with part of rowkey(in 
 composite rowkey) mainly for secondary index tables we need to create 
 reference files even when split row not lies in the storefile range so that 
 they can be rewritten to it's child regions by some custom half store file 
 reader which compare the part of row key with split row.
 The check of comparing split row with storefile range and returning directly 
 can be avoided by having special boolean attribute in table descriptor when 
 it set to true. Or else we can have coprocessor hooks so that in the hooks we 
 can create the references and bypass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HBASE-12583) Allow creating reference files even the split row not lies in the storefile range if required


[ 
https://issues.apache.org/jira/browse/HBASE-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227571#comment-14227571
 ] 

rajeshbabu edited comment on HBASE-12583 at 11/27/14 11:53 AM:
---

Here is the patch skip the storefile range check in 
HRegionFileSystem#splitStoreFile

[~stack]
Actually I thought of adding special attribute for table to skip the check.
But Ram suggested to do this using RegionSplitPolicy so that it will be 
meaningful and clean. So I have done the same way.

Please review.



was (Author: rajesh23):
Here is the patch skip the storefile range check in 
HRegionFileSystem#splitStoreFile

[~stack]
Actually I thought of adding special attribute for table to skip the check.
But Ram suggested to this using RegionSplitPolicy so that it will be meaningful 
and clean. So I have done the same way.

Please review.


 Allow creating reference files even the split row not lies in the storefile 
 range if required
 -

 Key: HBASE-12583
 URL: https://issues.apache.org/jira/browse/HBASE-12583
 Project: HBase
  Issue Type: Improvement
Reporter: rajeshbabu
Assignee: rajeshbabu
  Labels: Phoenix
 Fix For: 2.0.0, 0.98.9, 0.99.2

 Attachments: HBASE-12583.patch


 Currently in HRegionFileSystem#splitStoreFile we are not creating reference 
 files if the split row not lies in the storefile range that means one of the 
 child region doesn't have any data.
 {code}
// Check whether the split row lies in the range of the store file
 // If it is outside the range, return directly.
 if (top) {
   //check if larger than last key.
   KeyValue splitKey = KeyValueUtil.createFirstOnRow(splitRow);
   byte[] lastKey = f.createReader().getLastKey();
   // If lastKey is null means storefile is empty.
   if (lastKey == null) return null;
   if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
   splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0, 
 lastKey.length)  0) {
 return null;
   }
 } else {
   //check if smaller than first key
   KeyValue splitKey = KeyValueUtil.createLastOnRow(splitRow);
   byte[] firstKey = f.createReader().getFirstKey();
   // If firstKey is null means storefile is empty.
   if (firstKey == null) return null;
   if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
   splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0, 
 firstKey.length)  0) {
 return null;
   }
 }
 {code}
 In some cases when split row should be compared with part of rowkey(in 
 composite rowkey) mainly for secondary index tables we need to create 
 reference files even when split row not lies in the storefile range so that 
 they can be rewritten to it's child regions by some custom half store file 
 reader which compare the part of row key with split row.
 The check of comparing split row with storefile range and returning directly 
 can be avoided by having special boolean attribute in table descriptor when 
 it set to true. Or else we can have coprocessor hooks so that in the hooks we 
 can create the references and bypass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12583) Allow creating reference files even the split row not lies in the storefile range if required


[ 
https://issues.apache.org/jira/browse/HBASE-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227573#comment-14227573
 ] 

ramkrishna.s.vasudevan commented on HBASE-12583:


{code}
+  public void setSplitPolicy(RegionSplitPolicy spliPolicy) {
+this.splitPolicy = spliPolicy;
+  }
{code}
Why do we need to have a setter in HRegionFileSystem. Will not be enough just 
to pass the boolean to splitStoreFile?

 Allow creating reference files even the split row not lies in the storefile 
 range if required
 -

 Key: HBASE-12583
 URL: https://issues.apache.org/jira/browse/HBASE-12583
 Project: HBase
  Issue Type: Improvement
Reporter: rajeshbabu
Assignee: rajeshbabu
  Labels: Phoenix
 Fix For: 2.0.0, 0.98.9, 0.99.2

 Attachments: HBASE-12583.patch


 Currently in HRegionFileSystem#splitStoreFile we are not creating reference 
 files if the split row not lies in the storefile range that means one of the 
 child region doesn't have any data.
 {code}
// Check whether the split row lies in the range of the store file
 // If it is outside the range, return directly.
 if (top) {
   //check if larger than last key.
   KeyValue splitKey = KeyValueUtil.createFirstOnRow(splitRow);
   byte[] lastKey = f.createReader().getLastKey();
   // If lastKey is null means storefile is empty.
   if (lastKey == null) return null;
   if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
   splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0, 
 lastKey.length)  0) {
 return null;
   }
 } else {
   //check if smaller than first key
   KeyValue splitKey = KeyValueUtil.createLastOnRow(splitRow);
   byte[] firstKey = f.createReader().getFirstKey();
   // If firstKey is null means storefile is empty.
   if (firstKey == null) return null;
   if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
   splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0, 
 firstKey.length)  0) {
 return null;
   }
 }
 {code}
 In some cases when split row should be compared with part of rowkey(in 
 composite rowkey) mainly for secondary index tables we need to create 
 reference files even when split row not lies in the storefile range so that 
 they can be rewritten to it's child regions by some custom half store file 
 reader which compare the part of row key with split row.
 The check of comparing split row with storefile range and returning directly 
 can be avoided by having special boolean attribute in table descriptor when 
 it set to true. Or else we can have coprocessor hooks so that in the hooks we 
 can create the references and bypass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12583) Allow creating reference files even the split row not lies in the storefile range if required


[ 
https://issues.apache.org/jira/browse/HBASE-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227595#comment-14227595
 ] 

rajeshbabu commented on HBASE-12583:


[~ram_krish]
To pass the boolean in splitStoreFile we need to expose split policy from 
HRegion and get it in SplitTransaction.


 Allow creating reference files even the split row not lies in the storefile 
 range if required
 -

 Key: HBASE-12583
 URL: https://issues.apache.org/jira/browse/HBASE-12583
 Project: HBase
  Issue Type: Improvement
Reporter: rajeshbabu
Assignee: rajeshbabu
  Labels: Phoenix
 Fix For: 2.0.0, 0.98.9, 0.99.2

 Attachments: HBASE-12583.patch


 Currently in HRegionFileSystem#splitStoreFile we are not creating reference 
 files if the split row not lies in the storefile range that means one of the 
 child region doesn't have any data.
 {code}
// Check whether the split row lies in the range of the store file
 // If it is outside the range, return directly.
 if (top) {
   //check if larger than last key.
   KeyValue splitKey = KeyValueUtil.createFirstOnRow(splitRow);
   byte[] lastKey = f.createReader().getLastKey();
   // If lastKey is null means storefile is empty.
   if (lastKey == null) return null;
   if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
   splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0, 
 lastKey.length)  0) {
 return null;
   }
 } else {
   //check if smaller than first key
   KeyValue splitKey = KeyValueUtil.createLastOnRow(splitRow);
   byte[] firstKey = f.createReader().getFirstKey();
   // If firstKey is null means storefile is empty.
   if (firstKey == null) return null;
   if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
   splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0, 
 firstKey.length)  0) {
 return null;
   }
 }
 {code}
 In some cases when split row should be compared with part of rowkey(in 
 composite rowkey) mainly for secondary index tables we need to create 
 reference files even when split row not lies in the storefile range so that 
 they can be rewritten to it's child regions by some custom half store file 
 reader which compare the part of row key with split row.
 The check of comparing split row with storefile range and returning directly 
 can be avoided by having special boolean attribute in table descriptor when 
 it set to true. Or else we can have coprocessor hooks so that in the hooks we 
 can create the references and bypass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12476) HydraBase Consensus Protocol

[
https://issues.apache.org/jira/browse/HBASE-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227636#comment-14227636
]

stack commented on HBASE-12476:
---

Here are some notes on the 0002 patch itself. Mostly questions. No hurry w/
response.

Purge versions from sub-poms if the dependency is common with other modules.
Inherit from parent where you can. I found this in the consensus pom:

version2.15/version

and
version4.1/version

... for what I believe are common dependencies (Its fine to have dependencies
in modules with versions if the dependency only used by the submodule)

In the consensus module, there is an hbase-default.xml also? How does it relate
to the one over in hbase-common?

You can purge hadoop-1 references (unless that is how you refer to your hdfs.)
We don't support it in hbase 1.0.

Others have already remarked on undoing the local copies of HColumnDescriptor,
etc. You had them in here so this module could 'work' without dependency? (Kill
HServerAddress!) Do we need to move stuff around in hbase so you can have
minimal dependencies? All the classes you have copied local here, should we
move them back up and into hbase-common module if not there already?

Run the rat tool on your feature branch. Some of these files are still missing
licenses.

Nice. You have already ported over ConfigurationObserver. Good.

We'll have to redo FetchTask, etc., as protobuf? (the swifty annotations look
very nice)

In FetchTask I see in the run that it has this:

// TODO in part 2: fetch log files from the peer!

Does that mean this facility of raft -- catchup I presume -- is yet to be
implemented or is it rather that this class gets subclassed later?

If the former, is there more to do to have full implementation?

Each regionserver stands up an instance of a QuorumClient per region?

A quorum name is different from region name?

We'd have to move all of the quorum communication up on to hbase rpc and pb?
What you lot think? Keep swift?

The FSM stuff could do w/ a bit of package doc on how it works. That said, it
is easy to read. (Maybe we could use these primitives elsewhere in hbase)

Region splits handled?

We'd have to figure how to do the RMap stuff in master? And bootstrapping the
cluster?

A regionserver would have to do consensuservice?

WHen you say 'favored_nodes' in the rmap, does that imply we need 'favored
nodes' working out in apache hbase or is this just how you describe quorum
members for a region?

Is the read-side, reading from the quorum, a patch as big as this one (smile)?

Nice one.

HydraBase Consensus Protocol

Key: HBASE-12476
URL: https://issues.apache.org/jira/browse/HBASE-12476
Project: HBase
Issue Type: Sub-task
Components: Consensus, wal
Reporter: Gaurav Menghani
Assignee: Gaurav Menghani
Attachments: 0001-HydraBase-consensus-protocol.patch,
0002-HydraBase-consensus-protocol.patch

This is the first patch for the HydraBase consensus protocol implemented
according to the Raft consensus protocol
(https://ramcloud.stanford.edu/raft.pdf), as advertised in (HBASE-12259)

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12581) TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 addendum)


[ 
https://issues.apache.org/jira/browse/HBASE-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227638#comment-14227638
 ] 

stack commented on HBASE-12581:
---

Let me add sleep for now to all of these cell acl tests. One just failed here: 
https://builds.apache.org/job/HBase-TRUNK/5834/testReport/junit/org.apache.hadoop.hbase.security.access/TestCellACLs/testCellPermissions/

 TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 
 addendum)
 ---

 Key: HBASE-12581
 URL: https://issues.apache.org/jira/browse/HBASE-12581
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.2

 Attachments: 12581.addendum.txt, 12581.txt, acls.txt


 TestCellACLWithMultipleVersions failed after HBASE-12404 went in (though it 
 passed twice on hadoopqa!).  Fails locally too.  Here, make a Connection when 
 we go to check perms.  That seems to fix it. Going to commit since patch is 
 just more of hbase-12404... this is in essence and addendum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12594) Remove the usage of deprecated HbaseAdmin#getTableNames from shell list command

2014-11-27 Thread Ashish Singhi (JIRA)

Ashish Singhi created HBASE-12594:
-

 Summary: Remove the usage of deprecated HbaseAdmin#getTableNames 
from shell list command
 Key: HBASE-12594
 URL: https://issues.apache.org/jira/browse/HBASE-12594
 Project: HBase
  Issue Type: Bug
  Components: shell
Affects Versions: 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor


With this fix one behavior change a non-super user will notice is that he/she 
will be able to see only tables to which they have create/admin permissions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12581) TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 addendum)


 [ 
https://issues.apache.org/jira/browse/HBASE-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-12581:
--
Attachment: 12581.addendum.sleep.txt

Something like this? Its hacky but maybe ok for tests for now.

This issue is just post table creation until the acls in zk get updated? I 
think something less hacky would be good to have for unit tests.  Do we need 
such a facility for actual runtimes?

Thanks [~ram_krish]

 TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 
 addendum)
 ---

 Key: HBASE-12581
 URL: https://issues.apache.org/jira/browse/HBASE-12581
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.2

 Attachments: 12581.addendum.sleep.txt, 12581.addendum.txt, 12581.txt, 
 acls.txt


 TestCellACLWithMultipleVersions failed after HBASE-12404 went in (though it 
 passed twice on hadoopqa!).  Fails locally too.  Here, make a Connection when 
 we go to check perms.  That seems to fix it. Going to commit since patch is 
 just more of hbase-12404... this is in essence and addendum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12594) Remove the usage of deprecated HbaseAdmin#getTableNames from shell list command

2014-11-27 Thread Ashish Singhi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HBASE-12594:
--
Attachment: HBASE-12594.patch

Patch for trunk.
Some one please review.

 Remove the usage of deprecated HbaseAdmin#getTableNames from shell list 
 command
 ---

 Key: HBASE-12594
 URL: https://issues.apache.org/jira/browse/HBASE-12594
 Project: HBase
  Issue Type: Bug
  Components: shell
Affects Versions: 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor
 Attachments: HBASE-12594.patch


 With this fix one behavior change a non-super user will notice is that he/she 
 will be able to see only tables to which they have create/admin permissions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12594) Remove the usage of deprecated HbaseAdmin#getTableNames from shell list command

2014-11-27 Thread Ashish Singhi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HBASE-12594:
--
Status: Patch Available  (was: Open)

 Remove the usage of deprecated HbaseAdmin#getTableNames from shell list 
 command
 ---

 Key: HBASE-12594
 URL: https://issues.apache.org/jira/browse/HBASE-12594
 Project: HBase
  Issue Type: Bug
  Components: shell
Affects Versions: 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor
 Attachments: HBASE-12594.patch


 With this fix one behavior change a non-super user will notice is that he/she 
 will be able to see only tables to which they have create/admin permissions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12581) TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 addendum)


[ 
https://issues.apache.org/jira/browse/HBASE-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227654#comment-14227654
 ] 

stack commented on HBASE-12581:
---

I pushed the change to branch-1+.  Lets see how it does.

 TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 
 addendum)
 ---

 Key: HBASE-12581
 URL: https://issues.apache.org/jira/browse/HBASE-12581
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.2

 Attachments: 12581.addendum.sleep.txt, 12581.addendum.txt, 12581.txt, 
 acls.txt


 TestCellACLWithMultipleVersions failed after HBASE-12404 went in (though it 
 passed twice on hadoopqa!).  Fails locally too.  Here, make a Connection when 
 we go to check perms.  That seems to fix it. Going to commit since patch is 
 just more of hbase-12404... this is in essence and addendum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12581) TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 addendum)


[ 
https://issues.apache.org/jira/browse/HBASE-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227656#comment-14227656
 ] 

stack commented on HBASE-12581:
---

Should we open new issues or subtask to add in the wait on explicit change in 
acl?  Thanks [~ram_krish] I suppose no hurry unless needed at runtime -- if 
this silly sleep does the job for tests. A lag is to be expected but can you 
think of circumstances (other than tests) where the lag is going to mess us up?

 TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 
 addendum)
 ---

 Key: HBASE-12581
 URL: https://issues.apache.org/jira/browse/HBASE-12581
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.2

 Attachments: 12581.addendum.sleep.txt, 12581.addendum.txt, 12581.txt, 
 acls.txt


 TestCellACLWithMultipleVersions failed after HBASE-12404 went in (though it 
 passed twice on hadoopqa!).  Fails locally too.  Here, make a Connection when 
 we go to check perms.  That seems to fix it. Going to commit since patch is 
 just more of hbase-12404... this is in essence and addendum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)


[ 
https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227658#comment-14227658
 ] 

Hudson commented on HBASE-12404:


FAILURE: Integrated in HBase-TRUNK #5835 (See 
[https://builds.apache.org/job/HBase-TRUNK/5835/])
HBASE-12581 TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 
(HBASE-12404 addendum) -- SLEEP ADDENDUM (stack: rev 
0f8894cd6435ed6962ec3d7c81be4cb0d4f7657e)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLWithMultipleVersions.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLs.java


 Task 5 from parent: Replace internal HTable constructor use with 
 HConnection#getTable (0.98, 0.99)
 --

 Key: HBASE-12404
 URL: https://issues.apache.org/jira/browse/HBASE-12404
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Assignee: stack
 Fix For: 2.0.0, 0.99.2

 Attachments: 
 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt, 
 12404.v16.txt, 12404.v16.txt, 12404.v17.txt, 12404v10.txt, 12404v11.txt, 
 12404v12.txt, 12404v12.txt, 12404v13.txt, 12404v13.txt, 12404v14.txt, 
 12404v15.txt, 12404v18.txt, 12404v19.txt, 12404v19.txt, 12404v2.txt, 
 12404v20.txt, 12404v20.txt, 12404v3.txt, 12404v5.txt, 12404v6.txt, 
 12404v7.txt, 12404v9.txt


 Do the step 5. from the [~ndimiduk] list in parent issue.  Go through src 
 code and change all new HTable to instead be connection.getTable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12581) TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 addendum)


[ 
https://issues.apache.org/jira/browse/HBASE-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227659#comment-14227659
 ] 

Hudson commented on HBASE-12581:


FAILURE: Integrated in HBase-TRUNK #5835 (See 
[https://builds.apache.org/job/HBase-TRUNK/5835/])
HBASE-12581 TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 
(HBASE-12404 addendum) -- SLEEP ADDENDUM (stack: rev 
0f8894cd6435ed6962ec3d7c81be4cb0d4f7657e)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLWithMultipleVersions.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLs.java


 TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 
 addendum)
 ---

 Key: HBASE-12581
 URL: https://issues.apache.org/jira/browse/HBASE-12581
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.2

 Attachments: 12581.addendum.sleep.txt, 12581.addendum.txt, 12581.txt, 
 acls.txt


 TestCellACLWithMultipleVersions failed after HBASE-12404 went in (though it 
 passed twice on hadoopqa!).  Fails locally too.  Here, make a Connection when 
 we go to check perms.  That seems to fix it. Going to commit since patch is 
 just more of hbase-12404... this is in essence and addendum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12583) Allow creating reference files even the split row not lies in the storefile range if required

2014-11-27 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227673#comment-14227673
 ] 

Hadoop QA commented on HBASE-12583:
---

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12684051/HBASE-12583.patch
  against master branch at commit aa0bd50fd40d8090c5a98cbde063621eadd988f8.
  ATTACHMENT ID: 12684051

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/11853//console

This message is automatically generated.

 Allow creating reference files even the split row not lies in the storefile 
 range if required
 -

 Key: HBASE-12583
 URL: https://issues.apache.org/jira/browse/HBASE-12583
 Project: HBase
  Issue Type: Improvement
Reporter: rajeshbabu
Assignee: rajeshbabu
  Labels: Phoenix
 Fix For: 2.0.0, 0.98.9, 0.99.2

 Attachments: HBASE-12583.patch


 Currently in HRegionFileSystem#splitStoreFile we are not creating reference 
 files if the split row not lies in the storefile range that means one of the 
 child region doesn't have any data.
 {code}
// Check whether the split row lies in the range of the store file
 // If it is outside the range, return directly.
 if (top) {
   //check if larger than last key.
   KeyValue splitKey = KeyValueUtil.createFirstOnRow(splitRow);
   byte[] lastKey = f.createReader().getLastKey();
   // If lastKey is null means storefile is empty.
   if (lastKey == null) return null;
   if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
   splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0, 
 lastKey.length)  0) {
 return null;
   }
 } else {
   //check if smaller than first key
   KeyValue

[jira] [Commented] (HBASE-12581) TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 addendum)


[ 
https://issues.apache.org/jira/browse/HBASE-12581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227711#comment-14227711
 ] 

Hudson commented on HBASE-12581:


SUCCESS: Integrated in HBase-1.0 #518 (See 
[https://builds.apache.org/job/HBase-1.0/518/])
HBASE-12581 TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 
(HBASE-12404 addendum) -- SLEEP ADDENDUM (stack: rev 
f95728ac7d596ecca7d98007b309821353bdbb90)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLWithMultipleVersions.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLs.java


 TestCellACLWithMultipleVersions failing since task 5 HBASE-12404 (HBASE-12404 
 addendum)
 ---

 Key: HBASE-12581
 URL: https://issues.apache.org/jira/browse/HBASE-12581
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0, 0.99.2

 Attachments: 12581.addendum.sleep.txt, 12581.addendum.txt, 12581.txt, 
 acls.txt


 TestCellACLWithMultipleVersions failed after HBASE-12404 went in (though it 
 passed twice on hadoopqa!).  Fails locally too.  Here, make a Connection when 
 we go to check perms.  That seems to fix it. Going to commit since patch is 
 just more of hbase-12404... this is in essence and addendum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)