date:20121219


 [ 
https://issues.apache.org/jira/browse/HBASE-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-6585:
---

Attachment: HBASE-6885-0.94.patch

Attached the 0.94 patch

 Audit log messages should contain info about the higher level operation being 
 executed
 --

 Key: HBASE-6585
 URL: https://issues.apache.org/jira/browse/HBASE-6585
 Project: HBase
  Issue Type: Improvement
  Components: security
Affects Versions: 0.96.0
Reporter: Marcelo Vanzin
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: acl
 Fix For: 0.96.0, 0.94.4

 Attachments: HBASE-6585-v0.patch, HBASE-6585-v1.patch, 
 HBASE-6585-v2.patch, HBASE-6885-0.94.patch


 Currently, audit log messages contains the action for which access was 
 checked; this is one of READ, WRITE, CREATE or ADMIN.
 These give very little information to the person digging into the logs about 
 what was done, though. You can't ask who deleted rows from table x?, 
 because delete is translated to a WRITE action.
 It would be nice if the audit logs contained the higher-level operation, 
 either replacing or in addition to the RWCA information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6585) Audit log messages should contain info about the higher level operation being executed


[ 
https://issues.apache.org/jira/browse/HBASE-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535790#comment-13535790
 ] 

Hudson commented on HBASE-6585:
---

Integrated in HBase-TRUNK #3638 (See 
[https://builds.apache.org/job/HBase-TRUNK/3638/])
HBASE-6585 Audit log messages should contain info about the higher level 
operation being executed (Revision 1423775)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/conf/log4j.properties
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java


 Audit log messages should contain info about the higher level operation being 
 executed
 --

 Key: HBASE-6585
 URL: https://issues.apache.org/jira/browse/HBASE-6585
 Project: HBase
  Issue Type: Improvement
  Components: security
Affects Versions: 0.96.0
Reporter: Marcelo Vanzin
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: acl
 Fix For: 0.96.0, 0.94.4

 Attachments: HBASE-6585-v0.patch, HBASE-6585-v1.patch, 
 HBASE-6585-v2.patch, HBASE-6885-0.94.patch


 Currently, audit log messages contains the action for which access was 
 checked; this is one of READ, WRITE, CREATE or ADMIN.
 These give very little information to the person digging into the logs about 
 what was done, though. You can't ask who deleted rows from table x?, 
 because delete is translated to a WRITE action.
 It would be nice if the audit logs contained the higher-level operation, 
 either replacing or in addition to the RWCA information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7283) Backport HBASE-6564 + HBASE-7202 to 0.94


[ 
https://issues.apache.org/jira/browse/HBASE-7283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535798#comment-13535798
 ] 

Hudson commented on HBASE-7283:
---

Integrated in HBase-0.94 #644 (See 
[https://builds.apache.org/job/HBase-0.94/644/])
HBASE-7283 Backport HBASE-6564 + HBASE-7202 to 0.94 (Revision 1423774)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/backup/HFileArchiver.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/handler
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/handler/TestTableDeleteFamilyHandler.java


 Backport HBASE-6564 + HBASE-7202 to 0.94
 

 Key: HBASE-7283
 URL: https://issues.apache.org/jira/browse/HBASE-7283
 Project: HBase
  Issue Type: Task
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
 Fix For: 0.94.4

 Attachments: 7283.094.txt, HBASE-6564-0.94.patch, 
 HBASE-7202-0.94.patch


 * HBASE-6564: HDFS space is not reclaimed when a column family is deleted
 * HBASE-7202: Family Store Files are not archived on admin.deleteColumn()
 (the second one is a fix for the first, to use the archiver)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7202) Family Store Files are not archived on admin.deleteColumn()


[ 
https://issues.apache.org/jira/browse/HBASE-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535799#comment-13535799
 ] 

Hudson commented on HBASE-7202:
---

Integrated in HBase-0.94 #644 (See 
[https://builds.apache.org/job/HBase-0.94/644/])
HBASE-7283 Backport HBASE-6564 + HBASE-7202 to 0.94 (Revision 1423774)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/backup/HFileArchiver.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/handler
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/handler/TestTableDeleteFamilyHandler.java


 Family Store Files are not archived on admin.deleteColumn()
 ---

 Key: HBASE-7202
 URL: https://issues.apache.org/jira/browse/HBASE-7202
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.2, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 0.96.0

 Attachments: HBASE-7202-v1.patch, HBASE-7202-v2.patch


 using HBaseAdmin.deleteColumn() the files are not archived but deleted 
 directory.
 This causes problems with snapshots, and other systems that relies on files 
 to be archived.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6564) HDFS space is not reclaimed when a column family is deleted


[ 
https://issues.apache.org/jira/browse/HBASE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535800#comment-13535800
 ] 

Hudson commented on HBASE-6564:
---

Integrated in HBase-0.94 #644 (See 
[https://builds.apache.org/job/HBase-0.94/644/])
HBASE-7283 Backport HBASE-6564 + HBASE-7202 to 0.94 (Revision 1423774)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/backup/HFileArchiver.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFamilyHandler.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/handler
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/handler/TestTableDeleteFamilyHandler.java


 HDFS space is not reclaimed when a column family is deleted
 ---

 Key: HBASE-6564
 URL: https://issues.apache.org/jira/browse/HBASE-6564
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.94.1
Reporter: J Mohamed Zahoor
Assignee: J Mohamed Zahoor
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-6564-trunk.patch, HBASE-6564-v2.patch, 
 HBASE-6564-v3.patch, HBASE-6564-v4.patch, HBASE-6564-v5.patch


 When a column family of a table is deleted, the HDFS space of the column 
 family does not seem to be reclaimed even after a major compaction.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7381) Lightweight data transfer for Class Result


[ 
https://issues.apache.org/jira/browse/HBASE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535801#comment-13535801
 ] 

Hudson commented on HBASE-7381:
---

Integrated in HBase-0.94 #644 (See 
[https://builds.apache.org/job/HBase-0.94/644/])
HBASE-7381 Lightweight data transfer for Class Result (Cheng Hao) (Revision 
1423771)

 Result = FAILURE
larsh : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Result.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java


 Lightweight data transfer for Class Result
 --

 Key: HBASE-7381
 URL: https://issues.apache.org/jira/browse/HBASE-7381
 Project: HBase
  Issue Type: Improvement
  Components: Client
Reporter: Cheng Hao
Priority: Trivial
 Fix For: 0.94.4

 Attachments: result_lightweight_copy_v2.patch


 Currently,the data transferring between 2 Result objects in the same process, 
 will cause additional/unnecessary data parsing  copying; as we have to do 
 that via Writables.copyWritable(result1, result2), which internally is 
 serialization, data copying, and de-serialization.
 The use case are quite common when integrated with Hadoop job running;
 The protocol org.apache.hadoop.mapred.RecordReader defined in Hadoop, 
 provides 3 interfaces:
 1) K createKey();
 2) V createValue();
 3) boolean next(K key, V value) throws IOException;
 In the 3rd method implementation, most likely requires the value (should be 
 Result object) to be filled, with the Result object from HBase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6585) Audit log messages should contain info about the higher level operation being executed

[
https://issues.apache.org/jira/browse/HBASE-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535810#comment-13535810
]

Hadoop QA commented on HBASE-6585:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561667/HBASE-6885-0.94.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3608//console

This message is automatically generated.

Audit log messages should contain info about the higher level operation being
executed
--

Key: HBASE-6585
URL: https://issues.apache.org/jira/browse/HBASE-6585
Project: HBase
Issue Type: Improvement
Components: security
Affects Versions: 0.96.0
Reporter: Marcelo Vanzin
Assignee: Matteo Bertozzi
Priority: Minor
Labels: acl
Fix For: 0.96.0, 0.94.4

Attachments: HBASE-6585-v0.patch, HBASE-6585-v1.patch,
HBASE-6585-v2.patch, HBASE-6885-0.94.patch

Currently, audit log messages contains the action for which access was
checked; this is one of READ, WRITE, CREATE or ADMIN.
These give very little information to the person digging into the logs about
what was done, though. You can't ask who deleted rows from table x?,
because delete is translated to a WRITE action.
It would be nice if the audit logs contained the higher-level operation,
either replacing or in addition to the RWCA information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6651) Improve thread safety of HTablePool


[ 
https://issues.apache.org/jira/browse/HBASE-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535822#comment-13535822
 ] 

Hadoop QA commented on HBASE-6651:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12561665/HBASE-6651-V11.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 19 new 
or modified tests.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 findbugs{color}.  The patch appears to introduce 25 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

 {color:red}-1 core zombie tests{color}.  There are zombie tests. See build 
logs for details.

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3607//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3607//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3607//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3607//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3607//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3607//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3607//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3607//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3607//console

This message is automatically generated.

 Improve thread safety of HTablePool
 ---

 Key: HBASE-6651
 URL: https://issues.apache.org/jira/browse/HBASE-6651
 Project: HBase
  Issue Type: Bug
  Components: Client
Affects Versions: 0.94.1
Reporter: Hiroshi Ikeda
Assignee: Hiroshi Ikeda
 Fix For: 0.96.0

 Attachments: HBASE-6651.patch, HBASE-6651-V10.patch, 
 HBASE-6651-V11.patch, HBASE-6651-V2.patch, HBASE-6651-V3.patch, 
 HBASE-6651-V4.patch, HBASE-6651-V5.patch, HBASE-6651-V6.patch, 
 HBASE-6651-V7.patch, HBASE-6651-V8.patch, HBASE-6651-V9.patch, sample.zip, 
 sample.zip, sharedmap_for_hbaseclient.zip


 There are some operations in HTablePool accessing PoolMap in multiple places 
 without any explicit synchronization. 
 For example HTablePool.closeTablePool() calls PoolMap.values(), and calls 
 PoolMap.remove(). If other threads add new instances to the pool in the 
 middle of the calls, the newly added instances might be dropped. 
 (HTablePool.closeTablePool() also has another problem that calling it by 
 multiple threads causes accessing HTable by multiple threads.)
 Moreover, PoolMap is not thread safe for the same reason.
 For example PoolMap.put() calles ConcurrentMap.get() and calles 
 ConcurrentMap.put(). If other threads add a new instance to the concurent map 
 in the middle of the calls, the new instance might be dropped.
 And also implementations of Pool have the same problems.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6585) Audit log messages should contain info about the higher level operation being executed


[ 
https://issues.apache.org/jira/browse/HBASE-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535904#comment-13535904
 ] 

Hudson commented on HBASE-6585:
---

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #303 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/303/])
HBASE-6585 Audit log messages should contain info about the higher level 
operation being executed (Revision 1423775)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/conf/log4j.properties
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java


 Audit log messages should contain info about the higher level operation being 
 executed
 --

 Key: HBASE-6585
 URL: https://issues.apache.org/jira/browse/HBASE-6585
 Project: HBase
  Issue Type: Improvement
  Components: security
Affects Versions: 0.96.0
Reporter: Marcelo Vanzin
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: acl
 Fix For: 0.96.0, 0.94.4

 Attachments: HBASE-6585-v0.patch, HBASE-6585-v1.patch, 
 HBASE-6585-v2.patch, HBASE-6885-0.94.patch


 Currently, audit log messages contains the action for which access was 
 checked; this is one of READ, WRITE, CREATE or ADMIN.
 These give very little information to the person digging into the logs about 
 what was done, though. You can't ask who deleted rows from table x?, 
 because delete is translated to a WRITE action.
 It would be nice if the audit logs contained the higher-level operation, 
 either replacing or in addition to the RWCA information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7372) Check in the generated website so can point apache infrastructure at what to publish as our hbase.apache.org


[ 
https://issues.apache.org/jira/browse/HBASE-7372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535903#comment-13535903
 ] 

Hudson commented on HBASE-7372:
---

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #303 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/303/])
HBASE-7372 Check in the generated website so can point apache 
infrastructure at what to publish as our hbase.apache.org (Revision 1423766)
HBASE-7372 Check in the generated website so can point apache infrastructure at 
what to publish as our hbase.apache.org: ADDENDUM removing jxr x-ref content 
and reference in site.xml (Revision 1423724)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/hbase.apache.org
* /hbase/trunk/pom.xml
* /hbase/trunk/src/site/site.xml

stack : 
Files : 
* /hbase/trunk/hbase.apache.org/acid-semantics.html
* /hbase/trunk/hbase.apache.org/book.html
* /hbase/trunk/hbase.apache.org/book/book.html
* /hbase/trunk/hbase.apache.org/bulk-loads.html
* /hbase/trunk/hbase.apache.org/cygwin.html
* /hbase/trunk/hbase.apache.org/index.html
* /hbase/trunk/hbase.apache.org/integration.html
* /hbase/trunk/hbase.apache.org/issue-tracking.html
* /hbase/trunk/hbase.apache.org/license.html
* /hbase/trunk/hbase.apache.org/mail-lists.html
* /hbase/trunk/hbase.apache.org/metrics.html
* /hbase/trunk/hbase.apache.org/old_news.html
* /hbase/trunk/hbase.apache.org/project-info.html
* /hbase/trunk/hbase.apache.org/pseudo-distributed.html
* /hbase/trunk/hbase.apache.org/replication.html
* /hbase/trunk/hbase.apache.org/resources.html
* /hbase/trunk/hbase.apache.org/source-repository.html
* /hbase/trunk/hbase.apache.org/sponsors.html
* /hbase/trunk/hbase.apache.org/team-list.html
* /hbase/trunk/hbase.apache.org/xref
* /hbase/trunk/hbase.apache.org/xref-test
* /hbase/trunk/pom.xml
* /hbase/trunk/src/site/site.xml


 Check in the generated website so can point apache infrastructure at what to 
 publish as our hbase.apache.org
 

 Key: HBASE-7372
 URL: https://issues.apache.org/jira/browse/HBASE-7372
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Attachments: 7372v2.txt


 January 1st is deadline for changing how we publish our website.  We may no 
 longer rsync out to people.apache.org.  Apache infrastructure supplies two 
 options here: http://www.apache.org/dev/project-site.html  We could redo our 
 site in apache cms format.  Or we could just use svnpubsub and keep on w/ how 
 the site is currently generated and on checkin, have it autopublished.  I'll 
 go the latter route unless I hear otherwise.
 For svnpubsub, we need to point apache infrastructure at a directory that has 
 our checkedin site in it.  I was thinking ${hbasedir}/hbase.apache.org
 Let me raise this on the dev list too.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5339) Add support for compound keys to importtsv

2012-12-19 Thread Gustavo Anatoly (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535945#comment-13535945
 ] 

Gustavo Anatoly commented on HBASE-5339:


Hi, Lars.

I'm interested to contribute, with this improvement so I would like to know, 
what do you think 
about to create an inner class CompoundTsvParser inside ImportTsv, and inside
CompoundTsvParser create specific parser line class? 

Thanks.

 Add support for compound keys to importtsv
 --

 Key: HBASE-5339
 URL: https://issues.apache.org/jira/browse/HBASE-5339
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Lars George
Priority: Trivial

 Add support that you can combine some columns from the TSV with either a 
 given separator, no separator, or with a custom row key generator class. 
 Syntax could be:
 {code}
 -Dimporttsv.columns=HBASE_ROW_KEY_1,HBASE_ROW_KEY_2,cf1:col1,cf2:col3,HBASE_ROW_KEY_3
 -Dimporttsv.rowkey.separator=-
 {code}
 Another option of course is using the custom mapper class and handle this 
 there, but this also seems like a nice to have option, probably often 
 covering the 80% this sort of thing is needed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7294) Check for snapshot file cleaners on start


 [ 
https://issues.apache.org/jira/browse/HBASE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7294:
---

Attachment: HBASE-7294-v2.patch

fixed log.info() and class.getName()

 Check for snapshot file cleaners on start
 -

 Key: HBASE-7294
 URL: https://issues.apache.org/jira/browse/HBASE-7294
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, regionserver, snapshots, Zookeeper
Affects Versions: hbase-6055
Reporter: Jesse Yates
Assignee: Matteo Bertozzi
 Fix For: hbase-6055, 0.96.0

 Attachments: HBASE-7294-v1.patch, HBASE-7294-v2.patch


 Snapshots currently use the SnaphotHfileCleaner and SnapshotHLogCleaner to 
 ensure that any hfiles or hlogs (respectively) that are currently part of a 
 snapshot are not removed from their respective archive directories (.archive 
 and .oldlogs).
 From Matteo Bertozzi:
 {quote}
 currently the snapshot cleaner is not in hbase-default.xml
 and there's no warning/exception on snapshot/restore operation, if not 
 enabled.
 even if we add the cleaner to the hbase-default.xml how do we ensure that the 
 user doesn't remove it?
 Do we want to hardcode the cleaner at master startup?
 Do we want to add a check in snapshot/restore that throws an exception if the 
 cleaner is not enabled?
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7389) HBASE-7365 Snapshot clone/restore followup

Matteo Bertozzi created HBASE-7389:
--

 Summary: HBASE-7365 Snapshot clone/restore followup
 Key: HBASE-7389
 URL: https://issues.apache.org/jira/browse/HBASE-7389
 Project: HBase
  Issue Type: Sub-task
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi


Fix Snapshot clone/restore after HBASE-7365 merge

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7389) HBASE-7365 Snapshot clone/restore followup


[ 
https://issues.apache.org/jira/browse/HBASE-7389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13535998#comment-13535998
 ] 

Matteo Bertozzi commented on HBASE-7389:


review here https://reviews.apache.org/r/8674/

 HBASE-7365 Snapshot clone/restore followup
 --

 Key: HBASE-7389
 URL: https://issues.apache.org/jira/browse/HBASE-7389
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055, 0.96.0


 Fix Snapshot clone/restore after HBASE-7365 merge

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7311) Add snapshot information to hbase master webui


 [ 
https://issues.apache.org/jira/browse/HBASE-7311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7311:
---

Attachment: screenshot-snapshot-jsp.png
screenshot-master-status.png

 Add snapshot information to hbase master webui
 --

 Key: HBASE-7311
 URL: https://issues.apache.org/jira/browse/HBASE-7311
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jonathan Hsieh
Assignee: Matteo Bertozzi
 Fix For: hbase-6055, 0.96.0

 Attachments: HBASE-7311-v0.patch, screenshot-master-status.png, 
 screenshot-snapshot-jsp.png


 Similarly to how tables are listed in the web interface, snapshots should be 
 listed as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5778) Turn on WAL compression by default


 [ 
https://issues.apache.org/jira/browse/HBASE-5778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-5778:
--

Status: Patch Available  (was: Open)

 Turn on WAL compression by default
 --

 Key: HBASE-5778
 URL: https://issues.apache.org/jira/browse/HBASE-5778
 Project: HBase
  Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Priority: Blocker
 Fix For: 0.96.0

 Attachments: 5778.addendum, 5778-addendum.txt, HBASE-5778-0.94.patch, 
 HBASE-5778-0.94-v2.patch, HBASE-5778-0.94-v3.patch, HBASE-5778-0.94-v4.patch, 
 HBASE-5778-0.94-v5.patch, HBASE-5778-0.94-v6.patch, HBASE-5778-0.94-v7.patch, 
 HBASE-5778.patch, HBASE-5778-trunk-v6.patch, HBASE-5778-trunk-v7.patch


 I ran some tests to verify if WAL compression should be turned on by default.
 For a use case where it's not very useful (values two order of magnitude 
 bigger than the keys), the insert time wasn't different and the CPU usage 15% 
 higher (150% CPU usage VS 130% when not compressing the WAL).
 When values are smaller than the keys, I saw a 38% improvement for the insert 
 run time and CPU usage was 33% higher (600% CPU usage VS 450%). I'm not sure 
 WAL compression accounts for all the additional CPU usage, it might just be 
 that we're able to insert faster and we spend more time in the MemStore per 
 second (because our MemStores are bad when they contain tens of thousands of 
 values).
 Those are two extremes, but it shows that for the price of some CPU we can 
 save a lot. My machines have 2 quads with HT, so I still had a lot of idle 
 CPUs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7311) Add snapshot information to hbase master webui

[
https://issues.apache.org/jira/browse/HBASE-7311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536028#comment-13536028
]

Hadoop QA commented on HBASE-7311:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment

http://issues.apache.org/jira/secure/attachment/12561703/screenshot-snapshot-jsp.png
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3609//console

This message is automatically generated.

Add snapshot information to hbase master webui
--

Key: HBASE-7311
URL: https://issues.apache.org/jira/browse/HBASE-7311
Project: HBase
Issue Type: Sub-task
Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jonathan Hsieh
Assignee: Matteo Bertozzi
Fix For: hbase-6055, 0.96.0

Attachments: HBASE-7311-v0.patch, screenshot-master-status.png,
screenshot-snapshot-jsp.png

Similarly to how tables are listed in the web interface, snapshots should be
listed as well.

[jira] [Commented] (HBASE-7294) Check for snapshot file cleaners on start

[
https://issues.apache.org/jira/browse/HBASE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536026#comment-13536026
]

Hadoop QA commented on HBASE-7294:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561699/HBASE-7294-v2.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3610//console

This message is automatically generated.

Check for snapshot file cleaners on start
-

Key: HBASE-7294
URL: https://issues.apache.org/jira/browse/HBASE-7294
Project: HBase
Issue Type: Sub-task
Components: Client, master, regionserver, snapshots, Zookeeper
Affects Versions: hbase-6055
Reporter: Jesse Yates
Assignee: Matteo Bertozzi
Fix For: hbase-6055, 0.96.0

Attachments: HBASE-7294-v1.patch, HBASE-7294-v2.patch

Snapshots currently use the SnaphotHfileCleaner and SnapshotHLogCleaner to
ensure that any hfiles or hlogs (respectively) that are currently part of a
snapshot are not removed from their respective archive directories (.archive
and .oldlogs).
From Matteo Bertozzi:
{quote}
currently the snapshot cleaner is not in hbase-default.xml
and there's no warning/exception on snapshot/restore operation, if not
enabled.
even if we add the cleaner to the hbase-default.xml how do we ensure that the
user doesn't remove it?
Do we want to hardcode the cleaner at master startup?
Do we want to add a check in snapshot/restore that throws an exception if the
cleaner is not enabled?
{quote}

[jira] [Commented] (HBASE-5778) Turn on WAL compression by default

[
https://issues.apache.org/jira/browse/HBASE-5778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536029#comment-13536029
]

Hadoop QA commented on HBASE-5778:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment

http://issues.apache.org/jira/secure/attachment/12561641/HBASE-5778-0.94-v7.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 16 new
or modified tests.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3611//console

This message is automatically generated.

Turn on WAL compression by default
--

Key: HBASE-5778
URL: https://issues.apache.org/jira/browse/HBASE-5778
Project: HBase
Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Priority: Blocker
Fix For: 0.96.0

Attachments: 5778.addendum, 5778-addendum.txt, HBASE-5778-0.94.patch,
HBASE-5778-0.94-v2.patch, HBASE-5778-0.94-v3.patch, HBASE-5778-0.94-v4.patch,
HBASE-5778-0.94-v5.patch, HBASE-5778-0.94-v6.patch, HBASE-5778-0.94-v7.patch,
HBASE-5778.patch, HBASE-5778-trunk-v6.patch, HBASE-5778-trunk-v7.patch

I ran some tests to verify if WAL compression should be turned on by default.
For a use case where it's not very useful (values two order of magnitude
bigger than the keys), the insert time wasn't different and the CPU usage 15%
higher (150% CPU usage VS 130% when not compressing the WAL).
When values are smaller than the keys, I saw a 38% improvement for the insert
run time and CPU usage was 33% higher (600% CPU usage VS 450%). I'm not sure
WAL compression accounts for all the additional CPU usage, it might just be
that we're able to insert faster and we spend more time in the MemStore per
second (because our MemStores are bad when they contain tens of thousands of
values).
Those are two extremes, but it shows that for the price of some CPU we can
save a lot. My machines have 2 quads with HT, so I still had a lot of idle
CPUs.

[jira] [Commented] (HBASE-7294) Check for snapshot file cleaners on start


[ 
https://issues.apache.org/jira/browse/HBASE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536037#comment-13536037
 ] 

Jonathan Hsieh commented on HBASE-7294:
---

Please fix the To been... comment.   It is still grammatically incorrect.

 Check for snapshot file cleaners on start
 -

 Key: HBASE-7294
 URL: https://issues.apache.org/jira/browse/HBASE-7294
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, regionserver, snapshots, Zookeeper
Affects Versions: hbase-6055
Reporter: Jesse Yates
Assignee: Matteo Bertozzi
 Fix For: hbase-6055, 0.96.0

 Attachments: HBASE-7294-v1.patch, HBASE-7294-v2.patch


 Snapshots currently use the SnaphotHfileCleaner and SnapshotHLogCleaner to 
 ensure that any hfiles or hlogs (respectively) that are currently part of a 
 snapshot are not removed from their respective archive directories (.archive 
 and .oldlogs).
 From Matteo Bertozzi:
 {quote}
 currently the snapshot cleaner is not in hbase-default.xml
 and there's no warning/exception on snapshot/restore operation, if not 
 enabled.
 even if we add the cleaner to the hbase-default.xml how do we ensure that the 
 user doesn't remove it?
 Do we want to hardcode the cleaner at master startup?
 Do we want to add a check in snapshot/restore that throws an exception if the 
 cleaner is not enabled?
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-7311) Add snapshot information to hbase master webui


 [ 
https://issues.apache.org/jira/browse/HBASE-7311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh reassigned HBASE-7311:
-

Assignee: Jonathan Hsieh  (was: Matteo Bertozzi)

 Add snapshot information to hbase master webui
 --

 Key: HBASE-7311
 URL: https://issues.apache.org/jira/browse/HBASE-7311
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: hbase-6055, 0.96.0

 Attachments: HBASE-7311-v0.patch, screenshot-master-status.png, 
 screenshot-snapshot-jsp.png


 Similarly to how tables are listed in the web interface, snapshots should be 
 listed as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7311) Add snapshot information to hbase master webui


[ 
https://issues.apache.org/jira/browse/HBASE-7311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536040#comment-13536040
 ] 

Jonathan Hsieh commented on HBASE-7311:
---

nice screen shots.  Before committing, let's get someone who knows the webby 
stuff for to review the web actions. [~eclark] maybe take a look?

 Add snapshot information to hbase master webui
 --

 Key: HBASE-7311
 URL: https://issues.apache.org/jira/browse/HBASE-7311
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: hbase-6055, 0.96.0

 Attachments: HBASE-7311-v0.patch, screenshot-master-status.png, 
 screenshot-snapshot-jsp.png


 Similarly to how tables are listed in the web interface, snapshots should be 
 listed as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7294) Check for snapshot file cleaners on start


 [ 
https://issues.apache.org/jira/browse/HBASE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-7294:
---

Attachment: HBASE-7294-v3.patch

 Check for snapshot file cleaners on start
 -

 Key: HBASE-7294
 URL: https://issues.apache.org/jira/browse/HBASE-7294
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, regionserver, snapshots, Zookeeper
Affects Versions: hbase-6055
Reporter: Jesse Yates
Assignee: Matteo Bertozzi
 Fix For: hbase-6055, 0.96.0

 Attachments: HBASE-7294-v1.patch, HBASE-7294-v2.patch, 
 HBASE-7294-v3.patch


 Snapshots currently use the SnaphotHfileCleaner and SnapshotHLogCleaner to 
 ensure that any hfiles or hlogs (respectively) that are currently part of a 
 snapshot are not removed from their respective archive directories (.archive 
 and .oldlogs).
 From Matteo Bertozzi:
 {quote}
 currently the snapshot cleaner is not in hbase-default.xml
 and there's no warning/exception on snapshot/restore operation, if not 
 enabled.
 even if we add the cleaner to the hbase-default.xml how do we ensure that the 
 user doesn't remove it?
 Do we want to hardcode the cleaner at master startup?
 Do we want to add a check in snapshot/restore that throws an exception if the 
 cleaner is not enabled?
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5778) Turn on WAL compression by default

2012-12-19 Thread Jean-Daniel Cryans (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536047#comment-13536047
]

Jean-Daniel Cryans commented on HBASE-5778:
---

[~ram_krish] I wasn't aware of it so I guess it's still an issue. Will open a
Jira.

[~stack] Ok thanks, also I'll change the title.

Turn on WAL compression by default
--

[jira] [Commented] (HBASE-7294) Check for snapshot file cleaners on start

[
https://issues.apache.org/jira/browse/HBASE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536051#comment-13536051
]

Hadoop QA commented on HBASE-7294:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561711/HBASE-7294-v3.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3612//console

This message is automatically generated.

Check for snapshot file cleaners on start
-

Attachments: HBASE-7294-v1.patch, HBASE-7294-v2.patch,
HBASE-7294-v3.patch

[jira] [Updated] (HBASE-7212) Globally Barriered Procedure mechanism


 [ 
https://issues.apache.org/jira/browse/HBASE-7212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-7212:
--

Attachment: pre-hbase-7212.v5.patch
hbase-7212.v5.patch

pre patch and patch applied to the 12/18 snapshots branch.

 Globally Barriered Procedure mechanism
 --

 Key: HBASE-7212
 URL: https://issues.apache.org/jira/browse/HBASE-7212
 Project: HBase
  Issue Type: Sub-task
  Components: snapshots
Affects Versions: hbase-6055
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: hbase-6055

 Attachments: 121127-global-barrier-proc.pdf, hbase-7212.patch, 
 hbase-7212.v5.patch, pre-hbase-7212.patch, pre-hbase-7212.v5.patch


 This is a simplified version of what was proposed in HBASE-6573.  Instead of 
 claiming to be a 2pc or 3pc implementation (which implies logging at each 
 actor, and recovery operations) this is just provides a best effort global 
 barrier mechanism called a Procedure.  
 Users need only to implement a methods to acquireBarrier, to act when 
 insideBarrier, and to releaseBarrier that use the ExternalException 
 cooperative error checking mechanism.
 Globally consistent snapshots require the ability to quiesce writes to a set 
 of region servers before a the snapshot operation is executed.  Also if any 
 node fails, it needs to be able to notify them so that they abort.
 The first cut of other online snapshots don't need the fully barrier but may 
 still use this for its error propagation mechanisms.
 This version removes the extra layer incurred in the previous implementation 
 due to the use of generics, separates the coordinator and members, and 
 reduces the amount of inheritance used in favor of composition.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7212) Globally Barriered Procedure mechanism

[
https://issues.apache.org/jira/browse/HBASE-7212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536060#comment-13536060
]

Hadoop QA commented on HBASE-7212:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment

http://issues.apache.org/jira/secure/attachment/12561715/pre-hbase-7212.v5.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 75 new
or modified tests.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3613//console

This message is automatically generated.

Globally Barriered Procedure mechanism
--

Key: HBASE-7212
URL: https://issues.apache.org/jira/browse/HBASE-7212
Project: HBase
Issue Type: Sub-task
Components: snapshots
Affects Versions: hbase-6055
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: hbase-6055

Attachments: 121127-global-barrier-proc.pdf, hbase-7212.patch,
hbase-7212.v5.patch, pre-hbase-7212.patch, pre-hbase-7212.v5.patch

This is a simplified version of what was proposed in HBASE-6573. Instead of
claiming to be a 2pc or 3pc implementation (which implies logging at each
actor, and recovery operations) this is just provides a best effort global
barrier mechanism called a Procedure.
Users need only to implement a methods to acquireBarrier, to act when
insideBarrier, and to releaseBarrier that use the ExternalException
cooperative error checking mechanism.
Globally consistent snapshots require the ability to quiesce writes to a set
of region servers before a the snapshot operation is executed. Also if any
node fails, it needs to be able to notify them so that they abort.
The first cut of other online snapshots don't need the fully barrier but may
still use this for its error propagation mechanisms.
This version removes the extra layer incurred in the previous implementation
due to the use of generics, separates the coordinator and members, and
reduces the amount of inheritance used in favor of composition.

[jira] [Assigned] (HBASE-7311) Add snapshot information to hbase master webui


 [ 
https://issues.apache.org/jira/browse/HBASE-7311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh reassigned HBASE-7311:
-

Assignee: Matteo Bertozzi  (was: Jonathan Hsieh)

 Add snapshot information to hbase master webui
 --

 Key: HBASE-7311
 URL: https://issues.apache.org/jira/browse/HBASE-7311
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jonathan Hsieh
Assignee: Matteo Bertozzi
 Fix For: hbase-6055, 0.96.0

 Attachments: HBASE-7311-v0.patch, screenshot-master-status.png, 
 screenshot-snapshot-jsp.png


 Similarly to how tables are listed in the web interface, snapshots should be 
 listed as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7390) Add extra cases for assignement on the region server

nkeywal created HBASE-7390:
--

 Summary: Add extra cases for assignement on the region server
 Key: HBASE-7390
 URL: https://issues.apache.org/jira/browse/HBASE-7390
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Minor


We don't have a lot of tests on the region server itself.
Here are some.
Some of them are failing, feedback welcome.
See as well the attached state diagram for the ZK nodes on assignment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7390) Add extra test cases for assignement on the region server


 [ 
https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-7390:
---

Summary: Add extra test cases for assignement on the region server  (was: 
Add extra cases for assignement on the region server)

 Add extra test cases for assignement on the region server
 -

 Key: HBASE-7390
 URL: https://issues.apache.org/jira/browse/HBASE-7390
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Minor

 We don't have a lot of tests on the region server itself.
 Here are some.
 Some of them are failing, feedback welcome.
 See as well the attached state diagram for the ZK nodes on assignment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7306) [shell] Count shell command to return ruby bindable value.


[ 
https://issues.apache.org/jira/browse/HBASE-7306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536068#comment-13536068
 ] 

stack commented on HBASE-7306:
--

+1 on commit to 0.94.  Innocuous.

 [shell] Count shell command to return ruby bindable value.
 --

 Key: HBASE-7306
 URL: https://issues.apache.org/jira/browse/HBASE-7306
 Project: HBase
  Issue Type: Sub-task
  Components: Client, shell
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: 0.96.0

 Attachments: hbase-7306.patch


 Currently if you have a table foo with 5 rows in it, you can run this command:
 {code}
 hbase(main):001:0 x = count 'foo'
 3100 row(s) in 0.7030 seconds
 hbase(main):001:0 x
 nil
 {code}
 Ideally, the 'x' variable should be 5. so something like this coudl be done:
 {code}
 Kernel.exit 127 if x != 5
 {code}
 This is useful for having scripts that can check a condition and act.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7390) Add extra test cases for assignement on the region server


 [ 
https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-7390:
---

Attachment: 7390.v1.patch

 Add extra test cases for assignement on the region server
 -

 Key: HBASE-7390
 URL: https://issues.apache.org/jira/browse/HBASE-7390
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Minor
 Attachments: 7390.v1.patch, assignment_zk_states.jpg


 We don't have a lot of tests on the region server itself.
 Here are some.
 Some of them are failing, feedback welcome.
 See as well the attached state diagram for the ZK nodes on assignment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7390) Add extra test cases for assignement on the region server


 [ 
https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-7390:
---

Attachment: assignment_zk_states.jpg

 Add extra test cases for assignement on the region server
 -

 Key: HBASE-7390
 URL: https://issues.apache.org/jira/browse/HBASE-7390
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Minor
 Attachments: 7390.v1.patch, assignment_zk_states.jpg


 We don't have a lot of tests on the region server itself.
 Here are some.
 Some of them are failing, feedback welcome.
 See as well the attached state diagram for the ZK nodes on assignment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5416) Improve performance of scans with some kind of filters.

2012-12-19 Thread Max Lapan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536069#comment-13536069
]

Max Lapan commented on HBASE-5416:
--

is this part of acceptable inconsistency that was discussed above? I
understand this can be a valid scenario (speedup is huge at the cost of very
few (or none if pre-split) easily recoverable errors), but just wonder if you
are aware of that...

No. It is very strange, because spliting process is handled on lower level of
Store class and should be transparent to HRegion level (at least in 0.90.6
codebase, maybe something changed dramatically). In our production split is
quite common operation, run without issues. We had one problem (HBASE-6499),
which was caused by no one calling seek/reseek frequently, it may be similiar
issue.

Improve performance of scans with some kind of filters.
---

Key: HBASE-5416
URL: https://issues.apache.org/jira/browse/HBASE-5416
Project: HBase
Issue Type: Improvement
Components: Filters, Performance, regionserver
Affects Versions: 0.90.4
Reporter: Max Lapan
Assignee: Max Lapan
Fix For: 0.96.0

Attachments: 5416-Filtered_scans_v6.patch, 5416-v5.txt, 5416-v6.txt,
Filtered_scans.patch, Filtered_scans_v2.patch, Filtered_scans_v3.patch,
Filtered_scans_v4.patch, Filtered_scans_v5.1.patch, Filtered_scans_v5.patch,
Filtered_scans_v7.patch, HBASE-5416-v10.patch, HBASE-5416-v7-rebased.patch,
HBASE-5416-v8.patch, HBASE-5416-v9.patch

When the scan is performed, whole row is loaded into result list, after that
filter (if exists) is applied to detect that row is needed.
But when scan is performed on several CFs and filter checks only data from
the subset of these CFs, data from CFs, not checked by a filter is not needed
on a filter stage. Only when we decided to include current row. And in such
case we can significantly reduce amount of IO performed by a scan, by loading
only values, actually checked by a filter.
For example, we have two CFs: flags and snap. Flags is quite small (bunch of
megabytes) and is used to filter large entries from snap. Snap is very large
(10s of GB) and it is quite costly to scan it. If we needed only rows with
some flag specified, we use SingleColumnValueFilter to limit result to only
small subset of region. But current implementation is loading both CFs to
perform scan, when only small subset is needed.
Attached patch adds one routine to Filter interface to allow filter to
specify which CF is needed to it's operation. In HRegion, we separate all
scanners into two groups: needed for filter and the rest (joined). When new
row is considered, only needed data is loaded, filter applied, and only if
filter accepts the row, rest of data is loaded. At our data, this speeds up
such kind of scans 30-50 times. Also, this gives us the way to better
normalize the data into separate columns by optimizing the scans performed.

[jira] [Created] (HBASE-7391) Review/improve HLog compression's memory consumption

2012-12-19 Thread Jean-Daniel Cryans (JIRA)

Jean-Daniel Cryans created HBASE-7391:
-

Summary: Review/improve HLog compression's memory consumption
Key: HBASE-7391
URL: https://issues.apache.org/jira/browse/HBASE-7391
Project: HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Fix For: 0.96.0

From Ram in
http://mail-archives.apache.org/mod_mbox/hbase-dev/201205.mbox/%3C00bc01cd31e6$7caf1320$760d3960$%25vasude...@huawei.com%3E:

{quote}
One small observation after giving +1 on the RC.
The WAL compression feature causes OOME and causes Full GC.

The problem is, if we have 1500 regions and I need to create recovered.edits
for each of the region (I don’t have much data in the regions (~300MB)).
Now when I try to build the dictionary there is a Node object getting
created.
Each node object occupies 32 bytes.
We have 5 such dictionaries.

Initially we create indexToNodes array and its size is 32767.

So now we have 32*5*32767 = ~5MB.

Now I have 1500 regions.

So 5MB*1500 = ~7GB.(Excluding actual data). This seems to a very high
initial memory foot print and this never allows me to split the logs and I
am not able to make the cluster up at all.

Our configured heap size was 8GB, tested in 3 node cluster with 5000
regions, very less data( 1GB in hdfs cluster including replication), some
small data is spread evenly across all regions.

The formula is 32(Node object size)*5(No of dictionary)*32767(no of node
objects)*noofregions.
{quote}

[jira] [Updated] (HBASE-6585) Audit log messages should contain info about the higher level operation being executed


 [ 
https://issues.apache.org/jira/browse/HBASE-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-6585:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed to 0.94 as well. Thanks for the patch Matteo.

 Audit log messages should contain info about the higher level operation being 
 executed
 --

 Key: HBASE-6585
 URL: https://issues.apache.org/jira/browse/HBASE-6585
 Project: HBase
  Issue Type: Improvement
  Components: security
Affects Versions: 0.96.0
Reporter: Marcelo Vanzin
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: acl
 Fix For: 0.96.0, 0.94.4

 Attachments: HBASE-6585-v0.patch, HBASE-6585-v1.patch, 
 HBASE-6585-v2.patch, HBASE-6885-0.94.patch


 Currently, audit log messages contains the action for which access was 
 checked; this is one of READ, WRITE, CREATE or ADMIN.
 These give very little information to the person digging into the logs about 
 what was done, though. You can't ask who deleted rows from table x?, 
 because delete is translated to a WRITE action.
 It would be nice if the audit logs contained the higher-level operation, 
 either replacing or in addition to the RWCA information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7318) Add verbose logging option to HConnectionManager


[ 
https://issues.apache.org/jira/browse/HBASE-7318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536130#comment-13536130
 ] 

stack commented on HBASE-7318:
--

Patch looks good.  You tried it Sergey?  We don't see all the extra detail 
unless TRACE is enabled?  Thanks boss.

 Add verbose logging option to HConnectionManager
 

 Key: HBASE-7318
 URL: https://issues.apache.org/jira/browse/HBASE-7318
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.96.0
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-7318-v0.patch, HBASE-7318-v1.patch


 In the course of HBASE-7250 I found that client-side errors (as well as 
 server-side errors, but that's another question) are hard to debug.
 I have some local commits with useful, not-that-hacky HConnectionManager 
 logging added.
 Need to productionize it to be off by default but easy-to-enable for 
 debugging.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6788) Convert AuthenticationProtocol to protocol buffer service


[ 
https://issues.apache.org/jira/browse/HBASE-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536139#comment-13536139
 ] 

stack commented on HBASE-6788:
--

+1 on commit.

 Convert AuthenticationProtocol to protocol buffer service
 -

 Key: HBASE-6788
 URL: https://issues.apache.org/jira/browse/HBASE-6788
 Project: HBase
  Issue Type: Sub-task
  Components: Coprocessors
Reporter: Gary Helmling
Assignee: Gary Helmling
Priority: Blocker
 Fix For: 0.96.0

 Attachments: HBASE-6788_2.patch, HBASE-6788_3.patch, 
 HBASE-6788_4.patch, HBASE-6788_5.patch, HBASE-6788.patch


 With coprocessor endpoints now exposed as protobuf defined services, we 
 should convert over all of our built-in endpoints to PB services.
 AccessControllerProtocol was converted as part of HBASE-5448, but the 
 authentication token provider still needs to be changed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server


[ 
https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536158#comment-13536158
 ] 

stack commented on HBASE-7390:
--

Why does master not see SPLITTING?  Because it does not have a watcher?  It 
cannot create a znode when there is a SPLITTING znode in place though.  That is 
what we want, right?  Ditto for the OPENING?  Does the master not have a 
watcher on the OPENING znode?

Missing license.

Should test be called TestRegionServerNoMaster?

Odd testCloseByMasterWithoutZNode fails. I can think no reason why it should 
fail.  Ditto testMultipleOpen

Nice tests [~nkeywal]



 Add extra test cases for assignement on the region server
 -

 Key: HBASE-7390
 URL: https://issues.apache.org/jira/browse/HBASE-7390
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Minor
 Attachments: 7390.v1.patch, assignment_zk_states.jpg


 We don't have a lot of tests on the region server itself.
 Here are some.
 Some of them are failing, feedback welcome.
 See as well the attached state diagram for the ZK nodes on assignment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6499) StoreScanner's QueryMatcher not reset on store update


 [ 
https://issues.apache.org/jira/browse/HBASE-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-6499:
--

Attachment: 6499-trunk-v2.txt

Patch v2 is rebased on trunk.

 StoreScanner's QueryMatcher not reset on store update
 -

 Key: HBASE-6499
 URL: https://issues.apache.org/jira/browse/HBASE-6499
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.96.0
Reporter: Max Lapan
Assignee: Max Lapan
 Attachments: 6499-trunk-v2.txt, StoreScanner_not_reset_matcher.patch


 When underlying store changed (due compact, bulk load, etc), we destroy 
 current KeyValueHeap and recreate it using checkReseek call. Besides heap 
 recreation, it resets underlying QueryMatcher instance.
 The problem is that checkReseek not called by seek() and reseek(), only by 
 next(). If someone calls seek() just after store changed, it gets wrong 
 scanner results. Call to reseek may end up with NPE.
 AFAIK, current codebase don't call seek and reseek, but it is quite possible 
 in future. Personally, I spent lots of time to find source of wrong scanner 
 results in HBASE-5416.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6499) StoreScanner's QueryMatcher not reset on store update


 [ 
https://issues.apache.org/jira/browse/HBASE-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-6499:
--

Fix Version/s: 0.96.0

 StoreScanner's QueryMatcher not reset on store update
 -

 Key: HBASE-6499
 URL: https://issues.apache.org/jira/browse/HBASE-6499
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.96.0
Reporter: Max Lapan
Assignee: Max Lapan
 Fix For: 0.96.0

 Attachments: 6499-trunk-v2.txt, StoreScanner_not_reset_matcher.patch


 When underlying store changed (due compact, bulk load, etc), we destroy 
 current KeyValueHeap and recreate it using checkReseek call. Besides heap 
 recreation, it resets underlying QueryMatcher instance.
 The problem is that checkReseek not called by seek() and reseek(), only by 
 next(). If someone calls seek() just after store changed, it gets wrong 
 scanner results. Call to reseek may end up with NPE.
 AFAIK, current codebase don't call seek and reseek, but it is quite possible 
 in future. Personally, I spent lots of time to find source of wrong scanner 
 results in HBASE-5416.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6499) StoreScanner's QueryMatcher not reset on store update

2012-12-19 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536169#comment-13536169
 ] 

ramkrishna.s.vasudevan commented on HBASE-6499:
---

+1 on patch.

 StoreScanner's QueryMatcher not reset on store update
 -

 Key: HBASE-6499
 URL: https://issues.apache.org/jira/browse/HBASE-6499
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.96.0
Reporter: Max Lapan
Assignee: Max Lapan
 Fix For: 0.96.0

 Attachments: 6499-trunk-v2.txt, StoreScanner_not_reset_matcher.patch


 When underlying store changed (due compact, bulk load, etc), we destroy 
 current KeyValueHeap and recreate it using checkReseek call. Besides heap 
 recreation, it resets underlying QueryMatcher instance.
 The problem is that checkReseek not called by seek() and reseek(), only by 
 next(). If someone calls seek() just after store changed, it gets wrong 
 scanner results. Call to reseek may end up with NPE.
 AFAIK, current codebase don't call seek and reseek, but it is quite possible 
 in future. Personally, I spent lots of time to find source of wrong scanner 
 results in HBASE-5416.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server


[ 
https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536172#comment-13536172
 ] 

nkeywal commented on HBASE-7390:


This code there is very very smart: if there is a open in progress, it changes 
the internal state to close, then raises an exception through the call to 
checkIfRegionInTransition. As we changed the state, we will have, if we were 
currently opening, a message saying that we were trying to close a region 
already closing.

As well, it really means the other stuff going on is supposed to catch the 
state change?

{code}
  public CloseRegionResponse closeRegion(final RpcController controller,
  final CloseRegionRequest request) throws ServiceException {
//..
  Boolean openAction = regionsInTransitionInRS.get(encodedName);
  if (openAction != null) {
if (openAction.booleanValue()) {
  regionsInTransitionInRS.replace(encodedName, openAction, 
Boolean.FALSE);
}
checkIfRegionInTransition(encodedName, CLOSE);
  }
   
  requestCount.increment();
  LOG.info(Received close region:  + region.getRegionNameAsString() +
. Version of ZK closing node: + versionOfClosingNode +
. Destination server: + sn);
  HRegionInfo regionInfo = region.getRegionInfo();
  checkIfRegionInTransition(encodedName, CLOSE);
  boolean closed = closeRegion(
regionInfo, false, zk, versionOfClosingNode, sn);
  CloseRegionResponse.Builder builder =
CloseRegionResponse.newBuilder().setClosed(closed);
  return builder.build();
} catch (IOException ie) {
  throw new ServiceException(ie);
}
  }
{code}

 Add extra test cases for assignement on the region server
 -

 Key: HBASE-7390
 URL: https://issues.apache.org/jira/browse/HBASE-7390
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Minor
 Attachments: 7390.v1.patch, assignment_zk_states.jpg


 We don't have a lot of tests on the region server itself.
 Here are some.
 Some of them are failing, feedback welcome.
 See as well the attached state diagram for the ZK nodes on assignment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7295) Contention in HBaseClient.getConnection

2012-12-19 Thread Varun Sharma (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536173#comment-13536173
 ] 

Varun Sharma commented on HBASE-7295:
-

I cannot really make connection volatile since its declared inside the 
function and hence, the reference can only be assigned to, in a single thread.

 Contention in HBaseClient.getConnection
 ---

 Key: HBASE-7295
 URL: https://issues.apache.org/jira/browse/HBASE-7295
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.94.3
Reporter: Varun Sharma
Assignee: Varun Sharma
 Fix For: 0.96.0, 0.94.4

 Attachments: 7295-0.94.txt, 7295-0.94-v2.txt, 7295-0.94-v3.txt, 
 7295-0.94-v4.txt, 7295-0.94-v5.txt, 7295-trunk.txt, 7295-trunk.txt, 
 7295-trunk-v2.txt, 7295-trunk-v3.txt, 7295-trunk-v3.txt


 HBaseClient.getConnection() synchronizes on the connections object. We found 
 severe contention on a thrift gateway which was fanning out roughly 3000+ 
 calls per second to hbase region servers. The thrift gateway had 2000+ 
 threads for handling incoming connections. Threads were blocked on the 
 syncrhonized block - we set ipc.pool.size to 200. Since we are using 
 RoundRobin/ThreadLocal pool only - its not necessary to synchronize on 
 connections - it might lead to cases where we might go slightly over the 
 ipc.max.pool.size() but the additional connections would timeout after 
 maxIdleTime - underlying PoolMap connections object is thread safe.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server


[ 
https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536176#comment-13536176
 ] 

nkeywal commented on HBASE-7390:


bq. Why does master not see SPLITTING? Because it does not have a watcher? It 
cannot create a znode when there is a SPLITTING znode in place though. That is 
what we want, right? Ditto for the OPENING? Does the master not have a watcher 
on the OPENING znode?

I need to explore the splitting case, but there are no real reason imho (at 
least not a ZK reason).

bq. testMultipleOpen
I'm on this one (see comment above), it's not easy. The fix is trivial. What it 
is revealing is not.


 Add extra test cases for assignement on the region server
 -

 Key: HBASE-7390
 URL: https://issues.apache.org/jira/browse/HBASE-7390
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Minor
 Attachments: 7390.v1.patch, assignment_zk_states.jpg


 We don't have a lot of tests on the region server itself.
 Here are some.
 Some of them are failing, feedback welcome.
 See as well the attached state diagram for the ZK nodes on assignment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7392) Disable failing example unit tests TestZooKeeperScanPolicyObserver and TestBulkDeleteProtocol

stack created HBASE-7392:


 Summary: Disable failing example unit tests 
TestZooKeeperScanPolicyObserver and TestBulkDeleteProtocol
 Key: HBASE-7392
 URL: https://issues.apache.org/jira/browse/HBASE-7392
 Project: HBase
  Issue Type: Task
Reporter: stack


Jenkins run https://builds.apache.org/job/HBase-TRUNK/3638/ turned up two broke 
example tests.  They pass on a jdk6 machine locally but not on my jdk7 laptop.  
Somethings up.  My guess is that these failures have been there a while but 
only surfaced because we got further than we normally do on a jenkins run.

Tests have no output on jenkins.

If I run w/

$ MAVEN_OPTS= -Xmx3g mvn test -PlocalTests -Dtest=TestBulkDeleteProtocol 
-Dtest.output.tofile=false 

I get:

{code}
---
 T E S T S
---
Running org.apache.hadoop.hbase.coprocessor.example.TestBulkDeleteProtocol
2012-12-19 09:34:36,340 INFO  [main] hbase.HBaseTestingUtility(713): Starting 
up minicluster with 1 master(s) and 2 regionserver(s) and 2 datanode(s)
2012-12-19 09:34:46,362 INFO  [main] hbase.HBaseTestingUtility(343): Created 
new mini-cluster data directory: 
/Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/dfscluster_1c4634ed-2333-48ee-807d-c56f8c4ff20f
2012-12-19 09:34:46,362 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
test.cache.data to 
/Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/cache_data
 in system properties and HBase conf
2012-12-19 09:34:46,363 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
hadoop.tmp.dir to 
/Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/hadoop_tmp
 in system properties and HBase conf
2012-12-19 09:34:46,363 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
hadoop.log.dir to 
/Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/hadoop_logs
 in system properties and HBase conf
2012-12-19 09:34:46,364 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
mapred.local.dir to 
/Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/mapred_local
 in system properties and HBase conf
2012-12-19 09:34:46,364 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
mapred.temp.dir to 
/Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/mapred_temp
 in system properties and HBase conf
2012-12-19 09:34:46,365 INFO  [main] hbase.HBaseTestingUtility(536): read short 
circuit is ON for user stack
2012-12-19 09:34:46.438 java[16837:1703] Unable to load realm info from 
SCDynamicStore
2012-12-19 09:34:56,540 DEBUG [main] fs.HFileSystem(199): Starting 
addLocationsOrderInterceptor with class class 
org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks
2012-12-19 09:34:56,541 WARN  [main] fs.HFileSystem(215): The file system is 
not a DistributedFileSystem.Not adding block location reordering
2012-12-19 09:34:56,669 WARN  [main] namenode.FSNamesystem(564): The 
dfs.support.append option is in your configuration, however append is not 
supported. This configuration option is no longer required to enable sync.
2012-12-19 09:35:06,962 WARN  [main] namenode.FSNamesystem(564): The 
dfs.support.append option is in your configuration, however append is not 
supported. This configuration option is no longer required to enable sync.
2012-12-19 09:35:07,076 INFO  [main] log.Slf4jLog(67): Logging to 
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2012-12-19 09:35:07,151 INFO  [main] log.Slf4jLog(67): jetty-6.1.26
2012-12-19 09:35:07,184 INFO  [main] log.Slf4jLog(67): Extract 
jar:file:/Users/stack/.m2/repository/org/apache/hadoop/hadoop-core/1.1.1/hadoop-core-1.1.1.jar!/webapps/hdfs
 to 
/var/folders/bp/2z1cykc92rs6j24251cg__phgp/T/Jetty_localhost_57924_hdfsg9mqyr/webapp
2012-12-19 09:35:07,402 INFO  [main] log.Slf4jLog(67): Started 
SelectChannelConnector@localhost:57924
Starting DataNode 0 with dfs.data.dir: 
/Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/dfscluster_1c4634ed-2333-48ee-807d-c56f8c4ff20f/dfs/data/data1,/Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/dfscluster_1c4634ed-2333-48ee-807d-c56f8c4ff20f/dfs/data/data2
2012-12-19 09:35:17,505 INFO  [main] hbase.HBaseTestingUtility(825): Shutting 
down minicluster
2012-12-19 09:35:17,519 INFO  [main] hbase.HBaseTestingUtility(834): 
Minicluster is down
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 41.582 sec  
FAILURE!
{code}


And.


{code}

2012-12-19 09:38:52,368 INFO  [main] zookeeper.MiniZooKeeperCluster(197): 
Started MiniZK Cluster and connect 1 ZK server

[jira] [Updated] (HBASE-7392) Disable failing example unit tests TestZooKeeperScanPolicyObserver and TestBulkDeleteProtocol


 [ 
https://issues.apache.org/jira/browse/HBASE-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-7392:
-

Attachment: 7392.txt

Here is patch to disable the two failing tests.

 Disable failing example unit tests TestZooKeeperScanPolicyObserver and 
 TestBulkDeleteProtocol
 -

 Key: HBASE-7392
 URL: https://issues.apache.org/jira/browse/HBASE-7392
 Project: HBase
  Issue Type: Task
Reporter: stack
 Attachments: 7392.txt


 Jenkins run https://builds.apache.org/job/HBase-TRUNK/3638/ turned up two 
 broke example tests.  They pass on a jdk6 machine locally but not on my jdk7 
 laptop.  Somethings up.  My guess is that these failures have been there a 
 while but only surfaced because we got further than we normally do on a 
 jenkins run.
 Tests have no output on jenkins.
 If I run w/
 $ MAVEN_OPTS= -Xmx3g mvn test -PlocalTests -Dtest=TestBulkDeleteProtocol 
 -Dtest.output.tofile=false 
 I get:
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.coprocessor.example.TestBulkDeleteProtocol
 2012-12-19 09:34:36,340 INFO  [main] hbase.HBaseTestingUtility(713): Starting 
 up minicluster with 1 master(s) and 2 regionserver(s) and 2 datanode(s)
 2012-12-19 09:34:46,362 INFO  [main] hbase.HBaseTestingUtility(343): Created 
 new mini-cluster data directory: 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/dfscluster_1c4634ed-2333-48ee-807d-c56f8c4ff20f
 2012-12-19 09:34:46,362 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
 test.cache.data to 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/cache_data
  in system properties and HBase conf
 2012-12-19 09:34:46,363 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
 hadoop.tmp.dir to 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/hadoop_tmp
  in system properties and HBase conf
 2012-12-19 09:34:46,363 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
 hadoop.log.dir to 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/hadoop_logs
  in system properties and HBase conf
 2012-12-19 09:34:46,364 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
 mapred.local.dir to 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/mapred_local
  in system properties and HBase conf
 2012-12-19 09:34:46,364 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
 mapred.temp.dir to 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/mapred_temp
  in system properties and HBase conf
 2012-12-19 09:34:46,365 INFO  [main] hbase.HBaseTestingUtility(536): read 
 short circuit is ON for user stack
 2012-12-19 09:34:46.438 java[16837:1703] Unable to load realm info from 
 SCDynamicStore
 2012-12-19 09:34:56,540 DEBUG [main] fs.HFileSystem(199): Starting 
 addLocationsOrderInterceptor with class class 
 org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks
 2012-12-19 09:34:56,541 WARN  [main] fs.HFileSystem(215): The file system is 
 not a DistributedFileSystem.Not adding block location reordering
 2012-12-19 09:34:56,669 WARN  [main] namenode.FSNamesystem(564): The 
 dfs.support.append option is in your configuration, however append is not 
 supported. This configuration option is no longer required to enable sync.
 2012-12-19 09:35:06,962 WARN  [main] namenode.FSNamesystem(564): The 
 dfs.support.append option is in your configuration, however append is not 
 supported. This configuration option is no longer required to enable sync.
 2012-12-19 09:35:07,076 INFO  [main] log.Slf4jLog(67): Logging to 
 org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via 
 org.mortbay.log.Slf4jLog
 2012-12-19 09:35:07,151 INFO  [main] log.Slf4jLog(67): jetty-6.1.26
 2012-12-19 09:35:07,184 INFO  [main] log.Slf4jLog(67): Extract 
 jar:file:/Users/stack/.m2/repository/org/apache/hadoop/hadoop-core/1.1.1/hadoop-core-1.1.1.jar!/webapps/hdfs
  to 
 /var/folders/bp/2z1cykc92rs6j24251cg__phgp/T/Jetty_localhost_57924_hdfsg9mqyr/webapp
 2012-12-19 09:35:07,402 INFO  [main] log.Slf4jLog(67): Started 
 SelectChannelConnector@localhost:57924
 Starting DataNode 0 with dfs.data.dir: 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/dfscluster_1c4634ed-2333-48ee-807d-c56f8c4ff20f/dfs/data/data1,/Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/dfscluster_1c4634ed-2333-48ee-807d-c56f8c4ff20f/dfs/data/data2
 2012-12-19 09:35:17,505 INFO

[jira] [Commented] (HBASE-7145) ReusableStreamGzipCodec NPE upon reset with IBM JDK


[ 
https://issues.apache.org/jira/browse/HBASE-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536190#comment-13536190
 ] 

Ted Yu commented on HBASE-7145:
---

[~stack]:
Can you take a look at patch v3 ?

 ReusableStreamGzipCodec NPE upon reset with IBM JDK
 ---

 Key: HBASE-7145
 URL: https://issues.apache.org/jira/browse/HBASE-7145
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.94.0
Reporter: Yu Li
Assignee: Yu Li
  Labels: gzip, ibm-jdk
 Fix For: 0.96.0

 Attachments: 7145-trunk-v3.txt, HBASE-7145.1.patch.txt, 
 HBASE-7145.2_trunk.patch.txt


 This is the same issue as described in HADOOP-8419, repeat the issue 
 description here:
 The ReusableStreamGzipCodec will NPE upon reset after finish when the native 
 zlib codec is not loaded. When the native zlib is loaded the codec creates a 
 CompressorOutputStream that doesn't have the problem, otherwise, the 
 ReusableStreamGzipCodec uses GZIPOutputStream which is extended to provide 
 the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 
 SR10, GZIPOutputStream#finish will release the underlying deflater(calls the 
 deflater's end method), which causes NPE upon reset. This seems to be an IBM 
 JDK quirk as Sun JDK and OpenJDK doesn't have this issue.
 Since in HBASE-5387 HBase source has refactor its code not to use hadoop's 
 GzipCodec during real compress/decompress, it's necessary to make a separate 
 patch for HBase on the same issue

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7392) Disable failing example unit tests TestZooKeeperScanPolicyObserver and TestBulkDeleteProtocol


[ 
https://issues.apache.org/jira/browse/HBASE-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536192#comment-13536192
 ] 

stack commented on HBASE-7392:
--

TestRowCountEndpoint, also an example, passed on jenkins but failed locally for 
me so it is likely not just a jdk7 thing.

Will wait before applying to see if these fails happen again up on a jenkins 
run.

 Disable failing example unit tests TestZooKeeperScanPolicyObserver and 
 TestBulkDeleteProtocol
 -

 Key: HBASE-7392
 URL: https://issues.apache.org/jira/browse/HBASE-7392
 Project: HBase
  Issue Type: Task
Reporter: stack
 Attachments: 7392.txt


 Jenkins run https://builds.apache.org/job/HBase-TRUNK/3638/ turned up two 
 broke example tests.  They pass on a jdk6 machine locally but not on my jdk7 
 laptop.  Somethings up.  My guess is that these failures have been there a 
 while but only surfaced because we got further than we normally do on a 
 jenkins run.
 Tests have no output on jenkins.
 If I run w/
 $ MAVEN_OPTS= -Xmx3g mvn test -PlocalTests -Dtest=TestBulkDeleteProtocol 
 -Dtest.output.tofile=false 
 I get:
 {code}
 ---
  T E S T S
 ---
 Running org.apache.hadoop.hbase.coprocessor.example.TestBulkDeleteProtocol
 2012-12-19 09:34:36,340 INFO  [main] hbase.HBaseTestingUtility(713): Starting 
 up minicluster with 1 master(s) and 2 regionserver(s) and 2 datanode(s)
 2012-12-19 09:34:46,362 INFO  [main] hbase.HBaseTestingUtility(343): Created 
 new mini-cluster data directory: 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/dfscluster_1c4634ed-2333-48ee-807d-c56f8c4ff20f
 2012-12-19 09:34:46,362 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
 test.cache.data to 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/cache_data
  in system properties and HBase conf
 2012-12-19 09:34:46,363 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
 hadoop.tmp.dir to 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/hadoop_tmp
  in system properties and HBase conf
 2012-12-19 09:34:46,363 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
 hadoop.log.dir to 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/hadoop_logs
  in system properties and HBase conf
 2012-12-19 09:34:46,364 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
 mapred.local.dir to 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/mapred_local
  in system properties and HBase conf
 2012-12-19 09:34:46,364 INFO  [main] hbase.HBaseTestingUtility(553): Setting 
 mapred.temp.dir to 
 /Users/stack/checkouts/trunk/hbase-examples/target/test-data/cbfdeaed-c701-4a96-9277-0d9b9615a06e/mapred_temp
  in system properties and HBase conf
 2012-12-19 09:34:46,365 INFO  [main] hbase.HBaseTestingUtility(536): read 
 short circuit is ON for user stack
 2012-12-19 09:34:46.438 java[16837:1703] Unable to load realm info from 
 SCDynamicStore
 2012-12-19 09:34:56,540 DEBUG [main] fs.HFileSystem(199): Starting 
 addLocationsOrderInterceptor with class class 
 org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks
 2012-12-19 09:34:56,541 WARN  [main] fs.HFileSystem(215): The file system is 
 not a DistributedFileSystem.Not adding block location reordering
 2012-12-19 09:34:56,669 WARN  [main] namenode.FSNamesystem(564): The 
 dfs.support.append option is in your configuration, however append is not 
 supported. This configuration option is no longer required to enable sync.
 2012-12-19 09:35:06,962 WARN  [main] namenode.FSNamesystem(564): The 
 dfs.support.append option is in your configuration, however append is not 
 supported. This configuration option is no longer required to enable sync.
 2012-12-19 09:35:07,076 INFO  [main] log.Slf4jLog(67): Logging to 
 org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via 
 org.mortbay.log.Slf4jLog
 2012-12-19 09:35:07,151 INFO  [main] log.Slf4jLog(67): jetty-6.1.26
 2012-12-19 09:35:07,184 INFO  [main] log.Slf4jLog(67): Extract 
 jar:file:/Users/stack/.m2/repository/org/apache/hadoop/hadoop-core/1.1.1/hadoop-core-1.1.1.jar!/webapps/hdfs
  to 
 /var/folders/bp/2z1cykc92rs6j24251cg__phgp/T/Jetty_localhost_57924_hdfsg9mqyr/webapp
 2012-12-19 09:35:07,402 INFO  [main] log.Slf4jLog(67): Started 
 SelectChannelConnector@localhost:57924
 Starting DataNode 0 with dfs.data.dir:

[jira] [Commented] (HBASE-7145) ReusableStreamGzipCodec NPE upon reset with IBM JDK


[ 
https://issues.apache.org/jira/browse/HBASE-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536195#comment-13536195
 ] 

stack commented on HBASE-7145:
--

You do this +JVM jvm = new JVM();.  Is it needed?  Can all in JVM be 
static (I've not looked... makes it less useful if have to instantiate it to 
use it).  Then you do private static boolean HAS_BROKEN_FINISH = 
JVM.isBrokenFinish();

Thanks.

 ReusableStreamGzipCodec NPE upon reset with IBM JDK
 ---

 Key: HBASE-7145
 URL: https://issues.apache.org/jira/browse/HBASE-7145
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.94.0
Reporter: Yu Li
Assignee: Yu Li
  Labels: gzip, ibm-jdk
 Fix For: 0.96.0

 Attachments: 7145-trunk-v3.txt, HBASE-7145.1.patch.txt, 
 HBASE-7145.2_trunk.patch.txt


 This is the same issue as described in HADOOP-8419, repeat the issue 
 description here:
 The ReusableStreamGzipCodec will NPE upon reset after finish when the native 
 zlib codec is not loaded. When the native zlib is loaded the codec creates a 
 CompressorOutputStream that doesn't have the problem, otherwise, the 
 ReusableStreamGzipCodec uses GZIPOutputStream which is extended to provide 
 the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 
 SR10, GZIPOutputStream#finish will release the underlying deflater(calls the 
 deflater's end method), which causes NPE upon reset. This seems to be an IBM 
 JDK quirk as Sun JDK and OpenJDK doesn't have this issue.
 Since in HBASE-5387 HBase source has refactor its code not to use hadoop's 
 GzipCodec during real compress/decompress, it's necessary to make a separate 
 patch for HBase on the same issue

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7379) Port '[89-fb] prevent OOM possibility due to per connection responseQueue being unbounded' to trunk


[ 
https://issues.apache.org/jira/browse/HBASE-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536202#comment-13536202
 ] 

Ted Yu commented on HBASE-7379:
---

[~stack]:
Do you have comments about patch v2 ?

Thanks

 Port '[89-fb] prevent OOM possibility due to per connection responseQueue 
 being unbounded' to trunk
 ---

 Key: HBASE-7379
 URL: https://issues.apache.org/jira/browse/HBASE-7379
 Project: HBase
  Issue Type: Sub-task
Reporter: Ted Yu
Assignee: Ted Yu
 Fix For: 0.96.0

 Attachments: 7379-trunk.txt, 7379-trunk-v2.txt


 HBASE-6728 ported '[89-fb] prevent OOM possibility due to per connection 
 responseQueue being unbounded' to 0.94 branch
 This issue is to port it to trunk

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7294) Check for snapshot file cleaners on start


[ 
https://issues.apache.org/jira/browse/HBASE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536200#comment-13536200
 ] 

Ted Yu commented on HBASE-7294:
---

{code}
+   * Called at startup, to verify if snapshot operation are supported, and to 
avoid
{code}
'operation are' - 'operation is'
{code}
+  private void checkSnapshotSupport(final Configuration conf, final 
MasterFileSystem mfs)
{code}
Please add @param for the two parameters.
{code}
+// Mark snapshot feature has enabled if cleaners are present
{code}
'feature has' - 'feature as'
{code}
+// otherwise we end up snapshot with data loss.
{code}
'end up snapshot with' - 'end up with snapshot'
{code}
+'hbase.master.hfilecleaner.plugins' with ' +
{code}
Please refer to constants instead of hardcoding. Same with 
'hbase.master.logcleaner.plugins'

 Check for snapshot file cleaners on start
 -

 Key: HBASE-7294
 URL: https://issues.apache.org/jira/browse/HBASE-7294
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, regionserver, snapshots, Zookeeper
Affects Versions: hbase-6055
Reporter: Jesse Yates
Assignee: Matteo Bertozzi
 Fix For: hbase-6055, 0.96.0

 Attachments: HBASE-7294-v1.patch, HBASE-7294-v2.patch, 
 HBASE-7294-v3.patch


 Snapshots currently use the SnaphotHfileCleaner and SnapshotHLogCleaner to 
 ensure that any hfiles or hlogs (respectively) that are currently part of a 
 snapshot are not removed from their respective archive directories (.archive 
 and .oldlogs).
 From Matteo Bertozzi:
 {quote}
 currently the snapshot cleaner is not in hbase-default.xml
 and there's no warning/exception on snapshot/restore operation, if not 
 enabled.
 even if we add the cleaner to the hbase-default.xml how do we ensure that the 
 user doesn't remove it?
 Do we want to hardcode the cleaner at master startup?
 Do we want to add a check in snapshot/restore that throws an exception if the 
 cleaner is not enabled?
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4443) optimize/avoid seeking to previous block when key you are interested in is the first one of a block

2012-12-19 Thread Kannan Muthukkaruppan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-4443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kannan Muthukkaruppan updated HBASE-4443:
-

Assignee: Adela Maznikar

optimize/avoid seeking to previous block when key you are interested in is
the first one of a block
-

Key: HBASE-4443
URL: https://issues.apache.org/jira/browse/HBASE-4443
Project: HBase
Issue Type: Improvement
Reporter: Kannan Muthukkaruppan
Assignee: Adela Maznikar

This issue primarily affects cases when you are storing large blobs, i.e.
when blocks contain small number of keys, and the chances of the key you are
looking for being the first block of a key is higher.
Say, you are looking for row/col, and row/col/ts=5 is the latest version
of the key in the HFile and is at the beginning of block X.
The search for the key is done by looking for row/col/TS=Long.MAX_VAL, but
this will land us in block X-1 (because ts=Long.MAX_VAL sorts ahead of ts=5);
only to find that there is no matching row/col in block X-1, and then we'll
advance to block X to return the value.
Seems like we should be able to optimize this somehow.
Some possibilities:
1) Suppose we track that the file contains no deletes, and if the CF setting
has MAX_VERSIONS=1, we can know for sure that block X - 1 does not contain
any relevant data, and directly position the seek to block X. [This will also
require the memstore flusher to remove extra versions if MAX_VERSION=1 and
not allow the file to contain duplicate entries for the same ROW/COL.]
Tracking deletes will also avoid in many cases, the seek to the top of the
row to look for DeleteFamily.
2) Have a dense index (1 entry per KV in the index; this might be ok for
large object case since index vs. data ratio will still be low).
3) Have the index contain the last KV of each block also in addition to the
first KV. This doubles the size of the index though.

[jira] [Updated] (HBASE-7199) hbck should check lingering reference hfile and have option to sideline them automatically

2012-12-19 Thread Jimmy Xiang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-7199:
---

   Resolution: Fixed
Fix Version/s: 0.94.4
   0.96.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Integrated into trunk and 0.94.  Thanks Jon for the review.

 hbck should check lingering reference hfile and have option to sideline them 
 automatically
 --

 Key: HBASE-7199
 URL: https://issues.apache.org/jira/browse/HBASE-7199
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0, 0.94.4

 Attachments: trunk-7199.patch, trunk-7199_v2.patch


 Sometimes, some lingering reference hfile points to some region doesn't exist 
 any more.  This will prevent the region to open.  Hbck should be able to find 
 these files and sideline them automatically if requested.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5416) Improve performance of scans with some kind of filters.

2012-12-19 Thread ramkrishna.s.vasudevan (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536211#comment-13536211
]

ramkrishna.s.vasudevan commented on HBASE-5416:
---

Some initial comments
{code}
getAllowLazyCfLoadingRaw(), getAllowLazyCfLoading()
{code}
It can be renamed better?
{code}
public enum LazyCfLoadingValue
{code}
Enum constants can be in CAPS.

Improve performance of scans with some kind of filters.
---

[jira] [Updated] (HBASE-7145) ReusableStreamGzipCodec NPE upon reset with IBM JDK


 [ 
https://issues.apache.org/jira/browse/HBASE-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-7145:
--

Attachment: 7145-trunk-v4.txt

Patch v4 addresses Stack's comments.

 ReusableStreamGzipCodec NPE upon reset with IBM JDK
 ---

 Key: HBASE-7145
 URL: https://issues.apache.org/jira/browse/HBASE-7145
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.94.0
Reporter: Yu Li
Assignee: Yu Li
  Labels: gzip, ibm-jdk
 Fix For: 0.96.0

 Attachments: 7145-trunk-v3.txt, 7145-trunk-v4.txt, 
 HBASE-7145.1.patch.txt, HBASE-7145.2_trunk.patch.txt


 This is the same issue as described in HADOOP-8419, repeat the issue 
 description here:
 The ReusableStreamGzipCodec will NPE upon reset after finish when the native 
 zlib codec is not loaded. When the native zlib is loaded the codec creates a 
 CompressorOutputStream that doesn't have the problem, otherwise, the 
 ReusableStreamGzipCodec uses GZIPOutputStream which is extended to provide 
 the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 
 SR10, GZIPOutputStream#finish will release the underlying deflater(calls the 
 deflater's end method), which causes NPE upon reset. This seems to be an IBM 
 JDK quirk as Sun JDK and OpenJDK doesn't have this issue.
 Since in HBASE-5387 HBase source has refactor its code not to use hadoop's 
 GzipCodec during real compress/decompress, it's necessary to make a separate 
 patch for HBase on the same issue

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5416) Improve performance of scans with some kind of filters.

[
https://issues.apache.org/jira/browse/HBASE-5416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536236#comment-13536236
]

Sergey Shelukhin commented on HBASE-5416:
-

[~shmuma] I actually had to change (re)seek during rebase, can you take a look
if there's some error there? Other than that, I attached the test to linked
JIRA, I can repro it consistently by changing the initial regions-per-server
constant in the test to low value (1-5).

Improve performance of scans with some kind of filters.
---

[jira] [Updated] (HBASE-7357) HBaseClient and HBaseServer should use hbase.security.authentication when negotiating authentication

2012-12-19 Thread Gary Helmling (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Helmling updated HBASE-7357:
-

Fix Version/s: 0.94.4
   0.96.0

 HBaseClient and HBaseServer should use hbase.security.authentication when 
 negotiating authentication
 

 Key: HBASE-7357
 URL: https://issues.apache.org/jira/browse/HBASE-7357
 Project: HBase
  Issue Type: Bug
  Components: security
Reporter: Gary Helmling
Assignee: Gary Helmling
 Fix For: 0.96.0, 0.94.4

 Attachments: HBASE-7357_0.94_2.patch, HBASE-7357_0.94.patch, 
 HBASE-7357.patch


 This came up in the context of testing HBASE-6788.  Currently HBaseClient and 
 HBaseServer call UserGroupInformation.isSecurityEnabled() when determining 
 whether or not to use SASL to negotiate connections.  This means they are 
 using the hadoop.security.authentication configuration value.  Since this is 
 in the context of HBase RPC connections, it seems more correct to use the 
 hbase.security.authentication configuration value by calling 
 User.isHBaseSecurityEnabled().

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5416) Improve performance of scans with some kind of filters.

[
https://issues.apache.org/jira/browse/HBASE-5416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536238#comment-13536238
]

Sergey Shelukhin commented on HBASE-5416:
-

[~ram_krish] Thanks!

Improve performance of scans with some kind of filters.
---

[jira] [Commented] (HBASE-7294) Check for snapshot file cleaners on start


[ 
https://issues.apache.org/jira/browse/HBASE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536240#comment-13536240
 ] 

Jonathan Hsieh commented on HBASE-7294:
---

We need to make sure the snapshots unit tests pass as well -- I believe they 
should all fail once this gets in because I don't think all install the 
cleaners.

 Check for snapshot file cleaners on start
 -

 Key: HBASE-7294
 URL: https://issues.apache.org/jira/browse/HBASE-7294
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, regionserver, snapshots, Zookeeper
Affects Versions: hbase-6055
Reporter: Jesse Yates
Assignee: Matteo Bertozzi
 Fix For: hbase-6055, 0.96.0

 Attachments: HBASE-7294-v1.patch, HBASE-7294-v2.patch, 
 HBASE-7294-v3.patch


 Snapshots currently use the SnaphotHfileCleaner and SnapshotHLogCleaner to 
 ensure that any hfiles or hlogs (respectively) that are currently part of a 
 snapshot are not removed from their respective archive directories (.archive 
 and .oldlogs).
 From Matteo Bertozzi:
 {quote}
 currently the snapshot cleaner is not in hbase-default.xml
 and there's no warning/exception on snapshot/restore operation, if not 
 enabled.
 even if we add the cleaner to the hbase-default.xml how do we ensure that the 
 user doesn't remove it?
 Do we want to hardcode the cleaner at master startup?
 Do we want to add a check in snapshot/restore that throws an exception if the 
 cleaner is not enabled?
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7145) ReusableStreamGzipCodec NPE upon reset with IBM JDK

[
https://issues.apache.org/jira/browse/HBASE-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536242#comment-13536242
]

stack commented on HBASE-7145:
--

+1. Fix spacing on commit... looks like there is a tab in there... in the new
method added to JVM class.

ReusableStreamGzipCodec NPE upon reset with IBM JDK
---

Key: HBASE-7145
URL: https://issues.apache.org/jira/browse/HBASE-7145
Project: HBase
Issue Type: Sub-task
Affects Versions: 0.94.0
Reporter: Yu Li
Assignee: Yu Li
Labels: gzip, ibm-jdk
Fix For: 0.96.0

Attachments: 7145-trunk-v3.txt, 7145-trunk-v4.txt,
HBASE-7145.1.patch.txt, HBASE-7145.2_trunk.patch.txt

This is the same issue as described in HADOOP-8419, repeat the issue
description here:
The ReusableStreamGzipCodec will NPE upon reset after finish when the native
zlib codec is not loaded. When the native zlib is loaded the codec creates a
CompressorOutputStream that doesn't have the problem, otherwise, the
ReusableStreamGzipCodec uses GZIPOutputStream which is extended to provide
the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6
SR10, GZIPOutputStream#finish will release the underlying deflater(calls the
deflater's end method), which causes NPE upon reset. This seems to be an IBM
JDK quirk as Sun JDK and OpenJDK doesn't have this issue.
Since in HBASE-5387 HBase source has refactor its code not to use hadoop's
GzipCodec during real compress/decompress, it's necessary to make a separate
patch for HBase on the same issue

[jira] [Commented] (HBASE-7357) HBaseClient and HBaseServer should use hbase.security.authentication when negotiating authentication

2012-12-19 Thread Gary Helmling (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536241#comment-13536241
]

Gary Helmling commented on HBASE-7357:
--

Committed to 0.94 branch.

bq. Does this need a documentation change somewhere aswell? Can do as separate
jira if needed.

Yes, I checked the ref guide and it looks like we should update the security
docs to show hadoop.security.authentication in the configuration in addition to
hbase.security.authentication. I'll open another ticket for the doc update.

HBaseClient and HBaseServer should use hbase.security.authentication when
negotiating authentication

Key: HBASE-7357
URL: https://issues.apache.org/jira/browse/HBASE-7357
Project: HBase
Issue Type: Bug
Components: security
Reporter: Gary Helmling
Assignee: Gary Helmling
Fix For: 0.96.0, 0.94.4

Attachments: HBASE-7357_0.94_2.patch, HBASE-7357_0.94.patch,
HBASE-7357.patch

This came up in the context of testing HBASE-6788. Currently HBaseClient and
HBaseServer call UserGroupInformation.isSecurityEnabled() when determining
whether or not to use SASL to negotiate connections. This means they are
using the hadoop.security.authentication configuration value. Since this is
in the context of HBase RPC connections, it seems more correct to use the
hbase.security.authentication configuration value by calling
User.isHBaseSecurityEnabled().

[jira] [Updated] (HBASE-7145) ReusableStreamGzipCodec NPE upon reset with IBM JDK


 [ 
https://issues.apache.org/jira/browse/HBASE-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-7145:
--

Status: Open  (was: Patch Available)

 ReusableStreamGzipCodec NPE upon reset with IBM JDK
 ---

 Key: HBASE-7145
 URL: https://issues.apache.org/jira/browse/HBASE-7145
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.94.0
Reporter: Yu Li
Assignee: Yu Li
  Labels: gzip, ibm-jdk
 Fix For: 0.96.0

 Attachments: 7145-trunk-v3.txt, 7145-trunk-v4.txt, 7145-trunk-v5.txt, 
 HBASE-7145.1.patch.txt, HBASE-7145.2_trunk.patch.txt


 This is the same issue as described in HADOOP-8419, repeat the issue 
 description here:
 The ReusableStreamGzipCodec will NPE upon reset after finish when the native 
 zlib codec is not loaded. When the native zlib is loaded the codec creates a 
 CompressorOutputStream that doesn't have the problem, otherwise, the 
 ReusableStreamGzipCodec uses GZIPOutputStream which is extended to provide 
 the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 
 SR10, GZIPOutputStream#finish will release the underlying deflater(calls the 
 deflater's end method), which causes NPE upon reset. This seems to be an IBM 
 JDK quirk as Sun JDK and OpenJDK doesn't have this issue.
 Since in HBASE-5387 HBase source has refactor its code not to use hadoop's 
 GzipCodec during real compress/decompress, it's necessary to make a separate 
 patch for HBase on the same issue

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7145) ReusableStreamGzipCodec NPE upon reset with IBM JDK


 [ 
https://issues.apache.org/jira/browse/HBASE-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-7145:
--

Attachment: 7145-trunk-v5.txt

Patch v5 fixes the tab Stack pointed out.

 ReusableStreamGzipCodec NPE upon reset with IBM JDK
 ---

 Key: HBASE-7145
 URL: https://issues.apache.org/jira/browse/HBASE-7145
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.94.0
Reporter: Yu Li
Assignee: Yu Li
  Labels: gzip, ibm-jdk
 Fix For: 0.96.0

 Attachments: 7145-trunk-v3.txt, 7145-trunk-v4.txt, 7145-trunk-v5.txt, 
 HBASE-7145.1.patch.txt, HBASE-7145.2_trunk.patch.txt


 This is the same issue as described in HADOOP-8419, repeat the issue 
 description here:
 The ReusableStreamGzipCodec will NPE upon reset after finish when the native 
 zlib codec is not loaded. When the native zlib is loaded the codec creates a 
 CompressorOutputStream that doesn't have the problem, otherwise, the 
 ReusableStreamGzipCodec uses GZIPOutputStream which is extended to provide 
 the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 
 SR10, GZIPOutputStream#finish will release the underlying deflater(calls the 
 deflater's end method), which causes NPE upon reset. This seems to be an IBM 
 JDK quirk as Sun JDK and OpenJDK doesn't have this issue.
 Since in HBASE-5387 HBase source has refactor its code not to use hadoop's 
 GzipCodec during real compress/decompress, it's necessary to make a separate 
 patch for HBase on the same issue

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks

2012-12-19 Thread Nick Dimiduk (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536252#comment-13536252
 ] 

Nick Dimiduk commented on HBASE-5487:
-

I'll take a stab at this.

 Generic framework for Master-coordinated tasks
 --

 Key: HBASE-5487
 URL: https://issues.apache.org/jira/browse/HBASE-5487
 Project: HBase
  Issue Type: New Feature
  Components: master, regionserver, Zookeeper
Affects Versions: 0.94.0
Reporter: Mubarak Seyed
  Labels: noob

 Need a framework to execute master-coordinated tasks in a fault-tolerant 
 manner. 
 Master-coordinated tasks such as online-scheme change and delete-range 
 (deleting region(s) based on start/end key) can make use of this framework.
 The advantages of framework are
 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for 
 master-coordinated tasks
 2. Ability to abstract the common functions across Master - ZK and RS - ZK
 3. Easy to plugin new master-coordinated tasks without adding code to core 
 components

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5487) Generic framework for Master-coordinated tasks

2012-12-19 Thread Nick Dimiduk (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk reassigned HBASE-5487:
---

Assignee: Nick Dimiduk

 Generic framework for Master-coordinated tasks
 --

 Key: HBASE-5487
 URL: https://issues.apache.org/jira/browse/HBASE-5487
 Project: HBase
  Issue Type: New Feature
  Components: master, regionserver, Zookeeper
Affects Versions: 0.94.0
Reporter: Mubarak Seyed
Assignee: Nick Dimiduk
  Labels: noob

 Need a framework to execute master-coordinated tasks in a fault-tolerant 
 manner. 
 Master-coordinated tasks such as online-scheme change and delete-range 
 (deleting region(s) based on start/end key) can make use of this framework.
 The advantages of framework are
 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for 
 master-coordinated tasks
 2. Ability to abstract the common functions across Master - ZK and RS - ZK
 3. Easy to plugin new master-coordinated tasks without adding code to core 
 components

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7339) Splitting a hfilelink causes region servers to go down.

2012-12-19 Thread Renata Ghisloti Duarte de Souza (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Hsieh updated HBASE-7339:
--

Attachment: pre-hbase-7339.v2.patch
hbase-7339.v2.patch

patches applied for the snapshots branch (12/18 rebase)

Splitting a hfilelink causes region servers to go down.
---

Key: HBASE-7339
URL: https://issues.apache.org/jira/browse/HBASE-7339
Project: HBase
Issue Type: Sub-task
Components: snapshots
Affects Versions: hbase-6055
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: hbase-6055

Attachments: hbase-7339.patch, hbase-7339.v2.patch,
pre-hbase-7339.patch, pre-hbase-7339.v2.patch

Steps:
- Have a single region table t with 15 hfiles in it.
- Snapshot it. (was done using online snapshot from HBASE-7321)
- Clone a snapshot to table t'.
- t' has its region do a post-open task that attempts to compact region.
policy does not compact all files. (default seems to be 10)
- after compaction we have hfile links and real hfiles mixed in the region
- t' starts splitting
- creating split references, opening daughers fails
- hfile links are split, creating hfile link daughter refs.
{{hfile\-region\-table.parentregion}}
- these split hfile links are interpreted as hfile links with table
{{table.parentregion}} -
{{hfile\-region\-table.parentregion}} (groupings interpreted
incorrectly)
- Since this is after the splitting PONR, this aborts the server. It then
spreads to the next server.

[jira] [Commented] (HBASE-7145) ReusableStreamGzipCodec NPE upon reset with IBM JDK


[ 
https://issues.apache.org/jira/browse/HBASE-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536267#comment-13536267
 ] 

Renata Ghisloti Duarte de Souza commented on HBASE-7145:


+1 The patch solves the problem for me too.

 ReusableStreamGzipCodec NPE upon reset with IBM JDK
 ---

 Key: HBASE-7145
 URL: https://issues.apache.org/jira/browse/HBASE-7145
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.94.0
Reporter: Yu Li
Assignee: Yu Li
  Labels: gzip, ibm-jdk
 Fix For: 0.96.0

 Attachments: 7145-trunk-v3.txt, 7145-trunk-v4.txt, 7145-trunk-v5.txt, 
 HBASE-7145.1.patch.txt, HBASE-7145.2_trunk.patch.txt


 This is the same issue as described in HADOOP-8419, repeat the issue 
 description here:
 The ReusableStreamGzipCodec will NPE upon reset after finish when the native 
 zlib codec is not loaded. When the native zlib is loaded the codec creates a 
 CompressorOutputStream that doesn't have the problem, otherwise, the 
 ReusableStreamGzipCodec uses GZIPOutputStream which is extended to provide 
 the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 
 SR10, GZIPOutputStream#finish will release the underlying deflater(calls the 
 deflater's end method), which causes NPE upon reset. This seems to be an IBM 
 JDK quirk as Sun JDK and OpenJDK doesn't have this issue.
 Since in HBASE-5387 HBase source has refactor its code not to use hadoop's 
 GzipCodec during real compress/decompress, it's necessary to make a separate 
 patch for HBase on the same issue

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks


[ 
https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536271#comment-13536271
 ] 

Andrew Purtell commented on HBASE-5487:
---

Brownie points if coprocessors can use it too.

 Generic framework for Master-coordinated tasks
 --

 Key: HBASE-5487
 URL: https://issues.apache.org/jira/browse/HBASE-5487
 Project: HBase
  Issue Type: New Feature
  Components: master, regionserver, Zookeeper
Affects Versions: 0.94.0
Reporter: Mubarak Seyed
Assignee: Nick Dimiduk
  Labels: noob

 Need a framework to execute master-coordinated tasks in a fault-tolerant 
 manner. 
 Master-coordinated tasks such as online-scheme change and delete-range 
 (deleting region(s) based on start/end key) can make use of this framework.
 The advantages of framework are
 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for 
 master-coordinated tasks
 2. Ability to abstract the common functions across Master - ZK and RS - ZK
 3. Easy to plugin new master-coordinated tasks without adding code to core 
 components

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7357) HBaseClient and HBaseServer should use hbase.security.authentication when negotiating authentication


 [ 
https://issues.apache.org/jira/browse/HBASE-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-7357:
-

Release Note: 
This is a slightly incompatible change.
HBase SASL connection negotiation is now enabled by setting 
hbase.security.authentication rather than hadoop.security.authentication.
Hadoop Flags: Incompatible change,Reviewed

 HBaseClient and HBaseServer should use hbase.security.authentication when 
 negotiating authentication
 

 Key: HBASE-7357
 URL: https://issues.apache.org/jira/browse/HBASE-7357
 Project: HBase
  Issue Type: Bug
  Components: security
Reporter: Gary Helmling
Assignee: Gary Helmling
 Fix For: 0.96.0, 0.94.4

 Attachments: HBASE-7357_0.94_2.patch, HBASE-7357_0.94.patch, 
 HBASE-7357.patch


 This came up in the context of testing HBASE-6788.  Currently HBaseClient and 
 HBaseServer call UserGroupInformation.isSecurityEnabled() when determining 
 whether or not to use SASL to negotiate connections.  This means they are 
 using the hadoop.security.authentication configuration value.  Since this is 
 in the context of HBase RPC connections, it seems more correct to use the 
 hbase.security.authentication configuration value by calling 
 User.isHBaseSecurityEnabled().

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7357) HBaseClient and HBaseServer should use hbase.security.authentication when negotiating authentication


 [ 
https://issues.apache.org/jira/browse/HBASE-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-7357:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 HBaseClient and HBaseServer should use hbase.security.authentication when 
 negotiating authentication
 

 Key: HBASE-7357
 URL: https://issues.apache.org/jira/browse/HBASE-7357
 Project: HBase
  Issue Type: Bug
  Components: security
Reporter: Gary Helmling
Assignee: Gary Helmling
 Fix For: 0.96.0, 0.94.4

 Attachments: HBASE-7357_0.94_2.patch, HBASE-7357_0.94.patch, 
 HBASE-7357.patch


 This came up in the context of testing HBASE-6788.  Currently HBaseClient and 
 HBaseServer call UserGroupInformation.isSecurityEnabled() when determining 
 whether or not to use SASL to negotiate connections.  This means they are 
 using the hadoop.security.authentication configuration value.  Since this is 
 in the context of HBase RPC connections, it seems more correct to use the 
 hbase.security.authentication configuration value by calling 
 User.isHBaseSecurityEnabled().

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7339) Splitting a hfilelink causes region servers to go down.


[ 
https://issues.apache.org/jira/browse/HBASE-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536280#comment-13536280
 ] 

Hadoop QA commented on HBASE-7339:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12561761/pre-hbase-7339.v2.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 85 new 
or modified tests.

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3616//console

This message is automatically generated.

 Splitting a hfilelink causes region servers to go down.
 ---

 Key: HBASE-7339
 URL: https://issues.apache.org/jira/browse/HBASE-7339
 Project: HBase
  Issue Type: Sub-task
  Components: snapshots
Affects Versions: hbase-6055
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
 Fix For: hbase-6055

 Attachments: hbase-7339.patch, hbase-7339.v2.patch, 
 pre-hbase-7339.patch, pre-hbase-7339.v2.patch


 Steps:
 - Have a single region table t with 15 hfiles in it.
 - Snapshot it. (was done using online snapshot from HBASE-7321)
 - Clone a snapshot to table t'. 
 - t' has its region do a post-open task that attempts to compact region.  
 policy does not compact all files. (default seems to be 10)
 - after compaction we have hfile links and real hfiles mixed in the region
 - t' starts splitting
 - creating split references, opening daughers fails 
 - hfile links are split, creating hfile link daughter refs.  
 {{hfile\-region\-table.parentregion}}
 - these split hfile links are interpreted as hfile links with table 
 {{table.parentregion}} - 
 {{hfile\-region\-table.parentregion}}  (groupings interpreted 
 incorrectly)
 - Since this is after the splitting PONR, this aborts the server.  It then 
 spreads to the next server.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7351) Periodic health check chore


[ 
https://issues.apache.org/jira/browse/HBASE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536278#comment-13536278
 ] 

Andrew Purtell commented on HBASE-7351:
---

Thanks for the consideration [~avandana]. I'll take a detailed look while 
porting your latest 0.94 patch to trunk shortly.

 Periodic health check chore
 ---

 Key: HBASE-7351
 URL: https://issues.apache.org/jira/browse/HBASE-7351
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Vandana Ayyalasomayajula
Assignee: Vandana Ayyalasomayajula
Priority: Minor
 Attachments: HBASE-7331_94_1.patch, HBASE-7331_94.patch, 
 HBASE-7331_trunk.patch


 Similar to MAPREDUCE-211, region server should also have a mechanism to check 
 the health of the node.  It should run the health check script periodically 
 and if there is any errors, it should stop itself gracefully. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5487) Generic framework for Master-coordinated tasks


 [ 
https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5487:
--

Labels:   (was: noob)

Not sure how this had the noob tag --this sounds like a fairly major 
undertaking!


 Generic framework for Master-coordinated tasks
 --

 Key: HBASE-5487
 URL: https://issues.apache.org/jira/browse/HBASE-5487
 Project: HBase
  Issue Type: New Feature
  Components: master, regionserver, Zookeeper
Affects Versions: 0.94.0
Reporter: Mubarak Seyed
Assignee: Nick Dimiduk

 Need a framework to execute master-coordinated tasks in a fault-tolerant 
 manner. 
 Master-coordinated tasks such as online-scheme change and delete-range 
 (deleting region(s) based on start/end key) can make use of this framework.
 The advantages of framework are
 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for 
 master-coordinated tasks
 2. Ability to abstract the common functions across Master - ZK and RS - ZK
 3. Easy to plugin new master-coordinated tasks without adding code to core 
 components

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7374) Expose master table operations for coprocessors by way of MasterServices


 [ 
https://issues.apache.org/jira/browse/HBASE-7374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-7374:
--

   Resolution: Fixed
Fix Version/s: 0.94.4
   0.96.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to 0.94. Thanks.

 Expose master table operations for coprocessors by way of MasterServices
 

 Key: HBASE-7374
 URL: https://issues.apache.org/jira/browse/HBASE-7374
 Project: HBase
  Issue Type: Improvement
  Components: Coprocessors, master
Affects Versions: 0.96.0, 0.94.4
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Minor
 Fix For: 0.96.0, 0.94.4

 Attachments: HBASE-7374-0.94.patch, HBASE-7374.patch


 I have something I'm working on that, as a coprocessor, when it initializes 
 would like to add a column to a table should that column be missing. Exposing 
 master table operations for coprocessors by way of MasterServices was how I 
 solved this problem, and is generally useful for all master coprocessors.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6499) StoreScanner's QueryMatcher not reset on store update

[
https://issues.apache.org/jira/browse/HBASE-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536284#comment-13536284
]

Hadoop QA commented on HBASE-6499:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561736/6499-trunk-v2.txt
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:red}-1 findbugs{color}. The patch appears to introduce 28 new
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 core tests{color}. The patch passed unit tests in .

{color:red}-1 core zombie tests{color}. There are zombie tests. See build
logs for details.

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3614//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3614//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3614//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3614//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3614//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3614//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3614//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3614//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3614//console

This message is automatically generated.

StoreScanner's QueryMatcher not reset on store update
-

Key: HBASE-6499
URL: https://issues.apache.org/jira/browse/HBASE-6499
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.96.0
Reporter: Max Lapan
Assignee: Max Lapan
Fix For: 0.96.0

Attachments: 6499-trunk-v2.txt, StoreScanner_not_reset_matcher.patch

When underlying store changed (due compact, bulk load, etc), we destroy
current KeyValueHeap and recreate it using checkReseek call. Besides heap
recreation, it resets underlying QueryMatcher instance.
The problem is that checkReseek not called by seek() and reseek(), only by
next(). If someone calls seek() just after store changed, it gets wrong
scanner results. Call to reseek may end up with NPE.
AFAIK, current codebase don't call seek and reseek, but it is quite possible
in future. Personally, I spent lots of time to find source of wrong scanner
results in HBASE-5416.

[jira] [Commented] (HBASE-7360) Snapshot 0.94 Backport


[ 
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536287#comment-13536287
 ] 

Jonathan Hsieh commented on HBASE-7360:
---

There was some offline conversion about this patch and bringing Snapshots to 
apache 0.94.  Let's bring it here.  The patch on review board has gotten a +1.  

The question: should be bring this in to 0.94?  [~lhofhansl]? [~jyates]?





 Snapshot 0.94 Backport 
 ---

 Key: HBASE-7360
 URL: https://issues.apache.org/jira/browse/HBASE-7360
 Project: HBase
  Issue Type: Sub-task
  Components: snapshots
Affects Versions: 0.94.3
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055


 Backport snapshot code to 0.94
 The main changes needed to backport the snapshot are related to the protobuf 
 vs writable rpc.
 Offline Snapshot
  * HBASE-6610 - HFileLink: Hardlink alternative for snapshot restore
  * HBASE-6765 - Take a Snapshot interface
  * HBASE-6571 - Generic multi-thread/cross-process error handling framework
  * HBASE-6353 - Snapshots shell
  * HBASE-7107 - Snapshot References Utils (FileSystem Visitor)
  * HBASE-6836 - Offline snapshots
  * HBASE-6865 - Snapshot File Cleaners
  * HBASE-6777 - Snapshot Restore Interface
  * HBASE-6230 - Clone/Restore Snapshots
  * HBASE-6802 - Export Snapshot

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (HBASE-7360) Snapshot 0.94 Backport

[
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536287#comment-13536287
]

Jonathan Hsieh edited comment on HBASE-7360 at 12/19/12 6:49 PM:
-

There was some offline conversion about this patch and bringing Snapshots to
apache 0.94. Let's bring it here. The patch on review board has gotten a +1.

The question: should be bring this in to 0.94? [~lhofhansl]? [~jesse_yates]?

was (Author: jmhsieh):
There was some offline conversion about this patch and bringing Snapshots
to apache 0.94. Let's bring it here. The patch on review board has gotten a
+1.

The question: should be bring this in to 0.94? [~lhofhansl]? [~jyates]?

Snapshot 0.94 Backport
---

Key: HBASE-7360
URL: https://issues.apache.org/jira/browse/HBASE-7360
Project: HBase
Issue Type: Sub-task
Components: snapshots
Affects Versions: 0.94.3
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Fix For: hbase-6055

Backport snapshot code to 0.94
The main changes needed to backport the snapshot are related to the protobuf
vs writable rpc.
Offline Snapshot
* HBASE-6610 - HFileLink: Hardlink alternative for snapshot restore
* HBASE-6765 - Take a Snapshot interface
* HBASE-6571 - Generic multi-thread/cross-process error handling framework
* HBASE-6353 - Snapshots shell
* HBASE-7107 - Snapshot References Utils (FileSystem Visitor)
* HBASE-6836 - Offline snapshots
* HBASE-6865 - Snapshot File Cleaners
* HBASE-6777 - Snapshot Restore Interface
* HBASE-6230 - Clone/Restore Snapshots
* HBASE-6802 - Export Snapshot

[jira] [Commented] (HBASE-6499) StoreScanner's QueryMatcher not reset on store update


[ 
https://issues.apache.org/jira/browse/HBASE-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536290#comment-13536290
 ] 

Ted Yu commented on HBASE-6499:
---

Integrated to trunk.

Thanks for the patch, Max.

Thanks for the review, Ram.

 StoreScanner's QueryMatcher not reset on store update
 -

 Key: HBASE-6499
 URL: https://issues.apache.org/jira/browse/HBASE-6499
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.96.0
Reporter: Max Lapan
Assignee: Max Lapan
 Fix For: 0.96.0

 Attachments: 6499-trunk-v2.txt, StoreScanner_not_reset_matcher.patch


 When underlying store changed (due compact, bulk load, etc), we destroy 
 current KeyValueHeap and recreate it using checkReseek call. Besides heap 
 recreation, it resets underlying QueryMatcher instance.
 The problem is that checkReseek not called by seek() and reseek(), only by 
 next(). If someone calls seek() just after store changed, it gets wrong 
 scanner results. Call to reseek may end up with NPE.
 AFAIK, current codebase don't call seek and reseek, but it is quite possible 
 in future. Personally, I spent lots of time to find source of wrong scanner 
 results in HBASE-5416.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6499) StoreScanner's QueryMatcher not reset on store update


 [ 
https://issues.apache.org/jira/browse/HBASE-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-6499:
--

Hadoop Flags: Reviewed

 StoreScanner's QueryMatcher not reset on store update
 -

 Key: HBASE-6499
 URL: https://issues.apache.org/jira/browse/HBASE-6499
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.96.0
Reporter: Max Lapan
Assignee: Max Lapan
 Fix For: 0.96.0

 Attachments: 6499-trunk-v2.txt, StoreScanner_not_reset_matcher.patch


 When underlying store changed (due compact, bulk load, etc), we destroy 
 current KeyValueHeap and recreate it using checkReseek call. Besides heap 
 recreation, it resets underlying QueryMatcher instance.
 The problem is that checkReseek not called by seek() and reseek(), only by 
 next(). If someone calls seek() just after store changed, it gets wrong 
 scanner results. Call to reseek may end up with NPE.
 AFAIK, current codebase don't call seek and reseek, but it is quite possible 
 in future. Personally, I spent lots of time to find source of wrong scanner 
 results in HBASE-5416.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5498) Secure Bulk Load

[
https://issues.apache.org/jira/browse/HBASE-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536293#comment-13536293
]

Andrew Purtell commented on HBASE-5498:
---

My guess is the impact of the PB conversion in trunk became a time sink.
[~toffer] how would you like to proceed here? Carry it over the finish line, or
transfer to me for same?

Secure Bulk Load

Key: HBASE-5498
URL: https://issues.apache.org/jira/browse/HBASE-5498
Project: HBase
Issue Type: Improvement
Components: security
Reporter: Francis Liu
Assignee: Francis Liu
Fix For: 0.96.0, 0.94.5

Attachments: HBASE-5498_94_2.patch, HBASE-5498_94_3.patch,
HBASE-5498_94.patch, HBASE-5498_94.patch, HBASE-5498_draft_94.patch,
HBASE-5498_draft.patch, HBASE-5498_trunk_2.patch, HBASE-5498_trunk_3.patch,
HBASE-5498_trunk_4.patch, HBASE-5498_trunk.patch

Design doc:
https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Secure+Bulk+Load
Short summary:
Security as it stands does not cover the bulkLoadHFiles() feature. Users
calling this method will bypass ACLs. Also loading is made more cumbersome in
a secure setting because of hdfs privileges. bulkLoadHFiles() moves the data
from user's directory to the hbase directory, which would require certain
write access privileges set.
Our solution is to create a coprocessor which makes use of AuthManager to
verify if a user has write access to the table. If so, launches a MR job as
the hbase user to do the importing (ie rewrite from text to hfiles). One
tricky part this job will have to do is impersonate the calling user when
reading the input files. We can do this by expecting the user to pass an hdfs
delegation token as part of the secureBulkLoad() coprocessor call and extend
an inputformat to make use of that token. The output is written to a
temporary directory accessible only by hbase and then bulkloadHFiles() is
called.

[jira] [Updated] (HBASE-7390) Add extra test cases for assignement on the region server


 [ 
https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-7390:
---

Attachment: 7390.v2.patch

 Add extra test cases for assignement on the region server
 -

 Key: HBASE-7390
 URL: https://issues.apache.org/jira/browse/HBASE-7390
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Minor
 Attachments: 7390.v1.patch, 7390.v2.patch, assignment_zk_states.jpg


 We don't have a lot of tests on the region server itself.
 Here are some.
 Some of them are failing, feedback welcome.
 See as well the attached state diagram for the ZK nodes on assignment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7374) Expose master table operations for coprocessors by way of MasterServices

2012-12-19 Thread Enis Soztutar (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536295#comment-13536295
 ] 

Enis Soztutar commented on HBASE-7374:
--

bq. Enis Soztutar I'm going to commit this as is. I think we should handle the 
HBASE-7305 issues on 7305 by adding javadoc to the master preXXX and postXXX 
interface definitions, then by exposing the table lock API via MasterServices 
with a fat warning about the locking discipline.
Agreed. Thanks Andrew. 

 Expose master table operations for coprocessors by way of MasterServices
 

 Key: HBASE-7374
 URL: https://issues.apache.org/jira/browse/HBASE-7374
 Project: HBase
  Issue Type: Improvement
  Components: Coprocessors, master
Affects Versions: 0.96.0, 0.94.4
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Minor
 Fix For: 0.96.0, 0.94.4

 Attachments: HBASE-7374-0.94.patch, HBASE-7374.patch


 I have something I'm working on that, as a coprocessor, when it initializes 
 would like to add a column to a table should that column be missing. Exposing 
 master table operations for coprocessors by way of MasterServices was how I 
 solved this problem, and is generally useful for all master coprocessors.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7390) Add extra test cases for assignement on the region server


 [ 
https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-7390:
---

Fix Version/s: 0.96.0
 Assignee: nkeywal
   Status: Patch Available  (was: Open)

 Add extra test cases for assignement on the region server
 -

 Key: HBASE-7390
 URL: https://issues.apache.org/jira/browse/HBASE-7390
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Fix For: 0.96.0

 Attachments: 7390.v1.patch, 7390.v2.patch, assignment_zk_states.jpg


 We don't have a lot of tests on the region server itself.
 Here are some.
 Some of them are failing, feedback welcome.
 See as well the attached state diagram for the ZK nodes on assignment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7295) Contention in HBaseClient.getConnection

[
https://issues.apache.org/jira/browse/HBASE-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536297#comment-13536297
]

Lars Hofhansl commented on HBASE-7295:
--

Yeah, maybe use ConcurrentHashMap.putIfAbsent (I think one of your patches had
that). If you find the Connection is in there already at that point just close
the newly created one.

In the end I am somewhat skeptical that this is even an issue. Do you have a
matching number of handlers threads at the regionservers?
(hbase.regionserver.handler.count)

Contention in HBaseClient.getConnection
---

Key: HBASE-7295
URL: https://issues.apache.org/jira/browse/HBASE-7295
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.3
Reporter: Varun Sharma
Assignee: Varun Sharma
Fix For: 0.96.0, 0.94.4

Attachments: 7295-0.94.txt, 7295-0.94-v2.txt, 7295-0.94-v3.txt,
7295-0.94-v4.txt, 7295-0.94-v5.txt, 7295-trunk.txt, 7295-trunk.txt,
7295-trunk-v2.txt, 7295-trunk-v3.txt, 7295-trunk-v3.txt

HBaseClient.getConnection() synchronizes on the connections object. We found
severe contention on a thrift gateway which was fanning out roughly 3000+
calls per second to hbase region servers. The thrift gateway had 2000+
threads for handling incoming connections. Threads were blocked on the
syncrhonized block - we set ipc.pool.size to 200. Since we are using
RoundRobin/ThreadLocal pool only - its not necessary to synchronize on
connections - it might lead to cases where we might go slightly over the
ipc.max.pool.size() but the additional connections would timeout after
maxIdleTime - underlying PoolMap connections object is thread safe.

[jira] [Updated] (HBASE-7295) Contention in HBaseClient.getConnection


 [ 
https://issues.apache.org/jira/browse/HBASE-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-7295:
-

Fix Version/s: (was: 0.94.4)
   0.94.5

 Contention in HBaseClient.getConnection
 ---

 Key: HBASE-7295
 URL: https://issues.apache.org/jira/browse/HBASE-7295
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.94.3
Reporter: Varun Sharma
Assignee: Varun Sharma
 Fix For: 0.96.0, 0.94.5

 Attachments: 7295-0.94.txt, 7295-0.94-v2.txt, 7295-0.94-v3.txt, 
 7295-0.94-v4.txt, 7295-0.94-v5.txt, 7295-trunk.txt, 7295-trunk.txt, 
 7295-trunk-v2.txt, 7295-trunk-v3.txt, 7295-trunk-v3.txt


 HBaseClient.getConnection() synchronizes on the connections object. We found 
 severe contention on a thrift gateway which was fanning out roughly 3000+ 
 calls per second to hbase region servers. The thrift gateway had 2000+ 
 threads for handling incoming connections. Threads were blocked on the 
 syncrhonized block - we set ipc.pool.size to 200. Since we are using 
 RoundRobin/ThreadLocal pool only - its not necessary to synchronize on 
 connections - it might lead to cases where we might go slightly over the 
 ipc.max.pool.size() but the additional connections would timeout after 
 maxIdleTime - underlying PoolMap connections object is thread safe.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7351) Periodic health check chore

2012-12-19 Thread Vandana Ayyalasomayajula (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536296#comment-13536296
 ] 

Vandana Ayyalasomayajula commented on HBASE-7351:
-

[~apurtell] If you are okay with the latest patch for 0.94 branch, I can create 
a patch for the trunk. Thanks for your time. 

 Periodic health check chore
 ---

 Key: HBASE-7351
 URL: https://issues.apache.org/jira/browse/HBASE-7351
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Vandana Ayyalasomayajula
Assignee: Vandana Ayyalasomayajula
Priority: Minor
 Attachments: HBASE-7331_94_1.patch, HBASE-7331_94.patch, 
 HBASE-7331_trunk.patch


 Similar to MAPREDUCE-211, region server should also have a mechanism to check 
 the health of the node.  It should run the health check script periodically 
 and if there is any errors, it should stop itself gracefully. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (HBASE-7360) Snapshot 0.94 Backport

[
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536287#comment-13536287
]

Jonathan Hsieh edited comment on HBASE-7360 at 12/19/12 6:55 PM:
-

There was some offline conversion about wheter we bring this patch and
snapshots to apache 0.94. Let's do it here. The patch on review board has
gotten a +1.

The question: should be bring this in to 0.94? [~lhofhansl]? [~jesse_yates]?

was (Author: jmhsieh):
There was some offline conversion about this patch and bringing Snapshots
to apache 0.94. Let's bring it here. The patch on review board has gotten a
+1.

The question: should be bring this in to 0.94? [~lhofhansl]? [~jesse_yates]?

Snapshot 0.94 Backport
---

[jira] [Comment Edited] (HBASE-7360) Snapshot 0.94 Backport

[
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536287#comment-13536287
]

Jonathan Hsieh edited comment on HBASE-7360 at 12/19/12 6:55 PM:
-

There was some offline conversation about wheter we bring this patch and
snapshots to apache 0.94. Let's do it here. The patch on review board has
gotten a +1.

The question: should be bring this in to 0.94? [~lhofhansl]? [~jesse_yates]?

was (Author: jmhsieh):
There was some offline conversion about wheter we bring this patch and
snapshots to apache 0.94. Let's do it here. The patch on review board has
gotten a +1.

The question: should be bring this in to 0.94? [~lhofhansl]? [~jesse_yates]?

Snapshot 0.94 Backport
---

[jira] [Commented] (HBASE-7390) Add extra test cases for assignement on the region server


[ 
https://issues.apache.org/jira/browse/HBASE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536299#comment-13536299
 ] 

nkeywal commented on HBASE-7390:


v2 fixes the issue. I've just removed the code above, I don't see how it can 
possibly work: if we were previously opening the region, we're updating the RIT 
state to closing, but without stopping the opening or starting a close handler.

 Add extra test cases for assignement on the region server
 -

 Key: HBASE-7390
 URL: https://issues.apache.org/jira/browse/HBASE-7390
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Fix For: 0.96.0

 Attachments: 7390.v1.patch, 7390.v2.patch, assignment_zk_states.jpg


 We don't have a lot of tests on the region server itself.
 Here are some.
 Some of them are failing, feedback welcome.
 See as well the attached state diagram for the ZK nodes on assignment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7011) log rolling and cache flushing should be able to proceed in parallel


[ 
https://issues.apache.org/jira/browse/HBASE-7011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536301#comment-13536301
 ] 

Sergey Shelukhin commented on HBASE-7011:
-

Will do it as part of HBASE-7329 on trunk

 log rolling and cache flushing should be able to proceed in parallel
 

 Key: HBASE-7011
 URL: https://issues.apache.org/jira/browse/HBASE-7011
 Project: HBase
  Issue Type: Improvement
Reporter: Kannan Muthukkaruppan
Assignee: Kannan Muthukkaruppan

 Today, during a memstore flush (snapshot of memstore + flushing to disk), log 
 rolling cannot happen. This seems like a bad design, and an unnecessary 
 restriction. 
 Possible reasons cited for this in code are:
 (i) maintenance of the lastSeqWritten map.
 (ii) writing a completed-cache-flush marker into the same log before the 
 roll.
 It seems that we can implement a new design for (i) to avoid holding the lock 
 for the entire duration of the flush. And the motivation for (ii) is not even 
 clear. We should reason this out, and make sure we can relax the restriction. 
 [See related discussion in HBASE-6980.]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (HBASE-7360) Snapshot 0.94 Backport

[
https://issues.apache.org/jira/browse/HBASE-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536287#comment-13536287
]

Jonathan Hsieh edited comment on HBASE-7360 at 12/19/12 6:57 PM:
-

There was some offline conversation about wheather we bring this patch and
snapshots to apache 0.94. Let's do that conversation here to make why we are
deciding one way or the other. The patch on review board has gotten a +1.

The question: should we bring this in to 0.94? [~lhofhansl]? [~jesse_yates]?

(sorry for all the edits)

was (Author: jmhsieh):
There was some offline conversation about wheter we bring this patch and
snapshots to apache 0.94. Let's do it here. The patch on review board has
gotten a +1.

The question: should be bring this in to 0.94? [~lhofhansl]? [~jesse_yates]?

Snapshot 0.94 Backport
---

[jira] [Commented] (HBASE-7385) Do not abort regionserver if StoreFlusher.flushCache() fails

2012-12-19 Thread Enis Soztutar (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536304#comment-13536304
 ] 

Enis Soztutar commented on HBASE-7385:
--

bq. How will Memstore.snapshot() become idempotent?
snapshot already checks for whether there is a previous snapshot lying around, 
we can just remove the warning log. If flush is aborted, the memstore snapshot 
will live on, and new snapshot cannot be taken unless this one has been 
flushed. We might have to adjust the memstore size() calculations. 


 Do not abort regionserver if StoreFlusher.flushCache() fails
 

 Key: HBASE-7385
 URL: https://issues.apache.org/jira/browse/HBASE-7385
 Project: HBase
  Issue Type: Improvement
  Components: io, regionserver
Reporter: Enis Soztutar
Assignee: Enis Soztutar

 A rare NN failover may cause RS abort, in the following sequence of events: 
  - RS tries to flush the memstore
  - Create a file, start block, and acquire a lease
  - Block is complete, lease removed, but before we send the RPC response back 
 to the client, NN is killed.
  - New NN comes up, client retries the block complete again, the new NN 
 throws lease expired since the block was already complete.
  - RS receives the exception, and aborts.
 This is actually a NN+DFSClient issue that, the dfs client from RS does not 
 receive the rpc response about the block close, and upon retry on the new NN, 
 it gets the exception, since the file was already closed. However, although 
 this is DFS client specific, we can also make RS more resilient by not 
 aborting the RS upon exception from the flushCache(). We can change 
 StoreFlusher so that: 
 StoreFlusher.prepare() will become idempotent (so will Memstore.snapshot())
 StoreFlusher.flushCache() will throw with IOException upon DFS exception, but 
 we catch IOException, and just abort the flush request (not RS).
 StoreFlusher.commit() still cause RS abort on exception. This is also 
 debatable. If dfs is alive, and we can undo the flush changes, than we should 
 not abort. 
 logs: 
 {code}
 org.apache.hadoop.hbase.DroppedSnapshotException: region: 
 loadtest_ha,e658,1355820729877.298bcbd550b80507a379fe67eefbe5ea.
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1485)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1364)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:896)
   at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:845)
   at 
 org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
   at 
 org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
   at java.lang.Thread.run(Thread.java:662)
 Caused by: org.apache.hadoop.ipc.RemoteException: 
 org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on 
 /apps/hbase/data/loadtest_ha/298bcbd550b80507a379fe67eefbe5ea/.tmp/5cf8951ee12449ce8e4e6dd0bf1645c2
  File is not open for writing. [Lease.  Holder: 
 DFSClient_hb_rs_XXX,60020,1355813552066_203591774_25, pendingcreates: 1]
   at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1724)
   at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1707)
   at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:1762)
   at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:1750)
   at 
 org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:779)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:396)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)
   at org.apache.hadoop.ipc.Client.call(Client.java:1107)
   at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

[jira] [Commented] (HBASE-7329) remove flush-related records from WAL


[ 
https://issues.apache.org/jira/browse/HBASE-7329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536305#comment-13536305
 ] 

Sergey Shelukhin commented on HBASE-7329:
-

ok, have to read a lot of code to remove a lot of code, so the patch may be 
slower in the making :)

 remove flush-related records from WAL
 -

 Key: HBASE-7329
 URL: https://issues.apache.org/jira/browse/HBASE-7329
 Project: HBase
  Issue Type: Improvement
  Components: wal
Affects Versions: 0.96.0
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin

 Comments from many people in HBASE-6466 and HBASE-6980 indicate that flush 
 records in WAL are not useful. If so, they should be removed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7145) ReusableStreamGzipCodec NPE upon reset with IBM JDK

[
https://issues.apache.org/jira/browse/HBASE-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536307#comment-13536307
]

Hadoop QA commented on HBASE-7145:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12561748/7145-trunk-v4.txt
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.

{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:red}-1 findbugs{color}. The patch appears to introduce 28 new
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 core tests{color}. The patch passed unit tests in .

{color:red}-1 core zombie tests{color}. There are zombie tests. See build
logs for details.

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3615//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3615//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3615//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3615//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3615//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3615//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3615//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3615//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3615//console

This message is automatically generated.

ReusableStreamGzipCodec NPE upon reset with IBM JDK
---

Attachments: 7145-trunk-v3.txt, 7145-trunk-v4.txt, 7145-trunk-v5.txt,
HBASE-7145.1.patch.txt, HBASE-7145.2_trunk.patch.txt

[jira] [Commented] (HBASE-7331) Add access control for region open and close, row locking, and stopping the regionserver


[ 
https://issues.apache.org/jira/browse/HBASE-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536306#comment-13536306
 ] 

Lars Hofhansl commented on HBASE-7331:
--

Patch looks fine, does not seem to break existing functionality or introduce 
and incompatibilities.
I'll defer to you [~apurtell]. If you think it's save, let's commit to 0.94.

Aren't we getting rid of rowlocks?

 Add access control for region open and close, row locking, and stopping the 
 regionserver
 

 Key: HBASE-7331
 URL: https://issues.apache.org/jira/browse/HBASE-7331
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, security
Affects Versions: 0.94.3, 0.96.0
Reporter: Vandana Ayyalasomayajula
Assignee: Vandana Ayyalasomayajula
 Fix For: 0.96.0, 0.94.4

 Attachments: HBASE-7331_94_02.patch, HBASE-7331_94.patch, 
 HBASE-7331_trunk_02.patch, HBASE-7331_trunk.patch


 The following APIs in HRegionServer are either missing hooks to coprocessor 
 or the hooks are not implemented in the AccessController class for security. 
 As a result any unauthorized user can:
 1.Open a region
 2. Close a region
 3. Stop region server
 4. Lock a row
 5. Unlock a row.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7351) Periodic health check chore


[ 
https://issues.apache.org/jira/browse/HBASE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536310#comment-13536310
 ] 

Andrew Purtell commented on HBASE-7351:
---

Yes the v1 0.94 patch looks good, except for the occasional missing trivial 
s/region// . Ideally there could be one simple unit test to confirm the 
HealthChecker works as advertised. I was just thinking about this: a very small 
value for hbase.node.health.script.timeout with a shell script that sleeps 
too long, a script that is 'exit -1', a missing script (fails with exception), 
and finally a script that is 'exit 0'. What do you think?

 Periodic health check chore
 ---

 Key: HBASE-7351
 URL: https://issues.apache.org/jira/browse/HBASE-7351
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Vandana Ayyalasomayajula
Assignee: Vandana Ayyalasomayajula
Priority: Minor
 Attachments: HBASE-7331_94_1.patch, HBASE-7331_94.patch, 
 HBASE-7331_trunk.patch


 Similar to MAPREDUCE-211, region server should also have a mechanism to check 
 the health of the node.  It should run the health check script periodically 
 and if there is any errors, it should stop itself gracefully. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7331) Add access control for region open and close, row locking, and stopping the regionserver


[ 
https://issues.apache.org/jira/browse/HBASE-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13536313#comment-13536313
 ] 

Andrew Purtell commented on HBASE-7331:
---

bq. I'll defer to you Andrew Purtell. If you think it's save, let's commit to 
0.94.

Ok, I'll commit it.

bq. Aren't we getting rid of rowlocks?

We are, so the trunk patch doesn't have this, but they are still present in 
0.94.

 Add access control for region open and close, row locking, and stopping the 
 regionserver
 

 Key: HBASE-7331
 URL: https://issues.apache.org/jira/browse/HBASE-7331
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, security
Affects Versions: 0.94.3, 0.96.0
Reporter: Vandana Ayyalasomayajula
Assignee: Vandana Ayyalasomayajula
 Fix For: 0.96.0, 0.94.4

 Attachments: HBASE-7331_94_02.patch, HBASE-7331_94.patch, 
 HBASE-7331_trunk_02.patch, HBASE-7331_trunk.patch


 The following APIs in HRegionServer are either missing hooks to coprocessor 
 or the hooks are not implemented in the AccessController class for security. 
 As a result any unauthorized user can:
 1.Open a region
 2. Close a region
 3. Stop region server
 4. Lock a row
 5. Unlock a row.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7331) Add access control for region open and close, row locking, and stopping the regionserver