[jira] [Commented] (HBASE-7879) JUnit dependency in main from htrace
[ https://issues.apache.org/jira/browse/HBASE-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585708#comment-13585708 ] nkeywal commented on HBASE-7879: Committed. The patch includes removing the 'runtime' scope from our junit dependency. Thanks a lot for the quick fix release Jonathan! JUnit dependency in main from htrace Key: HBASE-7879 URL: https://issues.apache.org/jira/browse/HBASE-7879 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 7879.v1.patch HTrace main depends on Junit , it should be only test. I created a junit in the github, that's https://github.com/cloudera/htrace/issues/1. If it's not fixed, we will be able to drop it in our pom, but let's wait a little before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7879) JUnit dependency in main from htrace
[ https://issues.apache.org/jira/browse/HBASE-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-7879: --- Resolution: Fixed Fix Version/s: 0.96.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) JUnit dependency in main from htrace Key: HBASE-7879 URL: https://issues.apache.org/jira/browse/HBASE-7879 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 7879.v1.patch HTrace main depends on Junit , it should be only test. I created a junit in the github, that's https://github.com/cloudera/htrace/issues/1. If it's not fixed, we will be able to drop it in our pom, but let's wait a little before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7188: - Attachment: HBASE-7188-3.patch Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Client, IPC/RPC Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6222) Add per-KeyValue Security
[ https://issues.apache.org/jira/browse/HBASE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585714#comment-13585714 ] ramkrishna.s.vasudevan commented on HBASE-6222: --- Andy, The memstore flushes if more than 1 CF exists, will it have an impact on this new CF introduced? One more question is, Once in the AccessController hooks we have ensured that the permission is available by checking the new CF _acl_, when the actual scan goes we can avoid this Cell right? May be am missing something. Pls correct me if am wrong. Add per-KeyValue Security - Key: HBASE-6222 URL: https://issues.apache.org/jira/browse/HBASE-6222 Project: HBase Issue Type: New Feature Components: security Affects Versions: 0.96.0, 0.98.0 Reporter: stack Assignee: Andrew Purtell Attachments: 6222-aclcf.patch, 6222.pdf, cell-acls-kv-tags-not-for-review.zip, HBaseCellRow-LevelSecurityDesignDoc.docx, HBaseCellRow-LevelSecurityPRD.docx Saw an interesting article: http://www.fiercegovernmentit.com/story/sasc-accumulo-language-pro-open-source-say-proponents/2012-06-14 The Senate Armed Services Committee version of the fiscal 2013 national defense authorization act (S. 3254) would require DoD agencies to foreswear the Accumulo NoSQL database after Sept. 30, 2013, unless the DoD CIO certifies that there exists either no viable commercial open source database with security features comparable to [Accumulo] (such as the HBase or Cassandra databases)... Not sure what a 'commercial open source database' is, and I'm not sure whats going on in the article, but tra-la-la'ing, if we had per-KeyValue 'security' like Accumulo's, we might put ourselves in the running for federal contributions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7896) make rename_table working in 92/94
[ https://issues.apache.org/jira/browse/HBASE-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585718#comment-13585718 ] Matteo Bertozzi commented on HBASE-7896: Looking at the code and trying it out... a couple of notes, aside from the discussion above. The concept, of scanning META and for each region recreate the folder with the new name and move the files in is ok, and it's also used by the clone snapshot. This script doesn't consider ACL. This means that if you rename the table, the ACL from the old table name are not moved to the new table, and they are not even removed. The .regioninfo file doesn't seems to be created in the proper way... by comparing the HRegion code with the rename_table script {code} // HRegion.java code HRegion.checkRegioninfoOnFilesystem() this.regionInfo.write(out); out.write('\n'); out.write('\n'); out.write(Bytes.toBytes(this.regionInfo.toString())); # rename_table.rb out = fs.create(regioninfofile) newR.getRegionInfo().write(out) newHtd.write(out) out.close() {code} At the end of the script you create the zknode for the new table but you don't delete the old one... also the node is created in the enabled state. This means that if I do, is_enabled it returns true but the scan or other operations doesn't work since the table is not really enabled (regions assgned), and if you don't call enable you're in a weird state that can break monitoring tools or similar. make rename_table working in 92/94 -- Key: HBASE-7896 URL: https://issues.apache.org/jira/browse/HBASE-7896 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.92.2, 0.94.5 Reporter: Tianying Chang Assignee: Tianying Chang Fix For: 0.92.2, 0.94.6 Attachments: rename_table.rb The rename_table function is very useful for our customers. However, rename_table.rb does not work for 92/94. It has several bugs. It will be useful to fix them so that users can solve their problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7188: - Attachment: HBASE-7188-5.patch Added headers. Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Client, IPC/RPC Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch, HBASE-7188-5.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7879) JUnit dependency in main from htrace
[ https://issues.apache.org/jira/browse/HBASE-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585742#comment-13585742 ] Hudson commented on HBASE-7879: --- Integrated in HBase-TRUNK #3895 (See [https://builds.apache.org/job/HBase-TRUNK/3895/]) HBASE-7879 JUnit dependency in main from htrace (Revision 1449614) Result = FAILURE nkeywal : Files : * /hbase/trunk/pom.xml JUnit dependency in main from htrace Key: HBASE-7879 URL: https://issues.apache.org/jira/browse/HBASE-7879 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 7879.v1.patch HTrace main depends on Junit , it should be only test. I created a junit in the github, that's https://github.com/cloudera/htrace/issues/1. If it's not fixed, we will be able to drop it in our pom, but let's wait a little before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585747#comment-13585747 ] Hadoop QA commented on HBASE-7188: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570746/HBASE-7188-3.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 66 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.coprocessor.example.TestBulkDeleteProtocol Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4521//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4521//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4521//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4521//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4521//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4521//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4521//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4521//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4521//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4521//console This message is automatically generated. Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Client, IPC/RPC Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch, HBASE-7188-5.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7188: - Attachment: HBASE-7188-6.patch Moved tests as well. Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Client, IPC/RPC Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch, HBASE-7188-5.patch, HBASE-7188-6.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585760#comment-13585760 ] Hadoop QA commented on HBASE-7188: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570755/HBASE-7188-6.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 307 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestSnapshotFromAdmin Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4523//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4523//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4523//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4523//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4523//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4523//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4523//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4523//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4523//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4523//console This message is automatically generated. Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Client, IPC/RPC Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch, HBASE-7188-5.patch, HBASE-7188-6.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7924) thrift interface is inconsistently implemented on timestamp/range filtering
Guido Serra aka Zeph created HBASE-7924: --- Summary: thrift interface is inconsistently implemented on timestamp/range filtering Key: HBASE-7924 URL: https://issues.apache.org/jira/browse/HBASE-7924 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.94.5, 0.92.0 Reporter: Guido Serra aka Zeph a getRowsWithColumnsTs or a Scan object are being exposed (as by documentation and .thrift description file) only as *exact* timestamp matcher, no timerange functionality is (supposedly) being exposed instead, the Scan object is behaving as by documentation but the getRowsWithColumnsTs() beneath has a timerange behaviour see: HBASE-5694, HBASE-7907 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7924) thrift interface is inconsistently implemented on timestamp/range scan
[ https://issues.apache.org/jira/browse/HBASE-7924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guido Serra aka Zeph updated HBASE-7924: Summary: thrift interface is inconsistently implemented on timestamp/range scan (was: thrift interface is inconsistently implemented on timestamp/range filtering) thrift interface is inconsistently implemented on timestamp/range scan -- Key: HBASE-7924 URL: https://issues.apache.org/jira/browse/HBASE-7924 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.92.0, 0.94.5 Reporter: Guido Serra aka Zeph a getRowsWithColumnsTs or a Scan object are being exposed (as by documentation and .thrift description file) only as *exact* timestamp matcher, no timerange functionality is (supposedly) being exposed instead, the Scan object is behaving as by documentation but the getRowsWithColumnsTs() beneath has a timerange behaviour see: HBASE-5694, HBASE-7907 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7925) Back port HBASE-6881 into 0.94
rajeshbabu created HBASE-7925: - Summary: Back port HBASE-6881 into 0.94 Key: HBASE-7925 URL: https://issues.apache.org/jira/browse/HBASE-7925 Project: HBase Issue Type: Bug Reporter: rajeshbabu Assignee: rajeshbabu -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7924) thrift interface is inconsistently implemented on timestamp/range scan
[ https://issues.apache.org/jira/browse/HBASE-7924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guido Serra aka Zeph updated HBASE-7924: Description: a getRowsWithColumnsTs or a Scan object are being exposed (as by documentation and .thrift description file) only as *exact* timestamp matcher, no timerange functionality is (supposedly) being exposed - see: HBASE-7907 instead, the Scan object is behaving as by documentation but the getRowsWithColumnsTs() beneath has a timerange behaviour {code} if (tScan.isSetTimestamp()) { scan.setTimeRange(Long.MIN_VALUE, tScan.getTimestamp()); } {code} see: HBASE-5694 was: a getRowsWithColumnsTs or a Scan object are being exposed (as by documentation and .thrift description file) only as *exact* timestamp matcher, no timerange functionality is (supposedly) being exposed instead, the Scan object is behaving as by documentation but the getRowsWithColumnsTs() beneath has a timerange behaviour {code} if (tScan.isSetTimestamp()) { scan.setTimeRange(Long.MIN_VALUE, tScan.getTimestamp()); } {code} see: HBASE-5694, HBASE-7907 thrift interface is inconsistently implemented on timestamp/range scan -- Key: HBASE-7924 URL: https://issues.apache.org/jira/browse/HBASE-7924 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.92.0, 0.94.5 Reporter: Guido Serra aka Zeph a getRowsWithColumnsTs or a Scan object are being exposed (as by documentation and .thrift description file) only as *exact* timestamp matcher, no timerange functionality is (supposedly) being exposed - see: HBASE-7907 instead, the Scan object is behaving as by documentation but the getRowsWithColumnsTs() beneath has a timerange behaviour {code} if (tScan.isSetTimestamp()) { scan.setTimeRange(Long.MIN_VALUE, tScan.getTimestamp()); } {code} see: HBASE-5694 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7924) thrift interface is inconsistently implemented on timestamp/range scan
[ https://issues.apache.org/jira/browse/HBASE-7924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guido Serra aka Zeph updated HBASE-7924: Description: a getRowsWithColumnsTs or a Scan object are being exposed (as by documentation and .thrift description file) only as *exact* timestamp matcher, no timerange functionality is (supposedly) being exposed instead, the Scan object is behaving as by documentation but the getRowsWithColumnsTs() beneath has a timerange behaviour {code} if (tScan.isSetTimestamp()) { scan.setTimeRange(Long.MIN_VALUE, tScan.getTimestamp()); } {code} see: HBASE-5694, HBASE-7907 was: a getRowsWithColumnsTs or a Scan object are being exposed (as by documentation and .thrift description file) only as *exact* timestamp matcher, no timerange functionality is (supposedly) being exposed instead, the Scan object is behaving as by documentation but the getRowsWithColumnsTs() beneath has a timerange behaviour see: HBASE-5694, HBASE-7907 thrift interface is inconsistently implemented on timestamp/range scan -- Key: HBASE-7924 URL: https://issues.apache.org/jira/browse/HBASE-7924 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.92.0, 0.94.5 Reporter: Guido Serra aka Zeph a getRowsWithColumnsTs or a Scan object are being exposed (as by documentation and .thrift description file) only as *exact* timestamp matcher, no timerange functionality is (supposedly) being exposed instead, the Scan object is behaving as by documentation but the getRowsWithColumnsTs() beneath has a timerange behaviour {code} if (tScan.isSetTimestamp()) { scan.setTimeRange(Long.MIN_VALUE, tScan.getTimestamp()); } {code} see: HBASE-5694, HBASE-7907 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585763#comment-13585763 ] chunhui shen commented on HBASE-6721: - [~toffer] How about for this now? Is it available in your cluster? When do you consider to apply in trunk? Since we have all the tables' group info, could we remove GroupInfo.TABLEDESC_PROP_GROUP? Could we only use hbase table to storage group information? When we update the group information, why not update the memory the first? Then we need't roll back if persistence failed RegionServer Group based Assignment --- Key: HBASE-6721 URL: https://issues.apache.org/jira/browse/HBASE-6721 Project: HBase Issue Type: New Feature Reporter: Francis Liu Assignee: Vandana Ayyalasomayajula Fix For: 0.96.0 Attachments: HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, HBASE-6721_94_7.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, HBASE-6721-DesigDoc.pdf In multi-tenant deployments of HBase, it is likely that a RegionServer will be serving out regions from a number of different tables owned by various client applications. Being able to group a subset of running RegionServers and assign specific tables to it, provides a client application a level of isolation and resource allocation. The proposal essentially is to have an AssignmentManager which is aware of RegionServer groups and assigns tables to region servers based on groupings. Load balancing will occur on a per group basis as well. This is essentially a simplification of the approach taken in HBASE-4120. See attached document. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5694) getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly
[ https://issues.apache.org/jira/browse/HBASE-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585768#comment-13585768 ] Guido Serra aka Zeph commented on HBASE-5694: - k, [~ted_yu] I opened a bug report HBASE-7924 ... fix will follow getRowsWithColumnsTs() in Thrift service handles timestamps incorrectly --- Key: HBASE-5694 URL: https://issues.apache.org/jira/browse/HBASE-5694 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.92.1 Reporter: Wouter Bolsterlee Fix For: 0.94.0 Attachments: HBASE-5694.patch, HBASE-5694-trunk-20120402.patch, setTimestamp.patch The getRowsWithColumnsTs() method in the Thrift interface only applies the timestamp if columns are explicitly specified. However, this method also allows for columns to be unspecified (this is even used internally to implement e.g. getRows()). The cause of the bug is a minor scoping issue: the time range is set inside a wrong if statement. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585770#comment-13585770 ] Hadoop QA commented on HBASE-7188: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570751/HBASE-7188-5.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 255 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces lines longer than 100 {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4522//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4522//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4522//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4522//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4522//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4522//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4522//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4522//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4522//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4522//console This message is automatically generated. Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Client, IPC/RPC Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch, HBASE-7188-5.patch, HBASE-7188-6.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7893) hbase.hregion.majorcompaction not taking effect.
[ https://issues.apache.org/jira/browse/HBASE-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585787#comment-13585787 ] Anoop Sam John commented on HBASE-7893: --- I can see in this RS log that it is getting YouAreDeadException from Master. Also ZK session timeout exceptions present. Can you check the GC for this process. Is this suffering from long full GCs? hbase.hregion.majorcompaction not taking effect. Key: HBASE-7893 URL: https://issues.apache.org/jira/browse/HBASE-7893 Project: HBase Issue Type: Bug Components: Compaction Affects Versions: 0.92.2 Environment: Linux hostname 2.6.21.7-2.fc8xen-ec2-v1.0 #1 SMP Tue Sep 1 10:25:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux Fedora release 8 (Werewolf) Reporter: Pranav Sharma Attachments: hbase-hadoop-master-ip-10-118-138-149.log, hbase-hadoop-regionserver-ip-10-118-187-126.log For disabling auto major compaction, I configured hbase.hregion.majorcompaction = 0 in the config file and restarted the cluster. In spite of this, major compaction continues to run everyday. Here is the config I set: property namehbase.hregion.majorcompaction/name value0/value /property What other way can I disable auto major compaction? I expected this config to work. What am I doing wrong here? Please advice. Thanks a lot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7188: - Attachment: HBASE-7188-7.patch Fix the javadoc warning. Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Client, IPC/RPC Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch, HBASE-7188-5.patch, HBASE-7188-6.patch, HBASE-7188-7.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7893) hbase.hregion.majorcompaction not taking effect.
[ https://issues.apache.org/jira/browse/HBASE-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585791#comment-13585791 ] ramkrishna.s.vasudevan commented on HBASE-7893: --- Yes it should be GC problem. Lot of timeouts from HDFS side also. hbase.hregion.majorcompaction not taking effect. Key: HBASE-7893 URL: https://issues.apache.org/jira/browse/HBASE-7893 Project: HBase Issue Type: Bug Components: Compaction Affects Versions: 0.92.2 Environment: Linux hostname 2.6.21.7-2.fc8xen-ec2-v1.0 #1 SMP Tue Sep 1 10:25:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux Fedora release 8 (Werewolf) Reporter: Pranav Sharma Attachments: hbase-hadoop-master-ip-10-118-138-149.log, hbase-hadoop-regionserver-ip-10-118-187-126.log For disabling auto major compaction, I configured hbase.hregion.majorcompaction = 0 in the config file and restarted the cluster. In spite of this, major compaction continues to run everyday. Here is the config I set: property namehbase.hregion.majorcompaction/name value0/value /property What other way can I disable auto major compaction? I expected this config to work. What am I doing wrong here? Please advice. Thanks a lot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585798#comment-13585798 ] Hadoop QA commented on HBASE-7188: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570764/HBASE-7188-7.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 307 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestSnapshotFromAdmin Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4524//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4524//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4524//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4524//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4524//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4524//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4524//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4524//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4524//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4524//console This message is automatically generated. Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Client, IPC/RPC Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch, HBASE-7188-5.patch, HBASE-7188-6.patch, HBASE-7188-7.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7926) SmallTests pollute the META descriptor
[ https://issues.apache.org/jira/browse/HBASE-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-7926: --- Attachment: HBASE-7926-v0.patch SmallTests pollute the META descriptor -- Key: HBASE-7926 URL: https://issues.apache.org/jira/browse/HBASE-7926 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7926-v0.patch Running tests on my jenkins I've noticed that the META_TABLEDESC at some point gets changed, and gets SNAPPY encoding and other settings. A couple of SmallTest take the META_TABLEDESC as base and change it directly, to verify the serialization, without creating a copy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7926) SmallTests pollute the META descriptor
[ https://issues.apache.org/jira/browse/HBASE-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-7926: --- Status: Patch Available (was: Open) SmallTests pollute the META descriptor -- Key: HBASE-7926 URL: https://issues.apache.org/jira/browse/HBASE-7926 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7926-v0.patch Running tests on my jenkins I've noticed that the META_TABLEDESC at some point gets changed, and gets SNAPPY encoding and other settings. A couple of SmallTest take the META_TABLEDESC as base and change it directly, to verify the serialization, without creating a copy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7879) JUnit dependency in main from htrace
[ https://issues.apache.org/jira/browse/HBASE-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585807#comment-13585807 ] Hudson commented on HBASE-7879: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #419 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/419/]) HBASE-7879 JUnit dependency in main from htrace (Revision 1449614) Result = FAILURE nkeywal : Files : * /hbase/trunk/pom.xml JUnit dependency in main from htrace Key: HBASE-7879 URL: https://issues.apache.org/jira/browse/HBASE-7879 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 7879.v1.patch HTrace main depends on Junit , it should be only test. I created a junit in the github, that's https://github.com/cloudera/htrace/issues/1. If it's not fixed, we will be able to drop it in our pom, but let's wait a little before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7927) Bad classpath generation
nkeywal created HBASE-7927: -- Summary: Bad classpath generation Key: HBASE-7927 URL: https://issues.apache.org/jira/browse/HBASE-7927 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.96.0 Reporter: nkeywal I don't know why, but when you do a mvn dependency:tree, everything looks fine. When you look at the generated target/cached_classpath.txt you see 2 versions of netty: netty-3.2.4.Final.jar and netty-3.5.9.Final.jar. This is bad and can lead to unpredictable behavior. I haven't looked at the other dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7926) SmallTests pollute the META descriptor
[ https://issues.apache.org/jira/browse/HBASE-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585818#comment-13585818 ] Hadoop QA commented on HBASE-7926: -- {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570770/HBASE-7926-v0.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 6 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4525//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4525//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4525//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4525//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4525//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4525//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4525//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4525//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4525//console This message is automatically generated. SmallTests pollute the META descriptor -- Key: HBASE-7926 URL: https://issues.apache.org/jira/browse/HBASE-7926 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7926-v0.patch Running tests on my jenkins I've noticed that the META_TABLEDESC at some point gets changed, and gets SNAPPY encoding and other settings. A couple of SmallTest take the META_TABLEDESC as base and change it directly, to verify the serialization, without creating a copy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7845) optimize hfile index key
[ https://issues.apache.org/jira/browse/HBASE-7845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Xie updated HBASE-7845: - Summary: optimize hfile index key (was: optimize hfile index size like leveldb's ByteWiseComparatorImpl::FindShortestSeparator() style) optimize hfile index key Key: HBASE-7845 URL: https://issues.apache.org/jira/browse/HBASE-7845 Project: HBase Issue Type: Improvement Components: HFile Affects Versions: 0.96.0 Reporter: Liang Xie Assignee: Liang Xie Leveldb uses ByteWiseComparatorImpl::FindShortestSeparator() FindShortSuccessor() to reduce index key size, it would be helpful under special conditions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7845) optimize hfile index key
[ https://issues.apache.org/jira/browse/HBASE-7845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Xie updated HBASE-7845: - Attachment: HBASE-7845.txt optimize hfile index key Key: HBASE-7845 URL: https://issues.apache.org/jira/browse/HBASE-7845 Project: HBase Issue Type: Improvement Components: HFile Affects Versions: 0.96.0 Reporter: Liang Xie Assignee: Liang Xie Attachments: HBASE-7845.txt Leveldb uses ByteWiseComparatorImpl::FindShortestSeparator() FindShortSuccessor() to reduce index key size, it would be helpful under special conditions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7845) optimize hfile index key
[ https://issues.apache.org/jira/browse/HBASE-7845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Xie updated HBASE-7845: - Status: Patch Available (was: Open) optimize hfile index key Key: HBASE-7845 URL: https://issues.apache.org/jira/browse/HBASE-7845 Project: HBase Issue Type: Improvement Components: HFile Affects Versions: 0.96.0 Reporter: Liang Xie Assignee: Liang Xie Attachments: HBASE-7845.txt Leveldb uses ByteWiseComparatorImpl::FindShortestSeparator() FindShortSuccessor() to reduce index key size, it would be helpful under special conditions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7927) Bad classpath generation
[ https://issues.apache.org/jira/browse/HBASE-7927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585830#comment-13585830 ] Gustavo Anatoly commented on HBASE-7927: Hi, Nicolas. Try run {code}mvn clean install -DskipTests{code} and check again your cached_classpath.txt, on my file appear only one netty-3.5.9.Final.jar. If this behavior, persist you can try delete manually the dependency netty-3.2.4.Final.jar inside the maven repository and run {code}mvn clean install -DskipTests{code} to see where blow up the error, so we discover the main cause. Bad classpath generation Key: HBASE-7927 URL: https://issues.apache.org/jira/browse/HBASE-7927 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.96.0 Reporter: nkeywal I don't know why, but when you do a mvn dependency:tree, everything looks fine. When you look at the generated target/cached_classpath.txt you see 2 versions of netty: netty-3.2.4.Final.jar and netty-3.5.9.Final.jar. This is bad and can lead to unpredictable behavior. I haven't looked at the other dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7221) RowKey utility class for rowkey construction
[ https://issues.apache.org/jira/browse/HBASE-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585848#comment-13585848 ] Doug Meil commented on HBASE-7221: -- Lars? Is that reasonable? If somebody wants to use reverse-timestamp pattern (e.g., Long.MAX_VALUE - val) with this utility there is nothing stopping them. It just won't do it automatically (just like everybody else has to do now). I think this patch is ready, but it can still be extended in the future e.g., {code} builder.add(RowKeySchema.INT, descending) (code} ... but I'd rather not add half-implemented placeholders at this point. RowKey utility class for rowkey construction Key: HBASE-7221 URL: https://issues.apache.org/jira/browse/HBASE-7221 Project: HBase Issue Type: Improvement Components: util Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: HBASE_7221.patch, hbase-common_hbase_7221_2.patch, hbase-common_hbase_7221_v3.patch, hbase-common_hbase_7221_v4.patch, hbase-server_hbase_7221_v5.patch, hbase-server_hbase_7221_v6.patch A common question in the dist-lists is how to construct rowkeys, particularly composite keys. Put/Get/Scan specifies byte[] as the rowkey, but it's up to you to sensibly populate that byte-array, and that's where things tend to go off the rails. The intent of this RowKey utility class isn't meant to add functionality into Put/Get/Scan, but rather make it simpler for folks to construct said arrays. Example: {code} RowKey key = RowKey.create(RowKey.SIZEOF_MD5_HASH + RowKey.SIZEOF_LONG); key.addHash(a); key.add(b); byte bytes[] = key.getBytes(); {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log split work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585851#comment-13585851 ] Jean-Marc Spaggiari commented on HBASE-7824: Working for me too with mvn test -PlocalTests -Dtest=TestReplicationQueueFailoverCompressed Running org.apache.hadoop.hbase.replication.TestReplicationQueueFailoverCompressed Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 126.846 sec Improve master start up time when there is log split work - Key: HBASE-7824 URL: https://issues.apache.org/jira/browse/HBASE-7824 Project: HBase Issue Type: Bug Components: master Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.94.6 Attachments: hbase-7824.patch, hbase-7824_v2.patch When there is log split work going on, master start up waits till all log split work completes even though the log split has nothing to do with meta region servers. It's a bad behavior considering a master node can run when log split is happening while its start up is blocking by log split work. Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7845) optimize hfile index key
[ https://issues.apache.org/jira/browse/HBASE-7845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585855#comment-13585855 ] Hadoop QA commented on HBASE-7845: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570777/HBASE-7845.txt against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestHLog Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4526//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4526//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4526//console This message is automatically generated. optimize hfile index key Key: HBASE-7845 URL: https://issues.apache.org/jira/browse/HBASE-7845 Project: HBase Issue Type: Improvement Components: HFile Affects Versions: 0.96.0 Reporter: Liang Xie Assignee: Liang Xie Attachments: HBASE-7845.txt Leveldb uses ByteWiseComparatorImpl::FindShortestSeparator() FindShortSuccessor() to reduce index key size, it would be helpful under special conditions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7927) Bad classpath generation
[ https://issues.apache.org/jira/browse/HBASE-7927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585858#comment-13585858 ] nkeywal commented on HBASE-7927: Ok, thanks to you Gustavo I understand my error. The issue occurs only with -Dhadoop.profile=2.0, in this case we have twice netty. With hadoop 1 it's ok. I was generating the classpath with hadoop2 and checking the dependency with hadoop1. And it's actually something that already burnt me in the past in HBASE-7104. I'm gonna change the title of this jira to make this clear. Bad classpath generation Key: HBASE-7927 URL: https://issues.apache.org/jira/browse/HBASE-7927 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.96.0 Reporter: nkeywal I don't know why, but when you do a mvn dependency:tree, everything looks fine. When you look at the generated target/cached_classpath.txt you see 2 versions of netty: netty-3.2.4.Final.jar and netty-3.5.9.Final.jar. This is bad and can lead to unpredictable behavior. I haven't looked at the other dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7927) Two versions of netty with hadoop.profile=2.0: 3.5.9 and 3.2.4
[ https://issues.apache.org/jira/browse/HBASE-7927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-7927: --- Summary: Two versions of netty with hadoop.profile=2.0: 3.5.9 and 3.2.4 (was: Bad classpath generation) Two versions of netty with hadoop.profile=2.0: 3.5.9 and 3.2.4 -- Key: HBASE-7927 URL: https://issues.apache.org/jira/browse/HBASE-7927 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.96.0 Reporter: nkeywal I don't know why, but when you do a mvn dependency:tree, everything looks fine. When you look at the generated target/cached_classpath.txt you see 2 versions of netty: netty-3.2.4.Final.jar and netty-3.5.9.Final.jar. This is bad and can lead to unpredictable behavior. I haven't looked at the other dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5071) HFile has a possible cast issue.
[ https://issues.apache.org/jira/browse/HBASE-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585860#comment-13585860 ] Max Lapan commented on HBASE-5071: -- Add my notes on this bug. This could be helpful to somewone who as we are still use HFileV1. This bug was introduced by HBASE-3040 performance optimisation and cannot be fixed by Harsh's patch, which truncates index data (there are later problems rise on index parse). I fixed this issue in our installation by replacing readAllIndex whith BufferedInputStreams, which is transparent and have no index size limitations: https://github.com/Shmuma/hbase/commit/d0ef517482a0475588e229344558c31b47d5a269 HFile has a possible cast issue. Key: HBASE-5071 URL: https://issues.apache.org/jira/browse/HBASE-5071 Project: HBase Issue Type: Bug Components: HFile, io Affects Versions: 0.90.0 Reporter: Harsh J Labels: hfile Fix For: 0.96.0 HBASE-3040 introduced this line originally in HFile.Reader#loadFileInfo(...): {code} int allIndexSize = (int)(this.fileSize - this.trailer.dataIndexOffset - FixedFileTrailer.trailerSize()); {code} Which on trunk today, for HFile v1 is: {code} int sizeToLoadOnOpen = (int) (fileSize - trailer.getLoadOnOpenDataOffset() - trailer.getTrailerSize()); {code} This computed (and casted) integer is then used to build an array of the same size. But if fileSize is very large ( Integer.MAX_VALUE), then there's an easy chance this can go negative at some point and spew out exceptions such as: {code} java.lang.NegativeArraySizeException at org.apache.hadoop.hbase.io.hfile.HFile$Reader.readAllIndex(HFile.java:805) at org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:832) at org.apache.hadoop.hbase.regionserver.StoreFile$Reader.loadFileInfo(StoreFile.java:1003) at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:382) at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:438) at org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:267) at org.apache.hadoop.hbase.regionserver.Store.init(Store.java:209) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:2088) at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:358) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2661) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647) {code} Did we accidentally limit single region sizes this way? (Unsure about HFile v2's structure so far, so do not know if v2 has the same issue.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7927) Two versions of netty with hadoop.profile=2.0: 3.5.9 and 3.2.4
[ https://issues.apache.org/jira/browse/HBASE-7927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585864#comment-13585864 ] Gustavo Anatoly commented on HBASE-7927: I did: {code}mvn dependency:tree -Dhadoop.profile=2.0 ~/dependency.txt{code} and really appear two versions of netty. I never tried combine dependency:tree with maven profiles :) it works. Two versions of netty with hadoop.profile=2.0: 3.5.9 and 3.2.4 -- Key: HBASE-7927 URL: https://issues.apache.org/jira/browse/HBASE-7927 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.96.0 Reporter: nkeywal I don't know why, but when you do a mvn dependency:tree, everything looks fine. When you look at the generated target/cached_classpath.txt you see 2 versions of netty: netty-3.2.4.Final.jar and netty-3.5.9.Final.jar. This is bad and can lead to unpredictable behavior. I haven't looked at the other dependencies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted
[ https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-6469: -- Attachment: HBASE-6469_2.patch Patch addressing Ted's and Ram's comments. @Ted,@Ram Please review. Is it ok? Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted --- Key: HBASE-6469 URL: https://issues.apache.org/jira/browse/HBASE-6469 Project: HBase Issue Type: Bug Affects Versions: 0.94.2, 0.96.0 Reporter: Enis Soztutar Assignee: rajeshbabu Fix For: 0.96.0, 0.94.6 Attachments: 6469-expose-force-r3.patch, HBASE-6469_2.patch, HBASE-6469.patch In Enable/DisableTableHandler code, if something goes wrong in handling, the table state in zk is left as ENABLING / DISABLING. After that we cannot force any more action from the API or CLI, and the only recovery path is restarting the master. {code} if (done) { // Flip the table to enabled. this.assignmentManager.getZKTable().setEnabledTable( this.tableNameStr); LOG.info(Table ' + this.tableNameStr + ' was successfully enabled. Status: done= + done); } else { LOG.warn(Table ' + this.tableNameStr + ' wasn't successfully enabled. Status: done= + done); } {code} Here, if done is false, the table state is not changed. There is also no way to set skipTableStateCheck from cli / api. We have run into this issue a couple of times before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted
[ https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-6469: -- Status: Open (was: Patch Available) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted --- Key: HBASE-6469 URL: https://issues.apache.org/jira/browse/HBASE-6469 Project: HBase Issue Type: Bug Affects Versions: 0.94.2, 0.96.0 Reporter: Enis Soztutar Assignee: rajeshbabu Fix For: 0.96.0, 0.94.6 Attachments: 6469-expose-force-r3.patch, HBASE-6469_2.patch, HBASE-6469.patch In Enable/DisableTableHandler code, if something goes wrong in handling, the table state in zk is left as ENABLING / DISABLING. After that we cannot force any more action from the API or CLI, and the only recovery path is restarting the master. {code} if (done) { // Flip the table to enabled. this.assignmentManager.getZKTable().setEnabledTable( this.tableNameStr); LOG.info(Table ' + this.tableNameStr + ' was successfully enabled. Status: done= + done); } else { LOG.warn(Table ' + this.tableNameStr + ' wasn't successfully enabled. Status: done= + done); } {code} Here, if done is false, the table state is not changed. There is also no way to set skipTableStateCheck from cli / api. We have run into this issue a couple of times before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted
[ https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-6469: -- Status: Patch Available (was: Open) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted --- Key: HBASE-6469 URL: https://issues.apache.org/jira/browse/HBASE-6469 Project: HBase Issue Type: Bug Affects Versions: 0.94.2, 0.96.0 Reporter: Enis Soztutar Assignee: rajeshbabu Fix For: 0.96.0, 0.94.6 Attachments: 6469-expose-force-r3.patch, HBASE-6469_2.patch, HBASE-6469.patch In Enable/DisableTableHandler code, if something goes wrong in handling, the table state in zk is left as ENABLING / DISABLING. After that we cannot force any more action from the API or CLI, and the only recovery path is restarting the master. {code} if (done) { // Flip the table to enabled. this.assignmentManager.getZKTable().setEnabledTable( this.tableNameStr); LOG.info(Table ' + this.tableNameStr + ' was successfully enabled. Status: done= + done); } else { LOG.warn(Table ' + this.tableNameStr + ' wasn't successfully enabled. Status: done= + done); } {code} Here, if done is false, the table state is not changed. There is also no way to set skipTableStateCheck from cli / api. We have run into this issue a couple of times before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7807) Introduce HRegionFileSystem and move region fs related code
[ https://issues.apache.org/jira/browse/HBASE-7807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-7807: --- Attachment: HBASE-7807-v1.patch Introduce HRegionFileSystem and move region fs related code --- Key: HBASE-7807 URL: https://issues.apache.org/jira/browse/HBASE-7807 Project: HBase Issue Type: Sub-task Components: regionserver Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Attachments: HBASE-7807-v0.patch, HBASE-7807-v1.patch Add a new HRegionFileSystem class that allows to create an on-disk region without needs to instantiate an HRegion. This is the first refactor step, the next step is moving file creation and retrival inside this class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted
[ https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585941#comment-13585941 ] ramkrishna.s.vasudevan commented on HBASE-6469: --- @Rajesh bq.Enable/disable the tables again if something wrong happened in the middle Improve the comments for this class. Saw some formatting related issues. Will go thro once again tomorrow and let you know. Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted --- Key: HBASE-6469 URL: https://issues.apache.org/jira/browse/HBASE-6469 Project: HBase Issue Type: Bug Affects Versions: 0.94.2, 0.96.0 Reporter: Enis Soztutar Assignee: rajeshbabu Fix For: 0.96.0, 0.94.6 Attachments: 6469-expose-force-r3.patch, HBASE-6469_2.patch, HBASE-6469.patch In Enable/DisableTableHandler code, if something goes wrong in handling, the table state in zk is left as ENABLING / DISABLING. After that we cannot force any more action from the API or CLI, and the only recovery path is restarting the master. {code} if (done) { // Flip the table to enabled. this.assignmentManager.getZKTable().setEnabledTable( this.tableNameStr); LOG.info(Table ' + this.tableNameStr + ' was successfully enabled. Status: done= + done); } else { LOG.warn(Table ' + this.tableNameStr + ' wasn't successfully enabled. Status: done= + done); } {code} Here, if done is false, the table state is not changed. There is also no way to set skipTableStateCheck from cli / api. We have run into this issue a couple of times before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7928) Scanning .META. with startRow and/or stopRow is not giving proper results
Jean-Marc Spaggiari created HBASE-7928: -- Summary: Scanning .META. with startRow and/or stopRow is not giving proper results Key: HBASE-7928 URL: https://issues.apache.org/jira/browse/HBASE-7928 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari {code} try { HTable metaTable = new HTable(config, Bytes.toBytes(.META.)); Scan scan = new Scan(); scan.setStartRow(Bytes.toBytes(e)); scan.setStopRow(Bytes.toBytes(z)); ResultScanner scanner = metaTable.getScanner(scan); Result[] results = scanner.next(100); while (results.length 0) { for (Result result : results) { System.out.println(Bytes.toString(result.getRow())); } results = scanner.next(100); } scanner.close(); metaTable.close(); } catch (Exception e) { e.printStackTrace(); } {code} This code will not return any result even if there is 10 tables with names starting with d to w, including one table called entry. If you comment the setStopRow you will get results, but will still get rows starting with d even if setStartRow is set to e. Same code using with a user table is working fine. Facing the same issue with the shell. scan '.META.' , {STARTROW = 'e', LIMIT = 10} is returning rows starting by d. scan '.META.' , {STARTROW = 'e', STOPROW = 'v', LIMIT = 10} is not returning anything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted
[ https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585950#comment-13585950 ] Hadoop QA commented on HBASE-6469: -- {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570795/HBASE-6469_2.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4527//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4527//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4527//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4527//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4527//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4527//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4527//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4527//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4527//console This message is automatically generated. Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted --- Key: HBASE-6469 URL: https://issues.apache.org/jira/browse/HBASE-6469 Project: HBase Issue Type: Bug Affects Versions: 0.94.2, 0.96.0 Reporter: Enis Soztutar Assignee: rajeshbabu Fix For: 0.96.0, 0.94.6 Attachments: 6469-expose-force-r3.patch, HBASE-6469_2.patch, HBASE-6469.patch In Enable/DisableTableHandler code, if something goes wrong in handling, the table state in zk is left as ENABLING / DISABLING. After that we cannot force any more action from the API or CLI, and the only recovery path is restarting the master. {code} if (done) { // Flip the table to enabled. this.assignmentManager.getZKTable().setEnabledTable( this.tableNameStr); LOG.info(Table ' + this.tableNameStr + ' was successfully enabled. Status: done= + done); } else { LOG.warn(Table ' + this.tableNameStr + ' wasn't successfully enabled. Status: done= + done); } {code} Here, if done is false, the table state is not changed. There is also no way to set skipTableStateCheck from cli / api. We have run into this issue a couple of times before. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7928) Scanning .META. with startRow and/or stopRow is not giving proper results
[ https://issues.apache.org/jira/browse/HBASE-7928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585952#comment-13585952 ] ramkrishna.s.vasudevan commented on HBASE-7928: --- @JM Are you planning to work on this? If not can i take this up. Scanning .META. with startRow and/or stopRow is not giving proper results - Key: HBASE-7928 URL: https://issues.apache.org/jira/browse/HBASE-7928 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari {code} try { HTable metaTable = new HTable(config, Bytes.toBytes(.META.)); Scan scan = new Scan(); scan.setStartRow(Bytes.toBytes(e)); scan.setStopRow(Bytes.toBytes(z)); ResultScanner scanner = metaTable.getScanner(scan); Result[] results = scanner.next(100); while (results.length 0) { for (Result result : results) { System.out.println(Bytes.toString(result.getRow())); } results = scanner.next(100); } scanner.close(); metaTable.close(); } catch (Exception e) { e.printStackTrace(); } {code} This code will not return any result even if there is 10 tables with names starting with d to w, including one table called entry. If you comment the setStopRow you will get results, but will still get rows starting with d even if setStartRow is set to e. Same code using with a user table is working fine. Facing the same issue with the shell. scan '.META.' , {STARTROW = 'e', LIMIT = 10} is returning rows starting by d. scan '.META.' , {STARTROW = 'e', STOPROW = 'v', LIMIT = 10} is not returning anything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7928) Scanning .META. with startRow and/or stopRow is not giving proper results
[ https://issues.apache.org/jira/browse/HBASE-7928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585955#comment-13585955 ] stack commented on HBASE-7928: -- Is it just the .META. table that has this behavior? (.META. table is a bit odd because rows have a special format: tablename ',', regionname... it might because of this). Scanning .META. with startRow and/or stopRow is not giving proper results - Key: HBASE-7928 URL: https://issues.apache.org/jira/browse/HBASE-7928 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari {code} try { HTable metaTable = new HTable(config, Bytes.toBytes(.META.)); Scan scan = new Scan(); scan.setStartRow(Bytes.toBytes(e)); scan.setStopRow(Bytes.toBytes(z)); ResultScanner scanner = metaTable.getScanner(scan); Result[] results = scanner.next(100); while (results.length 0) { for (Result result : results) { System.out.println(Bytes.toString(result.getRow())); } results = scanner.next(100); } scanner.close(); metaTable.close(); } catch (Exception e) { e.printStackTrace(); } {code} This code will not return any result even if there is 10 tables with names starting with d to w, including one table called entry. If you comment the setStopRow you will get results, but will still get rows starting with d even if setStartRow is set to e. Same code using with a user table is working fine. Facing the same issue with the shell. scan '.META.' , {STARTROW = 'e', LIMIT = 10} is returning rows starting by d. scan '.META.' , {STARTROW = 'e', STOPROW = 'v', LIMIT = 10} is not returning anything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7926) SmallTests pollute the META descriptor
[ https://issues.apache.org/jira/browse/HBASE-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585962#comment-13585962 ] stack commented on HBASE-7926: -- +1 SmallTests pollute the META descriptor -- Key: HBASE-7926 URL: https://issues.apache.org/jira/browse/HBASE-7926 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7926-v0.patch Running tests on my jenkins I've noticed that the META_TABLEDESC at some point gets changed, and gets SNAPPY encoding and other settings. A couple of SmallTest take the META_TABLEDESC as base and change it directly, to verify the serialization, without creating a copy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7928) Scanning .META. with startRow and/or stopRow is not giving proper results
[ https://issues.apache.org/jira/browse/HBASE-7928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585967#comment-13585967 ] Jean-Marc Spaggiari commented on HBASE-7928: [~ram_krish] No, you can take it. Thanks. I'm focusing on HBCK for now. [~saint@gmail.com] Just the .META. for user table it's working fine. Scanning .META. with startRow and/or stopRow is not giving proper results - Key: HBASE-7928 URL: https://issues.apache.org/jira/browse/HBASE-7928 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari {code} try { HTable metaTable = new HTable(config, Bytes.toBytes(.META.)); Scan scan = new Scan(); scan.setStartRow(Bytes.toBytes(e)); scan.setStopRow(Bytes.toBytes(z)); ResultScanner scanner = metaTable.getScanner(scan); Result[] results = scanner.next(100); while (results.length 0) { for (Result result : results) { System.out.println(Bytes.toString(result.getRow())); } results = scanner.next(100); } scanner.close(); metaTable.close(); } catch (Exception e) { e.printStackTrace(); } {code} This code will not return any result even if there is 10 tables with names starting with d to w, including one table called entry. If you comment the setStopRow you will get results, but will still get rows starting with d even if setStartRow is set to e. Same code using with a user table is working fine. Facing the same issue with the shell. scan '.META.' , {STARTROW = 'e', LIMIT = 10} is returning rows starting by d. scan '.META.' , {STARTROW = 'e', STOPROW = 'v', LIMIT = 10} is not returning anything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7928) Scanning .META. with startRow and/or stopRow is not giving proper results
[ https://issues.apache.org/jira/browse/HBASE-7928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585971#comment-13585971 ] Jean-Marc Spaggiari commented on HBASE-7928: It seems it's only impacting .META. (and maybe a bit -ROOT- too...) {code} try { HTable metaTable = new HTable(config, Bytes.toBytes(-ROOT-)); Scan scan = new Scan(); scan.setStartRow(new byte[]{0}); //scan.setStopRow(new byte[]{127}); ResultScanner scanner = metaTable.getScanner(scan); Result[] results = scanner.next(100); while (results.length 0) { for (Result result : results) { System.out.println(Bytes.toString(result.getRow())); } results = scanner.next(100); } scanner.close(); metaTable.close(); } catch (Exception e) { e.printStackTrace(); } {code} This is working for for ROOT, but not if I setup the stoprow. Scanning .META. with startRow and/or stopRow is not giving proper results - Key: HBASE-7928 URL: https://issues.apache.org/jira/browse/HBASE-7928 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari {code} try { HTable metaTable = new HTable(config, Bytes.toBytes(.META.)); Scan scan = new Scan(); scan.setStartRow(Bytes.toBytes(e)); scan.setStopRow(Bytes.toBytes(z)); ResultScanner scanner = metaTable.getScanner(scan); Result[] results = scanner.next(100); while (results.length 0) { for (Result result : results) { System.out.println(Bytes.toString(result.getRow())); } results = scanner.next(100); } scanner.close(); metaTable.close(); } catch (Exception e) { e.printStackTrace(); } {code} This code will not return any result even if there is 10 tables with names starting with d to w, including one table called entry. If you comment the setStopRow you will get results, but will still get rows starting with d even if setStartRow is set to e. Same code using with a user table is working fine. Facing the same issue with the shell. scan '.META.' , {STARTROW = 'e', LIMIT = 10} is returning rows starting by d. scan '.META.' , {STARTROW = 'e', STOPROW = 'v', LIMIT = 10} is not returning anything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7928) Scanning .META. with startRow and/or stopRow is not giving proper results
[ https://issues.apache.org/jira/browse/HBASE-7928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan reassigned HBASE-7928: - Assignee: ramkrishna.s.vasudevan Scanning .META. with startRow and/or stopRow is not giving proper results - Key: HBASE-7928 URL: https://issues.apache.org/jira/browse/HBASE-7928 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari Assignee: ramkrishna.s.vasudevan {code} try { HTable metaTable = new HTable(config, Bytes.toBytes(.META.)); Scan scan = new Scan(); scan.setStartRow(Bytes.toBytes(e)); scan.setStopRow(Bytes.toBytes(z)); ResultScanner scanner = metaTable.getScanner(scan); Result[] results = scanner.next(100); while (results.length 0) { for (Result result : results) { System.out.println(Bytes.toString(result.getRow())); } results = scanner.next(100); } scanner.close(); metaTable.close(); } catch (Exception e) { e.printStackTrace(); } {code} This code will not return any result even if there is 10 tables with names starting with d to w, including one table called entry. If you comment the setStopRow you will get results, but will still get rows starting with d even if setStartRow is set to e. Same code using with a user table is working fine. Facing the same issue with the shell. scan '.META.' , {STARTROW = 'e', LIMIT = 10} is returning rows starting by d. scan '.META.' , {STARTROW = 'e', STOPROW = 'v', LIMIT = 10} is not returning anything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7928) Scanning .META. with startRow and/or stopRow is not giving proper results
[ https://issues.apache.org/jira/browse/HBASE-7928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585973#comment-13585973 ] ramkrishna.s.vasudevan commented on HBASE-7928: --- YA.. Just verified. Let me take a look. Scanning .META. with startRow and/or stopRow is not giving proper results - Key: HBASE-7928 URL: https://issues.apache.org/jira/browse/HBASE-7928 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari Assignee: ramkrishna.s.vasudevan {code} try { HTable metaTable = new HTable(config, Bytes.toBytes(.META.)); Scan scan = new Scan(); scan.setStartRow(Bytes.toBytes(e)); scan.setStopRow(Bytes.toBytes(z)); ResultScanner scanner = metaTable.getScanner(scan); Result[] results = scanner.next(100); while (results.length 0) { for (Result result : results) { System.out.println(Bytes.toString(result.getRow())); } results = scanner.next(100); } scanner.close(); metaTable.close(); } catch (Exception e) { e.printStackTrace(); } {code} This code will not return any result even if there is 10 tables with names starting with d to w, including one table called entry. If you comment the setStopRow you will get results, but will still get rows starting with d even if setStartRow is set to e. Same code using with a user table is working fine. Facing the same issue with the shell. scan '.META.' , {STARTROW = 'e', LIMIT = 10} is returning rows starting by d. scan '.META.' , {STARTROW = 'e', STOPROW = 'v', LIMIT = 10} is not returning anything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7928) Scanning .META. with startRow and/or stopRow is not giving proper results
[ https://issues.apache.org/jira/browse/HBASE-7928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585977#comment-13585977 ] Kevin Odell commented on HBASE-7928: hbase(main):003:0 create 'test2', 'cf1', {SPLITS = ['111', '222', '333']} 0 row(s) in 1.1520 seconds hbase(main):005:0 put 'test2', '111', 'cf1:1', '' 0 row(s) in 0.1180 seconds hbase(main):006:0 put 'test2', '111', 'cf1:1', '1112' 0 row(s) in 0.0080 seconds hbase(main):007:0 put 'test2', '112', 'cf1:1', '1112' 0 row(s) in 0.0070 seconds hbase(main):008:0 put 'test2', '113', 'cf1:1', '1112' 0 row(s) in 0.0070 seconds hbase(main):009:0 put 'test2', '211', 'cf1:1', '1112' 0 row(s) in 0.0070 seconds hbase(main):010:0 put 'test2', '212', 'cf1:1', '1112' 0 row(s) in 0.0630 seconds hbase(main):011:0 put 'test2', '213', 'cf1:1', '1112' 0 row(s) in 0.0070 seconds hbase(main):012:0 put 'test2', '311', 'cf1:1', '1112' 0 row(s) in 0.0150 seconds hbase(main):013:0 put 'test2', '312', 'cf1:1', '1112' 0 row(s) in 0.0090 seconds hbase(main):014:0 put 'test2', '313', 'cf1:1', '1112' 0 row(s) in 0.0060 seconds hbase(main):020:0 scan 'test2', {STARTROW = '1', LIMIT = 2} ROW COLUMN+CELL 111 column=cf1:1, timestamp=1361809376535, value=1112 112 column=cf1:1, timestamp=1361809383138, value=1112 hbase(main):021:0 scan 'test2', {STARTROW = '2', LIMIT = 2} ROW COLUMN+CELL 211 column=cf1:1, timestamp=1361809393858, value=1112 212 column=cf1:1, timestamp=1361809398016, value=1112 hbase(main):029:0 scan '.META.', {STARTROW = '111', LIMIT = 2} ROW COLUMN+CELL brian,,1360685707510.81752ad34514d0d73735574fcb5280d column=info:regioninfo, timestamp=1360685707961, value={NAME = 'brian,,1360685707510.81752ad34514d0d73735574fcb5280d7.', STARTKEY = '', ENDKEY = '', ENC 7. ODED = 81752ad34514d0d73735574fcb5280d7,} brian,,1360685707510.81752ad34514d0d73735574fcb5280d column=info:server, timestamp=1360685708115, value=cdh4-hbase-02:60020 7. brian,,1360685707510.81752ad34514d0d73735574fcb5280d column=info:serverstartcode, timestamp=1360685708115, value=1357233940879 7. sent_email_su_lapse1d,,1360634184398.44fee2ed9f35ece column=info:regioninfo, timestamp=1360634184882, value={NAME = 'sent_email_su_lapse1d,,1360634184398.44fee2ed9f35ece213df06ac6f69a908.', STARTKEY = '', E 213df06ac6f69a908. .META. appears to be ignoring any of the extra flags. Scanning .META. with startRow and/or stopRow is not giving proper results - Key: HBASE-7928 URL: https://issues.apache.org/jira/browse/HBASE-7928 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari Assignee: ramkrishna.s.vasudevan {code} try { HTable metaTable = new HTable(config, Bytes.toBytes(.META.)); Scan scan = new Scan(); scan.setStartRow(Bytes.toBytes(e)); scan.setStopRow(Bytes.toBytes(z)); ResultScanner scanner = metaTable.getScanner(scan); Result[] results = scanner.next(100); while (results.length 0) {
[jira] [Commented] (HBASE-7807) Introduce HRegionFileSystem and move region fs related code
[ https://issues.apache.org/jira/browse/HBASE-7807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585979#comment-13585979 ] Hadoop QA commented on HBASE-7807: -- {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570803/HBASE-7807-v1.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 24 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4528//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4528//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4528//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4528//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4528//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4528//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4528//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4528//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4528//console This message is automatically generated. Introduce HRegionFileSystem and move region fs related code --- Key: HBASE-7807 URL: https://issues.apache.org/jira/browse/HBASE-7807 Project: HBase Issue Type: Sub-task Components: regionserver Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Attachments: HBASE-7807-v0.patch, HBASE-7807-v1.patch Add a new HRegionFileSystem class that allows to create an on-disk region without needs to instantiate an HRegion. This is the first refactor step, the next step is moving file creation and retrival inside this class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7507) Make memstore flush be able to retry after exception
[ https://issues.apache.org/jira/browse/HBASE-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585985#comment-13585985 ] stack commented on HBASE-7507: -- Regards the 0.94 patch, why are these statics? + private static final int DEFAULT_FLUSH_RETRIES_NUMBER = 10; + private static int flush_retries_number; + private static int pauseTime; Patch looks fine. Pity as said already that we have to localize the retry and rather we can't put all the retries behind a filesystem facade; we can do this for 0.96 I wonder what happens retrying after an IOE.Is it ok retrying any IOE? Has the flush path been reviewed to make sure only IOE is failed NN op? Is it possible to get an IOE after the file has been successfully written? Just wondering. Would say commit -- because helps us work with HA NN (HANN). Make memstore flush be able to retry after exception Key: HBASE-7507 URL: https://issues.apache.org/jira/browse/HBASE-7507 Project: HBase Issue Type: Bug Affects Versions: 0.94.3 Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.96.0 Attachments: 7507-94.patch, 7507-trunk v1.patch, 7507-trunk v2.patch, 7507-trunkv3.patch We will abort regionserver if memstore flush throws exception. I thinks we could do retry to make regionserver more stable because file system may be not ok in a transient time. e.g. Switching namenode in the NamenodeHA environment {code} HRegion#internalFlushcache(){ ... try { ... }catch(Throwable t){ DroppedSnapshotException dse = new DroppedSnapshotException(region: + Bytes.toStringBinary(getRegionName())); dse.initCause(t); throw dse; } ... } MemStoreFlusher#flushRegion(){ ... region.flushcache(); ... try { }catch(DroppedSnapshotException ex){ server.abort(Replay of HLog required. Forcing server shutdown, ex); } ... } {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7507) Make memstore flush be able to retry after exception
[ https://issues.apache.org/jira/browse/HBASE-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585987#comment-13585987 ] stack commented on HBASE-7507: -- Oh, I'll commit in a while unless I am beaten to it. Make memstore flush be able to retry after exception Key: HBASE-7507 URL: https://issues.apache.org/jira/browse/HBASE-7507 Project: HBase Issue Type: Bug Affects Versions: 0.94.3 Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.96.0 Attachments: 7507-94.patch, 7507-trunk v1.patch, 7507-trunk v2.patch, 7507-trunkv3.patch We will abort regionserver if memstore flush throws exception. I thinks we could do retry to make regionserver more stable because file system may be not ok in a transient time. e.g. Switching namenode in the NamenodeHA environment {code} HRegion#internalFlushcache(){ ... try { ... }catch(Throwable t){ DroppedSnapshotException dse = new DroppedSnapshotException(region: + Bytes.toStringBinary(getRegionName())); dse.initCause(t); throw dse; } ... } MemStoreFlusher#flushRegion(){ ... region.flushcache(); ... try { }catch(DroppedSnapshotException ex){ server.abort(Replay of HLog required. Forcing server shutdown, ex); } ... } {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7920) Move isFamilyEssential(byte[] name) out of Filter interface in 0.94
[ https://issues.apache.org/jira/browse/HBASE-7920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7920: -- Hadoop Flags: Reviewed Status: Patch Available (was: In Progress) Move isFamilyEssential(byte[] name) out of Filter interface in 0.94 --- Key: HBASE-7920 URL: https://issues.apache.org/jira/browse/HBASE-7920 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.94.6 Attachments: 7920-94.txt, 7920-94-v2.txt As Dave Latham pointed out here: https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=13584553page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13584553 The addition of isFamilyEssential(byte[] name) in Filter interface in 0.94.5 broke backward compatibility. Agreement in subsequent discussions on HBASE-5416 was reached: isFamilyEssential(byte[] name) should be lifted from Filter interface into FilterBase class. This would allow 0.94.6 to be compatible with 0.94.1 to 0.94.4 releases. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HBASE-7920) Move isFamilyEssential(byte[] name) out of Filter interface in 0.94
[ https://issues.apache.org/jira/browse/HBASE-7920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-7920 started by Ted Yu. Move isFamilyEssential(byte[] name) out of Filter interface in 0.94 --- Key: HBASE-7920 URL: https://issues.apache.org/jira/browse/HBASE-7920 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.94.6 Attachments: 7920-94.txt, 7920-94-v2.txt As Dave Latham pointed out here: https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=13584553page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13584553 The addition of isFamilyEssential(byte[] name) in Filter interface in 0.94.5 broke backward compatibility. Agreement in subsequent discussions on HBASE-5416 was reached: isFamilyEssential(byte[] name) should be lifted from Filter interface into FilterBase class. This would allow 0.94.6 to be compatible with 0.94.1 to 0.94.4 releases. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7920) Move isFamilyEssential(byte[] name) out of Filter interface in 0.94
[ https://issues.apache.org/jira/browse/HBASE-7920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7920: -- Resolution: Fixed Status: Resolved (was: Patch Available) Move isFamilyEssential(byte[] name) out of Filter interface in 0.94 --- Key: HBASE-7920 URL: https://issues.apache.org/jira/browse/HBASE-7920 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.94.6 Attachments: 7920-94.txt, 7920-94-v2.txt As Dave Latham pointed out here: https://issues.apache.org/jira/browse/HBASE-5416?focusedCommentId=13584553page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13584553 The addition of isFamilyEssential(byte[] name) in Filter interface in 0.94.5 broke backward compatibility. Agreement in subsequent discussions on HBASE-5416 was reached: isFamilyEssential(byte[] name) should be lifted from Filter interface into FilterBase class. This would allow 0.94.6 to be compatible with 0.94.1 to 0.94.4 releases. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7929) Reapply hbase-7507 Make memstore flush be able to retry after exception to 0.94 branch.
stack created HBASE-7929: Summary: Reapply hbase-7507 Make memstore flush be able to retry after exception to 0.94 branch. Key: HBASE-7929 URL: https://issues.apache.org/jira/browse/HBASE-7929 Project: HBase Issue Type: Task Reporter: stack It was applied once then backed out because it seemed like it could be part responsible for destabilizing unit tests. Thinking is different now. Retrying application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7929) Reapply hbase-7507 Make memstore flush be able to retry after exception to 0.94 branch.
[ https://issues.apache.org/jira/browse/HBASE-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-7929. -- Resolution: Fixed Fix Version/s: 0.94.6 Assignee: stack Applied to 0.94 branch. Reapply hbase-7507 Make memstore flush be able to retry after exception to 0.94 branch. - Key: HBASE-7929 URL: https://issues.apache.org/jira/browse/HBASE-7929 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Fix For: 0.94.6 It was applied once then backed out because it seemed like it could be part responsible for destabilizing unit tests. Thinking is different now. Retrying application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7930) hbck should provide an option to fix empty REGIONINFO_QUALIFIER .META. rows.
Jean-Marc Spaggiari created HBASE-7930: -- Summary: hbck should provide an option to fix empty REGIONINFO_QUALIFIER .META. rows. Key: HBASE-7930 URL: https://issues.apache.org/jira/browse/HBASE-7930 Project: HBase Issue Type: Improvement Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Today when master and HBCK are reporting empty REGIONINFO_QUALIFIER .META. rows, we need to manually delete them from the .META. region. We need to enhance hbck to do that automatically. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7507) Make memstore flush be able to retry after exception
[ https://issues.apache.org/jira/browse/HBASE-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586001#comment-13586001 ] stack commented on HBASE-7507: -- Done. Hopefully it doesn't mess up the 0.94 tests. Flag me and I'll remove it again. Make memstore flush be able to retry after exception Key: HBASE-7507 URL: https://issues.apache.org/jira/browse/HBASE-7507 Project: HBase Issue Type: Bug Affects Versions: 0.94.3 Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.96.0 Attachments: 7507-94.patch, 7507-trunk v1.patch, 7507-trunk v2.patch, 7507-trunkv3.patch We will abort regionserver if memstore flush throws exception. I thinks we could do retry to make regionserver more stable because file system may be not ok in a transient time. e.g. Switching namenode in the NamenodeHA environment {code} HRegion#internalFlushcache(){ ... try { ... }catch(Throwable t){ DroppedSnapshotException dse = new DroppedSnapshotException(region: + Bytes.toStringBinary(getRegionName())); dse.initCause(t); throw dse; } ... } MemStoreFlusher#flushRegion(){ ... region.flushcache(); ... try { }catch(DroppedSnapshotException ex){ server.abort(Replay of HLog required. Forcing server shutdown, ex); } ... } {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6861) HFileOutputFormat set TIMERANGE wrongly
[ https://issues.apache.org/jira/browse/HBASE-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6861. -- Resolution: Duplicate Marking as duplicate of hbase-3776 as per Anoop HFileOutputFormat set TIMERANGE wrongly --- Key: HBASE-6861 URL: https://issues.apache.org/jira/browse/HBASE-6861 Project: HBase Issue Type: Bug Reporter: Eugene Morozov Priority: Minor Attachments: diff In case if timestamps for KeyValues specified differently for different column families, then TIMERANGEs of both HFiles would be wrong. Example (in pseudo code): my reducer has a condition if ( condition ) { keyValue = new KeyValue(.., CF1, .., timestamp, ..); } else { keyValue = new KeyValue(.., CF2, .., ..); // - no timestamp } context.write( keyValue ); These two keyValues would be written into two different HFiles. But the code, which is actually write do the following: // we now have the proper HLog writer. full steam ahead kv.updateLatestStamp(this.now); trt.includeTimestamp(kv); wl.writer.append(kv); Basically, two HFiles shares the same instance of trt (TimeRangeTracker), which leads to the same TIMERANGEs of both of them. Which is definitely incorrect, because first HFile must have TIMERANGE=timestamp...timestamp, cause we do not write any other timestamps there. And another HFile must have TIMERANGE=now...now by same meaning. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7928) Scanning .META. with startRow and/or stopRow is not giving proper results
[ https://issues.apache.org/jira/browse/HBASE-7928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586007#comment-13586007 ] ramkrishna.s.vasudevan commented on HBASE-7928: --- So the META_COMPARATOR is not able to find the row that starts with the start row specified. We try to create a DELETE_FAMILY row with the specified row. So that the scan starts from there and all other rows are less than that. In case of META it does not happen. So let me come up with a test case tomorrow. Scanning .META. with startRow and/or stopRow is not giving proper results - Key: HBASE-7928 URL: https://issues.apache.org/jira/browse/HBASE-7928 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Jean-Marc Spaggiari Assignee: ramkrishna.s.vasudevan {code} try { HTable metaTable = new HTable(config, Bytes.toBytes(.META.)); Scan scan = new Scan(); scan.setStartRow(Bytes.toBytes(e)); scan.setStopRow(Bytes.toBytes(z)); ResultScanner scanner = metaTable.getScanner(scan); Result[] results = scanner.next(100); while (results.length 0) { for (Result result : results) { System.out.println(Bytes.toString(result.getRow())); } results = scanner.next(100); } scanner.close(); metaTable.close(); } catch (Exception e) { e.printStackTrace(); } {code} This code will not return any result even if there is 10 tables with names starting with d to w, including one table called entry. If you comment the setStopRow you will get results, but will still get rows starting with d even if setStartRow is set to e. Same code using with a user table is working fine. Facing the same issue with the shell. scan '.META.' , {STARTROW = 'e', LIMIT = 10} is returning rows starting by d. scan '.META.' , {STARTROW = 'e', STOPROW = 'v', LIMIT = 10} is not returning anything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586023#comment-13586023 ] Ted Yu commented on HBASE-4210: --- [~apurtell]: What do you think ? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_Trunk.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-4210: -- Status: Open (was: Patch Available) Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_Trunk.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-4210: -- Attachment: HBASE-4210_94-V4.patch Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_Trunk.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-4210: -- Status: Patch Available (was: Open) Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-4210: -- Attachment: HBASE-4210_Trunk-V2.patch Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7188: - Component/s: snapshots Replication Admin Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Admin, Client, IPC/RPC, Replication, snapshots Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch, HBASE-7188-5.patch, HBASE-7188-6.patch, HBASE-7188-7.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7915) Secure ThriftServer needs to login before calling HBaseHandler
[ https://issues.apache.org/jira/browse/HBASE-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586035#comment-13586035 ] Gary Helmling commented on HBASE-7915: -- +1 on the patch. This should work. The underlying problem is that establish a network connection in a chained series of constructors (ThriftServerRunner - ThriftServerRunner$HBaseHandler - HBaseAdmin), which seems like something we should avoid. But HBaseAdmin is the real culprit, so any change there should really be a separate JIRA. Secure ThriftServer needs to login before calling HBaseHandler -- Key: HBASE-7915 URL: https://issues.apache.org/jira/browse/HBASE-7915 Project: HBase Issue Type: Bug Components: security, Thrift Affects Versions: 0.96.0, 0.94.5 Reporter: Arpit Gupta Assignee: Arpit Gupta Fix For: 0.96.0, 0.94.6 Attachments: HBASE-7915.branch-0.94.patch in ThriftServer.java the following call is made {code} serverRunner = new ThriftServerRunner(conf); {code} which invokes {code} public ThriftServerRunner(Configuration conf) throws IOException { this(conf, new ThriftServerRunner.HBaseHandler(conf)); } {code} All of this is happening before the user has logged in and fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7692: Status: Open (was: Patch Available) Add utility class to generate ordered byte[] serialization -- Key: HBASE-7692 URL: https://issues.apache.org/jira/browse/HBASE-7692 Project: HBase Issue Type: Improvement Components: util Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch The current Bytes utility class works, but produces output that does not maintain the native sort ordering of the input value. This results in, for example, a negative value that does not necessarily sort before a positive value. HBase should provide a canonical implementation of such a serialization format so that third-parties can reliably build on top of HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7692: Status: Patch Available (was: Open) Add utility class to generate ordered byte[] serialization -- Key: HBASE-7692 URL: https://issues.apache.org/jira/browse/HBASE-7692 Project: HBase Issue Type: Improvement Components: util Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch The current Bytes utility class works, but produces output that does not maintain the native sort ordering of the input value. This results in, for example, a negative value that does not necessarily sort before a positive value. HBase should provide a canonical implementation of such a serialization format so that third-parties can reliably build on top of HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7692: Attachment: HBASE-7692.v5.patch The git mirror caught up with svn over the weekend. Separate module patch updated accordingly. Add utility class to generate ordered byte[] serialization -- Key: HBASE-7692 URL: https://issues.apache.org/jira/browse/HBASE-7692 Project: HBase Issue Type: Improvement Components: util Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch The current Bytes utility class works, but produces output that does not maintain the native sort ordering of the input value. This results in, for example, a negative value that does not necessarily sort before a positive value. HBase should provide a canonical implementation of such a serialization format so that third-parties can reliably build on top of HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6222) Add per-KeyValue Security
[ https://issues.apache.org/jira/browse/HBASE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586041#comment-13586041 ] Ted Yu commented on HBASE-6222: --- [~apurtell]: The attached document is interesting. Do you have performance comparison for writes ? Thanks Add per-KeyValue Security - Key: HBASE-6222 URL: https://issues.apache.org/jira/browse/HBASE-6222 Project: HBase Issue Type: New Feature Components: security Affects Versions: 0.96.0, 0.98.0 Reporter: stack Assignee: Andrew Purtell Attachments: 6222-aclcf.patch, 6222.pdf, cell-acls-kv-tags-not-for-review.zip, HBaseCellRow-LevelSecurityDesignDoc.docx, HBaseCellRow-LevelSecurityPRD.docx Saw an interesting article: http://www.fiercegovernmentit.com/story/sasc-accumulo-language-pro-open-source-say-proponents/2012-06-14 The Senate Armed Services Committee version of the fiscal 2013 national defense authorization act (S. 3254) would require DoD agencies to foreswear the Accumulo NoSQL database after Sept. 30, 2013, unless the DoD CIO certifies that there exists either no viable commercial open source database with security features comparable to [Accumulo] (such as the HBase or Cassandra databases)... Not sure what a 'commercial open source database' is, and I'm not sure whats going on in the article, but tra-la-la'ing, if we had per-KeyValue 'security' like Accumulo's, we might put ourselves in the running for federal contributions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7930) hbck should provide an option to fix empty REGIONINFO_QUALIFIER .META. rows.
[ https://issues.apache.org/jira/browse/HBASE-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586047#comment-13586047 ] Dave Latham commented on HBASE-7930: Are there things that could generate this WARNing where the fix is not deleting the rows, but rather trying to repair them? For example, if there is still corresponding data on HDFS? hbck should provide an option to fix empty REGIONINFO_QUALIFIER .META. rows. Key: HBASE-7930 URL: https://issues.apache.org/jira/browse/HBASE-7930 Project: HBase Issue Type: Improvement Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Today when master and HBCK are reporting empty REGIONINFO_QUALIFIER .META. rows, we need to manually delete them from the .META. region. We need to enhance hbck to do that automatically. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7915) Secure ThriftServer needs to login before calling HBaseHandler
[ https://issues.apache.org/jira/browse/HBASE-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Gupta updated HBASE-7915: --- Attachment: HBASE-7915.patch Thanks Gary I have attached a similar patch for trunk Secure ThriftServer needs to login before calling HBaseHandler -- Key: HBASE-7915 URL: https://issues.apache.org/jira/browse/HBASE-7915 Project: HBase Issue Type: Bug Components: security, Thrift Affects Versions: 0.96.0, 0.94.5 Reporter: Arpit Gupta Assignee: Arpit Gupta Fix For: 0.96.0, 0.94.6 Attachments: HBASE-7915.branch-0.94.patch, HBASE-7915.patch in ThriftServer.java the following call is made {code} serverRunner = new ThriftServerRunner(conf); {code} which invokes {code} public ThriftServerRunner(Configuration conf) throws IOException { this(conf, new ThriftServerRunner.HBaseHandler(conf)); } {code} All of this is happening before the user has logged in and fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7915) Secure ThriftServer needs to login before calling HBaseHandler
[ https://issues.apache.org/jira/browse/HBASE-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Gupta updated HBASE-7915: --- Status: Patch Available (was: Open) Secure ThriftServer needs to login before calling HBaseHandler -- Key: HBASE-7915 URL: https://issues.apache.org/jira/browse/HBASE-7915 Project: HBase Issue Type: Bug Components: security, Thrift Affects Versions: 0.94.5, 0.96.0 Reporter: Arpit Gupta Assignee: Arpit Gupta Fix For: 0.96.0, 0.94.6 Attachments: HBASE-7915.branch-0.94.patch, HBASE-7915.patch in ThriftServer.java the following call is made {code} serverRunner = new ThriftServerRunner(conf); {code} which invokes {code} public ThriftServerRunner(Configuration conf) throws IOException { this(conf, new ThriftServerRunner.HBaseHandler(conf)); } {code} All of this is happening before the user has logged in and fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7878) recoverFileLease does not check return value of recoverLease
[ https://issues.apache.org/jira/browse/HBASE-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586049#comment-13586049 ] Eric Newton commented on HBASE-7878: Ted asked me to comment a bit further on the test that found this problem. We (the accumulo team) use our Continuous Ingest test, along with a script that randomly kills services, to verify that accumulo doesn't lose data during recovery. We had ported this test to HBase in the past by using Gora. We found an unrelated data loss a while back and posted it under HBASE-5754. The details about the test can be found in the github project [Goraci|https://github.com/keith-turner/goraci]. In this case, we found the data loss in Accumulo and tracked it down to the lease recovery. Since we used the same approach as HBase, I created this ticket. recoverFileLease does not check return value of recoverLease Key: HBASE-7878 URL: https://issues.apache.org/jira/browse/HBASE-7878 Project: HBase Issue Type: Bug Components: util Reporter: Eric Newton Assignee: Ted Yu Priority: Critical Fix For: 0.96.0, 0.94.6 Attachments: 7878-trunk-v2.txt, 7878-trunk-v3.txt I think this is a problem, so I'm opening a ticket so an HBase person takes a look. Apache Accumulo has moved its write-ahead log to HDFS. I modeled the lease recovery for Accumulo after HBase's lease recovery. During testing, we experienced data loss. I found it is necessary to wait until recoverLease returns true to know that the file has been truly closed. In FSHDFSUtils, the return result of recoverLease is not checked. In the unit tests created to check lease recovery in HBASE-2645, the return result of recoverLease is always checked. I think FSHDFSUtils should be modified to check the return result, and wait until it returns true. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log split work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586053#comment-13586053 ] Ted Yu commented on HBASE-7824: --- [~lhofhansl]: Do you want to take another look ? Thanks Improve master start up time when there is log split work - Key: HBASE-7824 URL: https://issues.apache.org/jira/browse/HBASE-7824 Project: HBase Issue Type: Bug Components: master Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.94.6 Attachments: hbase-7824.patch, hbase-7824_v2.patch When there is log split work going on, master start up waits till all log split work completes even though the log split has nothing to do with meta region servers. It's a bad behavior considering a master node can run when log split is happening while its start up is blocking by log split work. Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7926) SmallTests pollute the META descriptor
[ https://issues.apache.org/jira/browse/HBASE-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-7926: --- Resolution: Fixed Status: Resolved (was: Patch Available) committed to trunk, thanks for the review SmallTests pollute the META descriptor -- Key: HBASE-7926 URL: https://issues.apache.org/jira/browse/HBASE-7926 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7926-v0.patch Running tests on my jenkins I've noticed that the META_TABLEDESC at some point gets changed, and gets SNAPPY encoding and other settings. A couple of SmallTest take the META_TABLEDESC as base and change it directly, to verify the serialization, without creating a copy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7915) Secure ThriftServer needs to login before calling HBaseHandler
[ https://issues.apache.org/jira/browse/HBASE-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Gupta updated HBASE-7915: --- Attachment: HBASE-7915.patch added missing imports Secure ThriftServer needs to login before calling HBaseHandler -- Key: HBASE-7915 URL: https://issues.apache.org/jira/browse/HBASE-7915 Project: HBase Issue Type: Bug Components: security, Thrift Affects Versions: 0.96.0, 0.94.5 Reporter: Arpit Gupta Assignee: Arpit Gupta Fix For: 0.96.0, 0.94.6 Attachments: HBASE-7915.branch-0.94.patch, HBASE-7915.patch, HBASE-7915.patch in ThriftServer.java the following call is made {code} serverRunner = new ThriftServerRunner(conf); {code} which invokes {code} public ThriftServerRunner(Configuration conf) throws IOException { this(conf, new ThriftServerRunner.HBaseHandler(conf)); } {code} All of this is happening before the user has logged in and fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7641) Port HBASE-6669 'Add BigDecimalColumnInterpreter for doing aggregations using AggregationClient' to trunk
[ https://issues.apache.org/jira/browse/HBASE-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586061#comment-13586061 ] Julian Wissmann commented on HBASE-7641: Hi Ted, fixed that; removed the else keyword. Formatted with the eclipse formatter, however it did not split lines longer, than 100. Is it not supposed to do that, also? However, I fail to see the connection to the failing tests. I'll check tis again, before submitting another patch. Port HBASE-6669 'Add BigDecimalColumnInterpreter for doing aggregations using AggregationClient' to trunk - Key: HBASE-7641 URL: https://issues.apache.org/jira/browse/HBASE-7641 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Julian Wissmann Labels: features, newbie, patch Fix For: 0.96.0 Attachments: BigDecimalColumnInterpreter.java, HBASE-7641.patch, HBASE-7641v2.patch, hbase.proto, TestBigDecimalColumnInterpreter.java ColumnInterpreter implementation in trunk is different from that in 0.94 This issue ports BigDecimalColumnInterpreter to trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7930) hbck should provide an option to fix empty REGIONINFO_QUALIFIER .META. rows.
[ https://issues.apache.org/jira/browse/HBASE-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586063#comment-13586063 ] Jean-Marc Spaggiari commented on HBASE-7930: hi Dave, This is the code generating this warning: {code} byte [] bytes = result.getValue(HConstants.CATALOG_FAMILY, HConstants.REGIONINFO_QUALIFIER); if (bytes == null) { LOG.warn(REGIONINFO_QUALIFIER is empty in + result); return null; } {code} And it's the only place where this is printed into the logs. I grepped the code and was not able to find anywhere else where this is printed. So this warning is displayed only when we are not able to find the REGIONINFO_QUALIFIER in CATALOG_FAMILY the row. From what is remaining in the table: entry,keyvalue,1360762149493.288611f36e195c950754ba88e9402950. column=info:server, timestamp=1361046940033, value=node1:60020 entry,keyvalue,1360762149493.288611f36e195c950754ba88e9402950. column=info:serverstartcode, timestamp=1361046940033, value=1361046923347 We don't know where to look into HDFS. If we remove those 2 lines and HBCK found a region in the HDFS which is not into the .META., it will reconstruct the entries in the .META. based in the HDFS content. So I think we can safely remove those lines from the .META. and I don't think there is anything else which can generate this WARNing. hbck should provide an option to fix empty REGIONINFO_QUALIFIER .META. rows. Key: HBASE-7930 URL: https://issues.apache.org/jira/browse/HBASE-7930 Project: HBase Issue Type: Improvement Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Today when master and HBCK are reporting empty REGIONINFO_QUALIFIER .META. rows, we need to manually delete them from the .META. region. We need to enhance hbck to do that automatically. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7641) Port HBASE-6669 'Add BigDecimalColumnInterpreter for doing aggregations using AggregationClient' to trunk
[ https://issues.apache.org/jira/browse/HBASE-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586065#comment-13586065 ] Ted Yu commented on HBASE-7641: --- @Julian: Please wrap the long line manually. If you use Eclipse to work on HBase, you can easily see the long lines in your code by turning on 'Show print margin' option in Preferences - General - Editors - Text Editors and specifying 100 as Print margin column. Port HBASE-6669 'Add BigDecimalColumnInterpreter for doing aggregations using AggregationClient' to trunk - Key: HBASE-7641 URL: https://issues.apache.org/jira/browse/HBASE-7641 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Julian Wissmann Labels: features, newbie, patch Fix For: 0.96.0 Attachments: BigDecimalColumnInterpreter.java, HBASE-7641.patch, HBASE-7641v2.patch, hbase.proto, TestBigDecimalColumnInterpreter.java ColumnInterpreter implementation in trunk is different from that in 0.94 This issue ports BigDecimalColumnInterpreter to trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6222) Add per-KeyValue Security
[ https://issues.apache.org/jira/browse/HBASE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586073#comment-13586073 ] Andrew Purtell commented on HBASE-6222: --- [~yuzhih...@gmail.com] The results for writes are not interesting, boxplots align etc. I would have expected some impact to be noticeable so I plan to look into that more. Stay tuned. Add per-KeyValue Security - Key: HBASE-6222 URL: https://issues.apache.org/jira/browse/HBASE-6222 Project: HBase Issue Type: New Feature Components: security Affects Versions: 0.96.0, 0.98.0 Reporter: stack Assignee: Andrew Purtell Attachments: 6222-aclcf.patch, 6222.pdf, cell-acls-kv-tags-not-for-review.zip, HBaseCellRow-LevelSecurityDesignDoc.docx, HBaseCellRow-LevelSecurityPRD.docx Saw an interesting article: http://www.fiercegovernmentit.com/story/sasc-accumulo-language-pro-open-source-say-proponents/2012-06-14 The Senate Armed Services Committee version of the fiscal 2013 national defense authorization act (S. 3254) would require DoD agencies to foreswear the Accumulo NoSQL database after Sept. 30, 2013, unless the DoD CIO certifies that there exists either no viable commercial open source database with security features comparable to [Accumulo] (such as the HBase or Cassandra databases)... Not sure what a 'commercial open source database' is, and I'm not sure whats going on in the article, but tra-la-la'ing, if we had per-KeyValue 'security' like Accumulo's, we might put ourselves in the running for federal contributions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586080#comment-13586080 ] stack commented on HBASE-7188: -- [~eclark] Anything to the above test failing? It failed twice? Any notes on what you did moving the code? Did you have to dup some classes? e.g. Abortable? Chore? These seem more server-side than client classes? Server depends on client? Any editorial on the experience? I'm inclined to just commit and sort out the dead afterward. Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Admin, Client, IPC/RPC, Replication, snapshots Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch, HBASE-7188-5.patch, HBASE-7188-6.patch, HBASE-7188-7.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6222) Add per-KeyValue Security
[ https://issues.apache.org/jira/browse/HBASE-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586093#comment-13586093 ] Andrew Purtell commented on HBASE-6222: --- bq. In AccessController.requireCoveringPermission the close should be in finally block? Ok. bq. May be we can move this before the 'if' condition just before this. Ok. bq. @Test tag misses for testCellPermissions(). Was it intentional? No. Checked the test logs and it still runs, but will add this of course. bq. The memstore flushes if more than 1 CF exists, will it have an impact on this new CF introduced? The ACL CF is only hidden from the client. bq. Once in the AccessController hooks we have ensured that the permission is available by checking the new CF acl, when the actual scan goes we can avoid this Cell right? We could modify the Scan object in a preScannerOpen hook to exclude the ACL CF. The values from that family are not used in the filter. (I seem to remember exploring that idea is why no such exclusion presently.) Thanks for the review! Add per-KeyValue Security - Key: HBASE-6222 URL: https://issues.apache.org/jira/browse/HBASE-6222 Project: HBase Issue Type: New Feature Components: security Affects Versions: 0.96.0, 0.98.0 Reporter: stack Assignee: Andrew Purtell Attachments: 6222-aclcf.patch, 6222.pdf, cell-acls-kv-tags-not-for-review.zip, HBaseCellRow-LevelSecurityDesignDoc.docx, HBaseCellRow-LevelSecurityPRD.docx Saw an interesting article: http://www.fiercegovernmentit.com/story/sasc-accumulo-language-pro-open-source-say-proponents/2012-06-14 The Senate Armed Services Committee version of the fiscal 2013 national defense authorization act (S. 3254) would require DoD agencies to foreswear the Accumulo NoSQL database after Sept. 30, 2013, unless the DoD CIO certifies that there exists either no viable commercial open source database with security features comparable to [Accumulo] (such as the HBase or Cassandra databases)... Not sure what a 'commercial open source database' is, and I'm not sure whats going on in the article, but tra-la-la'ing, if we had per-KeyValue 'security' like Accumulo's, we might put ourselves in the running for federal contributions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7926) SmallTests pollute the META descriptor
[ https://issues.apache.org/jira/browse/HBASE-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586096#comment-13586096 ] Hudson commented on HBASE-7926: --- Integrated in HBase-TRUNK #3896 (See [https://builds.apache.org/job/HBase-TRUNK/3896/]) HBASE-7926 SmallTests pollute the META descriptor (Revision 1449782) Result = FAILURE mbertozzi : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestHColumnDescriptor.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java SmallTests pollute the META descriptor -- Key: HBASE-7926 URL: https://issues.apache.org/jira/browse/HBASE-7926 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7926-v0.patch Running tests on my jenkins I've noticed that the META_TABLEDESC at some point gets changed, and gets SNAPPY encoding and other settings. A couple of SmallTest take the META_TABLEDESC as base and change it directly, to verify the serialization, without creating a copy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7878) recoverFileLease does not check return value of recoverLease
[ https://issues.apache.org/jira/browse/HBASE-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7878: -- Attachment: 7878-trunk-v4.txt Patch v4 introduced hbase.lease.recovery.timeout (open to better naming) with default of 5 minutes. For hadoop 1.0, if the lease recovery takes longer than this timeout, append() is called. TestHLog#testAppendClose passes with this patch. I also tried to cover hadoop 2.0 where RecoveryInProgressException may be thrown. Comments are welcome. recoverFileLease does not check return value of recoverLease Key: HBASE-7878 URL: https://issues.apache.org/jira/browse/HBASE-7878 Project: HBase Issue Type: Bug Components: util Reporter: Eric Newton Assignee: Ted Yu Priority: Critical Fix For: 0.96.0, 0.94.6 Attachments: 7878-trunk-v2.txt, 7878-trunk-v3.txt, 7878-trunk-v4.txt I think this is a problem, so I'm opening a ticket so an HBase person takes a look. Apache Accumulo has moved its write-ahead log to HDFS. I modeled the lease recovery for Accumulo after HBase's lease recovery. During testing, we experienced data loss. I found it is necessary to wait until recoverLease returns true to know that the file has been truly closed. In FSHDFSUtils, the return result of recoverLease is not checked. In the unit tests created to check lease recovery in HBASE-2645, the return result of recoverLease is always checked. I think FSHDFSUtils should be modified to check the return result, and wait until it returns true. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7878) recoverFileLease does not check return value of recoverLease
[ https://issues.apache.org/jira/browse/HBASE-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586114#comment-13586114 ] Ted Yu commented on HBASE-7878: --- bq. Have you tested what happens when you call append? Using hadoop 1.0, I haven't come across the scenario where append() is called. recoverFileLease does not check return value of recoverLease Key: HBASE-7878 URL: https://issues.apache.org/jira/browse/HBASE-7878 Project: HBase Issue Type: Bug Components: util Reporter: Eric Newton Assignee: Ted Yu Priority: Critical Fix For: 0.96.0, 0.94.6 Attachments: 7878-trunk-v2.txt, 7878-trunk-v3.txt, 7878-trunk-v4.txt I think this is a problem, so I'm opening a ticket so an HBase person takes a look. Apache Accumulo has moved its write-ahead log to HDFS. I modeled the lease recovery for Accumulo after HBase's lease recovery. During testing, we experienced data loss. I found it is necessary to wait until recoverLease returns true to know that the file has been truly closed. In FSHDFSUtils, the return result of recoverLease is not checked. In the unit tests created to check lease recovery in HBASE-2645, the return result of recoverLease is always checked. I think FSHDFSUtils should be modified to check the return result, and wait until it returns true. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7878) recoverFileLease does not check return value of recoverLease
[ https://issues.apache.org/jira/browse/HBASE-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586132#comment-13586132 ] stack commented on HBASE-7878: -- bq. Using hadoop 1.0, I haven't come across the scenario where append() is called. Want to try manufacturing it? recoverFileLease does not check return value of recoverLease Key: HBASE-7878 URL: https://issues.apache.org/jira/browse/HBASE-7878 Project: HBase Issue Type: Bug Components: util Reporter: Eric Newton Assignee: Ted Yu Priority: Critical Fix For: 0.96.0, 0.94.6 Attachments: 7878-trunk-v2.txt, 7878-trunk-v3.txt, 7878-trunk-v4.txt I think this is a problem, so I'm opening a ticket so an HBase person takes a look. Apache Accumulo has moved its write-ahead log to HDFS. I modeled the lease recovery for Accumulo after HBase's lease recovery. During testing, we experienced data loss. I found it is necessary to wait until recoverLease returns true to know that the file has been truly closed. In FSHDFSUtils, the return result of recoverLease is not checked. In the unit tests created to check lease recovery in HBASE-2645, the return result of recoverLease is always checked. I think FSHDFSUtils should be modified to check the return result, and wait until it returns true. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586134#comment-13586134 ] Hadoop QA commented on HBASE-4210: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570817/HBASE-4210_Trunk-V2.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 9 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor {color:red}-1 core zombie tests{color}. There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//console This message is automatically generated. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586138#comment-13586138 ] Elliott Clark commented on HBASE-7188: -- Almost all of this was just moving classes. There's very little changes in classes. * First I moved all of the o.a.h.h.client namespace into hbase-client. * Moved ipc stuff related to the client. * Moved the needed zookeeper classes into hbase-client. * Protobuf utils were split, so that replication uses a different util. ** This is so that Wal and HFile aren't brought in as dependencies. * Split some enums out of security. ** Done in previous jira. * Split some enums out of Executors. ** EventType ** ExecutorType * Had to un-link some javadocs so the imports don't refer to the server classes. * Moved Exceptions into an exception namespace like you suggested before. ** Though I did have to move exceptions into the client module since some of the exceptions now refer to protobuf. ** There still might be more exceptions to move. I didn't duplicate any classes. I did move chore and Catalog tracker into client because: * Chore is used in HConnectionManager. * Abortable is used in CatalogTracker, and zookeeper classes. Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Admin, Client, IPC/RPC, Replication, snapshots Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch, HBASE-7188-5.patch, HBASE-7188-6.patch, HBASE-7188-7.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586139#comment-13586139 ] Hadoop QA commented on HBASE-7692: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570819/HBASE-7692.v5.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 68 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces lines longer than 100 {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4531//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-orderly.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4531//console This message is automatically generated. Add utility class to generate ordered byte[] serialization -- Key: HBASE-7692 URL: https://issues.apache.org/jira/browse/HBASE-7692 Project: HBase Issue Type: Improvement Components: util Reporter: Nick Dimiduk Assignee: Nick Dimiduk Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch The current Bytes utility class works, but produces output that does not maintain the native sort ordering of the input value. This results in, for example, a negative value that does not necessarily sort before a positive value. HBase should provide a canonical implementation of such a serialization format so that third-parties can reliably build on top of HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7915) Secure ThriftServer needs to login before calling HBaseHandler
[ https://issues.apache.org/jira/browse/HBASE-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586142#comment-13586142 ] Hadoop QA commented on HBASE-7915: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570827/HBASE-7915.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: {color:red}-1 core zombie tests{color}. There are 1 zombie test(s): at org.apache.hadoop.hbase.util.TestHBaseFsck.testFixByTable(TestHBaseFsck.java:1157) Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4530//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4530//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4530//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4530//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4530//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4530//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4530//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4530//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4530//console This message is automatically generated. Secure ThriftServer needs to login before calling HBaseHandler -- Key: HBASE-7915 URL: https://issues.apache.org/jira/browse/HBASE-7915 Project: HBase Issue Type: Bug Components: security, Thrift Affects Versions: 0.96.0, 0.94.5 Reporter: Arpit Gupta Assignee: Arpit Gupta Fix For: 0.96.0, 0.94.6 Attachments: HBASE-7915.branch-0.94.patch, HBASE-7915.patch, HBASE-7915.patch in ThriftServer.java the following call is made {code} serverRunner = new ThriftServerRunner(conf); {code} which invokes {code} public ThriftServerRunner(Configuration conf) throws IOException { this(conf, new ThriftServerRunner.HBaseHandler(conf)); } {code} All of this is happening before the user has logged in and fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7188) Move classes into hbase-client
[ https://issues.apache.org/jira/browse/HBASE-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586147#comment-13586147 ] stack commented on HBASE-7188: -- Go for it then. +1 Any suggestions for cleanup issues post commit that came up during your 'moving' experience? Move classes into hbase-client -- Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Components: Admin, Client, IPC/RPC, Replication, snapshots Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7188-0.patch, HBASE-7188-1.patch, HBASE-7188-2.patch, HBASE-7188-3.patch, HBASE-7188-5.patch, HBASE-7188-6.patch, HBASE-7188-7.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7900) Have client Mutations (Put/Delete/etc.) and Result implement CellScanner Interface
[ https://issues.apache.org/jira/browse/HBASE-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7900: - Attachment: 7900v6.txt Address @Matt Corgan comments up on rb. Made the Mutation family List? extends Cell and then changed everywhere (I think) where I was building up ListCell to be explicit ListKeyValue instead. Have client Mutations (Put/Delete/etc.) and Result implement CellScanner Interface -- Key: HBASE-7900 URL: https://issues.apache.org/jira/browse/HBASE-7900 Project: HBase Issue Type: Sub-task Components: IPC/RPC Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.96.0 Attachments: 7900.txt, 7900v2.txt, 7900v3.txt, 7900v4.txt, 7900v5.txt, 7900v6.txt Need to be able to Iterate the Cells carried by Puts, Deletes, etc., as well as Result so can build Cell blocks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7900) Have client Mutations (Put/Delete/etc.) and Result implement CellScanner Interface
[ https://issues.apache.org/jira/browse/HBASE-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586162#comment-13586162 ] Matt Corgan commented on HBASE-7900: I'm not too familiar with the RPC area of the code, but I thought everything looked good conceptually from the CellScanner perspective. I could not review in enough detail to be spotting bugs though. Have client Mutations (Put/Delete/etc.) and Result implement CellScanner Interface -- Key: HBASE-7900 URL: https://issues.apache.org/jira/browse/HBASE-7900 Project: HBase Issue Type: Sub-task Components: IPC/RPC Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.96.0 Attachments: 7900.txt, 7900v2.txt, 7900v3.txt, 7900v4.txt, 7900v5.txt, 7900v6.txt Need to be able to Iterate the Cells carried by Puts, Deletes, etc., as well as Result so can build Cell blocks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira