[jira] [Commented] (HBASE-7160) Improve IdLock and remove its minor defects
[ https://issues.apache.org/jira/browse/HBASE-7160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500080#comment-13500080 ] Hadoop QA commented on HBASE-7160: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554129/HBASE-7160-V2.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 99 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 24 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3363//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3363//console This message is automatically generated. Improve IdLock and remove its minor defects --- Key: HBASE-7160 URL: https://issues.apache.org/jira/browse/HBASE-7160 Project: HBase Issue Type: Improvement Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Priority: Minor Attachments: HBASE-7160.patch, HBASE-7160-V2.patch Combination of synchronizations and concurrent collections complicates the code, and it is hard to trace the code and to confirm its correctness. We should re-create the class and make it more understandable. In the current code, I find the following minor defects: (1) In the case that there is a waiting thread for a lock in getLockEntry() and another thread is releasing the lock by calling releaseLockEntry(), trying to get the lock with a 3rd thread by calling getLockEntry() falls into a busy loop until the waiting thread wakes up and gets the lock. Even if notify() wakes up the blocked thread and causes a context switch to the waked thread immediately, synchronization might block the waked thread and cause another context switch, and the busy loop might continue for a while. (2) In the same case as (1), since releasing the lock is merely notifying without removing the lock-entry from the map, interrupting the waiting thread might leave an unused lock-entry (entry.numWaiters == 0) in the map. This is a memory leak unless the id of the lock is used again. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7184) pre-check table name in table.jsp
[ https://issues.apache.org/jira/browse/HBASE-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liang xie updated HBASE-7184: - Status: Patch Available (was: Open) pre-check table name in table.jsp - Key: HBASE-7184 URL: https://issues.apache.org/jira/browse/HBASE-7184 Project: HBase Issue Type: Improvement Components: UI Affects Versions: 0.94.2, 0.96.0 Reporter: liang xie Assignee: liang xie Attachments: HBASE-7184.txt Currently the (table)name parameter in table.jsp isn't checked, it brings two problems at least: 1) 500 error for invalid value 2) directly written to JSP output, giving reflected XSS vulnerability we can do a parameter-checking -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7184) pre-check table name in table.jsp
[ https://issues.apache.org/jira/browse/HBASE-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liang xie updated HBASE-7184: - Attachment: HBASE-7184.txt pre-check table name in table.jsp - Key: HBASE-7184 URL: https://issues.apache.org/jira/browse/HBASE-7184 Project: HBase Issue Type: Improvement Components: UI Affects Versions: 0.94.2, 0.96.0 Reporter: liang xie Assignee: liang xie Attachments: HBASE-7184.txt Currently the (table)name parameter in table.jsp isn't checked, it brings two problems at least: 1) 500 error for invalid value 2) directly written to JSP output, giving reflected XSS vulnerability we can do a parameter-checking -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7184) pre-check table name in table.jsp
liang xie created HBASE-7184: Summary: pre-check table name in table.jsp Key: HBASE-7184 URL: https://issues.apache.org/jira/browse/HBASE-7184 Project: HBase Issue Type: Improvement Components: UI Affects Versions: 0.94.2, 0.96.0 Reporter: liang xie Assignee: liang xie Attachments: HBASE-7184.txt Currently the (table)name parameter in table.jsp isn't checked, it brings two problems at least: 1) 500 error for invalid value 2) directly written to JSP output, giving reflected XSS vulnerability we can do a parameter-checking -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7184) pre-check table name in table.jsp
[ https://issues.apache.org/jira/browse/HBASE-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500081#comment-13500081 ] liang xie commented on HBASE-7184: -- Before applied the attached patch, if i access host:port/table.jsp?name=... , will get : HTTP ERROR 500 Problem accessing /table.jsp. Reason: ... Caused by: org.apache.hadoop.hbase.TableNotFoundException: ... at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:999) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:864) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:821) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234) at org.apache.hadoop.hbase.client.HTable.init(HTable.java:174) at org.apache.hadoop.hbase.client.HTable.init(HTable.java:133) at org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:67) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98) at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221) at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1056) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) Powered by Jetty:// After applied patch, i got : Illegal table name parameter! pre-check table name in table.jsp - Key: HBASE-7184 URL: https://issues.apache.org/jira/browse/HBASE-7184 Project: HBase Issue Type: Improvement Components: UI Affects Versions: 0.94.2, 0.96.0 Reporter: liang xie Assignee: liang xie Attachments: HBASE-7184.txt Currently the (table)name parameter in table.jsp isn't checked, it brings two problems at least: 1) 500 error for invalid value 2) directly written to JSP output, giving reflected XSS vulnerability we can do a parameter-checking -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7184) pre-check table name in table.jsp
[ https://issues.apache.org/jira/browse/HBASE-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500102#comment-13500102 ] Hadoop QA commented on HBASE-7184: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554136/HBASE-7184.txt against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 99 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 24 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSide Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3364//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3364//console This message is automatically generated. pre-check table name in table.jsp - Key: HBASE-7184 URL: https://issues.apache.org/jira/browse/HBASE-7184 Project: HBase Issue Type: Improvement Components: UI Affects Versions: 0.94.2, 0.96.0 Reporter: liang xie Assignee: liang xie Attachments: HBASE-7184.txt Currently the (table)name parameter in table.jsp isn't checked, it brings two problems at least: 1) 500 error for invalid value 2) directly written to JSP output, giving reflected XSS vulnerability we can do a parameter-checking -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.
[ https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500108#comment-13500108 ] Cosmin Lehene commented on HBASE-2645: -- Wow, I think it was one of the longest lived blockers. Glad to see it fixed. Thanks :) HLog writer can do 1-2 sync operations after lease has been recovered for split process. Key: HBASE-2645 URL: https://issues.apache.org/jira/browse/HBASE-2645 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 0.90.4 Reporter: Cosmin Lehene Assignee: Todd Lipcon Priority: Blocker Fix For: 0.96.0 Attachments: 2645-addendum.txt, 2645_hacking.txt, 2645.txt, 2645v2.txt, 2645v3.txt, 2645v4.txt, hdfs_1.0_editswriter_recoverlease.txt, hdfs_trunk_editswriter_recoverlease.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt, org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. This test starts a thread that writes one edit to the log, syncs and counts. During this, a HLog.splitLog operation is started. splitLog recovers the log lease before reading the log, so that the original regionserver could not wake up and write after the split process started. The test compares the number of edits reported by the split process and by the writer thread. Writer thread (called zombie in the test) should report = than the splitLog (sync() might raise after the last edit gets written and the edit won't get counted by zombie thread). However it appears that the zombie counts 1-2 more edits. So it looks like it can sync without a lease. This might be a hdfs-0.20 related issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7184) pre-check table name in table.jsp
[ https://issues.apache.org/jira/browse/HBASE-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500112#comment-13500112 ] liang xie commented on HBASE-7184: -- emmm. seems the failure test case is unrelated with this issue, IMHO... pre-check table name in table.jsp - Key: HBASE-7184 URL: https://issues.apache.org/jira/browse/HBASE-7184 Project: HBase Issue Type: Improvement Components: UI Affects Versions: 0.94.2, 0.96.0 Reporter: liang xie Assignee: liang xie Attachments: HBASE-7184.txt Currently the (table)name parameter in table.jsp isn't checked, it brings two problems at least: 1) 500 error for invalid value 2) directly written to JSP output, giving reflected XSS vulnerability we can do a parameter-checking -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6777) Snapshot Restore interface
[ https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500325#comment-13500325 ] Jesse Yates commented on HBASE-6777: Committing today, unless there are objections Snapshot Restore interface -- Key: HBASE-6777 URL: https://issues.apache.org/jira/browse/HBASE-6777 Project: HBase Issue Type: Sub-task Components: Client, master, snapshots Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Fix For: hbase-6055 Attachments: hbase-6777-v9.patch, HBASE--v1.patch Add interfaces for restoring a snapshot -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7035) thrift server closes HTable of open Scanners
[ https://issues.apache.org/jira/browse/HBASE-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devin Bayer updated HBASE-7035: --- Status: Patch Available (was: Open) thrift server closes HTable of open Scanners Key: HBASE-7035 URL: https://issues.apache.org/jira/browse/HBASE-7035 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.94.4 Reporter: Devin Bayer Labels: thrift2 Original Estimate: 4h Remaining Estimate: 4h ThriftHBaseServiceHandler.openScanner() does this: 1. table = pool.getTable() 2. scanner = table.getScanner() 3. table.close() 4. return scanner While back porting the thrift server to 0.92.6, I found that table.close() calls connection.close(). Further calls to scanner.next() raise a ConnectionClosed exception. The unit tests do not catch this since they reuse an open HConnection instance. This might work on trunk, but depends on the implementations of HTablePool, HTable and HConnectionManager. Even with the pool wrapper, if the pool is full, table.close() may be called, which may invalidate the table. Also, HTable is not thread-safe, but they are being reused since they go back in the pool. I suggest storing the table handle along with the scanner. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7035) thrift server closes HTable of open Scanners
[ https://issues.apache.org/jira/browse/HBASE-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devin Bayer updated HBASE-7035: --- Attachment: old-hbase-thrift-v1.patch thrift server closes HTable of open Scanners Key: HBASE-7035 URL: https://issues.apache.org/jira/browse/HBASE-7035 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.94.4 Reporter: Devin Bayer Labels: thrift2 Attachments: old-hbase-thrift-v1.patch Original Estimate: 4h Remaining Estimate: 4h ThriftHBaseServiceHandler.openScanner() does this: 1. table = pool.getTable() 2. scanner = table.getScanner() 3. table.close() 4. return scanner While back porting the thrift server to 0.92.6, I found that table.close() calls connection.close(). Further calls to scanner.next() raise a ConnectionClosed exception. The unit tests do not catch this since they reuse an open HConnection instance. This might work on trunk, but depends on the implementations of HTablePool, HTable and HConnectionManager. Even with the pool wrapper, if the pool is full, table.close() may be called, which may invalidate the table. Also, HTable is not thread-safe, but they are being reused since they go back in the pool. I suggest storing the table handle along with the scanner. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7035) thrift server closes HTable of open Scanners
[ https://issues.apache.org/jira/browse/HBASE-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500435#comment-13500435 ] Devin Bayer commented on HBASE-7035: stack: patch is attached. It does work for us. thrift server closes HTable of open Scanners Key: HBASE-7035 URL: https://issues.apache.org/jira/browse/HBASE-7035 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.94.4 Reporter: Devin Bayer Labels: thrift2 Attachments: old-hbase-thrift-v1.patch Original Estimate: 4h Remaining Estimate: 4h ThriftHBaseServiceHandler.openScanner() does this: 1. table = pool.getTable() 2. scanner = table.getScanner() 3. table.close() 4. return scanner While back porting the thrift server to 0.92.6, I found that table.close() calls connection.close(). Further calls to scanner.next() raise a ConnectionClosed exception. The unit tests do not catch this since they reuse an open HConnection instance. This might work on trunk, but depends on the implementations of HTablePool, HTable and HConnectionManager. Even with the pool wrapper, if the pool is full, table.close() may be called, which may invalidate the table. Also, HTable is not thread-safe, but they are being reused since they go back in the pool. I suggest storing the table handle along with the scanner. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6865) Snapshot File Cleaners
[ https://issues.apache.org/jira/browse/HBASE-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesse Yates updated HBASE-6865: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to snapshot branch Snapshot File Cleaners -- Key: HBASE-6865 URL: https://issues.apache.org/jira/browse/HBASE-6865 Project: HBase Issue Type: Sub-task Reporter: Jesse Yates Assignee: Jesse Yates Fix For: hbase-6055 Attachments: hbase-6865-v2.patch HFile and HLog cleaners are necessary for fully implementing snapshots, but can be broken out into their own piece really cleanly. The HFile cleaner is necessary for both timestamp and globally consistent snapshots, but the HLog cleaner is necessary for globally consistent and offline snapshots. Putting everything together in one patch isn't too much overload as there is a lot of overlap between the cleaners. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7062) Move HLog stats to metrics 2
[ https://issues.apache.org/jira/browse/HBASE-7062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7062: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed revision 1411337. Move HLog stats to metrics 2 Key: HBASE-7062 URL: https://issues.apache.org/jira/browse/HBASE-7062 Project: HBase Issue Type: Sub-task Components: metrics Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7062-1.patch, HBASE-7062-2.patch, HBASE-7062-3.patch, HBASE-7062-4.patch, HBASE-7062-5.patch, HBASE-7062-6.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7185) Create hbase-rpc module
Elliott Clark created HBASE-7185: Summary: Create hbase-rpc module Key: HBASE-7185 URL: https://issues.apache.org/jira/browse/HBASE-7185 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7186) Split Classes for Client/Server module split.
Elliott Clark created HBASE-7186: Summary: Split Classes for Client/Server module split. Key: HBASE-7186 URL: https://issues.apache.org/jira/browse/HBASE-7186 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7148) Some files in hbase-examples module miss license header
[ https://issues.apache.org/jira/browse/HBASE-7148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500454#comment-13500454 ] Sergey Shelukhin commented on HBASE-7148: - Sorry, my mistake; I was away last week. Thanks Enis for taking care of this! Some files in hbase-examples module miss license header --- Key: HBASE-7148 URL: https://issues.apache.org/jira/browse/HBASE-7148 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: hbase-7148.patch Trunk build 3530 got to building hbase-examples module but failed: {code} [INFO] HBase - Examples .. FAILURE [3.222s] [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 29:21.569s [INFO] Finished at: Sun Nov 11 15:17:35 UTC 2012 [INFO] Final Memory: 68M/642M [INFO] [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.8:check (default) on project hbase-examples: Too many unapproved licenses: 20 - [Help 1] {code} Looks like license headers are missing in some of the files in hbase-examples module -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7187) Create empty hbase-client module
Elliott Clark created HBASE-7187: Summary: Create empty hbase-client module Key: HBASE-7187 URL: https://issues.apache.org/jira/browse/HBASE-7187 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7188) Move classes into hbase-client
Elliott Clark created HBASE-7188: Summary: Move classes into hbase-client Key: HBASE-7188 URL: https://issues.apache.org/jira/browse/HBASE-7188 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7012) Create hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-7012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500459#comment-13500459 ] Elliott Clark commented on HBASE-7012: -- Sure I'll give it a shot. Some of the changes are when splitting the classes only become really evident after the move. Other than that it should be pretty easy. Create hbase-client module -- Key: HBASE-7012 URL: https://issues.apache.org/jira/browse/HBASE-7012 Project: HBase Issue Type: Task Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7012-0.patch.gz I just tried creating a project that uses 0.95-SNAPSHOT and had to import org.apache.hbase:hbase-server as the module that I depend on. This will be confusing to users. In addition this brings in lots of dependencies that are not really needed. Let's create a client module that has all of the client in it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Assignee: Jimmy Xiang (was: Amitanand Aiyer) Status: Open (was: Patch Available) Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Attachment: trunk-6423_v2.patch Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Patch Available (was: Open) Addressed Ted's comments. Fixed TestHeapSize. Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500477#comment-13500477 ] Hadoop QA commented on HBASE-6423: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554204/trunk-6423_v2.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 5 new or modified tests. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3366//console This message is automatically generated. Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7185) Create hbase-rpc module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7185: - Attachment: HBASE-7185-0.patch Move Protobuf files into hbase-rpc module. This new module should make it easy for others to interact with hbase-server without depending on much from hbase. Eg this could be used by something like async hbase. Create hbase-rpc module --- Key: HBASE-7185 URL: https://issues.apache.org/jira/browse/HBASE-7185 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 Attachments: HBASE-7185-0.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7185) Create hbase-rpc module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7185: - Status: Patch Available (was: Open) Create hbase-rpc module --- Key: HBASE-7185 URL: https://issues.apache.org/jira/browse/HBASE-7185 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 Attachments: HBASE-7185-0.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96
[ https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500491#comment-13500491 ] Jonathan Hsieh commented on HBASE-6055: --- I've reached a pretty decent point on this branch now refactoring and simplifying the online snapshot code. Most of the work is in the plumbing, and most of the complexity remains in the plumbing. Here is a link https://github.com/jmhsieh/hbase/commits/snapshot-dev? It isn't worth looking at the individual patches -- if interested take a look at the code generally one package at a time. I'd suggest starting from errorhandling, then procedures since this work is fairly isolated and stable now. The will likely affect existing code, and online may change significantly during the merge process. Error handling: o.a.h.h.errorhandling.* * Added concept of ExternalException (an exception from an separate thread or process). * Removing generics by funneling everything through an ExternalException * Simplified Exception Propagation by only having a Dispatcher, Listener, and Checker. (No Visitors, Orchestrators, some Factories) * Made Exception Serialization static so that instances don't need to be passed around. * Added more meaningful usage and motivation documentation. Procedure framework: o.a.h.h.procedure.* * Separated Coordinator side from Member side * Reduced the number of constructors (and fewer objects threaded throughout). * Added concept of Procedure and Subprocedure -- these maintain state on each host. (this replaces just using strings everywhere). * Folded several threads that used latch ping-pong into single threads. * Renamed methods from 2pc nomenclature to barrier nomenclature. * Added more meaningful usage and motivation documentation. Online Snapshots: o.a.h.h.snapshot.* * Converted per regionserver only Procedures to simpler Callable/Future fork join implementation. * Removed different *ErrorHandlers and moved into Subprocedures. (this may be further eliminated) * Each Procedure contains an ExternalExceptionDispatcher * ExternalExceptions go to the SnapshotManager to abort the Procedure. I'm in process of merging code into the offline snapshot branch. It isn't clean but I'll be working on that for the next few days. (Clashes with updates in offline snapshot). Once I get the snapshot branch compiling again, I'll start posting the External Exception and Procedure stuff as a series of patches. My suggestion for the overall effort is to get the main offline snapshot branch code committed to the branch and then start looking into merging with trunk and 0.94. The online work I feel should remain a branch until its pieces are fleshed out. Snapshots in HBase 0.96 --- Key: HBASE-6055 URL: https://issues.apache.org/jira/browse/HBASE-6055 Project: HBase Issue Type: New Feature Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jesse Yates Assignee: Jesse Yates Fix For: hbase-6055, 0.96.0 Attachments: Snapshots in HBase.docx Continuation of HBASE-50 for the current trunk. Since the implementation has drastically changed, opening as a new ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7185) Create hbase-rpc module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500502#comment-13500502 ] Hadoop QA commented on HBASE-7185: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554208/HBASE-7185-0.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 99 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 24 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3367//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-rpc.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3367//console This message is automatically generated. Create hbase-rpc module --- Key: HBASE-7185 URL: https://issues.apache.org/jira/browse/HBASE-7185 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 Attachments: HBASE-7185-0.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7185) Create hbase-rpc module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7185: - Attachment: HBASE-7185-1.patch Missed a commit that cleaned up the rpc pom to not depend upon resource listener. Create hbase-rpc module --- Key: HBASE-7185 URL: https://issues.apache.org/jira/browse/HBASE-7185 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7185) Create hbase-rpc module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7185: - Attachment: (was: HBASE-7185-1.patch) Create hbase-rpc module --- Key: HBASE-7185 URL: https://issues.apache.org/jira/browse/HBASE-7185 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 Attachments: HBASE-7185-0.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7185) Create hbase-rpc module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7185: - Attachment: HBASE-7185-1.patch Create hbase-rpc module --- Key: HBASE-7185 URL: https://issues.apache.org/jira/browse/HBASE-7185 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7062) Move HLog stats to metrics 2
[ https://issues.apache.org/jira/browse/HBASE-7062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500524#comment-13500524 ] Hudson commented on HBASE-7062: --- Integrated in HBase-TRUNK #3553 (See [https://builds.apache.org/job/HBase-TRUNK/3553/]) HBASE-7062 Move HLog stats to metrics 2 (Revision 1411337) Result = FAILURE eclark : Files : * /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSource.java * /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal * /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWALSource.java * /hbase/trunk/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal * /hbase/trunk/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestMetricsHLogSource.java * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSourceImpl.java * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWALSourceImpl.java * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationSourceImpl.java * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/rest/MetricsRESTSourceImpl.java * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSourceImpl.java * /hbase/trunk/hbase-hadoop1-compat/src/main/resources/META-INF/services/org.apache.hadoop.hbase.regionserver.wal.MetricsWALSource * /hbase/trunk/hbase-hadoop1-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal * /hbase/trunk/hbase-hadoop1-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestMetricsWALSourceImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSourceImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWALSourceImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationSourceImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/rest/MetricsRESTSourceImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSourceImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/resources/META-INF/services/org.apache.hadoop.hbase.regionserver.wal.MetricsWALSource * /hbase/trunk/hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal * /hbase/trunk/hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestMetricsWALSourceImpl.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogMetrics.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWAL.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java Move HLog stats to metrics 2 Key: HBASE-7062 URL: https://issues.apache.org/jira/browse/HBASE-7062 Project: HBase Issue Type: Sub-task Components: metrics Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: HBASE-7062-1.patch, HBASE-7062-2.patch, HBASE-7062-3.patch, HBASE-7062-4.patch, HBASE-7062-5.patch, HBASE-7062-6.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7009) Port HBaseCluster interface/tests to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500525#comment-13500525 ] stack commented on HBASE-7009: -- So, is this good to go then lads? Port HBaseCluster interface/tests to 0.94 - Key: HBASE-7009 URL: https://issues.apache.org/jira/browse/HBASE-7009 Project: HBase Issue Type: Sub-task Components: test Affects Versions: 0.94.3 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.94.4 Attachments: HBASE-7009-p1.patch, HBASE-7009.patch, HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch Need to port. I am porting V5 patch from the original JIRA; I have a partially ported (V3) patch from Enis with protocol buffers being reverted to HRegionInterface/HMasterInterface -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96
[ https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500532#comment-13500532 ] Jonathan Hsieh commented on HBASE-6055: --- Branch to a squashed version. https://github.com/jmhsieh/hbase/tree/snapshot-dev-squash Snapshots in HBase 0.96 --- Key: HBASE-6055 URL: https://issues.apache.org/jira/browse/HBASE-6055 Project: HBase Issue Type: New Feature Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jesse Yates Assignee: Jesse Yates Fix For: hbase-6055, 0.96.0 Attachments: Snapshots in HBase.docx Continuation of HBASE-50 for the current trunk. Since the implementation has drastically changed, opening as a new ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Open (was: Patch Available) Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Attachment: trunk-6423_v2.1.patch Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Patch Available (was: Open) Rebased to trunk latest. Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96
[ https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500557#comment-13500557 ] Jesse Yates commented on HBASE-6055: I just +1'ed Matteo's restore changes, so I hope to roll that into the snapshot branch (https://github.com/jyates/hbase/tree/snapshots) in the next day or two, which would mean we have everything for offline snapshots. The offline snapshots themselves are a self contained bit of code and sizable enough to make it worthwhile to roll it into trunk on its own. Let's keep the online keep on a branch until its wrapped up - I'll start taking a look at the online when I have a chance, and look forward to code posted on RB. The progress sounds real sweet Jon! TL;DR lets do what Jon suggests above, especially as offline snapshots are basically done and ready to merge into trunk Snapshots in HBase 0.96 --- Key: HBASE-6055 URL: https://issues.apache.org/jira/browse/HBASE-6055 Project: HBase Issue Type: New Feature Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jesse Yates Assignee: Jesse Yates Fix For: hbase-6055, 0.96.0 Attachments: Snapshots in HBase.docx Continuation of HBASE-50 for the current trunk. Since the implementation has drastically changed, opening as a new ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96
[ https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500578#comment-13500578 ] Jonathan Hsieh commented on HBASE-6055: --- Just force pushed the snapshot-dev-squash branch -- it didn't delete files that were supposed to be gone. Snapshots in HBase 0.96 --- Key: HBASE-6055 URL: https://issues.apache.org/jira/browse/HBASE-6055 Project: HBase Issue Type: New Feature Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jesse Yates Assignee: Jesse Yates Fix For: hbase-6055, 0.96.0 Attachments: Snapshots in HBase.docx Continuation of HBASE-50 for the current trunk. Since the implementation has drastically changed, opening as a new ticket. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5422) StartupBulkAssigner would cause a lot of timeout on RIT when assigning large numbers of regions (timeout = 3 mins)
[ https://issues.apache.org/jira/browse/HBASE-5422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500592#comment-13500592 ] stack commented on HBASE-5422: -- I committed with an issue number of HBASE-5542 by mistake. I just corrected it by doing the following: {code} $ svn propset -r1292409 --revprop svn:log HBASE-5422 StartupBulkAssigner would cause a lot of timeout on RIT when assigning large numbers of regions (timeout = 3 mins) {code} I did it in 0.92, 0.94 and trunk. StartupBulkAssigner would cause a lot of timeout on RIT when assigning large numbers of regions (timeout = 3 mins) -- Key: HBASE-5422 URL: https://issues.apache.org/jira/browse/HBASE-5422 Project: HBase Issue Type: Bug Components: master Reporter: chunhui shen Assignee: chunhui shen Fix For: 0.92.1 Attachments: 5422-90.patch, 5422-90v2.patch, 5422-90v3.patch, 5422-v3.txt, hbase-5422.patch, hbase-5422v2.patch, hbase-5422v3.patch In our produce environment We find a lot of timeout on RIT when cluster up, there are about 7w regions in the cluster( 25 regionservers ). First, we could see the following log:(See the region 33cf229845b1009aa8a3f7b0f85c9bd0) master's log 2012-02-13 18:07:41,409 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:6-0x348f4a94723da5 Async create of unassigned node for 33cf229845b1009aa8a3f7b0f85c9bd0 with OFFLINE state 2012-02-13 18:07:42,560 DEBUG org.apache.hadoop.hbase.master.AssignmentManager$CreateUnassignedAsyncCallback: rs=item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. state=OFFLINE, ts=1329127661409, server=r03f11025.yh.aliyun.com,60020,1329127549907 2012-02-13 18:07:42,996 DEBUG org.apache.hadoop.hbase.master.AssignmentManager$ExistsUnassignedAsyncCallback: rs=item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. state=OFFLINE, ts=1329127661409 2012-02-13 18:10:48,072 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. state=PENDING_OPEN, ts=1329127662996 2012-02-13 18:10:48,072 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been PENDING_OPEN for too long, reassigning region=item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. 2012-02-13 18:11:16,744 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENED, server=r03f11025.yh.aliyun.com,60020,1329127549907, region=33cf229845b1009aa8a3f7b0f85c9bd0 2012-02-13 18:38:07,310 DEBUG org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: Handling OPENED event for 33cf229845b1009aa8a3f7b0f85c9bd0; deleting unassigned node 2012-02-13 18:38:07,310 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:6-0x348f4a94723da5 Deleting existing unassigned node for 33cf229845b1009aa8a3f7b0f85c9bd0 that is in expected state RS_ZK_REGION_OPENED 2012-02-13 18:38:07,314 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:6-0x348f4a94723da5 Successfully deleted unassigned node for region 33cf229845b1009aa8a3f7b0f85c9bd0 in expected state RS_ZK_REGION_OPENED 2012-02-13 18:38:07,573 DEBUG org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: Opened region item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. on r03f11025.yh.aliyun.com,60020,1329127549907 2012-02-13 18:50:54,428 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan was found (or we are ignoring an existing plan) for item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. so generated a random one; hri=item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0., src=, dest=r01b05043.yh.aliyun.com,60020,1329127549041; 29 (online=29, exclude=null) available servers 2012-02-13 18:50:54,428 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Assigning region item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. to r01b05043.yh.aliyun.com,60020,1329127549041 2012-02-13 19:31:50,514 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. state=PENDING_OPEN, ts=1329132528086 2012-02-13 19:31:50,514 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been PENDING_OPEN for too long, reassigning region=item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. Regionserver's log 2012-02-13 18:07:43,537 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Received request to open region:
[jira] [Updated] (HBASE-6230) [brainstorm] Restore snapshots for HBase 0.96
[ https://issues.apache.org/jira/browse/HBASE-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-6230: --- Attachment: HBASE-6230-v2.patch [brainstorm] Restore snapshots for HBase 0.96 --- Key: HBASE-6230 URL: https://issues.apache.org/jira/browse/HBASE-6230 Project: HBase Issue Type: Brainstorming Reporter: Jesse Yates Assignee: Matteo Bertozzi Attachments: HBASE-6230-v1.patch, HBASE-6230-v2.patch, SnapshotRestore-v0.pdf Discussion ticket around the definitions/expectations of different parts of snapshot restoration. This is complementary, but separate from the _how_ of taking a snapshot of a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500632#comment-13500632 ] Hadoop QA commented on HBASE-6423: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554219/trunk-6423_v2.1.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 5 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 99 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 22 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3369//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3369//console This message is automatically generated. Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7185) Create hbase-rpc module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500631#comment-13500631 ] Hadoop QA commented on HBASE-7185: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12554216/HBASE-7185-1.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 99 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 24 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3368//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-rpc.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3368//console This message is automatically generated. Create hbase-rpc module --- Key: HBASE-7185 URL: https://issues.apache.org/jira/browse/HBASE-7185 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7185) Create hbase-rpc module
[ https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500634#comment-13500634 ] Elliott Clark commented on HBASE-7185: -- Seems like all tests passed even though it timed out. Create hbase-rpc module --- Key: HBASE-7185 URL: https://issues.apache.org/jira/browse/HBASE-7185 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7180) RegionScannerImpl.next() is inefficient.
[ https://issues.apache.org/jira/browse/HBASE-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-7180: - Attachment: 7180-0.94-v1.txt Slightly better patch. Also changes the call in RegionServer to use the cheaper version of next(). Looking around at the code, we can also replace all the calls from the AggregationImplementation to use this cheaper next() method. RegionScannerImpl.next() is inefficient. Key: HBASE-7180 URL: https://issues.apache.org/jira/browse/HBASE-7180 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Attachments: 7180-0.94-SKETCH.txt, 7180-0.94-v1.txt We just came across a special scenario. For our Phoenix project (SQL runtime for HBase), we push a lot of work into HBase via coprocessors. One method is to wrap RegionScanner in coprocessor hooks and then do processing in the hook to avoid returning a lot of data to the client unnecessarily. In this specific case this is pretty bad. Since the wrapped RegionScanner's next() does not know that it is called this way is still does all of this on each invocation: # Starts a RegionOperation # Increments the request count # set the current read point on a thread local (because generally each call could come from a different thread) # Finally does the next on its StoreScanner(s) # Ends the RegionOperation When this is done in a tight loop millions of times (as is the case for us) it starts to become significant. Not sure what to do about this, really. Opening this issue for discussion. One way is to extend the RegionScanner with an internal next() method of sorts, so that all this overhead can be avoided. The coprocessor could call the regular next() methods once and then just call the cheaper internal version. Are there better/cleaner ways? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira