[jira] [Commented] (HBASE-7160) Improve IdLock and remove its minor defects

2012-11-19 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500080#comment-13500080
 ] 

Hadoop QA commented on HBASE-7160:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12554129/HBASE-7160-V2.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
99 warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 findbugs{color}.  The patch appears to introduce 24 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3363//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3363//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3363//console

This message is automatically generated.

 Improve IdLock and remove its minor defects
 ---

 Key: HBASE-7160
 URL: https://issues.apache.org/jira/browse/HBASE-7160
 Project: HBase
  Issue Type: Improvement
Reporter: Hiroshi Ikeda
Assignee: Hiroshi Ikeda
Priority: Minor
 Attachments: HBASE-7160.patch, HBASE-7160-V2.patch


 Combination of synchronizations and concurrent collections complicates the 
 code, and it is hard to trace the code and to confirm its correctness. We 
 should re-create the class and make it more understandable.
 In the current code, I find the following minor defects:
 (1) In the case that there is a waiting thread for a lock in getLockEntry() 
 and another thread is releasing the lock by calling releaseLockEntry(), 
 trying to get the lock with a 3rd thread by calling getLockEntry() falls into 
 a busy loop until the waiting thread wakes up and gets the lock.
 Even if notify() wakes up the blocked thread and causes a context switch to 
 the waked thread immediately, synchronization might block the waked thread 
 and cause another context switch, and the busy loop might continue for a 
 while.
 (2) In the same case as (1), since releasing the lock is merely notifying 
 without removing the lock-entry from the map, interrupting the waiting thread 
 might leave an unused lock-entry (entry.numWaiters == 0) in the map. This is 
 a memory leak unless the id of the lock is used again.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-7184) pre-check table name in table.jsp

2012-11-19 Thread liang xie (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liang xie updated HBASE-7184:
-

Status: Patch Available  (was: Open)

 pre-check table name in table.jsp
 -

 Key: HBASE-7184
 URL: https://issues.apache.org/jira/browse/HBASE-7184
 Project: HBase
  Issue Type: Improvement
  Components: UI
Affects Versions: 0.94.2, 0.96.0
Reporter: liang xie
Assignee: liang xie
 Attachments: HBASE-7184.txt


 Currently the (table)name parameter in table.jsp isn't checked, it brings two 
 problems at least:
 1) 500 error for invalid value
 2) directly written to JSP output, giving reflected XSS vulnerability
 we can do a parameter-checking

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-7184) pre-check table name in table.jsp

2012-11-19 Thread liang xie (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liang xie updated HBASE-7184:
-

Attachment: HBASE-7184.txt

 pre-check table name in table.jsp
 -

 Key: HBASE-7184
 URL: https://issues.apache.org/jira/browse/HBASE-7184
 Project: HBase
  Issue Type: Improvement
  Components: UI
Affects Versions: 0.94.2, 0.96.0
Reporter: liang xie
Assignee: liang xie
 Attachments: HBASE-7184.txt


 Currently the (table)name parameter in table.jsp isn't checked, it brings two 
 problems at least:
 1) 500 error for invalid value
 2) directly written to JSP output, giving reflected XSS vulnerability
 we can do a parameter-checking

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Created] (HBASE-7184) pre-check table name in table.jsp

2012-11-19 Thread liang xie (JIRA)
liang xie created HBASE-7184:


 Summary: pre-check table name in table.jsp
 Key: HBASE-7184
 URL: https://issues.apache.org/jira/browse/HBASE-7184
 Project: HBase
  Issue Type: Improvement
  Components: UI
Affects Versions: 0.94.2, 0.96.0
Reporter: liang xie
Assignee: liang xie
 Attachments: HBASE-7184.txt

Currently the (table)name parameter in table.jsp isn't checked, it brings two 
problems at least:

1) 500 error for invalid value
2) directly written to JSP output, giving reflected XSS vulnerability

we can do a parameter-checking

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-7184) pre-check table name in table.jsp

2012-11-19 Thread liang xie (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500081#comment-13500081
 ] 

liang xie commented on HBASE-7184:
--

Before applied the attached patch, if i access host:port/table.jsp?name=... , 
will get :

HTTP ERROR 500

Problem accessing /table.jsp. Reason:

...

Caused by:

org.apache.hadoop.hbase.TableNotFoundException: ...
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:999)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:864)
at 
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:821)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234)
at org.apache.hadoop.hbase.client.HTable.init(HTable.java:174)
at org.apache.hadoop.hbase.client.HTable.init(HTable.java:133)
at 
org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:67)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at 
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at 
org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at 
org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1056)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at 
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at 
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at 
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at 
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at 
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at 
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at 
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at 
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at 
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at 
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

Powered by Jetty://




After applied patch, i got :
Illegal table name parameter!

 pre-check table name in table.jsp
 -

 Key: HBASE-7184
 URL: https://issues.apache.org/jira/browse/HBASE-7184
 Project: HBase
  Issue Type: Improvement
  Components: UI
Affects Versions: 0.94.2, 0.96.0
Reporter: liang xie
Assignee: liang xie
 Attachments: HBASE-7184.txt


 Currently the (table)name parameter in table.jsp isn't checked, it brings two 
 problems at least:
 1) 500 error for invalid value
 2) directly written to JSP output, giving reflected XSS vulnerability
 we can do a parameter-checking

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-7184) pre-check table name in table.jsp

2012-11-19 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500102#comment-13500102
 ] 

Hadoop QA commented on HBASE-7184:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12554136/HBASE-7184.txt
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
99 warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 findbugs{color}.  The patch appears to introduce 24 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.client.TestFromClientSide

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3364//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3364//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3364//console

This message is automatically generated.

 pre-check table name in table.jsp
 -

 Key: HBASE-7184
 URL: https://issues.apache.org/jira/browse/HBASE-7184
 Project: HBase
  Issue Type: Improvement
  Components: UI
Affects Versions: 0.94.2, 0.96.0
Reporter: liang xie
Assignee: liang xie
 Attachments: HBASE-7184.txt


 Currently the (table)name parameter in table.jsp isn't checked, it brings two 
 problems at least:
 1) 500 error for invalid value
 2) directly written to JSP output, giving reflected XSS vulnerability
 we can do a parameter-checking

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-2645) HLog writer can do 1-2 sync operations after lease has been recovered for split process.

2012-11-19 Thread Cosmin Lehene (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500108#comment-13500108
 ] 

Cosmin Lehene commented on HBASE-2645:
--

Wow, I think it was one of the longest lived blockers. Glad to see it fixed. 
Thanks :) 

 HLog writer can do 1-2 sync operations after lease has been recovered for 
 split process.
 

 Key: HBASE-2645
 URL: https://issues.apache.org/jira/browse/HBASE-2645
 Project: HBase
  Issue Type: Bug
  Components: Filters
Affects Versions: 0.90.4
Reporter: Cosmin Lehene
Assignee: Todd Lipcon
Priority: Blocker
 Fix For: 0.96.0

 Attachments: 2645-addendum.txt, 2645_hacking.txt, 2645.txt, 
 2645v2.txt, 2645v3.txt, 2645v4.txt, hdfs_1.0_editswriter_recoverlease.txt, 
 hdfs_trunk_editswriter_recoverlease.txt, 
 org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt, 
 org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit-output.txt


 TestHLogSplit.testLogCannotBeWrittenOnceParsed is failing. 
 This test starts a thread that writes one edit to the log, syncs and counts. 
 During this, a HLog.splitLog operation is started. splitLog recovers the log 
 lease before reading the log, so that the original regionserver could not 
 wake up and write after the split process started.  
 The test compares the number of edits reported by the split process and by 
 the writer thread. Writer thread (called zombie in the test) should report = 
  than the splitLog (sync() might raise after the last edit gets written and 
 the edit won't get counted by zombie thread). However it appears that the 
 zombie counts 1-2 more edits. So it looks like it can sync without a lease.
 This might be a hdfs-0.20 related issue. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-7184) pre-check table name in table.jsp

2012-11-19 Thread liang xie (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500112#comment-13500112
 ] 

liang xie commented on HBASE-7184:
--

emmm.  seems the failure test case is unrelated with this issue, IMHO...

 pre-check table name in table.jsp
 -

 Key: HBASE-7184
 URL: https://issues.apache.org/jira/browse/HBASE-7184
 Project: HBase
  Issue Type: Improvement
  Components: UI
Affects Versions: 0.94.2, 0.96.0
Reporter: liang xie
Assignee: liang xie
 Attachments: HBASE-7184.txt


 Currently the (table)name parameter in table.jsp isn't checked, it brings two 
 problems at least:
 1) 500 error for invalid value
 2) directly written to JSP output, giving reflected XSS vulnerability
 we can do a parameter-checking

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-6777) Snapshot Restore interface

2012-11-19 Thread Jesse Yates (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500325#comment-13500325
 ] 

Jesse Yates commented on HBASE-6777:


Committing today, unless there are objections

 Snapshot Restore interface
 --

 Key: HBASE-6777
 URL: https://issues.apache.org/jira/browse/HBASE-6777
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, snapshots
Affects Versions: 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055

 Attachments: hbase-6777-v9.patch, HBASE--v1.patch


 Add interfaces for restoring a snapshot

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-7035) thrift server closes HTable of open Scanners

2012-11-19 Thread Devin Bayer (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devin Bayer updated HBASE-7035:
---

Status: Patch Available  (was: Open)

 thrift server closes HTable of open Scanners
 

 Key: HBASE-7035
 URL: https://issues.apache.org/jira/browse/HBASE-7035
 Project: HBase
  Issue Type: Bug
  Components: Thrift
Affects Versions: 0.94.4
Reporter: Devin Bayer
  Labels: thrift2
   Original Estimate: 4h
  Remaining Estimate: 4h

 ThriftHBaseServiceHandler.openScanner() does this:
 1. table = pool.getTable()
 2. scanner = table.getScanner()
 3. table.close()
 4. return scanner
 While back porting the thrift server to 0.92.6, I found that table.close() 
 calls connection.close(). Further calls to scanner.next() raise a 
 ConnectionClosed exception. The unit tests do not catch this since they reuse 
 an open HConnection instance.
 This might work on trunk, but depends on the implementations of HTablePool, 
 HTable and HConnectionManager. Even with the pool wrapper, if the pool is 
 full, table.close() may be called, which may invalidate the table. Also,  
 HTable is not thread-safe, but they are being reused since they go back in 
 the pool.
 I suggest storing the table handle along with the scanner.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-7035) thrift server closes HTable of open Scanners

2012-11-19 Thread Devin Bayer (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devin Bayer updated HBASE-7035:
---

Attachment: old-hbase-thrift-v1.patch

 thrift server closes HTable of open Scanners
 

 Key: HBASE-7035
 URL: https://issues.apache.org/jira/browse/HBASE-7035
 Project: HBase
  Issue Type: Bug
  Components: Thrift
Affects Versions: 0.94.4
Reporter: Devin Bayer
  Labels: thrift2
 Attachments: old-hbase-thrift-v1.patch

   Original Estimate: 4h
  Remaining Estimate: 4h

 ThriftHBaseServiceHandler.openScanner() does this:
 1. table = pool.getTable()
 2. scanner = table.getScanner()
 3. table.close()
 4. return scanner
 While back porting the thrift server to 0.92.6, I found that table.close() 
 calls connection.close(). Further calls to scanner.next() raise a 
 ConnectionClosed exception. The unit tests do not catch this since they reuse 
 an open HConnection instance.
 This might work on trunk, but depends on the implementations of HTablePool, 
 HTable and HConnectionManager. Even with the pool wrapper, if the pool is 
 full, table.close() may be called, which may invalidate the table. Also,  
 HTable is not thread-safe, but they are being reused since they go back in 
 the pool.
 I suggest storing the table handle along with the scanner.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-7035) thrift server closes HTable of open Scanners

2012-11-19 Thread Devin Bayer (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500435#comment-13500435
 ] 

Devin Bayer commented on HBASE-7035:


stack: patch is attached. It does work for us.

 thrift server closes HTable of open Scanners
 

 Key: HBASE-7035
 URL: https://issues.apache.org/jira/browse/HBASE-7035
 Project: HBase
  Issue Type: Bug
  Components: Thrift
Affects Versions: 0.94.4
Reporter: Devin Bayer
  Labels: thrift2
 Attachments: old-hbase-thrift-v1.patch

   Original Estimate: 4h
  Remaining Estimate: 4h

 ThriftHBaseServiceHandler.openScanner() does this:
 1. table = pool.getTable()
 2. scanner = table.getScanner()
 3. table.close()
 4. return scanner
 While back porting the thrift server to 0.92.6, I found that table.close() 
 calls connection.close(). Further calls to scanner.next() raise a 
 ConnectionClosed exception. The unit tests do not catch this since they reuse 
 an open HConnection instance.
 This might work on trunk, but depends on the implementations of HTablePool, 
 HTable and HConnectionManager. Even with the pool wrapper, if the pool is 
 full, table.close() may be called, which may invalidate the table. Also,  
 HTable is not thread-safe, but they are being reused since they go back in 
 the pool.
 I suggest storing the table handle along with the scanner.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-6865) Snapshot File Cleaners

2012-11-19 Thread Jesse Yates (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesse Yates updated HBASE-6865:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to snapshot branch

 Snapshot File Cleaners
 --

 Key: HBASE-6865
 URL: https://issues.apache.org/jira/browse/HBASE-6865
 Project: HBase
  Issue Type: Sub-task
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055

 Attachments: hbase-6865-v2.patch


 HFile and HLog cleaners are necessary for fully implementing snapshots, but 
 can be broken out into their own piece really cleanly. The HFile cleaner is 
 necessary for both timestamp and globally consistent snapshots, but the HLog 
 cleaner is necessary for globally consistent and offline snapshots. 
 Putting everything together in one patch isn't too much overload as there is 
 a lot of overlap between the cleaners.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-7062) Move HLog stats to metrics 2

2012-11-19 Thread Elliott Clark (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-7062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-7062:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed revision 1411337.

 Move HLog stats to metrics 2
 

 Key: HBASE-7062
 URL: https://issues.apache.org/jira/browse/HBASE-7062
 Project: HBase
  Issue Type: Sub-task
  Components: metrics
Affects Versions: 0.96.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Critical
 Fix For: 0.96.0

 Attachments: HBASE-7062-1.patch, HBASE-7062-2.patch, 
 HBASE-7062-3.patch, HBASE-7062-4.patch, HBASE-7062-5.patch, HBASE-7062-6.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Created] (HBASE-7185) Create hbase-rpc module

2012-11-19 Thread Elliott Clark (JIRA)
Elliott Clark created HBASE-7185:


 Summary: Create hbase-rpc module
 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Created] (HBASE-7186) Split Classes for Client/Server module split.

2012-11-19 Thread Elliott Clark (JIRA)
Elliott Clark created HBASE-7186:


 Summary: Split Classes for Client/Server module split.
 Key: HBASE-7186
 URL: https://issues.apache.org/jira/browse/HBASE-7186
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-7148) Some files in hbase-examples module miss license header

2012-11-19 Thread Sergey Shelukhin (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500454#comment-13500454
 ] 

Sergey Shelukhin commented on HBASE-7148:
-

Sorry, my mistake; I was away last week. Thanks Enis for taking care of this!

 Some files in hbase-examples module miss license header
 ---

 Key: HBASE-7148
 URL: https://issues.apache.org/jira/browse/HBASE-7148
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Enis Soztutar
 Fix For: 0.96.0

 Attachments: hbase-7148.patch


 Trunk build 3530 got to building hbase-examples module but failed:
 {code}
 [INFO] HBase - Examples .. FAILURE [3.222s]
 [INFO] 
 
 [INFO] BUILD FAILURE
 [INFO] 
 
 [INFO] Total time: 29:21.569s
 [INFO] Finished at: Sun Nov 11 15:17:35 UTC 2012
 [INFO] Final Memory: 68M/642M
 [INFO] 
 
 [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.8:check 
 (default) on project hbase-examples: Too many unapproved licenses: 20 - 
 [Help 1]
 {code}
 Looks like license headers are missing in some of the files in hbase-examples 
 module

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Created] (HBASE-7187) Create empty hbase-client module

2012-11-19 Thread Elliott Clark (JIRA)
Elliott Clark created HBASE-7187:


 Summary: Create empty hbase-client module
 Key: HBASE-7187
 URL: https://issues.apache.org/jira/browse/HBASE-7187
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Created] (HBASE-7188) Move classes into hbase-client

2012-11-19 Thread Elliott Clark (JIRA)
Elliott Clark created HBASE-7188:


 Summary: Move classes into hbase-client
 Key: HBASE-7188
 URL: https://issues.apache.org/jira/browse/HBASE-7188
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-7012) Create hbase-client module

2012-11-19 Thread Elliott Clark (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500459#comment-13500459
 ] 

Elliott Clark commented on HBASE-7012:
--

Sure I'll give it a shot.  Some of the changes are when splitting the classes 
only become really evident after the move.  Other than that it should be pretty 
easy.

 Create hbase-client module
 --

 Key: HBASE-7012
 URL: https://issues.apache.org/jira/browse/HBASE-7012
 Project: HBase
  Issue Type: Task
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Critical
 Fix For: 0.96.0

 Attachments: HBASE-7012-0.patch.gz


 I just tried creating a project that uses 0.95-SNAPSHOT and had to import 
 org.apache.hbase:hbase-server as the module that I depend on.  This will be 
 confusing to users.
 In addition this brings in lots of dependencies that are not really needed.
 Let's create a client module that has all of the client in it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores

2012-11-19 Thread Jimmy Xiang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6423:
---

Assignee: Jimmy Xiang  (was: Amitanand Aiyer)
  Status: Open  (was: Patch Available)

 Writes should not block reads on blocking updates to memstores
 --

 Key: HBASE-6423
 URL: https://issues.apache.org/jira/browse/HBASE-6423
 Project: HBase
  Issue Type: Bug
Reporter: Karthik Ranganathan
Assignee: Jimmy Xiang
 Attachments: trunk-6423.patch, trunk-6423_v2.patch


 We have a big data use case where we turn off WAL and have a ton of reads and 
 writes. We found that:
 1. flushing a memstore takes a while (GZIP compression)
 2. incoming writes cause the new memstore to grow in an unbounded fashion
 3. this triggers blocking memstore updates
 4. in turn, this causes all the RPC handler threads to block on writes to 
 that memstore
 5. we are not able to read during this time as RPC handlers are blocked
 At a higher level, we should not hold up the RPC threads while blocking 
 updates, and we should build in some sort of rate control.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores

2012-11-19 Thread Jimmy Xiang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6423:
---

Attachment: trunk-6423_v2.patch

 Writes should not block reads on blocking updates to memstores
 --

 Key: HBASE-6423
 URL: https://issues.apache.org/jira/browse/HBASE-6423
 Project: HBase
  Issue Type: Bug
Reporter: Karthik Ranganathan
Assignee: Jimmy Xiang
 Attachments: trunk-6423.patch, trunk-6423_v2.patch


 We have a big data use case where we turn off WAL and have a ton of reads and 
 writes. We found that:
 1. flushing a memstore takes a while (GZIP compression)
 2. incoming writes cause the new memstore to grow in an unbounded fashion
 3. this triggers blocking memstore updates
 4. in turn, this causes all the RPC handler threads to block on writes to 
 that memstore
 5. we are not able to read during this time as RPC handlers are blocked
 At a higher level, we should not hold up the RPC threads while blocking 
 updates, and we should build in some sort of rate control.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores

2012-11-19 Thread Jimmy Xiang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6423:
---

Status: Patch Available  (was: Open)

Addressed Ted's comments. Fixed TestHeapSize.

 Writes should not block reads on blocking updates to memstores
 --

 Key: HBASE-6423
 URL: https://issues.apache.org/jira/browse/HBASE-6423
 Project: HBase
  Issue Type: Bug
Reporter: Karthik Ranganathan
Assignee: Jimmy Xiang
 Attachments: trunk-6423.patch, trunk-6423_v2.patch


 We have a big data use case where we turn off WAL and have a ton of reads and 
 writes. We found that:
 1. flushing a memstore takes a while (GZIP compression)
 2. incoming writes cause the new memstore to grow in an unbounded fashion
 3. this triggers blocking memstore updates
 4. in turn, this causes all the RPC handler threads to block on writes to 
 that memstore
 5. we are not able to read during this time as RPC handlers are blocked
 At a higher level, we should not hold up the RPC threads while blocking 
 updates, and we should build in some sort of rate control.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-6423) Writes should not block reads on blocking updates to memstores

2012-11-19 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500477#comment-13500477
 ] 

Hadoop QA commented on HBASE-6423:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12554204/trunk-6423_v2.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 5 new 
or modified tests.

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3366//console

This message is automatically generated.

 Writes should not block reads on blocking updates to memstores
 --

 Key: HBASE-6423
 URL: https://issues.apache.org/jira/browse/HBASE-6423
 Project: HBase
  Issue Type: Bug
Reporter: Karthik Ranganathan
Assignee: Jimmy Xiang
 Attachments: trunk-6423.patch, trunk-6423_v2.patch


 We have a big data use case where we turn off WAL and have a ton of reads and 
 writes. We found that:
 1. flushing a memstore takes a while (GZIP compression)
 2. incoming writes cause the new memstore to grow in an unbounded fashion
 3. this triggers blocking memstore updates
 4. in turn, this causes all the RPC handler threads to block on writes to 
 that memstore
 5. we are not able to read during this time as RPC handlers are blocked
 At a higher level, we should not hold up the RPC threads while blocking 
 updates, and we should build in some sort of rate control.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-7185) Create hbase-rpc module

2012-11-19 Thread Elliott Clark (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-7185:
-

Attachment: HBASE-7185-0.patch

Move Protobuf files into hbase-rpc module.

This new module should make it easy for others to interact with hbase-server 
without depending on much from hbase. Eg this could be used by something like 
async hbase.

 Create hbase-rpc module
 ---

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-7185) Create hbase-rpc module

2012-11-19 Thread Elliott Clark (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-7185:
-

Status: Patch Available  (was: Open)

 Create hbase-rpc module
 ---

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96

2012-11-19 Thread Jonathan Hsieh (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500491#comment-13500491
 ] 

Jonathan Hsieh commented on HBASE-6055:
---

I've reached a pretty decent point on this branch now refactoring and 
simplifying the online snapshot code.  Most of the work is in the plumbing, and 
most of the complexity remains in the plumbing.   Here is a link

https://github.com/jmhsieh/hbase/commits/snapshot-dev?

It isn't worth looking at the individual patches -- if interested take a look 
at the code generally one package at a time.  I'd suggest starting from 
errorhandling, then procedures since this work is fairly isolated and stable 
now.  The will likely affect existing code, and online may change significantly 
during the merge process.

Error handling:  o.a.h.h.errorhandling.*
* Added concept of ExternalException (an exception from an separate thread or 
process).
* Removing generics by funneling everything through an ExternalException
* Simplified Exception Propagation by only having a Dispatcher, Listener, and 
Checker. (No Visitors, Orchestrators, some Factories)
* Made Exception Serialization static so that instances don't need to be passed 
around.
* Added more meaningful usage and motivation documentation.

Procedure framework: o.a.h.h.procedure.*
* Separated Coordinator side from Member side
* Reduced the number of constructors (and fewer objects threaded throughout).
* Added concept of Procedure and Subprocedure -- these maintain state on each 
host. (this replaces just using strings everywhere).
* Folded several threads that used latch ping-pong into single threads.
* Renamed methods from 2pc nomenclature to barrier nomenclature.
* Added more meaningful usage and motivation documentation.

Online Snapshots: o.a.h.h.snapshot.*
* Converted per regionserver only Procedures to simpler Callable/Future fork 
join implementation.
* Removed different *ErrorHandlers and moved into Subprocedures. (this may be 
further eliminated)
* Each Procedure contains an ExternalExceptionDispatcher
* ExternalExceptions go to the SnapshotManager to abort the Procedure.

I'm in process of merging code into the offline snapshot branch.  It isn't 
clean but I'll be working on that for the next few days. (Clashes with updates 
in offline snapshot).   Once I get the snapshot branch compiling again, I'll 
start posting the External Exception and Procedure stuff as a series of patches.

My suggestion for the overall effort is to get the main offline snapshot branch 
code committed to the branch and then start looking into merging with trunk and 
0.94.  The online work I feel should remain a branch until its pieces are 
fleshed out.


 Snapshots in HBase 0.96
 ---

 Key: HBASE-6055
 URL: https://issues.apache.org/jira/browse/HBASE-6055
 Project: HBase
  Issue Type: New Feature
  Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055, 0.96.0

 Attachments: Snapshots in HBase.docx


 Continuation of HBASE-50 for the current trunk. Since the implementation has 
 drastically changed, opening as a new ticket.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-7185) Create hbase-rpc module

2012-11-19 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500502#comment-13500502
 ] 

Hadoop QA commented on HBASE-7185:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12554208/HBASE-7185-0.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
99 warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 findbugs{color}.  The patch appears to introduce 24 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3367//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-rpc.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3367//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3367//console

This message is automatically generated.

 Create hbase-rpc module
 ---

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-7185) Create hbase-rpc module

2012-11-19 Thread Elliott Clark (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-7185:
-

Attachment: HBASE-7185-1.patch

Missed a commit that cleaned up the rpc pom to not depend upon resource 
listener.

 Create hbase-rpc module
 ---

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-7185) Create hbase-rpc module

2012-11-19 Thread Elliott Clark (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-7185:
-

Attachment: (was: HBASE-7185-1.patch)

 Create hbase-rpc module
 ---

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-7185) Create hbase-rpc module

2012-11-19 Thread Elliott Clark (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-7185:
-

Attachment: HBASE-7185-1.patch

 Create hbase-rpc module
 ---

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-7062) Move HLog stats to metrics 2

2012-11-19 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500524#comment-13500524
 ] 

Hudson commented on HBASE-7062:
---

Integrated in HBase-TRUNK #3553 (See 
[https://builds.apache.org/job/HBase-TRUNK/3553/])
HBASE-7062 Move HLog stats to metrics 2 (Revision 1411337)

 Result = FAILURE
eclark : 
Files : 
* 
/hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSource.java
* 
/hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal
* 
/hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWALSource.java
* 
/hbase/trunk/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal
* 
/hbase/trunk/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestMetricsHLogSource.java
* 
/hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java
* 
/hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSourceImpl.java
* 
/hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
* 
/hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal
* 
/hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWALSourceImpl.java
* 
/hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationSourceImpl.java
* 
/hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/rest/MetricsRESTSourceImpl.java
* 
/hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSourceImpl.java
* 
/hbase/trunk/hbase-hadoop1-compat/src/main/resources/META-INF/services/org.apache.hadoop.hbase.regionserver.wal.MetricsWALSource
* 
/hbase/trunk/hbase-hadoop1-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal
* 
/hbase/trunk/hbase-hadoop1-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestMetricsWALSourceImpl.java
* 
/hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java
* 
/hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/metrics/BaseSourceImpl.java
* 
/hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
* 
/hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal
* 
/hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWALSourceImpl.java
* 
/hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsReplicationSourceImpl.java
* 
/hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/rest/MetricsRESTSourceImpl.java
* 
/hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/thrift/MetricsThriftServerSourceImpl.java
* 
/hbase/trunk/hbase-hadoop2-compat/src/main/resources/META-INF/services/org.apache.hadoop.hbase.regionserver.wal.MetricsWALSource
* 
/hbase/trunk/hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal
* 
/hbase/trunk/hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestMetricsWALSourceImpl.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogMetrics.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWAL.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java


 Move HLog stats to metrics 2
 

 Key: HBASE-7062
 URL: https://issues.apache.org/jira/browse/HBASE-7062
 Project: HBase
  Issue Type: Sub-task
  Components: metrics
Affects Versions: 0.96.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Critical
 Fix For: 0.96.0

 Attachments: HBASE-7062-1.patch, HBASE-7062-2.patch, 
 HBASE-7062-3.patch, HBASE-7062-4.patch, HBASE-7062-5.patch, HBASE-7062-6.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-7009) Port HBaseCluster interface/tests to 0.94

2012-11-19 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500525#comment-13500525
 ] 

stack commented on HBASE-7009:
--

So, is this good to go then lads?

 Port HBaseCluster interface/tests to 0.94
 -

 Key: HBASE-7009
 URL: https://issues.apache.org/jira/browse/HBASE-7009
 Project: HBase
  Issue Type: Sub-task
  Components: test
Affects Versions: 0.94.3
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Fix For: 0.94.4

 Attachments: HBASE-7009-p1.patch, HBASE-7009.patch, 
 HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch


 Need to port. I am porting V5 patch from the original JIRA; I have a 
 partially ported (V3) patch from Enis with protocol buffers being reverted to 
 HRegionInterface/HMasterInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96

2012-11-19 Thread Jonathan Hsieh (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500532#comment-13500532
 ] 

Jonathan Hsieh commented on HBASE-6055:
---

Branch to a squashed version. 
https://github.com/jmhsieh/hbase/tree/snapshot-dev-squash


 Snapshots in HBase 0.96
 ---

 Key: HBASE-6055
 URL: https://issues.apache.org/jira/browse/HBASE-6055
 Project: HBase
  Issue Type: New Feature
  Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055, 0.96.0

 Attachments: Snapshots in HBase.docx


 Continuation of HBASE-50 for the current trunk. Since the implementation has 
 drastically changed, opening as a new ticket.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores

2012-11-19 Thread Jimmy Xiang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6423:
---

Status: Open  (was: Patch Available)

 Writes should not block reads on blocking updates to memstores
 --

 Key: HBASE-6423
 URL: https://issues.apache.org/jira/browse/HBASE-6423
 Project: HBase
  Issue Type: Bug
Reporter: Karthik Ranganathan
Assignee: Jimmy Xiang
 Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, 
 trunk-6423_v2.patch


 We have a big data use case where we turn off WAL and have a ton of reads and 
 writes. We found that:
 1. flushing a memstore takes a while (GZIP compression)
 2. incoming writes cause the new memstore to grow in an unbounded fashion
 3. this triggers blocking memstore updates
 4. in turn, this causes all the RPC handler threads to block on writes to 
 that memstore
 5. we are not able to read during this time as RPC handlers are blocked
 At a higher level, we should not hold up the RPC threads while blocking 
 updates, and we should build in some sort of rate control.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores

2012-11-19 Thread Jimmy Xiang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6423:
---

Attachment: trunk-6423_v2.1.patch

 Writes should not block reads on blocking updates to memstores
 --

 Key: HBASE-6423
 URL: https://issues.apache.org/jira/browse/HBASE-6423
 Project: HBase
  Issue Type: Bug
Reporter: Karthik Ranganathan
Assignee: Jimmy Xiang
 Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, 
 trunk-6423_v2.patch


 We have a big data use case where we turn off WAL and have a ton of reads and 
 writes. We found that:
 1. flushing a memstore takes a while (GZIP compression)
 2. incoming writes cause the new memstore to grow in an unbounded fashion
 3. this triggers blocking memstore updates
 4. in turn, this causes all the RPC handler threads to block on writes to 
 that memstore
 5. we are not able to read during this time as RPC handlers are blocked
 At a higher level, we should not hold up the RPC threads while blocking 
 updates, and we should build in some sort of rate control.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores

2012-11-19 Thread Jimmy Xiang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6423:
---

Status: Patch Available  (was: Open)

Rebased to trunk latest.

 Writes should not block reads on blocking updates to memstores
 --

 Key: HBASE-6423
 URL: https://issues.apache.org/jira/browse/HBASE-6423
 Project: HBase
  Issue Type: Bug
Reporter: Karthik Ranganathan
Assignee: Jimmy Xiang
 Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, 
 trunk-6423_v2.patch


 We have a big data use case where we turn off WAL and have a ton of reads and 
 writes. We found that:
 1. flushing a memstore takes a while (GZIP compression)
 2. incoming writes cause the new memstore to grow in an unbounded fashion
 3. this triggers blocking memstore updates
 4. in turn, this causes all the RPC handler threads to block on writes to 
 that memstore
 5. we are not able to read during this time as RPC handlers are blocked
 At a higher level, we should not hold up the RPC threads while blocking 
 updates, and we should build in some sort of rate control.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96

2012-11-19 Thread Jesse Yates (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500557#comment-13500557
 ] 

Jesse Yates commented on HBASE-6055:


I just +1'ed Matteo's restore changes, so I hope to roll that into the snapshot 
branch (https://github.com/jyates/hbase/tree/snapshots) in the next day or two, 
which would mean we have everything for offline snapshots.

The offline snapshots themselves are a self contained bit of code and sizable 
enough to make it worthwhile to roll it into trunk on its own. 

Let's keep the online keep on a branch until its wrapped up - I'll start taking 
a look at the online when I have a chance, and look forward to code posted on 
RB. The progress sounds real sweet Jon!


TL;DR lets do what Jon suggests above, especially as offline snapshots are 
basically done and ready to merge into trunk

 Snapshots in HBase 0.96
 ---

 Key: HBASE-6055
 URL: https://issues.apache.org/jira/browse/HBASE-6055
 Project: HBase
  Issue Type: New Feature
  Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055, 0.96.0

 Attachments: Snapshots in HBase.docx


 Continuation of HBASE-50 for the current trunk. Since the implementation has 
 drastically changed, opening as a new ticket.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96

2012-11-19 Thread Jonathan Hsieh (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500578#comment-13500578
 ] 

Jonathan Hsieh commented on HBASE-6055:
---

Just force pushed the snapshot-dev-squash branch -- it didn't delete files that 
were supposed to be gone.


 Snapshots in HBase 0.96
 ---

 Key: HBASE-6055
 URL: https://issues.apache.org/jira/browse/HBASE-6055
 Project: HBase
  Issue Type: New Feature
  Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055, 0.96.0

 Attachments: Snapshots in HBase.docx


 Continuation of HBASE-50 for the current trunk. Since the implementation has 
 drastically changed, opening as a new ticket.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-5422) StartupBulkAssigner would cause a lot of timeout on RIT when assigning large numbers of regions (timeout = 3 mins)

2012-11-19 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500592#comment-13500592
 ] 

stack commented on HBASE-5422:
--

I committed with an issue number of HBASE-5542 by mistake.  I just corrected it 
by doing the following:

{code}
$ svn propset -r1292409 --revprop svn:log HBASE-5422 StartupBulkAssigner would 
cause a lot of timeout on RIT when assigning large numbers of regions (timeout 
= 3 mins)
{code}

I did it in 0.92, 0.94 and trunk.

 StartupBulkAssigner would cause a lot of timeout on RIT when assigning large 
 numbers of regions (timeout = 3 mins)
 --

 Key: HBASE-5422
 URL: https://issues.apache.org/jira/browse/HBASE-5422
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: chunhui shen
Assignee: chunhui shen
 Fix For: 0.92.1

 Attachments: 5422-90.patch, 5422-90v2.patch, 5422-90v3.patch, 
 5422-v3.txt, hbase-5422.patch, hbase-5422v2.patch, hbase-5422v3.patch


 In our produce environment
 We find a lot of timeout on RIT when cluster up, there are about 7w regions 
 in the cluster( 25 regionservers ).
 First, we could see the following log:(See the region 
 33cf229845b1009aa8a3f7b0f85c9bd0)
 master's log
 2012-02-13 18:07:41,409 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:6-0x348f4a94723da5 Async create of unassigned node for 
 33cf229845b1009aa8a3f7b0f85c9bd0 with OFFLINE state 
 2012-02-13 18:07:42,560 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager$CreateUnassignedAsyncCallback:
  rs=item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. 
 state=OFFLINE, ts=1329127661409, 
 server=r03f11025.yh.aliyun.com,60020,1329127549907 
 2012-02-13 18:07:42,996 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager$ExistsUnassignedAsyncCallback:
  rs=item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. 
 state=OFFLINE, ts=1329127661409 
 2012-02-13 18:10:48,072 INFO 
 org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed 
 out: item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. 
 state=PENDING_OPEN, ts=1329127662996
 2012-02-13 18:10:48,072 INFO 
 org.apache.hadoop.hbase.master.AssignmentManager: Region has been 
 PENDING_OPEN for too long, reassigning 
 region=item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. 
 2012-02-13 18:11:16,744 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: Handling 
 transition=RS_ZK_REGION_OPENED, 
 server=r03f11025.yh.aliyun.com,60020,1329127549907, 
 region=33cf229845b1009aa8a3f7b0f85c9bd0 
 2012-02-13 18:38:07,310 DEBUG 
 org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: Handling OPENED 
 event for 33cf229845b1009aa8a3f7b0f85c9bd0; deleting unassigned node 
 2012-02-13 18:38:07,310 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:6-0x348f4a94723da5 Deleting existing unassigned node for 
 33cf229845b1009aa8a3f7b0f85c9bd0 that is in expected state 
 RS_ZK_REGION_OPENED 
 2012-02-13 18:38:07,314 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: 
 master:6-0x348f4a94723da5 Successfully deleted unassigned node for region 
 33cf229845b1009aa8a3f7b0f85c9bd0 in expected state RS_ZK_REGION_OPENED 
 2012-02-13 18:38:07,573 DEBUG 
 org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: Opened region 
 item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. on 
 r03f11025.yh.aliyun.com,60020,1329127549907 
 2012-02-13 18:50:54,428 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan 
 was found (or we are ignoring an existing plan) for 
 item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. so 
 generated a random one; 
 hri=item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0., 
 src=, dest=r01b05043.yh.aliyun.com,60020,1329127549041; 29 (online=29, 
 exclude=null) available servers 
 2012-02-13 18:50:54,428 DEBUG 
 org.apache.hadoop.hbase.master.AssignmentManager: Assigning region 
 item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. to 
 r01b05043.yh.aliyun.com,60020,1329127549041 
 2012-02-13 19:31:50,514 INFO 
 org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed 
 out: item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. 
 state=PENDING_OPEN, ts=1329132528086 
 2012-02-13 19:31:50,514 INFO 
 org.apache.hadoop.hbase.master.AssignmentManager: Region has been 
 PENDING_OPEN for too long, reassigning 
 region=item_20120208,\x009,1328794343859.33cf229845b1009aa8a3f7b0f85c9bd0. 
 Regionserver's log
 2012-02-13 18:07:43,537 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: Received request to open 
 region: 

[jira] [Updated] (HBASE-6230) [brainstorm] Restore snapshots for HBase 0.96

2012-11-19 Thread Matteo Bertozzi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-6230:
---

Attachment: HBASE-6230-v2.patch

 [brainstorm] Restore snapshots for HBase 0.96
 ---

 Key: HBASE-6230
 URL: https://issues.apache.org/jira/browse/HBASE-6230
 Project: HBase
  Issue Type: Brainstorming
Reporter: Jesse Yates
Assignee: Matteo Bertozzi
 Attachments: HBASE-6230-v1.patch, HBASE-6230-v2.patch, 
 SnapshotRestore-v0.pdf


 Discussion ticket around the definitions/expectations of different parts of 
 snapshot restoration.  This is complementary, but separate from the _how_ of 
 taking a snapshot of a table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-6423) Writes should not block reads on blocking updates to memstores

2012-11-19 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500632#comment-13500632
 ] 

Hadoop QA commented on HBASE-6423:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12554219/trunk-6423_v2.1.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 5 new 
or modified tests.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
99 warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 findbugs{color}.  The patch appears to introduce 22 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
  
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3369//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3369//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3369//console

This message is automatically generated.

 Writes should not block reads on blocking updates to memstores
 --

 Key: HBASE-6423
 URL: https://issues.apache.org/jira/browse/HBASE-6423
 Project: HBase
  Issue Type: Bug
Reporter: Karthik Ranganathan
Assignee: Jimmy Xiang
 Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, 
 trunk-6423_v2.patch


 We have a big data use case where we turn off WAL and have a ton of reads and 
 writes. We found that:
 1. flushing a memstore takes a while (GZIP compression)
 2. incoming writes cause the new memstore to grow in an unbounded fashion
 3. this triggers blocking memstore updates
 4. in turn, this causes all the RPC handler threads to block on writes to 
 that memstore
 5. we are not able to read during this time as RPC handlers are blocked
 At a higher level, we should not hold up the RPC threads while blocking 
 updates, and we should build in some sort of rate control.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-7185) Create hbase-rpc module

2012-11-19 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500631#comment-13500631
 ] 

Hadoop QA commented on HBASE-7185:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12554216/HBASE-7185-1.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 
2.0 profile.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
99 warning messages.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 findbugs{color}.  The patch appears to introduce 24 new 
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3368//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-rpc.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3368//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/3368//console

This message is automatically generated.

 Create hbase-rpc module
 ---

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (HBASE-7185) Create hbase-rpc module

2012-11-19 Thread Elliott Clark (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500634#comment-13500634
 ] 

Elliott Clark commented on HBASE-7185:
--

Seems like all tests passed even though it timed out.

 Create hbase-rpc module
 ---

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HBASE-7180) RegionScannerImpl.next() is inefficient.

2012-11-19 Thread Lars Hofhansl (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-7180:
-

Attachment: 7180-0.94-v1.txt

Slightly better patch.
Also changes the call in RegionServer to use the cheaper version of next().

Looking around at the code, we can also replace all the calls from the 
AggregationImplementation to use this cheaper next() method.


 RegionScannerImpl.next() is inefficient.
 

 Key: HBASE-7180
 URL: https://issues.apache.org/jira/browse/HBASE-7180
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
 Attachments: 7180-0.94-SKETCH.txt, 7180-0.94-v1.txt


 We just came across a special scenario.
 For our Phoenix project (SQL runtime for HBase), we push a lot of work into 
 HBase via coprocessors. One method is to wrap RegionScanner in coprocessor 
 hooks and then do processing in the hook to avoid returning a lot of data to 
 the client unnecessarily.
 In this specific case this is pretty bad. Since the wrapped RegionScanner's 
 next() does not know that it is called this way is still does all of this 
 on each invocation:
 # Starts a RegionOperation
 # Increments the request count
 # set the current read point on a thread local (because generally each call 
 could come from a different thread)
 # Finally does the next on its StoreScanner(s)
 # Ends the RegionOperation
 When this is done in a tight loop millions of times (as is the case for us) 
 it starts to become significant.
 Not sure what to do about this, really. Opening this issue for discussion.
 One way is to extend the RegionScanner with an internal next() method of 
 sorts, so that all this overhead can be avoided. The coprocessor could call 
 the regular next() methods once and then just call the cheaper internal 
 version.
 Are there better/cleaner ways?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira