[jira] [Updated] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the region assignment inconsistent
[
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5200:
--
Comment: was deleted
(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12513158/5200-v2.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
-1 javadoc. The javadoc tool appears to have generated -136 warning
messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 155 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.TestInfoServers
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.io.hfile.TestHFileBlock
org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/898//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/898//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/898//console
This message is automatically generated.)
> AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the
> region assignment inconsistent
> -
>
> Key: HBASE-5200
> URL: https://issues.apache.org/jira/browse/HBASE-5200
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.94.0, 0.90.7, 0.92.1
>
> Attachments: 5200-v2.txt, HBASE-5200.patch, HBASE-5200_1.patch,
> TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml
>
>
> This is the scenario
> Consider a case where the balancer is going on thus trying to close regions
> in a RS.
> Before we could close a master switch happens.
> On Master switch the set of nodes that are in RIT is collected and we first
> get Data and start watching the node
> After that the node data is added into RIT.
> Now by this time (before adding to RIT) if the RS to which close was called
> does a transition in AM.handleRegion() we miss the handling saying RIT state
> was null.
> {code}
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> a66d281d231dfcaea97c270698b26b6f from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> c12e53bfd48ddc5eec507d66821c4d23 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 59ae13de8c1eb325a0dd51f4902d2052 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> f45bc9614d7575f35244849af85aa078 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> cc3ecd7054fe6cd4a1159ed92fd62641 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 3af40478a17fee96b4a192b22c90d5a2 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.Assig
[jira] [Updated] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the region assignment inconsistent
[
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5200:
--
Attachment: 5200-v2.txt
Re-attach patch v2 for Hadoop QA.
> AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the
> region assignment inconsistent
> -
>
> Key: HBASE-5200
> URL: https://issues.apache.org/jira/browse/HBASE-5200
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.94.0, 0.90.7, 0.92.1
>
> Attachments: 5200-v2.txt, HBASE-5200.patch, HBASE-5200_1.patch,
> TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml
>
>
> This is the scenario
> Consider a case where the balancer is going on thus trying to close regions
> in a RS.
> Before we could close a master switch happens.
> On Master switch the set of nodes that are in RIT is collected and we first
> get Data and start watching the node
> After that the node data is added into RIT.
> Now by this time (before adding to RIT) if the RS to which close was called
> does a transition in AM.handleRegion() we miss the handling saying RIT state
> was null.
> {code}
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> a66d281d231dfcaea97c270698b26b6f from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> c12e53bfd48ddc5eec507d66821c4d23 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 59ae13de8c1eb325a0dd51f4902d2052 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> f45bc9614d7575f35244849af85aa078 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> cc3ecd7054fe6cd4a1159ed92fd62641 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 3af40478a17fee96b4a192b22c90d5a2 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> e6096a8466e730463e10d3d61f809b92 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 4806781a1a23066f7baed22b4d237e24 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> d69e104131accaefe21dcc01fddc7629 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> {code}
> In branch the CLOSING node is created by RS thus leading to more
> inconsistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the region assignment inconsistent
[
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5200:
--
Attachment: (was: 5200-v2.txt)
> AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the
> region assignment inconsistent
> -
>
> Key: HBASE-5200
> URL: https://issues.apache.org/jira/browse/HBASE-5200
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.94.0, 0.90.7, 0.92.1
>
> Attachments: 5200-v2.txt, HBASE-5200.patch, HBASE-5200_1.patch,
> TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml
>
>
> This is the scenario
> Consider a case where the balancer is going on thus trying to close regions
> in a RS.
> Before we could close a master switch happens.
> On Master switch the set of nodes that are in RIT is collected and we first
> get Data and start watching the node
> After that the node data is added into RIT.
> Now by this time (before adding to RIT) if the RS to which close was called
> does a transition in AM.handleRegion() we miss the handling saying RIT state
> was null.
> {code}
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> a66d281d231dfcaea97c270698b26b6f from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> c12e53bfd48ddc5eec507d66821c4d23 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 59ae13de8c1eb325a0dd51f4902d2052 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> f45bc9614d7575f35244849af85aa078 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> cc3ecd7054fe6cd4a1159ed92fd62641 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 3af40478a17fee96b4a192b22c90d5a2 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> e6096a8466e730463e10d3d61f809b92 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 4806781a1a23066f7baed22b4d237e24 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> d69e104131accaefe21dcc01fddc7629 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> {code}
> In branch the CLOSING node is created by RS thus leading to more
> inconsistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5186) Add metrics to ThriftServer
[ https://issues.apache.org/jira/browse/HBASE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5186: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Add metrics to ThriftServer > --- > > Key: HBASE-5186 > URL: https://issues.apache.org/jira/browse/HBASE-5186 > Project: HBase > Issue Type: Improvement >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.94.0 > > Attachments: 5186-v10.txt, 5186-v11.txt, 5186-v12.txt, 5186-v9.txt, > HBASE-5186.D1461.1.patch, HBASE-5186.D1461.2.patch, HBASE-5186.D1461.3.patch, > HBASE-5186.D1461.4.patch, HBASE-5186.D1461.5.patch, HBASE-5186.D1461.6.patch, > HBASE-5186.D1461.7.patch, HBASE-5186.D1461.8.patch > > > It will be useful to have some metrics (queue length, waiting time, > processing time ...) similar to Hadoop RPC server. This allows us to monitor > system health also provide a tool to diagnose the problem where thrift calls > are slow. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the region assignment inconsistent
[
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5200:
--
Attachment: 5200-v2.txt
Patch v2 incorporates review comments
> AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the
> region assignment inconsistent
> -
>
> Key: HBASE-5200
> URL: https://issues.apache.org/jira/browse/HBASE-5200
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.94.0, 0.90.7, 0.92.1
>
> Attachments: 5200-v2.txt, HBASE-5200.patch, HBASE-5200_1.patch,
> TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml
>
>
> This is the scenario
> Consider a case where the balancer is going on thus trying to close regions
> in a RS.
> Before we could close a master switch happens.
> On Master switch the set of nodes that are in RIT is collected and we first
> get Data and start watching the node
> After that the node data is added into RIT.
> Now by this time (before adding to RIT) if the RS to which close was called
> does a transition in AM.handleRegion() we miss the handling saying RIT state
> was null.
> {code}
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> a66d281d231dfcaea97c270698b26b6f from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> c12e53bfd48ddc5eec507d66821c4d23 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 59ae13de8c1eb325a0dd51f4902d2052 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> f45bc9614d7575f35244849af85aa078 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> cc3ecd7054fe6cd4a1159ed92fd62641 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 3af40478a17fee96b4a192b22c90d5a2 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> e6096a8466e730463e10d3d61f809b92 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 4806781a1a23066f7baed22b4d237e24 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> d69e104131accaefe21dcc01fddc7629 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> {code}
> In branch the CLOSING node is created by RS thus leading to more
> inconsistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked
[
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5329:
--
Summary: addRowLock() may allocate duplicate lock id, causing the client to
be blocked (was: when client call lockRow,region server may allocate
duplicated lock id, this may cause that client is blocked.)
> addRowLock() may allocate duplicate lock id, causing the client to be blocked
> -
>
> Key: HBASE-5329
> URL: https://issues.apache.org/jira/browse/HBASE-5329
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.3
> Environment: Red Hat Enterprise Linux Server release 5.4
>Reporter: liaoxiangui
>Priority: Critical
>
> {code}
> protected long addRowLock(Integer r, HRegion region) throws
> LeaseStillHeldException
> {
> long lockId = -1L;
> lockId = rand.nextLong(); //!!!may generate duplicated
> id，bug?
> String lockName = String.valueOf(lockId);
> rowlocks.put(lockName, r);
> this.leases.createLease(lockName, new RowLockListener(lockName,
> region));
> return lockId;
> }
> {code}
> In addRowLock(),rand may generate duplicated lock id, it may cause
> regionserver throw exception(Leases$LeaseStillHeldException).The client will
> be blocked until old rowlock is released.
> {code}
> 2012-02-03 15:21:50,084 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
> (fsOk: true)
> org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
> at
> org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5329) when client call lockRow,region server may allocate duplicated lock id, this may cause that client is blocked.
[
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5329:
--
Priority: Critical (was: Major)
> when client call lockRow,region server may allocate duplicated lock id, this
> may cause that client is blocked.
> ---
>
> Key: HBASE-5329
> URL: https://issues.apache.org/jira/browse/HBASE-5329
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.3
> Environment: Red Hat Enterprise Linux Server release 5.4
>Reporter: liaoxiangui
>Priority: Critical
>
> {code}
> protected long addRowLock(Integer r, HRegion region) throws
> LeaseStillHeldException
> {
> long lockId = -1L;
> lockId = rand.nextLong(); //!!!may generate duplicated
> id，bug?
> String lockName = String.valueOf(lockId);
> rowlocks.put(lockName, r);
> this.leases.createLease(lockName, new RowLockListener(lockName,
> region));
> return lockId;
> }
> {code}
> In addRowLock(),rand may generate duplicated lock id, it may cause
> regionserver throw exception(Leases$LeaseStillHeldException).The client will
> be blocked until old rowlock is released.
> {code}
> 2012-02-03 15:21:50,084 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
> (fsOk: true)
> org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
> at
> org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5329) when client call lockRow,region server may allocate duplicated lock id, this may cause that client is blocked.
[
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5329:
--
Description:
{code}
protected long addRowLock(Integer r, HRegion region) throws
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong(); //!!!may generate duplicated
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName,
region));
return lockId;
}
{code}
In addRowLock(),rand may generate duplicated lock id, it may cause regionserver
throw exception(Leases$LeaseStillHeldException).The client will be blocked
until old rowlock is released.
{code}
2012-02-03 15:21:50,084 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
{code}
was:
protected long addRowLock(Integer r, HRegion region) throws
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong(); //!!!may generate duplicated
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName,
region));
return lockId;
}
In addRowLock(),rand may generate duplicated lock id, it may cause regionserver
throw exception(Leases$LeaseStillHeldException).The client will be blocked
until old rowlock is released.
2012-02-03 15:21:50,084 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
I think we should call putIfAbsent().
If the return value from putIfAbsent() is not null, we should try to regenerate
a new lockId.
Do you want to upload a patch ?
> when client call lockRow,region server may allocate duplicated lock id, this
> may cause that client is blocked.
> ---
>
> Key: HBASE-5329
> URL: https://issues.apache.org/jira/browse/HBASE-5329
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.3
> Environment: Red Hat Enterprise Linux Server release 5.4
>Reporter: liaoxiangui
>
> {code}
> protected long addRowLock(Integer r, HRegion region) throws
> LeaseStillHeldException
> {
> long lockId = -1L;
> lockId = rand.nextLong(); //!!!may generate duplicated
> id，bug?
> String lockName = String.valueOf(lockId);
> rowlocks.put(lockName, r);
> this.leases.createLease(lockName, new RowLockListener(lockName,
> region));
> return lockId;
> }
> {code}
> In addRowLock(),rand may generate duplicated lock id, it may cause
> regionserver throw exception(Leases$LeaseStillHeldException).The client will
> be blocked until old rowlock is released.
> {code}
> 2012-02-03 15:21:50,084 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
> (fsOk: true)
> org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
> at
> org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
> at
[jira] [Updated] (HBASE-5324) Hbck fix dryrun/plan
[ https://issues.apache.org/jira/browse/HBASE-5324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5324: -- Description: Hbck fix should have a dryrun option, or show the planned operations/steps at first, then start the actual fix after confirmed by the user. (was: Hbck fix should have a dryrun option, or show the planed operations/steps at first, then start the actual fix after confirmed by the user.) Fix Version/s: 0.94.0 > Hbck fix dryrun/plan > - > > Key: HBASE-5324 > URL: https://issues.apache.org/jira/browse/HBASE-5324 > Project: HBase > Issue Type: New Feature > Components: hbck >Affects Versions: 0.94.0 >Reporter: Jimmy Xiang > Fix For: 0.94.0 > > > Hbck fix should have a dryrun option, or show the planned operations/steps at > first, then start the actual fix after confirmed by the user. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5186) Add metrics to ThriftServer
[ https://issues.apache.org/jira/browse/HBASE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5186: -- Fix Version/s: 0.94.0 Hadoop Flags: Reviewed > Add metrics to ThriftServer > --- > > Key: HBASE-5186 > URL: https://issues.apache.org/jira/browse/HBASE-5186 > Project: HBase > Issue Type: Improvement >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.94.0 > > Attachments: 5186-v10.txt, 5186-v11.txt, 5186-v12.txt, 5186-v9.txt, > HBASE-5186.D1461.1.patch, HBASE-5186.D1461.2.patch, HBASE-5186.D1461.3.patch, > HBASE-5186.D1461.4.patch, HBASE-5186.D1461.5.patch, HBASE-5186.D1461.6.patch, > HBASE-5186.D1461.7.patch, HBASE-5186.D1461.8.patch > > > It will be useful to have some metrics (queue length, waiting time, > processing time ...) similar to Hadoop RPC server. This allows us to monitor > system health also provide a tool to diagnose the problem where thrift calls > are slow. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5186) Add metrics to ThriftServer
[ https://issues.apache.org/jira/browse/HBASE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5186: -- Attachment: 5186-v12.txt Patch v12 adds logging to CallQueue.java > Add metrics to ThriftServer > --- > > Key: HBASE-5186 > URL: https://issues.apache.org/jira/browse/HBASE-5186 > Project: HBase > Issue Type: Improvement >Reporter: Scott Chen >Assignee: Scott Chen > Attachments: 5186-v10.txt, 5186-v11.txt, 5186-v12.txt, 5186-v9.txt, > HBASE-5186.D1461.1.patch, HBASE-5186.D1461.2.patch, HBASE-5186.D1461.3.patch, > HBASE-5186.D1461.4.patch, HBASE-5186.D1461.5.patch, HBASE-5186.D1461.6.patch, > HBASE-5186.D1461.7.patch, HBASE-5186.D1461.8.patch > > > It will be useful to have some metrics (queue length, waiting time, > processing time ...) similar to Hadoop RPC server. This allows us to monitor > system health also provide a tool to diagnose the problem where thrift calls > are slow. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5186) Add metrics to ThriftServer
[ https://issues.apache.org/jira/browse/HBASE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5186: -- Attachment: 5186-v11.txt Patch v11 adds protection to other methods of CallQueue.java > Add metrics to ThriftServer > --- > > Key: HBASE-5186 > URL: https://issues.apache.org/jira/browse/HBASE-5186 > Project: HBase > Issue Type: Improvement >Reporter: Scott Chen >Assignee: Scott Chen > Attachments: 5186-v10.txt, 5186-v11.txt, 5186-v9.txt, > HBASE-5186.D1461.1.patch, HBASE-5186.D1461.2.patch, HBASE-5186.D1461.3.patch, > HBASE-5186.D1461.4.patch, HBASE-5186.D1461.5.patch, HBASE-5186.D1461.6.patch, > HBASE-5186.D1461.7.patch, HBASE-5186.D1461.8.patch > > > It will be useful to have some metrics (queue length, waiting time, > processing time ...) similar to Hadoop RPC server. This allows us to monitor > system health also provide a tool to diagnose the problem where thrift calls > are slow. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5186) Add metrics to ThriftServer
[ https://issues.apache.org/jira/browse/HBASE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5186: -- Attachment: 5186-v10.txt Patch v10 addresses concurrency issue in drainTo(). > Add metrics to ThriftServer > --- > > Key: HBASE-5186 > URL: https://issues.apache.org/jira/browse/HBASE-5186 > Project: HBase > Issue Type: Improvement >Reporter: Scott Chen >Assignee: Scott Chen > Attachments: 5186-v10.txt, 5186-v9.txt, HBASE-5186.D1461.1.patch, > HBASE-5186.D1461.2.patch, HBASE-5186.D1461.3.patch, HBASE-5186.D1461.4.patch, > HBASE-5186.D1461.5.patch, HBASE-5186.D1461.6.patch, HBASE-5186.D1461.7.patch, > HBASE-5186.D1461.8.patch > > > It will be useful to have some metrics (queue length, waiting time, > processing time ...) similar to Hadoop RPC server. This allows us to monitor > system health also provide a tool to diagnose the problem where thrift calls > are slow. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5186) Add metrics to ThriftServer
[ https://issues.apache.org/jira/browse/HBASE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5186: -- Attachment: 5186-v9.txt Patch v9 is almost same as patch v8. I used an iterator to traverse underlyingQueue in drainTo(). @Scott: If v9 looks good to you, I will integrate tomorrow. > Add metrics to ThriftServer > --- > > Key: HBASE-5186 > URL: https://issues.apache.org/jira/browse/HBASE-5186 > Project: HBase > Issue Type: Improvement >Reporter: Scott Chen >Assignee: Scott Chen > Attachments: 5186-v9.txt, HBASE-5186.D1461.1.patch, > HBASE-5186.D1461.2.patch, HBASE-5186.D1461.3.patch, HBASE-5186.D1461.4.patch, > HBASE-5186.D1461.5.patch, HBASE-5186.D1461.6.patch, HBASE-5186.D1461.7.patch, > HBASE-5186.D1461.8.patch > > > It will be useful to have some metrics (queue length, waiting time, > processing time ...) similar to Hadoop RPC server. This allows us to monitor > system health also provide a tool to diagnose the problem where thrift calls > are slow. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5295) Improve the Thrift API to switch on/off writing to wal for Mutations
[ https://issues.apache.org/jira/browse/HBASE-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5295: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Improve the Thrift API to switch on/off writing to wal for Mutations > - > > Key: HBASE-5295 > URL: https://issues.apache.org/jira/browse/HBASE-5295 > Project: HBase > Issue Type: Improvement > Components: thrift >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Attachments: D1515.1.patch, D1515.1.patch, D1515.1.patch, > D1515.1.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch > > > The thrift api currently does not support switching off updating wal for > Puts/Deletes. Support it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5212) Fix test TestTableMapReduce against 0.23.
[ https://issues.apache.org/jira/browse/HBASE-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5212: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Fix test TestTableMapReduce against 0.23. > - > > Key: HBASE-5212 > URL: https://issues.apache.org/jira/browse/HBASE-5212 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Mahadev konar >Assignee: Gregory Chanan > Fix For: 0.94.0 > > Attachments: 5212-v2.txt, HBASE-5212-v3.patch, HBASE-5212.patch > > > As reported by Andrew on the hadoop mailing list, mvn -Dhadoop.profile=23 > clean test -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce fails > on 0.92 branch. There are minor changes to HBase poms required to fix that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5212) Fix test TestTableMapReduce against 0.23.
[ https://issues.apache.org/jira/browse/HBASE-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5212: -- Fix Version/s: (was: 0.92.1) 0.94.0 Hadoop Flags: Reviewed Integrated to TRUNK. Thanks for the patch, Mahadev and Gregory. > Fix test TestTableMapReduce against 0.23. > - > > Key: HBASE-5212 > URL: https://issues.apache.org/jira/browse/HBASE-5212 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Mahadev konar >Assignee: Gregory Chanan > Fix For: 0.94.0 > > Attachments: 5212-v2.txt, HBASE-5212-v3.patch, HBASE-5212.patch > > > As reported by Andrew on the hadoop mailing list, mvn -Dhadoop.profile=23 > clean test -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce fails > on 0.92 branch. There are minor changes to HBase poms required to fix that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5315) Asynchronous table creation shall verify that all the regions are online
[ https://issues.apache.org/jira/browse/HBASE-5315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5315: -- Summary: Asynchronous table creation shall verify that all the regions are online (was: Asynchronously table creation shall verify all the regions online) > Asynchronous table creation shall verify that all the regions are online > > > Key: HBASE-5315 > URL: https://issues.apache.org/jira/browse/HBASE-5315 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang > > HBaseAdmin creates table in an asynchronous way and it needs to verify all > the regions online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5310) HConnectionManager server cache key enhancement
[ https://issues.apache.org/jira/browse/HBASE-5310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5310: -- Summary: HConnectionManager server cache key enhancement (was: HConnectionManager cache server name enhancement) Hadoop Flags: Reviewed Integrated to TRUNK. Thanks for the patch Jimmy > HConnectionManager server cache key enhancement > --- > > Key: HBASE-5310 > URL: https://issues.apache.org/jira/browse/HBASE-5310 > Project: HBase > Issue Type: Improvement > Components: client >Affects Versions: 0.94.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.94.0 > > Attachments: hbase-5310.txt > > > HConnectionManager uses deprecated HServerAddress to create server cache key > which needs to resolve the address every time. > It should be better to use HRegionLocation.getHostnamePort() instead. > In our cluster we have some DNS issue, resolving an address fails sometime > which kills the application since it is a runtime > exception IllegalArgumentException thrown at > HServerAddress.getResolvedAddress. This change will fix this issue as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5283) Request counters may become negative for heavily loaded regions
[
https://issues.apache.org/jira/browse/HBASE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5283:
--
Fix Version/s: (was: 0.92.1)
Integrated to TRUNK.
Thanks for the patch Mubarak.
Thanks for the review, Stack.
> Request counters may become negative for heavily loaded regions
> ---
>
> Key: HBASE-5283
> URL: https://issues.apache.org/jira/browse/HBASE-5283
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Zhihong Yu
>Assignee: Mubarak Seyed
> Fix For: 0.94.0
>
> Attachments: 5283.txt, HBASE-5283.trunk.v1.patch
>
>
> Requests counter showing negative count, example under 'Requests' column:
> -645470239
> {code}
> Name Region Server Start Key End Key Requests
> usertable,user2037516127892189021,1326756873774.16833e4566d1daef109b8fdcd1f4b5a6.
> xxx.com:60030 user2037516127892189021 user2296868939942738705
>-645470239
> {code}
> RegionLoad.readRequestsCount and RegionLoad.writeRequestsCount are of int
> type. Our Ops has been running lots of heavy load operation.
> RegionLoad.getRequestsCount() overflows int.MAX_VALUE. It is set to D986E7E1.
> In table.jsp, RegionLoad.getRequestsCount() is assigned to long type.
> D986E7E1 is converted to long D986E7E1 which is -645470239 in decimal.
> Suggested fix is to make readRequestsCount and writeRequestsCount long type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5283) Request counters may become negative for heavily loaded regions
[
https://issues.apache.org/jira/browse/HBASE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5283:
--
Attachment: 5283.txt
> Request counters may become negative for heavily loaded regions
> ---
>
> Key: HBASE-5283
> URL: https://issues.apache.org/jira/browse/HBASE-5283
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Zhihong Yu
>Assignee: Mubarak Seyed
> Fix For: 0.94.0, 0.92.1
>
> Attachments: 5283.txt, HBASE-5283.trunk.v1.patch
>
>
> Requests counter showing negative count, example under 'Requests' column:
> -645470239
> {code}
> Name Region Server Start Key End Key Requests
> usertable,user2037516127892189021,1326756873774.16833e4566d1daef109b8fdcd1f4b5a6.
> xxx.com:60030 user2037516127892189021 user2296868939942738705
>-645470239
> {code}
> RegionLoad.readRequestsCount and RegionLoad.writeRequestsCount are of int
> type. Our Ops has been running lots of heavy load operation.
> RegionLoad.getRequestsCount() overflows int.MAX_VALUE. It is set to D986E7E1.
> In table.jsp, RegionLoad.getRequestsCount() is assigned to long type.
> D986E7E1 is converted to long D986E7E1 which is -645470239 in decimal.
> Suggested fix is to make readRequestsCount and writeRequestsCount long type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5290) [FindBugs] Synchronization on boxed primitive
[
https://issues.apache.org/jira/browse/HBASE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5290:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> [FindBugs] Synchronization on boxed primitive
> -
>
> Key: HBASE-5290
> URL: https://issues.apache.org/jira/browse/HBASE-5290
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Liyin Tang
>Assignee: Ben West
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 5290-v3.txt, 5290-v4.txt, HBASE-5290.patch,
> HBASE-5290.patch, HBASE-5290v2.patch
>
>
> This bug is reported by the findBugs tool, which is a static analysis tool.
> Bug: Synchronization on Integer in
> org.apache.hadoop.hbase.regionserver.compactions.CompactSelection.emptyFileList()
> The code synchronizes on a boxed primitive constant, such as an Integer.
> {code}
> private static Integer count = 0;
> ...
> synchronized(count) {
> count++;
> }
> ...
> {code}
> Since Integer objects can be cached and shared, this code could be
> synchronizing on the same object as other, unrelated code, leading to
> unresponsiveness and possible deadlock
> See CERT CON08-J. Do not synchronize on objects that may be reused for more
> information.
> Confidence: Normal, Rank: Troubling (14)
> Pattern: DL_SYNCHRONIZATION_ON_BOXED_PRIMITIVE
> Type: DL, Category: MT_CORRECTNESS (Multithreaded correctness)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5283) Request counters may become negative for heavily loaded regions
[
https://issues.apache.org/jira/browse/HBASE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5283:
--
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
> Request counters may become negative for heavily loaded regions
> ---
>
> Key: HBASE-5283
> URL: https://issues.apache.org/jira/browse/HBASE-5283
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Zhihong Yu
>Assignee: Mubarak Seyed
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-5283.trunk.v1.patch
>
>
> Requests counter showing negative count, example under 'Requests' column:
> -645470239
> {code}
> Name Region Server Start Key End Key Requests
> usertable,user2037516127892189021,1326756873774.16833e4566d1daef109b8fdcd1f4b5a6.
> xxx.com:60030 user2037516127892189021 user2296868939942738705
>-645470239
> {code}
> RegionLoad.readRequestsCount and RegionLoad.writeRequestsCount are of int
> type. Our Ops has been running lots of heavy load operation.
> RegionLoad.getRequestsCount() overflows int.MAX_VALUE. It is set to D986E7E1.
> In table.jsp, RegionLoad.getRequestsCount() is assigned to long type.
> D986E7E1 is converted to long D986E7E1 which is -645470239 in decimal.
> Suggested fix is to make readRequestsCount and writeRequestsCount long type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3850) Log more details when a scanner lease expires
[
https://issues.apache.org/jira/browse/HBASE-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-3850:
--
Fix Version/s: 0.94.0
Hadoop Flags: Reviewed
> Log more details when a scanner lease expires
> -
>
> Key: HBASE-3850
> URL: https://issues.apache.org/jira/browse/HBASE-3850
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Reporter: Benoit Sigoure
>Assignee: Darren Haas
>Priority: Critical
> Fix For: 0.94.0
>
> Attachments: HBASE-3850.trunk.v1.patch
>
>
> The message logged by the RegionServer when a Scanner lease expires isn't as
> useful as it could be. {{Scanner 4765412385779771089 lease expired}} - most
> clients don't log their scanner ID, so it's really hard to figure out what
> was going on. I think it would be useful to at least log the name of the
> region on which the Scanner was open, and it would be great to have the
> ip:port of the client that had that lease too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5259) Normalize the RegionLocation in TableInputFormat by the reverse DNS lookup.
[ https://issues.apache.org/jira/browse/HBASE-5259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5259: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Normalize the RegionLocation in TableInputFormat by the reverse DNS lookup. > --- > > Key: HBASE-5259 > URL: https://issues.apache.org/jira/browse/HBASE-5259 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Fix For: 0.94.0 > > Attachments: D1413.1.patch, D1413.1.patch, D1413.1.patch, > D1413.1.patch, D1413.2.patch, D1413.2.patch, D1413.2.patch, D1413.2.patch, > D1413.3.patch, D1413.3.patch, D1413.3.patch, D1413.3.patch, HBASE-5259.patch > > > Assuming the HBase and MapReduce running in the same cluster, the > TableInputFormat is to override the split function which divides all the > regions from one particular table into a series of mapper tasks. So each > mapper task can process a region or one part of a region. Ideally, the mapper > task should run on the same machine on which the region server hosts the > corresponding region. That's the motivation that the TableInputFormat sets > the RegionLocation so that the MapReduce framework can respect the node > locality. > The code simply set the host name of the region server as the > HRegionLocation. However, the host name of the region server may have > different format with the host name of the task tracker (Mapper task). The > task tracker always gets its hostname by the reverse DNS lookup. And the DNS > service may return different host name format. For example, the host name of > the region server is correctly set as a.b.c.d while the reverse DNS lookup > may return a.b.c.d. (With an additional doc in the end). > So the solution is to set the RegionLocation by the reverse DNS lookup as > well. No matter what host name format the DNS system is using, the > TableInputFormat has the responsibility to keep the consistent host name > format with the MapReduce framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5074) support checksums in HBase block cache
[ https://issues.apache.org/jira/browse/HBASE-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5074: -- Comment: was deleted (was: mbautin has commented on the revision "[jira] [HBASE-5074] Support checksums in HBase block cache". Added CCs: dhruba @Dhruba: The "checksum at the end of block" approach seems reasonable and the implementation looks good! Specific comments inline. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java:357 What is the purpose of the hfs parameter here? src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java:49 s/preceeding/preceding/ src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java:50 s/deermines/determines/ src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java:51 s/does not need/do not need/ src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java:119 s/major/minor/ src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java:260 Rename the existing expectVersion to expectMajorVersion for clarity. src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java:343 Rename to expectMajorVersion for clarity. Also, does the version field of this class now only contain the major version? If so, rename it to majorVersion. src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java:345 Add the word "major" to the error message. src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java:462 Rename to getMajorVersion src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java:402 Can we modify the parameter type and get rid of this cast? src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java:415 This is not a constructor, but a factory method. src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java:417 Add "ForTest" to method name for clarity. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:91 s/has/have/ src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:95 Consider replacing the "_V0" suffix with something more meaningful like "_NO_CHECKSUM". src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:102 Consider using a suffix "_WITH_CHECKSUM". src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:123 When the number of bytes per checksum becomes configurable, will that require a persistent data format change? What will the upgrade procedure be in that case? src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:138 It is not clear from this call that 0 is minor version. Create a constant with a meaningful name (e.g. MINOR_VERSION_NO_CHECKSUM). src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:149 Consider adding "WithChecksum" to variable name. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:395-398 This is becoming bulky. Factor out the common term (uncompressedSizeWithoutHeader + headerSize() + cksumBytes) into a local variable. Also avoid evaluating headerSize() twice. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:400-402 Reuse the new local variables from the above comments here. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:441-442 Update this comment, since the meaning of "extraBytes" has changed from just being the room for the next block's header to a much more complex role. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:757-758 Should we throw an IOException instead since this method already throws it? src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:771-772 tmp is a particularly bad variable name. Combine these two lines and get rid of tmp. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:808-809 Get rid of tmp and combine these two lines. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:788 This method and the above one seem to share a lot of code. Is it possible to get rid of code duplication? Also, these two methods seem isolated enough to be moved to another class, maybe even as static methods. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:866-870 Do we need this in case of minorVersion = 0? Or do we always write new files with checksums? src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:973-975 Somehow the fact that checksum format is different for compressed and uncompressed blocks has escaped me halfway through the review. Maybe it is worth explicitly mentioning that in javadoc. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:999-1011 Use System.arraycopy instead of loops. Add "ForTest" to method name to discourage its use in production. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:1202 Nice! Thanks for locking down these internal base classes and methods. src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlo
[jira] [Updated] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the region assignment inconsistent
[
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5200:
--
Fix Version/s: 0.94.0
Hadoop Flags: Reviewed
Summary: AM.ProcessRegionInTransition() and AM.handleRegion() race
thus leaving the region assignment inconsistent (was:
AM.ProcessRegionInTransition() and AM.handleRegion() races thus leaving the
region assignment inconsistent.)
> AM.ProcessRegionInTransition() and AM.handleRegion() race thus leaving the
> region assignment inconsistent
> -
>
> Key: HBASE-5200
> URL: https://issues.apache.org/jira/browse/HBASE-5200
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.94.0, 0.90.7, 0.92.1
>
> Attachments: HBASE-5200.patch, HBASE-5200_1.patch,
> TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml
>
>
> This is the scenario
> Consider a case where the balancer is going on thus trying to close regions
> in a RS.
> Before we could close a master switch happens.
> On Master switch the set of nodes that are in RIT is collected and we first
> get Data and start watching the node
> After that the node data is added into RIT.
> Now by this time (before adding to RIT) if the RS to which close was called
> does a transition in AM.handleRegion() we miss the handling saying RIT state
> was null.
> {code}
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> a66d281d231dfcaea97c270698b26b6f from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> c12e53bfd48ddc5eec507d66821c4d23 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 59ae13de8c1eb325a0dd51f4902d2052 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> f45bc9614d7575f35244849af85aa078 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> cc3ecd7054fe6cd4a1159ed92fd62641 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 3af40478a17fee96b4a192b22c90d5a2 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> e6096a8466e730463e10d3d61f809b92 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 4806781a1a23066f7baed22b4d237e24 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> d69e104131accaefe21dcc01fddc7629 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> {code}
> In branch the CLOSING node is created by RS thus leading to more
> inconsistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5297) Update metrics numOpenConnections and callQueueLen directly in HBaseServer
[ https://issues.apache.org/jira/browse/HBASE-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5297: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Update metrics numOpenConnections and callQueueLen directly in HBaseServer > -- > > Key: HBASE-5297 > URL: https://issues.apache.org/jira/browse/HBASE-5297 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Scott Chen >Assignee: Scott Chen >Priority: Minor > Fix For: 0.94.0 > > Attachments: HBASE-5297.D1509.1.patch, HBASE-5297.D1509.2.patch, > HBASE-5297.D1509.3.patch > > > It's better to directly update the metrics outside HBaseRpcMetrics so that > HBaseRpcMetrics doesn't have to hold reference to HBaseServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5256) Use WritableUtils.readVInt() in RegionLoad.readFields()
[ https://issues.apache.org/jira/browse/HBASE-5256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5256: -- Status: Patch Available (was: Open) > Use WritableUtils.readVInt() in RegionLoad.readFields() > --- > > Key: HBASE-5256 > URL: https://issues.apache.org/jira/browse/HBASE-5256 > Project: HBase > Issue Type: Task >Reporter: Zhihong Yu >Assignee: Mubarak Seyed > Fix For: 0.94.0 > > Attachments: HBASE-5256.trunk.v1.patch > > > Currently in.readInt() is used in RegionLoad.readFields() > More metrics would be added to RegionLoad in the future, we should utilize > WritableUtils.readVInt() to reduce the amount of data exchanged between > Master and region servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5290) [FindBugs] Synchronization on boxed primitive
[
https://issues.apache.org/jira/browse/HBASE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5290:
--
Fix Version/s: 0.94.0
Hadoop Flags: Reviewed
> [FindBugs] Synchronization on boxed primitive
> -
>
> Key: HBASE-5290
> URL: https://issues.apache.org/jira/browse/HBASE-5290
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Liyin Tang
>Assignee: Ben West
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 5290-v3.txt, 5290-v4.txt, HBASE-5290.patch,
> HBASE-5290.patch, HBASE-5290v2.patch
>
>
> This bug is reported by the findBugs tool, which is a static analysis tool.
> Bug: Synchronization on Integer in
> org.apache.hadoop.hbase.regionserver.compactions.CompactSelection.emptyFileList()
> The code synchronizes on a boxed primitive constant, such as an Integer.
> {code}
> private static Integer count = 0;
> ...
> synchronized(count) {
> count++;
> }
> ...
> {code}
> Since Integer objects can be cached and shared, this code could be
> synchronizing on the same object as other, unrelated code, leading to
> unresponsiveness and possible deadlock
> See CERT CON08-J. Do not synchronize on objects that may be reused for more
> information.
> Confidence: Normal, Rank: Troubling (14)
> Pattern: DL_SYNCHRONIZATION_ON_BOXED_PRIMITIVE
> Type: DL, Category: MT_CORRECTNESS (Multithreaded correctness)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5290) [FindBugs] Synchronization on boxed primitive
[
https://issues.apache.org/jira/browse/HBASE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5290:
--
Attachment: 5290-v4.txt
Patch v4 should be close to the final form.
Since numOutstandingOffPeakCompactions is static, the lock object should be
static as well.
Since numOutstandingOffPeakCompactions isn't referenced by any other class, I
made it package private in case TestCompactSelection needs to access it in
future.
> [FindBugs] Synchronization on boxed primitive
> -
>
> Key: HBASE-5290
> URL: https://issues.apache.org/jira/browse/HBASE-5290
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Liyin Tang
>Assignee: Liyin Tang
>Priority: Minor
> Attachments: 5290-v3.txt, 5290-v4.txt, HBASE-5290.patch,
> HBASE-5290.patch, HBASE-5290v2.patch
>
>
> This bug is reported by the findBugs tool, which is a static analysis tool.
> Bug: Synchronization on Integer in
> org.apache.hadoop.hbase.regionserver.compactions.CompactSelection.emptyFileList()
> The code synchronizes on a boxed primitive constant, such as an Integer.
> {code}
> private static Integer count = 0;
> ...
> synchronized(count) {
> count++;
> }
> ...
> {code}
> Since Integer objects can be cached and shared, this code could be
> synchronizing on the same object as other, unrelated code, leading to
> unresponsiveness and possible deadlock
> See CERT CON08-J. Do not synchronize on objects that may be reused for more
> information.
> Confidence: Normal, Rank: Troubling (14)
> Pattern: DL_SYNCHRONIZATION_ON_BOXED_PRIMITIVE
> Type: DL, Category: MT_CORRECTNESS (Multithreaded correctness)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5290) [FindBugs] Synchronization on boxed primitive
[
https://issues.apache.org/jira/browse/HBASE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5290:
--
Attachment: 5290-v3.txt
Patch v3 changes compactionCountLock to private.
> [FindBugs] Synchronization on boxed primitive
> -
>
> Key: HBASE-5290
> URL: https://issues.apache.org/jira/browse/HBASE-5290
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Liyin Tang
>Assignee: Liyin Tang
>Priority: Minor
> Attachments: 5290-v3.txt, HBASE-5290.patch, HBASE-5290.patch,
> HBASE-5290v2.patch
>
>
> This bug is reported by the findBugs tool, which is a static analysis tool.
> Bug: Synchronization on Integer in
> org.apache.hadoop.hbase.regionserver.compactions.CompactSelection.emptyFileList()
> The code synchronizes on a boxed primitive constant, such as an Integer.
> {code}
> private static Integer count = 0;
> ...
> synchronized(count) {
> count++;
> }
> ...
> {code}
> Since Integer objects can be cached and shared, this code could be
> synchronizing on the same object as other, unrelated code, leading to
> unresponsiveness and possible deadlock
> See CERT CON08-J. Do not synchronize on objects that may be reused for more
> information.
> Confidence: Normal, Rank: Troubling (14)
> Pattern: DL_SYNCHRONIZATION_ON_BOXED_PRIMITIVE
> Type: DL, Category: MT_CORRECTNESS (Multithreaded correctness)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5290) [FindBugs] Synchronization on boxed primitive
[
https://issues.apache.org/jira/browse/HBASE-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5290:
--
Description:
This bug is reported by the findBugs tool, which is a static analysis tool.
Bug: Synchronization on Integer in
org.apache.hadoop.hbase.regionserver.compactions.CompactSelection.emptyFileList()
The code synchronizes on a boxed primitive constant, such as an Integer.
{code}
private static Integer count = 0;
...
synchronized(count) {
count++;
}
...
{code}
Since Integer objects can be cached and shared, this code could be
synchronizing on the same object as other, unrelated code, leading to
unresponsiveness and possible deadlock
See CERT CON08-J. Do not synchronize on objects that may be reused for more
information.
Confidence: Normal, Rank: Troubling (14)
Pattern: DL_SYNCHRONIZATION_ON_BOXED_PRIMITIVE
Type: DL, Category: MT_CORRECTNESS (Multithreaded correctness)
was:
This bug is reported by the findBugs tool, which is a static analysis tool.
Bug: Synchronization on Integer in
org.apache.hadoop.hbase.regionserver.compactions.CompactSelection.emptyFileList()
The code synchronizes on a boxed primitive constant, such as an Integer.
private static Integer count = 0;
...
synchronized(count) {
count++;
}
...
Since Integer objects can be cached and shared, this code could be
synchronizing on the same object as other, unrelated code, leading to
unresponsiveness and possible deadlock
See CERT CON08-J. Do not synchronize on objects that may be reused for more
information.
Confidence: Normal, Rank: Troubling (14)
Pattern: DL_SYNCHRONIZATION_ON_BOXED_PRIMITIVE
Type: DL, Category: MT_CORRECTNESS (Multithreaded correctness)
> [FindBugs] Synchronization on boxed primitive
> -
>
> Key: HBASE-5290
> URL: https://issues.apache.org/jira/browse/HBASE-5290
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Liyin Tang
>Assignee: Liyin Tang
>Priority: Minor
> Attachments: HBASE-5290.patch, HBASE-5290.patch, HBASE-5290v2.patch
>
>
> This bug is reported by the findBugs tool, which is a static analysis tool.
> Bug: Synchronization on Integer in
> org.apache.hadoop.hbase.regionserver.compactions.CompactSelection.emptyFileList()
> The code synchronizes on a boxed primitive constant, such as an Integer.
> {code}
> private static Integer count = 0;
> ...
> synchronized(count) {
> count++;
> }
> ...
> {code}
> Since Integer objects can be cached and shared, this code could be
> synchronizing on the same object as other, unrelated code, leading to
> unresponsiveness and possible deadlock
> See CERT CON08-J. Do not synchronize on objects that may be reused for more
> information.
> Confidence: Normal, Rank: Troubling (14)
> Pattern: DL_SYNCHRONIZATION_ON_BOXED_PRIMITIVE
> Type: DL, Category: MT_CORRECTNESS (Multithreaded correctness)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5259) Normalize the RegionLocation in TableInputFormat by the reverse DNS lookup.
[ https://issues.apache.org/jira/browse/HBASE-5259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5259: -- Fix Version/s: 0.94.0 Changing TableInputFormatBase in mapreduce should be good enough. > Normalize the RegionLocation in TableInputFormat by the reverse DNS lookup. > --- > > Key: HBASE-5259 > URL: https://issues.apache.org/jira/browse/HBASE-5259 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Fix For: 0.94.0 > > Attachments: D1413.1.patch, D1413.1.patch, D1413.1.patch, > D1413.1.patch, D1413.2.patch, D1413.2.patch, D1413.2.patch, D1413.2.patch, > D1413.3.patch, D1413.3.patch, D1413.3.patch, D1413.3.patch, HBASE-5259.patch > > > Assuming the HBase and MapReduce running in the same cluster, the > TableInputFormat is to override the split function which divides all the > regions from one particular table into a series of mapper tasks. So each > mapper task can process a region or one part of a region. Ideally, the mapper > task should run on the same machine on which the region server hosts the > corresponding region. That's the motivation that the TableInputFormat sets > the RegionLocation so that the MapReduce framework can respect the node > locality. > The code simply set the host name of the region server as the > HRegionLocation. However, the host name of the region server may have > different format with the host name of the task tracker (Mapper task). The > task tracker always gets its hostname by the reverse DNS lookup. And the DNS > service may return different host name format. For example, the host name of > the region server is correctly set as a.b.c.d while the reverse DNS lookup > may return a.b.c.d. (With an additional doc in the end). > So the solution is to set the RegionLocation by the reverse DNS lookup as > well. No matter what host name format the DNS system is using, the > TableInputFormat has the responsibility to keep the consistent host name > format with the MapReduce framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5200) AM.ProcessRegionInTransition() and AM.handleRegion() races thus leaving the region assignment inconsistent.
[
https://issues.apache.org/jira/browse/HBASE-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5200:
--
Attachment: TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml
> AM.ProcessRegionInTransition() and AM.handleRegion() races thus leaving the
> region assignment inconsistent.
> ---
>
> Key: HBASE-5200
> URL: https://issues.apache.org/jira/browse/HBASE-5200
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.90.7, 0.92.1
>
> Attachments: HBASE-5200.patch,
> TEST-org.apache.hadoop.hbase.master.TestRestartCluster.xml
>
>
> This is the scenario
> Consider a case where the balancer is going on thus trying to close regions
> in a RS.
> Before we could close a master switch happens.
> On Master switch the set of nodes that are in RIT is collected and we first
> get Data and start watching the node
> After that the node data is added into RIT.
> Now by this time (before adding to RIT) if the RS to which close was called
> does a transition in AM.handleRegion() we miss the handling saying RIT state
> was null.
> {code}
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> a66d281d231dfcaea97c270698b26b6f from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> c12e53bfd48ddc5eec507d66821c4d23 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,358 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 59ae13de8c1eb325a0dd51f4902d2052 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> f45bc9614d7575f35244849af85aa078 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> cc3ecd7054fe6cd4a1159ed92fd62641 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 3af40478a17fee96b4a192b22c90d5a2 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> e6096a8466e730463e10d3d61f809b92 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> 4806781a1a23066f7baed22b4d237e24 from server
> HOST-192-168-47-204,20020,1326342744518 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> 2012-01-13 10:50:46,359 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Received CLOSED for region
> d69e104131accaefe21dcc01fddc7629 from server
> HOST-192-168-47-205,20020,1326363111288 but region was in the state null and
> not in expected PENDING_CLOSE or CLOSING states
> {code}
> In branch the CLOSING node is created by RS thus leading to more
> inconsistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4218) Data Block Encoding of KeyValues (aka delta encoding / prefix compression)
[ https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-4218: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12508340/0001-Delta-encoding.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 92 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -142 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 80 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.TestHeapSize Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/578//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/578//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/578//console This message is automatically generated.) > Data Block Encoding of KeyValues (aka delta encoding / prefix compression) > --- > > Key: HBASE-4218 > URL: https://issues.apache.org/jira/browse/HBASE-4218 > Project: HBase > Issue Type: Improvement > Components: io >Affects Versions: 0.94.0 >Reporter: Jacek Migdal >Assignee: Mikhail Bautin > Labels: compression > Fix For: 0.94.0 > > Attachments: 0001-Delta-encoding-fixed-encoded-scanners.patch, > 0001-Delta-encoding.patch, 4218-2012-01-14.txt, 4218-v16.txt, 4218.txt, > D447.1.patch, D447.10.patch, D447.11.patch, D447.12.patch, D447.13.patch, > D447.14.patch, D447.15.patch, D447.16.patch, D447.17.patch, D447.18.patch, > D447.19.patch, D447.2.patch, D447.20.patch, D447.21.patch, D447.22.patch, > D447.23.patch, D447.24.patch, D447.25.patch, D447.26.patch, D447.3.patch, > D447.4.patch, D447.5.patch, D447.6.patch, D447.7.patch, D447.8.patch, > D447.9.patch, Data-block-encoding-2011-12-23.patch, > Delta-encoding-2012-01-17_11_09_09.patch, > Delta-encoding-2012-01-25_00_45_29.patch, > Delta-encoding-2012-01-25_16_32_14.patch, > Delta-encoding.patch-2011-12-22_11_52_07.patch, > Delta-encoding.patch-2012-01-05_15_16_43.patch, > Delta-encoding.patch-2012-01-05_16_31_44.patch, > Delta-encoding.patch-2012-01-05_16_31_44_copy.patch, > Delta-encoding.patch-2012-01-05_18_50_47.patch, > Delta-encoding.patch-2012-01-07_14_12_48.patch, > Delta-encoding.patch-2012-01-13_12_20_07.patch, > Delta_encoding_with_memstore_TS.patch, open-source.diff > > > A compression for keys. Keys are sorted in HFile and they are usually very > similar. Because of that, it is possible to design better compression than > general purpose algorithms, > It is an additional step designed to be used in memory. It aims to save > memory in cache as well as speeding seeks within HFileBlocks. It should > improve performance a lot, if key lengths are larger than value lengths. For > example, it makes a lot of sense to use it when value is a counter. > Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) > shows that I could achieve decent level of compression: > key compression ratio: 92% > total compression ratio: 85% > LZO on the same data: 85% > LZO after delta encoding: 91% > While having much better performance (20-80% faster decompression ratio than > LZO). Moreover, it should allow far more efficient seeking which should > improve performance a bit. > It seems that a simple compression algorithms are good enough. Most of the > savings are due to prefix compression, int128 encoding, timestamp diffs and > bitfields to avoid duplication. That way, comparisons of compressed data can > be much faster than a byte comparator (thanks to prefix compression and > bitfields). > In order to implement it in HBase two important changes in design will be > needed: > -solidify interface to HFileBlock / HFileReader Scanner to provide seeking > and iterating; access to uncompressed buffer in HFileBlock will have bad > performance > -extend comparators to support comparison assuming that N first bytes are > equal (or some fields are equal) > Link to a discussion about something similar: > http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windows&subj=Re+prefix+compression -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/Conta
[jira] [Updated] (HBASE-5074) support checksums in HBase block cache
[ https://issues.apache.org/jira/browse/HBASE-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5074: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12512350/D1521.1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 50 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -140 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 161 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestStore org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed org.apache.hadoop.hbase.regionserver.wal.TestWALReplay org.apache.hadoop.hbase.client.TestInstantSchemaChangeSplit org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler org.apache.hadoop.hbase.io.hfile.TestHFileBlock org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles org.apache.hadoop.hbase.replication.TestMultiSlaveReplication org.apache.hadoop.hbase.util.TestMergeTool org.apache.hadoop.hbase.util.TestMergeTable org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.coprocessor.TestWALObserver Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/867//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/867//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/867//console This message is automatically generated.) > support checksums in HBase block cache > -- > > Key: HBASE-5074 > URL: https://issues.apache.org/jira/browse/HBASE-5074 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Attachments: D1521.1.patch, D1521.1.patch > > > The current implementation of HDFS stores the data in one block file and the > metadata(checksum) in another block file. This means that every read into the > HBase block cache actually consumes two disk iops, one to the datafile and > one to the checksum file. This is a major problem for scaling HBase, because > HBase is usually bottlenecked on the number of random disk iops that the > storage-hardware offers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5074) support checksums in HBase block cache
[ https://issues.apache.org/jira/browse/HBASE-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5074: -- Status: Patch Available (was: Open) > support checksums in HBase block cache > -- > > Key: HBASE-5074 > URL: https://issues.apache.org/jira/browse/HBASE-5074 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Attachments: D1521.1.patch, D1521.1.patch > > > The current implementation of HDFS stores the data in one block file and the > metadata(checksum) in another block file. This means that every read into the > HBase block cache actually consumes two disk iops, one to the datafile and > one to the checksum file. This is a major problem for scaling HBase, because > HBase is usually bottlenecked on the number of random disk iops that the > storage-hardware offers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5074) support checksums in HBase block cache
[ https://issues.apache.org/jira/browse/HBASE-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5074: -- Comment: was deleted (was: tedyu has commented on the revision "[jira] [HBASE-5074] Support checksums in HBase block cache". Good job, Dhruba. I like this comment from HFileSystem: + * In future, if we want to make hlogs be in a different filesystem, + * this is the place to make it happen. I only see one setVerifyChecksum() call in the HFileSystem ctor. The readfs is used by createReaderWithEncoding(). Shall we give readfs more flexibility where checksum verification can be configured dynamically ? REVISION DETAIL https://reviews.facebook.net/D1521 ) > support checksums in HBase block cache > -- > > Key: HBASE-5074 > URL: https://issues.apache.org/jira/browse/HBASE-5074 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Attachments: D1521.1.patch, D1521.1.patch > > > The current implementation of HDFS stores the data in one block file and the > metadata(checksum) in another block file. This means that every read into the > HBase block cache actually consumes two disk iops, one to the datafile and > one to the checksum file. This is a major problem for scaling HBase, because > HBase is usually bottlenecked on the number of random disk iops that the > storage-hardware offers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5297) Update metrics numOpenConnections and callQueueLen directly in HBaseServer
[ https://issues.apache.org/jira/browse/HBASE-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5297: -- Fix Version/s: 0.94.0 > Update metrics numOpenConnections and callQueueLen directly in HBaseServer > -- > > Key: HBASE-5297 > URL: https://issues.apache.org/jira/browse/HBASE-5297 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Scott Chen >Assignee: Scott Chen >Priority: Minor > Fix For: 0.94.0 > > Attachments: HBASE-5297.D1509.1.patch, HBASE-5297.D1509.2.patch > > > It's better to directly update the metrics outside HBaseRpcMetrics so that > HBaseRpcMetrics doesn't have to hold reference to HBaseServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5297) Update metrics numOpenConnections and callQueueLen directly in HBaseServer
[ https://issues.apache.org/jira/browse/HBASE-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5297: -- Status: Patch Available (was: Open) > Update metrics numOpenConnections and callQueueLen directly in HBaseServer > -- > > Key: HBASE-5297 > URL: https://issues.apache.org/jira/browse/HBASE-5297 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Scott Chen >Assignee: Scott Chen >Priority: Minor > Attachments: HBASE-5297.D1509.1.patch, HBASE-5297.D1509.2.patch > > > It's better to directly update the metrics outside HBaseRpcMetrics so that > HBaseRpcMetrics doesn't have to hold reference to HBaseServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5153) Add retry logic in HConnectionImplementation#resetZooKeeperTrackers
[ https://issues.apache.org/jira/browse/HBASE-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5153: -- Fix Version/s: (was: 0.92.1) (was: 0.94.0) Dropping Fix versions as Lars suggested. > Add retry logic in HConnectionImplementation#resetZooKeeperTrackers > --- > > Key: HBASE-5153 > URL: https://issues.apache.org/jira/browse/HBASE-5153 > Project: HBase > Issue Type: Bug > Components: client >Affects Versions: 0.90.4 >Reporter: Jieshan Bean >Assignee: Jieshan Bean > Fix For: 0.90.6 > > Attachments: 5153-92.txt, 5153-trunk-v2.txt, 5153-trunk.txt, > 5153-trunk.txt, HBASE-5153-V2.patch, HBASE-5153-V3.patch, > HBASE-5153-V4-90.patch, HBASE-5153-V5-90.patch, > HBASE-5153-V6-90-minorchange.patch, HBASE-5153-V6-90.txt, > HBASE-5153-trunk-v2.patch, HBASE-5153-trunk.patch, HBASE-5153.patch, > HBase-5153-90-addendum.patch, TestResults-hbase5153.out > > > HBASE-4893 is related to this issue. In that issue, we know, if multi-threads > share a same connection, once this connection got abort in one thread, the > other threads will got a > "HConnectionManager$HConnectionImplementation@18fb1f7 closed" exception. > It solve the problem of "stale connection can't removed". But the orignal > HTable instance cann't be continue to use. The connection in HTable should be > recreated. > Actually, there's two aproach to solve this: > 1. In user code, once catch an IOE, close connection and re-create HTable > instance. We can use this as a workaround. > 2. In HBase Client side, catch this exception, and re-create connection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5153) Add retry logic in HConnectionImplementation#resetZooKeeperTrackers
[ https://issues.apache.org/jira/browse/HBASE-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5153: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12512100/HBase-5153-90-addendum.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/858//console This message is automatically generated.) > Add retry logic in HConnectionImplementation#resetZooKeeperTrackers > --- > > Key: HBASE-5153 > URL: https://issues.apache.org/jira/browse/HBASE-5153 > Project: HBase > Issue Type: Bug > Components: client >Affects Versions: 0.90.4 >Reporter: Jieshan Bean >Assignee: Jieshan Bean > Fix For: 0.94.0, 0.90.6, 0.92.1 > > Attachments: 5153-92.txt, 5153-trunk-v2.txt, 5153-trunk.txt, > 5153-trunk.txt, HBASE-5153-V2.patch, HBASE-5153-V3.patch, > HBASE-5153-V4-90.patch, HBASE-5153-V5-90.patch, > HBASE-5153-V6-90-minorchange.patch, HBASE-5153-V6-90.txt, > HBASE-5153-trunk-v2.patch, HBASE-5153-trunk.patch, HBASE-5153.patch, > HBase-5153-90-addendum.patch, TestResults-hbase5153.out > > > HBASE-4893 is related to this issue. In that issue, we know, if multi-threads > share a same connection, once this connection got abort in one thread, the > other threads will got a > "HConnectionManager$HConnectionImplementation@18fb1f7 closed" exception. > It solve the problem of "stale connection can't removed". But the orignal > HTable instance cann't be continue to use. The connection in HTable should be > recreated. > Actually, there's two aproach to solve this: > 1. In user code, once catch an IOE, close connection and re-create HTable > instance. We can use this as a workaround. > 2. In HBase Client side, catch this exception, and re-create connection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3134) [replication] Add the ability to enable/disable streams
[ https://issues.apache.org/jira/browse/HBASE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-3134: -- Fix Version/s: 0.94.0 > [replication] Add the ability to enable/disable streams > --- > > Key: HBASE-3134 > URL: https://issues.apache.org/jira/browse/HBASE-3134 > Project: HBase > Issue Type: New Feature > Components: replication >Reporter: Jean-Daniel Cryans >Assignee: Teruyoshi Zenmyo >Priority: Minor > Labels: replication > Fix For: 0.94.0 > > Attachments: HBASE-3134.patch > > > This jira was initially in the scope of HBASE-2201, but was pushed out since > it has low value compared to the required effort (and when want to ship > 0.90.0 rather soonish). > We need to design a way to enable/disable replication streams in a > determinate fashion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5289) NullPointerException in resetZooKeeperTrackers in HConnectionManager / HConnectionImplementation
[
https://issues.apache.org/jira/browse/HBASE-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5289:
--
Fix Version/s: 0.92.1
0.94.0
> NullPointerException in resetZooKeeperTrackers in HConnectionManager /
> HConnectionImplementation
>
>
> Key: HBASE-5289
> URL: https://issues.apache.org/jira/browse/HBASE-5289
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.90.5
>Reporter: Krystian Nowak
> Fix For: 0.94.0, 0.92.1
>
>
> This might happen on heavy load in case of lagging HBase when sharing one
> HConnection by multiple threads:
> {noformat}
> 2012-01-26 13:59:38,396 ERROR [http://*:8080-251-EventThread]
> zookeeper.ClientCnxn$EventThread(532): Error while calling watcher
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.resetZooKeeperTrackers(HConnectionManager.java:533)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.abort(HConnectionManager.java:1536)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:344)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:262)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
> {noformat}
> The following code is not protected against NPE:
> {code}
> private synchronized void resetZooKeeperTrackers()
> throws ZooKeeperConnectionException {
> LOG.info("Trying to reconnect to zookeeper");
> masterAddressTracker.stop();
> masterAddressTracker = null;
> rootRegionTracker.stop();
> rootRegionTracker = null;
> clusterId = null;
> this.zooKeeper = null;
> setupZookeeperTrackers();
> }
> {code}
> In some cases as proven by the log snippet above it might happen that either
> masterAddressTracker or rootRegionTracker might be null.
> Because of the NPE the code can't reach setupZookeeperTrackers() call.
> This should be fixed at least the way as shown in one of the patches in
> HBASE-5153
> {code}
> LOG.info("Trying to reconnect to zookeeper.");
> if (this.masterAddressTracker != null) {
> this.masterAddressTracker.stop();
> this.masterAddressTracker = null;
> }
> if (this.rootRegionTracker != null) {
> this.rootRegionTracker.stop();
> this.rootRegionTracker = null;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5283) Request counters may become negative for heavily loaded regions
[
https://issues.apache.org/jira/browse/HBASE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5283:
--
Affects Version/s: 0.92.0
Fix Version/s: 0.92.1
0.94.0
> Request counters may become negative for heavily loaded regions
> ---
>
> Key: HBASE-5283
> URL: https://issues.apache.org/jira/browse/HBASE-5283
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Zhihong Yu
> Fix For: 0.94.0, 0.92.1
>
>
> Requests counter showing negative count, example under 'Requests' column:
> -645470239
> {code}
> Name Region Server Start Key End Key Requests
> usertable,user2037516127892189021,1326756873774.16833e4566d1daef109b8fdcd1f4b5a6.
> xxx.com:60030 user2037516127892189021 user2296868939942738705
>-645470239
> {code}
> RegionLoad.readRequestsCount and RegionLoad.writeRequestsCount are of int
> type. Our Ops has been running lots of heavy load operation.
> RegionLoad.getRequestsCount() overflows int.MAX_VALUE. It is set to D986E7E1.
> In table.jsp, RegionLoad.getRequestsCount() is assigned to long type.
> D986E7E1 is converted to long D986E7E1 which is -645470239 in decimal.
> Suggested fix is to make readRequestsCount and writeRequestsCount long type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5258: -- Description: When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad was: When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? Summary: Move coprocessors set out of RegionLoad (was: Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad) > Move coprocessors set out of RegionLoad > --- > > Key: HBASE-5258 > URL: https://issues.apache.org/jira/browse/HBASE-5258 > Project: HBase > Issue Type: Task >Reporter: Zhihong Yu >Priority: Critical > > When I worked on HBASE-5256, I revisited the code related to Ser/De of > coprocessors set in RegionLoad. > I think the rationale for embedding coprocessors set is for maximum > flexibility where each region can load different coprocessors. > This flexibility is causing extra cost in the region server to Master > communication and increasing the footprint of Master heap. > Would HServerLoad be a better place for this set ? > If required, region server should calculate disparity of loaded coprocessors > among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5258) Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5258: -- Priority: Critical (was: Major) Raising priority as this task is about making the correct design choices. > Move coprocessors set out of RegionLoad, region server should calculate > disparity of loaded coprocessors among regions and send report through > HServerLoad > -- > > Key: HBASE-5258 > URL: https://issues.apache.org/jira/browse/HBASE-5258 > Project: HBase > Issue Type: Task >Reporter: Zhihong Yu >Priority: Critical > > When I worked on HBASE-5256, I revisited the code related to Ser/De of > coprocessors set in RegionLoad. > I think the rationale for embedding coprocessors set is for maximum > flexibility where each region can load different coprocessors. > This flexibility is causing extra cost in the region server to Master > communication and increasing the footprint of Master heap. > Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5271) Result.getValue and Result.getColumnLatest return the wrong column.
[
https://issues.apache.org/jira/browse/HBASE-5271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5271:
--
Attachment: 5271-90.txt
5271-v2.txt
Updated patch by wrapping a long line.
> Result.getValue and Result.getColumnLatest return the wrong column.
> ---
>
> Key: HBASE-5271
> URL: https://issues.apache.org/jira/browse/HBASE-5271
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.90.5
>Reporter: Ghais Issa
> Fix For: 0.94.0, 0.90.6, 0.92.1
>
> Attachments: 5271-90.txt, 5271-v2.txt,
> fixKeyValueMatchingColumn.diff, testGetValue.diff
>
>
> In the following example result.getValue returns the wrong column
> KeyValue kv = new KeyValue(Bytes.toBytes("r"), Bytes.toBytes("24"),
> Bytes.toBytes("2"), Bytes.toBytes(7L));
> Result result = new Result(new KeyValue[] { kv });
> System.out.println(Bytes.toLong(result.getValue(Bytes.toBytes("2"),
> Bytes.toBytes("2"; //prints 7.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5271) Result.getValue and Result.getColumnLatest return the wrong column.
[
https://issues.apache.org/jira/browse/HBASE-5271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5271:
--
Fix Version/s: 0.92.1
0.90.6
0.94.0
Hadoop Flags: Reviewed
> Result.getValue and Result.getColumnLatest return the wrong column.
> ---
>
> Key: HBASE-5271
> URL: https://issues.apache.org/jira/browse/HBASE-5271
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.90.5
>Reporter: Ghais Issa
> Fix For: 0.94.0, 0.90.6, 0.92.1
>
> Attachments: fixKeyValueMatchingColumn.diff, testGetValue.diff
>
>
> In the following example result.getValue returns the wrong column
> KeyValue kv = new KeyValue(Bytes.toBytes("r"), Bytes.toBytes("24"),
> Bytes.toBytes("2"), Bytes.toBytes(7L));
> Result result = new Result(new KeyValue[] { kv });
> System.out.println(Bytes.toLong(result.getValue(Bytes.toBytes("2"),
> Bytes.toBytes("2"; //prints 7.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5231) Backport HBASE-3373 (per-table load balancing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5231: -- Attachment: 5231.addendum > Backport HBASE-3373 (per-table load balancing) to 0.92 > -- > > Key: HBASE-5231 > URL: https://issues.apache.org/jira/browse/HBASE-5231 > Project: HBase > Issue Type: Improvement >Reporter: Zhihong Yu > Fix For: 0.92.1 > > Attachments: 5231-v2.txt, 5231.addendum, 5231.txt > > > This JIRA backports per-table load balancing to 0.90 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5255) Use singletons for OperationStatus to save memory
[
https://issues.apache.org/jira/browse/HBASE-5255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5255:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> Use singletons for OperationStatus to save memory
> -
>
> Key: HBASE-5255
> URL: https://issues.apache.org/jira/browse/HBASE-5255
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 0.90.5, 0.92.0
>Reporter: Benoit Sigoure
>Assignee: Benoit Sigoure
>Priority: Minor
> Labels: performance
> Fix For: 0.94.0, 0.92.1
>
> Attachments: 5255-92.txt, 5255-v2.txt,
> HBASE-5255-0.92-Use-singletons-to-remove-unnecessary-memory-allocati.patch,
> HBASE-5255-trunk-Use-singletons-to-remove-unnecessary-memory-allocati.patch
>
>
> Every single {{Put}} causes the allocation of at least one
> {{OperationStatus}}, yet {{OperationStatus}} is almost always stateless, so
> these allocations are unnecessary and could be avoided. Attached patch adds
> a few singletons and uses them, with no public API change. I didn't test the
> patches, but you get the idea.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5189) Add metrics to keep track of region-splits in RS
[ https://issues.apache.org/jira/browse/HBASE-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5189: -- Hadoop Flags: Reviewed Status: Patch Available (was: Open) > Add metrics to keep track of region-splits in RS > > > Key: HBASE-5189 > URL: https://issues.apache.org/jira/browse/HBASE-5189 > Project: HBase > Issue Type: Improvement > Components: metrics, regionserver >Affects Versions: 0.92.0, 0.90.5 >Reporter: Mubarak Seyed >Assignee: Mubarak Seyed >Priority: Minor > Labels: noob > Attachments: HBASE-5189.trunk.v1.patch, HBASE-5189.trunk.v2.patch > > > For write-heavy workload with region-size 1 GB, region-split is considerably > high. We do normally grep the NN log (grep "mkdir*.split" NN.log | sort | > uniq -c) to get the count. > I would like to have a counter incremented each time region-split execution > succeeds and this counter exposed via the metrics stuff in HBase. > - regionSplitSuccessCount > - regionSplitFailureCount (will help us to correlate the timestamp range in > RS logs across all RS) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5189) Add metrics to keep track of region-splits in RS
[ https://issues.apache.org/jira/browse/HBASE-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5189: -- Fix Version/s: 0.94.0 > Add metrics to keep track of region-splits in RS > > > Key: HBASE-5189 > URL: https://issues.apache.org/jira/browse/HBASE-5189 > Project: HBase > Issue Type: Improvement > Components: metrics, regionserver >Affects Versions: 0.90.5, 0.92.0 >Reporter: Mubarak Seyed >Assignee: Mubarak Seyed >Priority: Minor > Labels: noob > Fix For: 0.94.0 > > Attachments: HBASE-5189.trunk.v1.patch, HBASE-5189.trunk.v2.patch > > > For write-heavy workload with region-size 1 GB, region-split is considerably > high. We do normally grep the NN log (grep "mkdir*.split" NN.log | sort | > uniq -c) to get the count. > I would like to have a counter incremented each time region-split execution > succeeds and this counter exposed via the metrics stuff in HBase. > - regionSplitSuccessCount > - regionSplitFailureCount (will help us to correlate the timestamp range in > RS logs across all RS) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5258) Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5258: -- Summary: Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad (was: Move coprocessors set out of RegionLoad) > Move coprocessors set out of RegionLoad, region server should calculate > disparity of loaded coprocessors among regions and send report through > HServerLoad > -- > > Key: HBASE-5258 > URL: https://issues.apache.org/jira/browse/HBASE-5258 > Project: HBase > Issue Type: Task >Reporter: Zhihong Yu > > When I worked on HBASE-5256, I revisited the code related to Ser/De of > coprocessors set in RegionLoad. > I think the rationale for embedding coprocessors set is for maximum > flexibility where each region can load different coprocessors. > This flexibility is causing extra cost in the region server to Master > communication and increasing the footprint of Master heap. > Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511589/5179-90v18.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/844//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.94.0, 0.90.6, 0.92.1 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v18.txt, 5179-90v2.patch, > 5179-90v3.patch, 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, > 5179-90v7.patch, 5179-90v8.patch, 5179-90v9.patch, 5179-92v17.patch, > 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, > Errorlog, hbase-5179.patch, hbase-5179v10.patch, hbase-5179v12.patch, > hbase-5179v17.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Attachment: 5179-90v18.txt Patch v18 addresses Stack's comments. The sleep() isn't for unit test. I lowered wait interval to 500ms. I created waitUntilNoLogDir(HServerAddress serverAddress) so that -ROOT- and .META. servers can reuse the logic. Renamed logDirExists() to getLogDirIfExists() > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.94.0, 0.90.6, 0.92.1 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v18.txt, 5179-90v2.patch, > 5179-90v3.patch, 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, > 5179-90v7.patch, 5179-90v8.patch, 5179-90v9.patch, 5179-92v17.patch, > 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, > Errorlog, hbase-5179.patch, hbase-5179v10.patch, hbase-5179v12.patch, > hbase-5179v17.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5243) LogSyncerThread not getting shutdown waiting for the interrupted flag
[ https://issues.apache.org/jira/browse/HBASE-5243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5243: -- Attachment: 5243-92.addendum The addendum fixes broken 0.92 build > LogSyncerThread not getting shutdown waiting for the interrupted flag > - > > Key: HBASE-5243 > URL: https://issues.apache.org/jira/browse/HBASE-5243 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.5 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.90.6, 0.92.1 > > Attachments: 5243-92.addendum, HBASE-5243_0.90.patch, > HBASE-5243_0.90_1.patch, HBASE-5243_trunk.patch > > > In the LogSyncer run() we keep looping till this.isInterrupted flag is set. > But in some cases the DFSclient is consuming the Interrupted exception. So > we are running into infinite loop in some shutdown cases. > I would suggest that as we are the ones who tries to close down the > LogSyncerThread we can introduce a variable like > Close or shutdown and based on the state of this flag along with > isInterrupted() we can make the thread stop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5255) Use singletons for OperationStatus to save memory
[
https://issues.apache.org/jira/browse/HBASE-5255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5255:
--
Attachment: 5255-92.txt
Patch for 0.92 branch
> Use singletons for OperationStatus to save memory
> -
>
> Key: HBASE-5255
> URL: https://issues.apache.org/jira/browse/HBASE-5255
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 0.92.0, 0.90.5
>Reporter: Benoit Sigoure
>Assignee: Benoit Sigoure
>Priority: Minor
> Labels: performance
> Fix For: 0.94.0, 0.92.1
>
> Attachments: 5255-92.txt, 5255-v2.txt,
> HBASE-5255-0.92-Use-singletons-to-remove-unnecessary-memory-allocati.patch,
> HBASE-5255-trunk-Use-singletons-to-remove-unnecessary-memory-allocati.patch
>
>
> Every single {{Put}} causes the allocation of at least one
> {{OperationStatus}}, yet {{OperationStatus}} is almost always stateless, so
> these allocations are unnecessary and could be avoided. Attached patch adds
> a few singletons and uses them, with no public API change. I didn't test the
> patches, but you get the idea.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5231) Backport HBASE-3373 (per-table load balancing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5231: -- Attachment: 5231-v2.txt Patch v2 which I am going to integrate later today. > Backport HBASE-3373 (per-table load balancing) to 0.92 > -- > > Key: HBASE-5231 > URL: https://issues.apache.org/jira/browse/HBASE-5231 > Project: HBase > Issue Type: Improvement >Reporter: Zhihong Yu > Fix For: 0.92.1 > > Attachments: 5231-v2.txt, 5231.txt > > > This JIRA backports per-table load balancing to 0.90 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5255) Use singletons for OperationStatus to save memory
[
https://issues.apache.org/jira/browse/HBASE-5255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5255:
--
Attachment: 5255-v2.txt
Patch v2 makes exceptionMsg and code fields final.
> Use singletons for OperationStatus to save memory
> -
>
> Key: HBASE-5255
> URL: https://issues.apache.org/jira/browse/HBASE-5255
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 0.92.0, 0.90.5
>Reporter: Benoit Sigoure
>Assignee: Benoit Sigoure
>Priority: Minor
> Labels: performance
> Fix For: 0.94.0, 0.92.1
>
> Attachments: 5255-v2.txt,
> HBASE-5255-0.92-Use-singletons-to-remove-unnecessary-memory-allocati.patch,
> HBASE-5255-trunk-Use-singletons-to-remove-unnecessary-memory-allocati.patch
>
>
> Every single {{Put}} causes the allocation of at least one
> {{OperationStatus}}, yet {{OperationStatus}} is almost always stateless, so
> these allocations are unnecessary and could be avoided. Attached patch adds
> a few singletons and uses them, with no public API change. I didn't test the
> patches, but you get the idea.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5139) Compute (weighted) median using AggregateProtocol
[ https://issues.apache.org/jira/browse/HBASE-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5139: -- Attachment: 5139.addendum Addendum that handles startRow being null for the case where median is in the first region. > Compute (weighted) median using AggregateProtocol > - > > Key: HBASE-5139 > URL: https://issues.apache.org/jira/browse/HBASE-5139 > Project: HBase > Issue Type: Sub-task >Reporter: Zhihong Yu >Assignee: Zhihong Yu > Attachments: 5139-v2.txt, 5139.addendum > > > Suppose cf:cq1 stores numeric values and optionally cf:cq2 stores weights. > This task finds out the median value among the values of cf:cq1 (See > http://www.stat.ucl.ac.be/ISdidactique/Rhelp/library/R.basic/html/weighted.median.html) > This can be done in two passes. > The first pass utilizes AggregateProtocol where the following tuple is > returned from each region: > (partial-sum-of-values, partial-sum-of-weights) > The start rowkey (supplied by coprocessor framework) would be used to sort > the tuples. This way we can determine which region (called R) contains the > (weighted) median. partial-sum-of-weights can be 0 if unweighted median is > sought > The second pass involves scanning the table, beginning with startrow of > region R and computing partial (weighted) sum until the threshold of S/2 is > crossed. The (weighted) median is returned. > However, this approach wouldn't work if there is mutation in the underlying > table between pass one and pass two. > In that case, sequential scanning seems to be the solution which is slower > than the above approach. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5246) Regenerate code with thrift 0.8.0
[ https://issues.apache.org/jira/browse/HBASE-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5246: -- Fix Version/s: 0.94.0 > Regenerate code with thrift 0.8.0 > - > > Key: HBASE-5246 > URL: https://issues.apache.org/jira/browse/HBASE-5246 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Scott Chen >Assignee: Scott Chen >Priority: Minor > Fix For: 0.94.0 > > Attachments: HBASE-5246.D1371.1.patch > > > In HBASE-5201, We have upgrated libthrift.jar to 0.8.0. > The source codes generated from "*.thrift" should be upgraded also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5246) Regenerate code with thrift 0.8.0
[ https://issues.apache.org/jira/browse/HBASE-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5246: -- Hadoop Flags: Reviewed Summary: Regenerate code with thrift 0.8.0 (was: Regenerate codes with thrift 0.8.0) > Regenerate code with thrift 0.8.0 > - > > Key: HBASE-5246 > URL: https://issues.apache.org/jira/browse/HBASE-5246 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Scott Chen >Assignee: Scott Chen >Priority: Minor > Attachments: HBASE-5246.D1371.1.patch > > > In HBASE-5201, We have upgrated libthrift.jar to 0.8.0. > The source codes generated from "*.thrift" should be upgraded also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5246) Regenerate codes with thrift 0.8.0
[ https://issues.apache.org/jira/browse/HBASE-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5246: -- Status: Patch Available (was: Open) > Regenerate codes with thrift 0.8.0 > -- > > Key: HBASE-5246 > URL: https://issues.apache.org/jira/browse/HBASE-5246 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Scott Chen >Assignee: Scott Chen >Priority: Minor > Attachments: HBASE-5246.D1371.1.patch > > > In HBASE-5201, We have upgrated libthrift.jar to 0.8.0. > The source codes generated from "*.thrift" should be upgraded also. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5237) Addendum for HBASE-5160 and HBASE-4397
[
https://issues.apache.org/jira/browse/HBASE-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5237:
--
Status: Patch Available (was: Open)
> Addendum for HBASE-5160 and HBASE-4397
> --
>
> Key: HBASE-5237
> URL: https://issues.apache.org/jira/browse/HBASE-5237
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.5
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.92.0, 0.90.6
>
> Attachments: HBASE-5237_0.90.patch, HBASE-5237_trunk.patch
>
>
> As part of HBASE-4397 there is one more scenario where the patch has to be
> applied.
> {code}
> RegionPlan plan = getRegionPlan(state, forceNewPlan);
> if (plan == null) {
> debugLog(state.getRegion(),
> "Unable to determine a plan to assign " + state);
> return; // Should get reassigned later when RIT times out.
> }
> {code}
> I think in this scenario also
> {code}
> this.timeoutMonitor.setAllRegionServersOffline(true);
> {code}
> this should be done.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511367/5179-92v17.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/823//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-92v17.patch, 5179-v11-92.txt, > 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, Errorlog, > hbase-5179.patch, hbase-5179v10.patch, hbase-5179v12.patch, > hbase-5179v17.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Attachment: 5179-92v17.patch Patch for 0.92 > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-92v17.patch, 5179-v11-92.txt, > 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, Errorlog, > hbase-5179.patch, hbase-5179v10.patch, hbase-5179v12.patch, > hbase-5179v17.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Attachment: hbase-5179v17.patch Patch v17 for trunk. > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, > 5179-v3.txt, 5179-v4.txt, Errorlog, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v12.patch, hbase-5179v17.patch, hbase-5179v5.patch, > hbase-5179v6.patch, hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Attachment: (was: 5179-92v17.patch) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, > 5179-v3.txt, 5179-v4.txt, Errorlog, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v12.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Attachment: (was: hbase-5179v17.patch) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-92v17.patch, 5179-v11-92.txt, > 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, Errorlog, > hbase-5179.patch, hbase-5179v10.patch, hbase-5179v12.patch, > hbase-5179v5.patch, hbase-5179v6.patch, hbase-5179v7.patch, > hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: v17patch for 92) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-92v17.patch, 5179-v11-92.txt, > 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, Errorlog, > hbase-5179.patch, hbase-5179v10.patch, hbase-5179v12.patch, > hbase-5179v17.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511358/5179-92v17.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/821//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-92v17.patch, 5179-v11-92.txt, > 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, Errorlog, > hbase-5179.patch, hbase-5179v10.patch, hbase-5179v12.patch, > hbase-5179v17.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Attachment: (was: hbase-5179v17.patch) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-92v17.patch, 5179-v11-92.txt, > 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, Errorlog, > hbase-5179.patch, hbase-5179v10.patch, hbase-5179v12.patch, > hbase-5179v17.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4913) Per-CF compaction Via the Shell
[ https://issues.apache.org/jira/browse/HBASE-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-4913: -- Status: Patch Available (was: Open) > Per-CF compaction Via the Shell > --- > > Key: HBASE-4913 > URL: https://issues.apache.org/jira/browse/HBASE-4913 > Project: HBase > Issue Type: Sub-task > Components: client, regionserver >Reporter: Nicolas Spiegelberg >Assignee: Mubarak Seyed > Fix For: 0.94.0 > > Attachments: HBASE-4913.trunk.v1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5241) Deletes should not mask Puts that come after it.
[ https://issues.apache.org/jira/browse/HBASE-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5241: -- Description: Suppose that we have a delete row, and then followed by the put. The delete row can mask the put, unless there was a major compaction in between. Now that we are flushing the memstoreTS to disk, along with the KVs, we should be able to differentiate whether or not the Put happened after the Delete and offer better delete semantics. Couldn't find a pre-existing JIRA that already discusses this, so creating one. Seems related to https://issues.apache.org/jira/browse/HBASE-2406, but is not quite the same. was: Suppose that we have a delete row, and then followed by the put. The delete row can mask the put, unless there was a major compaction in between. Now that we are flushing the memstoreTS to disk, along with the KVs, we should be able to differentiate weather or not the Put happened after the Delete and offer better delete semantics. Couldn't find a pre-existing JIRA that already discusses this, so creating one. Seems related to https://issues.apache.org/jira/browse/HBASE-2406, but is not quiet the same. Priority: Major (was: Minor) Hadoop Flags: Incompatible change This is a nice initiative. > Deletes should not mask Puts that come after it. > > > Key: HBASE-5241 > URL: https://issues.apache.org/jira/browse/HBASE-5241 > Project: HBase > Issue Type: Improvement >Reporter: Amitanand Aiyer > > Suppose that we have a delete row, and then followed by the put. The delete > row > can mask the put, unless there was a major compaction in between. > Now that we are flushing the memstoreTS to disk, along with the KVs, we > should be able > to differentiate whether or not the Put happened after the Delete and offer > better > delete semantics. > Couldn't find a pre-existing JIRA that already discusses this, so creating > one. > Seems related to https://issues.apache.org/jira/browse/HBASE-2406, but is not > quite the same. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5225) Backport HBASE-3845 -data loss because lastSeqWritten can miss memstore edits
[ https://issues.apache.org/jira/browse/HBASE-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5225: -- Description: Critical defect. Patch from HBASE-3845 was not integrated to 0.90. (was: Critical defect. not merged to 0.90.) > Backport HBASE-3845 -data loss because lastSeqWritten can miss memstore edits > - > > Key: HBASE-5225 > URL: https://issues.apache.org/jira/browse/HBASE-5225 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.4 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.90.6 > > Attachments: HBASE-3845-90.patch > > > Critical defect. Patch from HBASE-3845 was not integrated to 0.90. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511281/5179-90v17.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/817//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, > 5179-v3.txt, 5179-v4.txt, Errorlog, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v12.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5231) Backport HBASE-3373 (per-table load balancing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5231: -- Fix Version/s: 0.92.1 > Backport HBASE-3373 (per-table load balancing) to 0.92 > -- > > Key: HBASE-5231 > URL: https://issues.apache.org/jira/browse/HBASE-5231 > Project: HBase > Issue Type: Improvement >Reporter: Zhihong Yu > Fix For: 0.92.1 > > Attachments: 5231.txt > > > This JIRA backports per-table load balancing to 0.90 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Attachment: 5179-90v17.txt Patch v17 makes all new methods in DeadServer package private. Also aligned new method names in DeadServer with existing method names. Wrapped long lines. This version should be close to Stack's standard. > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, > 5179-v3.txt, 5179-v4.txt, Errorlog, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v12.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12510304/hbase-5179v5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -147 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 80 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.regionserver.wal.TestHLog org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/741//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/741//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/741//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v17.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, > 5179-v3.txt, 5179-v4.txt, Errorlog, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v12.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12510277/5179-v4.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -147 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/737//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/737//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/737//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v2.patch, 5179-90v3.patch, 5179-90v4.patch, > 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, 5179-90v8.patch, > 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, > 5179-v4.txt, Errorlog, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v12.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12510206/5179-v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -147 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 78 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestImportTsv Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/730//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/730//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/730//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v2.patch, 5179-90v3.patch, 5179-90v4.patch, > 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, 5179-90v8.patch, > 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, > 5179-v4.txt, Errorlog, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v12.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511256/5179-90v16.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/816//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v16.patch, 5179-90v2.patch, 5179-90v3.patch, 5179-90v4.patch, > 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, 5179-90v8.patch, > 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, > 5179-v4.txt, Errorlog, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v12.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5223) TestMetaReaderEditor is missing call to CatalogTracker.stop()
[ https://issues.apache.org/jira/browse/HBASE-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5223: -- Resolution: Fixed Assignee: Zhihong Yu Status: Resolved (was: Patch Available) > TestMetaReaderEditor is missing call to CatalogTracker.stop() > - > > Key: HBASE-5223 > URL: https://issues.apache.org/jira/browse/HBASE-5223 > Project: HBase > Issue Type: Test >Reporter: Zhihong Yu >Assignee: Zhihong Yu > Fix For: 0.94.0, 0.92.1 > > Attachments: 5223.txt > > > I noticed that TestMetaReaderEditor hung on 0.92 Jenkins builds - see > https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/249/console > It turns out that CatalogTracker.stop() is missing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511218/5179-90v15.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/815//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v2.patch, 5179-90v3.patch, 5179-90v4.patch, 5179-90v5.patch, > 5179-90v6.patch, 5179-90v7.patch, 5179-90v8.patch, 5179-90v9.patch, > 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, > Errorlog, hbase-5179.patch, hbase-5179v10.patch, hbase-5179v12.patch, > hbase-5179v5.patch, hbase-5179v6.patch, hbase-5179v7.patch, > hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511216/5179-90v14.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/814//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v14.patch, 5179-90v15.patch, > 5179-90v2.patch, 5179-90v3.patch, 5179-90v4.patch, 5179-90v5.patch, > 5179-90v6.patch, 5179-90v7.patch, 5179-90v8.patch, 5179-90v9.patch, > 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, > Errorlog, hbase-5179.patch, hbase-5179v10.patch, hbase-5179v12.patch, > hbase-5179v5.patch, hbase-5179v6.patch, hbase-5179v7.patch, > hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5231) Backport HBASE-3373 (per-table load balancing) to 0.92
[
https://issues.apache.org/jira/browse/HBASE-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5231:
--
Attachment: 5231.txt
Patch for 0.92
The following tests pass:
{code}
751 mt -Dtest=TestRegionRebalancing
752 mt -Dtest=TestDefaultLoadBalancer
{code}
> Backport HBASE-3373 (per-table load balancing) to 0.92
> --
>
> Key: HBASE-5231
> URL: https://issues.apache.org/jira/browse/HBASE-5231
> Project: HBase
> Issue Type: Improvement
>Reporter: Zhihong Yu
> Attachments: 5231.txt
>
>
> This JIRA backports per-table load balancing to 0.90
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511158/5179-90v13.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/813//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, > 5179-v3.txt, 5179-v4.txt, Errorlog, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v12.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Attachment: 5179-90v13.txt > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v13.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, > 5179-v3.txt, 5179-v4.txt, Errorlog, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v12.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5231) Backport HBASE-3373 (per-table load balancing) to 0.92
[ https://issues.apache.org/jira/browse/HBASE-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5231: -- Summary: Backport HBASE-3373 (per-table load balancing) to 0.92 (was: Backport HBASE-3373 (per-table load balancing) to 0.90) Thanks for the reminder, J-D. This should go to 0.92 first. > Backport HBASE-3373 (per-table load balancing) to 0.92 > -- > > Key: HBASE-5231 > URL: https://issues.apache.org/jira/browse/HBASE-5231 > Project: HBase > Issue Type: Improvement >Reporter: Zhihong Yu > > This JIRA backports per-table load balancing to 0.90 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511134/Errorlog against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/812//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v12.patch, 5179-90v2.patch, 5179-90v3.patch, 5179-90v4.patch, > 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, 5179-90v8.patch, > 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, > 5179-v4.txt, Errorlog, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v12.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2315) BookKeeper for write-ahead logging
[ https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-2315: -- Comment: was deleted (was: I am out of office in an all-day business meeting today and tomorrow, I will not be checking email. If this is urgent then please call my cell phone (or send an SMS), otherwise I will reply to your email when I get back. Thanks for your patience, -- amr ) > BookKeeper for write-ahead logging > -- > > Key: HBASE-2315 > URL: https://issues.apache.org/jira/browse/HBASE-2315 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: Flavio Junqueira > Attachments: HBASE-2315.patch, bookkeeperOverview.pdf, > zookeeper-dev-bookkeeper.jar > > > BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high > throughput write-ahead logging service. This issue provides an implementation > of write-ahead logging for hbase using BookKeeper. Apart from expected > throughput improvements, BookKeeper also has stronger durability guarantees > compared to the implementation currently used by hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Attachment: 5179-90v11.patch Patch v11 for 0.90 branch. Applied same simplification technique to DeadServer.java > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v11.patch, > 5179-90v2.patch, 5179-90v3.patch, 5179-90v4.patch, 5179-90v5.patch, > 5179-90v6.patch, 5179-90v7.patch, 5179-90v8.patch, 5179-90v9.patch, > 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, > hbase-5179.patch, hbase-5179v10.patch, hbase-5179v5.patch, > hbase-5179v6.patch, hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5204) Backward compatibility fixes for 0.92
[
https://issues.apache.org/jira/browse/HBASE-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5204:
--
Attachment: (was: 5204.addendum)
> Backward compatibility fixes for 0.92
> -
>
> Key: HBASE-5204
> URL: https://issues.apache.org/jira/browse/HBASE-5204
> Project: HBase
> Issue Type: Bug
> Components: ipc
>Reporter: Benoit Sigoure
>Assignee: Benoit Sigoure
>Priority: Blocker
> Labels: backwards-compatibility
> Fix For: 0.92.0
>
> Attachments:
> 0001-Add-some-backward-compatible-support-for-reading-old.patch,
> 0002-Make-sure-that-a-connection-always-uses-a-protocol.patch,
> 0003-Change-the-code-used-when-serializing-HTableDescript.patch, 5204-92.txt,
> 5204-trunk.txt, 5204.addendum
>
>
> Attached are 3 patches that are necessary to allow compatibility between
> HBase 0.90.x (and previous releases) and HBase 0.92.0.
> First of all, I'm well aware that 0.92.0 RC4 has been thumbed up by a lot of
> people and would probably wind up being released as 0.92.0 tomorrow, so I
> sincerely apologize for creating this issue so late in the process. I spent
> a lot of time trying to work around the quirks of 0.92 but once I realized
> that with a few very quasi-trivial changes compatibility would be made
> significantly easier, I immediately sent these 3 patches to Stack, who
> suggested I create this issue.
> The first patch is required as without it clients sending a 0.90-style RPC to
> a 0.92-style server causes the server to die uncleanly. It seems that 0.92
> ships with {{\-XX:OnOutOfMemoryError="kill \-9 %p"}}, and when a 0.92 server
> fails to deserialize a 0.90-style RPC, it attempts to allocate a large buffer
> because it doesn't read fields of 0.90-style RPCs properly. This allocation
> attempt immediately triggers an OOME, which causes the JVM to die abruptly of
> a {{SIGKILL}}. So whenever a 0.90.x client attempts to connect to HBase, it
> kills whichever RS is hosting the {{\-ROOT-}} region.
> The second patch fixes a bug introduced by HBASE-2002, which added support
> for letting clients specify what "protocol" they want to speak. If a client
> doesn't properly specify what protocol to use, the connection's {{protocol}}
> field will be left {{null}}, which causes any subsequent RPC on that
> connection to trigger an NPE in the server, even though the connection was
> successfully established from the client's point of view. The fix is to
> simply give the connection a default protocol, by assuming the client meant
> to speak to a RegionServer.
> The third patch fixes an oversight that slipped in HBASE-451, where a change
> to {{HbaseObjectWritable}} caused all the codes used to serialize
> {{Writables}} to shift by one. This was carefully avoided in other changes
> such as HBASE-1502, which cleanly removed entries for {{HMsg}} and
> {{HMsg[]}}, so I don't think this breakage in HBASE-451 was intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5204) Backward compatibility fixes for 0.92
[
https://issues.apache.org/jira/browse/HBASE-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5204:
--
Attachment: 5204.addendum
This is the correct addendum.
> Backward compatibility fixes for 0.92
> -
>
> Key: HBASE-5204
> URL: https://issues.apache.org/jira/browse/HBASE-5204
> Project: HBase
> Issue Type: Bug
> Components: ipc
>Reporter: Benoit Sigoure
>Assignee: Benoit Sigoure
>Priority: Blocker
> Labels: backwards-compatibility
> Fix For: 0.92.0
>
> Attachments:
> 0001-Add-some-backward-compatible-support-for-reading-old.patch,
> 0002-Make-sure-that-a-connection-always-uses-a-protocol.patch,
> 0003-Change-the-code-used-when-serializing-HTableDescript.patch, 5204-92.txt,
> 5204-trunk.txt, 5204.addendum
>
>
> Attached are 3 patches that are necessary to allow compatibility between
> HBase 0.90.x (and previous releases) and HBase 0.92.0.
> First of all, I'm well aware that 0.92.0 RC4 has been thumbed up by a lot of
> people and would probably wind up being released as 0.92.0 tomorrow, so I
> sincerely apologize for creating this issue so late in the process. I spent
> a lot of time trying to work around the quirks of 0.92 but once I realized
> that with a few very quasi-trivial changes compatibility would be made
> significantly easier, I immediately sent these 3 patches to Stack, who
> suggested I create this issue.
> The first patch is required as without it clients sending a 0.90-style RPC to
> a 0.92-style server causes the server to die uncleanly. It seems that 0.92
> ships with {{\-XX:OnOutOfMemoryError="kill \-9 %p"}}, and when a 0.92 server
> fails to deserialize a 0.90-style RPC, it attempts to allocate a large buffer
> because it doesn't read fields of 0.90-style RPCs properly. This allocation
> attempt immediately triggers an OOME, which causes the JVM to die abruptly of
> a {{SIGKILL}}. So whenever a 0.90.x client attempts to connect to HBase, it
> kills whichever RS is hosting the {{\-ROOT-}} region.
> The second patch fixes a bug introduced by HBASE-2002, which added support
> for letting clients specify what "protocol" they want to speak. If a client
> doesn't properly specify what protocol to use, the connection's {{protocol}}
> field will be left {{null}}, which causes any subsequent RPC on that
> connection to trigger an NPE in the server, even though the connection was
> successfully established from the client's point of view. The fix is to
> simply give the connection a default protocol, by assuming the client meant
> to speak to a RegionServer.
> The third patch fixes an oversight that slipped in HBASE-451, where a change
> to {{HbaseObjectWritable}} caused all the codes used to serialize
> {{Writables}} to shift by one. This was carefully avoided in other changes
> such as HBASE-1502, which cleanly removed entries for {{HMsg}} and
> {{HMsg[]}}, so I don't think this breakage in HBASE-451 was intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511030/5179-90v10.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/809//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v2.patch, > 5179-90v3.patch, 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, > 5179-90v7.patch, 5179-90v8.patch, 5179-90v9.patch, 5179-v11-92.txt, > 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, hbase-5179.patch, > hbase-5179v10.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Attachment: 5179-90v10.patch Patch for 0.90 branch > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v10.patch, 5179-90v2.patch, > 5179-90v3.patch, 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, > 5179-90v7.patch, 5179-90v8.patch, 5179-90v9.patch, 5179-v11-92.txt, > 5179-v11.txt, 5179-v2.txt, 5179-v3.txt, 5179-v4.txt, hbase-5179.patch, > hbase-5179v10.patch, hbase-5179v5.patch, hbase-5179v6.patch, > hbase-5179v7.patch, hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5179) Concurrent processing of processFaileOver and ServerShutdownHandler may cause region to be assigned before log splitting is completed, causing data loss
[ https://issues.apache.org/jira/browse/HBASE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5179: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511020/5179-v11-92.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/807//console This message is automatically generated.) > Concurrent processing of processFaileOver and ServerShutdownHandler may cause > region to be assigned before log splitting is completed, causing data loss > > > Key: HBASE-5179 > URL: https://issues.apache.org/jira/browse/HBASE-5179 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.6 > > Attachments: 5179-90.txt, 5179-90v2.patch, 5179-90v3.patch, > 5179-90v4.patch, 5179-90v5.patch, 5179-90v6.patch, 5179-90v7.patch, > 5179-90v8.patch, 5179-90v9.patch, 5179-v11-92.txt, 5179-v11.txt, 5179-v2.txt, > 5179-v3.txt, 5179-v4.txt, hbase-5179.patch, hbase-5179v10.patch, > hbase-5179v5.patch, hbase-5179v6.patch, hbase-5179v7.patch, > hbase-5179v8.patch, hbase-5179v9.patch > > > If master's processing its failover and ServerShutdownHandler's processing > happen concurrently, it may appear following case. > 1.master completed splitLogAfterStartup() > 2.RegionserverA restarts, and ServerShutdownHandler is processing. > 3.master starts to rebuildUserRegions, and RegionserverA is considered as > dead server. > 4.master starts to assign regions of RegionserverA because it is a dead > server by step3. > However, when doing step4(assigning region), ServerShutdownHandler may be > doing split log, Therefore, it may cause data loss. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5204) Backward compatibility fixes for 0.92
[
https://issues.apache.org/jira/browse/HBASE-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5204:
--
Attachment: 5204.addendum
How about this addendum ?
> Backward compatibility fixes for 0.92
> -
>
> Key: HBASE-5204
> URL: https://issues.apache.org/jira/browse/HBASE-5204
> Project: HBase
> Issue Type: Bug
> Components: ipc
>Reporter: Benoit Sigoure
>Assignee: Benoit Sigoure
>Priority: Blocker
> Labels: backwards-compatibility
> Fix For: 0.92.0
>
> Attachments:
> 0001-Add-some-backward-compatible-support-for-reading-old.patch,
> 0002-Make-sure-that-a-connection-always-uses-a-protocol.patch,
> 0003-Change-the-code-used-when-serializing-HTableDescript.patch, 5204-92.txt,
> 5204-trunk.txt, 5204.addendum
>
>
> Attached are 3 patches that are necessary to allow compatibility between
> HBase 0.90.x (and previous releases) and HBase 0.92.0.
> First of all, I'm well aware that 0.92.0 RC4 has been thumbed up by a lot of
> people and would probably wind up being released as 0.92.0 tomorrow, so I
> sincerely apologize for creating this issue so late in the process. I spent
> a lot of time trying to work around the quirks of 0.92 but once I realized
> that with a few very quasi-trivial changes compatibility would be made
> significantly easier, I immediately sent these 3 patches to Stack, who
> suggested I create this issue.
> The first patch is required as without it clients sending a 0.90-style RPC to
> a 0.92-style server causes the server to die uncleanly. It seems that 0.92
> ships with {{\-XX:OnOutOfMemoryError="kill \-9 %p"}}, and when a 0.92 server
> fails to deserialize a 0.90-style RPC, it attempts to allocate a large buffer
> because it doesn't read fields of 0.90-style RPCs properly. This allocation
> attempt immediately triggers an OOME, which causes the JVM to die abruptly of
> a {{SIGKILL}}. So whenever a 0.90.x client attempts to connect to HBase, it
> kills whichever RS is hosting the {{\-ROOT-}} region.
> The second patch fixes a bug introduced by HBASE-2002, which added support
> for letting clients specify what "protocol" they want to speak. If a client
> doesn't properly specify what protocol to use, the connection's {{protocol}}
> field will be left {{null}}, which causes any subsequent RPC on that
> connection to trigger an NPE in the server, even though the connection was
> successfully established from the client's point of view. The fix is to
> simply give the connection a default protocol, by assuming the client meant
> to speak to a RegionServer.
> The third patch fixes an oversight that slipped in HBASE-451, where a change
> to {{HbaseObjectWritable}} caused all the codes used to serialize
> {{Writables}} to shift by one. This was carefully avoided in other changes
> such as HBASE-1502, which cleanly removed entries for {{HMsg}} and
> {{HMsg[]}}, so I don't think this breakage in HBASE-451 was intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
