[jira] [Updated] (HBASE-5141) Memory leak in MonitoredRPCHandlerImpl
[ https://issues.apache.org/jira/browse/HBASE-5141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-5141: -- Attachment: Screen Shot 2012-01-06 at 3.03.09 PM.png This screen shot shows how MonitoredRPCHandlerImpl are all using 6% of the heap because they are holding on the packets. > Memory leak in MonitoredRPCHandlerImpl > -- > > Key: HBASE-5141 > URL: https://issues.apache.org/jira/browse/HBASE-5141 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Jean-Daniel Cryans >Priority: Blocker > Fix For: 0.92.0, 0.94.0 > > Attachments: Screen Shot 2012-01-06 at 3.03.09 PM.png > > > I got a pretty reliable way of OOME'ing my region servers. Using a big > payload (64MB in my case), a default heap and default number of handlers, > it's not too long that all the MonitoredRPCHandlerImpl hold on a 64MB > reference and once a compaction kicks in it kills everything. > The issue is that even after the RPC call is done, the packet still lives in > MonitoredRPCHandlerImpl. > Will attach a screen shot of jprofiler's analysis in a moment. > This is a blocker for 0.92.0, anyone using a high number of handlers and > bigish values will kill themselves. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5141) Memory leak in MonitoredRPCHandlerImpl
[ https://issues.apache.org/jira/browse/HBASE-5141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-5141: -- Attachment: HBASE-5141.patch Testing this patch, I like it better than passing null as the monitor takes care of itself. > Memory leak in MonitoredRPCHandlerImpl > -- > > Key: HBASE-5141 > URL: https://issues.apache.org/jira/browse/HBASE-5141 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Jean-Daniel Cryans >Priority: Blocker > Fix For: 0.92.0, 0.94.0 > > Attachments: HBASE-5141.patch, Screen Shot 2012-01-06 at 3.03.09 > PM.png > > > I got a pretty reliable way of OOME'ing my region servers. Using a big > payload (64MB in my case), a default heap and default number of handlers, > it's not too long that all the MonitoredRPCHandlerImpl hold on a 64MB > reference and once a compaction kicks in it kills everything. > The issue is that even after the RPC call is done, the packet still lives in > MonitoredRPCHandlerImpl. > Will attach a screen shot of jprofiler's analysis in a moment. > This is a blocker for 0.92.0, anyone using a high number of handlers and > bigish values will kill themselves. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5141) Memory leak in MonitoredRPCHandlerImpl
[ https://issues.apache.org/jira/browse/HBASE-5141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-5141: -- Assignee: Jean-Daniel Cryans Status: Patch Available (was: Open) > Memory leak in MonitoredRPCHandlerImpl > -- > > Key: HBASE-5141 > URL: https://issues.apache.org/jira/browse/HBASE-5141 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Jean-Daniel Cryans >Assignee: Jean-Daniel Cryans >Priority: Blocker > Fix For: 0.92.0, 0.94.0 > > Attachments: HBASE-5141-v2.patch, HBASE-5141.patch, Screen Shot > 2012-01-06 at 3.03.09 PM.png > > > I got a pretty reliable way of OOME'ing my region servers. Using a big > payload (64MB in my case), a default heap and default number of handlers, > it's not too long that all the MonitoredRPCHandlerImpl hold on a 64MB > reference and once a compaction kicks in it kills everything. > The issue is that even after the RPC call is done, the packet still lives in > MonitoredRPCHandlerImpl. > Will attach a screen shot of jprofiler's analysis in a moment. > This is a blocker for 0.92.0, anyone using a high number of handlers and > bigish values will kill themselves. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5141) Memory leak in MonitoredRPCHandlerImpl
[ https://issues.apache.org/jira/browse/HBASE-5141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-5141: -- Attachment: HBASE-5141-v2.patch This second patch survives my little test. What I was missing was that the packet also contains a reference to the params, so I have to clear out both (that was a bit confusing). > Memory leak in MonitoredRPCHandlerImpl > -- > > Key: HBASE-5141 > URL: https://issues.apache.org/jira/browse/HBASE-5141 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Jean-Daniel Cryans >Priority: Blocker > Fix For: 0.92.0, 0.94.0 > > Attachments: HBASE-5141-v2.patch, HBASE-5141.patch, Screen Shot > 2012-01-06 at 3.03.09 PM.png > > > I got a pretty reliable way of OOME'ing my region servers. Using a big > payload (64MB in my case), a default heap and default number of handlers, > it's not too long that all the MonitoredRPCHandlerImpl hold on a 64MB > reference and once a compaction kicks in it kills everything. > The issue is that even after the RPC call is done, the packet still lives in > MonitoredRPCHandlerImpl. > Will attach a screen shot of jprofiler's analysis in a moment. > This is a blocker for 0.92.0, anyone using a high number of handlers and > bigish values will kill themselves. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5141) Memory leak in MonitoredRPCHandlerImpl
[ https://issues.apache.org/jira/browse/HBASE-5141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5141: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Memory leak in MonitoredRPCHandlerImpl > -- > > Key: HBASE-5141 > URL: https://issues.apache.org/jira/browse/HBASE-5141 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Jean-Daniel Cryans >Assignee: Jean-Daniel Cryans >Priority: Blocker > Fix For: 0.92.0, 0.94.0 > > Attachments: HBASE-5141-v2.patch, HBASE-5141.patch, Screen Shot > 2012-01-06 at 3.03.09 PM.png > > > I got a pretty reliable way of OOME'ing my region servers. Using a big > payload (64MB in my case), a default heap and default number of handlers, > it's not too long that all the MonitoredRPCHandlerImpl hold on a 64MB > reference and once a compaction kicks in it kills everything. > The issue is that even after the RPC call is done, the packet still lives in > MonitoredRPCHandlerImpl. > Will attach a screen shot of jprofiler's analysis in a moment. > This is a blocker for 0.92.0, anyone using a high number of handlers and > bigish values will kill themselves. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
