[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars George updated HBASE-3852: --- Issue Type: Sub-task (was: Bug) Parent: HBASE-8818 ThriftServer leaks scanners --- Key: HBASE-3852 URL: https://issues.apache.org/jira/browse/HBASE-3852 Project: HBase Issue Type: Sub-task Components: Thrift Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Assignee: Lars George Attachments: 3852.txt, HBASE-3852-0.94.patch, thrift2-scanner.patch The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[
https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Daniel Cryans updated HBASE-3852:
--
Attachment: HBASE-3852-0.94.patch
Attaching patch.
It got a lot messier then I expected but that's because I forgot that that code
relies on the new scanner stuff that we used at SU.
So kind, of like the thrift2 patch, I use a wrapper for the scanners. This is
what you store in the scanner map. When you use it you first remove it from the
map and then later you put it back with a new timestamp.
Removing the scanner from the list has 2 nice properties: we don't race when
scanner the map for timeouts and if you hit this code in scannerGetList:
{code}
if (null == results) {
return new ArrayListTRowResult();
}
{code}
Then the scanner is already removed from the map so the leak there is fixed too.
ThriftServer leaks scanners
---
Key: HBASE-3852
URL: https://issues.apache.org/jira/browse/HBASE-3852
Project: HBase
Issue Type: Bug
Components: Thrift
Affects Versions: 0.90.2
Reporter: Jean-Daniel Cryans
Attachments: 3852.txt, HBASE-3852-0.94.patch, thrift2-scanner.patch
The scannerMap in ThriftServer relies on the user to clean it by closing the
scanner. If that doesn't happen, the ResultScanner will stay in the thrift
server's memory and if any pre-fetching was done, it will also start
accumulating Results (with all their data).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-3852: - Fix Version/s: (was: 0.94.5) This was moved from release to release, with nobody owning it. I am removing this from 0.94 altogether. Should probably be marked as Won't fix (I wish there was a Nobody cares state). Pull back in if you disagree. ThriftServer leaks scanners --- Key: HBASE-3852 URL: https://issues.apache.org/jira/browse/HBASE-3852 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Attachments: 3852.txt, thrift2-scanner.patch The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-3852: - Fix Version/s: (was: 0.94.4) 0.94.5 And on to 0.94.5 ThriftServer leaks scanners --- Key: HBASE-3852 URL: https://issues.apache.org/jira/browse/HBASE-3852 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Fix For: 0.94.5 Attachments: 3852.txt, thrift2-scanner.patch The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-3852: - Fix Version/s: (was: 0.94.3) 0.94.4 And on to 0.94.4 ThriftServer leaks scanners --- Key: HBASE-3852 URL: https://issues.apache.org/jira/browse/HBASE-3852 Project: HBase Issue Type: Bug Components: Thrift Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Fix For: 0.94.4 Attachments: 3852.txt, thrift2-scanner.patch The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-3852: - Fix Version/s: (was: 0.94.1) 0.94.2 ThriftServer leaks scanners --- Key: HBASE-3852 URL: https://issues.apache.org/jira/browse/HBASE-3852 Project: HBase Issue Type: Bug Components: thrift Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.94.2 Attachments: 3852.txt, thrift2-scanner.patch The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-3852: -- Component/s: thrift ThriftServer leaks scanners --- Key: HBASE-3852 URL: https://issues.apache.org/jira/browse/HBASE-3852 Project: HBase Issue Type: Bug Components: thrift Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.94.1 Attachments: 3852.txt, thrift2-scanner.patch The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-3852: - Fix Version/s: (was: 0.94.0) 0.94.1 Let's try for 0.94.1 ThriftServer leaks scanners --- Key: HBASE-3852 URL: https://issues.apache.org/jira/browse/HBASE-3852 Project: HBase Issue Type: Bug Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.94.1 Attachments: 3852.txt, thrift2-scanner.patch The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bob Copeland updated HBASE-3852: Attachment: thrift2-scanner.patch We hit this running thrift2 in production. With a lot of scanners buffering a lot (100) of rows per call, thrift servers would OOME in no time. This patch just periodically expires old ones. I can respin/rework the patch if the approach is sound. ThriftServer leaks scanners --- Key: HBASE-3852 URL: https://issues.apache.org/jira/browse/HBASE-3852 Project: HBase Issue Type: Bug Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.94.0 Attachments: 3852.txt, thrift2-scanner.patch The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-3852: -- Fix Version/s: (was: 0.92.0) 0.94.0 Moving out of 0.92 for now. ThriftServer leaks scanners --- Key: HBASE-3852 URL: https://issues.apache.org/jira/browse/HBASE-3852 Project: HBase Issue Type: Bug Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.94.0 Attachments: 3852.txt The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-3852: -- Attachment: 3852.txt TestThriftServer passes. ThriftServer leaks scanners --- Key: HBASE-3852 URL: https://issues.apache.org/jira/browse/HBASE-3852 Project: HBase Issue Type: Bug Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Assignee: Ted Yu Priority: Critical Fix For: 0.92.0 Attachments: 3852.txt The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
