[
https://issues.apache.org/jira/browse/HBASE-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4965:
-------------------------
Resolution: Fixed
Fix Version/s: 0.94.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed trunk.
Outstanding then is addressing these leaks. You've done the more onerous ones
N? Should we make issues for the rest?
On the ulimit thingy, Giri said he'd take a look. I'll follow up w/ him.
Good stuff.
Thanks for being persistent. If issues w/ this patch post-commit we can make
new issues to address.
> Monitor the open file descriptors and the threads counters during the unit
> tests
> --------------------------------------------------------------------------------
>
> Key: HBASE-4965
> URL: https://issues.apache.org/jira/browse/HBASE-4965
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.94.0
> Environment: all
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4965-v2.txt, 4965-v2.txt, 4965-v3.txt, 4965_all.patch,
> 4965_all.patch, 4965_all.patch, ResourceChecker.java,
> ResourceCheckerJUnitRule.java
>
>
> We're seeing a lot of issues with hadoop-qa related to threads or file
> descriptors.
> Monitoring these counters would ease the analysis.
> Note as well that
> - if we want to execute the tests in the same jvm (because the test is small
> or because we want to share the cluster) we can't afford to leak too many
> resources
> - if the tests leak, it's more difficult to detect a leak in the software
> itself.
> I attach piece of code that I used. It requires two lines in a unit test
> class to:
> - before every test, count the threads and the open file descriptor
> - after every test, compare with the previous value.
> I ran it on some tests; we have for example:
> - client.TestMultiParallel#testBatchWithManyColsInOneRowGetAndPut: 232
> threads (was 231), 390 file descriptors (was 390). => TestMultiParallel uses
> 232 threads!
> - client.TestMultipleTimestamps#testWithColumnDeletes: 152 threads (was 151),
> 283 file descriptors (was 282).
> - client.TestAdmin#testCheckHBaseAvailableClosesConnection: 477 threads (was
> 294), 815 file descriptors (was 461)
> - client.TestMetaMigrationRemovingHTD#testMetaMigration: 149 threads (was
> 148), 310 file descriptors (was 307).
> It's not always leaks, we can expect some pooling effects. But still...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
