[ https://issues.apache.org/jira/browse/HBASE-14420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024722#comment-15024722 ]
stack commented on HBASE-14420: ------------------------------- In 1.3 build, saw this just now in https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3/jdk=latest1.8,label=Hadoop/393/consoleText kalashnikov:hbase.git stack$ python ./dev-support/findHangingTests.py https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.3/jdk=latest1.8,label=Hadoop/393/consoleText Fetching the console output from the URL Printing hanging tests Hanging test : org.apache.hadoop.hbase.mapreduce.TestImportTSVWithOperationAttributes Hanging test : org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed ... Looks like stuff not being shutdown if I look in the output. Wonder why zombie detector not finding these? Because they die out before end of test? It seems to be the test mapreduce.TestImportTSVWithOperationAttributes#testMROnTableWithInvalidOperationAttr that is problematic.... we stop reporting from this test in the output. > Zombie Stomping Session > ----------------------- > > Key: HBASE-14420 > URL: https://issues.apache.org/jira/browse/HBASE-14420 > Project: HBase > Issue Type: Umbrella > Components: test > Reporter: stack > Assignee: stack > Priority: Critical > Attachments: hangers.txt, none_fix (1).txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt, > none_fix.txt, none_fix.txt, none_fix.txt, none_fix.txt > > > Patch build are now failing most of the time because we are dropping zombies. > I confirm we are doing this on non-apache build boxes too. > Left-over zombies consume resources on build boxes (OOME cannot create native > threads). Having to do multiple test runs in the hope that we can get a > non-zombie-making build or making (arbitrary) rulings that the zombies are > 'not related' is a productivity sink. And so on... > This is an umbrella issue for a zombie stomping session that started earlier > this week. Will hang sub-issues of this one. Am running builds back-to-back > on little cluster to turn out the monsters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)