[ https://issues.apache.org/jira/browse/CASSANDRA-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12965930#action_12965930 ]
Hudson commented on CASSANDRA-1752: ----------------------------------- Integrated in Cassandra-0.6 #14 (See [https://hudson.apache.org/hudson/job/Cassandra-0.6/14/]) avoid opening readers on anticompacted to-be-streamed temporary files patch by thobbs; reviewed by mdennis and jbellis for CASSANDRA-1752 > repair leaving FDs unclosed > --------------------------- > > Key: CASSANDRA-1752 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1752 > Project: Cassandra > Issue Type: Bug > Components: Core > Reporter: Jonathan Ellis > Assignee: Tyler Hobbs > Fix For: 0.6.9 > > Attachments: 1752-0.6-v2.txt, 1752-0.6-v3.txt, 1752-0.6.txt > > > "We noticed that after a `nodetool repair` was ran, several of our nodes > reported high disk usage; -- even one node hit 100% disk usage. After a > restart of that node, disk usage drop instantly by 80 gigabytes -- well that > was confusing, but we quickly formed the theory that Cassandra must of been > holding open references to deleted file descriptors. > "Later, i found this node as an example, it is using about 8-10 gigabytes > more than it should be -- 118 gigabytes reported by df, yet du reports only > 106 gigabytes in the cassandra directory (nothing else on the mahcine). As > you can see from the lsof listing, it is holding open FDs to files that no > longer exist on the filesystem, and there are no open streams or as far as I > can tell other reasons for the deleted sstable to be open. > "This seems to be related to running a repair, as we haven't seen it in any > other situations before." > A quick check of FileStreamTask shows that the obvious base is covered: > {code} > finally > { > try > { > raf.close(); > } > catch (IOException e) > { > throw new AssertionError(e); > } > } > {code} > So it seems that either the transfer loop is never finishing to get to that > finally block (in which case why isn't it showing up in outbound streams?) or > something else is the problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.