[
https://issues.apache.org/jira/browse/LUCENE-5716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14013590#comment-14013590
]
Dawid Weiss commented on LUCENE-5716:
-------------------------------------
I spoke with Robert at Buzzwords about it a bit. It'd be interesting to capture
file handle leaks from tests (unclosed sockets, unclosed files). There are
essentially no java built-in mechanisms to do it. One could use the operating
system's tools such as lsof, but this would measure ALL file handles, including
handles of open jar files, etc. It's also an interesting puzzle to see if one
can do it from Java in a portable (?) way.
There are a few options.
1) Override bootclasspath for a particular release of the JVM and modify system
classes to allow resource tracking. A similar "solution" kind of exists already
in FileInputStream:
{code}
public int read() throws IOException {
Object traceContext = IoTrace.fileReadBegin(path);
int b = 0;
try {
b = read0();
} finally {
IoTrace.fileReadEnd(traceContext, b == -1 ? 0 : 1);
}
return b;
}
{code}
the obscure IoTrace class unfortunately only tracks file writes, not its
open/close status. A similar hack could be applied to capture constructors and
close calls though.
2) Use bytecode transformation and capture all places where interesting objects
are created/ closed. AspectJ is closest to my heart since it allows fast
prototyping and is essentially just a bytecode manipulation framework. I've
written a simple aspect tracing FileInputStream usage, here's the code.
https://github.com/dweiss/fhandle-tracing
The aspect itself:
https://github.com/dweiss/fhandle-tracing/blob/master/src/main/aspect/com/carrotsearch/aspects/FileInputStreamTracker.aj
There is a JUnit test in there and if you run mvn test, you can see that it's
actually working quite nice. Not everything can be easily addressed (for
example, it's difficult to advise close in classes that inherit from FIS but
don't override this method), but alternative solutions to the problem also
exist (capture all close calls, capture all constructors of
FileInputStream+.new). Doable and should work for 99% of the use cases I think.
3) use jvmti instrumentation agents to essentially provide the same
instrumentation as above. I don't think it is functionally any different from
(2) and I like aspect's syntax for fast hacking. The only exception might be
(didn't check) if we could redefine and reload the definitions of core Java
classes (FileInputStream, for example) so that we can instrument constructors
and methods of these base classes directly. This would be quite nice because
then we'd be able to make it portable across all JVMs and it should work for
all code fragments, including standard library itself.
I'll keep experimenting with (2) for now as it's the low-hanging fruit (already
works), but if somebody wants to inspect (3) it'd be quite interesting.
> Track file handle leaks (FileDescriptor, NIO Path SPI and Socket mostly).
> -------------------------------------------------------------------------
>
> Key: LUCENE-5716
> URL: https://issues.apache.org/jira/browse/LUCENE-5716
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
