> From: André Warnier [mailto:a...@ice-sa.com]
> It has been previously established that a socket in a
> long-time-lingering CLOSE-WAIT status, is due to one or the other side
> of a TCP connection not properly closing its side of the
> connection when
> it is done with it.
> I also surmise (without having a definite proof of this), that this is
> essentially "bad", as it ties up some resources that could be
> otherwise freed.
At the very least it'll tie up a kernel data structure for the socket itself.
I don't know modern Linux kernels well enough to know how buffers are
allocated, but I suspect you won't be wasting much memory on buffers as they'll
be allocated on-demand. You're probably talking tens to low hundreds of bytes
for each one of these.
You will also be consuming resources in whichever program is not closing the
sockets correctly.
> So my question is : considering the situation above, is there
> something
> I can do locally to free these lingering CLOSE_WAIT sockets, and under
> which conditions ?
> For example, I see this line :
> tcp6 12 0 ::ffff:127.0.0.1:41764 ::ffff:127.0.0.1:11002
> CLOSE_WAIT 29649/java
> which tells me that there is a local process 29649/java,
> whith a "local"
> socket port 41674 in the CLOSE_WAIT state, related to another socket
> #11002 on the same host.
> On the other hand, I see this line :
> tcp 0 0 127.0.0.1:11002 127.0.0.1:41764
> FIN_WAIT2 -
> which shows a "local" socket on port 11002, related to this
> other local
> socket port #41764, with no process-id/program displayed.
> What does that tell me ?
The process that was on port 11002 closed its end of the socket and sent a FIN.
Process 29649 hasn't closed its end of the socket yet.
> I also know that the process-id 29649 corresponds to a local java
> process, of the daemon variety, multi-threaded. That program
> "talks to"
> another known server program, written in C, of which instances are
> started on an ad-hoc base by inetd, and which "listens" on port 11002
> (in fact it is inetd who does, and it passes this socket on to the
> process it forks, I understand that).
The local Java process may have a resource leak. It appears not to have closed
the socket it was using to communicate with the server.
A possible reason for the lack of a PID on port 11002 is that the socket was
handed across from inetd to the C daemon - not sure about this.
> What it looks like to me in this case, is that at some point
> one of the
> threads of process # 29649 opened a client socket #41674 to the local
> inetd port #11002; that inetd then started the underlying
> server process
> (the C program); that the underlying C program then at some point
> exited; but that process #41674 never closes one of the sides of its
> connection with port #11002.
Agree.
> Can I somehow detect this condition, and "force" the
> offending thread of
> process #29649 to close that socket (or just force this
> thread to exit) ?
Threads are flows of control. Threads do not reference objects other than from
their stack and any thread-local storage - and there are plenty of other places
that can hold onto objects! The socket may well be referenced from an object
on the heap (not the stack) that's ultimately referenced by a static variable
in a class, for example, in which case zapping a thread may well do nothing.
You need to find out what, if anything, is holding onto the socket.
If you have some way of forcing that Java process to collect garbage, you
should do so. It's possible for sockets that haven't been close()d to hang
around, unreferenced but not yet garbage collected. A full GC would collect
any of these, finalizing them as it does and hence closing the socket. If a
full GC doesn't close the socket, some other object is still referencing it.
If a full GC doesn't clear the problem, you may need to go in with some
memory-tracing tool and find out what's holding onto the socket. It's a long,
long time since I had to do this in Java, so I have no idea of the appropriate
tools - my brain's telling me Son of Strike, which is for the .Net CLR and
*definitely* wrong!
Does that help? Or is it clear as mud?
- Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
