RE: Too many CLOSE_WAIT socket connections

[Mike Burati]

Both Unix and Windows appear to have TCP Time Wait timeouts set too high by
default (about 5 minutes) where the OS leaves closed sockets in the
CLOSE_WAIT state queuing up (which can make it real easy to hit the maxfd
limit on a heavily loaded web server).

I believe the value is something like tcp_time_wait_interval (kernel param)
on Unix systems, and I can't remember what the Windows Registry key is for
the equivalent setting, but it's name is similar.

Set those smaller (eg, 30 or 60 seconds) and you should avoid the problem
you're seeing.


-----Original Message-----
From: Matteo Tamburini [mailto:[EMAIL PROTECTED]
Sent: Friday, October 31, 2003 1:56 PM
To: [EMAIL PROTECTED]
Subject: Too many CLOSE_WAIT socket connections


Hi. 
I've got a thread (axis client) which calls an axis web service several
times (some thousands).
My client code creates the Service object, than creates the Call object
(exactly as shown in the Axis User's Guide). 
Finally the invoke method from the Call class is called.
Everything just works fine for the first thousand times, but then I get an
exception: 
AxisFault
 faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
 faultSubcode: 
 faultString: java.net.SocketException: Too many open files
 faultActor: 
 faultNode: 
 faultDetail: 
        {http://xml.apache.org/axis/}stackTrace: java.net.SocketException:
Too many open files
        at java.net.Socket.createImpl(Socket.java:313)
        at java.net.Socket.<init>(Socket.java:286)
        at java.net.Socket.<init>(Socket.java:119)
        at
org.apache.axis.components.net.DefaultSocketFactory.create(DefaultSocketFact
ory.java:129)
        at
org.apache.axis.transport.http.HTTPSender.getSocket(HTTPSender.java:157)
        at
org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:114)
        at
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:
71)
        at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:150)
        at org.apache.axis.SimpleChain.invoke(SimpleChain.java:120)
        at org.apache.axis.client.AxisClient.invoke(AxisClient.java:180)
        at org.apache.axis.client.Call.invokeEngine(Call.java:2564)
        at org.apache.axis.client.Call.invoke(Call.java:2553)
        at org.apache.axis.client.Call.invoke(Call.java:2248)
        at org.apache.axis.client.Call.invoke(Call.java:2171)
        at org.apache.axis.client.Call.invoke(Call.java:1691)
        at
it.matteo.SyncWorkers.DonorSyncClient.wsSync(DonorSyncClient.java:325)
        at
it.matteo.SyncWorkers.DonorSyncClient.run(DonorSyncClient.java:201)

Using "netstat" I can see in the client lots (about one thousand) open
socket connections in CLOSE_WAIT state. It seems that terminating this ws
client process makes the OS allowed to correctly close these (too many)
opened sockets.

I thought that after each "invoke" the used socket would be closed, but each
socket remains open, and I can't figure out why.
I expected the sockets to be reused, but each time a socket is opened, it
remais opened in CLOSE_WAIT state and a new one is taken, until no more
sockets can be opened.

CLOSE_WAIT is a state without timeout. When a socket is in CLOSE_WAIT state
the OS is waiting the process to release the socket. Isn't it?

The server wich publishes the web service is tomcat4, connected to
apache_1.3 using mod_jk2.

So, I'm trying to figure out what's wrong...
Any help would be appreciated.
Thank you in advance.

Bye,
Matteo.