You seem to be hitting maxProcessors. If you do, no more threads are created, and connections start getting into SYN_RECV state, simply because Tomcat is not accept()ing them, exactly as you told it to do.
So, either reduce your concurrency level for the tests, or increase maxProcessors.
Maybe there is also sth wrong with the (deprecated) connector code, as suggested by someone else. But even with non-deprecated connectors and recent Tomcats, you will have a very bad user experience if you hit maxProcessors.
If you were using Apache in front of Tomcat (which I know is not your case), thigs get even worse. In that case, you must ensure that Apache's MaxClients is below Tomcat's maxProcessors, and if you have a firewall in the middle, you must also ensure that you are using worker.something.socket_timeout=300 or something like that in workers.properties, so that your firewall does not mess things up. (Assuming mod_jk)
Hope that helps.
Antonio Fiol
Frode E. Moe wrote:
On Mon, Oct 27, 2003 at 13:29:50 +0100, Remy Maucherat wrote:
On Fri, Oct 24, 2003 at 16:01:48 +0200, [EMAIL PROTECTED] wrote:Any reason for using this connector ? It's deprecated.
When everything quiets down, tomcat seems to be unable to service
any http requests. Any attempt to connect to port 10003 with a web
browser eventually causes a "No data in response"-style error message.
At this point it seems the http connector listening on port 10003 has
gone "zombie" on me:
I've figured out what's going wrong. The following exception is printed once on standard out when things fail:
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start(Native Method)
at org.apache.catalina.connector.http.HttpProcessor.threadStart(HttpProcessor.java:1178)
at org.apache.catalina.connector.http.HttpProcessor.start(HttpProcessor.java:1262)
at org.apache.catalina.connector.http.HttpConnector.newProcessor(HttpConnector.java:937)
at org.apache.catalina.connector.http.HttpConnector.createProcessor(HttpConnector.java:868)
at org.apache.catalina.connector.http.HttpConnector.run(HttpConnector.java:1075)
I looked into the source for HttpProcessor briefly; perhaps the
"thread.start()" on line 1178 or the threadStart() call on line 1262 could be surrounded with a try/catch(OutOfMemoryError), and throw a LifecycleException?
BTW, once there's an OOM exception, the VM is in an unpredictable state, and must be restarted.
If it is of any interest, I have recently started experiencing the same problem again, while doing some profiling work on a webapp.
I'm now running Tomcat 4.1.29 on Linux 2.4.24 (i386) with Sun Java 1.4.2_02. I have also changed server.xml to use the newer Coyote connectors:
<Connector className="org.apache.coyote.tomcat4.CoyoteConnector" port="10003" minProcessors="50" maxProcessors="500" enableLookups="true" redirectPort="8443" acceptCount="10" debug="100" connectionTimeout="60000" useURIValidationHack="false" disableUploadTimeout="true" />
When running with the following java options: "-Xrunhprof cpu=samples,thread=y;depth=10,cutoff=0,format=a,file=/tmp/log.txt -Xms64m -Xmsm1024m" and "pounding" the webapp with 1000 HTTP requests simulatenously, I got the following on System.out:
07.jan.2004 09:27:41 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run SEVERE: Caught exception executing [EMAIL PROTECTED], terminating thread java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start(Native Method) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.<init>(ThreadPool.java:630) at org.apache.tomcat.util.threads.ThreadPool.openThreads(ThreadPool.java:505) at org.apache.tomcat.util.threads.ThreadPool.findControlRunnable(ThreadPool.java:324) at org.apache.tomcat.util.threads.ThreadPool.runIt(ThreadPool.java:304) at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:557) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:666) at java.lang.Thread.run(Thread.java:534)
..after which any attempt to connect to the http port (10003 in this
case) results in java holding a half-opened connection in SYN_RECV, as shown by "netstat -apeln | grep 10003":
tcp 0 0 0.0.0.0:10003 0.0.0.0:* LISTEN 1001 47392370 27230/java tcp 0 0 127.0.0.1:10003 127.0.0.1:41269 SYN_RECV 1001 0 -
(and several in CLOSE_WAIT state)
Other threads keep running, so the VM _seems_ stable, except for the fact that it's no longer possible to connect to the HTTP port.
To me, it seems like it should be possible to recover from this error. Any comments?
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
smime.p7s
Description: S/MIME Cryptographic Signature
