RE: Tomcat 4.1.30 ThreadPool.adjustLimits
Yoav, Thanks for the + feedback. Apologies for response delay - I'm digest only and also had wicked work backlog past couple weeks. Will submit a Bugzillia item as requested. Additional questions seem relevant prior to that (in the "be careful what you ask for..." department ;-). With the outlined patch (MAX_THREAD_MIN=2) in place... (1) I config minProcessors="1", maxProcessors="2" and I appear to get servicing of a single request. Increment maxProcessers and I get concurrent servicing of 2 requests, and so on. What exactly is the additional thread used for (shutdown listener? thread pool manager? something else?). From an audit trail perspective, this would seem to be valuable information to retain in the context of any change. (2) When I stack up requests to a given server - throw a lot of concurrent requests at it - I see one of the following entries in the (catalina.out) log file: [ERROR] ThreadPool - -All threads (2) are currently busy, waiting. Increase maxThreads (2) or check the servlet status SEVERE: All threads (2) are currently busy, waiting. Increase maxThreads (2) or check the servlet status The above generated (indirectly) out of ThreadPool.runIt() when it has to block on a "run thread" request. This anything to worry about? I do have acceptCount="6" set for what that's worth... Regards, -Jim Layer- -Original Message- From: Shapira, Yoav [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 04, 2004 8:52 AM To: Tomcat Users List Subject: RE: Tomcat 4.1.30 ThreadPool.adjustLimits Hi, That was an excellent message. Your approach is flawless. You can keep using your hack, which isn't that crude at all. You can keep using the 5.x Connector code by and large with 4.1.30 (though not earlier versions of Tomcat 4.1), it should be just fine. And I tend to agree than 10 as MAX_THREADS_MIN is too high: something like 2 or 3 seems reasonable to me. Please open a Bugzilla item asking for this to be modified, and I will address it. Thanks, Yoav Shapira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Tomcat 4.1.30 ThreadPool.adjustLimits
I have several questions that apparently center around the org.apache.tomcat.util.threads.ThreadPool class contained in the tomcat-util.jar file included in the Tomcat V4.1.30 distribution. Background: Using V4.1.30 in a standalone, relatively minimal configuration with the Coyote HTTP/1.1 Connector. Deployed on several hundred (ultimately several thousand) UNIX servers of various flavors. Least common denominator server types supporting only JVM 1.2.2 limit us to 4.x Tomcat. Tomcat used solely for foreign system access to server data via specialized servlets in a confined/controlled environment (not internet exposed). Resource considerations require limiting Tomcat to a tiny fraction of it's concurrent connection capacity (primary function of servers is to host a vast load of multi-user business management applications). What I'm after is limiting to a max of 4 concurrent connections on the "stock" configuration. Initially though I could simply set maxProcessors="5" and minProcessors="1" (earlier experimentation suggested actual concurrent connections was one less than maxProcessors value). I unfortunately misinterpreted the warning message generated during startup, "WARNING: maxThreads setting (5) too low, set to 10", to be a suggestion. I now understand this is a statement that Tomcat (ThreadPool.adjustLimits() method logic, in particular) actually reset the maxProcessors value to 10. Went looking for ThreadPool in V4.1.30 source (based on warning message origin tags) without success, then remembered something about TC V4.1.30 using TC V5.x Coyote connector. Then located what I believe to be the operative logic in jakarta-tomcat-connectors portion of Tomcat V5.0.19 source (version I happened to have handy). Sure enough, method adjustLimits() forces maxThreads value (apparently set by digester logic based on TC4.x.x maxProcessors) to MAX_THREADS_MIN (hardwired to 10 in source) if value supplied in server.xml is less than that. I knocked off a copy of org.apache.tomcat.util.threads.ThreadPool, changed MAX_THREADS_MIN to 2, compiled, and surgically deployed resulting classes under server/classes (effectively overriding the ThreadPool class in server/lib/tomcat-util.jar) on a test box. Appears to be behaving reasonably, now limiting concurrent connections to that specified in server.xml file. I know - crude hack - but I didn't have much recourse without fielding something to front-end Tomcat. My questions: (1) Is there some other reason for this essentially arbitrary (upward) adjustment of the maxProcessors/maxThreads value - beyond the rational provided in the source commentary "... fix provides reasonable settings for a single CPU with medium load."? (2) Is the ThreadPool source from Tomcat V5.0.19 the appropriate "version" to use as I have outlined above? (It appeared to be the closest date match to the Tomcat V4.1.30 release, but I couldn't find any specific references to where the tomcat-util.jar file in V4.1.30 originated.) While my little "hack" appears to be working - I'm still testing the crap out of it - would be nice to know if there is something I may have missed, so any input from those intimate with V4.1.30 & V5.x logic would be most appreciated. Regards, Jim Layer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: TC V4.1.29 Coyote Connector / minProcessors maxProcessors / Confusion
The second observation: two requests to the same servlet always handled serially. Tomcat normally creates one instance of a servlet for every declaration in web.xml. The exception is SingleThreadModel (deprecated, don't use this) servlets. There's no provision in tomcat to create more instances as the load on the server increases (maybe there is such a provision and I've missed it?). Instead, the processing threads should call the servlet methods in parallel. I'm not sure why you're not seeing this behavior, perhaps someone else could comment, as I have to run to a meeting ;) Yoav Shapira Yes - All you mentioned per Servlet spec expected behavior - why I was surprised at what I was seeing. I have additional qualification data but believe I should post under a new thread / subject appropriate. -Jim Layer- egg-on-face update: Wasn't controlling my variables closely enough. Weird non-concurrent behavior I observed was confined to brower-based tests. Screwed up on the earlier multi-instance client test tool exercise - works as expected when I'm explicitly controlling client behavior (headers, etc.). Thanks (Yoav) again for your help, Regards, -Jim Layer- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: TC V4.1.29 Coyote Connector / minProcessors maxProcessors / Confusion
Shapira, Yoav writes: I think your testing, servlets, configuration is all fine. The maxProcessors-1 observation is something I've noticed in the past, but as you say I don't think many people care because they deal with ~75 maxProcessors (the default value). Thanks - glad to know I'm not off in the weeds on that... The second observation: two requests to the same servlet always handled serially. Tomcat normally creates one instance of a servlet for every declaration in web.xml. The exception is SingleThreadModel (deprecated, don't use this) servlets. There's no provision in tomcat to create more instances as the load on the server increases (maybe there is such a provision and I've missed it?). Instead, the processing threads should call the servlet methods in parallel. I'm not sure why you're not seeing this behavior, perhaps someone else could comment, as I have to run to a meeting ;) Yoav Shapira Yes - All you mentioned per Servlet spec expected behavior - why I was surprised at what I was seeing. I have additional qualification data but believe I should post under a new thread / subject appropriate. -Jim Layer- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: TC V4.1.29 Coyote Connector / minProcessors maxProcessors / Confusion
Yoav,
Thanks for the quick reply...
Create a servlet that takes a long time to process, so you can easily
observe the connections. Then make sure you make two concurrent
requests to that servlet, and that the servlet's HTML output doesn't
contain images, CSS references, or any other entities which would cause
a separate HTTP connection to be initiated by the browser for that one
page. This type of page is the most common cause for perceived problems
with concurrent request processing.
I wasn't working with any images, etc. - reduced quickly to KISS level
configs & testing as a consequence of more than a couple of decades
worth of battle scars ;-)
Simple servlet, doGet implementation, simple HTML out, 30 second sleep.
Built several variants, different only in class name. Manual deployment,
no WAR file, a set of servlet classes in several different contexts.
Servlets HelloWait, HelloWait2, HelloWait3, etc. Contexts cctest1,
cctest2, etc.
Used a multi-tabbed browser (Mozilla) to test, though behavior the same
with separate browsers on multiple clients (enlisted the aid of several
of my cohorts). Also used lovely little multi-threaded Java HTTP test
client gadget to slam in requests from multiple clients concurrently.
First observation:
Behavior in general indicates number of concurrent requests serviced
equal to maxProcessors - 1 (not maxProcessers-minProcessors as I
suggested in earlier email).
Second observation:
This one really threw me. Independent of maxProcessors value, multiple
concurrent requests to the same servelet in the same context appear
to be handled serially - one at a time. Multiple concurrent requests,
each to a different servlet in the same context or a servlet in
another context appear to be handled concurrently, subject to my
first observation.
I was ready to buy the maxProcessers-1 thing as a simple implementation
oversight - not surprising since most use Tomcat for much higher volume
stuff. Second observation suggests something really strange with 4.1.29
(most of our earlier testing experimentation was with 4.1.27) or I have
made a really toady configuration error.
I have attached 2 representative servlet source samples - pretty
trivial, server.xml, default web.xml (renamed as web_default.xml) and
example context web.xml for servlets. Would have zipped but found
that not permitted for list mail. Maybe something in these I'm doing
wrong?
Tomcat running (for test execution) on an HP dual proc (Intel PIII 1.133
GHz) platform running Linux - RedHat 7.1 - little or no extraneous load
during the testing.
Any insight might have would be well appreciated.
Regards,
-Jim-
/*
* HelloWait.java
*/
import java.io.*;
import java.net.*;
import java.util.Date;
import java.text.DateFormat;
import javax.servlet.*;
import javax.servlet.http.*;
/**
* Servlet used to test concurrent connections. Waits for 30 seconds
* before returning from doGet method.
*
* @author LayerJB
*/
public class HelloWait extends HttpServlet
{
/**
* Process the HTTP GET request.
*
* @param req An HttpServletRequest object that contains the
* request the client has made of the servlet.
* @param res An HttpServletResponse object that contains the
* response the servlet sends to the object.
*
* @exception ServletExceptionThrown if the request for the
*
GET could not be handled.
* @exception IOException Thrown if an input or
output
*
error is detected when the servlet
*
handles the request.
*/
public void doGet (HttpServletRequest req, HttpServletResponse res)
throws ServletException, IOException
{
DateFormat dateFormat = DateFormat.getDateTimeInstance();
String sStart = dateFormat.format(new Date());
res.setContentType("text/html");
ServletOutputStream out = res.getOutputStream();
out.println("HelloWait");
out.println("doGet start: " + sStart +
"We're waiting 30 seconds...");
out.flush();
try
{
Thread.sleep(3);
}
catch (InterruptedException ignore) {};
out.println("Done Waiting: " +
dateFormat.format(new Date()) +
"");
out.close();
}
}
/*
* HelloWait2.java
*/
import java.io.*;
import java.net.*;
import java.util.Date;
import java.text.DateFormat;
import javax.servlet.*;
import javax.servlet.http.*;
/**
* Servlet used
TC V4.1.29 Coyote Connector / minProcessors maxProcessors / Confusion
I'm a little confused regarding TC V4.1.29 Coyote HTTP/1.1 Connector attributes minProcessors and maxProcessors. Server config docs state minProcessors == number of connection threads to start initially and maxProcessors == the maximum number of connection threads that the server will create. This suggested to me that with minProcessors="1" and maxProcessors="2", the server would allow 2 concurrent connections. I tried the above values and found the server would handle 1 connection. Increasing to maxProcessors="3", 2 concurrent connections were available, and so on. Seems maximum_concurrent_connections=(maxProcessors - minProcessors), a slightly different relationship than I gathered from the docs. I was hoping someone could confirm this as normal, or is there maybe a bug that I should perhaps watch for correction of later. Additional minutia (to perhaps avoid adjacent questions ;-): Deploying Tomcat Standalone on a variety of Unix platforms. Likely will update to V4.1.30 shortly. Avoiding V5.0.x because a significant number of the (older) target systems are stuck on JVM 1.2.2 Using TC for moving data / messages, servlets only, no web pages, JSP, etc. Limiting concurrent connection count to relatively low numbers due to resource constraints on many of these platforms. Any feedback would be greatly appreciated. Regards, jblayer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
