On 10.07.2009 07:31, Aneesha Pant wrote: > We have configured 2 tomcats with 1 Apache on the same box, but during > performance testing, we are continuously getting these errors in the > mod_jk log > Please help
Since 111 on most OSes is "connection refused", I am guessing you got OutOfmemoryError on your Tomcat and as a result it doesn't listen any more for incoming requests. Another possibility is that your thread pool is exhausted. So check your connector configuration for the number of threads and compare to the expected concurrency in your load testing scenario. Java thread dumps will show you, what all those threads are doing (e.g. waiting for). Regards, Rainer > [Thu Jul 09 18:50:53 2009] [13862:50960] [info] > ajp_send_request::jk_ajp_common.c (1241): (worker2) all endpoints are > disconnected > [Thu Jul 09 18:50:53 2009] [13796:50960] [info] > ajp_connect_to_endpoint::jk_ajp_common.c (876): Failed opening socket to > (130.11.1.165:8009) (errno=111) > [Thu Jul 09 18:50:53 2009] [13842:50960] [info] > ajp_connect_to_endpoint::jk_ajp_common.c (876): Failed opening socket to > (130.11.1.165:8009) (errno=111) > [Thu Jul 09 18:50:53 2009] [13807:50960] [info] service::jk_lb_worker.c > (1098): service failed, worker worker1 is in error state > [Thu Jul 09 18:50:53 2009] [14122:50960] [info] service::jk_lb_worker.c > (1160): All tomcat instances are busy or in error state > [Thu Jul 09 18:50:53 2009] [14032:50960] [info] service::jk_lb_worker.c > (1160): All tomcat instances are busy or in error state > [Thu Jul 09 18:50:53 2009] [13500:50960] [info] > ajp_connect_to_endpoint::jk_ajp_common.c (876): Failed opening socket to > (130.11.1.165:8009) (errno=111) > [Thu Jul 09 18:50:53 2009] [13922:50960] [info] > ajp_connect_to_endpoint::jk_ajp_common.c (876): Failed opening socket to > (130.11.1.165:8009) (errno=111) > [Thu Jul 09 18:50:53 2009] [13828:50960] [info] jk_handler::mod_jk.c > (2254): Service error=0 for worker=loadbalancer > [Thu Jul 09 18:50:53 2009] [13908:50960] [info] > ajp_connect_to_endpoint::jk_ajp_common.c (876): Failed opening socket to > (130.11.1.165:8009) (errno=111) > [Thu Jul 09 18:50:53 2009] [13952:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [14033:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [13728:50960] [info] > ajp_connect_to_endpoint::jk_ajp_common.c (876): Failed opening socket to > (130.11.1.165:8009) (errno=111) > [Thu Jul 09 18:50:53 2009] [14098:50960] [info] service::jk_lb_worker.c > (1160): All tomcat instances are busy or in error state > [Thu Jul 09 18:50:53 2009] [14082:50960] [info] service::jk_lb_worker.c > (1098): service failed, worker worker1 is in error state > [Thu Jul 09 18:50:53 2009] [14067:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [14069:50960] [info] > ajp_service::jk_ajp_common.c (1941): (worker1) sending request to tomcat > failed, recoverable operation attempt=2 > [Thu Jul 09 18:50:53 2009] [13973:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [13902:50960] [info] > ajp_send_request::jk_ajp_common.c (1241): (worker2) all endpoints are > disconnected > [Thu Jul 09 18:50:53 2009] [14050:50960] [info] > ajp_service::jk_ajp_common.c (1941): (worker1) sending request to tomcat > failed, recoverable operation attempt=2 > [Thu Jul 09 18:50:53 2009] [13943:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [13909:50960] [error] > ajp_service::jk_ajp_common.c (1953): (worker1) Connecting to tomcat > failed. Tomcat is probably not started or is listening on the wrong port > [Thu Jul 09 18:50:53 2009] [14244:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [14171:50960] [info] > ajp_service::jk_ajp_common.c (1941): (worker1) sending request to tomcat > failed, recoverable operation attempt=2 > [Thu Jul 09 18:50:53 2009] [14114:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [13849:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [13970:50960] [info] > ajp_send_request::jk_ajp_common.c (1241): (worker2) all endpoints are > disconnected > [Thu Jul 09 18:50:53 2009] [14168:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [14014:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [13915:50960] [info] service::jk_lb_worker.c > (1160): All tomcat instances are busy or in error state > [Thu Jul 09 18:50:53 2009] [14062:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [14121:50960] [info] > ajp_service::jk_ajp_common.c (1941): (worker1) sending request to tomcat > failed, recoverable operation attempt=2 > [Thu Jul 09 18:50:53 2009] [14047:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [14039:50960] [info] > ajp_send_request::jk_ajp_common.c (1241): (worker2) all endpoints are > disconnected > [Thu Jul 09 18:50:53 2009] [14042:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [13726:50960] [info] > ajp_send_request::jk_ajp_common.c (1241): (worker2) all endpoints are > disconnected > [Thu Jul 09 18:50:53 2009] [13713:50960] [info] > ajp_send_request::jk_ajp_common.c (1241): (worker2) all endpoints are > disconnected > [Thu Jul 09 18:50:53 2009] [14101:50960] [info] > ajp_send_request::jk_ajp_common.c (1244): (worker2) increase the backend > idle connection timeout or the connection_pool_minsize > [Thu Jul 09 18:50:53 2009] [14102:50960] [info] > ajp_send_request::jk_ajp_common.c (1244): (worker2) increase the backend > idle connection timeout or the connection_pool_minsize > [Thu Jul 09 18:50:53 2009] [13855:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [14043:50960] [info] > ajp_send_request::jk_ajp_common.c (1244): (worker2) increase the backend > idle connection timeout or the connection_pool_minsize > [Thu Jul 09 18:50:53 2009] [13887:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker2) error connecting to > the backend server (errno=111) > [Thu Jul 09 18:50:53 2009] [13841:50960] [info] > ajp_send_request::jk_ajp_common.c (1273): (worker1) error connecting to > the backend server (errno=111) > > > > > -----Original Message----- > From: Aneesha Pant > Sent: 15 June 2009 5:27 PM > To: Tomcat Users List > Subject: RE: Problem with apache tomcat while running load test > > There is nothing in any of the log files. The application we are using > works on Tomcat 5.0.16, have not yet tested on latest tomcat. Yes Mod_jk > the version is mod_jk-1.2.25-httpd-2.0.59.so > mod_jk-1.2.23-apache-2.2.x-linux-i68 > > -----Original Message----- > From: Peter Crowther [mailto:peter.crowt...@melandra.com] > Sent: 15 June 2009 4:50 PM > To: 'Tomcat Users List' > Subject: RE: Problem with apache tomcat while running load test > > Any messages in any of the (many) log files? > Connector version? I assume mod_jk, but which? > Out of interest, why are you using an unsupported Tomcat version? > > - Peter > > ________________________________ > From: Aneesha Pant [mailto:aneesha.p...@edelcap.com] > Sent: 15 June 2009 12:09 > To: users@tomcat.apache.org > Subject: Problem with apache tomcat while running load test > > We are trying to run a load test using Borland Silk route on a > application running on Tomcat 5.0.16 on Apache 2.2.3.11. we have > configured 4 tomcats with 1 Apache using worker.properties file. when we > try to run 1000 concurrent users doing transactions on the website the > site stops responding after about 10 min. on the server side we have > observed that the cpu utilisation shoots upto 100% for server process > (Apache). the user cpu utilisation is not much. Then the site hangs and > the cpu become 100% idle. this is a very strange behavior, we observed > that the max httpd connects was going upto 2050, so we increased the > maxclients to 3072 (earlier tried with 1024 & 2048). We have tried to > increase the RAM of the TOM cat from 1GB to 2GB each, but no significant > difference in the behavior. > > Server Dell 2950 > OS Red HAT- 5.2 > > > Aneesha Pant > AVP - BSG & Tech > Dir: +91 (22) 4079 5085 > <http://www.edelcap.com/> > <http://www.edelcap.com/> > <http://www.edelcap.com/> > <http://www.edelcap.com/> > [cid:image8b3bb3.JPG@e1c938af.c4614c12] > Edelweiss Capital Limited > 14th Floor, Express Tower, > Nariamn Point,Mumbai 400 021 > Board:+91 (22) 2286 4400 Ext. 5085 > www.edelcap.com<http://www.edelcap.com> > ------------------------------------------------------------------------ > -----------------------------------------------------------------------[ > cid:imageb1a1a6.gif@b1d8bcc8.1bb44f41]<https://https//www.edelweissmf.co > m/Products/Enhancer.aspx> --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
