urgent help needed

2007-05-17 Thread bhavik shah

Hi,
We have tomcat4.1 installed on HPUX Box.
I stopped the tomcat server
I deploy one application on the tomcat server and start it

when I tried to navigate to http://hostname:portno it gives me the following
error:
HTTP Status 404 - /
--

*type* Status report

*message* */*

*description* *The requested resource (/) is not available.*

**

But when i tried to navigate to my application I can navigate it.

i.e. http://hostname:portno/virtual_dir

after that I saw the following error in localhost_log file:

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path /admin

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path /webdav

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path /examples

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path /tomcat-docs

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path /manager

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path

2007-05-17 10:38:14 WebappLoader[]: Deploying class repositories to work
directory /bdhp4238/opt/docu/tst/tomcat4/work/Standalone/localhost/_

2007-05-17 10:38:14 WebappLoader[]: Deploy JAR
/WEB-INF/lib/dmvirtuallink.jar to
/opt/docu/tst/tomcat4/webapps/ROOT/WEB-INF/lib/dmvirtuallink.jar

2007-05-17 10:38:14 ContextConfig[] Exception processing JAR at resource
path /WEB-INF/lib/dmvirtuallink.jar

javax.servlet.ServletException: Exception processing JAR at resource path
/WEB-INF/lib/dmvirtuallink.jar

at org.apache.catalina.startup.ContextConfig.tldScanJar(Unknown Source)

at org.apache.catalina.startup.ContextConfig.tldScan(Unknown Source)

at org.apache.catalina.startup.ContextConfig.start(Unknown Source)

at org.apache.catalina.startup.ContextConfig.lifecycleEvent(Unknown Source)

at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(Unknown
Source)

at org.apache.catalina.core.StandardContext.start(Unknown Source)

at org.apache.catalina.core.ContainerBase.start(Unknown Source)

at org.apache.catalina.core.StandardHost.start(Unknown Source)

at org.apache.catalina.core.ContainerBase.start(Unknown Source)

at org.apache.catalina.core.StandardEngine.start(Unknown Source)

at org.apache.catalina.core.StandardService.start(Unknown Source)

at org.apache.catalina.core.StandardServer.start(Unknown Source)

at org.apache.catalina.startup.Catalina.start(Unknown Source)

at org.apache.catalina.startup.Catalina.execute(Unknown Source)

at org.apache.catalina.startup.Catalina.process(Unknown Source)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java
:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:324)

at org.apache.catalina.startup.Bootstrap.main(Unknown Source)

- Root Cause -

java.io.FileNotFoundException

at org.apache.naming.resources.DirContextURLConnection.getInputStream(Unknown
Source)

at sun.net.www.protocol.jar.URLJarFile.retrieve(URLJarFile.java:161)

at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:42)

at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:68)

at sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java
:85)

at sun.net.www.protocol.jar.JarURLConnection.getJarFile(
JarURLConnection.java:69)

at org.apache.catalina.startup.ContextConfig.tldScanJar(Unknown Source)

at org.apache.catalina.startup.ContextConfig.tldScan(Unknown Source)

at org.apache.catalina.startup.ContextConfig.start(Unknown Source)

at org.apache.catalina.startup.ContextConfig.lifecycleEvent(Unknown Source)

at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(Unknown
Source)

at org.apache.catalina.core.StandardContext.start(Unknown Source)

at org.apache.catalina.core.ContainerBase.start(Unknown Source)

at org.apache.catalina.core.StandardHost.start(Unknown Source)

at org.apache.catalina.core.ContainerBase.start(Unknown Source)

at org.apache.catalina.core.StandardEngine.start(Unknown Source)

at org.apache.catalina.core.StandardService.start(Unknown Source)

at org.apache.catalina.core.StandardServer.start(Unknown Source)

at org.apache.catalina.startup.Catalina.start(Unknown Source)

at org.apache.catalina.startup.Catalina.execute(Unknown Source)

at org.apache.catalina.startup.Catalina.process(Unknown Source)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java
:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:324)

at org.apache.catalina.startup.Bootstrap.main(Unknown Source)

2007-05-17 10:38:14 ContextConfig[]: Marking this application unavailable
due to previous error(s)

2007-05-17

Re: urgent help needed

2007-05-17 Thread Filip Hanik - Dev Lists

there is something wrong in this sucker
/opt/docu/tst/tomcat4/webapps/ROOT/WEB-INF/lib/dmvirtuallink.jar

are you using symlinks?
if so, 
http://tomcat.apache.org/tomcat-4.1-doc/config/context.html#allowLinking


you end up getting a FileNotFoundException, turning on debug logging, 
might yield more info


Filip

bhavik shah wrote:

Hi,
We have tomcat4.1 installed on HPUX Box.
I stopped the tomcat server
I deploy one application on the tomcat server and start it

when I tried to navigate to http://hostname:portno it gives me the 
following

error:
HTTP Status 404 - /
--

*type* Status report

*message* */*

*description* *The requested resource (/) is not available.*

**

But when i tried to navigate to my application I can navigate it.

i.e. http://hostname:portno/virtual_dir

after that I saw the following error in localhost_log file:

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path /admin

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path /webdav

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path /examples

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path /tomcat-docs

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path /manager

2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
context path

2007-05-17 10:38:14 WebappLoader[]: Deploying class repositories to work
directory /bdhp4238/opt/docu/tst/tomcat4/work/Standalone/localhost/_

2007-05-17 10:38:14 WebappLoader[]: Deploy JAR
/WEB-INF/lib/dmvirtuallink.jar to
/opt/docu/tst/tomcat4/webapps/ROOT/WEB-INF/lib/dmvirtuallink.jar

2007-05-17 10:38:14 ContextConfig[] Exception processing JAR at resource
path /WEB-INF/lib/dmvirtuallink.jar

javax.servlet.ServletException: Exception processing JAR at resource path
/WEB-INF/lib/dmvirtuallink.jar

at org.apache.catalina.startup.ContextConfig.tldScanJar(Unknown Source)

at org.apache.catalina.startup.ContextConfig.tldScan(Unknown Source)

at org.apache.catalina.startup.ContextConfig.start(Unknown Source)

at org.apache.catalina.startup.ContextConfig.lifecycleEvent(Unknown 
Source)


at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(Unknown
Source)

at org.apache.catalina.core.StandardContext.start(Unknown Source)

at org.apache.catalina.core.ContainerBase.start(Unknown Source)

at org.apache.catalina.core.StandardHost.start(Unknown Source)

at org.apache.catalina.core.ContainerBase.start(Unknown Source)

at org.apache.catalina.core.StandardEngine.start(Unknown Source)

at org.apache.catalina.core.StandardService.start(Unknown Source)

at org.apache.catalina.core.StandardServer.start(Unknown Source)

at org.apache.catalina.startup.Catalina.start(Unknown Source)

at org.apache.catalina.startup.Catalina.execute(Unknown Source)

at org.apache.catalina.startup.Catalina.process(Unknown Source)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java

:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:324)

at org.apache.catalina.startup.Bootstrap.main(Unknown Source)

- Root Cause -

java.io.FileNotFoundException

at 
org.apache.naming.resources.DirContextURLConnection.getInputStream(Unknown 


Source)

at sun.net.www.protocol.jar.URLJarFile.retrieve(URLJarFile.java:161)

at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:42)

at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:68)

at 
sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java

:85)

at sun.net.www.protocol.jar.JarURLConnection.getJarFile(
JarURLConnection.java:69)

at org.apache.catalina.startup.ContextConfig.tldScanJar(Unknown Source)

at org.apache.catalina.startup.ContextConfig.tldScan(Unknown Source)

at org.apache.catalina.startup.ContextConfig.start(Unknown Source)

at org.apache.catalina.startup.ContextConfig.lifecycleEvent(Unknown 
Source)


at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(Unknown
Source)

at org.apache.catalina.core.StandardContext.start(Unknown Source)

at org.apache.catalina.core.ContainerBase.start(Unknown Source)

at org.apache.catalina.core.StandardHost.start(Unknown Source)

at org.apache.catalina.core.ContainerBase.start(Unknown Source)

at org.apache.catalina.core.StandardEngine.start(Unknown Source)

at org.apache.catalina.core.StandardService.start(Unknown Source)

at org.apache.catalina.core.StandardServer.start(Unknown Source)

at org.apache.catalina.startup.Catalina.start(Unknown Source)

at org.apache.catalina.startup.Catalina.execute(Unknown Source)

at org.apache.catalina.startup.Catalina.process(Unknown Source)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at 
sun.reflect.NativeMethodAccessorImpl.invoke(Nat

Re: urgent help needed

2007-05-17 Thread bhavik shah

I remove the jar file and it worked fine

Regards
bhavik


On 5/17/07, Filip Hanik - Dev Lists <[EMAIL PROTECTED]> wrote:


there is something wrong in this sucker
/opt/docu/tst/tomcat4/webapps/ROOT/WEB-INF/lib/dmvirtuallink.jar

are you using symlinks?
if so,
http://tomcat.apache.org/tomcat-4.1-doc/config/context.html#allowLinking

you end up getting a FileNotFoundException, turning on debug logging,
might yield more info

Filip

bhavik shah wrote:
> Hi,
> We have tomcat4.1 installed on HPUX Box.
> I stopped the tomcat server
> I deploy one application on the tomcat server and start it
>
> when I tried to navigate to http://hostname:portno it gives me the
> following
> error:
> HTTP Status 404 - /
> --
>
> *type* Status report
>
> *message* */*
>
> *description* *The requested resource (/) is not available.*
>
> **
>
> But when i tried to navigate to my application I can navigate it.
>
> i.e. http://hostname:portno/virtual_dir
>
> after that I saw the following error in localhost_log file:
>
> 2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
> context path /admin
>
> 2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
> context path /webdav
>
> 2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
> context path /examples
>
> 2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
> context path /tomcat-docs
>
> 2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
> context path /manager
>
> 2007-05-17 09:08:52 StandardHost[localhost]: Removing web application at
> context path
>
> 2007-05-17 10:38:14 WebappLoader[]: Deploying class repositories to work
> directory /bdhp4238/opt/docu/tst/tomcat4/work/Standalone/localhost/_
>
> 2007-05-17 10:38:14 WebappLoader[]: Deploy JAR
> /WEB-INF/lib/dmvirtuallink.jar to
> /opt/docu/tst/tomcat4/webapps/ROOT/WEB-INF/lib/dmvirtuallink.jar
>
> 2007-05-17 10:38:14 ContextConfig[] Exception processing JAR at resource
> path /WEB-INF/lib/dmvirtuallink.jar
>
> javax.servlet.ServletException: Exception processing JAR at resource
path
> /WEB-INF/lib/dmvirtuallink.jar
>
> at org.apache.catalina.startup.ContextConfig.tldScanJar(Unknown Source)
>
> at org.apache.catalina.startup.ContextConfig.tldScan(Unknown Source)
>
> at org.apache.catalina.startup.ContextConfig.start(Unknown Source)
>
> at org.apache.catalina.startup.ContextConfig.lifecycleEvent(Unknown
> Source)
>
> at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(Unknown
> Source)
>
> at org.apache.catalina.core.StandardContext.start(Unknown Source)
>
> at org.apache.catalina.core.ContainerBase.start(Unknown Source)
>
> at org.apache.catalina.core.StandardHost.start(Unknown Source)
>
> at org.apache.catalina.core.ContainerBase.start(Unknown Source)
>
> at org.apache.catalina.core.StandardEngine.start(Unknown Source)
>
> at org.apache.catalina.core.StandardService.start(Unknown Source)
>
> at org.apache.catalina.core.StandardServer.start(Unknown Source)
>
> at org.apache.catalina.startup.Catalina.start(Unknown Source)
>
> at org.apache.catalina.startup.Catalina.execute(Unknown Source)
>
> at org.apache.catalina.startup.Catalina.process(Unknown Source)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java
> :39)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:324)
>
> at org.apache.catalina.startup.Bootstrap.main(Unknown Source)
>
> - Root Cause -
>
> java.io.FileNotFoundException
>
> at
> org.apache.naming.resources.DirContextURLConnection.getInputStream
(Unknown
>
> Source)
>
> at sun.net.www.protocol.jar.URLJarFile.retrieve(URLJarFile.java:161)
>
> at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:42)
>
> at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:68)
>
> at
> sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java
> :85)
>
> at sun.net.www.protocol.jar.JarURLConnection.getJarFile(
> JarURLConnection.java:69)
>
> at org.apache.catalina.startup.ContextConfig.tldScanJar(Unknown Source)
>
> at org.apache.catalina.startup.ContextConfig.tldScan(Unknown Source)
>
> at org.apache.catalina.startup.ContextConfig.start(Unknown Source)
>
> at org.apache.catalina.startup.ContextConfig.lifecycleEvent(Unknown
> Source)
>
> at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(Unknown
> Source)
>
> at org.apache.catalina.core.StandardContext.start(Unknown Source)
>
> at org.apache.catalina.core.ContainerBase.start(Unknown Source)
>
> at org.apache.catalina.core.StandardHost.start(Unknown Source)
>
> at org.apache.catalina.core.ContainerBase.start(Unknown Source)
>
> at org.apache.catalina.core.StandardEngine.start(Unknown Source)
>
> at org.apache.catalina.core.StandardService.start(Unknown Source)
>
> at org.apache.catalina.core.S

URGENT HELP NEEDED: mod_jk loadbalancing issues

2006-07-08 Thread Edmon Begoli

We are running into some serious load and scalability issues related to the
mod_jk.

I read some threads that indicate that these issues may have been fixed in
post 1.2.15 version.

Available downoalds are only for 1.2.15.

Some websites are indicating that the latest one should be 1.2.16.

Can someone please tell me what is the latest version of mod_jk that we
should be using.


We are running 1.2.15 complied on RedHat ES 4.2.x. with Apache 2.0.52

--
Thank you,
Edmon


Re: URGENT HELP NEEDED: mod_jk loadbalancing issues

2006-07-08 Thread Rainer Jung
Hello,

if you would give detailed information about your problems, we could
advise you, whether it is known, and whether it has been fixed in some
version.

The latest release is 1.2.15, we tried to produce a 1.2.16 release, that
but it has now been withdrawn because of a regression bug. Another
release (1.2.17) will be tried next week.

Rainer

Edmon Begoli schrieb:
> We are running into some serious load and scalability issues related to the
> mod_jk.
> 
> I read some threads that indicate that these issues may have been fixed in
> post 1.2.15 version.
> 
> Available downoalds are only for 1.2.15.
> 
> Some websites are indicating that the latest one should be 1.2.16.
> 
> Can someone please tell me what is the latest version of mod_jk that we
> should be using.
> 
> 
> We are running 1.2.15 complied on RedHat ES 4.2.x. with Apache 2.0.52
> 

-
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: URGENT HELP NEEDED: mod_jk loadbalancing issues

2006-07-09 Thread Edmon Begoli

Hi,

First of all - thanks Rainer for willing to look into this issue.

Environment description:

Web Server: Apache 2.0.52
mod_jk: 1.2.15
OS: RedHat ES 4.2
App. Server: Tomcat 5.5.15

Cluster topology: 4 webservers with mod_jk doing sticky loadbalancing into
10 tomcats

MOD_JK Configuration: Pretty much basic mod_jk and Tomcat settings for AJP13

Description of the issue:

Under very heavy loads for our standards (about thousand of concurrent
users)
we start seeing errors (listed at end botton of this e-mail) in the mod_jk
error log.

Our concern is mod_jk behavior on the web server. We closely monitor apache
and Apache itself is not under heavy stress. However, shortly after we
start
seeing these messages in the mod_jk's error log, web server becomes
unresponsive
and we have to re-start it.

While the web server server is unresponsive we monitor 8009 port - nothing
is going on it.

We monitor Tomcats and they are fine.

As soon as we re-start web servers traffic resumes. Then under heavy loads
things break again

We are planning on increasing the number of max threads on the Tomcats'
AJP connector to imrpove the throughput of the Tomcats.

However, our concern is with the web server side lock ups. It seems that
mod_jk cannot
recover after this oversaturation.


TAIL -F FROM THE MOD_JK LOG RIGHT BEFORE  IT "DIES"
__


[Fri Jul 07 09:59:43 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:43 2006] [info]  ajp_service::jk_ajp_common.c (1749):
Sending request to tomcat failed,  recoverable operation attempt=1

[Fri Jul 07 09:59:44 2006] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
response message from tomcat, network problems or tomcat is down (:8009), err=-110

[Fri Jul 07 09:59:44 2006] [error] ajp_get_reply::jk_ajp_common.c (1503):
Tomcat is down or refused connection. No response has been sent to the
client (yet)

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1749):
Sending request to tomcat failed,  recoverable operation attempt=1

[Fri Jul 07 09:59:44 2006] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
response message from tomcat, network problems or tomcat is down (:8009), err=-110

[Fri Jul 07 09:59:44 2006] [error] ajp_get_reply::jk_ajp_common.c (1503):
Tomcat is down or refused connection. No response has been sent to the
client (yet)

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1749):
Sending request to tomcat failed,  recoverable operation attempt=1

[Fri Jul 07 09:59:44 2006] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
response message from tomcat, network problems or tomcat is down (:8009), err=-110

[Fri Jul 07 09:59:44 2006] [error] ajp_get_reply::jk_ajp_common.c (1503):
Tomcat is down or refused connection. No response has been sent to the
client (yet)

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1749):
Sending request to tomcat failed,  recoverable operation attempt=1

[Fri Jul 07 09:59:45 2006] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
response message from tomcat, network problems or tomcat is down (:8009), err=-110

[Fri Jul 07 09:59:45 2006] [error] ajp_get_reply::jk_ajp_common.c (1503):
Tomcat is down or refused connection. No response has been sent to the
client (yet)

[Fri Jul 07 09:59:45 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:45 2006] [info]  ajp_service::jk_ajp_common.c (1749):
Sending request to tomcat failed,  recoverable operation attempt=1

[Fri Jul 07 09:59:46 2006] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
response message from tomcat, network problems or tomcat is down (:8009), err=-110

[Fri Jul 07 09:59:46 2006] [error] ajp_get_reply::jk_ajp_common.c (1503):
Tomcat is down or refused connection. No response has been sent to the
client (yet)

[Fri Jul 07 09:59:46 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:46 2006] [info]  ajp_service::jk_ajp_common.c (1749):
Sending request to tomcat failed,  recoverable operation attempt=1

[Fri Jul 07 09:59:46 2006] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
response message from tomcat, network problems or tomcat is down (:8009), err=-110

[Fri Jul 07 

Re: URGENT HELP NEEDED: mod_jk loadbalancing issues

2006-07-09 Thread Alex Turner

What is your load average under these circumstance?

Is each of the 10 tomcats on a seperate physical machine?

What are your max thread settings in Tomcat?

Are your tomcat machines CPU bound or I/O bound under full load?

Alex

On 7/9/06, Edmon Begoli <[EMAIL PROTECTED]> wrote:


Hi,

First of all - thanks Rainer for willing to look into this issue.

Environment description:

Web Server: Apache 2.0.52
mod_jk: 1.2.15
OS: RedHat ES 4.2
App. Server: Tomcat 5.5.15

Cluster topology: 4 webservers with mod_jk doing sticky loadbalancing into
10 tomcats

MOD_JK Configuration: Pretty much basic mod_jk and Tomcat settings for
AJP13

Description of the issue:

Under very heavy loads for our standards (about thousand of concurrent
users)
we start seeing errors (listed at end botton of this e-mail) in the mod_jk
error log.

Our concern is mod_jk behavior on the web server. We closely monitor
apache
and Apache itself is not under heavy stress. However, shortly after we
start
seeing these messages in the mod_jk's error log, web server becomes
unresponsive
and we have to re-start it.

While the web server server is unresponsive we monitor 8009 port - nothing
is going on it.

We monitor Tomcats and they are fine.

As soon as we re-start web servers traffic resumes. Then under heavy loads
things break again

We are planning on increasing the number of max threads on the Tomcats'
AJP connector to imrpove the throughput of the Tomcats.

However, our concern is with the web server side lock ups. It seems that
mod_jk cannot
recover after this oversaturation.


TAIL -F FROM THE MOD_JK LOG RIGHT BEFORE  IT "DIES"
__


[Fri Jul 07 09:59:43 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:43 2006] [info]  ajp_service::jk_ajp_common.c (1749):
Sending request to tomcat failed,  recoverable operation attempt=1

[Fri Jul 07 09:59:44 2006] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
response message from tomcat, network problems or tomcat is down (:8009), err=-110

[Fri Jul 07 09:59:44 2006] [error] ajp_get_reply::jk_ajp_common.c (1503):
Tomcat is down or refused connection. No response has been sent to the
client (yet)

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1749):
Sending request to tomcat failed,  recoverable operation attempt=1

[Fri Jul 07 09:59:44 2006] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
response message from tomcat, network problems or tomcat is down (:8009), err=-110

[Fri Jul 07 09:59:44 2006] [error] ajp_get_reply::jk_ajp_common.c (1503):
Tomcat is down or refused connection. No response has been sent to the
client (yet)

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1749):
Sending request to tomcat failed,  recoverable operation attempt=1

[Fri Jul 07 09:59:44 2006] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
response message from tomcat, network problems or tomcat is down (:8009), err=-110

[Fri Jul 07 09:59:44 2006] [error] ajp_get_reply::jk_ajp_common.c (1503):
Tomcat is down or refused connection. No response has been sent to the
client (yet)

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1749):
Sending request to tomcat failed,  recoverable operation attempt=1

[Fri Jul 07 09:59:45 2006] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
response message from tomcat, network problems or tomcat is down (:8009), err=-110

[Fri Jul 07 09:59:45 2006] [error] ajp_get_reply::jk_ajp_common.c (1503):
Tomcat is down or refused connection. No response has been sent to the
client (yet)

[Fri Jul 07 09:59:45 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:45 2006] [info]  ajp_service::jk_ajp_common.c (1749):
Sending request to tomcat failed,  recoverable operation attempt=1

[Fri Jul 07 09:59:46 2006] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
response message from tomcat, network problems or tomcat is down (:8009), err=-110

[Fri Jul 07 09:59:46 2006] [error] ajp_get_reply::jk_ajp_common.c (1503):
Tomcat is down or refused connection. No response has been sent to the
client (yet)

[Fri Jul 07 09:59:46 2006] [info]  ajp_service::jk_ajp_common.c (1721):
Receiving from tomcat failed, recoverable operation attempt=0

[Fri Jul 07 09:59:46 2006] [info]  ajp_service::jk_ajp_common

Re: URGENT HELP NEEDED: mod_jk loadbalancing issues

2006-07-10 Thread Edmon Begoli

Each Tomcat machine is running in 512 MB Max.
(We recently increased that to ~850 MB. We are trying not to go up to far to
avoid prolonged garbage collector pauses)

CPU, I/O utilization under heavy loads is not excessive. 80% max usually 60%

Max. thread settings are default.

I was thinking to bump it by 35-50% as well as the backlog.

BTW - What would be the impoact/side effects of increasing the Connectors
max threads and backlog size.

Thanks,
Edmon


_



What is your load average under these circumstance?


Is each of the 10 tomcats on a seperate physical machine?

What are your max thread settings in Tomcat?

Are your tomcat machines CPU bound or I/O bound under full load?

Alex

On 7/9/06, Edmon Begoli <[EMAIL PROTECTED]> wrote:
>
> Hi,
>
> First of all - thanks Rainer for willing to look into this issue.
>
> Environment description:
>
> Web Server: Apache 2.0.52
> mod_jk: 1.2.15
> OS: RedHat ES 4.2
> App. Server: Tomcat 5.5.15
>
> Cluster topology: 4 webservers with mod_jk doing sticky loadbalancing
into
> 10 tomcats
>
> MOD_JK Configuration: Pretty much basic mod_jk and Tomcat settings for
> AJP13
>
> Description of the issue:
>
> Under very heavy loads for our standards (about thousand of concurrent
> users)
> we start seeing errors (listed at end botton of this e-mail) in the
mod_jk
> error log.
>
> Our concern is mod_jk behavior on the web server. We closely monitor
> apache
> and Apache itself is not under heavy stress. However, shortly after we
> start
> seeing these messages in the mod_jk's error log, web server becomes
> unresponsive
> and we have to re-start it.
>
> While the web server server is unresponsive we monitor 8009 port -
nothing
> is going on it.
>
> We monitor Tomcats and they are fine.
>
> As soon as we re-start web servers traffic resumes. Then under heavy
loads
> things break again
>
> We are planning on increasing the number of max threads on the Tomcats'
> AJP connector to imrpove the throughput of the Tomcats.
>
> However, our concern is with the web server side lock ups. It seems that
> mod_jk cannot
> recover after this oversaturation.
>
>
> TAIL -F FROM THE MOD_JK LOG RIGHT BEFORE  IT "DIES"
> __
>
>
> [Fri Jul 07 09:59:43 2006] [info]  ajp_service::jk_ajp_common.c (1721):
> Receiving from tomcat failed, recoverable operation attempt=0
>
> [Fri Jul 07 09:59:43 2006] [info]  ajp_service::jk_ajp_common.c (1749):
> Sending request to tomcat failed,  recoverable operation attempt=1
>
> [Fri Jul 07 09:59:44 2006] [error]
> ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
> response message from tomcat, network problems or tomcat is down ( REMOVED>:8009), err=-110
>
> [Fri Jul 07 09:59:44 2006] [error] ajp_get_reply::jk_ajp_common.c
(1503):
> Tomcat is down or refused connection. No response has been sent to the
> client (yet)
>
> [Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1721):
> Receiving from tomcat failed, recoverable operation attempt=0
>
> [Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1749):
> Sending request to tomcat failed,  recoverable operation attempt=1
>
> [Fri Jul 07 09:59:44 2006] [error]
> ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
> response message from tomcat, network problems or tomcat is down ( REMOVED>:8009), err=-110
>
> [Fri Jul 07 09:59:44 2006] [error] ajp_get_reply::jk_ajp_common.c
(1503):
> Tomcat is down or refused connection. No response has been sent to the
> client (yet)
>
> [Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1721):
> Receiving from tomcat failed, recoverable operation attempt=0
>
> [Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1749):
> Sending request to tomcat failed,  recoverable operation attempt=1
>
> [Fri Jul 07 09:59:44 2006] [error]
> ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
> response message from tomcat, network problems or tomcat is down ( REMOVED>:8009), err=-110
>
> [Fri Jul 07 09:59:44 2006] [error] ajp_get_reply::jk_ajp_common.c
(1503):
> Tomcat is down or refused connection. No response has been sent to the
> client (yet)
>
> [Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1721):
> Receiving from tomcat failed, recoverable operation attempt=0
>
> [Fri Jul 07 09:59:44 2006] [info]  ajp_service::jk_ajp_common.c (1749):
> Sending request to tomcat failed,  recoverable operation attempt=1
>
> [Fri Jul 07 09:59:45 2006] [error]
> ajp_connection_tcp_get_message::jk_ajp_common.c (961): Can't receive the
> response message from tomcat, network problems or tomcat is down ( REMOVED>:8009), err=-110
>
> [Fri Jul 07 09:59:45 2006] [error] ajp_get_reply::jk_ajp_common.c
(1503):
> Tomcat is down or refused connection. No response has been sent to the
> client (yet)
>
> [Fri Jul 07 09:59:45 2006] [info]  ajp_service::jk_ajp_common.c (1721):
> Receiving from tomcat failed, recove