Hello,
I've spent a week looking into mod_jk, Apache, etc., and the problem appears to
be due to a bug in the AJP connector within JBoss:
https://jira.jboss.org/browse/JBPAPP-366
This bug is not fixed in the latest release of JBoss 4.2.3, which is rather
disappointing. After downloading the j
> From: John Baker [mailto:jba...@javasystemsolutions.com]
> Subject: Re: 2 second delays in mod_jk while "maintaining workers"
> some of our JBoss instances are showing a large number
> ajp threads that seem to be in keep alive mode but are
> well beyond the con
On Thursday 09 September 2010 16:08:04 you wrote:
> On 09/09/2010 04:48 PM, John Baker wrote:
> >
> >
> > On Thursday 09 September 2010 15:45:44 you wrote:
> >> Nice.
> >
> > I spoke too soon. I'm now trying to figure out how to print out the IP
> > address of the socket (I don't really do C) s
On 09/09/2010 04:48 PM, John Baker wrote:
On Thursday 09 September 2010 15:45:44 you wrote:
Nice.
I spoke too soon. I'm now trying to figure out how to print out the IP address
of the socket (I don't really do C) so I can log the socket that caused the
poll to timeout, and compare with a
Interestingly, some of our JBoss instances are showing a large number ajp
threads that seem to be in keep alive mode but are well beyond the
connectionTimeout defined in server.xml (which is set to 9):
Max threads: 40 Current thread count: 40 Current thread busy: 40
Max processing time: 5563
I would be happy to share all my evidence and write a report once we get to the
bottom of this problem.
Any hints on printing out the socket IP (i.e. of Tomcat)?
On Thursday 09 September 2010 15:47:33 you wrote:
> On 09/09/2010 03:22 PM, John Baker wrote:
> >
> > Do you fancy putting that change
On Thursday 09 September 2010 15:45:44 you wrote:
> Nice.
I spoke too soon. I'm now trying to figure out how to print out the IP address
of the socket (I don't really do C) so I can log the socket that caused the
poll to timeout, and compare with a tcpdump.
On 09/09/2010 03:22 PM, John Baker wrote:
Do you fancy putting that change into the next release of mod_jk?
BTW, do you fancy opening BZ report fill in what we
tried so far and the actual solution, so we can track that down
for the next releases and patches?
Regards
--
^TM
-
On 09/09/2010 03:22 PM, John Baker wrote:
On Thursday 09 September 2010 13:59:50 you wrote:
If that doesn't help, it's obvious the Tomcat
doesn't close the socket, so should be investigated
why. Like said before, either the Tomcat doesn't
respond to shutdown or the shutdown's FIN packet
isn't s
On Thursday 09 September 2010 13:59:50 you wrote:
> >
> > If that doesn't help, it's obvious the Tomcat
> > doesn't close the socket, so should be investigated
> > why. Like said before, either the Tomcat doesn't
> > respond to shutdown or the shutdown's FIN packet
> > isn't send to the Tomcat or
On Sep 9, 2010, at 8:34 AM, Mladen Turk wrote:
> On 09/09/2010 02:09 PM, John Baker wrote:
>> Thanks for the feedback.
>>
>> Can you tell me why this if statement exists:
>>
>> if (poll(&fds, 1, timeout)> 0)
>>{
>>...
>>}
>>else
>> break;
>
On 09/09/2010 02:09 PM, John Baker wrote:
Thanks for the feedback.
Can you tell me why this if statement exists:
if (poll(&fds, 1, timeout)> 0)
{
...
}
else
break;
It appears to be at fault.
poll returns positive number if there is so
Thanks for the feedback.
Can you tell me why this if statement exists:
if (poll(&fds, 1, timeout) > 0)
{
...
}
else
break;
It appears to be at fault.
John
-
To unsubscribe,
On 09/08/2010 05:51 PM, John Baker wrote:
I think it would be helpful if you could walk us through the code.
It is very simple actually.
1. shutdown write end of our connection to Tomcat
This should cause the soket.read() in Tomcat
to throw exception which is used to close the socket
2.
Original Message--
From: John Baker
To: Tomcat Users List
ReplyTo: Tomcat Users List
Subject: Re: 2 second delays in mod_jk while "maintaining workers"
Sent: 8 Sep 2010 16:41
> On 09/08/2010 05:08 PM, John Baker wrote:
> The code *is* required.
> It is used when the client di
> On 09/08/2010 05:08 PM, John Baker wrote:
> The code *is* required.
> It is used when the client disconnects while the backend
> still has some data in the AJP buffer. Drain is needed
> to read that excess data.
Why does it always report 0 bytes read?
> If you can compile mod_jk and test, try
please can you remove me off the mailing list
many thanks
On Wed, Sep 8, 2010 at 4:29 PM, Mladen Turk wrote:
> On 09/08/2010 05:08 PM, John Baker wrote:
>
>> Hello,
>>
>> I don't thikn the shutdown call is to blame - I think it's the large pile
>> of code below. i.e. everything below the if (s
On 09/08/2010 05:08 PM, John Baker wrote:
Hello,
I don't thikn the shutdown call is to blame - I think it's the large pile
of code below. i.e. everything below the if (shutdown(..)) below. The
question is, what does it all do and does it actually work? It appears to
be the 'drain' code, but g
Hello,
I don't thikn the shutdown call is to blame - I think it's the large pile
of code below. i.e. everything below the if (shutdown(..)) below. The
question is, what does it all do and does it actually work? It appears to
be the 'drain' code, but given it often results in this message:
"Shu
Hello
We've just noticed that the maintenance mode operates on all workers, so
having one worker run maintenance for the rest is making tracing the
problem difficult. Reading down the logs, we can see it finds a worker
and iterates through workers perforing maintenance.
Is this correct?
Moving
On 07.09.2010 19:46, John Baker wrote:
Where can I find documentation on JkWatchdog?
General documntation on mod_jk:
http://tomcat.apache.org/connectors-doc/
JK directives: http://tomcat.apache.org/connectors-doc/reference/apache.html
workers.properties:
http://tomcat.apache.org/connectors-
Where can I find documentation on JkWatchdog?
On Tuesday 07 September 2010 13:51:23 you wrote:
> - As a workaround: using a JkWatchdog moves the maintain into a separate
> thread. But during the socket closing a lock is held, which blocks other
> threads from accessing the same worker.
---
On 07.09.2010 12:40, Rainer Jung wrote:
On 07.09.2010 12:32, John Baker wrote:
Rainer,
On Tuesday 07 September 2010 11:09:46 you wrote:
I don't like "socket_timeout" ...
worker.basic.socket_timeout=90
but I would like socket_connect_timeout.
The next two are possibly a bit short, because
On 07.09.2010 12:32, John Baker wrote:
Rainer,
On Tuesday 07 September 2010 11:09:46 you wrote:
I don't like "socket_timeout" ...
worker.basic.socket_timeout=90
but I would like socket_connect_timeout.
The next two are possibly a bit short, because if the backend e.g. does
a Java Garbage C
On 09/07/2010 12:20 PM, John Baker wrote:
On Tuesday 07 September 2010 11:13:07 you wrote:
It's obvious that
shutdown(socket, SHUT_WR)
poll(socket, 2 seconds)
close(socket)
caused poll call to time out, meaning that
the JBoss side didn't respond to the
shutdown(socket, SHUT_WR) call by
closi
On 07.09.2010 12:29, Mladen Turk wrote:
On 09/07/2010 12:09 PM, Rainer Jung wrote:
Are we sure, that those log lines are correct? This was generated by
mod_jk 1.2.30 without any code changes, right?
The other option would be to set JkWatchdogInterval 60
In that case wc_maintain would happen
Rainer,
On Tuesday 07 September 2010 11:09:46 you wrote:
> I don't like "socket_timeout" ...
>
> > worker.basic.socket_timeout=90
>
> but I would like socket_connect_timeout.
>
> The next two are possibly a bit short, because if the backend e.g. does
> a Java Garbage Collection which miht take
On 09/07/2010 12:09 PM, Rainer Jung wrote:
Are we sure, that those log lines are correct? This was generated by mod_jk
1.2.30 without any code changes, right?
The other option would be to set JkWatchdogInterval 60
In that case wc_maintain would happen in separate thread
without observable re
On Tuesday 07 September 2010 11:13:07 you wrote:
> It's obvious that
>
> shutdown(socket, SHUT_WR)
> poll(socket, 2 seconds)
> close(socket)
>
> caused poll call to time out, meaning that
> the JBoss side didn't respond to the
> shutdown(socket, SHUT_WR) call by
> closing it's side of the conne
Rainer,
On Tuesday 07 September 2010 11:09:46 you wrote:
> > [Tue Sep 07 10:20:20.617 2010] [18806:46962404156768] [debug]
> > find_match::jk_uri_worker_map.c (850): Attempting to map context URI
> > '/*=lb-jboss51-integration' source 'JkMount'
> > [Tue Sep 07 10:20:20.617 2010] [18806:469624041
On 09/07/2010 11:53 AM, John Baker wrote:
TM,
On Tuesday 07 September 2010 09:47:17 you wrote:
If you have enough resources, try to disable
connectionTimeout on AJP connector and see weather
the same will happen again.
I've removed the connectionTimeout attribute and the problem persists - I
On 07.09.2010 11:36, John Baker wrote:
Rainer,
Is this acceptable? I am using a tail and an egrep to match the various
statements you wish to see. if it's missing anything, plesae let me know what
to add to the grep.
Yes, that's better!
[Tue Sep 07 10:20:20.617 2010] [18806:4696240415676
TM,
On Tuesday 07 September 2010 09:47:17 you wrote:
> If you have enough resources, try to disable
> connectionTimeout on AJP connector and see weather
> the same will happen again.
I've removed the connectionTimeout attribute and the problem persists - I can
see the "lingering bytes in 2 sec"
Rainer,
Is this acceptable? I am using a tail and an egrep to match the various
statements you wish to see. if it's missing anything, plesae let me know what
to add to the grep.
[Tue Sep 07 10:20:20.617 2010] [18806:46962404156768] [debug]
find_match::jk_uri_worker_map.c (850): Attempting
On 09/07/2010 10:11 AM, John Baker wrote:
On Tuesday 07 September 2010 08:59:27 you wrote:
It means that socket shutdown failed.
Do you have firewall between mod_jk and JBoss or some non-standard
network driver (e.g running under some VM)?
We are using VMs but there should be no firewall. I s
On Tuesday 07 September 2010 08:59:27 you wrote:
> It means that socket shutdown failed.
> Do you have firewall between mod_jk and JBoss or some non-standard
> network driver (e.g running under some VM)?
We are using VMs but there should be no firewall. I should probably re-iterate
that the prob
On 07.09.2010 09:05, John Baker wrote:
On Tuesday 07 September 2010 06:28:33 you wrote:
On 09/06/2010 11:59 PM, John Baker wrote:
What's jk_maintain?
Function that maintains the workers
(closes excess connections inactive for a long time)
Anyhow, like Rainer said, if that's the case
you sho
On 09/07/2010 09:05 AM, John Baker wrote:
On Tuesday 07 September 2010 06:28:33 you wrote:
On 09/06/2010 11:59 PM, John Baker wrote:
What's jk_maintain?
Function that maintains the workers
(closes excess connections inactive for a long time)
Anyhow, like Rainer said, if that's the case
you
On Tuesday 07 September 2010 06:28:33 you wrote:
> On 09/06/2010 11:59 PM, John Baker wrote:
> > What's jk_maintain?
> >
>
> Function that maintains the workers
> (closes excess connections inactive for a long time)
>
> Anyhow, like Rainer said, if that's the case
> you should have something like
On 09/06/2010 11:59 PM, John Baker wrote:
On Monday 06 September 2010 22:57:21 you wrote:
I didn't look at the code now, but the 2 seconds remind me of the
connection draining during socket shutdown, which could be related to
jk_maintain?
What's jk_maintain?
Function that maintains the work
On Monday 06 September 2010 22:57:21 you wrote:
> I didn't look at the code now, but the 2 seconds remind me of the
> connection draining during socket shutdown, which could be related to
> jk_maintain?
What's jk_maintain?
-
To
On 06.09.2010 19:56, Mladen Turk wrote:
On 09/06/2010 04:16 PM, John Baker wrote:
I've set the Jk logging to trace and you can see the debug statements and
the 2s delays:
Do you use NFS share by any chance to store the
mod_jk log directory data?
I didn't look at the code now, but the 2 sec
On Monday 06 September 2010 18:56:20 you wrote:
> On 09/06/2010 04:16 PM, John Baker wrote:
> >
> > I've set the Jk logging to trace and you can see the debug statements and
> > the 2s delays:
> >
>
> Do you use NFS share by any chance to store the
> mod_jk log directory data?
Nope. All log fi
On 09/06/2010 04:16 PM, John Baker wrote:
I've set the Jk logging to trace and you can see the debug statements and
the 2s delays:
Do you use NFS share by any chance to store the
mod_jk log directory data?
What happens if you set
worker.lb-jboss51-integration.lock=O
Regards
--
^TM
---
44 matches
Mail list logo