Logs show reasonable request handling duration, but proxied clients timing out

[eric.berg]

For sometime I've had complaints from one of the users of a particular
mod_perl handler that is timing out on their end.  They are using a
programmatic Java client, accessing a mod_perl registry handler that
simply returns a text file.

 

The client is accessing my servers via a proxy which then connects to a
load balancer that balances between two of my boxes.

 

The client uses a 500 millisecond read timeout which is often reached,
causing the client process to throw exceptions.  However, when I look at
my logs, the %D param shows durations well below this limit.

 

At times I do not see the requests at all.

 

What should I be looking at on my servers to see if the problem is on my
end?

 

As much as it appears that the problem is in the proxies, I want to make
sure that I've covered all the bases from my end before I escalate the
issue.

 

There are no fixup handlers or log handlers, which I believe might
impact post-logging timings.

 

Thanks, everybody.

 

Eric


___

This e-mail may contain information that is confidential, privileged or 
otherwise protected from disclosure. If you are not an intended recipient of 
this e-mail, do not duplicate or redistribute it by any means. Please delete it 
and any attachments and notify the sender that you have received it in error. 
Unless specifically indicated, this e-mail is not an offer to buy or sell or a 
solicitation to buy or sell any securities, investment products or other 
financial product or service, an official confirmation of any transaction, or 
an official statement of Barclays. Any views or opinions presented are solely 
those of the author and do not necessarily represent those of Barclays. This 
e-mail is subject to terms available at the following link: 
www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the 
foregoing.  Barclays Capital is the investment banking division of Barclays 
Bank PLC, a company registered in England (number 1026167) with its registered 
office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be 
sent from other members of the Barclays Group.
___

Re: Logs show reasonable request handling duration, but proxied clients timing out

[Perrin Harkins]

On Tue, Jan 5, 2010 at 11:39 AM,   wrote:
> The client uses a 500 millisecond read timeout which is often reached,
> causing the client process to throw exceptions.  However, when I look at my
> logs, the %D param shows durations well below this limit.
>
> At times I do not see the requests at all.

Sounds like a network problem.

> What should I be looking at on my servers to see if the problem is on my
> end?

You could try turning off keep-alive, in case the proxy handles that
poorly.  You could also set up a monitor script to hit the URL every 5
minutes and log the times.  Maybe try doing that directly and through
the proxy to compare.

- Perrin