Re: proxy_ajp connect timeout fix.

[Jess Holle]

Also note that the mod_jk code works just fine here, i.e. its socket 
connection timeouts are obeyed without further hackery.


This is via jk_connect.c's nb_connect(), not APR, though -- so chalk one 
up for by-passing APR?


--
Jess Holle

Jess Holle wrote:

Ruediger Pluem wrote:

I guess you should move this over to d...@apr as this is likely a problem
with the windows specific connect call not returning immediately.
  
I moved this over to d...@apr as suggested, but have not received any 
responses there.


Note that I applied Matt Stevenson's suggested fix from earlier in 
this thread 
[http://marc.info/?l=apache-httpd-devm=122358323701009w=2] and the 
connection timeout then worked on Windows as expected with 8 dead 
ports being checked in between 1 and 2 seconds -- which is what I'd 
expect given a connectiontimeout of 160ms.


It would seem proxy_util.c should not have to do this but rather that 
whatever is needed to get connection timeouts to work on a given 
platform should be done in apr_socket_connect().


This raises another question, though.  Earlier in this thread there 
were claims that Matt Stevenson's patch had adverse performance 
impacts, e.g. on HTTPS.  Can someone explain how this could be?  I ask 
in part as unless/until someone figures out the right fix in APR, I'll 
have to use Matt's patch -- and would like to understand the downsides 
and mitigate them if possible.


--
Jess Holle

Re: proxy_ajp connect timeout fix.

[Jess Holle]

Ruediger Pluem wrote:

I guess you should move this over to d...@apr as this is likely a problem
with the windows specific connect call not returning immediately.
  
I moved this over to d...@apr as suggested, but have not received any 
responses there.


Note that I applied Matt Stevenson's suggested fix from earlier in this 
thread [http://marc.info/?l=apache-httpd-devm=122358323701009w=2] and 
the connection timeout then worked on Windows as expected with 8 dead 
ports being checked in between 1 and 2 seconds -- which is what I'd 
expect given a connectiontimeout of 160ms.


It would seem proxy_util.c should not have to do this but rather that 
whatever is needed to get connection timeouts to work on a given 
platform should be done in apr_socket_connect().


This raises another question, though.  Earlier in this thread there were 
claims that Matt Stevenson's patch had adverse performance impacts, e.g. 
on HTTPS.  Can someone explain how this could be?  I ask in part as 
unless/until someone figures out the right fix in APR, I'll have to use 
Matt's patch -- and would like to understand the downsides and mitigate 
them if possible.


--
Jess Holle

Re: proxy_ajp connect timeout fix.

[Ruediger Pluem]

On 12/17/2008 12:38 AM, Jess Holle wrote:
 The errno assignments you added did the trick.
 
 Unfortunately, I'm still missing the overall goal.  I have many proxy
 balance members like:
 
BalancerMember ajp://localhost:8010 route=tomcat1 min=16 max=80
smax=40 ttl=900 keepalive=Off timeout=9 retry=30
connectiontimeout=160ms flushpackets=on
 
BalancerMember ajp://localhost:8011 route=tomcat2 min=16 max=80
smax=40 ttl=900 keepalive=Off timeout=9 retry=30
connectiontimeout=160ms flushpackets=on
 
...
 
 However, the error log says:
 
[Tue Dec 16 17:32:*25* 2008] [error] (OS 10061)No connection could be
 made because the target machine actively refused it.  : proxy: AJP:
 attempt to connect to 127.0.0.1:8011 (localhost) failed
[Tue Dec 16 17:32:25 2008] [error] ap_proxy_connect_backend disabling
 worker for (localhost)
[Tue Dec 16 17:32:25 2008] [error] proxy: AJP: failed to make
 connection to backend: localhost
 
[Tue Dec 16 17:32:*26* 2008] [error] (OS 10061)No connection could be
 made because the target machine actively refused it.  : proxy: AJP:
 attempt to connect to 127.0.0.1:8012 (localhost) failed
[Tue Dec 16 17:32:26 2008] [error] ap_proxy_connect_backend disabling
 worker for (localhost)
[Tue Dec 16 17:32:26 2008] [error] proxy: AJP: failed to make
 connection to backend: localhost
...
 
 Each port (on Windows) still consistently takes right around 1 full
 second to reject. despite having set connectiontimeout to be 160ms.
 
 Something seems to still be awry here as 160ms is significantly less
 than 1000ms...

I guess you should move this over to d...@apr as this is likely a problem
with the windows specific connect call not returning immediately.

Regards

Rüdiger

Re: proxy_ajp connect timeout fix.

[Jess Holle]

Did anyone test this on Windows?

I ask as I am trying connectiontimeout=160ms as part of my Apache 2.2.11 
configuration and am getting a configuration error.  I get the same 
error with ping and other parameters which now use 
ap_timeout_parameter_parse().


My BalanceMember looks something like:

   BalancerMember ajp://localhost:8010 route=tomcat1 min=16 max=80 
smax=40 ttl=900 keepalive=Off timeout=9 retry=30 
connectiontimeout=160ms flushpackets=on


--
Jess Holle

Re: proxy_ajp connect timeout fix.

[Ruediger Pluem]

On 12/16/2008 11:17 PM, Jess Holle wrote:
 Did anyone test this on Windows?

I stumbled across the same issue on Red Hat AS 5 today.
Try to patch your APR with

http://svn.apache.org/viewvc?rev=727052view=rev

from APR trunk. This should fix this.

 
 I ask as I am trying connectiontimeout=160ms as part of my Apache 2.2.11
 configuration and am getting a configuration error.  I get the same
 error with ping and other parameters which now use
 ap_timeout_parameter_parse().
 
 My BalanceMember looks something like:
 
BalancerMember ajp://localhost:8010 route=tomcat1 min=16 max=80
 smax=40 ttl=900 keepalive=Off timeout=9 retry=30
 connectiontimeout=160ms flushpackets=on

Regards

Rüdiger

Re: proxy_ajp connect timeout fix.

[Jess Holle]

Thanks!

Ruediger Pluem wrote:

On 12/16/2008 11:17 PM, Jess Holle wrote:
  

Did anyone test this on Windows?



I stumbled across the same issue on Red Hat AS 5 today.
Try to patch your APR with

http://svn.apache.org/viewvc?rev=727052view=rev

from APR trunk. This should fix this.

  

I ask as I am trying connectiontimeout=160ms as part of my Apache 2.2.11
configuration and am getting a configuration error.  I get the same
error with ping and other parameters which now use
ap_timeout_parameter_parse().

My BalanceMember looks something like:

   BalancerMember ajp://localhost:8010 route=tomcat1 min=16 max=80
smax=40 ttl=900 keepalive=Off timeout=9 retry=30
connectiontimeout=160ms flushpackets=on



Regards

Rüdiger


  

Re: proxy_ajp connect timeout fix.

[Jess Holle]

The errno assignments you added did the trick.

Unfortunately, I'm still missing the overall goal.  I have many proxy 
balance members like:


   BalancerMember ajp://localhost:8010 route=tomcat1 min=16 max=80
   smax=40 ttl=900 keepalive=Off timeout=9 retry=30
   connectiontimeout=160ms flushpackets=on

   BalancerMember ajp://localhost:8011 route=tomcat2 min=16 max=80
   smax=40 ttl=900 keepalive=Off timeout=9 retry=30
   connectiontimeout=160ms flushpackets=on

   ...

However, the error log says:

   [Tue Dec 16 17:32:*25* 2008] [error] (OS 10061)No connection could be made 
because the target machine actively refused it.  : proxy: AJP: attempt to 
connect to 127.0.0.1:8011 (localhost) failed
   [Tue Dec 16 17:32:25 2008] [error] ap_proxy_connect_backend disabling worker 
for (localhost)
   [Tue Dec 16 17:32:25 2008] [error] proxy: AJP: failed to make connection to 
backend: localhost

   [Tue Dec 16 17:32:*26* 2008] [error] (OS 10061)No connection could be made 
because the target machine actively refused it.  : proxy: AJP: attempt to 
connect to 127.0.0.1:8012 (localhost) failed
   [Tue Dec 16 17:32:26 2008] [error] ap_proxy_connect_backend disabling worker 
for (localhost)
   [Tue Dec 16 17:32:26 2008] [error] proxy: AJP: failed to make connection to 
backend: localhost
   ...
 

Each port (on Windows) still consistently takes right around 1 full 
second to reject. despite having set connectiontimeout to be 160ms.


Something seems to still be awry here as 160ms is significantly less 
than 1000ms...


--
Jess Holle

Ruediger Pluem wrote:

On 12/16/2008 11:17 PM, Jess Holle wrote:
  

Did anyone test this on Windows?


I stumbled across the same issue on Red Hat AS 5 today.
Try to patch your APR with

http://svn.apache.org/viewvc?rev=727052view=rev

from APR trunk. This should fix this

I ask as I am trying connectiontimeout=160ms as part of my Apache 2.2.11
configuration and am getting a configuration error.  I get the same
error with ping and other parameters which now use
ap_timeout_parameter_parse().

My BalanceMember looks something like:

   BalancerMember ajp://localhost:8010 route=tomcat1 min=16 max=80
smax=40 ttl=900 keepalive=Off timeout=9 retry=30
connectiontimeout=160ms flushpackets=on


Regards

Rüdiger
  

Re: proxy_ajp connect timeout fix.

[Ruediger Pluem]

On 10/15/2008 07:39 AM, Mladen Turk wrote:
 Ruediger Pluem wrote:

 I guess this leaves us to the question whether we need to be able to set
 connectiontimeouts below one second.
 
 Right now we are using a simple atoi for parsing those
 config values. Some more advanced function for parsing the
 time should be added thought.
 Eg.
 timeout=30 (defaults to seconds)
 timeout=30ms
 timeout=30us
 timeout=30s[ec]
 timeout=1min
 timeout=1h
 
 This would be similar what we have for parsing some
 file sizes across the conf (1024, 1K, 1M, ...)
 
 I'm not aware of any standard defining that, but
 since we underneath use apr_time anything
 from 1us should be valid time (weather it makes
 sense is a different story).

How about the following:

Index: modules/proxy/mod_proxy.c
===
--- modules/proxy/mod_proxy.c   (Revision 704917)
+++ modules/proxy/mod_proxy.c   (Arbeitskopie)
@@ -41,8 +41,60 @@
 return sizeof(proxy_worker_stat);
 }

+/**
+ * Parse a given timeout parameter string into an apr_interval_time_t value.
+ * The unit of the time interval is given as postfix string to the numeric
+ * string. Currently the following units are understood:
+ *
+ * s: seconds
+ * m[s] : milliseconds
+ *
+ * If no unit is contained in the given timeout parameter the default_time_unit
+ * will be used instead.
+ * @param timeout_parameter The string containing the timeout parameter.
+ * @param timeout The timeout value to be returned.
+ * @param default_time_unit The default time unit to use if none is specified
+ * in timeout_parameter.
+ * @return Status value indicating whether the parsing was successful or not.
+ */
+/*
+ * XXX: Once this function has its final status parameter wise it makes sense
+ * to move it to some of the util??? files under server/ as public API.
+ */
+static apr_status_t ap_timeout_parameter_parse(const char *timeout_parameter,
+   apr_interval_time_t *timeout,
+   const char *default_time_unit)
+{
+char *endp;
+const char *time_str;
+apr_int64_t tout;

+tout = apr_strtoi64(timeout_parameter, endp, 10);
+if (errno) {
+return errno;
+}
+if (!endp || !*endp) {
+time_str = default_time_unit;
+}
+else {
+time_str = endp;
+}

+switch (*time_str) {
+/* Time is in seconds */
+case 's':
+*timeout = (apr_interval_time_t) apr_time_from_sec(tout);
+break;
+/* Time is in miliseconds */
+case 'm':
+*timeout = (apr_interval_time_t) tout * 1000;
+break;
+default:
+return APR_EGENERAL;
+}
+return APR_SUCCESS;
+}
+
 /*
  * A Web proxy module. Stages:
  *
@@ -77,6 +129,8 @@
 {

 int ival;
+apr_interval_time_t timeout;
+
 if (!strcasecmp(key, loadfactor)) {
 /* Normalized load factor. Used with BalancerMamber,
  * it is a number between 1 and 100.
@@ -267,12 +321,13 @@
 worker-flush_wait = ival * 1000;/* change to microseconds */
 }
 else if (!strcasecmp(key, ping)) {
-/* Ping/Pong timeout in seconds.
+/* Ping/Pong timeout in given unit (default is second).
  */
-ival = atoi(val);
-if (ival  1)
-return Ping/Pong timeout must be at least one second;
-worker-ping_timeout = apr_time_from_sec(ival);
+if (ap_timeout_parameter_parse(val, timeout, s) != APR_SUCCESS)
+return Ping/Pong timeout has wrong format;
+if (timeout  1)
+return Ping/Pong timeout must be at least one millisecond;
+worker-ping_timeout = timeout;
 worker-ping_timeout_set = 1;
 }
 else if (!strcasecmp(key, lbset)) {
@@ -282,13 +337,14 @@
 worker-lbset = ival;
 }
 else if (!strcasecmp(key, connectiontimeout)) {
-/* Request timeout in seconds.
+/* Request timeout in given unit (default is second).
  * Defaults to connection timeout
  */
-ival = atoi(val);
-if (ival  1)
-return Connectiontimeout must be at least one second.;
-worker-conn_timeout = apr_time_from_sec(ival);
+if (ap_timeout_parameter_parse(val, timeout, s) != APR_SUCCESS)
+return Connectiontimeout has wrong format;
+if (timeout  1)
+return Connectiontimeout must be at least one millisecond.;
+worker-conn_timeout = timeout;
 worker-conn_timeout_set = 1;
 }
 else {

Regards

Rüdiger

Re: proxy_ajp connect timeout fix.

[Mladen Turk]

Ruediger Pluem wrote:




This would be similar what we have for parsing some
file sizes across the conf (1024, 1K, 1M, ...)

I'm not aware of any standard defining that, but
since we underneath use apr_time anything
from 1us should be valid time (weather it makes
sense is a different story).


How about the following:



+1

More time modifies (minutes and hours, 'M', 'hH') would be
handy, but that's it.
Think it's even backportable to 2.2.x.

Regards
--
^(TM)

Re: proxy_ajp connect timeout fix.

[Jess Holle]

Mladen Turk wrote:

Ruediger Pluem wrote:

This would be similar what we have for parsing some
file sizes across the conf (1024, 1K, 1M, ...)

I'm not aware of any standard defining that, but
since we underneath use apr_time anything
from 1us should be valid time (weather it makes
sense is a different story).


How about the following:

+1

More time modifies (minutes and hours, 'M', 'hH') would be
handy, but that's it.
More modifiers would be /nice/ but seconds and milliseconds suffice for 
my current needs.

Think it's even backportable to 2.2.x.

That would be good as I need it in 2.2.x :-)

--
Jess Holle

Re: proxy_ajp connect timeout fix.

[Ruediger Pluem]

On 10/09/2008 10:11 PM, Matt Stevenson wrote:
 Had a bit more time, here is a patch that should work for Unix which have 
 apr_wait_for_io_or_timeout available. I can't test windows/others so that's 
 the reason for the ifdef.
 
 ProxyPass / balance://hotcluster/
 Proxy balance://hotcluster
   # defaultish tomcat
   BalancerMember ajp://10.136.130.111:7009  loadfactor=1 connectiontimeout=2
   # below IP is not reachable, acts like a down box
   BalancerMember ajp://192.168.0.7:7010  loadfactor=1 connectiontimeout=2
 /Proxy
 
 
 Index: modules/proxy/proxy_util.c
 ===
 --- modules/proxy/proxy_util.c  (revision 703219)
 +++ modules/proxy/proxy_util.c  (working copy)
 @@ -2358,9 +2358,18 @@
   proxy: %s: fam %d socket created to connect to %s,
   proxy_function, backend_addr-family, worker-hostname);
 
 +/* use non blocking for connect timeouts to work. The ifdef
 +   limits to unix systems which have apr_wait_for_io_or_timeout.
 +   TODO: remove the ifdef and see what works/breaks */
 +#ifdef USE_WAIT_FOR_IO
 +apr_socket_opt_set(newsock,  APR_SO_NONBLOCK, 1);
 +#endif
  /* make the connection out of the socket */
  rv = apr_socket_connect(newsock, backend_addr);
 
 +#ifdef USE_WAIT_FOR_IO
 +apr_socket_opt_set(newsock,  APR_SO_NONBLOCK, 0);
 +#endif
  /* if an error occurred, loop round and try again */
  if (rv != APR_SUCCESS) {
  apr_socket_close(newsock);
 

Have you really checked that this patch solves your problem and that the problem
is not solved without?
While reviewing this patch for backport I noticed that it is wrong and that it 
shouldn't
be needed. The call to apr_socket_timeout_set before apr_socket_connect already 
sets the
socket to non-blocking mode because the timeout of the socket is -1 after 
creation. A further
call to apr_socket_timeout_set (after the connect call does not do this, 
because the old
and the new timeout are =0). The further code expects the socket to be in 
non-blocking
mode, otherwise we have regressions with ssl. This can be notified by running 
t/ssl/proxy
on 2.2.x which runs much much slower with the patch applied. This does not 
happen
on trunk because the socket is set back to non blocking by the core output 
filter
(async write completion).
So r703998 should be reverted on trunk.

Regards

Rüdiger

Re: proxy_ajp connect timeout fix.

[Ruediger Pluem]

On 10/15/2008 02:28 AM, Ruediger Pluem wrote:
 
 On 10/09/2008 10:11 PM, Matt Stevenson wrote:
 Had a bit more time, here is a patch that should work for Unix which have 
 apr_wait_for_io_or_timeout available. I can't test windows/others so 
 that's the reason for the ifdef.

 ProxyPass / balance://hotcluster/
 Proxy balance://hotcluster
   # defaultish tomcat
   BalancerMember ajp://10.136.130.111:7009  loadfactor=1 connectiontimeout=2
   # below IP is not reachable, acts like a down box
   BalancerMember ajp://192.168.0.7:7010  loadfactor=1 connectiontimeout=2
 /Proxy


 Index: modules/proxy/proxy_util.c
 ===
 --- modules/proxy/proxy_util.c  (revision 703219)
 +++ modules/proxy/proxy_util.c  (working copy)
 @@ -2358,9 +2358,18 @@
   proxy: %s: fam %d socket created to connect to %s,
   proxy_function, backend_addr-family, 
 worker-hostname);

 +/* use non blocking for connect timeouts to work. The ifdef
 +   limits to unix systems which have apr_wait_for_io_or_timeout.
 +   TODO: remove the ifdef and see what works/breaks */
 +#ifdef USE_WAIT_FOR_IO
 +apr_socket_opt_set(newsock,  APR_SO_NONBLOCK, 1);
 +#endif
  /* make the connection out of the socket */
  rv = apr_socket_connect(newsock, backend_addr);

 +#ifdef USE_WAIT_FOR_IO
 +apr_socket_opt_set(newsock,  APR_SO_NONBLOCK, 0);
 +#endif
  /* if an error occurred, loop round and try again */
  if (rv != APR_SUCCESS) {
  apr_socket_close(newsock);

 
 Have you really checked that this patch solves your problem and that the 
 problem
 is not solved without?
 While reviewing this patch for backport I noticed that it is wrong and that 
 it shouldn't
 be needed. The call to apr_socket_timeout_set before apr_socket_connect 
 already sets the
 socket to non-blocking mode because the timeout of the socket is -1 after 
 creation. A further
 call to apr_socket_timeout_set (after the connect call does not do this, 
 because the old
 and the new timeout are =0). The further code expects the socket to be in 
 non-blocking
 mode, otherwise we have regressions with ssl. This can be notified by running 
 t/ssl/proxy
 on 2.2.x which runs much much slower with the patch applied. This does not 
 happen
 on trunk because the socket is set back to non blocking by the core output 
 filter
 (async write completion).
 So r703998 should be reverted on trunk.

I checked 2.2.x and trunk in the meantime and they behaves as they should 
*without* the patch.
If I try to connect to a non existing host the apr_socket_connect call returns
after the timeout set via connectiontimeout.
I guess this leaves us to the question whether we need to be able to set
connectiontimeouts below one second.
I reverted r703998 on trunk.

Regards

Rüdiger

Re: proxy_ajp connect timeout fix.

[Jess Holle]

Ruediger Pluem wrote:

I checked 2.2.x and trunk in the meantime and they behaves as they should 
*without* the patch.
If I try to connect to a non existing host the apr_socket_connect call returns
after the timeout set via connectiontimeout.
I guess this leaves us to the question whether we need to be able to set
connectiontimeouts below one second.
I reverted r703998 on trunk.
  
I need some means of getting proxy connections to sockets with nothing 
listening on them to fail in less than 1 second as a workaround to 
Windows' noncompliant RST handling.


I believe Mladen just added this to mod_jk (where I also need such a 
capability for the IIS/Tomcat connector).


This seems cleaner than the GetTcpTable mess I was creating (as I'd 
assume local connections should take significantly less than 1 second to 
connect when successful).


--
Jess Holle

Re: proxy_ajp connect timeout fix.

[Mladen Turk]

Ruediger Pluem wrote:


I guess this leaves us to the question whether we need to be able to set
connectiontimeouts below one second.


Right now we are using a simple atoi for parsing those
config values. Some more advanced function for parsing the
time should be added thought.
Eg.
timeout=30 (defaults to seconds)
timeout=30ms
timeout=30us
timeout=30s[ec]
timeout=1min
timeout=1h

This would be similar what we have for parsing some
file sizes across the conf (1024, 1K, 1M, ...)

I'm not aware of any standard defining that, but
since we underneath use apr_time anything
from 1us should be valid time (weather it makes
sense is a different story).

Regards
--
^(TM)

Re: proxy_ajp connect timeout fix.

[Ruediger Pluem]

On 10/09/2008 10:11 PM, Matt Stevenson wrote:
 Had a bit more time, here is a patch that should work for Unix which have 
 apr_wait_for_io_or_timeout available. I can't test windows/others so that's 
 the reason for the ifdef.
 
 ProxyPass / balance://hotcluster/
 Proxy balance://hotcluster
   # defaultish tomcat
   BalancerMember ajp://10.136.130.111:7009  loadfactor=1 connectiontimeout=2
   # below IP is not reachable, acts like a down box
   BalancerMember ajp://192.168.0.7:7010  loadfactor=1 connectiontimeout=2
 /Proxy
 

Committed to trunk slightly modified as r703998 
(http://svn.apache.org/viewvc?rev=703998view=rev).
Thanks for the patch.

Regards

Rüdiger

Re: proxy_ajp connect timeout fix.

[Ruediger Pluem]

On 10/08/2008 10:55 PM, Matt Stevenson wrote:
 Hi,
 
 I've used mod_jk (1/2) for years. I've always had an issue when a backend 
 server goes down, not tomcat/jboss stopped
 but dead. Recently some people I work with have been using mod_proxy and 
 mod_proxy_ajp. This seems to have the same issue.
 
 The code (proxy_util.c) assumes that apr_socket_timeout_set works for all 
 connects. I don't believe it does, not unless
 it is in non blocking mode. I wrote the code below 
 forap_proxy_connect_backend before I looked deeper into the apr libs
 (sorry its not in diff format, and for the hard 2 sec timeout). The code 
 seems to work fine for linux (and probably other
 unix). I've basically redone the apr code in apr_wait_for_io_or_timeout 
 (should have dug deeper first).
 
 Anyway the current release code doesn't seem to work for me for down boxes 
 (to test point an ajp proxy at a non existant
 IP on the network and a live server). I think if you put the socket in 
 non-blocking mode first and with a timeout apr will
 try to handle a connect timeout (I haven't had a chance to try), switch back 
 to non blocking after connect.

Exactly. This might be the only thing missing in ap_proxy_connect_backend: 
Setting the socket to non blocking before
calling apr_socket_connect and setting it to blocking again afterwards.

Regards

Rüdiger

Re: proxy_ajp connect timeout fix.

[Matt Stevenson]

Had a bit more time, here is a patch that should work for Unix which have 
apr_wait_for_io_or_timeout available. I can't test windows/others so that's 
the reason for the ifdef.

ProxyPass / balance://hotcluster/
Proxy balance://hotcluster
  # defaultish tomcat
  BalancerMember ajp://10.136.130.111:7009  loadfactor=1 connectiontimeout=2
  # below IP is not reachable, acts like a down box
  BalancerMember ajp://192.168.0.7:7010  loadfactor=1 connectiontimeout=2
/Proxy


Index: modules/proxy/proxy_util.c
===
--- modules/proxy/proxy_util.c  (revision 703219)
+++ modules/proxy/proxy_util.c  (working copy)
@@ -2358,9 +2358,18 @@
  proxy: %s: fam %d socket created to connect to %s,
  proxy_function, backend_addr-family, worker-hostname);

+/* use non blocking for connect timeouts to work. The ifdef
+   limits to unix systems which have apr_wait_for_io_or_timeout.
+   TODO: remove the ifdef and see what works/breaks */
+#ifdef USE_WAIT_FOR_IO
+apr_socket_opt_set(newsock,  APR_SO_NONBLOCK, 1);
+#endif
 /* make the connection out of the socket */
 rv = apr_socket_connect(newsock, backend_addr);

+#ifdef USE_WAIT_FOR_IO
+apr_socket_opt_set(newsock,  APR_SO_NONBLOCK, 0);
+#endif
 /* if an error occurred, loop round and try again */
 if (rv != APR_SUCCESS) {
 apr_socket_close(newsock);


Regards
matt



- Original Message 
From: Matt Stevenson [EMAIL PROTECTED]
To: dev@httpd.apache.org
Sent: Wednesday, October 8, 2008 9:55:43 PM
Subject: proxy_ajp connect timeout fix.

Hi,

I've used mod_jk (1/2) for years. I've always had an issue when a backend 
server goes down, not tomcat/jboss stopped but dead. Recently some people I 
work with have been using mod_proxy and mod_proxy_ajp. This seems to have the 
same issue.

The code (proxy_util.c) assumes that apr_socket_timeout_set works for all 
connects. I don't believe it does, not unless it is in non blocking mode. I 
wrote the code below forap_proxy_connect_backend before I looked deeper into 
the apr libs (sorry its not in diff format, and for the hard 2 sec timeout). 
The code seems to work fine for linux (and probably other unix). I've basically 
redone the apr code in apr_wait_for_io_or_timeout (should have dug deeper 
first).

Anyway the current release code doesn't seem to work for me for down boxes (to 
test point an ajp proxy at a non existant IP on the network and a live server). 
I think if you put the socket in non-blocking mode first and with a timeout apr 
will try to handle a connect timeout (I haven't had a chance to try), switch 
back to non blocking after connect.

Regards
Matt


if (worker-keepalive) {
if ((rv =  apr_socket_opt_set(newsock,
APR_SO_KEEPALIVE, 1))  != APR_SUCCESS) {
ap_log_error(APLOG_MARK, APLOG_ERR, rv,  s,
 apr_socket_opt_set(SO_KEEPALIVE): Failed to  set
  Keepalive);
}
 }
ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s,
  proxy: %s: fam %d socket created to connect to %s,
  proxy_function, backend_addr-family, worker-hostname);

apr_socket_opt_set(newsock,  APR_SO_NONBLOCK, 1);
apr_socket_timeout_set(newsock, 0);

rv = apr_socket_connect(newsock,  backend_addr);
if( rv != APR_SUCCESS   APR_STATUS_IS_EINPROGRESS(rv)){

apr_pollfd_t pfds[1];
 apr_status_t status;
apr_int32_t  nfds;


pfds[0].reqevents = APR_POLLOUT;
 pfds[0].desc_type = APR_POLL_SOCKET;
pfds[0].desc.s =  newsock;

rv = apr_poll(pfds[0], 1, nfds,  apr_time_from_sec(2));
}

/* if an error occurred, loop round and try again */
if  (rv != APR_SUCCESS) {
apr_socket_close(newsock);
 loglevel = backend_addr-next ? APLOG_DEBUG : APLOG_ERR;
 ap_log_error(APLOG_MARK, loglevel, rv, s,
 proxy:  %s: attempt to connect to %pI (%s) failed,
  proxy_function,
  backend_addr,
 worker-hostname);
 backend_addr = backend_addr-next;
continue;
 }

apr_socket_opt_set(newsock, APR_SO_NONBLOCK, 0);

/* Set a timeout on the socket */


  

Re: proxy_ajp connect timeout fix.

[Ruediger Pluem]

On 10/09/2008 10:11 PM, Matt Stevenson wrote:
 Had a bit more time, here is a patch that should work for Unix which have 
 apr_wait_for_io_or_timeout available. I can't test windows/others so that's 
 the reason for the ifdef.
 
 ProxyPass / balance://hotcluster/
 Proxy balance://hotcluster
   # defaultish tomcat
   BalancerMember ajp://10.136.130.111:7009  loadfactor=1 connectiontimeout=2
   # below IP is not reachable, acts like a down box
   BalancerMember ajp://192.168.0.7:7010  loadfactor=1 connectiontimeout=2
 /Proxy
 
 
 Index: modules/proxy/proxy_util.c
 ===
 --- modules/proxy/proxy_util.c  (revision 703219)
 +++ modules/proxy/proxy_util.c  (working copy)
 @@ -2358,9 +2358,18 @@
   proxy: %s: fam %d socket created to connect to %s,
   proxy_function, backend_addr-family, worker-hostname);
 
 +/* use non blocking for connect timeouts to work. The ifdef
 +   limits to unix systems which have apr_wait_for_io_or_timeout.
 +   TODO: remove the ifdef and see what works/breaks */
 +#ifdef USE_WAIT_FOR_IO
 +apr_socket_opt_set(newsock,  APR_SO_NONBLOCK, 1);
 +#endif
  /* make the connection out of the socket */
  rv = apr_socket_connect(newsock, backend_addr);
 
 +#ifdef USE_WAIT_FOR_IO
 +apr_socket_opt_set(newsock,  APR_SO_NONBLOCK, 0);
 +#endif
  /* if an error occurred, loop round and try again */
  if (rv != APR_SUCCESS) {
  apr_socket_close(newsock);
 

According to the APR source files only older versions of BeOS do not provide 
apr_wait_for_io_or_timeout.
So I would say that it is pretty much save to remove the ifdef's

Regards

Rüdiger