Re: OS Keep-alive on forward proxy

2010-08-05 Thread Rainer Jung

On 05.08.2010 05:45, Ryujiro Shibuya wrote:

Hi,

Can you tell if the OS Keep-alive feature should be expected to work on
forward proxy?

The mod_proxy of latest httpd 2.2 and trunk (2.3) has OS Keep-alive
(Keep-alive connection to origin server) feature, but it seems not working
when we configure the httpd as the forward proxy server.
If the client connection is kept alive, the proxy sends indeed Connection:
Keep-alive to the origin server but the proxy doesn't actually try to keep
the OS connection alive; it closes the OS connection immediately after
receiving the response.

The inconsistency between sending Connection: Keep-alive and immediate
connection closing looks like a defect (and I've just posted a bug report
49699). But apart from the inconsistency, I wonder if the OS Keep-alive of
mod_proxy is designed to work for the forward proxy in the first place.


OS Keepalive and HTTP Keepalive are two totally different things.

OS Keepalive: if a connection is open but idle for a long time 
(configurable), then the OS automatically sends empty TCP packets over 
the connection in OS dependent intervals.


HTTP Keepalive: Signalling via HTTP Connection header, whether the 
connections is supposed to be kept open and eventually reused for 
another HTTP request/response.


OS Keepalive does not imply HTTP Keepalive. For all reasonable 
configurations (except possible server to server communication), HTTP 
Keepalive timeout will be much shorter than the minimum idle time, 
before OS Keepalive packets are send. So OS Keepalive for most HTTP 
communications is not relevant.


And yes: the forward proxy does *not* do HTTP Keepalive. Technical 
reason: the connections to the origin server are pooled and retrieved 
from and returned to the pool for each request. A forward proxy usually 
talks to many diferent origin servers. Keeping those connections open in 
a naive way would lead to a lot of not well used pools. Assuming that 
during one client connection the origin server often is used for 
multiple requests this could be improved, but would bloat the already 
complicated proxy code even more.


Regards,

Rainer


Re: Per request DocumentRoot

2010-08-05 Thread Ondřej Surý
I have a question how to proceed. Is there really no interest in doing
this? (And fixing mod-vhost-alias as well.)

Or should I fill the bug and attach the patch? Or ...?

Thanks,
Ondrej

On Mon, Jul 19, 2010 at 09:30, Ondřej Surý ond...@sury.org wrote:
 We could do something interesting with moving this to the req rec in 2.4
 (on 2.3 trunk) but it wouldn't solve your issue today with 2.2 or 2.0,
 which will not change.

 I am fine with that it doesn't get into 2.2 or 2.0. But I think that
 fixing this in the right place would allow all other vhost modules to
 benefit from that. mod-vhost-alias could be fixed as well.

 Here's the first version of patch for 2.3-trunk which implements
 per-request document_root.

 I have also updated m-v-a to use this new functionality.

 It compiles cleanly, however I'm not sure it it's complete. I hope I
 found all places where new request_rec is initialized, but I may have
 missed something.

 Ondrej
 --
 Ondřej Surý ond...@sury.org
 http://blog.rfc1925.org/




-- 
Ondřej Surý ond...@sury.org


Re: Per request DocumentRoot

2010-08-05 Thread William A. Rowe Jr.
On 8/5/2010 3:12 AM, Ondřej Surý wrote:
 I have a question how to proceed. Is there really no interest in doing
 this? (And fixing mod-vhost-alias as well.)
 
 Or should I fill the bug and attach the patch? Or ...?

Please don't confuse ignoring your patch with lack of interest.  Everyone
here is a volunteer and dedicates the cycles they have free to attacking
the most pressing issue they see at the time.

You certainly can, some people track bugzilla better than the dev list.
Others don't follow bugzilla closely, so both a bug and occassional dev@
reminder are both good.


Re: Per request DocumentRoot

2010-08-05 Thread Ondřej Surý
William,

On Thu, Aug 5, 2010 at 10:18, William A. Rowe Jr. wr...@rowe-clan.net wrote:
 On 8/5/2010 3:12 AM, Ondřej Surý wrote:
 I have a question how to proceed. Is there really no interest in doing
 this? (And fixing mod-vhost-alias as well.)

 Or should I fill the bug and attach the patch? Or ...?

 Please don't confuse ignoring your patch with lack of interest.  Everyone
 here is a volunteer and dedicates the cycles they have free to attacking
 the most pressing issue they see at the time.

Sorry if that offended you or anybody else. I didn't ment to sound
harsh as I was just merely asking what to do now.

 You certainly can, some people track bugzilla better than the dev list.
 Others don't follow bugzilla closely, so both a bug and occassional dev@
 reminder are both good.

Thanks, I'll do that.

Ondrej
-- 
Ondřej Surý ond...@sury.org


Re: Per request DocumentRoot

2010-08-05 Thread William A. Rowe Jr.
On 8/5/2010 3:22 AM, Ondřej Surý wrote:
 William,
 
 On Thu, Aug 5, 2010 at 10:18, William A. Rowe Jr. wr...@rowe-clan.net wrote:
 On 8/5/2010 3:12 AM, Ondřej Surý wrote:
 I have a question how to proceed. Is there really no interest in doing
 this? (And fixing mod-vhost-alias as well.)

 Or should I fill the bug and attach the patch? Or ...?

 Please don't confuse ignoring your patch with lack of interest.  Everyone
 here is a volunteer and dedicates the cycles they have free to attacking
 the most pressing issue they see at the time.
 
 Sorry if that offended you or anybody else. I didn't ment to sound
 harsh as I was just merely asking what to do now.

You didn't and I didn't mean to sound irritated, either, just trying to frame
what happens on this particular list for your info.

Many 'nice to have' things for Apache 2.4 are ignored as more critical issues
in 2.2 are addressed, but with rising interest in frequent alpha/betas, we can
all hope that improves :)


RE: OS Keep-alive on forward proxy

2010-08-05 Thread Ryujiro Shibuya
Hi Rainer,

I meant to talk about the HTTP Keep-alive to the origin server (OS), not
about TCP Keep-alive of operating system (OS).
Excuse me for my confusing description.

 And yes: the forward proxy does *not* do HTTP Keepalive. Technical 
 reason: the connections to the origin server are pooled and retrieved 
 from and returned to the pool for each request. A forward proxy usually 
 talks to many diferent origin servers. Keeping those connections open in 
 a naive way would lead to a lot of not well used pools. Assuming that 
 during one client connection the origin server often is used for 
 multiple requests this could be improved, but would bloat the already 
 complicated proxy code even more.
 
And thank you for the answer and reasons! That is what I wanted to hear.
So I understand the HTTP Keep-alive to the origin server is only supported
for reverse proxy, as the design of httpd 2.2/2.3.

Regards,
Ryujiro Shibuya





Re: Question about APR SHM

2010-08-05 Thread Alex Wulms
Op 31-07-10 18:07, Alex Wulms schreef:
 Hi,

 I have got a question about APR SHM. The comments of the function
 apr_shm_baseaddr_get(const apr_shm_t *m) indicate that the resulting
 address is only valid in the callers address space, since the API does
 not guarantee that other attaching processes will maintain the same
 address mapping.
 ...
   
Hi,

In order to play it safe, I have made my code function completely on top
of the RMM  (relocatable memory management) API already present in APR.

As a side-product, I have written a re-usable RMM hash table (derived
from apr_hash.c). You can find the code in the GIT repository located here:
http://repo.or.cz/w/httpd-crcsyncproxy.git

It concerns files crccache/rmm_hash.h and crccache/rmm_hash.c in the
repository.

Please feel free to include them in a future version of APR.

Kind regards,
Alex



Re: Failing startup for vhost configuration problems

2010-08-05 Thread Niklas Edmundsson

On Thu, 5 Aug 2010, Graham Dumpleton wrote:


On Thursday, August 5, 2010, Stefan Fritsch s...@sfritsch.de wrote:

On Tuesday 03 August 2010, Dan Poirier wrote:

I'd like to propose that in 2.3/2.4, we fail startup for any of the
virtual host misconfigurations for which behavior is undefined but
right now we only issue a warning.

snip

Perhaps warnings around bad MPM configuration can be reviewed as well.

From memory it doesn't at the moment warn you when something like

StartServers is set over what it can be based on MaxClients and
ThreadsPerChild. I think it may just silently reduce the value. Since
MPM settings are often misunderstood, better hints about bad
configuration there would be useful.


+1 on both suggestions. This would be a good time to change/improve on 
this front.


/Nikke
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
 Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se  | ni...@acc.umu.se
---
 Window-screen: An arrangement for keeping flies in.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=


Re: OS Keep-alive on forward proxy

2010-08-05 Thread Paul Fee
Rainer Jung wrote:

snip

 And yes: the forward proxy does *not* do HTTP Keepalive. Technical
 reason: the connections to the origin server are pooled and retrieved
 from and returned to the pool for each request. A forward proxy usually
 talks to many diferent origin servers. Keeping those connections open in
 a naive way would lead to a lot of not well used pools. Assuming that
 during one client connection the origin server often is used for
 multiple requests this could be improved, but would bloat the already
 complicated proxy code even more.

Has mod_proxy operated in that way for a while now?  I gained most of my 
experience with mod_proxy using Apache 2.0.X.  My understanding was that 
proxy to OS connections were tightly coupled to the client to proxy 
connection.  There was a deliberate decision not to reuse proxy-OS 
connections for requests coming from other client-proxy connections as this 
may be a security risk.

The OS may attribute authorization to a connection and a subsequent request 
on this persistent connection could inherit these attributes.  Each HTTP 
request *should* be stateless and hence the next request on the same socket 
should be independent, but there was the risk that a remote (non-Apache) 
origin server may not work that way.  If the proxy-OS connection is pooled 
and reused by a different client-proxy request, does that risk confusing an 
origin server that expects all requests on the same connection to come from 
the same client?

... or have I misunderstood your description?

Thanks,
Paul


RE: OS Keep-alive on forward proxy

2010-08-05 Thread Plüm, Rüdiger, VF-Group
 

 -Original Message-
 From: Paul Fee 
 Sent: Donnerstag, 5. August 2010 11:18
 To: dev@httpd.apache.org
 Subject: Re: OS Keep-alive on forward proxy
 
 Rainer Jung wrote:
 
 snip
 
  And yes: the forward proxy does *not* do HTTP Keepalive. Technical
  reason: the connections to the origin server are pooled and 
 retrieved
  from and returned to the pool for each request. A forward 
 proxy usually
  talks to many diferent origin servers. Keeping those 
 connections open in
  a naive way would lead to a lot of not well used pools. 
 Assuming that
  during one client connection the origin server often is used for
  multiple requests this could be improved, but would bloat 
 the already
  complicated proxy code even more.
 
 Has mod_proxy operated in that way for a while now?  I gained 

Since 2.2.

 most of my 
 experience with mod_proxy using Apache 2.0.X.  My 
 understanding was that 
 proxy to OS connections were tightly coupled to the client to proxy 

That was true in 2.0.x yes.

 connection.  There was a deliberate decision not to reuse proxy-OS 
 connections for requests coming from other client-proxy 
 connections as this 
 may be a security risk.
 
 The OS may attribute authorization to a connection and a 
 subsequent request 
 on this persistent connection could inherit these attributes. 
  Each HTTP 
 request *should* be stateless and hence the next request on 
 the same socket 
 should be independent, but there was the risk that a remote 
 (non-Apache) 
 origin server may not work that way.  If the proxy-OS 
 connection is pooled 
 and reused by a different client-proxy request, does that 
 risk confusing an 
 origin server that expects all requests on the same 
 connection to come from 
 the same client?

It would be a bug in this server to expect them to origin from the
same client as you correctly state that HTTP is a stateless protocol.
Nevertheless you can turn off connection pooling in the case you
are dealing with a faulty origin server.

Regards

Rüdiger



Changing Connection header use in forward proxy

2010-08-05 Thread Rainer Jung
It seems the forward proxy by default sets Connection: Keep-Alive 
although it later closes the connection when it detects 
is_address_reusable == 1 in the worker.


Doesn't it make sense to issue Connection close instead?

Since this happens in the code where we have a connection and a request, 
but no worker struct available, is it safe to apply this whenever 
proxyreq is PROXYREQ_PROXY? Or can one somehow configure a non-default 
forward worker with an explicit URL to e.g. a frequently used origin 
server which would then be expected to reuse connections? I don't expect 
that to be posible, but I'm not sure.


Regards,

Rainer





Re: Per request DocumentRoot

2010-08-05 Thread Ondřej Surý
On Thu, Aug 5, 2010 at 10:25, William A. Rowe Jr. wr...@rowe-clan.net wrote:
 On 8/5/2010 3:22 AM, Ondřej Surý wrote:
 William,

 On Thu, Aug 5, 2010 at 10:18, William A. Rowe Jr. wr...@rowe-clan.net 
 wrote:
 On 8/5/2010 3:12 AM, Ondřej Surý wrote:
 I have a question how to proceed. Is there really no interest in doing
 this? (And fixing mod-vhost-alias as well.)

 Or should I fill the bug and attach the patch? Or ...?

 Please don't confuse ignoring your patch with lack of interest.  Everyone
 here is a volunteer and dedicates the cycles they have free to attacking
 the most pressing issue they see at the time.

 Sorry if that offended you or anybody else. I didn't ment to sound
 harsh as I was just merely asking what to do now.

 You didn't and I didn't mean to sound irritated, either, just trying to frame
 what happens on this particular list for your info.

Ok :).

 Many 'nice to have' things for Apache 2.4 are ignored as more critical issues
 in 2.2 are addressed, but with rising interest in frequent alpha/betas, we can
 all hope that improves :)

I've filled this under #49705

https://issues.apache.org/bugzilla/show_bug.cgi?id=49705

Ondrej
-- 
Ondřej Surý ond...@sury.org


Re: Failing startup for vhost configuration problems

2010-08-05 Thread Graham Dumpleton
On Thursday, August 5, 2010, Niklas Edmundsson ni...@acc.umu.se wrote:
 On Thu, 5 Aug 2010, Graham Dumpleton wrote:


 On Thursday, August 5, 2010, Stefan Fritsch s...@sfritsch.de wrote:

 On Tuesday 03 August 2010, Dan Poirier wrote:

 I'd like to propose that in 2.3/2.4, we fail startup for any of the
 virtual host misconfigurations for which behavior is undefined but
 right now we only issue a warning.

 snip

 Perhaps warnings around bad MPM configuration can be reviewed as well.

 From memory it doesn't at the moment warn you when something like

 StartServers is set over what it can be based on MaxClients and
 ThreadsPerChild. I think it may just silently reduce the value. Since
 MPM settings are often misunderstood, better hints about bad
 configuration there would be useful.


 +1 on both suggestions. This would be a good time to change/improve on this 
 front.

Another thing which would be nice related to MPM settings and other
core settings is the ability to do:

  httpd -t -DDUMP_CORE_CONFIG

The idea being that in doing this it would log what all the MPM
related settings are, ie. The final result of taking user settings,
applying defaults where no user supplied settings, plus any
adjustments due to stupid settings. Add to that critical core settings
which come into play as to performance of a site such as Timeout,
KeepAlive, KeepAliveTimeout, ListenBacklog, EnableSendFile etc.

This way when trying to assist someone to understand whether their
configuration is appropriate one can get all the key important
settings easily and not have to get them to troll through their
configuration files, work out whether they are even being included,
adding defaults for things not set etc.

All up this would certainly make the job of helping newbies work out
how they screwed up there site by changing these settings in the first
place, a lot easier.

Graham


RE: Changing Connection header use in forward proxy

2010-08-05 Thread Plüm, Rüdiger, VF-Group
 

 -Original Message-
 From: Rainer Jung 
 Sent: Donnerstag, 5. August 2010 11:51
 To: dev@httpd.apache.org
 Subject: Changing Connection header use in forward proxy
 
 It seems the forward proxy by default sets Connection: Keep-Alive 
 although it later closes the connection when it detects 
 is_address_reusable == 1 in the worker.
 
 Doesn't it make sense to issue Connection close instead?
 
 Since this happens in the code where we have a connection and 
 a request, 
 but no worker struct available, is it safe to apply this whenever 
 proxyreq is PROXYREQ_PROXY? Or can one somehow configure a 
 non-default 
 forward worker with an explicit URL to e.g. a frequently used origin 
 server which would then be expected to reuse connections? I 
 don't expect 
 that to be posible, but I'm not sure.

I haven't tried so far, but IMHO it should be possible to setup
pooled connections for frequently used targets of a forward proxy
by adding

Proxy http://www.frequentlyused.com/
   # Set an arbitrary parameter to trigger the creation of a worker
   ProxySet keepalive=on
/Proxy

If I am correct with this assumption the we should not proceed the way
you proposed.

How about the following untested patch instead?

Index: modules/proxy/proxy_util.c
===
--- modules/proxy/proxy_util.c  (revision 982130)
+++ modules/proxy/proxy_util.c  (working copy)
@@ -1568,6 +1568,7 @@
 *balancer = NULL;
 *worker = conf-forward;
 access_status = OK;
+apr_table_set(r-subprocess_env, proxy-nokeepalive, 1);
 }
 }
 else if (r-proxyreq == PROXYREQ_REVERSE) {
@@ -1578,6 +1579,7 @@
 *balancer = NULL;
 *worker = conf-reverse;
 access_status = OK;
+apr_table_set(r-subprocess_env, proxy-nokeepalive, 1);
 }
 }
 }

Regards

Rüdiger



Rephrasing worker already used

2010-08-05 Thread Rainer Jung
The message worker ... already used by another worker seems to not say 
what's actually happening.


Situation:

ProxyPass /X http://a.b.c.d/A
ProxyPass /Y http://a.b.c.d/AB

Wouldn't a message

worker http://a.b.c.d/A; will also be used for URL /Y

be more correct?

In addition: we do overwrite the worker config (additional attributes) 
when parsing the second ProxyPass. Should we add a message


Merging configuration for worker http://a.b.c.d/A;

Finally: I guess we need to document the behaviour, especially that you 
have the option to reuse/overwrite or use separate workers, depending on 
the order of the ProxyPass statements.


Any opinion?

Regards,

Rainer



RE: Rephrasing worker already used

2010-08-05 Thread Plüm, Rüdiger, VF-Group
 

 -Original Message-
 From: Rainer Jung 
 Sent: Donnerstag, 5. August 2010 13:14
 To: dev@httpd.apache.org
 Subject: Rephrasing  worker already used
 
 The message worker ... already used by another worker seems 
 to not say 
 what's actually happening.
 
 Situation:
 
 ProxyPass /X http://a.b.c.d/A
 ProxyPass /Y http://a.b.c.d/AB
 
 Wouldn't a message
 
 worker http://a.b.c.d/A; will also be used for URL /Y
 
 be more correct?
 
 In addition: we do overwrite the worker config (additional 
 attributes) 
 when parsing the second ProxyPass. Should we add a message
 
 Merging configuration for worker http://a.b.c.d/A;

Actually it doesn't merge the configuration. For some parameters like
timeout they get overwritten with the defaults if not configured again in the
ProxyPass directive. So

ProxyPass /X http://a.b.c.d/A timeout=5
ProxyPass /Y http://a.b.c.d/AB

Will result in a worker http://a.b.c.d/A with timeout set to the default value
(ProxyTimeout / Timeout).

Regards

Rüdiger



RE: OS Keep-alive on forward proxy

2010-08-05 Thread Paul Fee
Plüm, Rüdiger, VF-Group wrote:

  
 
 -Original Message-
 From: Paul Fee
 Sent: Donnerstag, 5. August 2010 11:18
 To: dev@httpd.apache.org
 Subject: Re: OS Keep-alive on forward proxy
 
 Rainer Jung wrote:
 
 snip
 
  And yes: the forward proxy does *not* do HTTP Keepalive. Technical
  reason: the connections to the origin server are pooled and
 retrieved
  from and returned to the pool for each request. A forward
 proxy usually
  talks to many diferent origin servers. Keeping those
 connections open in
  a naive way would lead to a lot of not well used pools.
 Assuming that
  during one client connection the origin server often is used for
  multiple requests this could be improved, but would bloat
 the already
  complicated proxy code even more.
 
 Has mod_proxy operated in that way for a while now?  I gained
 
 Since 2.2.
 
 most of my
 experience with mod_proxy using Apache 2.0.X.  My
 understanding was that
 proxy to OS connections were tightly coupled to the client to proxy
 
 That was true in 2.0.x yes.
 
 connection.  There was a deliberate decision not to reuse proxy-OS
 connections for requests coming from other client-proxy
 connections as this
 may be a security risk.
 
 The OS may attribute authorization to a connection and a
 subsequent request
 on this persistent connection could inherit these attributes.
  Each HTTP
 request *should* be stateless and hence the next request on
 the same socket
 should be independent, but there was the risk that a remote
 (non-Apache)
 origin server may not work that way.  If the proxy-OS
 connection is pooled
 and reused by a different client-proxy request, does that
 risk confusing an
 origin server that expects all requests on the same
 connection to come from
 the same client?
 
 It would be a bug in this server to expect them to origin from the
 same client as you correctly state that HTTP is a stateless protocol.
 Nevertheless you can turn off connection pooling in the case you
 are dealing with a faulty origin server.
 
 Regards
 
 Rüdiger

That's useful information, it's not mentioned in the overview of new 
features in 2.2 and I missed it in the detailed changelog.  Thanks for 
correcting my misunderstanding.

Regarding disabling connection pooling, I looked at the source and see two 
ways to achieve this:

1) The disablereuse parameter of the ProxyPass directive.
2) The proxy-initial-not-pooled Apache environment variable set on a per-
request basis.

Both these relate to reverse proxy requests.

Does connection pooling apply to forward proxy requests?  If so, are there 
configuration options to control it?

Would disabling connection pooling fix the defect that Ryujiro Shibuya 
reported (with the penalty of losing the performance gains for pooling)?

Thanks,
Paul


RE: OS Keep-alive on forward proxy

2010-08-05 Thread Plüm, Rüdiger, VF-Group
 

 -Original Message-
 From: Paul Fee 
 Sent: Donnerstag, 5. August 2010 17:32
 To: dev@httpd.apache.org
 Subject: RE: OS Keep-alive on forward proxy
 

 Does connection pooling apply to forward proxy requests?  If 
 so, are there 
 configuration options to control it?

Forward proxies do no connection pooling. At least not by default,
see the parallel thread fro details.

Regards

Rüdiger



Re: OS Keep-alive on forward proxy

2010-08-05 Thread Rainer Jung

In adition to Rüdiger's reply:

On 05.08.2010 17:31, Paul Fee wrote:

Plüm, Rüdiger, VF-Group wrote:

-Original Message-
From: Paul Fee
Sent: Donnerstag, 5. August 2010 11:18
To: dev@httpd.apache.org
Subject: Re: OS Keep-alive on forward proxy

Rainer Jung wrote:

snip


And yes: the forward proxy does *not* do HTTP Keepalive. Technical
reason: the connections to the origin server are pooled and

retrieved

from and returned to the pool for each request. A forward

proxy usually

talks to many diferent origin servers. Keeping those

connections open in

a naive way would lead to a lot of not well used pools.

Assuming that

during one client connection the origin server often is used for
multiple requests this could be improved, but would bloat

the already

complicated proxy code even more.


Has mod_proxy operated in that way for a while now?  I gained


Since 2.2.


most of my
experience with mod_proxy using Apache 2.0.X.  My
understanding was that
proxy to OS connections were tightly coupled to the client to proxy


That was true in 2.0.x yes.


connection.  There was a deliberate decision not to reuse proxy-OS
connections for requests coming from other client-proxy
connections as this
may be a security risk.

The OS may attribute authorization to a connection and a
subsequent request
on this persistent connection could inherit these attributes.
  Each HTTP
request *should* be stateless and hence the next request on
the same socket
should be independent, but there was the risk that a remote
(non-Apache)
origin server may not work that way.  If the proxy-OS
connection is pooled
and reused by a different client-proxy request, does that
risk confusing an
origin server that expects all requests on the same
connection to come from
the same client?


It would be a bug in this server to expect them to origin from the
same client as you correctly state that HTTP is a stateless protocol.
Nevertheless you can turn off connection pooling in the case you
are dealing with a faulty origin server.

Regards

Rüdiger


That's useful information, it's not mentioned in the overview of new
features in 2.2 and I missed it in the detailed changelog.  Thanks for
correcting my misunderstanding.

Regarding disabling connection pooling, I looked at the source and see two
ways to achieve this:

1) The disablereuse parameter of the ProxyPass directive.
2) The proxy-initial-not-pooled Apache environment variable set on a per-
request basis.

Both these relate to reverse proxy requests.

Does connection pooling apply to forward proxy requests?  If so, are there
configuration options to control it?

Would disabling connection pooling fix the defect that Ryujiro Shibuya
reported (with the penalty of losing the performance gains for pooling)?


The default worker for forward proxying does not use connection pooling 
in the naive sense. It closes each connection after each request.


What Ryujiro Shibuya was observing was that Apache signals it would keep 
the connection open even in forward proxy mode, but then acually closes 
the connection. We are discussing a fix to this, namely always 
signalling Connection: close from the beginning for the default 
forward and revere proxy workers.


You can define explicit workers though (e.g. using ProxyPass for reverse 
and as Rüdiger wrote likely also in forward proxy mode, which then will 
use HTTP Keep-Alive (by default, depending on several config options).


I just started to scribble an addition to the mod_proxy docs to explain 
a bit more about the concept of workers. Let's see how far this gets.


Regards,

Rainer


_default_ and * in vhost hostnames

2010-08-05 Thread Dan Poirier
Suppose we just make _default_ a synonym of * in virtual host hostnames?

Eric brought this up a while back, with little response at the time.

Right now, you can only use * in the hostname of a name-based virtual
host, and only use _default_ in the hostname of an IP-based virtual
host.

Their meanings are similar already, and any subtle differences in how
we've defined them aren't really relevant, since you can't use both in
the same virtual host anyway, so there's never any need to distinguish
them.

So, I'm proposing we simplify the code, and documentation, by making
_default_ just another name for *, and maybe if we see _default_,
issuing a warning that its use is deprecated.

Thoughts?

Dan


Re: Changing Connection header use in forward proxy

2010-08-05 Thread Rainer Jung

On 05.08.2010 13:03, Plüm, Rüdiger, VF-Group wrote:

-Original Message-
From: Rainer Jung
Sent: Donnerstag, 5. August 2010 11:51
To: dev@httpd.apache.org
Subject: Changing Connection header use in forward proxy

It seems the forward proxy by default sets Connection: Keep-Alive
although it later closes the connection when it detects
is_address_reusable == 1 in the worker.

Doesn't it make sense to issue Connection close instead?

Since this happens in the code where we have a connection and
a request,
but no worker struct available, is it safe to apply this whenever
proxyreq is PROXYREQ_PROXY? Or can one somehow configure a
non-default
forward worker with an explicit URL to e.g. a frequently used origin
server which would then be expected to reuse connections? I
don't expect
that to be posible, but I'm not sure.


I haven't tried so far, but IMHO it should be possible to setup
pooled connections for frequently used targets of a forward proxy
by adding

Proxy http://www.frequentlyused.com/
# Set an arbitrary parameter to trigger the creation of a worker
ProxySet keepalive=on
/Proxy


Actually after staring at the code for a while IMHO there is no 
difference between a forward or a reverse worker. There's only a default 
forward and a default reverse worker and then additional workers as 
configured.


It does make a difference whether a request is handled in forward or 
reverse mode, but that is independent of workers.


You can define a worker originally meant for reverse use like in

   ProxyPass /something http://www.origin.com

and allow forward mode

   ProxyRequests On

If you then send

   GET http://www.origin.com/ HTTP/1.0

your request will be handled by the worker defined through the above 
ProxyPass (including pooling etc.). So forward worker or reverse 
worker only makes sense for those builtin default workers (which are 
kind of fallback and not configurable).



If I am correct with this assumption the we should not proceed the way
you proposed.

How about the following untested patch instead?

Index: modules/proxy/proxy_util.c
===
--- modules/proxy/proxy_util.c  (revision 982130)
+++ modules/proxy/proxy_util.c  (working copy)
@@ -1568,6 +1568,7 @@
  *balancer = NULL;
  *worker = conf-forward;
  access_status = OK;
+apr_table_set(r-subprocess_env, proxy-nokeepalive, 1);
  }
  }
  else if (r-proxyreq == PROXYREQ_REVERSE) {
@@ -1578,6 +1579,7 @@
  *balancer = NULL;
  *worker = conf-reverse;
  access_status = OK;
+apr_table_set(r-subprocess_env, proxy-nokeepalive, 1);
  }
  }
  }


Thanks, looks good, will test.

Regards,

Rainer


Talking about proxy workers

2010-08-05 Thread Rainer Jung
I created a patch for the mod_proxy docs to clarify what a worker is. 
The old part of the docs never talk about workers and then later down 
when the 2.2 attributes are described we suddenly start using that 
terminology.


I got also confused by the worker sharing (worker already used by 
worker) as well as by the ordering issues for ProxyPass so I added all 
this to the following docs patch:


http://people.apache.org/~rjung/patches/mod_proxy_docs_workers.patch

I think the additions are correct and helpful, but it would be nice if 
one of our proxy specialists could take a quick view at it before I 
commit. I hope the addition is not to confusing.


Actually looking at the generated HTML page is nicer, so you can find it 
here:


http://people.apache.org/~rjung/httpd/trunk/manual/mod/mod_proxy.html.en

The new things are the section Workers

http://people.apache.org/~rjung/httpd/trunk/manual/mod/mod_proxy.html.en#workers

and the extended warning about ordering in

http://people.apache.org/~rjung/httpd/trunk/manual/mod/mod_proxy.html.en#proxypass

IMHO the same docs addition would be good for 2.2.

Regards,

Rainer


Re: Talking about proxy workers

2010-08-05 Thread Eric Covener

 http://people.apache.org/~rjung/httpd/trunk/manual/mod/mod_proxy.html.en#workers

A direct worker is usually configured using any of ProxyPass,
ProxyPassMatch or ProxySet.

I don't know much about Proxy, but can this hammer home a bit more
that these directives create a new worker implicitly based on [some
parts of?] the destination URL?  And what happens when there's
overlap?

-- 
Eric Covener
cove...@gmail.com


Re: _default_ and * in vhost hostnames

2010-08-05 Thread Eric Covener
On Thu, Aug 5, 2010 at 1:00 PM, Dan Poirier poir...@pobox.com wrote:
 Suppose we just make _default_ a synonym of * in virtual host hostnames?

+1, I still never found a practical purpose for two wildcards.

I tend to think of having only one kind of vhost that undergoes
ip-based selection, then optionally further selection based on name,
if the best addr:port match landed on a string that matches a
configured NVH directive.

The only config I am aware of broken by this is something degenerate
with NVH _default_:80  and overlapping *:80 or _default_:80 where
all but the first vhost are dead code today but wouldn't be with NVH
*:80

-- 
Eric Covener
cove...@gmail.com