RE: Talking about proxy workers

2010-08-11 Thread Ryujiro Shibuya
 -Original Message-
 From: Paul Fee [mailto:p...@talk21.com]
 Sent: Tuesday, August 10, 2010 9:18 PM
 To: dev@httpd.apache.org
 Subject: Re: Talking about proxy workers
 
 Paul Fee wrote:
 
  Rainer Jung wrote:
 
  Minor additions inside.
 
  On 06.08.2010 14:49, Plüm, Rüdiger, VF-Group wrote:
 
 
  -Original Message-
  From: Paul Fee
  Sent: Freitag, 6. August 2010 14:44
  To: dev@httpd.apache.org
  Subject: Re: Talking about proxy workers
 
 
  Also, is it possible to setup these three reuse styles for a
  forward proxy?
 
  1: No reuse, close the connection after this request.
 
  Yes, this the default.
 
  2: Reuse connection, but only for the client that caused its creation.
 
  No.
 
  Even if you configure pooled connections like in the example given in
 3,
  the connections are returned to the pool after each request/response
  cycle. They are not directly associated with the client connection.
 
  But: if the MPM is prefork, the client connection is handled by a single
  process which doesn't handle any other requests during the life of the
  client connection. Since pools are process local, in this case the pool
  will always return the same connection (the only connection in the
  pool). Note that this pooled connection will not be closed when the
  client connection is closed. It can live longer or shorter than the
  client connection and you can't tie their lifetime together.
 
  Whether the proxy operates in forward or reverse mode doesn't matter,
 it
  only matters how the pool aka worker is configured. See 3.
 
  3: Pool connection for reuse by any client.
 
  Yes, but this is needed separately for every origin server you forward
  to:
 
  Proxy http://www.frequentlyused.com/
  # Set an arbitrary parameter to trigger the creation of a worker
  ProxySet keepalive=on
  /Proxy
 
  Pools are associated with workers, and workers are identified by origin
  URL. In case of a reverse proxy you often only have a few origin
  servers, so pooling works fine. In case of a forward proxy you often
  have an enormous amount of origin servers, each only visited every now
  and then. So using persistent connections is less effective. It would
  only make some sense, if we could optionally tie together client
  connections and origin server connections.
 
  Regards,
 
  Rainer
 
  I'm using the worker MPM, so connection sharing between clients can
  happen.
 
  As you've pointed out, pooling works well for reverse proxies as there
 are
  few backends and the hit rate is high.  For forward proxies, there are
  numerous destinations and the pool hit rate will be low.  The pool has
 a
  cost due to multi-threaded access to a single data structure, I presume
  locks protect the connection pool.  Locks can limit scalability.
 
  I'm wondering if pools should be restricted to the reverse proxy case.
  Forward proxies would couple the proxy-origin server connection to the
  client side connection.  Since connections can not be shared, there's
 no
  need for locking.  We'd loss the opportunity to share, but since the
  probability of a pool hit by another client is low, that loss should be
  acceptable.
 
  Essentially, I'm asking if it would make sense to implement 2: Reuse
  connection, but only for the client that caused its creation.  This could
  be a configurable proxy worker setting.
 
  Thanks,
  Paul
 
 Here's a suggestion to refine connection pooling for forward proxies.
 
 Can a connection to an origin server be tightly coupled to the client
 connection for the lifetime of the client connection?  Then, when the client
 connection closes, the origin server connection can be placed in the pool
 for possible reuse by another incoming client connection.
 
 This would allow a client to reuse its own origin server connection without
 having to do a lookup in the pool.  We'd save on lookup costs and pool
 locking.  However connections would still go into the pool after the client
 has finished with it, for the potential benefit of other clients.
 
 When a client makes a request, mod_proxy looks to see if there's an existing
 origin server connection coupled to the request_rec.  If there's no
 connection or the connection is not to the correct origin server, then
 perform a pool lookup.  If that fails, then create a fresh connection.
 
 Does this should feasible?  Can we do this with mod_proxy already?  Is it
 worth implementing?
 
 Thanks,
 Paul

Current implementation seems to close the origin server connection every time 
even if the client connection is kept opened for the next client request 
(which is highly possibly for the same origin server).

I agree with Paul's idea, which seems a good way to enable both of ...
  (1) keeping origin server connection opened for exclusive use of a client,
  (2) connection pooling after use.
In the forward proxy, in my opinion, the advantage of (1) cannot be ignored, 
even though some might think the advantage of (2) may be limited than

Re: Talking about proxy workers

2010-08-11 Thread Jim Jagielski

On Aug 11, 2010, at 5:24 AM, Ryujiro Shibuya wrote:
 
 Current implementation seems to close the origin server connection every time 
 even if the client connection is kept opened for the next client request 
 (which is highly possibly for the same origin server).
 
 I agree with Paul's idea, which seems a good way to enable both of ...
  (1) keeping origin server connection opened for exclusive use of a client,
  (2) connection pooling after use.
 In the forward proxy, in my opinion, the advantage of (1) cannot be ignored, 
 even though some might think the advantage of (2) may be limited than that in 
 the reverse proxy.
 

It would be most ideal if we pulled our forward and reverse
proxies as sep modules instead of a single one, but that's
an exercise for post 2.4 I think...



Re: Talking about proxy workers

2010-08-10 Thread Paul Fee
Paul Fee wrote:

 Rainer Jung wrote:
 
 Minor additions inside.
 
 On 06.08.2010 14:49, Plüm, Rüdiger, VF-Group wrote:


 -Original Message-
 From: Paul Fee
 Sent: Freitag, 6. August 2010 14:44
 To: dev@httpd.apache.org
 Subject: Re: Talking about proxy workers


 Also, is it possible to setup these three reuse styles for a
 forward proxy?

 1: No reuse, close the connection after this request.

 Yes, this the default.

 2: Reuse connection, but only for the client that caused its creation.

 No.
 
 Even if you configure pooled connections like in the example given in 3,
 the connections are returned to the pool after each request/response
 cycle. They are not directly associated with the client connection.
 
 But: if the MPM is prefork, the client connection is handled by a single
 process which doesn't handle any other requests during the life of the
 client connection. Since pools are process local, in this case the pool
 will always return the same connection (the only connection in the
 pool). Note that this pooled connection will not be closed when the
 client connection is closed. It can live longer or shorter than the
 client connection and you can't tie their lifetime together.
 
 Whether the proxy operates in forward or reverse mode doesn't matter, it
 only matters how the pool aka worker is configured. See 3.
 
 3: Pool connection for reuse by any client.

 Yes, but this is needed separately for every origin server you forward
 to:

 Proxy http://www.frequentlyused.com/
 # Set an arbitrary parameter to trigger the creation of a worker
 ProxySet keepalive=on
 /Proxy
 
 Pools are associated with workers, and workers are identified by origin
 URL. In case of a reverse proxy you often only have a few origin
 servers, so pooling works fine. In case of a forward proxy you often
 have an enormous amount of origin servers, each only visited every now
 and then. So using persistent connections is less effective. It would
 only make some sense, if we could optionally tie together client
 connections and origin server connections.
 
 Regards,
 
 Rainer
 
 I'm using the worker MPM, so connection sharing between clients can
 happen.
 
 As you've pointed out, pooling works well for reverse proxies as there are
 few backends and the hit rate is high.  For forward proxies, there are
 numerous destinations and the pool hit rate will be low.  The pool has a
 cost due to multi-threaded access to a single data structure, I presume
 locks protect the connection pool.  Locks can limit scalability.
 
 I'm wondering if pools should be restricted to the reverse proxy case.
 Forward proxies would couple the proxy-origin server connection to the
 client side connection.  Since connections can not be shared, there's no
 need for locking.  We'd loss the opportunity to share, but since the
 probability of a pool hit by another client is low, that loss should be
 acceptable.
 
 Essentially, I'm asking if it would make sense to implement 2: Reuse
 connection, but only for the client that caused its creation.  This could
 be a configurable proxy worker setting.
 
 Thanks,
 Paul

Here's a suggestion to refine connection pooling for forward proxies.

Can a connection to an origin server be tightly coupled to the client 
connection for the lifetime of the client connection?  Then, when the client 
connection closes, the origin server connection can be placed in the pool 
for possible reuse by another incoming client connection.

This would allow a client to reuse its own origin server connection without 
having to do a lookup in the pool.  We'd save on lookup costs and pool 
locking.  However connections would still go into the pool after the client 
has finished with it, for the potential benefit of other clients.

When a client makes a request, mod_proxy looks to see if there's an existing 
origin server connection coupled to the request_rec.  If there's no 
connection or the connection is not to the correct origin server, then 
perform a pool lookup.  If that fails, then create a fresh connection.

Does this should feasible?  Can we do this with mod_proxy already?  Is it 
worth implementing?

Thanks,
Paul


Re: Talking about proxy workers

2010-08-09 Thread Paul Fee
Rainer Jung wrote:

 Minor additions inside.
 
 On 06.08.2010 14:49, Plüm, Rüdiger, VF-Group wrote:


 -Original Message-
 From: Paul Fee
 Sent: Freitag, 6. August 2010 14:44
 To: dev@httpd.apache.org
 Subject: Re: Talking about proxy workers


 Also, is it possible to setup these three reuse styles for a
 forward proxy?

 1: No reuse, close the connection after this request.

 Yes, this the default.

 2: Reuse connection, but only for the client that caused its creation.

 No.
 
 Even if you configure pooled connections like in the example given in 3,
 the connections are returned to the pool after each request/response
 cycle. They are not directly associated with the client connection.
 
 But: if the MPM is prefork, the client connection is handled by a single
 process which doesn't handle any other requests during the life of the
 client connection. Since pools are process local, in this case the pool
 will always return the same connection (the only connection in the
 pool). Note that this pooled connection will not be closed when the
 client connection is closed. It can live longer or shorter than the
 client connection and you can't tie their lifetime together.
 
 Whether the proxy operates in forward or reverse mode doesn't matter, it
 only matters how the pool aka worker is configured. See 3.
 
 3: Pool connection for reuse by any client.

 Yes, but this is needed separately for every origin server you forward
 to:

 Proxy http://www.frequentlyused.com/
 # Set an arbitrary parameter to trigger the creation of a worker
 ProxySet keepalive=on
 /Proxy
 
 Pools are associated with workers, and workers are identified by origin
 URL. In case of a reverse proxy you often only have a few origin
 servers, so pooling works fine. In case of a forward proxy you often
 have an enormous amount of origin servers, each only visited every now
 and then. So using persistent connections is less effective. It would
 only make some sense, if we could optionally tie together client
 connections and origin server connections.
 
 Regards,
 
 Rainer

I'm using the worker MPM, so connection sharing between clients can happen.

As you've pointed out, pooling works well for reverse proxies as there are 
few backends and the hit rate is high.  For forward proxies, there are 
numerous destinations and the pool hit rate will be low.  The pool has a 
cost due to multi-threaded access to a single data structure, I presume 
locks protect the connection pool.  Locks can limit scalability.

I'm wondering if pools should be restricted to the reverse proxy case.  
Forward proxies would couple the proxy-origin server connection to the 
client side connection.  Since connections can not be shared, there's no 
need for locking.  We'd loss the opportunity to share, but since the 
probability of a pool hit by another client is low, that loss should be 
acceptable.

Essentially, I'm asking if it would make sense to implement 2: Reuse 
connection, but only for the client that caused its creation.  This could 
be a configurable proxy worker setting.

Thanks,
Paul


Re: Talking about proxy workers

2010-08-06 Thread Rainer Jung

On 05.08.2010 21:30, Eric Covener wrote:


http://people.apache.org/~rjung/httpd/trunk/manual/mod/mod_proxy.html.en#workers


A direct worker is usually configured using any of ProxyPass,
ProxyPassMatch or ProxySet.



I don't know much about Proxy, but can this hammer home a bit more
that these directives create a new worker implicitly based on [some
parts of?] the destination URL?


Good point. I updated the patch and HTML page to stress the 
identification of workers by their URL.



And what happens when there's
overlap?


There's a warning box at the end of the Workers section talking about 
that. I slightly rephrased it to also contain the trm overlap.


New patch:

http://people.apache.org/~rjung/patches/mod_proxy_docs_workers-v2.patch

HTML-page updated in place.

Thanks!

Regards,

Rainer


Re: Talking about proxy workers

2010-08-06 Thread Eric Covener
On Fri, Aug 6, 2010 at 3:54 AM, Rainer Jung rainer.j...@kippdata.de wrote:
 On 05.08.2010 21:30, Eric Covener wrote:


 http://people.apache.org/~rjung/httpd/trunk/manual/mod/mod_proxy.html.en#workers

 A direct worker is usually configured using any of ProxyPass,
 ProxyPassMatch or ProxySet.

 I don't know much about Proxy, but can this hammer home a bit more
 that these directives create a new worker implicitly based on [some
 parts of?] the destination URL?

 Good point. I updated the patch and HTML page to stress the identification
 of workers by their URL.

 And what happens when there's
 overlap?

 There's a warning box at the end of the Workers section talking about that.
 I slightly rephrased it to also contain the trm overlap.

+1, the additions are helpful to this proxy laymen!

-- 
Eric Covener
cove...@gmail.com


Re: Talking about proxy workers

2010-08-06 Thread Jeff Trawick
On Fri, Aug 6, 2010 at 3:54 AM, Rainer Jung rainer.j...@kippdata.de wrote:
 On 05.08.2010 21:30, Eric Covener wrote:


 http://people.apache.org/~rjung/httpd/trunk/manual/mod/mod_proxy.html.en#workers

 A direct worker is usually configured using any of ProxyPass,
 ProxyPassMatch or ProxySet.

 I don't know much about Proxy, but can this hammer home a bit more
 that these directives create a new worker implicitly based on [some
 parts of?] the destination URL?

 Good point. I updated the patch and HTML page to stress the identification
 of workers by their URL.

 And what happens when there's
 overlap?

 There's a warning box at the end of the Workers section talking about that.
 I slightly rephrased it to also contain the trm overlap.

 New patch:

 http://people.apache.org/~rjung/patches/mod_proxy_docs_workers-v2.patch

nits:

+  There are two builtin workers, the default forward proxy worker and the

built-in

+  optionally included in a directive module=mod_proxyProxy/directive
+  directive.

How about using container at the end instead of directive? (shrug)

+  . Direct workers can use connection pooling,
+  HTTP Keep-Alive and individual configurations for example
+  for timeouts.

(dumb question: what's the diff between keepalive and connection
pooling?  reuse for one client vs. reuse for any client?)

That last part sounds a little awkward.  Maybe something like this:

A number of processing options can be specified for direct workers,
including connection pooling, HTTP Keep-Alive, and I/O timeout values.

+   Which options are available is depending on the
+  protocol used by the worker (and given in the origin server URL).
+  Available protocols include codeajp/code, codefcgi/code,
+  codeftp/code, codehttp/code and codescgi/code./p
+

The set of options available for the worker depends on the protocol,
which is specified in
the origin server URL.  Available protocols include 


+  pA balancer worker is created, if its worker URL uses

no comma

+  pWorker sharing happens, if the worker URLs overlap. More precisely
+  if the URL of some worker is a leading substring of the URL of another
+  worker defined later in the configuration file.

pWorker sharing happens if the worker URLs overlap, which occurs when
the URL of some worker is a leading substring of the URL of another
worker defined later in the configuration file.

+  In this case the later worker isn't actually created. Instead
the previous
+  worker is used. The benefit is, that there is only one connection pool,
+  so connections are more often reused. Unfortunately all the
configuration attributes
+  given explicitly for the later worker overwrite the respective
configuration
+  of the previous worker!/p


This sounds like a discussion of pros and cons.  There's no pro, since
the user didn't intend to configure it this way, right?


Re: Talking about proxy workers

2010-08-06 Thread Mark Watts
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 06/08/10 12:13, Jeff Trawick wrote:
 On Fri, Aug 6, 2010 at 3:54 AM, Rainer Jung rainer.j...@kippdata.de wrote:
 On 05.08.2010 21:30, Eric Covener wrote:


 http://people.apache.org/~rjung/httpd/trunk/manual/mod/mod_proxy.html.en#workers

 A direct worker is usually configured using any of ProxyPass,
 ProxyPassMatch or ProxySet.

 I don't know much about Proxy, but can this hammer home a bit more
 that these directives create a new worker implicitly based on [some
 parts of?] the destination URL?

 Good point. I updated the patch and HTML page to stress the identification
 of workers by their URL.

 And what happens when there's
 overlap?

 There's a warning box at the end of the Workers section talking about that.
 I slightly rephrased it to also contain the trm overlap.

 New patch:

 http://people.apache.org/~rjung/patches/mod_proxy_docs_workers-v2.patch
 
 nits:
 
 +  There are two builtin workers, the default forward proxy worker and the
 
 built-in
 
 +  optionally included in a directive 
 module=mod_proxyProxy/directive
 +  directive.
 
 How about using container at the end instead of directive? (shrug)
 
 +  . Direct workers can use connection pooling,
 +  HTTP Keep-Alive and individual configurations for example
 +  for timeouts.
 
 (dumb question: what's the diff between keepalive and connection
 pooling?  reuse for one client vs. reuse for any client?)
 
 That last part sounds a little awkward.  Maybe something like this:
 
 A number of processing options can be specified for direct workers,
 including connection pooling, HTTP Keep-Alive, and I/O timeout values.
 
 +   Which options are available is depending on the
 +  protocol used by the worker (and given in the origin server URL).
 +  Available protocols include codeajp/code, codefcgi/code,
 +  codeftp/code, codehttp/code and codescgi/code./p
 +
 
 The set of options available for the worker depends on the protocol,
 which is specified in
 the origin server URL.  Available protocols include 
 
 
 +  pA balancer worker is created, if its worker URL uses
 
 no comma
 
 +  pWorker sharing happens, if the worker URLs overlap. More precisely
 +  if the URL of some worker is a leading substring of the URL of another
 +  worker defined later in the configuration file.
 
 pWorker sharing happens if the worker URLs overlap, which occurs when
 the URL of some worker is a leading substring of the URL of another
 worker defined later in the configuration file.
 
 +  In this case the later worker isn't actually created. Instead
 the previous
 +  worker is used. The benefit is, that there is only one connection pool,
 +  so connections are more often reused. Unfortunately all the
 configuration attributes
 +  given explicitly for the later worker overwrite the respective
 configuration
 +  of the previous worker!/p
 
 
 This sounds like a discussion of pros and cons.  There's no pro, since
 the user didn't intend to configure it this way, right?


Can we have some examples put in that section - its a little wordy and I
found it a little hard to understand, and I use mod_proxy quite a lot!

Mark

- -- 
Mark Watts BSc RHCE MBCS
Senior Systems Engineer, IPR Secure Managed Hosting
www.QinetiQ.com
QinetiQ - Delivering customer-focused solutions
GPG Key: http://www.linux-corner.info/mwatts.gpg
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iEYEARECAAYFAkxb+FIACgkQBn4EFUVUIO2c8gCg1Qd2zkUxtvzMRAi1FrccaK0+
UXcAoOR/Nmle+KN0/PaIKTMGMo6DJkf1
=KyEK
-END PGP SIGNATURE-


Re: Talking about proxy workers

2010-08-06 Thread Paul Fee
Mark Watts wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On 06/08/10 12:13, Jeff Trawick wrote:
 On Fri, Aug 6, 2010 at 3:54 AM, Rainer Jung rainer.j...@kippdata.de
 wrote:
 On 05.08.2010 21:30, Eric Covener wrote:


 
http://people.apache.org/~rjung/httpd/trunk/manual/mod/mod_proxy.html.en#workers

 A direct worker is usually configured using any of ProxyPass,
 ProxyPassMatch or ProxySet.

 I don't know much about Proxy, but can this hammer home a bit more
 that these directives create a new worker implicitly based on [some
 parts of?] the destination URL?

 Good point. I updated the patch and HTML page to stress the
 identification of workers by their URL.

 And what happens when there's
 overlap?

 There's a warning box at the end of the Workers section talking about
 that. I slightly rephrased it to also contain the trm overlap.

 New patch:

 http://people.apache.org/~rjung/patches/mod_proxy_docs_workers-v2.patch
 
 nits:
 
 +  There are two builtin workers, the default forward proxy worker
 and the
 
 built-in
 
 +  optionally included in a directive
 module=mod_proxyProxy/directive
 +  directive.
 
 How about using container at the end instead of directive? (shrug)
 
 +  . Direct workers can use connection pooling,
 +  HTTP Keep-Alive and individual configurations for example
 +  for timeouts.
 
 (dumb question: what's the diff between keepalive and connection
 pooling?  reuse for one client vs. reuse for any client?)
 
 That last part sounds a little awkward.  Maybe something like this:
 
 A number of processing options can be specified for direct workers,
 including connection pooling, HTTP Keep-Alive, and I/O timeout values.
 
 +   Which options are available is depending on the
 +  protocol used by the worker (and given in the origin server URL).
 +  Available protocols include codeajp/code, codefcgi/code,
 +  codeftp/code, codehttp/code and codescgi/code./p
 +
 
 The set of options available for the worker depends on the protocol,
 which is specified in
 the origin server URL.  Available protocols include 
 
 
 +  pA balancer worker is created, if its worker URL uses
 
 no comma
 
 +  pWorker sharing happens, if the worker URLs overlap. More
 precisely
 +  if the URL of some worker is a leading substring of the URL of
 another
 +  worker defined later in the configuration file.
 
 pWorker sharing happens if the worker URLs overlap, which occurs when
 the URL of some worker is a leading substring of the URL of another
 worker defined later in the configuration file.
 
 +  In this case the later worker isn't actually created. Instead
 the previous
 +  worker is used. The benefit is, that there is only one connection
 pool,
 +  so connections are more often reused. Unfortunately all the
 configuration attributes
 +  given explicitly for the later worker overwrite the respective
 configuration
 +  of the previous worker!/p
 
 
 This sounds like a discussion of pros and cons.  There's no pro, since
 the user didn't intend to configure it this way, right?
 
 
 Can we have some examples put in that section - its a little wordy and I
 found it a little hard to understand, and I use mod_proxy quite a lot!
 
 Mark

Can I request an example of how to setup a worker for a forward proxy.  
Directives such as ProxyPass are for reverse proxies.  Can parameters such 
as disablereuse be used with a proxy block?

Also, is it possible to setup these three reuse styles for a forward proxy?

1: No reuse, close the connection after this request.
2: Reuse connection, but only for the client that caused its creation.
3: Pool connection for reuse by any client.

Could you provide example configurations for these please?

Thanks,
Paul


RE: Talking about proxy workers

2010-08-06 Thread Plüm, Rüdiger, VF-Group
 

 -Original Message-
 From: Paul Fee 
 Sent: Freitag, 6. August 2010 14:44
 To: dev@httpd.apache.org
 Subject: Re: Talking about proxy workers

 
 Also, is it possible to setup these three reuse styles for a 
 forward proxy?
 
 1: No reuse, close the connection after this request.

Yes, this the default.

 2: Reuse connection, but only for the client that caused its creation.

No.

 3: Pool connection for reuse by any client.

Yes, but this is needed separately for every origin server you forward to:

Proxy http://www.frequentlyused.com/
   # Set an arbitrary parameter to trigger the creation of a worker
   ProxySet keepalive=on
/Proxy

Regards

Rüdiger



RE: Talking about proxy workers

2010-08-06 Thread Paul Fee
Plüm, Rüdiger, VF-Group wrote:
 3: Pool connection for reuse by any client.
 
 Yes, but this is needed separately for every origin server you forward to:
 
 Proxy http://www.frequentlyused.com/
# Set an arbitrary parameter to trigger the creation of a worker
ProxySet keepalive=on
 /Proxy

Can I use wildcards to enable connection pooling for all forward proxy 
destinations?

Proxy *
   ProxySet keepalive=on
/Proxy

Thanks,
Paul


Re: Talking about proxy workers

2010-08-06 Thread Rainer Jung

On 06.08.2010 13:13, Jeff Trawick wrote:

nits:

+  There are two builtin workers, the default forward proxy worker and the

built-in


Done


+  optionally included in adirective module=mod_proxyProxy/directive
+  directive.

How about using container at the end instead of directive? (shrug)


Hah! I was actually thinking about that, but thought I had used 
contained instead of included. But I wasn't, so changed to container.



+  . Direct workers can use connection pooling,
+  HTTP Keep-Alive and individual configurations for example
+  for timeouts.

(dumb question: what's the diff between keepalive and connection
pooling?  reuse for one client vs. reuse for any client?)


Good point. I think we do not have a concept of reuse for the same 
client. Connections are always returned to the pool after each 
request/response cycle. Even if a connection doesn't use keepalive, it 
can be pooled. I think the object in the pool is a bit more than just 
the TCP connection, but I didn't really investigate. There's room for 
improvement here but I was to lame to think more about it right now.



That last part sounds a little awkward.  Maybe something like this:

A number of processing options can be specified for direct workers,
including connection pooling, HTTP Keep-Alive, and I/O timeout values.


Done.


+   Which options are available is depending on the
+  protocol used by the worker (and given in the origin server URL).
+  Available protocols includecodeajp/code,codefcgi/code,
+codeftp/code,codehttp/code  andcodescgi/code./p
+

The set of options available for the worker depends on the protocol,
which is specified in
the origin server URL.  Available protocols include 


Done.


+pA balancer worker is created, if its worker URL uses

no comma


OK


+pWorker sharing happens, if the worker URLs overlap. More precisely
+  if the URL of some worker is a leading substring of the URL of another
+  worker defined later in the configuration file.

pWorker sharing happens if the worker URLs overlap, which occurs when
the URL of some worker is a leading substring of the URL of another
worker defined later in the configuration file.


Done


+  In this case the later worker isn't actually created. Instead
the previous
+  worker is used. The benefit is, that there is only one connection pool,
+  so connections are more often reused. Unfortunately all the
configuration attributes
+  given explicitly for the later worker overwrite the respective
configuration
+  of the previous worker!/p


This sounds like a discussion of pros and cons.  There's no pro, since
the user didn't intend to configure it this way, right?


Technically there is a small pro, namely the origin server is confronted 
with less connections. I don't know enough about the implementation 
history (and didn't try to reconstruct it) to decide, whether this was 
done deliberately or it is just a consequence of the straightforward 
implementation. I do expect that the side effects of how it works now 
are highly unexpected, especially as users of mod_proxy start to 
configure worker more often with explicit configuration options (pool 
config, timeouts). I do plan to propose a slight change, namely not 
copying over default values to the alrady configured worker before 
applying the attributes of the second configuration to it.


I'd say the underlying problem is, that workers are identified by the 
URL. It would be cleaner if workers had a name. IMHO we have to live 
with the existing concepts and side effects for 2.4.


Thanks for your constructive comments. I'll work through the other 
replies, add some examples and repost another version.


Regards,

Rainer


Re: Talking about proxy workers

2010-08-06 Thread Rainer Jung

Minor additions inside.

On 06.08.2010 14:49, Plüm, Rüdiger, VF-Group wrote:




-Original Message-
From: Paul Fee
Sent: Freitag, 6. August 2010 14:44
To: dev@httpd.apache.org
Subject: Re: Talking about proxy workers




Also, is it possible to setup these three reuse styles for a
forward proxy?

1: No reuse, close the connection after this request.


Yes, this the default.


2: Reuse connection, but only for the client that caused its creation.


No.


Even if you configure pooled connections like in the example given in 3, 
the connections are returned to the pool after each request/response 
cycle. They are not directly associated with the client connection.


But: if the MPM is prefork, the client connection is handled by a single 
process which doesn't handle any other requests during the life of the 
client connection. Since pools are process local, in this case the pool 
will always return the same connection (the only connection in the 
pool). Note that this pooled connection will not be closed when the 
client connection is closed. It can live longer or shorter than the 
client connection and you can't tie their lifetime together.


Whether the proxy operates in forward or reverse mode doesn't matter, it 
only matters how the pool aka worker is configured. See 3.



3: Pool connection for reuse by any client.


Yes, but this is needed separately for every origin server you forward to:

Proxy http://www.frequentlyused.com/
# Set an arbitrary parameter to trigger the creation of a worker
ProxySet keepalive=on
/Proxy


Pools are associated with workers, and workers are identified by origin 
URL. In case of a reverse proxy you often only have a few origin 
servers, so pooling works fine. In case of a forward proxy you often 
have an enormous amount of origin servers, each only visited every now 
and then. So using persistent connections is less effective. It would 
only make some sense, if we could optionally tie together client 
connections and origin server connections.


Regards,

Rainer


Re: Talking about proxy workers

2010-08-05 Thread Eric Covener

 http://people.apache.org/~rjung/httpd/trunk/manual/mod/mod_proxy.html.en#workers

A direct worker is usually configured using any of ProxyPass,
ProxyPassMatch or ProxySet.

I don't know much about Proxy, but can this hammer home a bit more
that these directives create a new worker implicitly based on [some
parts of?] the destination URL?  And what happens when there's
overlap?

-- 
Eric Covener
cove...@gmail.com