Re: mod_serf is in trunk
On Tue, November 13, 2007 5:39 pm, Justin Erenkrantz wrote: > I'm very much okay with having competition and bake-offs. We have a > number of modules that do similar things (mod_alias / mod_rewrite, > etc.). > > I find that mod_proxy is incredibly complex and doesn't even do the > things that it claims to do properly. Such as? It would be very useful to know what these are, so they they can either be addressed in the current code, or avoided in the new code. Regards, Graham --
Re: mod_serf is in trunk
On Tue, November 13, 2007 7:17 pm, Akins, Brian wrote: > We found it much easier to write our own proxy rather than try to plug > away > at mod_proxy... As I recall, you needed a proxy server that was capable of behaving like an origin server at the cost of RFC compliance. Under circumstances like these, writing a custom module makes perfect sense. Regards, Graham --
Re: mod_serf is in trunk
Ruediger Pluem wrote: On 11/13/2007 04:39 PM, Justin Erenkrantz wrote: I find that mod_proxy is incredibly complex and doesn't even do the things that it claims to do properly. Rather than spend an inordinate amount of time trying to fix it, I think we'd be better off trying to go in a completely different direction with our proxy code and see where it takes us. It is true that mod_proxy is complex, but partly this is because it has many features, so I think we need to differentiate here: Reminds me of the delta between mod_tls and mod_ssl, one trivial module which lets openssl (or gnutls or whatever) do what they do well, and the other where we stick our fingers into every corner of the protocol. Suggesting one or the other is "right" seems besides the point. Bill
Re: mod_serf is in trunk
On 11/13/2007 04:39 PM, Justin Erenkrantz wrote: > On Nov 13, 2007 6:06 AM, Plüm, Rüdiger, VF-Group > <[EMAIL PROTECTED]> wrote: >> I agree here. While I would see a benefit of providing a http(s) client >> API to httpd via serf and thus getting rid of the somewhat strange >> way mod_proxy_http does its requests to a backend system ,I see no >> benefit at all adding another "http reverse proxy only module" for >> the same reasons above. So also -1 from me. > > I'm very much okay with having competition and bake-offs. We have a > number of modules that do similar things (mod_alias / mod_rewrite, > etc.). Competition is fine and maybe my response was a little harsh, but OTOH at least for me the initial mail regarding mod_serf had a little bit of the attitude 'mod_proxy as a whole is crap and here is an nice and tight replacement'. But maybe I am too touchy here :-). > > I find that mod_proxy is incredibly complex and doesn't even do the > things that it claims to do properly. Rather than spend an inordinate > amount of time trying to fix it, I think we'd be better off trying to > go in a completely different direction with our proxy code and see > where it takes us. It is true that mod_proxy is complex, but partly this is because it has many features, so I think we need to differentiate here: 1. There is the framework mod_proxy offers that allows to plugin different backends (currently http(s) / ajp / fcgi) and that offers load balancing with provider based policies. Maybe I am wrong, but I cannot see serf helping us here. If we see need for improvements here I guess its easier to fix them then to start from scratch, at least I believe so unless I see a nice and tight new framework :-). 2. There is mod_proxy_http which had numerous bugs in past and I guess still has some. Thanks to Nick we made huge progress to fix some of the RFC violations here. But as said I totally agree that the approach how mod_proxy_http does http requests is to say the least 'weird'. Using the input filter chain for the response and the connection hook for creating connections has some nasty side effects. So I am very much +1 for an alternate mod_proxy_http module based on serf. It would give us the best out of 1. and serf. But to be fair to mod_proxy_http: I think it currently attracts much more attention and testing than serf and so much more bugs appear on the surface (especially regarding RFC violation). Of course this neither means that we would suffer the same with serf if it was the base of mod_proxy_http nor that I think that serf is untested code. 3. I remember a general discussion about the current brigade and bucket system in httpd and the opinions that serf's concept and implementation of buckets and brigades is better and should be adapted by httpd. As I did not have the time so far to get a deeper view into the brigade and bucket system of serf I cannot really comment on this. But if the concept is better it makes sense to adapt it for 3.0. But this goes far beyond mod_proxy. > > Paul has an initial 'starter' version for us to play with - others can > add features. As said I would have been more lucky if this module were a replacement for mod_proxy_http and would fit in to the mod_proxy framework. > >> IMHO we should focus on providing a http(s) client API to httpd for 2.4 or >> 3.0 >> via serf whose main consumer would be mod_proxy_http. Other consumers >> would be possible mod_cache for prefetching tasks and mod_ssl for >> OCSP checks. Currently there are problems of doing the needed http requests >> for OCSP with mod_proxy due to the early stage in connection handling in >> which these requests are needed. >> It may even make sense to put this http(s) client API in apr-util and >> use serf as the provider of this functionality in apr-util. > > I think adding such a layer to apr-util would be unnecessary bloat as > we don't need to have a layer of indirection here. If we want to use > serf, call serf directly... -- justin I think this also depends on our general policy regarding external libraries: Currently we use the SSL library (openssl in most cases) and the pcre library directly in httpd. All other libraries are used via apr-util and the SSL layer is on the move to apr-util. OTOH I think serf and pcre have API's we want to use directly in httpd in contrast to e.g. ldap, dbm or also SSL API where there are multiple libraries that implement the functions with at least slightly different API's and where apr-util helps us to write programs that are independent of these differences. So yes having an apr-util API layer on top of serf would only make sense if we would like to give the user a choice of different libraries that implement that apr-util layer. Regards Rüdiger
Re: mod_serf is in trunk
On 11/13/2007 05:13 PM, Justin Erenkrantz wrote: > On Nov 13, 2007 11:10 AM, Mads Toftum <[EMAIL PROTECTED]> wrote: >> The name makes me think of it as a provider module like httpd - in fact >> I think that'd be quite useful (especially going by Justins reluctance >> to add it to apr-util which would have been my preferred location). > > Exposing some low-level helper stuff via a provider API/vtable would > be a way to permit any other module to do HTTP requests. That's worth > chewing on... -- justin +1 on that. Possible consumers would be mod_cache (prefetching) and mod_ssl (OCSP). Regards Rüdiger
Re: mod_serf is in trunk
On 11/13/07 11:28 AM, "Jim Jagielski" <[EMAIL PROTECTED]> wrote: > Agreed that mod_proxy has the potential of joining the ranks > of mod_rewrite and mod_ssl as the Modules Most Likely To Make > One Lose Their Minds And Run Screaming Hysterically Through > The Halls. We found it much easier to write our own proxy rather than try to plug away at mod_proxy... -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Re: mod_serf is in trunk
On Nov 13, 2007, at 10:39 AM, Justin Erenkrantz wrote: I find that mod_proxy is incredibly complex and doesn't even do the things that it claims to do properly. But it does NOT do the stuff it doesn't claim to do quite well :) Agreed that mod_proxy has the potential of joining the ranks of mod_rewrite and mod_ssl as the Modules Most Likely To Make One Lose Their Minds And Run Screaming Hysterically Through The Halls.
Re: mod_serf is in trunk
On Nov 13, 2007 11:10 AM, Mads Toftum <[EMAIL PROTECTED]> wrote: > The name makes me think of it as a provider module like httpd - in fact > I think that'd be quite useful (especially going by Justins reluctance > to add it to apr-util which would have been my preferred location). Exposing some low-level helper stuff via a provider API/vtable would be a way to permit any other module to do HTTP requests. That's worth chewing on... -- justin
Re: mod_serf is in trunk
On Tue, Nov 13, 2007 at 10:47:53AM -0500, Jim Jagielski wrote: > I think the idea is that mod_serf is not intended to be a complete > replacement for mod_proxy *at this time*... It's a cool start > and a basis to build on. > The name makes me think of it as a provider module like httpd - in fact I think that'd be quite useful (especially going by Justins reluctance to add it to apr-util which would have been my preferred location). vh Mads Toftum -- http://soulfood.dk
Re: mod_serf is in trunk
On Nov 13, 2007 6:06 AM, Plüm, Rüdiger, VF-Group <[EMAIL PROTECTED]> wrote: > I agree here. While I would see a benefit of providing a http(s) client > API to httpd via serf and thus getting rid of the somewhat strange > way mod_proxy_http does its requests to a backend system ,I see no > benefit at all adding another "http reverse proxy only module" for > the same reasons above. So also -1 from me. I'm very much okay with having competition and bake-offs. We have a number of modules that do similar things (mod_alias / mod_rewrite, etc.). I find that mod_proxy is incredibly complex and doesn't even do the things that it claims to do properly. Rather than spend an inordinate amount of time trying to fix it, I think we'd be better off trying to go in a completely different direction with our proxy code and see where it takes us. Paul has an initial 'starter' version for us to play with - others can add features. > IMHO we should focus on providing a http(s) client API to httpd for 2.4 or 3.0 > via serf whose main consumer would be mod_proxy_http. Other consumers > would be possible mod_cache for prefetching tasks and mod_ssl for > OCSP checks. Currently there are problems of doing the needed http requests > for OCSP with mod_proxy due to the early stage in connection handling in > which these requests are needed. > It may even make sense to put this http(s) client API in apr-util and > use serf as the provider of this functionality in apr-util. I think adding such a layer to apr-util would be unnecessary bloat as we don't need to have a layer of indirection here. If we want to use serf, call serf directly... -- justin
Re: mod_serf is in trunk
On Nov 13, 2007, at 8:55 AM, Axel-Stephane SMORGRAV wrote: Just out of curiosity, how would you do this with mod_serf: ProxyPass /foo http://127.0.0.1/ ProxyPassReverse /foo http://127.0.0.1/ ProxyPassReverse /foo http://localhost/ I think the idea is that mod_serf is not intended to be a complete replacement for mod_proxy *at this time*... It's a cool start and a basis to build on. Sorta like Amsterdam is likely to be serf-based as well :)
Re: mod_serf is in trunk
On 11/13/07, Paul Querna <[EMAIL PROTECTED]> wrote: > I've added mod_serf in r594425: > http://svn.apache.org/viewvc?view=rev&revision=594425 Nice! > I've grown exceptionally... tired of looking at mod_proxy. mod_serf is > nice and tight at 440 lines or so. A cool low number. Fits snugly with the idea of having httpd 3.0 revolving around a serf core. > With just a little more work, I think it could be a production level > reverse proxy. > > Oh yeah, and death to ProxyPass, ProxyPassMatch and all similar > directives that should just go inside Location Blocks. mod_serf will > put everything inside a location block like this: > > > SerfPass http://www.exmaple.com/ > SerfPreseveHost On > SerfReversePass off > SerfTimeout 5 > > > So no more semi-global configurations options like mod_proxy has. > > Thoughts? Cool addition, definitely worth exploring further. Sander
Re: mod_serf is in trunk
On Nov 12, 2007, at 11:34 PM, Paul Querna wrote: I've added mod_serf in r594425: http://svn.apache.org/viewvc?view=rev&revision=594425 I've grown exceptionally... tired of looking at mod_proxy. mod_serf is nice and tight at 440 lines or so. With just a little more work, I think it could be a production level reverse proxy. Oh yeah, and death to ProxyPass, ProxyPassMatch and all similar directives that should just go inside Location Blocks. mod_serf will put everything inside a location block like this: SerfPass http://www.exmaple.com/ SerfPreseveHost On SerfReversePass off SerfTimeout 5 So no more semi-global configurations options like mod_proxy has. Thoughts? At the very least, this is super cool since it provides a code-based usage "guideline" on using serf within httpd. It does seem a lot leaner than mod_proxy, but then again, mod_proxy is more than "just" a reverse proxy impl but more a framework for backend connections...
RE: mod_serf is in trunk
Message d'origine- > De : Paul Querna [mailto:[EMAIL PROTECTED] > Envoyé : mardi 13 novembre 2007 05:34 > À : dev@httpd.apache.org > Objet : mod_serf is in trunk > > I've added mod_serf in r594425: > http://svn.apache.org/viewvc?view=rev&revision=594425 > > I've grown exceptionally... tired of looking at mod_proxy. mod_serf is nice > and tight at 440 lines or so. > > With just a little more work, I think it could be a production level reverse > proxy. > > Oh yeah, and death to ProxyPass, ProxyPassMatch and all similar directives > that should just go inside Location Blocks. mod_serf will put everything > inside a location block like this: > > > SerfPass http://www.exmaple.com/ > SerfPreseveHost On > SerfReversePass off > SerfTimeout 5 > > > So no more semi-global configurations options like mod_proxy has. > > Thoughts? Just out of curiosity, how would you do this with mod_serf: ProxyPass /foo http://127.0.0.1/ ProxyPassReverse /foo http://127.0.0.1/ ProxyPassReverse /foo http://localhost/ -ascs
Re: mod_serf is in trunk
> -Ursprüngliche Nachricht- > Von: Graham Leggett > Gesendet: Dienstag, 13. November 2007 11:28 > An: dev@httpd.apache.org > Cc: dev@httpd.apache.org > Betreff: Re: mod_serf is in trunk > > > On Tue, November 13, 2007 6:34 am, Paul Querna wrote: > > > I've added mod_serf in r594425: > > http://svn.apache.org/viewvc?view=rev&revision=594425 > > > > I've grown exceptionally... tired of looking at mod_proxy. > mod_serf is > > nice and tight at 440 lines or so. > > > > With just a little more work, I think it could be a production level > > reverse proxy. > > I am not sure I follow the reasoning here. > > mod_proxy is a pluggable framework, and as such offers significant > flexibility, which is why we have mod_proxy_http, mod_proxy_ftp, > mod_proxy_connect, mod_proxy_ajp and mod_proxy_balancer > modules, that the > end user is free to use or not use these modules as needed. > All of these > modules exist because end users actively use this functionality. > > mod_serf, as I can see so far, is a simple module, offers > reverse proxy > http only, and is smaller[1] than mod_proxy only because it does less. > Assuming of course the issue of the missing documentation is fixed, I > don't see how continued development on mod_serf could bring > us to anything > other than what we have now, only written differently. > > Now, say mod_serf gets documented properly and finds itself > in a release. > How do you answer the following end user questions? > > "There are two reverse proxy modules in httpd, which should I > use and why?" > > "Does this mean you are taking AJP (etc) out of the server?" > > The answers "the code is tighter" and "I am tired of looking > at the code" > answer none of these end user questions, so I am -1 on this > until I see a > proper justification for the new module. I agree here. While I would see a benefit of providing a http(s) client API to httpd via serf and thus getting rid of the somewhat strange way mod_proxy_http does its requests to a backend system ,I see no benefit at all adding another "http reverse proxy only module" for the same reasons above. So also -1 from me. IMHO we should focus on providing a http(s) client API to httpd for 2.4 or 3.0 via serf whose main consumer would be mod_proxy_http. Other consumers would be possible mod_cache for prefetching tasks and mod_ssl for OCSP checks. Currently there are problems of doing the needed http requests for OCSP with mod_proxy due to the early stage in connection handling in which these requests are needed. It may even make sense to put this http(s) client API in apr-util and use serf as the provider of this functionality in apr-util. Regards Rüdiger
Re: mod_serf is in trunk
On Tue, November 13, 2007 6:34 am, Paul Querna wrote: > I've added mod_serf in r594425: > http://svn.apache.org/viewvc?view=rev&revision=594425 > > I've grown exceptionally... tired of looking at mod_proxy. mod_serf is > nice and tight at 440 lines or so. > > With just a little more work, I think it could be a production level > reverse proxy. I am not sure I follow the reasoning here. mod_proxy is a pluggable framework, and as such offers significant flexibility, which is why we have mod_proxy_http, mod_proxy_ftp, mod_proxy_connect, mod_proxy_ajp and mod_proxy_balancer modules, that the end user is free to use or not use these modules as needed. All of these modules exist because end users actively use this functionality. mod_serf, as I can see so far, is a simple module, offers reverse proxy http only, and is smaller[1] than mod_proxy only because it does less. Assuming of course the issue of the missing documentation is fixed, I don't see how continued development on mod_serf could bring us to anything other than what we have now, only written differently. Now, say mod_serf gets documented properly and finds itself in a release. How do you answer the following end user questions? "There are two reverse proxy modules in httpd, which should I use and why?" "Does this mean you are taking AJP (etc) out of the server?" The answers "the code is tighter" and "I am tired of looking at the code" answer none of these end user questions, so I am -1 on this until I see a proper justification for the new module. [1] Assuming it is actually smaller. The module is 440 lines + the size of the serf library, while proxy is self contained. > Oh yeah, and death to ProxyPass, ProxyPassMatch and all similar > directives that should just go inside Location Blocks. mod_serf will > put everything inside a location block like this: > > > SerfPass http://www.exmaple.com/ > SerfPreseveHost On > SerfReversePass off > SerfTimeout 5 > > > So no more semi-global configurations options like mod_proxy has. Proxy already supports this, which is no different: ProxyPass http://www.example.com/ ProxyPassReverse http://www.example.com That said, a far deeper issue lies underneath the proxy, and that is: "Why does the server treat Alias, ScriptAlias and ProxyPass as different directives, when in reality they are all the same thing?" Ideally at the core of http, there needs to be a generic way of mapping URLs to other URLs, in such a way that functionality like load balancing can be achieved easily and generically. The module mod_proxy (the parent framework module) will then become part of the (http) core, and mod_proxy_http and friends become just mod_http and friends. In addition, I should be able to set access control for .htaccess as for example allowing file:, but not allowing cgi:, or fastcgi:, or http:, or ftp:. But this is a goal that is most effectively tackled in amsterdam. Regards, Graham --
