About mod_mbox future improvements

2005-06-22 Thread Maxime Petazzoni
Hi list !

A few weeks ago, I showed interest in participating to mod_mbox
development. As a proof of my will to become part of the team, I've
already made two interesting improvements to actual module code :

- Email obfuscation (patch submitted last week to this list)
- A brand new XML+XSLT based interface, currently using client side
  XSLT processing.

Both of them are in action on my local mod_mbox setup :
http://skikda.bulix.org/archives/

But for now, I'd like to speak and start to discuss about mod_mbox
future, and -to be a bit more accurate- to what I might do as my SoC
summer project.

Please note that whether or not I'm accepted into this program, I'm
willing to work on mod_mbox for the same reasons I claimed at the
beginning of my SoC application (available at
http://blog.bulix.org/index.php/pages/googlesoc/show).

There are two main points that currently come to me that need to be
discussed since it involves mod_mbox basis design.

* XSLT processing

As far as I remember, the choice of outputting XML satisfies
everybody. The problem relates to where we'll do the XSLT
processing. There are two main solutions :

- client-side processing, as by now. The user requires a capable
  browser such as Gecko-based browsers.

- server-side processing, using mod_transform (or something else,
  ideas ? I don't know much about server-side output filters)

Client-side processing's advantage is to fasten global response time
since the server have less work to do. Main drawback is the lack of
XSLT-capable browser (even KHTML doesn't seem to do it).

Server-side processing corrects this problem, but may slow down the
response (isn't mod_mbox designed to be as quick as possible ?).

* A mailing-list management application ?

It has been said recently on this list that you (as in developers
involved in mod_mbox development) wish to make mod_mbox become a
full-featured mailing-list management complete application, that can
handle multiple mailing lists, and especially from a row rsync
checkout.

I completely agree with this last point (rsync), but I must say that I
have some doubts about the first one (making mod_mbox a complete
management application).

Indeed, making mod_mbox a application that can manage multiple lists
correctly (e.g. with additional list information better than
auto-generated subscribe/unsubscribe email addresses) will require
some sort of database, which is exactly what the first point wants to
avoid !

That's why I believe this should be the job of third party software
that will just pass the job to mod_mbox when it comes to serve a
specific mailing-list archive.

Well, that's all for tonight folks !

Good evening/morning/afternoon,

- Sam

-- 
Maxime Petazzoni (http://www.bulix.org)
 -- gone crazy, back soon. leave message.


signature.asc
Description: Digital signature


[STATUS] (httpd-test: perl-framework) Wed Jun 22 23:47:04 2005

2005-06-22 Thread Rodent of Unusual Size
httpd-test/perl-framework STATUS:   -*-text-*-
Last modified at [$Date: 2004-11-24 19:36:41 -0500 (Wed, 24 Nov 2004) $]

Stuff to do:
* finish the t/TEST exit code issue (ORed with 0x2C if
  framework failed)

* change existing tests that frob the DocumentRoot (e.g.,
  t/modules/access.t) to *not* do that; instead, have
  Makefile.PL prepare appropriate subdirectory configs
  for them.  Why?  So t/TEST can be used to test a
  remote server.

* problems with -d perl mode, doesn't work as documented
  Message-ID: [EMAIL PROTECTED]
  Date: Sat, 20 Oct 2001 12:58:33 +0800
  Subject: Re: perldb

Tests to be written:

* t/apache
  - simulations of network failures (incomplete POST bodies,
chunked and unchunked; missing POST bodies; slooow
client connexions, such as taking 1 minute to send
1KiB; ...)

* t/modules/autoindex
  - something seems possibly broken with inheritance on 2.0

* t/ssl
  - SSLPassPhraseDialog exec:
  - SSLRandomSeed exec:


[STATUS] (httpd-test: flood) Wed Jun 22 23:46:53 2005

2005-06-22 Thread Rodent of Unusual Size
flood STATUS:   -*-text-*-
Last modified at [$Date: 2004-11-24 19:36:41 -0500 (Wed, 24 Nov 2004) $]

Release:

1.0:   Released July 23, 2002
milestone-03:  Tagged January 16, 2002
ASF-transfer:  Released July 17, 2001
milestone-02:  Tagged August 13, 2001
milestone-01:  Tagged July 11, 2001 (tag lost during transfer)

RELEASE SHOWSTOPPERS:

* Everything needs to work perfectly

Other bugs that need fixing:

* I get a SIGBUS on Darwin with our examples/round-robin-ssl.xml
  config, on the second URL. I'm using OpenSSL 0.9.6c 21 dec 2001.
  
* iPlanet sends Content-length - there is a hack in there now
  to recognize it.  However, all HTTP headers need to be normalized
  before checking their values.  This isn't easy to do.  Grr.

* OpenSSL 0.9.6
  Segfaults under high load.  Upgrade to OpenSSL 0.9.6b.
   Aaron says: I just found a big bug that might have been causing
   this all along (we weren't closing ssl sockets).
   How can I reproduce the problem you were seeing
   to verify if this was the fix?

* SEGVs when /tmp/.rnd doesn't exist are bad. Make it configurable
  and at least bomb with a good error message. (See Doug's patch.)
   Status: This is fixed, no?

* If APR has disabled threads, flood should as well. We might want
  to have an enable/disable parameter that does this also, providing
  an error if threads are desired but not available.

* flood needs to clear pools more often. With a long running test
  it can chew up memory very quickly. We should just bite the bullet
  and create/destroy/clear pools for each level of our model:
  farm, farmer, profile, url/request-cycle, etc.

* APR needs to have a unified interface for ephemeral port
  exhaustion, but aparently Solaris and Linux return different
  errors at the moment. Fix this in APR then take advantage of
  it in flood.

* The examples/analyze-relative scripts fail when there are less
  than 5 unique URLs.

Other features that need writing:

* More analysis and graphing scripts are needed

* Write robust tool (using tethereal perhaps) to take network dumps 
  and convert them to flood's XML format.
Status: Justin volunteers.  Aaron had a script somewhere that is
a start. Jacek is working on a Mozilla application, codename
Flood URL bag (much like Live HTTP Headers) and small
HTTP proxy.

* Get chunked encoding support working.
Status: Justin volunteers.  He got sidetracked by the httpd
implementation of input filtering and never finished 
this.  This is a stopgap until apr-serf is completed.

* Maybe we should make randfile and capath runtime directives that
  come out of the XML, instead of autoconf parameters.

* We are using apr_os_thread_current() and getpid() in some places
  when what we really want is a GUID. The GUID will be used to
  correlate raw output data with each farmer. We may wish to print
  a unique ID for each of farm, farmer, profile, and url to help in
  postprocessing.

* We are using strtol() in some places and strtoll() in others.
  Pick one (Aaron says strtol(), but he's not sure).

* Validation of responses (known C-L, specific strings in response)
   Status: Justin volunteers

* HTTP error codes (ie. teach it about 302s)
   Justin says: Yeah, this won't be with round_robin as implemented.  
Need a linked list-based profile where we can insert 
new URLs into the sequence.

* Farmer (Single-thread, multiple profiles)
   Status: Aaron says: If you have threads, then any Farmer can be
   run as part of any Farm. If you don't have threads, you can
   currently only run one Farmer named Joe right now (this will
   be changed so that if you don't have threads, flood will attempt
   to run all Farmers in serial under one process).

* Collective (Single-host, multiple farms)
  This is a number of Farms that have been fork()ed into child processes.

* Megaconglomerate (Multiple hosts each running a collective)
  This is a number of Collectives running on a number of hosts, invoked
  via RSH/SSH or maybe even some proprietary mechanism.

* Other types of urllists
a) Random / Random-weighted
b) Sequenced (useful with cookie propogation)
c) Round-robin
d) Chaining of the above strategies
  Status: Round-robin is complete.

* Other types of reports
  Status: Aaron says: simple reports are functional. Justin added
  a new type that simply prints the approx. timestamp when
  the test was run, and the result as OK/FAIL; it is called
  easy reports (see flood_easy_reports.h).

Re: 2.1.6 on Friday

2005-06-22 Thread Graham Leggett
Paul Querna said:

 I would like to roll another alpha on this Friday, June 24th.

 I hope to resolve these issues that blocked 2.1.5 from going out:

 1) Compile on Win32.
 2) Proper use of strcasecmp to check for identity encoding. (d'oh)
 3) Fix any cases where the protocol is not set/NULL.

 Anything else anyone wants to get in?

There was an outstanding bug report which complained that the Microsoft
LDAP libraries used to build the Windows mod_ldap binary were too old - is
it possibl to make sure that the most recent service pack of the LDAP
client library is used for any of the builds?

Regards,
Graham
--



Accessing to per Directory configuration from an input filter: HOW?

2005-06-22 Thread luca regini
I need to write an input filter that is able to change the value of some cookies. However the name of the cookie to be changed is a per- directory value. So i have an input filter whose behavior depends programmatically from per-directory configuration. How should i gain information about per -dir configuration from within a filter??



Thanks in advance 
Luca


Comments on bug #35280: FTP proxy breaks RFC 2428 when trying to fall back from EPSV to PASV

2005-06-22 Thread Viipuri, Timo
Hi,

What do you think of bug #35280 and the patch attached to fix it. I think at
least handling the response 522: Protocol not supported correctly (as RFC
2428 suggests) is important. Handling the response 505: Command Blocked is
not that vital but I suppose including it also in the patch wouldn't hurt.

Or do you have ideas why the patch shouldn't be included? Thanks in advance
for comments!

 Timo


Re: Accessing to per Directory configuration from an input filter: HOW?

2005-06-22 Thread Nick Kew
luca regini wrote:
 I need to write an input filter that is able to change the value of some 
 cookies. However the name of the cookie to be changed is a per- directory 
 value. So i have an input filter whose behavior depends programmatically 
 from per-directory configuration. How should i gain information about per 
 -dir configuration from within a filter??
   Thanks in advance 
 Luca
 

Same as from anywhere else.

But you don't want to do that in an input filter.  Use the header_parser
hook.  Or, if it needs to happen before/after some other module which
views cookies in a different hook, move as appropriate.

-- 
Nick Kew


Re: Accessing to per Directory configuration from an input filter: HOW?

2005-06-22 Thread luca regini
I am not able to find any example of use of this hook to alter the value of a cookie. For my ( limited and erroneous) understanding an hook cannot alter requestsnor produce content. Thisare tasks made for filters.


Luca
On 6/22/05, Nick Kew [EMAIL PROTECTED] wrote:
luca regini wrote: I need to write an input filter that is able to change the value of some cookies. However the name of the cookie to be changed is a per- directory
 value. So i have an input filter whose behavior depends programmatically from per-directory configuration. How should i gain information about per -dir configuration from within a filter?? Thanks in advance
 LucaSame as from anywhere else.But you don't want to do that in an input filter.Use the header_parserhook.Or, if it needs to happen before/after some other module whichviews cookies in a different hook, move as appropriate.
--Nick Kew


Re: Accessing to per Directory configuration from an input filter: HOW?

2005-06-22 Thread luca regini
I add to my previous mail that thectx parameter in the ap_add_output_filter function can be used to pass an arbitrary structure to the filter. So this should be the right place to pass information to the filter.


Luca


On 6/22/05, Nick Kew [EMAIL PROTECTED] wrote:
luca regini wrote: I need to write an input filter that is able to change the value of some cookies. However the name of the cookie to be changed is a per- directory
 value. So i have an input filter whose behavior depends programmatically from per-directory configuration. How should i gain information about per -dir configuration from within a filter?? Thanks in advance
 LucaSame as from anywhere else.But you don't want to do that in an input filter.Use the header_parserhook.Or, if it needs to happen before/after some other module whichviews cookies in a different hook, move as appropriate.
--Nick Kew


[PATCH] 1.3 TraceEnable [on|off|extended]

2005-06-22 Thread William A. Rowe, Jr.
I've spent a large number of cycles investigating the Watchfire report 
(http://www.watchfire.com/resources/HTTP-Request-Smuggling.pdf) and
come up with a genuine reason to adopt the attached patch.

One advantage of TRACE is to determine exactly what the server thinks
it just saw.  This is ideal for walking requests through proxies.

Unfortunately, TRACE does not support request bodies(*) (RFC2616 9.8;
A TRACE request MUST NOT include an entity.), so it turned out 
to be somewhat useless, initially.  Supporting such behavior on
a very explicit basis (for testing the server or proxies, etc), since
proved to be very helpful for me.  (*) Note that 1.3, 2.0 and trunk
all support TRACE request bodies on proxied connections, the attached
patch also addresses this error.

Finally, after reviewing the report, I'm now convinced there is some
legitimate reason to deny TRACE if the administrator chooses.  Although
I don't subscribe to security by obscurity, too much transparency can
sometimes be a useful thing for exploit tools.  I still laugh at the
premise that TRACE 'caused' the IE xss vulnerabilities, of course.
Note that TRACE is not mandatory per RFC 2616 (only GET, HEAD are.)

So the attached patch introduces the per-host directive

TraceEnable on|off|extended

where extended permits a message body, up to 64kb at the target server,
and of an unlimited size through a proxy server.  The default remains
'on', of course, denying a TRACE body request even via proxy.

Following the semantics of TRACE, the request body is returned to the
host verbatim as part of the response, following the headers, exactly
as sent.

Patches will follow for 2.0 and trunk.  I have several days of other
work to catch up on, so won't jump on this till Friday eve.  Comments
on this patch before I start would be appreciated.

Bill
Index: src/modules/proxy/proxy_http.c
===
--- src/modules/proxy/proxy_http.c  (revision 191530)
+++ src/modules/proxy/proxy_http.c  (working copy)
@@ -15,6 +15,7 @@
 
 /* HTTP routines for Apache proxy */
 
+#define CORE_PRIVATE   /* To inspect core_server_conf-trace_enable */
 #include mod_proxy.h
 #include http_log.h
 #include http_main.h
@@ -141,6 +142,24 @@
 memset(server, '\0', sizeof(server));
 server.sin_family = AF_INET;
 
+if (r-method_number == M_TRACE) {
+core_server_config *coreconf = (core_server_config *)
+ ap_get_module_config(r-server-module_config, core_module);
+
+if (coreconf-trace_enable == AP_TRACE_DISABLE)
+return ap_proxyerror(r, HTTP_METHOD_NOT_ALLOWED,
+ TRACE denied by server configuration);
+
+/* Can't test ap_should_client_block, we aren't ready to send
+ * the client a 100 Continue response till the connection has
+ * been established
+ */
+if (coreconf-trace_enable != AP_TRACE_EXTENDED 
+ (r-read_length || (!r-read_chunked  (r-remaining = 0
+return ap_proxyerror(r, HTTP_REQUEST_ENTITY_TOO_LARGE,
+ TRACE with request body is not allowed);
+}
+
 /* We break the URL into host, port, path-search */
 
 urlptr = strstr(url, ://);
Index: src/include/http_core.h
===
--- src/include/http_core.h (revision 191530)
+++ src/include/http_core.h (working copy)
@@ -344,8 +344,18 @@
 int recursion_limit_set; /* boolean */
 int redirect_limit;  /* maximum number of internal redirects */
 int subreq_limit;/* maximum nesting level of subrequests */
+
+/* TRACE control */
+int trace_enable;/* see AP_TRACE_ below */
+
 } core_server_config;
 
+/* trace_enable options */
+#define AP_TRACE_UNSET-1
+#define AP_TRACE_DISABLE   0
+#define AP_TRACE_ENABLE1
+#define AP_TRACE_EXTENDED  2
+
 /* for http_config.c */
 CORE_EXPORT(void) ap_core_reorder_directories(pool *, server_rec *);
 
Index: src/main/http_protocol.c
===
--- src/main/http_protocol.c(revision 191530)
+++ src/main/http_protocol.c(working copy)
@@ -1637,11 +1637,14 @@
 }
 
 /* Build the Allow field-value from the request handler method mask.
- * Note that we always allow TRACE, since it is handled below.
+ * Note that we determine TRACE on a per-server basis.
  */
 static char *make_allow(request_rec *r)
 {
-return 2 + ap_pstrcat(r-pool,
+core_server_config *conf =
+ap_get_module_config(r-server-module_config, core_module);
+
+char *res = ap_pstrcat(r-pool,
(r-allowed  (1  M_GET))   ? , GET, HEAD : ,
(r-allowed  (1  M_POST))  ? , POST  : ,
(r-allowed  (1  M_PUT))   ? , PUT   : ,
@@ -1656,24 +1659,87 @@
(r-allowed  (1  M_MOVE))  ? , MOVE  : ,

Re: 2.1.6 on Friday

2005-06-22 Thread William A. Rowe, Jr.
At 02:55 AM 6/22/2005, Graham Leggett wrote:

There was an outstanding bug report which complained that the Microsoft
LDAP libraries used to build the Windows mod_ldap binary were too old - is
it possibl to make sure that the most recent service pack of the LDAP
client library is used for any of the builds?

The biggest issue is with the ldap TLS support, it seems to
vary greatly between releases.  There are some trivial hacks
I could do to avoid using binding-by-ordinal (the root of the
problem) but won't be able to get that by Friday.

Disable TLS on win32 in the meantime?

Bill




Re: 2.1.5 available for testing

2005-06-22 Thread William A. Rowe, Jr.
At 02:12 AM 6/20/2005, Paul Querna wrote:
William A. Rowe, Jr. wrote:

Also, possibly across platforms is a fault in ssl_engine_init,
where the host-protocol is still NULL, and we are trying to
strcmp it to 'https'.  I spent part of my weekend trying to
grok what change has broken this, but strcmp to NULL is popping
a segfault.  Not worthy of rejecting 2.1.5 on it's own, this is
still a minor irritation.  FYI - mod_ssl was loaded without SSL
being defined, so no ssl host actually exists.

In all cases the protocol should be set, and never be null. Look at  
ap_setup_listeners() in server/listen.c.

All linux cases as usual :-?

ap_setup_listeners is only called in the parent.  On mpm_winnt,
this happens in the open_logs hook of the parent.

get_listeners_from_parent in the mpm_winnt creates those per-child
listen records.  Memory is not inherited between parent and child
so the patch to modify this broke win32.

This code seems a bit too convoluted for me to delve into right
now, and I'm not quite clear what problem the patch was ment to
solve in the first place (?)

Bill





Re: apache developers documentation!!!

2005-06-22 Thread Joshua Slive


Nick Kew wrote:

On Tuesday 21 June 2005 21:32, arebassa arebassa wrote:


Hi all,

 I'm very newbie to apache development and I'd like to know more
about it. Is there any documentation about the functionalities of the
differents parts of code? how the code is structured? what is a bucket
brigade?... I have started looking at the documentation at
http://docx.webperf.org/index.html but I wander if there is any
documentation more focused on newbies.



If I might blow my own trumpet:-

(1) http://www.apachetutor.org/  - go to the Developer tutorials (which is
the only section of the site with any contents).

(2) http://www.apachecon.com/ - come to our module developers tutorial
and other talks.


Note that http://httpd.apache.org/docs-2.0/developer/ is supposed to be 
the canonical location for developer docs and links.  Nick, could you 
add a link to your site?


Joshua.


Re: 2.1.5 available for testing

2005-06-22 Thread Paul Querna

William A. Rowe, Jr. wrote:


At 02:12 AM 6/20/2005, Paul Querna wrote:
 


William A. Rowe, Jr. wrote:

   


Also, possibly across platforms is a fault in ssl_engine_init,
where the host-protocol is still NULL, and we are trying to
strcmp it to 'https'.  I spent part of my weekend trying to
grok what change has broken this, but strcmp to NULL is popping
a segfault.  Not worthy of rejecting 2.1.5 on it's own, this is
still a minor irritation.  FYI - mod_ssl was loaded without SSL
being defined, so no ssl host actually exists.
 

In all cases the protocol should be set, and never be null. Look at  
ap_setup_listeners() in server/listen.c.
   



All linux cases as usual :-?

 


Ugh.  FreeBSD actually, and all the *nix MPMs do this correctly :-P


ap_setup_listeners is only called in the parent.  On mpm_winnt,
this happens in the open_logs hook of the parent.
 

I wish we had more 'rules' for MPMs.  Right now its hit and miss. In 
this case, I missed mpm_winnt...



get_listeners_from_parent in the mpm_winnt creates those per-child
listen records.  Memory is not inherited between parent and child
so the patch to modify this broke win32.
 

Okay, I will look at breaking this out to a separate function, and then 
calling it from the correct places in mpm_winnt.



This code seems a bit too convoluted for me to delve into right
now, and I'm not quite clear what problem the patch was ment to
solve in the first place (?)
 

The point is to inherit a Protocol in a Virtual Host from the Listening 
Socket, if one wasn't explicitly set.


-Paul


Re: [PATCH] 1.3 TraceEnable [on|off|extended]

2005-06-22 Thread William A. Rowe, Jr.
At 08:56 AM 6/22/2005, William A. Rowe, Jr. wrote:

Unfortunately, TRACE does not support request bodies(*) (RFC2616 9.8;
A TRACE request MUST NOT include an entity.), so it turned out 
to be somewhat useless, initially.

FYI there is one small issue still.  The resulting Allow: null
response to denied TRACE request.  TRACE doesn't go through the
normal processing, so methods aren't added.  And since TRACE is
denied, it's removed too.

Result - we should use UNKNOWN_METHOD rather than METHOD_NOT_ALLOWED
since the later requires an Allow: header.  And, omit empty Allow:

Bill




Re: 2.1.5 available for testing

2005-06-22 Thread William A. Rowe, Jr.
At 01:57 AM 6/20/2005, jean-frederic clere wrote:

I still need some more time to check POST with 2 different content-lengths and 
GET with content-length.

Well, GET with c-l is permitted.  2 C-L headers would be rejected
due to the '##, ##' format, where the ', ' is non-numeric.

After your patch, in 2.1 we now accept (discard) the C-L headers
(even multiples) if we also have T-E: chunked.

Prior to your patch we ment to reject the request with C-L and T-E
if Paul's patch was entirely correct.

Prior to either patch we totally mishandled such requests.  So the
only question which remains is; which behavior do we prefer?
As the RFC states this is not acceptable, my gut says reject ANY
request with both C-L and T-E of non-identity.

Quoting 4.4 of RFC 2616;

   For compatibility with HTTP/1.0 applications, HTTP/1.1 requests
   containing a message-body MUST include a valid Content-Length header
   field unless the server is known to be HTTP/1.1 compliant. If a
   request contains a message-body and a Content-Length is not given,
   the server SHOULD respond with 400 (bad request) if it cannot
   determine the length of the message, or with 411 (length required) if
   it wishes to insist on receiving a valid Content-Length.

   All HTTP/1.1 applications that receive entities MUST accept the
   chunked transfer-coding (section 3.6), thus allowing this mechanism
   to be used for messages when the message length cannot be determined
   in advance.

   Messages MUST NOT include both a Content-Length header field and a
   non-identity transfer-coding. If the message does include a non-
   identity transfer-coding, the Content-Length MUST be ignored.

So I can see where we would ignore C-L, I'm mostly concerned that
we never proxy or pass on a C-L passed by a client through our
proxy to another UA for non-identity encodings.

But rejecting outright seems safer :)
Bill






Re: 2.1.5 available for testing

2005-06-22 Thread Paul Querna

Joe Orton wrote:


On Wed, Jun 22, 2005 at 03:02:50PM -0500, William Rowe wrote:
 


Prior to either patch we totally mishandled such requests.  So the
only question which remains is; which behavior do we prefer?
As the RFC states this is not acceptable, my gut says reject ANY
request with both C-L and T-E of non-identity.
   



I don't see any reason to reject anything, 2616 dictates precisely how 
to handle requests which are malformed in this way.  I do think the 
check against identity is actually redundant, though; not least 
because the 2616 errata remove all references to the word.


I think correct behaviour is to just follow the letter of Section 4.4, 
point 3, as below:


If a message is received with both a
Transfer-Encoding header field and a Content-Length header field,
the latter MUST be ignored.

(and it's also going to be better to check for T-E before C-L since 
99.99% of requests received are not going to have a T-E header so it'll 
fall through slightly quicker)


 


+1 to the posted patch.

-Paul



Re: apache developers documentation!!!

2005-06-22 Thread Ben Laurie

Akins, Brian wrote:




On 6/21/05 5:29 PM, Nick Kew [EMAIL PROTECTED] wrote:
 
  (2) http://www.apachecon.com/ - come to our module developers tutorial

  and other talks.

When will there be another apachecon US?


December.

--
ApacheCon Europe   http://www.apachecon.com/

http://www.apache-ssl.org/ben.html   http://www.thebunker.net/

There is no limit to what a man can do or how far he can go if he
doesn't mind who gets the credit. - Robert Woodruff


Re: 2.1.5 available for testing

2005-06-22 Thread Jeff Trawick
On 6/22/05, Paul Querna [EMAIL PROTECTED] wrote:
 Joe Orton wrote:
 
 On Wed, Jun 22, 2005 at 03:02:50PM -0500, William Rowe wrote:
 
 
 Prior to either patch we totally mishandled such requests.  So the
 only question which remains is; which behavior do we prefer?
 As the RFC states this is not acceptable, my gut says reject ANY
 request with both C-L and T-E of non-identity.
 
 
 
 I don't see any reason to reject anything, 2616 dictates precisely how
 to handle requests which are malformed in this way.  I do think the
 check against identity is actually redundant, though; not least
 because the 2616 errata remove all references to the word.
 
 I think correct behaviour is to just follow the letter of Section 4.4,
 point 3, as below:
 
If a message is received with both a
  Transfer-Encoding header field and a Content-Length header field,
  the latter MUST be ignored.
 
 (and it's also going to be better to check for T-E before C-L since
 99.99% of requests received are not going to have a T-E header so it'll
 fall through slightly quicker)
 
 
 
 +1 to the posted patch.

+1 here as well

What about proxy reading from origin server?

Origin server sends this to Apache acting as proxy:
HTTP/1.1 200 OK\r\n
Content-Length: 99\r\n
Transfer-Encoding: chunked\r\n
\r\n
0A\r\n
aa\r\n
00\r\n
\r\n

Client receives this:

HTTP/1.1 200 OK
Date: Wed, 22 Jun 2005 23:12:31 GMT
Content-Length: 99
Content-Type: text/plain; charset=ISO-8859-1

aa(END)

Add this patch:

Index: modules/proxy/mod_proxy_http.c
===
--- modules/proxy/mod_proxy_http.c  (revision 191655)
+++ modules/proxy/mod_proxy_http.c  (working copy)
@@ -1128,7 +1128,17 @@
r-headers_out,
save_table);
 }
-
+
+/* can't have both Content-Length and Transfer-Encoding */
+if (apr_table_get(r-headers_out, Transfer-Encoding)
+ apr_table_get(r-headers_out, Content-Length)) {
+/* 2616 section 4.4, point 3: if both Transfer-Encoding
+ * and Content-Length are received, the latter MUST be
+ * ignored; so unset it here to prevent any confusion
+ * later. */
+apr_table_unset(r-headers_out, Content-Length);
+}
+
 /* strip connection listed hop-by-hop headers from response */
 backend-close +=
ap_proxy_liststr(apr_table_get(r-headers_out,
  Connection),

Client now gets this:

HTTP/1.1 200 OK
Date: Wed, 22 Jun 2005 23:22:19 GMT
Transfer-Encoding: chunked
Content-Type: text/plain; charset=ISO-8859-1

a
aa
0


[STATUS] (httpd-2.0) Wed Jun 22 23:45:45 2005

2005-06-22 Thread Rodent of Unusual Size
APACHE 2.0 STATUS:  -*-text-*-
Last modified at [$Date: 2005-06-20 01:47:52 -0400 (Mon, 20 Jun 2005) $]

The current version of this file can be found at:
http://svn.apache.org/repos/asf/httpd/httpd/branches/2.0.x/STATUS

Release history:

2.0.55  : in development
2.0.54  : released April 17, 2005 as GA.
2.0.53  : released February 7, 2005 as GA.
2.0.52  : released September 28, 2004 as GA.
2.0.51  : released September 15, 2004 as GA.
2.0.50  : released June 30, 2004 as GA.
2.0.49  : released March 19, 2004 as GA.
2.0.48  : released October 29, 2003 as GA.
2.0.47  : released July 09, 2003 as GA.
2.0.46  : released May 28, 2003 as GA.
2.0.45  : released April 1, 2003 as GA.
2.0.44  : released January 20, 2003 as GA.
2.0.43  : released October 3, 2002 as GA.
2.0.42  : released September 24, 2002 as GA.
2.0.41  : rolled September 16, 2002.  not released.
2.0.40  : released August 9, 2002 as GA.
2.0.39  : released June 17, 2002 as GA.
2.0.38  : rolled June 16, 2002.  not released.
2.0.37  : rolled June 11, 2002.  not released.
2.0.36  : released May 6, 2002 as GA.
2.0.35  : released April 5, 2002 as GA.
2.0.34  : tagged March 26, 2002.
2.0.33  : tagged March 6, 2002.  not released.
2.0.32  : released Feburary 16, 2002 as beta.
2.0.31  : rolled Feburary 1, 2002.  not released.
2.0.30  : tagged January 8, 2002.  not rolled.
2.0.29  : tagged November 27, 2001.  not rolled.
2.0.28  : released November 13, 2001 as beta.
2.0.27  : rolled November 6, 2001
2.0.26  : tagged October 16, 2001.  not rolled.
2.0.25  : rolled August 29, 2001
2.0.24  : rolled August 18, 2001
2.0.23  : rolled August 9, 2001
2.0.22  : rolled July 29, 2001
2.0.21  : rolled July 20, 2001
2.0.20  : rolled July 8, 2001
2.0.19  : rolled June 27, 2001
2.0.18  : rolled May 18, 2001
2.0.17  : rolled April 17, 2001
2.0.16  : rolled April 4, 2001
2.0.15  : rolled March 21, 2001
2.0.14  : rolled March 7, 2001
2.0a9   : released December 12, 2000
2.0a8   : released November 20, 2000
2.0a7   : released October 8, 2000
2.0a6   : released August 18, 2000
2.0a5   : released August 4, 2000
2.0a4   : released June 7, 2000
2.0a3   : released April 28, 2000
2.0a2   : released March 31, 2000
2.0a1   : released March 10, 2000

Please consult the following STATUS files for information on related projects:

* http://svn.apache.org/repos/asf/apr/apr/branches/0.9.x/STATUS
* http://svn.apache.org/repos/asf/apr/apr-util/branches/0.9.x/STATUS
* http://svn.apache.org/repos/asf/httpd/httpd/branches/2.0.x/docs/STATUS

Contributors looking for a mission:

* Just do an egrep on TODO or XXX in the source.

* Review the bug database at: http://issues.apache.org/bugzilla/

* Review the PatchAvailable bugs in the bug database:

  
http://issues.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDproduct=Apache+httpd-2.0keywords=PatchAvailable

  After testing, you can append a comment saying Reviewed and tested.

* Open bugs in the bug database.

CURRENT RELEASE NOTES:

* Forward binary compatibility is expected of Apache 2.0.x releases, such
  that no MMN major number changes will occur.  Such changes can only be
  made in the trunk.

* All commits to branches/2.0.x must be reflected in SVN trunk,
  as well, if they apply.  Logical progression is commit to trunk,
  get feedback and votes in STATUS, and then merge into branches/2.0.x.

RELEASE SHOWSTOPPERS:

PATCHES TO BACKPORT FROM TRUNK:
  [ please place SVN revisions from trunk here, so it is easy to
identify exactly what the proposed changes are! ]
  [ please append new backports at the end of this list not the top. ]

*) ap_proxy_canonenc() is over-eager in handling '%' for reverse
   proxies (PR: 29554).
 Index: modules/proxy/proxy_util.c
 -   if (isenc  ch == '%') {
 +   if (isenc  (isenc != PROXYREQ_REVERSE)  ch == '%') {

   +1: jim, pquerna

*) Prevent hang writing to piped logger at graceful restart time.
   PR: 26467
 http://svn.apache.org/viewcvs?rev=170281view=rev
 http://svn.apache.org/viewcvs.cgi?rev=171093view=rev
   +1: trawick, jorton, pquerna

*) Fix fd leak in piped logging code, fix error handling, and remove
   dead errno handling.
 http://svn.apache.org/viewcvs?rev=170441view=rev
 http://svn.apache.org/viewcvs?rev=170537view=rev
 http://svn.apache.org/viewcvs?rev=170719view=rev
   all-in-one patch incremental to the PR 26467 fix:
 http://people.apache.org/~jorton/ap_pipedlog2.diff
   +1: jorton, trawick
   [yes, I will write a CHANGES entry too]

*) several changes to improve logging of connection-oriented errors, 
including
   

[STATUS] (httpd-2.1) Wed Jun 22 23:45:54 2005

2005-06-22 Thread Rodent of Unusual Size
APACHE 2.1 STATUS:  -*-text-*-
Last modified at [$Date: 2005-06-17 03:03:13 -0400 (Fri, 17 Jun 2005) $]

The current version of this file can be found at:
http://svn.apache.org/repos/asf/httpd/httpd/trunk/STATUS

Release history:
[NOTE that only Alpha/Beta releases occur in 2.1 development]

2.1.6   : in development
2.1.5   : Tagged on 6/17/2005. 
2.1.4   : not released.
2.1.3   : Released on  2/22/2005 as alpha.
2.1.2   : Released on 12/08/2004 as alpha.
2.1.1   : Released on 11/19/2004 as alpha.
2.1.0   : not released.

Please consult the following STATUS files for information on related projects:

* http://svn.apache.org/repos/asf/apr/apr/trunk/STATUS
* http://svn.apache.org/repos/asf/apr/apr-util/trunk/STATUS
* http://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/STATUS

Contributors looking for a mission:

* Just do an egrep on TODO or XXX in the source.

* Review the bug database at: http://issues.apache.org/bugzilla/

* Review the PatchAvailable bugs in the bug database:

  
http://issues.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDproduct=Apache+httpd-2.0keywords=PatchAvailable

  After testing, you can append a comment saying Reviewed and tested.

* Open bugs in the bug database.

CURRENT RELEASE NOTES:

RELEASE SHOWSTOPPERS:

* Handling of non-trailing / config by non-default handler is broken
  http://marc.theaimsgroup.com/?l=apache-httpd-devm=105451701628081w=2
  jerenkrantz asks: Why should this block a release?

* the edge connection filter cannot be removed 
  http://marc.theaimsgroup.com/?l=apache-httpd-devm=105366252619530w=2
  jerenkrantz asks: Why should this block a release?
  stas replies: because it requires a rewrite of the filters stack
implementation (you have suggested that) and once 2.2 is
released you can't do that anymore. 

CURRENT VOTES:

* httpd-std.conf and friends

  a) httpd-std.conf should be tailored by install (from src or
 binbuild) even if user has existing httpd.conf
 +1:   trawick, slive, gregames, ianh, Ken, wrowe, jwoolley, jim, nd,
   erikabele
   wrowe - prefer httpd.default.conf to avoid ambiguity with cvs

  b) tailored httpd-std.conf should be copied by install to
 sysconfdir/examples
 -0:   striker

  c) tailored httpd-std.conf should be installed to
 sysconfdir/examples or manualdir/exampleconf/
 +1:   slive, trawick, Ken, nd (prefer the latter), erikabele

  d) Installing a set of default config files when upgrading a server
 doesn't make ANY sense at all.
 +1:   ianh - medium/big sites don't use 'standard config' anyway, as it
  usually needs major customizations
 -1:   Ken, wrowe, jwoolley, jim, nd, erikabele
   wrowe - diff is wonderful when comparing old/new default configs,
   even for customized sites that ianh mentions
   jim - ... assuming that the default configs have been updated
 with the required inline docs to explain the
 changes

* If the parent process dies, should the remaining child processes
  gracefully self-terminate. Or maybe we should make it a runtime
  option, or have a concept of 2 parent processes (one being a 
  hot spare).
  See: Message-ID: [EMAIL PROTECTED]

  Self-destruct: Ken, Martin, Lars
  Not self-destruct: BrianP, Ian, Cliff, BillS
  Make it runtime configurable: Aaron, jim, Justin, wrowe, rederpj, nd

  /* The below was a concept on *how* to handle the problem */
  Have 2 parents: +1: jim
  -1: Justin, wrowe, rederpj, nd
  +0: Lars, Martin (while standing by, could it do
something useful?)

* Make the worker MPM the default MPM for threaded Unix boxes.
  +1:   Justin, Ian, Cliff, BillS, striker, wrowe, nd
  +0:   BrianP, Aaron (mutex contention is looking better with the
latest code, let's continue tuning and testing), rederpj, jim
  -0:   Lars

  pquerna: Do we want to change this for 2.2?

RELEASE NON-SHOWSTOPPERS BUT WOULD BE REAL NICE TO WRAP THESE UP:

* Patches submitted to the bug database:
  
http://issues.apache.org/bugzilla/buglist.cgi?bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDproduct=Apache+httpd-2.0keywords=PatchAvailable

* The Event MPM does not work on Solaris 10. PR 34040.

* Filter stacks and subrequests, redirects and fast redirects.
  There's at least one PR that suffers from the current unclean behaviour
  (which lets the server send garbage): PR 17629
  nd says: Every subrequest should get its own filter stack with the
   subreq_core filter as bottom-most. That filter does two things:
   

Re: 2.1.5 available for testing

2005-06-22 Thread William A. Rowe, Jr.
++1 To Joe's comments.

Jeff's fix is technically right, but scares the nibbles out
of me.  If, for example, an exploit is able to inject the
T-E on top of the legit C-L, I really suspect we should not 
trust the origin server at all.

For origin servers (as opposed to clients) make this choice
between ignore C-L, or fail, configurable?

My observation is that there are far more varied clients in
the world than servers, each with unique faults.  But the
RFC 2616 is clear...

   Messages MUST NOT include both a Content-Length header field and a
   non-identity transfer-coding. If the message does include a non-
   identity transfer-coding, the Content-Length MUST be ignored.

   When a Content-Length is given in a message where a message-body is
   allowed, its field value MUST exactly match the number of OCTETs in
   the message-body. HTTP/1.1 user agents MUST notify the user when an
   invalid length is received and detected.

...and server authors can be expected to be less buggy than clients.
Permissive in what we accept, strict in what we send implies some
strictness in what we trust to pass on to the client.

So +.5 to Jeff's patch, and let's discuss if the proxy response should 
then be trusted at all with T-E and C-L, in httpd-2.x where we support 
keepalives.

[FYI +1 in httpd-1.3, that proxy does not use keepalives.]

Bill




Bill

At 06:40 PM 6/22/2005, Jeff Trawick wrote:
On 6/22/05, Paul Querna [EMAIL PROTECTED] wrote:
 Joe Orton wrote:
 
 On Wed, Jun 22, 2005 at 03:02:50PM -0500, William Rowe wrote:
 
 
 Prior to either patch we totally mishandled such requests.  So the
 only question which remains is; which behavior do we prefer?
 As the RFC states this is not acceptable, my gut says reject ANY
 request with both C-L and T-E of non-identity.
 
 
 
 I don't see any reason to reject anything, 2616 dictates precisely how
 to handle requests which are malformed in this way.  I do think the
 check against identity is actually redundant, though; not least
 because the 2616 errata remove all references to the word.
 
 I think correct behaviour is to just follow the letter of Section 4.4,
 point 3, as below:
 
If a message is received with both a
  Transfer-Encoding header field and a Content-Length header field,
  the latter MUST be ignored.
 
 (and it's also going to be better to check for T-E before C-L since
 99.99% of requests received are not going to have a T-E header so it'll
 fall through slightly quicker)
 
 
 
 +1 to the posted patch.

+1 here as well

What about proxy reading from origin server?

Origin server sends this to Apache acting as proxy:
HTTP/1.1 200 OK\r\n
Content-Length: 99\r\n
Transfer-Encoding: chunked\r\n
\r\n
0A\r\n
aa\r\n
00\r\n
\r\n

Client receives this:

HTTP/1.1 200 OK
Date: Wed, 22 Jun 2005 23:12:31 GMT
Content-Length: 99
Content-Type: text/plain; charset=ISO-8859-1

aa(END)

Add this patch:

Index: modules/proxy/mod_proxy_http.c
===
--- modules/proxy/mod_proxy_http.c  (revision 191655)
+++ modules/proxy/mod_proxy_http.c  (working copy)
@@ -1128,7 +1128,17 @@
r-headers_out,
save_table);
 }
-
+
+/* can't have both Content-Length and Transfer-Encoding */
+if (apr_table_get(r-headers_out, Transfer-Encoding)
+ apr_table_get(r-headers_out, Content-Length)) {
+/* 2616 section 4.4, point 3: if both Transfer-Encoding
+ * and Content-Length are received, the latter MUST be
+ * ignored; so unset it here to prevent any confusion
+ * later. */
+apr_table_unset(r-headers_out, Content-Length);
+}
+
 /* strip connection listed hop-by-hop headers from response */
 backend-close +=
ap_proxy_liststr(apr_table_get(r-headers_out,
  
 Connection),

Client now gets this:

HTTP/1.1 200 OK
Date: Wed, 22 Jun 2005 23:22:19 GMT
Transfer-Encoding: chunked
Content-Type: text/plain; charset=ISO-8859-1

a
aa
0