Re: svn commit: r393037 - in /httpd/httpd/trunk: CHANGES server/protocol.c

[Roy T. Fielding]

On Apr 11, 2006, at 2:55 PM, Nick Kew wrote:


On Tuesday 11 April 2006 22:10, William A. Rowe, Jr. wrote:

Ruediger Pluem wrote:

On 04/11/2006 04:00 AM, Roy T. Fielding wrote:
It probably needs to be updated for RFC 3986 anyway.  The path  
should

be set to "", not NULL.  The HTTP server should take care of the
redirect from "" to "/", which in this case means the http-proxy
needs to check for "" when it sends a request and respond with a
redirect that adds the "/".


Um, it's not really a redirect; it's just a normalisation.  Shouldn't
really invoke any redirect logic, whether internal or external.


The server should redirect any time the characters in the request URI
are changed, since that impacts the digests used in various access
control mechanisms.

Roy

Re: svn commit: r393037 - in /httpd/httpd/trunk: CHANGES server/protocol.c

[Roy T. Fielding]

On Apr 10, 2006, at 2:50 PM, Ruediger Pluem wrote:
I also thought initially to fix this in apr-util, but right know I  
am not
sure about it, because IMHO apr_uri_parse should do generic uri  
parsing.
Setting an empty uri to "/" seems to be HTTP specific, so I am not  
sure

if we should do this in apr_uri_parse. At least we would need to check
whether the scheme is http or https.


It probably needs to be updated for RFC 3986 anyway.  The path should
be set to "", not NULL.  The HTTP server should take care of the
redirect from "" to "/", which in this case means the http-proxy
needs to check for "" when it sends a request and respond with a
redirect that adds the "/".

Roy

Re: ECONNRESET, low keepalives, and pipelined requests?

[Roy T. Fielding]

On Feb 9, 2006, at 10:17 PM, Justin Erenkrantz wrote:

On IRC, Paul pointed out this bug (now fixed):

http://issues.apache.org/bugzilla/show_bug.cgi?id=35292

2.0.50 probably has this bug - in that it'll won't do lingering close
correctly - and perhaps that's what I'm running into.


You're testing against 2.0.50?  Crikey.


Any cute ideas on how to work around this?  The real problem is that
there's no way for the server to tell me what its configured
MaxKeepAliveRequests setting is.  If I knew that, I could respect it -
instead I have to discover it experimentally...


That's why we used to send a Keep-Alive: header on responses that
indicated how many requests were left.  Don't get me started...

Roy

Re: ECONNRESET, low keepalives, and pipelined requests?

[Roy T. Fielding]

On Feb 9, 2006, at 9:36 PM, Justin Erenkrantz wrote:


Has anyone ever seen a situation where httpd (or the OS) will RST a
connection because there's too much unread data or such?

I'm doing some pipelined requests with serf against a 2.0.50 httpd  
on RH7.1
server (2.4.2 kernel?).  I'm getting ECONNRESET on the client after  
I try
to read or write a large number of requests.  httpd's side is  
sending the

RSTs - but there's nothing in the httpd logs.

Can an RST happen without a process dying?  Isn't that the most common
reason for the RST flag?  (We've checked and no httpd are dying,  
AFAICT.)


Bumping the MaxKeepAliveRequests from 10 to 100 apparently solves  
this; but
that's just odd - yet it implies that httpd is in some control over  
this

behavior.

Yet, if it were httpd, why isn't it responding to all of the previous
requests before it hit the MaxKeepAliveRequests?  (There is no  
response
with 'Connection: Close' being sent - it just drops off in the  
middle of

writing the response body as far as we can see.)  So, why would it
terminate the connection *before* responding to all of the outstanding
responses that are under the MaxKeepAliveRequests limit?  Is httpd  
writing

the response and Linux just dropping it?


Keep in mind that a RST also tells the recipient to throw away any
data that it has received since the last ACK.  Thus, you would never
see the server's last response unless you use an external network
monitor (like another PC running ethereal connected to your client PC
with a non-switching hub).

My guess is that, when MaxKeepAliveRequests is reached, the server
process closes the connection and tells the client.  If lingering
close hasn't been broken, it will then continue reading some data
from the client precisely to avoid this lost response problem.
Serf should be looking for Connection: close on the last response
it received and close the connection, starting again on a different one.

I suggest you check the lingering close code to see if someone has
disabled it on Linux.  People do that some times because they think
it is a performance drag to linger on close, neglecting to consider
that the client will be left clueless if a RST is sent before the
client acks receipt of the server's response.

Roy

Re: CHANGES attribution reminder

[Roy T. Fielding]

On Jan 21, 2006, at 2:29 PM, Ruediger Pluem wrote:

Ok. Then I had a different understanding from my osmosis :-).
Any other comments on this?
I have no problem adopting the above rules for future CHANGE entries.


Jim is correct.

It is easy to forget now because Subversion doesn't have the
rcstemplate feature, but commits should still have:

  PR:
  Obtained from:
  Submitted by:
  Reviewed by:

in the log when they are applicable.

CVS:  
--

CVS: PR:
CVS:   If this change addresses a PR in the problem report tracking
CVS:   database, then enter the PR number(s) here.
CVS: Obtained from:
CVS:   If this change has been taken from another system, such as NCSA,
CVS:   then name the system in this line, otherwise delete it.
CVS: Submitted by:
CVS:   If this code has been contributed to Apache by someone else;  
i.e.,
CVS:   they sent us a patch or a new module, then include their name/ 
email

CVS:   address here. If this is your work then delete this line.
CVS: Reviewed by:
CVS:   If we are doing pre-commit code reviews and someone else has
CVS:   reviewed your changes, include their name(s) here.
CVS:   If you have not had it reviewed then delete this line.


Roy

please set up a mod_python core group

[Roy T. Fielding]

It looks like mod_python is making good progress and everyone
is collaborating in the Apache way of testing and voting.
That's great!

Unfortunately, I have almost no insight into who these great people
are that are doing the RM task and testing and voting and preparing
for a next release.  That's not so great, since it is my job (as
VP of Apache HTTP Server Project) to be sure that the ASF knows all
this work is being done in its name and so that all of the people
doing it are appropriately recognized for their work.

So, please, take a few moments to decide amongst yourselves who
should have binding votes on mod_python (i.e., who has earned it),
keeping in mind that you need at least three binding +1 votes in
order to make any release at Apache, and send me a list of names
and email addresses of those people so that I can properly
record them in our records.

Cheers,

Roy T. Fielding<http://roy.gbiv.com/>
for the Apache HTTP Server PMC

filesystem directives

[Roy T. Fielding]

For someone looking for something to do,

The authorization code makes an assumption that filesystems allowing
file ownership is a platform-specific define.  That is not
the case for the same reason that case-sensitivity is not based
on the platform.  All of the filesystem characteristics should be
a runtime configuration scoped within the Directory directives, e.g.

   FileSystemCaseSensitive no
   FileSystemHasOwners yes

which can then be looked up in the per-dir config while performing
operations. [How to do this efficiently is unknown to me.]

Note that both of those are settable per disk in OS X, and in other
cases they will be dependent on the remote filesystem host (SMB).
The default should be based on some combination of platform and
perhaps platform-specific tests.  We've needed this feature for a
long time.

Roy

Re: Merging branch authz-dev - Authorization and Access Control 2.3 vs. 2.2

[Roy T. Fielding]

On Jan 11, 2006, at 7:19 AM, Joshua Slive wrote:


[Your merge today prompted me to dig out a response I started but
never finished.]

I am still worried that we are underestimating the pain that this will
cause.  In my opinion, a config change that requires substantial
changes to every httpd.conf and many .htaccess files requires a major
version bump (to 3.0) unless it can, in some way, be made seamless to
the end user.  And there is no way to deny that this will put a large
roadblock in the way of upgraders.


It isn't just your opinion -- incompatible configuration changes
means third-parties have to change their source code, which means a
major version bump is required.  So either somebody gets busy on
implementing backward-compatibility or this stuff gets bumped to 3.x.

We could decide that the next release will be 3.0, but I doubt it.

Roy

Re: svn commit: r366174 - /httpd/mod_mbox/trunk/module-2.0/mod_mbox_mime.c

[Roy T. Fielding]

On Jan 5, 2006, at 4:49 AM, [EMAIL PROTECTED] wrote:


+In order to handle empty boundaries, we'll look for the
+boundary plus the \n. */
+
+   boundary_line = apr_pstrcat(p, "--", mail->boundary, "\n", NULL);

/* The start boundary */
-   bound = ap_strstr(mail->body, mail->boundary);
+   bound = ap_strstr(mail->body, boundary_line);


That seems a bit risky -- MIME parts are supposed to have CRLF for
line terminators, but that code will only search for LF on Unix.

Would it make more sense to use a regex?

Roy

Fwd: I-D ACTION:draft-eastlake-sha2-01.txt

[Roy T. Fielding]

It might be a good project for someone to take this I-D and convert
it to an apache utility library.

Roy

Begin forwarded message:


From: [EMAIL PROTECTED]
Date: January 4, 2006 12:50:01 PM PST
To: i-d-announce@ietf.org
Subject: I-D ACTION:draft-eastlake-sha2-01.txt
Reply-To: [EMAIL PROTECTED]
Message-Id: <[EMAIL PROTECTED]>

A New Internet-Draft is available from the on-line Internet-Drafts  
directories.



Title   : US Secure Hash Algorithms (SHA)
Author(s)   : D. Eastlake 3rd, T. Hansen
Filename: draft-eastlake-sha2-01.txt
Pages   : 99
Date: 2006-1-4

The United States of America has adopted a suite of secure hash
   algorithms (SHAs), including four beyond SHA-1, as part of a  
Federal

   Information Processing Standard (FIPS), specifically SHA-224 [RFC
   3874], SHA-256, SHA-384, and SHA-512.  The purpose of this document
   is to make open source code performing these hash functions
   conveniently available to the Internet community. The sample code
   supports input strings of arbitrary bit length. SHA-1's sample code
   from [RFC 3174] has also been updated to handle an input string of
   arbitrary length. Most of the text herein was adapted by the  
authors

   from FIPS 180-2.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-eastlake-sha2-01.txt

Re: svn commit: r360461 - in /httpd/httpd/trunk: CHANGES include/ap_mmn.h include/httpd.h server/protocol.c

[Roy T. Fielding]

On Jan 3, 2006, at 12:02 AM, William A. Rowe, Jr. wrote:


Roy T. Fielding wrote:

On Jan 2, 2006, at 2:14 PM, Roy T. Fielding wrote:

Now, if you want to tell me that those changes produced a net
performance benefit on prefork (and thus are applicable to other   
MPMs),
then I am all ears.  I am easily convinced by comparative  
performance

figures when the comparison is meaningful.


lol, of course you choose the non-threaded MPM as a reference,  
which therefore
should receive no meaningful performance change.  The difference  
between an
async wakeup and a poll result should be null for one socket pool,  
one process,
one thread (of course select is a differently ugly beast, and if  
there were

a platform that supported async with no poll, I'd laugh.)


You seem to misunderstand me -- if I compare two prefork servers, one
with the change and one without the change, and the one with the change
performs better (by whatever various measures of performance you can  
test),

then that is an argument for making the change regardless of the other
concerns.

If, instead, you say that the change improves the event MPM by 10% and
degrades performance on prefork by 1%, then I am -1 on that change.
Prefork is our workhorse MPM.  The task then is to isolate MPM-specific
changes so that they have no significant impact on the critical path
of our primary MPM, even if that means using #ifdefs.

Alternatively, rewrite the server to remove all MPMs other than
event and then show that the new server is better than our existing
server, and we can adopt that for 3.0.


BTW, part of the reason I say that is because I have considered
replacing the same code with something that doesn't parse the
header fields until the request header/body separator line is
seen, since that would allow the entire request header to be parsed
in-place for the common case.


Well ... you are using protocol knowledge that will render us http- 
bound
when it comes time to efficently bind waka (no crlf delims in a  
binary format
protocol, no?) or ftp (pushes a 'greeting' before going back to  
sleep.)


Well, I am assuming that the MIME header parsing code is in the
protocol-specific part of the server, yes.

Roy

Re: svn commit: r360461 - in /httpd/httpd/trunk: CHANGES include/ap_mmn.h include/httpd.h server/protocol.c

[Roy T. Fielding]

On Jan 2, 2006, at 2:14 PM, Roy T. Fielding wrote:


Now, if you want to tell me that those changes produced a net
performance benefit on prefork (and thus are applicable to other  
MPMs),

then I am all ears.  I am easily convinced by comparative performance
figures when the comparison is meaningful.


BTW, part of the reason I say that is because I have considered
replacing the same code with something that doesn't parse the
header fields until the request header/body separator line is
seen, since that would allow the entire request header to be parsed
in-place for the common case.

Roy

Re: svn commit: r360461 - in /httpd/httpd/trunk: CHANGES include/ap_mmn.h include/httpd.h server/protocol.c

[Roy T. Fielding]

On Jan 2, 2006, at 1:37 PM, Brian Pane wrote:


"Significantly faster than prefork" has never been a litmus test for
assessing new features, and I'm -1 for adding it now.  A reasonable
technical metric for validating the async changes would "significantly
more scalable than the 2.2 Event MPM" or "memory footprint
competitive with IIS/Zeus/phttpd/one's-competitive-benchmark-of- 
choice."


Those aren't features.  They are properties of the resulting system
assuming all goes well.


The bit about degrading the rest of the server is a wonderful sound
bite, but we need to engineer the httpd based on data, not FUD.


I said leave it on the async branch until you have data.  You moved
it to trunk before you've even implemented the async part, which I
think is wrong because the way you implemented it damages the
performance of prefork and needlessly creates an incompatible MMN.
Maybe it would be easier for me to understand why the event loop is
being controlled at such a high level if I could see it work first.

Now, if you want to tell me that those changes produced a net
performance benefit on prefork (and thus are applicable to other MPMs),
then I am all ears.  I am easily convinced by comparative performance
figures when the comparison is meaningful.

Roy

Re: svn commit: r360461 - in /httpd/httpd/trunk: CHANGES include/ap_mmn.h include/httpd.h server/protocol.c

[Roy T. Fielding]

On Dec 31, 2005, at 9:55 PM, Brian Pane wrote:

On Dec 31, 2005, at 6:50 PM, Roy T. Fielding wrote:


The nonblocking yield should happen inside ap_rgetline (or its
replacement), not in the calling routine.  The thread has nothing
to do until that call is finished or it times out.


On the contrary, the thread has some very important work to do before
that call finishes or times out: it has other connections to  
process.  If
the thread waits until the ap_rgetline completes, a server  
configuration
sized for multiple threads per connection will be vulnerable to a  
trivially

implementable DoS.


When I say "thread", I mean a single stream of control with execution
stack, not OS process/thread.  An event MPM is going to have a single
stream of control per event, right?  What I am saying is that the
control should block in rgetline and yield (return to the event pool)
inside that function.  That way, the complications due to yielding are
limited to the I/O routines that might block a thread rather than
being spread all over the server code.

Am I missing something?  This is not a new topic -- Dean Gaudet had
quite a few rants on the subject in the archives.


 In any case,
this code should be independent of the MPM and no MPM is going
to do something useful with a partial HTTP request.

I say -1 to adding the input buffer variables to the request_rec.
Those variables can be local to the input loop.


Are you proposing that the buffers literally become local variables?
That generally won't work; the input loop isn't necessarily contained
within a single function call, and the reading of a single request's
input can cross threads.


I am saying it doesn't belong in the request_rec.  It belongs on the
execution stack that the yield routine has to save in order to return
to this execution path on the next event.  The request does not care
about partial lines.


It would be feasible to build up the pending request in a structure
other than the request_rec, so that ap_read_async_request() can
operate on, say, an ap_partial_request_t instead of a request_rec.
My preference so far, though, has been to leave the responsibility
for knowing how to parse request headers encapsulated within
the request_rec and its associated "methods."


Maybe you should just keep those changes on the async branch for now.
The rest of the server cannot be allowed to degrade just because you
want to introduce a new MPM.  After the async branch is proven to be
significantly faster than prefork, then we can evaluate whether or
not the additional complications are worth it.

Roy

Re: svn commit: r360461 - in /httpd/httpd/trunk: CHANGES include/ap_mmn.h include/httpd.h server/protocol.c

[Roy T. Fielding]

On Dec 31, 2005, at 3:45 PM, [EMAIL PROTECTED] wrote:


Author: brianp
Date: Sat Dec 31 15:45:11 2005
New Revision: 360461

URL: http://svn.apache.org/viewcvs?rev=360461&view=rev
Log:
Refactoring of ap_read_request() to store partial request state
in the request rec.  The point of this is to allow asynchronous
MPMs do do nonblocking reads of requests.  (Backported from the
async-read-dev branch)


Umm, this needs more eyes.

It doesn't seem to me to be doing anything useful.  It just adds
a set of unused input buffer fields to the wrong memory structure,
resulting in what should be a minor (not major) MMN bump, and then
rearranges a critical-path function into several subroutines.

The nonblocking yield should happen inside ap_rgetline (or its
replacement), not in the calling routine.  The thread has nothing
to do until that call is finished or it times out.  In any case,
this code should be independent of the MPM and no MPM is going
to do something useful with a partial HTTP request.

I say -1 to adding the input buffer variables to the request_rec.
Those variables can be local to the input loop.  I don't see any
point in placing this on trunk until it can do something useful.

Roy

Re: Event MPM: Spinning on cleanups?

[Roy T. Fielding]

On Dec 30, 2005, at 5:51 PM, Brian Pane wrote:

I haven't been able to find the bug yet.  As a next step, I'll try  
using

valgrind on a build with pool debugging enabled.


On entry to allocator_free, if

   (node == node->next && node->index > current_free_index)

is true, then the do { } while ((node = next) != NULL);
will go into an infinite loop.  This is because

if (max_free_index != APR_ALLOCATOR_MAX_FREE_UNLIMITED
&& index > current_free_index) {
node->next = freelist;
freelist = node;
}

does not update current_free_index.  I don't know if that is the
problem, but it may be the symptom.

Roy

erain removed from list

[Roy T. Fielding]

It looks like an autobot was fooled into subscribing here.
I have removed it and added it to the deny list.

Roy

Re: Vote for mod_mbox 0.2 release

[Roy T. Fielding]

On Dec 23, 2005, at 10:34 AM, Maxime Petazzoni wrote:

Ok, updated tarballs have been uploaded to
. Vote is restarted.


They should be called 0.2.1, though I'll let that pass as there were
no code changes.  However, you do need to remember to check the
file permissions after uploading the files.  They need to be

   chmod 664 *

I fixed them myself the last time, but don't have time right now.

Roy

Re: Vote for mod_mbox 0.2 release

[Roy T. Fielding]

On Dec 21, 2005, at 5:15 PM, Paul Querna wrote:
I agree with all points, except the last.  They do work, they are  
running mail-archives.apache.org.


No, an edited version of them is doing a poor job of occasionally
being manually used to update our mailing lists.  I just ran into
that problem again today when I looked at wadi-dev at incubator,
which does not yet appear in the archives because the scripts
don't actually work.  They really need to be updated so that they
pick up all the archives in a dir tree automatically -- something
we could not do at the time, but can now that all of our public
archives are in a single tree.


If I had spare time, I would love to remove ZSH :)


It's on my list.  Right below finishing waka. ;-)

Roy

Re: Vote for mod_mbox 0.2 release

[Roy T. Fielding]

On Dec 21, 2005, at 4:51 PM, Maxime Petazzoni wrote:


* Paul Querna <[EMAIL PROTECTED]> [2005-12-21 16:45:09]:


What specifically is wrong with the scripts directory?


Scripts are mostly specific to the ASF setup for
mail-archives.apache.org. They're not involved in making the module
work for the lambda user or even for anybody who's not part of the
infrastructure team ...

I did not changed my mind on this point : these scripts do not belong
to the mod_mbox repository, or at least not to trunk/ (and thus,
releases).


I think they should stay in trunk.  I just don't think they should
be included in the tarball until someone cares enough to fix or
replace them.

Roy

Re: Vote for mod_mbox 0.2 release

[Roy T. Fielding]

On Dec 21, 2005, at 4:45 PM, Paul Querna wrote:

Roy T. Fielding wrote:

Sorry, NOTICE is not a credits file.  Add a README instead that lists
all of the contributors.  Likewise, the scripts directory sucks  
beyond

description.  If you aren't going to fix it, then remove it from the
release.


What specifically is wrong with the scripts directory?


They don't work.  They include actual company names that have since
been bought by people outside our influence.  They include a domain
name that I own which isn't active right now.  They are written in
a language that nobody other than Justin understands.  And, well,
they don't work.

Roy

Re: Vote for mod_mbox 0.2 release

[Roy T. Fielding]

On Dec 21, 2005, at 2:54 PM, Sander Temme wrote:

On Dec 21, 2005, at 11:16 PM, Maxime Petazzoni wrote:


Of course, since this tag is currently running like a charm on Ajax,
my vote is +1 for GA.


Like a charm, indeed. Zero cores since this morning CET.

+1 for GA.


-1.  Er, sorry, I was about to vote +1 and then noticed that the
legal NOTICE file contains

==
This product includes software developed by
The Apache Software Foundation (http://www.apache.org/).

Originally developed by Justin Erenkrantz, eBuilt.

The SLL sort in mbox_sort.c is based on the public-domain algorithm by
Philip J. Erdelsky ([EMAIL PROTECTED]).  You may find the algorithm and
notes at: 

The threading code in mbox_thread.c is based Jamie Zawinski's
description of the Netscape 3.x threading algorithm at:


The 'mime_decode_qp' and `mime_decode_b64' routine are taken from
metamail 2.7, which is copyright (c) 1991 Bell Communications
Research, Inc. (Bellcore).

This product includes software developed by Edward Rudd and Paul
Querna (http://www.outoforder.cc/).

Most of mod_mbox development is now handled by Maxime Petazzoni.

==

Sorry, NOTICE is not a credits file.  Add a README instead that lists
all of the contributors.  Likewise, the scripts directory sucks beyond
description.  If you aren't going to fix it, then remove it from the
release.

Roy

Re: mod_mbox 0.2 goes alpha

[Roy T. Fielding]

You guys are confusing each other to bits.  We don't have release
candidates, we NEVER use m.n.v.rc1 versioning, and the thing that
Sam produced is called a tarball.  We call it that because we don't
want people to believe it is a release until the PMC has voted to
release it.  That's all there is to it.

Sam, you just need to follow the same process as all of our other
releases -- send a message to dev asking for votes on the tarball
for declaring it the 0.2.0 alpha release. Give it three days and,
at the end, if you have at least three +1s  (including your own)
and a majority of positive votes for release, then you can move it
to the actual release dist and work on an announcement to go out
24hrs later.

Roy

Re: 2.2 mod_http_proxy and "partial" pages

[Roy T. Fielding]

On Dec 16, 2005, at 12:41 AM, Plüm, Rüdiger, VIS wrote:

I do not intend to do close the connection by myself. Currently it
will be closed because c->keepalive is set to AP_CONN_CLOSE
(a approach also suggested in Roys patch).


Right, the important bit is that the code managing the client
connection is what should close the client connection, not the
code that is managing the outbound (server) connection.  For all
we know, the client connection might be an SMTP notifier.

If we wanted to get really fancy, we could check to see if the
response code has been sent yet and change the whole response to
a 502 bad gateway, but that probably isn't worth the effort.


The only addition I want to make is that in the chunked case
the chunked filter should not sent the closing chunk to make
it clear to the client that something had broken.


I agree with that change.


The question that remains to me: Does it hurt that the core output
filter removes the error bucket once it has seen it?


I have no idea. I looked at the filtering code and the way it uses
error buckets (in fact, the only place that uses error buckets)
and I don't understand why it was written this way at all.  It is
more efficient to just use the EOS bucket data to indicate an
error in all of those cases, since they all result in an EOS
anyway and we just waste memory/time with the special bucket type.

Roy

Re: svn commit: r355823 - in /httpd/httpd/trunk/modules/proxy: mod_proxy.c mod_proxy.h mod_proxy_http.c

[Roy T. Fielding]

On Dec 10, 2005, at 5:30 PM, Ruediger Pluem wrote:
But with your approach you need to double the code in *each* scheme  
handler.


mod_mem_cache doesn't look like it works right now.  I haven't looked
at the other schemes yet.

r355823 tries to avoid this by doing it a little later in the proxy  
handler.
I currently comment only on the position *where* you do it and not  
what.
But of course in the proxy handler I do not have rv add hand any  
longer.
The remaining part of the patch could be done also in the proxy  
handler

with e->data set to an int that is non null.
Ok. Other approach would be to put your code in a function of  
proxy_util.c
where every scheme handler can call it. That sounds also like a  
good solution.


Ideally it would be in APR, but that runs into version problems.  Since
it is a general streams issue, I would put the function in httpd.h, but
right now I just want to know if it works at all (since I have no way
of testing it).

I think this is the full patch, but there may be a better place to
check for EOS errors.

Roy

Index: server/core_filters.c
===
--- server/core_filters.c   (revision 355728)
+++ server/core_filters.c   (working copy)
@@ -678,6 +678,11 @@
 for (i = offset; i < nvec; ) {
 apr_bucket *bucket = APR_BRIGADE_FIRST(bb);
 if (APR_BUCKET_IS_METADATA(bucket)) {
+if (APR_BUCKET_IS_EOS(bucket) && bucket->data) {
+/* stream aborted and we have not ended it  
yet */

+c->keepalive = AP_CONN_CLOSE;
+bucket->data = NULL;
+}
 APR_BUCKET_REMOVE(bucket);
 apr_bucket_destroy(bucket);
 }
Index: modules/proxy/mod_proxy_http.c
===
--- modules/proxy/mod_proxy_http.c  (revision 355728)
+++ modules/proxy/mod_proxy_http.c  (working copy)
@@ -1481,12 +1481,16 @@
 }
 else if (rv != APR_SUCCESS) {
 /* In this case, we are in real trouble  
because

- * our backend bailed on us, so abort our
- * connection to our user too.
+ * our backend bailed on us.
  */
 ap_log_cerror(APLOG_MARK, APLOG_ERR, rv, c,
   "proxy: error reading  
response");

-c->aborted = 1;
+r->no_cache = 1;
+e = apr_bucket_eos_create(c->bucket_alloc);
+e->data = &rv;
+APR_BRIGADE_INSERT_TAIL(bb, e);
+ap_pass_brigade(r->output_filters, bb);
+backend->close = 1;
 break;
 }
 /* next time try a non-blocking read */
Index: modules/cache/mod_disk_cache.c
===
--- modules/cache/mod_disk_cache.c  (revision 355728)
+++ modules/cache/mod_disk_cache.c  (working copy)
@@ -1010,7 +1010,7 @@
  * sanity checks.
  */
 if (APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(bb))) {
-if (r->connection->aborted) {
+if (r->connection->aborted || r->no_cache) {
 ap_log_error(APLOG_MARK, APLOG_INFO, 0, r->server,
  "disk_cache: Discarding body for URL %s "
  "because connection has been aborted.",

Re: svn commit: r355823 - in /httpd/httpd/trunk/modules/proxy: mod_proxy.c mod_proxy.h mod_proxy_http.c

[Roy T. Fielding]

On Dec 10, 2005, at 5:02 PM, Justin Erenkrantz wrote:
That will get mod_cache not to pick up on it, right.  However, it  
won't
force the connection to close.  Or, give anything a chance to close  
it: be

it HTTP/1.1 or Waka.  =)  -- justin


Right, that was just the proxy part -- the other half is not done yet.

Roy

Re: svn commit: r355823 - in /httpd/httpd/trunk/modules/proxy: mod_proxy.c mod_proxy.h mod_proxy_http.c

[Roy T. Fielding]

On Dec 10, 2005, at 4:15 PM, [EMAIL PROTECTED] wrote:


Author: rpluem
Date: Sat Dec 10 16:15:27 2005
New Revision: 355823

URL: http://svn.apache.org/viewcvs?rev=355823&view=rev
Log:
* Move handling of backends that broke after the headers have been  
sent

  into the proxy handler of mod_proxy.

  This patch still sets r->connection->aborted to 1 which is currently
  vetoed by Roy. Moving it from the scheme handler to the proxy  
handler
  should ease the reimplementation of this, as the scheme handlers  
only
  needs to return PROXY_BACKEND_BROKEN to signal the above  
situation to

  the proxy handler.


Oy, crikey, I don't want that -- the proxy handler doesn't have the
information needed to complete the response.  As I said before, the
right way to do it is to send the error downstream.  There is no
need to leave the filter in an error state.

As it turns out, we also have a request_rec->no_cache variable that
is supposed to be set in this case.  I am curious whether the following
patch will work (after reverting r355823).

Roy


Index: modules/proxy/mod_proxy_http.c
===
--- modules/proxy/mod_proxy_http.c  (revision 355728)
+++ modules/proxy/mod_proxy_http.c  (working copy)
@@ -1481,12 +1481,16 @@
 }
 else if (rv != APR_SUCCESS) {
 /* In this case, we are in real trouble  
because

- * our backend bailed on us, so abort our
- * connection to our user too.
+ * our backend bailed on us.
  */
 ap_log_cerror(APLOG_MARK, APLOG_ERR, rv, c,
   "proxy: error reading  
response");

-c->aborted = 1;
+r->no_cache = 1;
+e = apr_bucket_eos_create(c->bucket_alloc);
+e->data = &rv;
+APR_BRIGADE_INSERT_TAIL(bb, e);
+ap_pass_brigade(r->output_filters, bb);
+backend->close = 1;
 break;
 }
 /* next time try a non-blocking read */
Index: modules/cache/mod_disk_cache.c
===
--- modules/cache/mod_disk_cache.c  (revision 355728)
+++ modules/cache/mod_disk_cache.c  (working copy)
@@ -1010,7 +1010,7 @@
  * sanity checks.
  */
 if (APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(bb))) {
-if (r->connection->aborted) {
+if (r->connection->aborted || r->no_cache) {
 ap_log_error(APLOG_MARK, APLOG_INFO, 0, r->server,
  "disk_cache: Discarding body for URL %s "
  "because connection has been aborted.",

Re: AW: 2.2 mod_http_proxy and "partial" pages

[Roy T. Fielding]

On Dec 8, 2005, at 9:21 PM, Justin Erenkrantz wrote:
Even with an EOS bucket, how will we indicate that the connection  
should be

aborted?  (i.e. don't try to do any further writes?)


The inbound connection hasn't been aborted -- only the outbound.
We don't even want to abort the inbound connection until the client
has received as much of the response as we received, since that
will help the client diagnose the error.  The only thing we need
to do is invalidate the cache entry, which would be done when
mod_*_cache checks the EOS bucket (which it already does)
for an error (the new feature).


I would extend the EOS bucket data to be an errno and then have
mod_cache check for that data != 0 when it does its EOS check.


For httpd's filters, an EOS bucket data doesn't attempt a close of the
stream: in fact, EOS doesn't do anything to the socket.  By the  
time we
start writing the body, all of the filters that know HTTP are long  
gone.
(The only thing that might be left is a chunking output filter.)   
Plus,
with apr's bucket types, there is no mechanism to associate that  
type of

error condition.


No, but we can graft it onto the currently unused EOS bucket data.


I am surprised it wasn't implemented that way in the first place.


I am too, but I'd like to be able to resolve this veto without  
radically

having to redesign filters in httpd.  =)


Me too -- I'll see what I can come up with during the hackathon.
When I looked at it earlier, a fix seemed feasible with only two
lines of code plus whatever log messages are desired.

FWIW, serf doesn't have these sorts of problems; that's one of the  
(many)

things we corrected with serf's filter design.  -- justin


Yes, but will it be done before waka? ;-)

Roy

Re: AW: 2.2 mod_http_proxy and "partial" pages

[Roy T. Fielding]

On Dec 8, 2005, at 6:58 AM, Justin Erenkrantz wrote:


On Wed, Dec 07, 2005 at 06:16:42PM -0800, Roy Fielding wrote:

Setting the inbound c->aborted within a filter just to indicate
that the outbound connection has died is wrong -- it prevents
other parts of the server (that are working fine) from correctly
handling the bad gateway response.


FWIW, it's not a filter - it's the handler itself that is setting
c->aborted.


The handler is just the top of the filter stack.  The point is that
it doesn't know anything about the other end (the c connection) and
thus can't know if it should be aborted.  waka, for example, has an
abort mechanism on the wire and thus would be horribly abused by
such a workaround.

If you have another idea how to specify via the handler that the  
connection
needs to be dropped, I'm all ears.  But, I couldn't see one.  --  
justin


I would extend the EOS bucket data to be an errno and then have
mod_cache check for that data != 0 when it does its EOS check.

I am surprised it wasn't implemented that way in the first place.

Roy

Re: AW: 2.2 mod_http_proxy and "partial" pages

[Roy T. Fielding]

On Dec 7, 2005, at 8:44 AM, Plüm, Rüdiger, VIS wrote:

Nope, that's the flag we set when we want the core to drop
the connection. I thought that it would be set by the filters
when a connection was dropped, but, as I said earlier in this
thread, I'm wrong.  The filters will never ever set it.  -- justin


Ok. Then I withdraw my objections against the setting of c->aborted.
I just understood its purpose wrong. Thanks for clarification.


No, you understood its purpose correctly.  I have no idea what Justin
is talking about -- maybe the proxy's connection to the outbound server?
c->aborted is only set on the inbound connection when a previous write
to that connection has failed.

Setting the inbound c->aborted within a filter just to indicate
that the outbound connection has died is wrong -- it prevents
other parts of the server (that are working fine) from correctly
handling the bad gateway response.

-1 VETO

Find another solution before the next release.  I have no objection
to such a workaround as a patch, but we don't introduce new spaghetti
code just to fix a trivial problem.

Roy

Re: 2.2 mod_http_proxy and "partial" pages

[Roy T. Fielding]

On Dec 6, 2005, at 12:45 PM, Brian Akins wrote:
That would work, I suppose, but wouldn't the client get the same  
impression "proxy is broken"?  I generally only deal with "reverse  
proxies," so if origin is broken, whole site is broken...


Sorry, reverse proxy is not a proxy -- it is a gateway.

You are right that it won't make a difference in the gateway case,
though from a software design perspective, pipe-and-filter architectures
depend on sending errors down the pipe so that each filter can handle
the error.  Having the filter set a global variable that is normally
private to the client connection, just for the sake of obtaining the
side-effect desired specifically for HTTP and mod_cache, is, well,
unclean.

Roy

Re: 2.2 mod_http_proxy and "partial" pages

[Roy T. Fielding]

On Dec 6, 2005, at 9:38 AM, Justin Erenkrantz wrote:


On Tue, Dec 06, 2005 at 12:22:18PM -0500, Brian Akins wrote:

There *might* be a breakage if the server aborted it's half of the
connection partway through the response.  I don't have the time  
to fully
look at the code, but there might be a code path that does so.   
-- justin


From what I can tell, stock proxy and cache is vulnerable to that.


After a bit more of thinking, the right thing to do would be to have
mod_proxy force a dropped connection to the client.  Since the backend
bailed on us, there's no way for us to finish the response - we've  
likely

already guaranteed the headers, but have no way to keep the connection
'correct' with respect to the metadata we've already sent.  We must  
bail.


That would depend on the current state of the proxy's response to
the client, since it may have not even sent the status code on the
wire.  If not, then marking the connection as aborted will just make
it look like the proxy is broken, so we should send a 503 error
message instead.

If the response code has already been sent on the wire, then HTTP
has no way to communicate the break other than body != C-L.
Note, however, that waka does have such a mechanism, so assuming
that the entire chain is broken is incorrect  Instead, send an error
bucket down the chain and have the proxy mark the connection as bad
after sending the last good data packet.

Roy

Content-Length or Transfer-Encoding on HEAD responses

[Roy T. Fielding]

On Dec 5, 2005, at 10:19 AM, William A. Rowe, Jr. wrote:


William A. Rowe, Jr. wrote:
But the obvious tangle becomes - do we return T-E chunked - and is  
that

a valid header as a HEAD response with no body?


From 2616 4.3,

   For response messages, whether or not a message-body is included  
with

   a message is dependent on both the request method and the response
   status code (section 6.1.1). All responses to the HEAD request  
method
   MUST NOT include a message-body, even though the presence of  
entity-

   header fields might lead one to believe they do.

I read this that the T-E entity-header field should be forwarded.


I disagree. T-E is a general-header field and must not
be provided if there is no message-body (see section 4.4).

   1.Any response message which "MUST NOT" include a message-body (such
 as the 1xx, 204, and 304 responses and any response to a HEAD
 request) is always terminated by the first empty line after the
 header fields, regardless of the entity-header fields present in
 the message.


Under 4.4

   2.If a Transfer-Encoding header field (section 14.41) is present  
and

 has any value other than "identity", then the transfer-length is
 defined by use of the "chunked" transfer-coding (section 3.6),
 unless the message is terminated by closing the connection.


Which is obviously conditioned on number 1 above it.

But the only way we know we will send the same T-E and C-L values  
is to

determine the C-L as a result of piping the results through the output
filters to the network stack.  Your patch suggests we should ignore  
that

potential issue.


That is completely irrelevant to forwarding a HEAD response.

Roy

Re: svn commit: r351702 - /httpd/httpd/branches/2.2.x/Makefile.win

[Roy T. Fielding]

On Dec 2, 2005, at 5:24 AM, William A. Rowe, Jr. wrote:


With this single change, there is now a source package available at;

  httpd.apache.org/dev/dist/httpd-2.2.0-win32-src.zip

However, without this single change it was impossible for nmake -f  
to run

to completion.

Is it unreasonable to publish this .zip if we deposit this patch  
under the
general /dist/httpd/patches/apply-to-2.2.0/ tree for those working  
from
a tarball?  If not, I'll scuttle (or leave in place for folks  
looking for
an unreleased candidate to examine), and we'll start all over at  
2.2.1 with

the initial release.


Go ahead and publish this package under the 2.2.0-win32 name.  The
changes fall within the scope of platform-specific ones that we allow
third-party package maintainers to make, so there's no reason we can't
do the same.  Yes, please deposit the patch as well.  Thanks,

Roy

Re: [vote] 2.2.0 tarballs

[Roy T. Fielding]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Nov 28, 2005, at 11:55 PM, Paul Querna wrote:


Available from:
http://people.apache.org/~pquerna/dev/httpd-2.2.0/

Please vote on releasing as GA/Stable.


+1 for release as 2.2.0.  I have verified the signatures, compared
the contents, diffed versus the post-tarball httpd-2.2 changes,
and tested on OS X 10.4.3 using Xcode 2.2.  You can add the following
signatures to the asc files if you like, minus the indent, assuming
you can verify them afterward).

httpd-2.2.0.tar.bz2.asc:
  -BEGIN PGP SIGNATURE-
  Version: GnuPG v1.4.2 (Darwin)

  iD8DBQBDjqBqW5aAEOBPmokRAmt6AJ0VpGAZAeSw6sDKp/NftIVFFeaF4ACeL+CB
  Ljif/NCrZXEpktnVTt3uCPs=
  =mtfD
  -END PGP SIGNATURE-

httpd-2.2.0.tar.gz.asc:
  -BEGIN PGP SIGNATURE-
  Version: GnuPG v1.4.2 (Darwin)

  iD8DBQBDjqBXW5aAEOBPmokRAlXHAKCUE/dZdsXikx11CuYkBoCr28WYqgCfYuTv
  +qPQnF9uCcmV+n/ZUFW8jTo=
  =HJ0S
  -END PGP SIGNATURE-

Cheers,

Roy T. Fielding<http://roy.gbiv.com/>
Chief Scientist, Day Software  <http://www.day.com/>

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2 (Darwin)

iD8DBQFDjqMxW5aAEOBPmokRAiouAKCH5He8ly1ngnyUcM+CB2F6tTjpHgCdHfzy
+XMSmJzDWHeA4GI9K09w+Fs=
=SNvw
-END PGP SIGNATURE-

Re: [vote] 2.2.0 tarballs

[Roy T. Fielding]

On Nov 30, 2005, at 10:12 PM, William A. Rowe, Jr. wrote:


I'm 100% conviced next to nobody on this list has been developing  
and testing
httpd-2.2/apr-1.2 without their own in-tree tweaks.  I'm as guilty  
as anyone.


So we've been compiling and improving the code, but the build/ 
install status
is -worse- than httpd-2.0, ergo this is not "the best version of  
apache now

available" and is -not- ready for GA.


I just built from scratch using the tarball and the same options
that any typical user would set: i.e.,

  ./configure --prefix=/dist/test22 --enable-modules=all

Zero problems.

I don't understand what you are talking about -- developers don't
run ./buildconf on the source package.  Only we do that.  Even after
I do a

  make extraclean
  ./buildconf
  ./configure --prefix=/dist/test22 --enable-modules=most

again I have zero problems.  The included versions of apr and apr-util
are used in all of my tests.  I've never installed apr-1.x in the OS
system libraries.  Why would anyone outside this list do that?

Those hyper to release, why not make it usable by Joe anybody  
instead of only

httpd-dev hacker who's used to 'fudging the build'?


Whatever problem you are encountering, please fix it on trunk.

I see no evidence that it will cause people outside the APR core
development group any grief for httpd-2.2, and even then a workaround
can be described on the website.

Roy

Re: svn commit: r349582 - /httpd/httpd/tags/2.2.0/

[Roy T. Fielding]

On Nov 28, 2005, at 8:07 PM, Justin Erenkrantz wrote:


On Mon, Nov 28, 2005 at 07:54:24PM -0800, Roy Fielding wrote:

What rules are you talking about?  GA just means it isn't alpha or
beta -- it
has nothing whatsoever to do with the version number.  2.2 is now our
STABLE
branch, not our GA branch.


See VERSIONING.


I've seen it.  2.2.0 is our stable branch.  I believe the vote  
indicates that
the 2.2.0 release should match the code in 2.1.10 modulo the changes  
necessary
to bump the version and update the documentation.  However, nobody  
bypasses
our release voting procedure just because they committed some  
document in CVS.

Release votes are on completed, verifiable, signed source tar balls.


What documentation are you talking about?


  ABOUT_APACHE
  CHANGES
  docs/manual/*
  INSTALL
  LAYOUT
  README
  */*/README
  include/ap_release.h
  include/ap_mmn.h (probably okay as is)

and that is just what goes inside the tarball.  We also have to  
update the
STATUS files, the website (I see Paul has started that), create  
docs-2.2,

and all of the other things people will remember as soon as we cross the
tarball threshold.

I mentioned those on our trip to Wells Fargo last week.  I would have  
fixed

them myself by now, but I made the mistake of updating my OS X 10.3.9 to
10.4.3 and Xcode 2.2 first, which led to three wasted days trying to fix
lame fink bugs instead.


In any case, we vote on complete source tarballs, not some
expectation of
a tag.  There can't be any 2.2.0 release votes yet because it doesn't
exist
as a released tarball, nor can it exist until we have updated the  
docs.


Yes, and we voted on the tarball contents which would be identical to
2.1.10.  The only change is the release number inside ap_release.h.
I don't consider that a material change that would alter my vote.


So if I were to go the website and cp 2.1.10.tar* 2.2.0.tar* you
wouldn't change your vote on that copy?  You wouldn't mind that the
tarball has ap_version wrong, the docs were last updated in 2002,
and we would need to do a 2.2.1 to clean that trivial stuff up?

I am not asking you to change your vote once a 2.2.0 tarball is created.
I am telling you that you can't vote on 2.2.0 until a tarball is created
that calls itself 2.2.0 and thus is available for review.  I refuse to
believe that anyone on the PMC has reviewed a release package that  
hasn't

even been packaged yet.

Feel free to veto any technical change that would cause 2.2.0 to differ
significantly from 2.1.10.  Feel free to make your vote on 2.2.0 on the
basis of a recursive diff between the two tarball packages.


The majority of votes cast were for GA.  At least to me and Rich (and
likely others), that implied 2.2.0.  A few people (but not enough to
alter the majority) specifically said that they would not vote for
2.2.0.  -- justin


The only vote I saw was a 2.1.10 release vote and a statement of  
intention
to make 2.2.0 match the code in 2.1.10.  That does not imply 2.2.0 is  
being
released based on that vote, which is patently absurd.  If you think  
anything

in our guidelines implies such a thing, then please point it out so that
I can delete it.

Roy

Re: svn commit: r349582 - /httpd/httpd/tags/2.2.0/

[Roy T . Fielding]

On Nov 28, 2005, at 6:13 PM, Paul Querna wrote:


Roy T. Fielding wrote:

On Nov 28, 2005, at 5:53 PM, [EMAIL PROTECTED] wrote:


Author: pquerna
Date: Mon Nov 28 17:53:31 2005
New Revision: 349582

URL: http://svn.apache.org/viewcvs?rev=349582&view=rev
Log:
Tag 2.2.0 from 2.1.10 tag to prepare for the 2.2.0 release


What?  No, sorry, -1.  Your vote was on releasing 2.1.10.  You  
should be
releasing 2.1.10 to the public as GA.  2.2.0 is different -- it  
requires

new documentation and a new tarball, and thus new votes.


Our rules also say that we cannot release 2.1.10 as GA.  The next  
GA release should be called 2.2.0.


What rules are you talking about?  GA just means it isn't alpha or  
beta -- it
has nothing whatsoever to do with the version number.  2.2 is now our  
STABLE

branch, not our GA branch.

I don't intend to release a 2.2.0 tarball without another vote.   
They will be identical to the 2.1.10 tarballs, except for the  
version number. (give me a few more minutes and they will be posted).


Please don't do that.  Change the version to 2.0.0-dev and then give  
people
like me time to go in and edit the documentation (which is kinda hard  
for

me to do at the moment because fink blew away my svn install and I can't
build it any more).  We have two weeks before ApacheCon, so there is  
no rush.


In any case, we vote on complete source tarballs, not some  
expectation of
a tag.  There can't be any 2.2.0 release votes yet because it doesn't  
exist

as a released tarball, nor can it exist until we have updated the docs.

If you have a better way to manage the once-every-3 year changeover  
from development to stable branches, please share it.


This is exactly what I said I would do in the [vote] thread for  
2.1.10.


No, it isn't -- you said that it was a vote to release 2.1.10.  I  
assumed

that meant you were going to bump the version number in CVS.  There were
several people who said they were +1 on 2.1.10 and NOT 2.2.0, and our
voting guidelines have never allowed a release vote to take place before
the release was even prepared.

Roy

Re: svn commit: r349582 - /httpd/httpd/tags/2.2.0/

[Roy T. Fielding]

On Nov 28, 2005, at 5:53 PM, [EMAIL PROTECTED] wrote:


Author: pquerna
Date: Mon Nov 28 17:53:31 2005
New Revision: 349582

URL: http://svn.apache.org/viewcvs?rev=349582&view=rev
Log:
Tag 2.2.0 from 2.1.10 tag to prepare for the 2.2.0 release


What?  No, sorry, -1.  Your vote was on releasing 2.1.10.  You should be
releasing 2.1.10 to the public as GA.  2.2.0 is different -- it requires
new documentation and a new tarball, and thus new votes.

Roy

Re: Need for a Release Candidate status

[Roy T . Fielding]

On Nov 22, 2005, at 5:35 AM, Jim Jagielski wrote:


Wayne S. Frazee wrote:


"The Apache HTTP Server Project is proud to announce the release of 
version

2.1.9-beta of the Apache HTTP Server ("Apache").

This version of Apache is a Beta release of the unstable development 
branch.
New features include Smart Filtering, Improved Caching, AJP Proxy, 
Proxy
Load Balancing, Graceful Shutdown support, Large File Support, the 
Event

MPM, and refactored Authentication/Authorization."

A little bit of beta or emphasis would go a long way.  People tend 
not to


Yep... instead having it as 2.2.0 RC1 would really make people sit
up and take notice. Plus, as I've stated before, it adds a good
procedure to our evolving release structure.


In a word, no.  We have a release structure and it is working just
fine right now.  I don't see any need to fiddle with it by adding
yet another level of version identifiers.

Roy

Re: Policies on binaries?

[Roy T. Fielding]

On Nov 18, 2005, at 11:39 AM, William A. Rowe, Jr. wrote:


Roy's raised the recent issue that all ASF releases under his domain
(httpd project chair) should be reviewed -by at least three pmc 
members-

as well as the contributors and committers.

This raises an obvious question; are binary distributions releases?


No, they are packages.

Or are the merely repackaging the source code that was already 
approved?

It seems none of the platform binaries are reviewed today.


They are only signed by the person making them.  I don't see any
reason why we need to vote on binary packages if we trust the person
who signs them to have made the package from the released source.
We don't get to vote on distro packages either.

Roy

board report for HTTP server project

[Roy T . Fielding]

Much to my surprise, I apparently have a board report due yesterday
for this Wednesday's board meeting.  Do we have any ASF issues that
need reporting to the board, aside from what is in STATUS*?
Any choice commentary?  Does anyone else feel like we have too many
dev lists for one project?

Roy

Re: cache trouble (Re: [vote] 2.1.9 as beta)

[Roy T. Fielding]

On Nov 7, 2005, at 3:10 PM, Ruediger Pluem wrote:
Not for every page, but if I get it right once you lock out one bad 
boy via



deny ipaddress

than it should be sent. AFAIK this not done automatically currently 
once you add a deny

directive somewhere. Does this need to be changed?


I can't remember which directive applies where, but if the
access control is set to deny all and allow some, where some
is a locally restricted subset of all, then cache-control
private is required on non-error responses unless the request
included Authorization (in which case cache-control private
is optional because it is already implied with Auth).

If the directive is set to allow all and deny some, then
it is reasonable to assume that the access control is for
service reasons, not authentication, and thus anyone who
receives the message should be allowed to cache it for others.

It would be wise to make both configurable.

Roy

Re: cache trouble (Re: [vote] 2.1.9 as beta)

[Roy T. Fielding]

On Nov 7, 2005, at 3:03 PM, Ruediger Pluem wrote:

Just checking if I understood things correctly:

If I have a forward proxy to which I limit access via IP based access 
control
I should add Cache-Control: private to any response I get back from 
the backend

(either a Remote Proxy or the origin server).


No, access control on a forward proxy has nothing to do with
cache-control.  Cache-control is defined by the origin server.

This response would not be cached by mod_cache unless I overwrite it 
with

CacheStorePrivate on.

If I set CacheStorePrivate to on the reponse gets cached by mod_cache,


Yes, though I would hope that you would set that within a
Location directive specific to a given set of URIs.


but the next request for this (fresh) resource will not check the
access control and
deliver it to any client, regardless of the IP. Correct?


The forward proxy would deliver it to any client that had the
ability to GET from that proxy.

Roy

Re: cache trouble (Re: [vote] 2.1.9 as beta)

[Roy T. Fielding]

On Nov 7, 2005, at 2:09 PM, Ruediger Pluem wrote:

The problem is that without Cache-Control: private, any downstream 
cache

would have the exact same problem.  There's no way for it to know that
the response differs based on IPs unless the Origin says so.  -- 
justin


This is true. But in the case of a forward proxy that is used to give 
office users access to the internet in general based on there IP this 
is no problem.


Then either the forward proxy has an external agreement with the
source (and can override the cache-control) or it has no clue
about the source and cannot safely cache the content.  In any
case, the messages that we send must be correctly marked as
private because that is our configuration.

If the forward proxy sends a request that has no Cookie and
no Authorization, then it is likely that a cache-control private
response is indicative of IP-based access control.  If you want
to code up a forward ProxyCache override based on that, go wild.

Roy

Re: cache trouble (Re: [vote] 2.1.9 as beta)

[Roy T. Fielding]

On Nov 7, 2005, at 1:01 PM, Paul Querna wrote:

If there is a compelling reason to support not adding Cache-Control:
private to authenticated requests, then it's definitely an option, 
but I

think we should default to the safe option for now.


The compelling reason is that this implies that even for the DEFAULT
configuration of apache, we should be sending cache-control private, 
for

EVERY page served.


Why?


This also implies that if we you use mod_rewrite based on any
non-Varied-Header information, you should be setting Cache-Control:
Private too.


No, you should be setting Vary: * if the content varies.  That is
also required by HTTP.

The default in all cases should be HTTP-compliant.  You can define
additional directives for overriding compliance by consent of
the owner, but we shouldn't ship a server that doesn't work
correctly by default.

Roy

Re: cache trouble (Re: [vote] 2.1.9 as beta)

[Roy T. Fielding]

On Nov 7, 2005, at 2:06 AM, Nick Kew wrote:


On Monday 07 November 2005 03:26, Paul Querna wrote:


Cache-control: private

is what should be added for any resource under access control.


I'd prefer something a little less drastic, like 'faking' a header out
of remote-ip.


Why?  All you accomplish is to cause this problem of a downstream
cache not knowing that there is access control.  If you don't want
access control, then don't use access control.

Roy

Re: cache trouble (Re: [vote] 2.1.9 as beta)

[Roy T . Fielding]

On Nov 4, 2005, at 10:56 AM, William A. Rowe, Jr. wrote:

It leaves us wondering; how can allow from/deny from n.n.n.n be mapped 
to
RFC 2616 semantics, or at least, without running the many server hooks 
on
later requests?  The only way I can see, is that we should have any 
more
explicit allow from/deny from leave a marker in the request record 
from that
authorization phase, and mark it nocache if the request doesn't 
otherwise

set the authentication required headers.


Cache-control: private

is what should be added for any resource under access control.

Roy

Re: svn commit: r329849 - in /httpd/httpd/trunk: CHANGES modules/proxy/proxy_util.c

[Roy T. Fielding]

c = ap_strchr_c(url, ':');
if (c == NULL || c[1] != '/' || c[2] != '/' || c[3] == '\0')
   return NULL;


BTW, unless url is somehow limited to http and ftp, the above
is bad code.  The proxy should be able to handle arbitrary
schemes (eventually), which means requiring scheme://host
is bogus.

Roy

Re: [PATCH] PR 18757 Content-Length removed from proxied HEAD responses

[Roy T. Fielding]

On Oct 19, 2005, at 8:35 AM, Greg Ames wrote:
let's say it is a HEAD request for a local static file.  the 
default_handler calls ap_set_content_length which creates a C-L 
header.  but then the body could be run through some length changing 
filter, such as mod_deflate or mod_include, invalidating the 
default_handler's C-L.


Then that filter must remove or modify the C-L.

in that case we want to update the C-L to reflect what's currently in 
the brigade.  so we need to behave differently when there is no body 
at all, like proxy, and when there is a temporary body which will be 
thrown away shortly.


That sounds like spaghetti code to me.  Fix the bug in the filters,
not the side-effects.

+1 on this patch as a temporary fix if you add commentary with it,
but I'd rather have a server that isn't so fragile in the long term.

Roy

Re: [PATCH] PR 18757 Content-Length removed from proxied HEAD responses

[Roy T. Fielding]

On Oct 18, 2005, at 3:58 PM, Greg Ames wrote:


Index: server/protocol.c
===
--- server/protocol.c   (revision 326257)
+++ server/protocol.c   (working copy)
@@ -1302,7 +1302,13 @@
  * We can only set a C-L in the response header if we haven't 
already

  * sent any buckets on to the next output filter for this request.
  */
-if (ctx->data_sent == 0 && eos) {
+if (ctx->data_sent == 0 && eos &&
+ /* don't whack the C-L if it has already been set for a HEAD
+  * by something like proxy.  the brigade only has an EOS 
bucket

+  * in this case, making r->bytes_sent zero.
+  */
+!(r->header_only && r->bytes_sent == 0 &&
+apr_table_get(r->headers_out, "Content-Length"))) {
 ap_set_content_length(r, r->bytes_sent);
 }


Why "&& r->bytes_sent == 0" ?  If it is a HEAD request and C-L
has already been set, then r->bytes_sent should be irrelevant,
especially given ctx->data_sent == 0.

Roy

Re: [PATCH] remove block on OPTIONS from mod_cgi

[Roy T . Fielding]

On Oct 15, 2005, at 12:54 PM, André Malo wrote:
I'm wondering, how many scripts this will break. Perhaps we should 
provide

an option?


It can't break a script.  It could expose a broken script, but then
those scripts are already exposed to all of the other methods.
It is far better to fix the script locally than disable a feature
of HTTP globally out of fear that bugs might exist elsewhere.

In any case, the option is LimitExcept.

Roy

[PATCH] remove block on OPTIONS from mod_cgi

[Roy T . Fielding]

I objected to this code when it was first added (13 Aug 1996)
as part of Alexei's huge HTTP/1.1 patch and for some unknown
reason I forgot to remove it.




It has been featured in multiple PRs since then for both GNATS (4490)
and Bugzilla (PR 15242).  Are there any objections to removing it now,
both here and in mod_cgid?  It is currently preventing folks from
implementing new services (WEBDAV and Atom, among others) via CGI.

Roy


Index: mod_cgi.c
===
--- mod_cgi.c   (revision 321466)
+++ mod_cgi.c   (working copy)
@@ -756,13 +756,6 @@

 p = r->main ? r->main->pool : r->pool;

-if (r->method_number == M_OPTIONS) {
-/* 99 out of 100 CGI scripts, this is all they support */
-r->allowed |= (AP_METHOD_BIT << M_GET);
-r->allowed |= (AP_METHOD_BIT << M_POST);
-return DECLINED;
-}
-
 argv0 = apr_filepath_name_get(r->filename);
 nph = !(strncmp(argv0, "nph-", 4));
 conf = ap_get_module_config(r->server->module_config, &cgi_module);

Re: strange behavior of Redirect on ajax

[Roy T. Fielding]

On Oct 13, 2005, at 7:50 PM, Roy T. Fielding wrote:


I just tried to add

Redirect Permanent /dist/jakarta/tomcat 
http://www.apache.org/dist/tomcat/tomcat


to a .htaccess on ajax and the prefix match/replace didn't work.


Hmm, false alarm -- this looks like an undocumented feature that
was added way back in 0.8.17.  Alias and Redirect only match complete
path segments, not URI prefixes (partial names).

Roy

Re: strange behavior of Redirect on ajax

[Roy T. Fielding]

On Oct 14, 2005, at 12:39 PM, Roy T. Fielding wrote:


On Oct 14, 2005, at 2:55 AM, Joe Orton wrote:


On Thu, Oct 13, 2005 at 07:50:27PM -0700, Roy T. Fielding wrote:

I just tried to add

Redirect Permanent /dist/jakarta/tomcat
http://www.apache.org/dist/tomcat/tomcat

to a .htaccess on ajax and the prefix match/replace didn't work.


What directory level was the .htaccess file placed in?  I can't
reproduce any problems with some simple cases similar to that.


It was in <http://www.apache.org/dist/jakarta/> and ajax is running
2.0.55.  The code doesn't look like it has changed in years.


And the request log shows

www.apache.org 70.183.59.151 - - [14/Oct/2005:04:35:53 +0200] "GET 
/dist/jakarta/tomcat-3/ HTTP/1.1" 301 370 "-" "Mozilla/5.0 (Macintosh; 
U; PPC Mac OS X Mach-O; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6"
www.apache.org 70.183.59.151 - - [14/Oct/2005:04:34:55 +0200] "GET 
/dist/tomcat/tomcat HTTP/1.1" 404 344 "-" "Mozilla/5.0 (Macintosh; U; 
PPC Mac OS X Mach-O; en-US; rv:1.7.10) Gecko/20050716 Firefox/1.0.6"


I'll trace the behavior on my test machine once I remember how
to build httpd 2.0.55. ;-)

Roy

Re: strange behavior of Redirect on ajax

[Roy T. Fielding]

On Oct 14, 2005, at 2:55 AM, Joe Orton wrote:


On Thu, Oct 13, 2005 at 07:50:27PM -0700, Roy T. Fielding wrote:

I just tried to add

Redirect Permanent /dist/jakarta/tomcat
http://www.apache.org/dist/tomcat/tomcat

to a .htaccess on ajax and the prefix match/replace didn't work.


What directory level was the .htaccess file placed in?  I can't
reproduce any problems with some simple cases similar to that.


It was in <http://www.apache.org/dist/jakarta/> and ajax is running
2.0.55.  The code doesn't look like it has changed in years.

Roy

strange behavior of Redirect on ajax

[Roy T. Fielding]

I just tried to add

Redirect Permanent /dist/jakarta/tomcat 
http://www.apache.org/dist/tomcat/tomcat


to a .htaccess on ajax and the prefix match/replace didn't work.
I had to replace it with

RedirectMatch Permanent /dist/jakarta/tomcat(.*)$ 
http://www.apache.org/dist/tomcat/tomcat$1


to get the intended results.

It makes me wonder if all Redirect directives are broken. :(

Roy

Re: 3700 core files for mod_mbox

[Roy T. Fielding]

On that note, we should look for someone else to take over
mail-archives.apache.org.  I'm getting flooded with a bunch of errors
due to Roy removing or renaming some ASF lists.


What errors?  Do you mean 404s on httpd (what I would expect) or
script errors on mbox processing (what I wouldn't expect unless
the mod_mbox scripts are horribly lame).

We should probably add redirects to www's httpd.conf whenever
a public mailing list name changes, but it won't happen until
someone adds documentation about that in apmail/README.

Roy

Re: svn commit: r290519 - /httpd/httpd/trunk/CHANGES

[Roy T. Fielding]

The RFC 2616 says

   The presence of a message-body in a request is signaled by the
   inclusion of a Content-Length or Transfer-Encoding header field in
   the request's message-headers. A message-body MUST NOT be included in
   a request if the specification of the request method (section 5.1.1)
   does not allow sending an entity-body in requests. A server SHOULD
   read and forward a message-body on any request; if the request method
   does not include defined semantics for an entity-body, then the
   message-body SHOULD be ignored when handling the request.

so I have no clue what


+  *) Added TraceEnable [on|off|extended] per-server directive to alter
+ the behavior of the TRACE method.  This addresses a flaw in proxy
+ conformance to RFC 2616 - previously the proxy server would 
accept

+ a TRACE request body although the RFC prohibited it.  The default
+ remains 'TraceEnable on'.  [William Rowe]


claims to be saying.  The proxy server SHOULD accept and forward
the message body and SHOULD ignore it.  Which means the options should
be on|off|rejectbody if you want to configure such behavior.

Roy

Re: svn commit: r265755 - in /apr/apr-util/trunk/ldap: apr_ldap_init.c apr_ldap_option.c

[Roy T. Fielding]

We have no need to support anything other than the latest updates
of Win2k and WinXP.  Anything other than that should not be running
a server and can continue using our old builds if needed.

Roy

status report for ASF board

[Roy T. Fielding]

I need to prepare a status report for the HTTP server project
for the ASF board meeting on Wednesday (IIRC).  Given that
I'll be on vacation for two weeks starting Monday, it would
be nice to have some text ready to send in by Sunday night.

Our last report on 2005_05_18 consisted of

===
   Attachment D: Status report for the Apache HTTP Server Project

   Activity is picking up again.  We are happy to say we have
   invited a number of new committers into apreq, docs, httpd
   and mod_python.

   There were a couple of releases: libapreq2-2.05-dev and
   httpd-2.0.54.

   There is increased interest in branching httpd 2.1 and working
   towards an initial 2.2 release.
===

Since that time, we have made no official changes to the PMC
(that will change very soon) and have added a number of
committers for the Google SoC projects.  Do we have a summary
of the SoC projects and committers somewhere?

We have also cut one alpha release called httpd 2.1.6. Other
releases are pending some fixes in APR.

Sander Striker has decided to step down as chair and VP of the
HTTP server project after having been volunteered into being
President of ASF.  I have been asked to come out of retirement
and be the new conveyer of choice words between this project
and the board.

Should I add anything else?  Could the people who accepted PMC
or committer status in the past three months please remind me
to include your name in the report?  Thanks.

Cheers,

Roy T. Fielding<http://roy.gbiv.com/>
Chief Scientist, Day Software  <http://www.day.com/>

Re: RTC killed the open source project

[Roy T . Fielding]

On Aug 9, 2005, at 11:28 AM, Aaron Bannert wrote:


I'm not talking about 2.2. I'm talking about a severe slowdown in the
pace of development on this entire project.


Less talk, more do.


And it sounds like you've been feeling the effects of all the red tape
too, with being fed up trying to follow through with a release.

There are two separate issues:

* The difficulty in *producing* releases is due to RMs being too afraid
  to make releases that aren't perfect. This phobia is a disease, 
because

  it prevents the release-early-release-often motto from happening.


Yes, but that doesn't prevent you from being the RM.


* The difficulty in *making progress toward a releasable repository* is
  being caused by the STATUS file bottleneck. The STATUS file 
bottleneck

  is caused because we have Review-Then-Commit rules set up, which
  means everyone has to serialize EVERY CHANGE on that file. (You're
  talking to the thread programming guy here, remember? That's a point
  of contention. :)


I still don't know WTF you are talking about.  RTC only applies
to the 1.3 and 2.0 branches right now, neither of which have seen
active development for years because we don't want them to be
actively developed.  The only reason we have a STATUS file is
so that the current RM can keep track of what has been proposed
and reviewed without searching the mail archives.  We could switch
over to Jira and do the same automatically (assuming it didn't
keel over and die from the load), but the STATUS file is supposed
to be the RM's to-do list, not everyone's bag of concerns. STATUS
doesn't prevent anyone from cutting a new release tarball.


RTC came with good intentions (higher quality code), but it's clear now
that it's not working and it needs to change.


RTC is working fine.  What isn't working is the lack of releases
on the CTR branches, which is the only reason people still complain
about slow 2.0 releases (because there are no other 2.x releases).

I can't be RM right now because I am going on vacation next week.
If I were RM, then I would simply arrange the tags such that some
set of revisions worked and cut a tarball.  If apr-util doesn't
work, revert it to a fixed version.  If Netware doesn't work, remove
it from the list of platforms. If win32 can't be built by the RM,
then discard it as irrelevant.  Cut the tarball and vote.  If that
doesn't pass, fix the errors, cut another tarball, and vote.  Keep
going until a release is made.

Roy

Re: RTC killed the open source project

[Roy T. Fielding]

On Aug 8, 2005, at 10:55 PM, Aaron Bannert wrote:


I can't believe you guys are still debating the merits of RTC over CTR
after all this time. RTC killed the momentum in this project a long 
time

ago.


That doesn't make sense, Aaron.  The other branches are CTR and
I don't see anyone making releases of those either.  We were
operating in RTC mode for the first year of Apache and that was
our most productive time in terms of number of releases.

Personally, I think what is killing momentum is the number of
single-developer platforms we claim to support that end up dragging
the release cycle beyond the capacity of any single RM.  That's why
I haven't driven a release since 1.3.  APR just makes it worse.
Please, if you want to help, don't bicker about RTC -- just
create a working APR release so that we can do an httpd release.

Roy

Re: svn commit: r230592 - in /httpd/httpd/branches/2.0.x: CHANGES STATUS modules/proxy/proxy_http.c

[Roy T. Fielding]

On Aug 8, 2005, at 5:24 PM, William A. Rowe, Jr. wrote:


At 06:27 PM 8/8/2005, Roy T. Fielding wrote:

Any change that hasn't been released can be vetoed if a technical
explanation is given.


Roy; I -totally- agree with your position.  However, emails going
back to 1997 of http://httpd.apache.org/dev/voting.html describe
a very specific process; votes are collected, result is announced,
extra +1's are welcome, extra -1's are invalid.


I am sure a lot of people have had an opinion on that since the
voting rules were finished in 1995.  However, the only opinion that
has ever been agreed to by the group as a whole is that a veto
is a veto and the only change it can't undo is a released change.

In particular, the voting.html file talks about a process in
which we were preparing releases once per week -- the result
of a vote meant a new release is done.  It doesn't say that
explicitly because it was assumed at that time, so the part
that you are reading about votes being invalid after the patch
selection process is done should be read as "after the release
is done".  There is a reason why the file is called stale.

I am all for cleaning up the documentation.  I am not inclined
to change our process.


The voting.html is called out as a stale document.  Unfortunately,
the newer and maintained http://httpd.apache.org/dev/guidelines.html
does not issue an opinion on the matter.

It really seems that if this is the new policy it must be set
to stone.  Two questions in guidelines.html ask;

 * We should clarify under what conditions a veto can be
   rescinded or overridden.

 * Should we set a time limit on vetos of patches? Two weeks?

There ya go; the current guidelines don't spell it out.


No, they spell it out quite fine.  The decision on both questions
was "no".  I don't know why the questions were left in the file
after it was approved.

Roy

Re: svn commit: r230592 - in /httpd/httpd/branches/2.0.x: CHANGES STATUS modules/proxy/proxy_http.c

[Roy T. Fielding]

Bill, if you spent more time making your changes understandable
by documenting what they change instead of various random things
totally unrelated to each patch, then maybe people like me could
review them.  Also, it would help a great deal if you would make
a complete set of changes locally, compile and test them locally,
and then present them as a set of logical change sets that achieve
a specific purpose.  Right now I am not even bothering to review
your commits because they are obviously untested and consist of
patch-upon-patch-upon-patch fixing what can only be described as
sloppy programming.

For the 2.0 branch, the changes should fix a very specific bug.
There shouldn't be any style changes or variable renaming.
I don't care if it makes it easier to compare with trunk -- I
only want to compare it to 2.0.x-1.  If it helps, generate the
patch against the last 2.0.x release tag instead of the branch.
If necessary, we should back out all the 2.0.x changes and release
a 2.0.x without them.  I do not consider HTTP request smuggling
to be a blocking issue that requires a redesign of the entire
request reading process.  If I did, the answer would be to EOL
the 2.0 branch and move all development off of it.

For the 2.1+ branches, it is imperative that commits be complete
change sets that have been tested on your local machine to solve
the problem you are trying to fix.  Some people do that by adding
to the test suite during the process of every change.  When I see
more than three commits per change set, I know that the commits
did not work and somebody is just wasting the group's time by
making everyone review unfinished changes.  I can understand that
in the heat of the moment every once in a long while, but you are
doing it every single time you work on something in httpd.  Just
slow down and make sure the changes are comprehensible before
they are committed and others won't be in such a bad mood when
it comes time to vote on them.

Roy

Re: svn commit: r230592 - in /httpd/httpd/branches/2.0.x: CHANGES STATUS modules/proxy/proxy_http.c

[Roy T. Fielding]

Any change that hasn't been released can be vetoed if a technical
explanation is given.  If you don't have time to fix it immediately,
then the person who committed the change is required to back the
change out to the point where we are back to the prior release code.

If you don't like that, then release more often.  RTC or CTR is
irrelevant to the basic principle that code changes can be vetoed
by any of the core developers.

Roy


On Aug 7, 2005, at 11:15 AM, Joe Orton wrote:


On Sat, Aug 06, 2005 at 06:54:45PM -0500, William Rowe wrote:

At 05:28 PM 8/6/2005, Joe Orton wrote:

That patch went through the normal 2.0.x review process and received
three +1s and no vetoes.  You absolutely cannot come along a few 
months
later and say "oh, actually, -1" and rip stuff out that you now 
decide

you don't like.


It received 3 +1 votes, a slim review.  It was never released,
so it's not in fact 'done'.  If unreleased changes are incorrect,
they need to be fixed, or needs to be reverted.


If you now think the changes are incorrect then you need to go through
the review process to correct them.  We've done this before.


 You missed the chance to veto


How so?

You can't veto a release.  You can veto code; certainly if there
is a 'deadline' it doesn't start until we begin talking about
released code, and that isn't the case here.


No, you can't veto "code".  You vote on *changes to the code*.  That's
what we've been doing for the last N years with 2.0.  That's how the
previous state of the 2.0.x branch was obtained.  Again, if you think
that the tree should be reverted to an older state, then you need to go
through the normal process.

-- if you want to change 
the state of the 2.0.x tree now then you need to go through

the review process like everyone else does.


I'll respectfully disagree, but I have to ask...


You're making a complete mockery of the time and effort expended by
those who maintain the 2.0.x tree.  Please restore the 2.0.x tree to 
the

state which was attained through the normal voting process by the
committers, and stop arguing the toss.  Then follow the process like
everyone else does to try and move *forward*, not backward.

I hope I speak for all the committers here.  If anyone thinks this
request is out of line, please speak up.


Why do you bring this up now when I mentioned that I had vetoed
the change a good three weeks ago, in STATUS, and advised on
list that it would be reverted?


Because you putting random crap in STATUS is meaningless.  The R-T-C
process under which the 2.0.x tree is maintained is not.

joe

Re: NTLM HTTP Authentication is insecure by design - a new writeup by Amit Klein

[Roy T. Fielding]

On Jul 18, 2005, at 12:30 PM, William A. Rowe, Jr. wrote:

NTLM HTTP Authentication
 (and possibly other connection-oriented
 HTTP authentication and authorization protocols)
 is insecure by design


Yep, no shit -- that's what the Microsoft fools were told when
they introduced it.


*) The web server (IIS/6.0) must receive a Via-less request. The
Microsoft implementation assumes that the Via header is always sent
by a proxy server, and this is indeed mandated by the HTTP/1.1 RFC
2616 (http://www.ietf.org/rfc/rfc2616.txt), section 14.45:
 The Via general-header field MUST be used by gateways and proxies
 to indicate the intermediate protocols and recipients between the
 user agent and the server on requests [...]


Yep.


However, it seems that not all servers adhere to this standard. For
example, Apache 2.0.54 mod_proxy does not generate a Via header by
default (see the ProxyVia directive -
http://httpd.apache.org/docs-2.0/mod/mod_proxy.html#proxyvia, yet
the default httpd.conf file contains a commented-out "ProxyVia On"
directive, so it's possible that many Apache proxy deployments do


WTF?  That's a bug.


*) Proxy vendors ­ do not to share TCP connections to the server
among several clients. Yes, it improves performance, but it's also
insecure and enables/aids 3 different attacks (the one described
here, HTTP Request Smuggling and HTTP Response Splitting).
Also, comply to the RFC and send the HTTP Via request header by
default (Apache Group - please take note).


As reverse proxy is never enabled without intent, the impact of
Apache on this vector is very low (note that Amit deliberately
introduced this into his reproduction case) - but anyone who has
intentionally used Apache as a reverse proxy to protect sensitive
IIS servers behind their DMZ using NTLM auth is vulnerable (as are
users of various NTLM Apache auth modules sitting behind Apache
reverse proxies.)


We don't really need to get in between their gun and their foot.


My thinking is that rather than 'revealing' the reverse proxy
origin server, we should be dumping the auth headers if they are
NTLM.


I don't think that will work from the browser's perspective.
I would just mark the connection as being in use by a single
client and remove it when the client closes.

Roy

Re: Hackathon Attendees

[Roy T. Fielding]

Yeah, no kidding.  We haven't made any decisions yet.  Decisions
are what happens when people vote on releases or changes.  Did one
of those go by and I didn't notice?  Nope.


Yup - branching 2.1.x - 2.2.x - trunk.


No, branching is part of the task of an RM.  It is not a decision
of the group, nor is it subject to vote, for the same reason we
do not allow people to veto releases.  As long as the RM is doing
it for a good reason (i.e., preparing a tarball, not just dicking
around), then we want them to use the version management tools the
way that they are intended to be used.

Nobody is forced to support the new branch.  Nobody is forced to
vote to release it.  It doesn't exist for the public until we vote
to release it, so just keep working on what you are doing and ignore
the dude behind the curtains.  At least until he asks us to make
a decision.

Roy

Re: svn commit: r219520 - /httpd/httpd/branches/2.2.x/

[Roy T. Fielding]

Thanks Paul, you just collided with the refactoring of 2.1.x proxy.


That isn't possible.  Trunk is still trunk.


Thank you for not announcing this first.


That isn't necessary.


Thank you for ignoring the entire user community in your zeal
to move forward with ... what?


Preparing a tarball for later vote, as requested.


Thank you for helping the entire dev community begin refactoring
httpd 2.3 long before there is a 2.2 release to begin committing
... what?


Fuck refactoring.  You can refactor the code after it is released.
And after that we can create a 2.4 release if necessary.


I see nothing going on in trunk that jeopardizes 2.2, I see alot
of things that are unstable in 2.1-dev, and 2.1-dev is NOT BETA.


It isn't much of anything at the moment, nor is that relevant.


The simplest thing I guess, going forward, since we all have the
right to branch and copy as we will on dev, is to simply svn cp
over your old work every time we feel like it.  There is no way
you are declaring httpd-2.2 C-T-R yet, although that's what 2.2
should be once branched.


Or you could simply keep working on trunk like everyone else
and let releases be made when a tarball gets three +1s.
Version numbers are cheap.  Telling the entire group to stop while
you work on the next big patch is extremely expensive.

Roy

Re: Hackathon Attendees

[Roy T. Fielding]

On Jul 18, 2005, at 10:26 AM, William A. Rowe, Jr. wrote:


Please remember that decisions need to be made on [EMAIL PROTECTED]
Hopefully high-bandwidth discussions will generate some pretty
slick code and solutions.  Although those in the rest of the
world will be lagging a bit, the history of the project, the
decisions and discussion need to continue to occur on [EMAIL PROTECTED]


Yeah, no kidding.  We haven't made any decisions yet.  Decisions
are what happens when people vote on releases or changes.  Did one
of those go by and I didn't notice?  Nope.

Roy

Re: svn commit: r218978 - in /httpd/httpd/trunk: CHANGES modules/proxy/mod_proxy_http.c

[Roy T. Fielding]

  * RFC 2616 says

"All HTTP/1.1 applications MUST be able to receive and decode the
 "chunked" transfer-coding, and MUST ignore chunk-extension 
extensions

 they do not understand."

I read this as "an HTTP/1.1 server must accept "chunked" or quit
reporting it complies with the HTTP/1.1 specification".


No, you should read it as an HTTP/1.1 server MUST be able to receive
and decode the "chunked" transfer-coding, because that's what I wrote.
In other words, it must be able to parse the message.


All of it, except for the preference to RB_STREAM_CHUNKED when,
perhaps, we could be more sub-optimal, falling back on RB_SPOOL_CL.

Many RB_STREAM_CL choices, before, were equally dangerous, and that
C-L == length_read test in the stream_reqbody_chunked() was meant
to exclude future abuse.


We should be sending C-L if the brigade includes the entire message.
CL is always preferred for requests.  The only time we should send
T-E on a proxied request is if we cannot buffer enough of the
received request to know how large it will be.


  * No longer upgrade HTTP/1.0 requests to the origin server as
HTTP/1.1 unless the force-proxy-request-1.1 envvar is set.
[William Rowe]


-1 (veto), since it is a clear violation of

   http://www.ietf.org/rfc/rfc2145.txt

and the intent of HTTP versions.

Roy

Re: [patch 2.0] http body request/response/trace conformance

[Roy T. Fielding]

-1  This is a violation of HTTP:

+if (apr_table_get(r->subprocess_env, "force-proxy-request-1.0")
+ || ((r->proto_num < HTTP_VERSION(1,1))
+ && !apr_table_get(r->subprocess_env, 
"force-proxy-request-1.1"))) {


In all cases except when specifically configured otherwise,
the proxy must send HTTP/1.1 if it supports responses in HTTP/1.1.

Roy

Re: mod_mbox user interface

[Roy T. Fielding]

On Jul 11, 2005, at 9:06 PM, Maxime Petazzoni wrote:

As promised : a first try at XHTML mockup. Comments about DOM
structure and XHTML semantics are welcome !

http://skikda.bulix.org/~sam/ajax/jsless/



2005 |  It looks great except for all the wasted whitespace.
 Jul |  One thing I look for in a mail interface is to be narrow
 Jun |  enough so that the window can be displayed side-by-side
 May |  with a terminal window for cut-n-paste.  You could fix
 Apr |  that by changing the wide mbox list to a simple one
 Mar |  limited to four characters, like the one on the left.
 Feb |
 Jan |  And you can place the number of messages per month in
 |  a roll-over title instead of a table.
2004 |
 Dec |
 Nov |
 Oct |
 ... |

What do you think?

Roy

Re: [VOTE] mod_ftp for HTTP Server Project

[Roy T. Fielding]

I therefore Call A Vote on whether we should support mod_ftp for
inclusion into the Incubator and if we should accept mod_ftp upon
graduation from the Incubator.


+1

Roy

Re: Philosophy, empty body still a request body?

[Roy T. Fielding]

Ditto what Joe quoted.

On Jul 7, 2005, at 10:46 AM, William A. Rowe, Jr. wrote:

Content-Length:

is most definitely invalid according to the grammar.  Although
the grammar doesn't account for

Content-Length: 0

although LWS is obviously assumed by many clients.  I'd suggest
that is a grammar bug :)


What is a grammar bug?  An empty field value for Content-Length
is an error.  All of the other conditions are spelled out in
section 4.4, so I don't understand what needs clarification.

Roy

Re: [Patch 1.3] Strict proxy C-L / T-E conformance

[Roy T. Fielding]

On Jul 5, 2005, at 8:56 PM, William A. Rowe, Jr. wrote:


Attached is the mystery patch [omitted from the last note - whoops].

IMHO we should apply the same to ap_http_filter() in 2.1's
http_filters.c


It looks like a band-aid to me -- how does this module know that
the server doesn't support other transfer encodings?  Wouldn't
it be better to register a set of transfer encoding filters and
then error if none matches?  Oh, never mind, this is for 1.3. +0.

Roy

Re: Philosophy, empty body still a request body?

[Roy T. Fielding]

On Jul 5, 2005, at 1:41 PM, William A. Rowe, Jr. wrote:


RFC2616 says TRACE doesn't accept a body.

Patches I'd committed to 1.3.x and working on 2.1.x (to port
to 2.0.x) enforce this with TraceEnable On.

But what about a 0-byte body?

Content-Length: 0


Is the same as no message body.


Transfer-Encoding: chunked

0


Is not the same as no message body.


both imply a body, with no body to follow.


Nope.  Both imply that the entity body is of zero length.
Only the first one says there is no message body.

Roy

Re: mod_mbox development plan

[Roy T . Fielding]

On Jul 3, 2005, at 4:18 AM, Maxime Petazzoni wrote:

+1. XML has the particularity to describe only the data, as opposed to
XHTML which stores data *and* structure information.


No, XML is a data structuring (mark-up) metalanguage.  The only
difference between arbitrary XML and XHTML is that XHTML elements
have names that are backwards-compatible with HTML presentation
semantics, which provide an appropriate default presentation for
those browsers that do not support CSS.  The field names are placed
in the class attribute instead of the element name.  An XML or CSS
processing engine doesn't care which one is used -- they are
equally expressive and they are both XML.


XML semantics
allows us to represent mailing list data with mailing list data
semantics instead of web page semantics. Given mailing list semantics
in the problem space, you can then transform to any solution space
semantics using XSL.


There are no XML semantics -- it is just a structuring language with
a bit of hierarchical containment.  An RDF/XML interface would
provide some added value, but that's even more of a mess.


According to me, this is the cleaner way of doing things (regarding
the module's source code).


Why don't you just make the elements table-driven?


Don't bother with the DTD -- just be sure it is well-formed.


Of course I will :) Writing a DTD may only be done when we'll have
settled on what we output (and how), and when I'll have time to do so
(or during a boring afternoon, something like this).


DTDs have no useful purpose in XML -- they are not extensible and
do not comprehend namespaces.  If you really want a formalism, then
define a schema (RELAX-NG or XML Schema).

Roy

Re: mod_mbox development plan

[Roy T . Fielding]

On Jul 3, 2005, at 1:29 AM, Paul Querna wrote:

I believe the core goal is separation of data from presentation.


No, the core goal is to provide an ultra-efficient interface to
large mail archives.  The data is already available in mbox form.

The standard reply is to use some kind of template language.  A glue.  
Let

it be PHP, XSLT, ClearSilver, or any of the other hundreds out there.


XSLT isn't a template language -- it is a transformation engine.


Why do we want glue?

I believe that XHTML, no matter how masterfully done, is messy.  Its 
not

a good method to sort data into logical groupings.  Yes, CSS does allow
some separation of the presentation, but I do not believe it is
completely viable for all design desires.

I believe that hand coded (X)HTML inside C source files is wrong.  It
hurts developer time.  Why should I have to recompile my module, stop
and start apache, just to change simple things like a font or the order
of something?


Use CSS.  CSS files are template presentation mechanisms that can
be specialized on a per-site basis regardless of the data sent from
server to browser.  Have a look at WordPress to see how effective
this can be without requiring additional overhead on the server.
They use PHP as a templating mechanism, but most people ignore that
and simply modify the css files to show or hide data as they wish.


This is where the jump to a specific solution from the goal came.  We
could debate the possible solutions for months, even write our own
language, but the simple answer is that none of them are perfect, but I
believe most people will agree that doing raw XHTML even with masterful
CSS is painful.


I don't agree.  It is far less painful than doing XML with browser
negotiation and high-overhead transforms on the server-side.  We
are not talking about an arbitrary data interface wherein the admin
needs to configure the placement and names of data fields. Mail
messages have their own naming mechanism and data items can be
selectively configured for delivery using a simple configuration
mechanism.  The "template" mechanism can be limited to header and
footer files without any loss of generality to the user.


Alternatives welcome. We already have a patch that does XML+XSLT.


Agreed.  Just keep an open mind, try the alternatives, and be prepared
to defend the implementation on the basis of its performance.


Just because we use XSL for some parts, it doesn't mean that we have to
apply it to all output.  Data meant for ajax can easily be passed
through, and not converted to XHTML.


Right, the ajax interface can be a completely separate tree of URIs
that only needs to worry about its particular style of index/threading.


+1 to client-side.


Good Luck.  I don't want to spend the summer testing XSL support on 6 
or

7 different browsers/versions.  This is not even counting alternative
browsers like Lynx, which have no support at all for XSLT on the client
side.


XSL support?  Forget that -- I was talking about CSS and ajax.
XHTML works on all relevant browsers and looks as good as it gets
on Lynx/links.  We only have to make it look pretty on Firefox,
Safari, and Konqueror.  And there is no problem at all if the ajax
stuff only works on Firefox.


Doing this kind of thing server-side gets us to (X)HTML. That is a
better known quantity. I believe there is a better expectation for that
to work on all browsers.

Regardless of how we get there, I think the browser should be
downloading XHTML.


Then it doesn't make any sense to use XSLT.  I wouldn't use a
sledge-hammer for tapping nails.

Roy

Re: mod_mbox development plan

[Roy T . Fielding]

On Jul 2, 2005, at 7:46 PM, Maxime Petazzoni wrote:


The main goal of my SoC project is to enhance mod_mbox's interface by
using newer web development techniques and/or technologies, while
avoiding any noticeable slowdowns.


That's good, but keep in mind the general design principle to
avoid doing things on the server that could be done on the client.


The first thing that has to be done is, of course, get rid of the hard
coded HTML and switch to something more flexible. Two main solutions
are available :

 - a template system such as ClearSilver
 - XML + XSLT


I guess I am having a hard time bridging that leap from goal
to a fairly specific solution.  What are the real advantages of
being flexible on the server?  I mean, as opposed to simply using
a fixed data format with XHTML and class names?


Since I already have a good knowledge of XML and XSLT, I chose these
technologies for mod_mbox's output. Making the necessary changes to
mod_mbox was a good introduction to it's source code, and I already
sent the resulting patch (also featuring email obfuscation).

The DTD is not yet written for mod_mbox's XML output format because
what has currently been done may need changes (I'm far from being an
expert in XML data semantics).


Don't bother with the DTD -- just be sure it is well-formed.


XML output brings about another question : where should the XSL
transformation be done ? Two solutions are available :

 - server side processing with mod_transform. No slowdown thanks to
the XSLT caching feature. This solution has two main drawbacks :

   * The client receives HTML code instead of XML, which will make
additional features such as dynamic interface (with AJAX) difficult or
impossible to implement (since we don't know what the DOM tree will
look like after the XSL transformation, we can't implement DOM dynamic
updates)

   * mod_transform is not part of the HTTPd project yet. It needs to
be compiled, installed and setup separately. Using server side
processing would bring a dependency to an "unofficial" 3rd party
module.


Also, "no slowdown due to caching" is only applicable for request
patterns that involve frequent duplication of a small set.  It doesn't
work that way when the archive contains several million messages,
since the cache hit rate will be too low to compensate for the
transform cost.


 - client side processing : just give the XML output and the XSLT to
the client's browser, and let it manage the transformation itself. On
the one hand, this solution allows nice (and wanted) features that
AJAX could provide ; on the other hand, it requires an XSLT capable
browser (Gecko-based).


This is not a problem with XHTML.


In the light of the above, I personally prefer the client-side
processing solution. Anyway, I've tested and now know how to set up
both of these solutions.


+1 to client-side.


The XML output is a good start, but it's not enough to make mod_mbox's
interface good enough : we need more interactivity, especially when
browsing mail threads. The AJAX (Asynchronous Javascript and XML)
development technique is an interesting solution to this need. By
providing a quick, neat and dynamic interface, it can make the archive
browsing more user-friendly.

I'm currently making some AJAX experiments (I've never used it before,
but I'm here to learn :) in order to figure out what it can do for
mod_mbox and how. I'm working on an interface mockup that I'll soon
submit for review, too.


That sounds like a good plan to me.

Roy

Re: mod_smtpd project planning

[Roy T . Fielding]

On Jun 29, 2005, at 6:09 AM, Ian Holsman wrote:


and where do we put the code?


Wherever you want the code to be, so long as it isn't released
until you get three binding +1s.

This isn't the first time that new people working on httpd-related
projects have been given limited commit access.  I don't give a
rat's ass whether they finish their assigned projects or not.
What I want them to do is become part of our community long enough
to figure out how fun it can be to be part of a smart group of
people working towards a greater goal.

The dev list needs some new blood (and new ideas) and these folks
aren't going to learn anything about real Apache development if
they are stuck out in the suburbs.

Roy

Re: mod_smtpd project planning

[Roy T. Fielding]

On Jun 28, 2005, at 4:20 PM, Paul Querna wrote:

So, to accommodate this, we would need a mod_smtpd space in subversion 
and a [EMAIL PROTECTED] mailing list.


-1.  The point of SoC is to get more people involved in open
source projects, not to encourage the ASF to accept
non-collaborative code bases in exchange for a bit of cash.
Use the main dev list and encourage the students to participate
in the larger project as well as the specific one.

Roy

Re: Rev 2: [PATCH] 1.3 TraceEnable [on|off|extended]

[Roy T. Fielding]

On Jun 23, 2005, at 6:53 AM, William A. Rowe, Jr. wrote:


The patch, in final form, tested and works for T-E with C-L > body,
T-E with C-L < body, C-L only, T-E only and no body.  It correctly
denies proxy TRACE with a body by default, and will deny all TRACE
requests for 'TraceEnable off'.

Votes please, before I invest in patching 2.x?


The correct response code to send is 403 Forbidden, not 405,
since the method is being handled. That will simplify your
patch considerably.

I have submitted a change request for s/MUST/MAY/ in the text of
2616's definition of 405, since the Allow header field should not
be required.

Roy

mod_deflate setting Vary

[Roy T . Fielding]

On Apr 17, 2005, at 1:25 PM, [EMAIL PROTECTED] wrote:
+
+  *) mod_deflate: Merge the Vary header, isntead of Setting it. Fixes
+ applications that send the Vary Header themselves, and also apply
+ mod_defalte as an output filter. [Paul Querna]
   *) mod_rewrite: use buffered I/O for RewriteMap txt: files.  This
  can result in a major performance improvement when the files are
Modified: httpd/httpd/trunk/modules/filters/mod_deflate.c
URL:  
http://svn.apache.org/viewcvs/httpd/httpd/trunk/modules/filters/ 
mod_deflate.c?view=diff&r1=161690&r2=161691
=== 
===
--- httpd/httpd/trunk/modules/filters/mod_deflate.c (original)
+++ httpd/httpd/trunk/modules/filters/mod_deflate.c Sun Apr 17  
13:25:57 2005
@@ -315,7 +315,7 @@
  * the Accept-Encoding, we need to note that we were looking
  * for this header and downstream proxies should be aware of  
that.
  */
-apr_table_setn(r->headers_out, "Vary", "Accept-Encoding");
+apr_table_mergen(r->headers_out, "Vary", "Accept-Encoding");
Is deflate setting Vary here because it is changing the
Content-Encoding, or is this part also done for Transfer-Encoding?
It should not be setting Vary for changes to the Transfer-Encoding.
Roy

Re: RFC: UserDir off by default for 2.1/2.2

[Roy T . Fielding]

IMO, it should be off by default on all httpd versions, just
as the config should default to no access.  Personally, I would
prefer that all of the defaults be set internal to the server
such that a running httpd with an empty status file would only
be capable of responding successfully to "/" with a simple
"You need to configure the server now."  Everything else should
be a 403 or 404 until it is explicitly configured.
+1 on patch.
Roy

Re: So what is the real status of 2.1.x...

[Roy T . Fielding]

On Mar 29, 2005, at 1:36 PM, William A. Rowe, Jr. wrote:
to remove his -1.  Every time I have tried to remove his stated
arguments against going beta (I lost count at 4 different rationales
against beta), OtherBill suddenly presents more arguments as to why
httpd can't enter beta.
Justin, your answer was, with several specific problems to solve,
to shove a tarball down our throats.  Screw that.  Good way
to ensure my -1 your every attempt.
Bill, why don't you just fix whatever it is that you think of as
broken rather than send negative votes?  It can't possibly take
as long as this to patch a bug, so maybe there is no bug to be
patched, or the patch requires too much effort to consider, or
you are simply too busy arguing about it to supply a fix?  Pick
one and solve it -- nobody here is going to stop you from fixing
a bug in APR.
If you think this is a showstopper, then I have no problem removing
win32 from the list of supported platforms for Apache httpd -- in my
opinion, that community of users is too lazy to support their own
platform and we (as a project) cannot continue to rely on one person
to do all of the necessary work to keep it up to date.  I see no
reason to treat Win32 as any more significant than Irix, at least not
until we have a group of dedicated developers who care enough about
that platform to keep up with its never-ending baggage of broken
system libraries and compilers.
It terms of technical issues, APR has no business shipping an
implementation of iconv and I don't care if whatever is in that
directory can't compile -- if it isn't supported, then delete iconv
from APR and let people install it separately.
Meanwhile, please end the paranoid crap you keep shoveling
about Justin doing this or Justin doing that.  He is following
the guidelines and trying to make progress while at the same
time trying to make sense of your cryptic problem reports and
not step on any toes.  At some point we have to stop trying to
reach consensus on every platform and just make progress on
those platforms with bodies to support them.
Cheers,
Roy T. Fielding<http://roy.gbiv.com/>
Chief Scientist, Day Software  <http://www.day.com/>

Re: So what is the real status of 2.1.x...

[Roy T . Fielding]

2.1.3   : Released on  2/22/2005 as alpha.
It had enough votes for beta, but I am not sure on the RM's decision.
It isn't the RM's decision.  A majority vote of the PMC is a decision
to release provided there are at least three +1s.  The RM is just the
person doing the heavy lifting.
Roy

Happy Birthday, we are 10

[Roy T . Fielding]

10 years ago today, the Apache Group decloaked with the creation
of the new-httpd archive and initial accounts on hyperreal.org.
I had the lucky timing of having the first message archived on
the list, though we had actually been talking about what to do
for at least a week before that (sadly, without any archives).
Thanks to everyone for making Apache possible and for continuing
these efforts today.  I wish I had time to wax historical for a
bit and talk about all the great things that happened along the
way, but I have to rush out for another flight (to Boston).
In any case, I am extremely proud of what we have accomplished
and very grateful to have met so many friends along the way.
Cheers,
Roy T. Fielding<http://roy.gbiv.com/>
Chief Scientist, Day Software  <http://www.day.com/>

Re: Problem with redirections

[Roy T . Fielding]

On Feb 19, 2005, at 11:02 AM, Arne Thomassen wrote:
But I disagree with your interpretation of the specification. 14.23 
says
"port number [...] as obtained from the original URI", and 3.2.2 says
"If the port is empty or not given, port 80 is assumed". So the client
"obtained" the port number by "assuming" that the standard port is
meant.
3.2.2 talks about the URI definition for http.  14.23 specifies what
to put in the Host header field.
Additionally, 3.2.3 (URI comparison) says: "[...] octet-by-octet
comparison of the entire URIs, with these exceptions: A port that is
empty or not given is equivalent to the default port for that
URI-reference [...]". So the Apache httpd redirects to an *equivalent*
URI while the Location header field would only allow a redirection to
something "*other* than the Request-URI" (14.30). I read "equivalent" 
to
mean "basically the same", not "other".
Unfortunately, many clients treat those URI as different, and thus
you will encounter some servers that are configured to redirect
to a single canonical form to reduce aliases (aliases have a negative
effect on cache performance and google ranking).
All this is why I originally implemented it this way. Maybe I
misinterpreted the spec. Should it be clarified with an erratum, or am 
I
just completely unreasonable?
The latter -- the spec is clear, you are just reading beyond what
the text says.  More examples would have been better.
As mentioned earlier, the problem only occurred with Apache httpd, not
with other server programs.
That's because Apache httpd is more configurable than other server
programs.
Also, you should be
sending HTTP/1.1 requests, not HTTP/1.0.
The client is able to do this, but I experienced problems with the
HTTP/1.1 implementation of some httpd servers. So the client uses
HTTP/1.0 by default nowadays, and users can configure it to use 
HTTP/1.1
for servers where they want it.
That's bizarre.  On all other browsers, HTTP/1.1 is both faster
and more robust than HTTP/1.0.  And its been at least five years
since I've heard of a broken HTTP/1.1 origin server.  Are you sure
that you aren't testing through a broken firewall or transparent
proxy?  Microsoft complained about HTTP/1.1 until I pointed out
to them that their lame corporate firewall (Gauntlet) was changing
the data stream.  They just never noticed it because HTTP/1.0
does not detect content errors.
Roy

Re: Problem with redirections

[Roy T . Fielding]

On Feb 18, 2005, at 4:51 PM, Arne Thomassen wrote:
sometimes there seems to be a problem with the interoperation of the 
web
browser retawq (<http://retawq.sourceforge.net/>) and the Apache httpd 
-
the httpd sends a redirection response pointing back to the original
request URI, which would cause an infinite loop of redirections.
I just tried it with telnet and can confirm the redirects, though
note that the redirect does not happen if you exclude the :80 or
:443 port information from Host.
It would appear that those sites have been configured to
canonicalize the URI used for access, and thus are directing
your client not to include the port info in Host. RFC 2616
section 14.23 Host:
   The Host request-header field specifies the Internet host and port
   number of the resource being requested, as obtained from the original
   URI given by the user or referring resource (generally an HTTP URL,
   as described in section 3.2.2). The Host field value MUST represent
   the naming authority of the origin server or gateway given by the
   original URL.
Thus, your client is not HTTP-compliant.  Also, you should be
sending HTTP/1.1 requests, not HTTP/1.0.
So there are two questions: Can/will you solve the problem inside httpd
somehow?
No.
And how could I change the request header to avoid the problem
with existing httpd versions?
Yes, don't add the port to Host unless it was present in the URI.
Cheers,
Roy T. Fielding<http://roy.gbiv.com/>
Chief Scientist, Day Software  <http://www.day.com/>

Re: Auth LDAP ssl/tls differences

[Roy T. Fielding]

Does anyone have any references to specifying STARTTLS as part
of the URI to the ldap server?  Any other comments on this patch
before I integrate into httpd-2.1?
No, it should be specified by a configuration option, as suggested
in the PR.
Roy

Re: C-L or T-E: chunked for proxied request bodies

[Roy T . Fielding]

On Jan 1, 2005, at 1:38 PM, Justin Erenkrantz wrote:
Apache 1.3 does not support chunked request bodies.  (All 2.x GA 
releases
do though.)  So, that factors very much in the reason that I don't 
think
we should send chunks by default: all requests  to 1.3.x would receive
some 4xx (413??) error if we tried to pass chunks.  -- justin
Umm, yes it does support chunked request bodies -- I wrote that code
long before 2.0 even started.  Only the CGI module doesn't support it
because CGI depends on knowing the length in advance.
Roy

Re: removing AddDefaultCharset from config file

[Roy T. Fielding]

On Dec 10, 2004, at 4:19 AM, Joe Orton wrote:
My understanding was that the forced default charset *does* prevent
browsers (or maybe, MSIE) from guessing the charset as UTF-7; UTF-7
being the special case as it's already an "escaped" encoding and hence
defies normal escaping-of-client-provided-data tricks.  Is that not
correct?
Yes and no -- it is both the source of the problem and the biggest
reason that we should NOT set charset as a default.
Consider the following two identical content resources, the first
being sent as
 Content-Type: text/html; charset=ISO-8859-15
  http://www-uxsup.csx.cam.ac.uk/~jw35/docs/cross-site-demo.html
and the second being sent with only
 Content-Type: text/html
  http://www.ics.uci.edu/~fielding/xss-demo.html
I've tested the above with all of my browsers.  Safari and MSIE-Mac do 
not
support utf-7 at all.  Firefox (Mac and Win) supports utf-7 but only 
when
manually set (it does not auto-detect utf-7, even when read from a 
local file).

MSIE (Windows), of course, does the least intelligent thing -- it does
not allow users to select utf-7 manually, but does auto-detect and 
interpret
utf-7 if it is read from a local file, or if "auto-detect" is enabled
regardless of the content-type charset parameter -- setting charset has
no effect on MSIE's auto-detect results.  In other words, it
is only at risk for XSS via utf-7 if auto-detect is enabled.

The problem we have created is that AddDefaultCharset causes entire
sites to default to one charset, usually iso-8859-1.  And because it
is set by default (no brains spent thinking about the right value),
it is often set that way even when installed in non-Latin countries
[and there is also a problem in Europe, since iso-8859-15 is where
the euro symbol was added].  As a result, normal users get a higher
frequency of wrong charset declarations in HTTP, for which the only
"standards-compliant" solution short of manually adjusting every
page received is to turn on auto-detect!  In other words, our default
is now causing more users to be vulnerable to utf-7 XSS attacks than
they would otherwise be if we never sent a default charset.
In any case, the only tutorials on cross-site scripting that still
emphasize setting charset is our own (written by Marc) and CERT's
(based on input from Marc).  Those were intended to be temporary
workarounds until folks had a chance to fix the real problems, which
were non-validating scripts that echo untrusted content to users.
After doing another afternoon of research on this one, I am now 
convinced
that AddDefaultCharset does far more harm than good.

Roy

removing AddDefaultCharset from config file

[Roy T . Fielding]

I've looked back at the Jan-Feb 2000 discussion regarding cross-site
scripting in an attempt to find out why AddDefaultCharset is being
set to iso-8859-1 in 2.x (but not in 1.3.x).  I can't find any rationale
for that behavior -- in fact, several people pointed out that it would
be inappropriate to set any default, which is why it was not set in 1.3.
The purpose of AddDefaultCharset is to provide sites that suffer from
poorly written scripts and cross-site scripting issues an immediate
handle by which they can force a single charset.  As it turns out, 
forcing
a charset does nothing to reduce the problem of cross-site scripting
because the browser will either auto-detect (and switch) or the user,
upon seeing a bunch of gibberish, will go up to the menu and switch
the charset just out of curiosity.  The real solutions were to
stop reflecting client-provided data back to the browser without first
carefully validating or percent-encoding it.

To make matters worse, the documentation in the default config is
completely wrong:
# Specify a default charset for all pages sent out. This is
# always a good idea and opens the door for future 
internationalisation
# of your web site, should you ever want it. Specifying it as
# a default does little harm; as the standard dictates that a page
# is in iso-8859-1 (latin1) unless specified otherwise i.e. you
# are merely stating the obvious. There are also some security
# reasons in browsers, related to javascript and URL parsing
# which encourage you to always set a default char set.
#
AddDefaultCharset ISO-8859-1

First, it only applies to text/plain and text/html, in spite of the
convoluted implementation in core.c.  Second, setting a default in the
server config actually hinders internationalization because normal 
authors
don't understand config files.  Furthermore, it causes harm because
it overrides the indicators present in the content. There is some 
argument
to make for doing that to CGI and SSI output for the sake of protecting
idiots from themselves, but not for flat files that do not contain any
generated content.  And the security reasons are not fixed by overriding
the charset anyway -- that just makes it easier for people to ignore the
real problems of unencoded data.  All that is really needed is the
availability of the directive so that *if* a site or tree is subject to
the XSS problem, then the server admins can set a default.

In short, unless someone can think of a justification for the above
being in the default config for 2.x, I will delete it soon and close
the festering PR 23421.
Roy

Re: svn commit: r111386 - /httpd/httpd/trunk/CHANGES /httpd/httpd/trunk/include/httpd.h /httpd/httpd/trunk/modules/http/http_protocol.c

[Roy T . Fielding]

On Dec 9, 2004, at 8:46 AM, Justin Erenkrantz wrote:
--On Thursday, December 9, 2004 11:26 AM -0500 Geoffrey Young 
<[EMAIL PROTECTED]> wrote:

well, I guess it depends on whether the goal is to help (for some 
definition
of help) support official HTTP variants (if indeed that's what 3229 
is), or
just for things we actually take the time to implement fully.
I think it only makes sense for us to have the status lines for the 
things we actually implement.  I'm not going to veto it, but just that 
I think it's foolish for us to add status lines for the goofy 
'variants' of HTTP that we'll never support.  IETF's stamp of approval 
means little as they've produced their fair share of crappy RFCs 
trying to hop on the HTTP bandwagon.
I will veto it.  -1.  I consider 3229 to be harmful to HTTP and do not 
wish to
support it in the current form.  Folks can still implement it with 
extensions
if needed.

Roy

Re: svn commit: r109866 - /httpd/httpd/trunk/modules/loggers/mod_log_config.c

[Roy T . Fielding]

On Dec 6, 2004, at 7:53 PM, Paul Querna wrote:
I do not agree that using writev() would allow multiple children from 
scribbling over each other's log entries.  I have not been able to 
cause this to happen on my local machines.
You might want to consider what happens on all of the not so recent 
operating
systems that run Apache, especially those that don't even implement 
writev.
See what happens when APR_HAVE_WRITEV is not defined to 1.

Roy

Re: svn commit: r109866 - /httpd/httpd/trunk/modules/loggers/mod_log_config.c

[Roy T. Fielding]

-1 (veto) -- the message is copied to a single buffer for the write
because that is the only way to guarantee an atomic append under
Unix without locks, thus preventing multiple children from scribbling
over each other's log entries.  Please revert this change ASAP.
Roy
On Dec 5, 2004, at 12:58 PM, Paul Querna wrote:
Joe Orton wrote:
On Sun, Dec 05, 2004 at 07:05:23AM -, Paul Querna wrote:
Author: pquerna
Date: Sat Dec  4 23:05:23 2004
New Revision: 109866
URL: http://svn.apache.org/viewcvs?view=rev&rev=109866
Log:
mod_log_config.c: Use iovecs to write the log line to eliminate a 
memcpy
IIRC, writev'ing several small blocks to a file is actually generally
more expensive than doing a memcpy in userspace and calling write.  
Did
you benchmark this to be faster/better/...?
I did a local mini-benchmark of write w/ memcpy vs writev... and they 
came out to almost exactly the same on average with small sets of 
data.

-Paul

Re: [PATCH] another mod_deflate vs 304 response case

[Roy T. Fielding]

Quoting "William A. Rowe, Jr." <[EMAIL PROTECTED]>:

> >Okay, but why the next three lines?  Why would Content-Encoding: gzip
> >*ever* be set on a 304?

Because Content-* header fields in a 304 response describe
what the response entity would contain if it were a 200 response.
Therefore, the header field must be the same as it would be for
a 200.  The body must be dropped by the HTTP filter.

Roy

Re: More informative SVN subject line (Re: svn commit: r76284 - apr/apr/trunk)

[Roy T. Fielding]

I happen to agree that the commit messages suck, but the right thing
to do is have a look at the script and suggest a patch on the
infrastructure mailing list.  I would do it myself, but have a paper
to write first.  I also think that placement of the Log text after
the long list of files is obviously broken, and the commit template
does not include prefixes for
   Submitted by:
   Reviewed by:
   Obtained from:
which are really really important.
I don't think that this discussion belongs on every mailing list
that uses subversion -- move it to infrastructure.
Roy

Re: cvs commit: httpd-2.0/server protocol.c

[Roy T . Fielding]

What would make more sense is "Error while reading HTTP request line. 
(remote browser didn't send a request?)". This indicates exactly what 
httpd was trying to do when the error occurred, and gives a hint of 
why the error might have occurred.
We used to have such a message.  It was removed from httpd because too
many users complained about the log file growing too fast, particularly
since that is the message which will be logged every time a browser
connects and then its initial request packet gets dropped by the 
network.

This is not an error that the server admin can solve -- it is normal
life on the Internet.  We really shouldn't be logging it except when
on DEBUG level.
Roy