Re: [PATCH 40026] ServerTokens Off

[Rasmus Lerdorf]

Sebastian Nohn wrote:

I fear that many users of Apache would actually turn off the
Server header for no or for the wrong reasons (which may "harm" our
market share), and therefore I'm -1 on including this patch.


It would not change apaches market share. If you are talking about
netcraft (and similar stats): I personally think, "ego" is a bad reason
for constricting people.


We have had a config option in PHP for years to completely hide the fact 
that a site is using PHP.  I don't think it has hurt us in any way. 
Sure, our Netcraft numbers would probably be higher without it and 
occasionally we see a dip due to some large site turning it off, but 
isn't this all about writing useful software and not about marketing 
gimmicks?


-Rasmus

Re: [VOTE] Apache HTTP Server 1.3.36 Candidate

[Rasmus Lerdorf]

Jim Jagielski wrote:

Please test and vote on releasing Apache httpd 1.3.36

Download from:
http://httpd.apache.org/dev/dist/

Changes:
http://httpd.apache.org/dev/dist/CHANGES_1.3


Works ok on Debian-unstable with the standard set of Debian patches 
applied.  It might be time to roll in this ndbm patch to fix 
mod_auth_dbm and mod_rewrite on Debian and Redhat.


diff -Naurd build-tree.ORIG/apache_1.3.34/src/helpers/find-dbm-lib 
build-tree/apache_1.3.34/src/helpers/find-dbm-lib
--- build-tree.ORIG/apache_1.3.34/src/helpers/find-dbm-lib 
2002-01-22 08:39:15.0 +0100
+++ build-tree/apache_1.3.34/src/helpers/find-dbm-lib   2003-06-29 
09:46:15.0 +0200

@@ -13,16 +13,19 @@
*-linux*)
# many systems don't have -ldbm
DBM_LIB=""
-   if ./helpers/TestCompile lib ndbm dbm_open; then
+   if ./helpers/TestCompile lib db __db_ndbm_open; then
+   DBM_LIB="-ldb"
+   CFLAGS="$CFLAGS -DDB_DBM_HSEARCH=1"
+   elif ./helpers/TestCompile lib db1 dbm_open; then
+   # For Red Hat 7, if not handled by the ndbm case above
+   DBM_LIB="-ldb1"
+   CFLAGS="$CFLAGS -I/usr/include/db1"
+   elif ./helpers/TestCompile lib ndbm dbm_open; then
DBM_LIB="-lndbm"
if ./helpers/TestCompile lib db1 dbm_open; then
# Red Hat needs this; ndbm.h lives in db1
CFLAGS="$CFLAGS -I/usr/include/db1"
fi
-   elif ./helpers/TestCompile lib db1 dbm_open; then
-   # For Red Hat 7, if not handled by the ndbm case above
-   DBM_LIB="-ldb1"
-   CFLAGS="$CFLAGS -I/usr/include/db1"
 elif ./helpers/TestCompile lib gdbm dbm_open; then
 DBM_LIB="-lgdbm"
 CFLAGS="$CFLAGS -I/usr/include/gdbm"
diff -Naurd 
build-tree.ORIG/apache_1.3.34/src/modules/standard/mod_auth_dbm.c 
build-tree/apache_1.3.34/src/modules/standard/mod_auth_dbm.c
--- build-tree.ORIG/apache_1.3.34/src/modules/standard/mod_auth_dbm.c 
2002-03-13 22:05:33.0 +0100
+++ build-tree/apache_1.3.34/src/modules/standard/mod_auth_dbm.c 
2003-06-29 09:46:15.0 +0200

@@ -81,8 +81,15 @@
 #define dbm_fetch sdbm_fetch
 #define dbm_close sdbm_close
 #else
+#ifndef DB_DBM_HSEARCH
+#define DB_DBM_HSEARCH 0
+#endif
+#if DB_DBM_HSEARCH != 0
+#include 
+#else
 #include 
 #endif
+#endif

 /*
  * Module definition information - the part between the -START and -END
diff -Naurd 
build-tree.ORIG/apache_1.3.34/src/modules/standard/mod_rewrite.h 
build-tree/apache_1.3.34/src/modules/standard/mod_rewrite.h
--- build-tree.ORIG/apache_1.3.34/src/modules/standard/mod_rewrite.h 
2002-03-13 22:05:34.0 +0100
+++ build-tree/apache_1.3.34/src/modules/standard/mod_rewrite.h 
2003-06-29 09:46:15.0 +0200

@@ -130,7 +130,14 @@
  * so we also need to know the file extension
  */
 #ifndef NO_DBM_REWRITEMAP
+#ifndef DB_DBM_HSEARCH
+#define DB_DBM_HSEARCH 0
+#endif
+#if DB_DBM_HSEARCH != 0
+#include 
+#else
 #include 
+#endif
 #if defined(DBM_SUFFIX)
 #define NDBM_FILE_SUFFIX DBM_SUFFIX
 #elif defined(__FreeBSD__) || (defined(DB_LOCK) && defined(DB_SHMEM))
diff -Naurd 
build-tree.ORIG/apache-contrib-1.0.8a/mod_eaccess/mod_eaccess.c 
build-tree/apache-contrib-1.0.8a/mod_eaccess/mod_eaccess.c
--- build-tree.ORIG/apache-contrib-1.0.8a/mod_eaccess/mod_eaccess.c 
2001-03-05 13:30:48.0 +0100
+++ build-tree/apache-contrib-1.0.8a/mod_eaccess/mod_eaccess.c 
2003-06-29 09:49:04.0 +0200

@@ -263,11 +263,11 @@
 #include "util_md5.h"
 #include "http_conf_globals.h"
 #include 
-#if defined (__GLIBC__) 
&&  \

-defined (__GLIBC_MINOR__)  &&  \
-__GLIBC__ >= 2 &&  \
-__GLIBC_MINOR__ >= 1
-#include 
+#ifndef DB_DBM_HSEARCH
+#define DB_DBM_HSEARCH 0
+#endif
+#if DB_DBM_HSEARCH != 0
+#include 
 #else
 #include 
 #endif

Would also be nice to get the Debian config.layout section in:

#   Debian GNU policy conforming path layout.

prefix:/usr
exec_prefix:   $prefix
bindir:$prefix/bin
sbindir:   $prefix/lib+
libexecdir:$prefix/libexec
mandir:$prefix/share/man
sysconfdir:/etc+
datadir:   $prefix/lib
iconsdir:  $prefix/share/apache/icons
htdocsdir: $datadir/htdocs
cgidir:$datadir/cgi-bin
includedir:$prefix/include+
localstatedir: /var
runtimedir:$localstatedir/run
logfiledir:$localstatedir/log+
proxycachedir: $localstatedir/cache+

Re: Laying undead myths to rest

[Rasmus Lerdorf]

William A. Rowe, Jr. wrote:

Joseph Dane wrote:

Joshua Slive <[EMAIL PROTECTED]> writes:


In very early versions of the Apache HTTP Server, the
AddType directive was also used to activate
special server-side processing (such as mod_include
or PHP) by assigning "magic" MIME types to files.  This  can create
problems in more recent versions and should be avoided in favor of
using the AddHandler directive.


for the record (not necessarily for the docs) can you expand on the
sort of problems that might arise?


It actually avoids more problems than it creates, consider the 
example.php.txt
file, which if done with AddHandler will always run through the php 
handler,
while if done with mime types will devolve to text/plain through the 
standard

handler (which is what's implied by the filename ordering.)


Right, which is not what the average Joe expects.

Nick, these 's over here were actually around in 1996 when this was 
added and understand very well the difference between AddType and 
AddHandler.  The folks who understand the difference can of course use 
either, but for those who don't, AddType's behaviour is the one people 
understand.  If we asked people to go and change all their AddTypes to 
AddHandler it could actually cause a number of nasty security problems 
so we have no motivation to do that.


-Rasmus

Re: non-detaching httpd

[Rasmus Lerdorf]

Enrico Weigelt wrote:
> Hi folks,
> 
> 
> I'd like to stop httpd from detaching itself, so I can start
> it from init. 
> 
> How can this be done ?

httpd -X  if you want a single non-forking non-detached process
httpd -F  if you just want the main process to be non-detached and still
  have it fork off children

-Rasmus

Re: Post-2.2 Thoughts

[Rasmus Lerdorf]

Paul Querna wrote:
> So, here is my short-list-made-up-this-afternoon of things I would like
> to look at doing after 2.2 is branched/released as GA.  I welcome
> additions too.
> 
> 1) Eliminate the HTTP in HTTPD.  I would like to be able to compile
> httpd with --disable-http.  Currently the 'http core' is half mangled
> with the 'core core'.
> 
> 2) Add a major Protocol Module.  I would like to add a major protocol
> module, besides http, into the mainline distribution.  I believe adding
> a FTP Server Module would be the best one to start with.  Many of the
> paradigms are the same, but it is different enough to help with Goal #1,
> eliminating the dependence on http.

Is there a demand for this, or are we doing this just because we can?
It seems like an odd way to force code cleanup to me.

-Rasmus

Re: Listen-Protocol Branch

[Rasmus Lerdorf]

Paul Querna wrote:
> The Problem:
> 
> We do not know what protocol will be used to handle a connection until
> runtime.  We currently cannot determine this at startup.
> 
> 
> This results in:
> 
> Primary: Accept Filters and TCP_DEFFER_ACCEPT are currently not used
> correctly.  The 'httpready' accept filter is currently applied to all
> sockets.  This means protocols like nntp, where the server must send
> data first, don't work at all. Rasmus said it has the potential to cause
> problems with SSL Servers, but I haven't personally observed this.

On further testing this turns out not to be the case.  The machines I
saw problems on were actually using a modified httpready filter that
fell back to waiting for a newline instead of triggering immediately.
The standard httpready filter triggers immediately when it doesn't
understand the request, so it should degrade nicely for SSL requests.

-Rasmus

Re: Accept Filters, was Re: Timeout for requests

[Rasmus Lerdorf]

Paul Querna wrote:
In real life, I don't believe this is detrimental, since if the
accf_http filter sees data it doesn't understand, it acts just like
accf_data -- and mod_ssl reads the data just like normal.
Hrm..  I am not sure I am convinced of that based on what I have seen on 
some misconfigured Apache1 servers.

There was a thread discussing refactoring of how accept filters and
TCP_DEFER_ACCEPT should be applied, but the root problem is that we do
not know that a socket is SSL, until after we have accept()'ed that
socket.  
Ah, in Apache1 that isn't a problem.  I can see how that complicates 
things in Apache2.

-Rasmus

Re: Timeout for requests

[Rasmus Lerdorf]

Paul Querna wrote:
Rasmus Lerdorf wrote:
Ivan Barrera A. wrote:

I haven't been able to enable acceptfilters on linux. Where can i get a
howto or some info ?

Sorry, I led you slightly astray.  SO_ACCEPTFILTER is for FreeBSD.  In
Linux the option is referred to as TCP_DEFER_ACCEPT and I don't actually
think this is implemented in the public Apache code.  If I get around to
it, I'll submit a patch.  But there should be patches floating around
out there for it.

Ya, I got messed up in my other email too.
SO_ACCEPTFILTER is in 2.0.xx.
TCP_DEFER_ACCEPT is in 2.1.xx.
-Paul
By the way Paul, I have been meaning ask, are you falling back from 
httpready to dataready on SSL requests in 2.x?  I don't see it in 
server/listen.c, but I am not really up on the 2.x code.  We can't use 
httpready on an SSL request for obvious reasons.

-Rasmus

Re: Timeout for requests

[Rasmus Lerdorf]

Nick Kew wrote:
I have some recollection of that problem, but not the solution.  It's
actually somewhat topical for my client right now.  You and Paul have
told us about FreeBSD and Linux; is there also a Solaris equivalent?
(probably not required as they're gradually upgrading it to Linux,
but would be good to have).
As far as I know Solaris doesn't have accept filtering of any sort.  But 
it has been a number of years since I cared about Solaris, so I could be 
wrong.  A quick search didn't turn up anything obvious.

-Rasmus

Re: Timeout for requests

[Rasmus Lerdorf]

Ivan Barrera A. wrote:
I haven't been able to enable acceptfilters on linux. Where can i get a
howto or some info ?
Sorry, I led you slightly astray.  SO_ACCEPTFILTER is for FreeBSD.  In 
Linux the option is referred to as TCP_DEFER_ACCEPT and I don't actually 
think this is implemented in the public Apache code.  If I get around to 
it, I'll submit a patch.  But there should be patches floating around 
out there for it.

-Rasmus

Re: Timeout for requests

[Rasmus Lerdorf]

Nick Kew wrote:
Turn on accept filtering and this problem goes away.  Or at least it
moves to be a kernel-level issue instead of an Apache one.
How does that work with large requests?  Doesn't the whole principle
leave you the choice of just moving the DOS attack or breaking
pipelining?
You mean the httpready filter?  The accept will trigger once the buffer 
is full, so yes, large requests will defeat it eventually, but you still 
get the benefit of not tying up an Apache process until the buffer has 
been filled.  The question was regarding just opening up lots of 
connections and letting them sit there, so the request size didn't 
matter in the context of the question.

And yes, if you have KeepAlive enabled, there is no protection for 
subsequent slow or stalled requests, but there is a KeepAlive timeout 
there.  Most busy sites disable KeepAlive anyway since it is a DoS 
feature in the sense that you tend to get a lot of processes sitting 
around waiting on slow clients.

I did fix an issue last year where even with accept filtering enabled 
you could DoS any Apache server by simply opening MaxClients connections 
and trickling a carriage return to each connection very slowly.  So for 
people seeing DoS issues like this, I would suggest upgrading to the 
latest version, turning on accept filtering and turning off keepalive.

-Rasmus

Re: Timeout for requests

[Rasmus Lerdorf]

Ivan Barrera A. wrote:
How about linux ? how about Windows ? how about (put your favorite OS
here) ?
Linux has SO_ACCEPTFILTER which doesn't trigger the accept until there 
is data, so accept filtering works on Linux too.  Windows?  No idea. 
But I bet an Apache DoS would be the least of your worries there.

Well.. First time i heard about httpready (which looks really nice).
I've been looking for something like this.. how come nobody mention it
before ?
It's in the docs.
And final, why can't apache itself have some decent DoS avoiding feature
? not always there will be 3Ârd party tools to help on that..
It's there in Apache for operating systems that provide mechanisms to 
help.  If you have suggestions for how to do it on other operating 
systems, please send a patch.

-Rasmus

Re: Timeout for requests

[Rasmus Lerdorf]

Ivan Barrera A. wrote:
It doesn't.
How so?  With accept filtering, simply opening up the socket doesn't 
trigger Apache in any way, so how is Apache DoS'ed in that case?  And 
under FreeBSD with the httpready filter you can trickle really slow 
requests over the socket and Apache still won't see anything until there 
is a full request ready to be handled.

-Rasmus

Re: Timeout for requests

[Rasmus Lerdorf]

Turn on accept filtering and this problem goes away.  Or at least it 
moves to be a kernel-level issue instead of an Apache one.

-Rasmus
Ivan Barrera A. wrote:
Hi...
 I'm still fighting (probably for a lost cause.. but my boss ask me to
do this).
 In the socket activity there are some troubles dealing with timeouts.
It is pretty easy to Anyone DoS any apache webserver.
 I want to propose implementing a request timeout time, or at least a
check for incoming data.
 If you open many sockets to an apache server, you can keep it alive,
and make apache keep it open for a looong time, eating resources. If you
limit the numbers of conecctions per ip, you still can DoS apache using
2 or more other ips.
 All this was tedtes with Timeout set to 20, KeepAlive set to 5, and all
relevant options to their lowest value.
 (one of the common scrips used to kill apache, is apache-squ1rt, i use
this other to test)
 Use this perl script to test :
#!/usr/bin/perl
my $Child = 150;
my $Sleep = 10;
use IO::Socket;
use strict;
my($c);
my(@SOCKET);
my($t);
local $| = 1;
$c=0;
for(0..$Child) {
  @SOCKET[$c] = new IO::Socket::INET( Proto   => "tcp",
PeerAddr=> "127.0.0.1:80");
  $c++;
}
for(0..$Child) {
  if ( defined @SOCKET[$c]) {
$t = @SOCKET[$c];
print $t "GET / HTTP/1.1";
  }
}
while(1){
  $c=0;
  # For each children
  for(0..$Child) {
if ( defined @SOCKET[$c]) {
  $t = @SOCKET[$c];
  print $t "host: test.test";
}
$c++;
  }
  sleep ($Sleep);
}
$c=0;
for(0..$Child) {
  close(@SOCKET[$c++]);
}

Re: Puzzling News

[Rasmus Lerdorf]

William A. Rowe, Jr. wrote:
For anyone who wants to argue that this is a PHP-caused anomaly, note also
http://www.securityspace.com/s_survey/data/man.200501/apachemods.html
-10% followed by +11% the following month:
http://www.securityspace.com/s_survey/data/man.200502/apachemods.html
Big swings like that are almost always artificial.  One or two big 
domain parkers switching versions or toggling the server header.  You 
can't deduce much from that.

-Rasmus

Re: UNIX MPMs

[Rasmus Lerdorf]

Paul A. Houle wrote:
On Linux I've done some benchmarking and found that worker isn't 
any  faster than prefork at serving static pages.  (Is it any different 
on  other platforms,  such as Solaris?)  In principle you might save RAM 
by  running prefork,  but in this day and age you can fit 16 GB in a 1U 
pretty  easily and it's cheaper than hiring a programmer who doesn't 
know how to  track down race conditions,  never mind one that does.
If you know of such a programmer that can quickly identify and fix race 
conditions, please send him my way.  I will give him a job in a second.

-Rasmus

Re: Warning from accept filters on startup

[Rasmus Lerdorf]

On Wed, 12 Jan 2005, Paul Querna wrote:
> - The entire AcceptFilter/TCP_DEFER_ACCEPT code needs to be refactored
> to properly work with different protocols.

Has anybody here had a look at implementing httpready for Linux or is the
'just use Tux' camp still winning this one?  I had a quick look, but it
seemed non-trivial to me.

-Rasmus

Re: Dumb APR_BUCKET_BUFF_SIZE question

[Rasmus Lerdorf]

Cliff Woolley wrote:
It might be.  We've considered having it be configurable before.  There
are just a lot of implications in changing the value; for example, it
affects the memory footprint of the server, it affects how much data gets
read in to memory per read() call on a file bucket (which might alter the
decision of whether read() or mmap() is preferable, for example), it
affects how well the data fits into memory pages, and it affects how much
data is buffered in one block for the scenarios you described.  There are
probably others I'm not thinking of right now, too.  Lots of different
axes that must be considered when trying to pick an optimal value.  But
still, if you want to make it configurable, go right ahead.  :)
Right, I saw all the things it affected which was precisely why I 
wondered why it wasn't made configurable.  I can see perhaps protecting 
users from themselves, but other parts of Apache2 doesn't really follow 
that.  The WindowSize config option in mod_deflate would be a very bad 
idea to change away from the default of 15, for example.  Anyway, I will 
go play a bit with the bucket size on 64-bit boxes.

-Rasmus

Re: Dumb APR_BUCKET_BUFF_SIZE question

[Rasmus Lerdorf]

Cliff Woolley wrote:
On Fri, 7 Jan 2005, Rasmus Lerdorf wrote:

Why is it hardcoded to be 8000?  It would seem like you could easily be
unlucky and just miss the cutoff and end up with a 6000 byte heap bucket
followed by a 3000 byte transient bucket, for example, as a result of 3
3000 byte ap_rwrites.  For that particular case it might be quite
beneficial to increase APR_BUCKET_BUFF_SIZE to 9000 which would suggest
that it might be something that should be configurable.

In fact, it used to be 9000.  Then we realized that that was causing
cache/page alignment problems.  So we changed it to be just a fuzz less
than 8KB to allow it plus the bucket allocator structures to fit into one
8KB or two 4KB pages.
I still think it would be worthwhile to make it configurable.  Linux or 
FreeBSD5 on IA64 with 16k pages, for example, might show some decent 
gains by setting that to 15000.  Or do a getpagesize() call on startup 
to determine it dynamically.

-Rasmus

Dumb APR_BUCKET_BUFF_SIZE question

[Rasmus Lerdorf]

Why is it hardcoded to be 8000?  It would seem like you could easily be 
unlucky and just miss the cutoff and end up with a 6000 byte heap bucket 
followed by a 3000 byte transient bucket, for example, as a result of 3 
3000 byte ap_rwrites.  For that particular case it might be quite 
beneficial to increase APR_BUCKET_BUFF_SIZE to 9000 which would suggest 
that it might be something that should be configurable.

-Rasmus

Re: How to change ap_document_root variable from a apache2 module ?

[Rasmus Lerdorf]

On Thu, 30 Dec 2004, Paul Querna wrote:
> Rasmus Lerdorf wrote:
> [snip]
> >>Hint: document root is a configuration value used *by default* in
> >>filename translation.  Any module can implement its own translation
> >>hook, but trying to change a core configuration value is not only
> >>utterly pointless (your module can use its own variables), it's
> >>working against the apache design, even in a non-threaded MPM.
> >
> >
> > Whether it is against the design or not, it is a useful thing to be able
> > to do.  And I fully realize that my module can do whatever it wants, but
> > other modules won't know about it unless I modify them.
>
> I agree this could be a useful 'feature'.  I believe in a new API for
> configuration, this should be taken into consideration. However, the
> current hack of overwriting the values of a CORE config struct is
> completely bogus.
>
> >
> > This can be done quite safely in Apache1, by the way.
>
> I don't believe it can. Code?

Well, since you don't need to worry about thread safety as long as you set
it on every request, or reset it after each request you are fine.
Something like:

  foo->old = ap_document_root(r);
  conf = ap_get_module_config(s->module_config, &core_module);
  conf->ap_document_root = new;
  ap_register_cleanup(r->pool, foo, my_cleanup, ap_null_cleanup);

Then set it back to foo->old in my_cleanup().

And yes, I agree it is a hack to change anything in the core_module on a
per-request basis, but there are a lot of things that are very useful
hacks in Apache1 that I am hoping to see the richer and more flexible
Apache2 API address.

-Rasmus

Re: How to change ap_document_root variable from a apache2 module ?

[Rasmus Lerdorf]

On Fri, 31 Dec 2004, Nick Kew wrote:
> On Thu, 30 Dec 2004, Rasmus Lerdorf wrote:
>
> > Nobody is saying anything about doing it anywhere within a request.
>
> Huh?  What are you smoking?
>
> > obviously you can't change the doc_root after the fact, but I see no
> > reason to not be able to do it prior to each request in the filename
> > translation hook
>
> Erm, where can I find a filename translation hook outside the request?

You can't, but the translation hook is not *anywhere* within the request,
it is at the beginning.

> >  so other hooks which use the doc_root will function
> > correctly.
>
> Hint: document root is a configuration value used *by default* in
> filename translation.  Any module can implement its own translation
> hook, but trying to change a core configuration value is not only
> utterly pointless (your module can use its own variables), it's
> working against the apache design, even in a non-threaded MPM.

Whether it is against the design or not, it is a useful thing to be able
to do.  And I fully realize that my module can do whatever it wants, but
other modules won't know about it unless I modify them.

This can be done quite safely in Apache1, by the way.

-Rasmus

Re: How to change ap_document_root variable from a apache2 module ?

[Rasmus Lerdorf]

On Fri, 31 Dec 2004, Nick Kew wrote:
> On Thu, 30 Dec 2004, Paul Querna wrote:
>
> > > Just sets the root server's docroot then now.  Not being able able to
> > > manipulate the document_root is a rather severe limitation as far as I am
> > > concerned.
> >
> > I agree. But to fix it properly, we need a different system for doing
> > configuration stuff. The current system is just no good for doing stuff
> > on demand/on request.
>
> Huh?  Why is this a problem?  A module might just have a legitimate
> reason to change document_root at post_config.  But to do so anywhere
> within a request is nonsense: by all means set file paths (c.f. Alias,
> etc), but don't call it document root.

Nobody is saying anything about doing it anywhere within a request.
obviously you can't change the doc_root after the fact, but I see no
reason to not be able to do it prior to each request in the filename
translation hook so other hooks which use the doc_root will function
correctly.

-Rasmus

Re: How to change ap_document_root variable from a apache2 module ?

[Rasmus Lerdorf]

On Thu, 30 Dec 2004, Paul Querna wrote:
> It is impossible with the current way virtual hosts and document roots
> are handled.

I guess doing it per-conf Apache1-style, as in:

  conf = ap_get_module_config()
  conf->ap_document_root = ...;

Just sets the root server's docroot then now.  Not being able able to
manipulate the document_root is a rather severe limitation as far as I am
concerned.

-Rasmus

Re: Time for 1.3.32 ?

[Rasmus Lerdorf]

On Wed, 8 Sep 2004, [ISO-8859-15] André Malo wrote:
> Actually I'm talking about the two proposals on the top. If you are
> interested in backport voting, you need to touch the STATUS file anyway and
> should follow the commits there.

I'd still suggest posting them here.  Until the lawyers here finish
chewing on the ASF CLA stuff I have no CVS access, so I can't actually
look at this stuff.  And no, I don't agree that the flow is 2.1->2.0->1.3.
The trees are way too different for that to make any sense.

-Rasmus

Re: Time for 1.3.32 ?

[Rasmus Lerdorf]

On Wed, 8 Sep 2004, [ISO-8859-15] André Malo wrote:

> * Jim Jagielski <[EMAIL PROTECTED]> wrote:
>
> > In general, people don't look for 1.3 patches in the 2.0 STATUS file
> > and vice-versa :)
>
> As far as I can see, the current way to make changes is 2.1 -> 2.0 -> 1.3.
> So it makes sense for me to look into 2.0 for possible 1.3 changes, but not
> vice versa ;-)
>
> nd
>

Re: Time for 1.3.32 ?

[Rasmus Lerdorf]

On Tue, 7 Sep 2004, [ISO-8859-15] André Malo wrote:
> * Jim Jagielski <[EMAIL PROTECTED]> wrote:
>
> > I'd like to propose a 1.3.32 release with a T&R either late this
> > week or early next.
>
> Sounds good.
> Though I'd like to point to the 2.0 status file, where a bugfix (to 2.0
> and 1.3) is waiting for approval :)

Could you save us some hunting and post it then?

-Rasmus

Re: [PATCH] backport static module checking in mod_so to 1.3

[Rasmus Lerdorf]

On Tue, 7 Sep 2004, Geoffrey Young wrote:
> the attached patch is a direct port of the logic from 2.0 to 1.3.  thoughts
> or insights on why this wouldn't be a good idea for 1.3 or other feedback
> appreciated.

Seems like a good idea to me.

-Rasmus

Re: [PATCH] "better" method of ap_set_keepalive being multicall safe

[Rasmus Lerdorf]

I had used the same approach but just had it external.  Putting it inside
is a better idea.  I still think it is dumb that this isn't called at a
much earlier stage, but changing that is more likely to break stuff than
this approach.

-Rasmus

On Sat, 28 Aug 2004, Jim Jagielski wrote:

> Here is, I think, a better patch. The assumption is that it's only
> in ap_set_keepalive that we set conn->keepalive to 1, so we
> can "overload" that flag to not only represent "Yes, keepalive
> the connection" but also as a "we've called ap_set_keepalive
> for this request already" flag.

Re: cvs commit: apache-1.3/src/main http_protocol.c http_request.c

[Rasmus Lerdorf]

On Fri, 27 Aug 2004 [EMAIL PROTECTED] wrote:
>/*
>   + * We need to ensure that r->connection->keepalive is valid in
>   + * order to determine if we can discard the request body below.
>   + */
>   +ap_set_keepalive(r);

I realize you probably just took my wording there, but it is a bit
misleading.  I would change it to:

   We need to ensure that r->connection->keepalive is set in order
   to determine if we need to call ap_discard_request_body() to read
   the rest of the request body for this request.  There is no point
   reading the body for this request if we are not in keepalive mode
   since we are in ap_die() and about to toss this request anyway.

-Rasmus

Re: cvs commit: apache-1.3/src/main http_protocol.c http_request.c

[Rasmus Lerdorf]

Yeah, I am not keen on the notes table approach either.  My original patch
to fix this didn't do that.  Not that my patch was great either.

The problem with calling ap_set_keepalive() multiple times is that the
function doesn't just set the keepalive flag, it also increments the
connection's keepalive counter on the call.

And the reason we are even messing with this is because we need to check
if we are on a keepalive connection in ap_die() so we know whether to
bother reading the rest of a request.  Normally ap_set_keepalive() isn't
called until we send the response headers which is too late in the game
for the ap_die() case.

-Rasmus

On Fri, 27 Aug 2004, Roy T. Fielding wrote:

> This doesn't look right.  Checking the notes table is a serious
> performance hit, and why does it matter how many times keepalives
> is incremented on an error path? There must be a better way to do this.
>
> Roy
>
>
> On Friday, August 27, 2004, at 04:44  PM, [EMAIL PROTECTED] wrote:
>
> > jim 2004/08/27 16:44:42
> >
> >   Modified:src  CHANGES
> >src/main http_protocol.c http_request.c
> >   Log:
> >   Make ap_set_keepalive more statefully aware, allowing it
> >   to be called multiple times (to correctly set keepalive)
> >   but not increment keepalives when not needed. This allows
> >   us to handle a special case where we need to discard
> >   body content "early"
> >
> >   Revision  ChangesPath
> >   1.1949+3 -2  apache-1.3/src/CHANGES
> >
> >   Index: CHANGES
> >   ===
> >   RCS file: /home/cvs/apache-1.3/src/CHANGES,v
> >   retrieving revision 1.1948
> >   retrieving revision 1.1949
> >   diff -u -r1.1948 -r1.1949
> >   --- CHANGES   27 Aug 2004 19:29:57 -  1.1948
> >   +++ CHANGES   27 Aug 2004 23:44:41 -  1.1949
> >   @@ -24,9 +24,10 @@
> > was not checked properly. This affects mod_usertrack and
> > core. PR 28218.  [André Malo]
> >
> >   -  *) No longer breaks mod_dav, frontpage and others.  Backs out
> >   - a patch which prevented discarding the request body for
> > requests
> >   +  *) No longer breaks mod_dav, frontpage and others.  Repair a patch
> >   + in 1.3.31 which prevented discarding the request body for
> > requests
> > that will be keptalive but are not currently keptalive. PR
> > 29237.
> >   + [Jim Jagielski]
> >
> >  *) COMPATIBILITY: Added new compile-time flag:
> > UCN_OFF_HONOR_PHYSICAL_PORT.
> > It controls how UseCanonicalName Off determines the port value
> > if
> >
> >
> >
> >   1.336 +12 -1 apache-1.3/src/main/http_protocol.c
> >
> >   Index: http_protocol.c
> >   ===
> >   RCS file: /home/cvs/apache-1.3/src/main/http_protocol.c,v
> >   retrieving revision 1.335
> >   retrieving revision 1.336
> >   diff -u -r1.335 -r1.336
> >   --- http_protocol.c   15 Apr 2004 15:51:51 -  1.335
> >   +++ http_protocol.c   27 Aug 2004 23:44:41 -  1.336
> >   @@ -391,6 +391,7 @@
> >int wimpy = ap_find_token(r->pool,
> >   ap_table_get(r->headers_out,
> > "Connection"), "close");
> >const char *conn = ap_table_get(r->headers_in, "Connection");
> >   +const char *herebefore = ap_table_get(r->notes,
> > "ap_set_keepalive-called");
> >
> >/* The following convoluted conditional determines whether or
> > not
> > * the current connection should remain persistent after this
> > response
> >   @@ -442,7 +443,17 @@
> >int left = r->server->keep_alive_max -
> > r->connection->keepalives;
> >
> >r->connection->keepalive = 1;
> >   -r->connection->keepalives++;
> >   + /*
> >   +  * ap_set_keepalive could be called multiple times (eg: in
> >   +  * ap_die() followed by ap_send_http_header()) during this
> >   +  * one single request. To ensure that we don't incorrectly
> >   +  * increment the keepalives counter for each call, we
> >   +  * use notes to store a state flag.
> >   +  */
> >   + if (!herebefore) {
> >   +r->connection->keepalives++;
> >   +ap_table_setn(r->notes, "ap_set_keepalive-called", "1");
> >   + }
> >
> >/* If they sent a Keep-Alive token, send one back */
> >if (ka_sent) {
> >
> >
> >
> >   1.176 +7 -1  apache-1.3/src/main/http_request.c
> >
> >   Index: http_request.c
> >   ===
> >   RCS file: /home/cvs/apache-1.3/src/main/http_request.c,v
> >   retrieving revision 1.175
> >   retrieving revision 1.176
> >   diff -u -r1.175 -r1.176
> >   --- http_request.c28 May 2004 12:07:02 -  1.175
> >   +++ http_request.c27 Aug 2004 23:44:41 -  1.176
> >   @@ -1051,12 +1051,18 @@
> >}
> >
> >/*
> >   + * We need to ensure that r->connection->keepalive is valid in
> >  

Re: Time for 1.3.32 ?

[Rasmus Lerdorf]

On Fri, 20 Aug 2004, Jim Jagielski wrote:
> I'm reviewing this... I'm mostly investigating whether
> the check for keepalive!=1 before calling ap_set_keepalive
> in ap_send_http_header and ap_send_error_response is
> too ap_die() specific. It seems to me that
> ap_set_keepalive should be smarter internally about
> "double or more calls" per request.

Yup, it should.  I tried to minimize the amount of code I touched to fix
the problem at hand.

The keepalive!=1 check prevents double-counting the request in the
keepalive counter.  if keepalive!=1 we know that either ap_set_keepalive()
has not been called yet, or it has been called and keepalive is off, so in
either case we won't be double-counting the request.

I still think the real problem is two-fold:

 1. ap_set_keepalive() is called way too late in the request

 2. it is impossible to tell if r->connection->keepalive has
been set since 0 can both mean it hasn't been set and that
it has been set and keepalive is off

My fix/hack just forces ap_set_keepalive to be called earlier on an
ap_die() request and then I try to make sure that the subsequent calls
won't double-increment the keepalive counter.

If ap_set_keepalive() was fixed to be immune to being called twice during
a request then those keepalive!=1 checks can be removed.

-Rasmus

Re: Time for 1.3.32 ?

[Rasmus Lerdorf]

Ok, how about this.

Add a call to ap_set_keepalive(r) in ap_die() before the check to see
if we should be discarding the request body.  Then, since ap_set_keepalive
increments the keepalives counter on the connection if keepalive is
determined to be enabled, add a check to the calls in ap_send_http_header
and ap_send_error_response since these could be called after ap_die and
we don't want to increment the keepalives counter twice on this request.

   

Index: http_protocol.c
===
diff -u -r1.76 http_protocol.c
--- http_protocol.c 21 May 2004 11:43:31 -  1.76
+++ http_protocol.c 8 Jul 2004 16:49:03 -
@@ -2074,7 +2074,7 @@
 PUSH_EBCDIC_OUTPUTCONVERSION_STATE_r(r, 1);
 #endif /*CHARSET_EBCDIC*/
   

-ap_set_keepalive(r);
+if(r->connection->keepalive != 1) ap_set_keepalive(r);
   

 #ifdef YAHOO
 #ifdef GZIP
@@ -3063,7 +3063,7 @@
 ap_hard_timeout("send 304", r);
   

 ap_basic_http_header(r);
-ap_set_keepalive(r);
+if(r->connection->keepalive != 1) ap_set_keepalive(r);
   

 ap_table_do((int (*)(void *, const char *, const char *)) 
ap_send_header_field,
 (void *) r, r->headers_out,
Index: http_request.c
===
diff -u -r1.29 http_request.c
--- http_request.c  21 May 2004 11:43:31 -  1.29
+++ http_request.c  8 Jul 2004 16:49:03 -
@@ -1116,6 +1116,12 @@
 }
   

 /*
+ * We need r->connection->keepalive set correctly in order to determine if
+ * we can discard the request body in the next condition
+ */
+ap_set_keepalive(r);
+
+/*
  * If we want to keep the connection, be sure that the request body
  * (if any) has been read.
  */


On Tue, 6 Jul 2004, Jim Jagielski wrote:

> Yes, we do, and we're still waiting for a patch. However,
> I can't see us delaying 1.3.32 for an "unreasonable"
> amount of time.
> 
> On Jul 5, 2004, at 10:54 AM, Rasmus Lerdorf wrote:
> 
> > We still have that outstanding issue of conn->keepalive being bogus in
> > ap_die() because it hasn't been set yet and thus we can't discard the
> > request body in situations where we really need to.  See my previous  
> > long
> > explanation of that problem.
> >
> > -Rasmus
> >
> > On Sat, 3 Jul 2004, Jim Jagielski wrote:
> >
> >> Let's use STATUS :)
> >>
> >> =?ISO-8859-15?Q?Andr=E9?= Malo wrote:
> >>>
> >>> * Jeff Trawick <[EMAIL PROTECTED]> wrote:
> >>>
> >>>> well, if you're going to be that way then consider my simple Win32  
> >>>> patch to
> >>>> fix reporting of proper error by spawnl(), which needs another +1 :)
> >>>>
> >>>> (see thread "[1.3 PATCH] restore failing errno for Win32 spawn  
> >>>> errors" on
> >>>> this list)
> >>>
> >>> +1 from me for that errno patch ;)
> >>>
> >>> nd
> >>> --
> >>> "Umfassendes Werk (auch fuer Umsteiger vom Apache 1.3)"
> >>>   -- aus einer Rezension
> >>>
> >>> <http://pub.perlig.de/books.html#apache2>
> >>>
> >>
> >>
> >> --
> >> == 
> >> =
> >>Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]
> >> http://www.jaguNET.com/
> >>   "A society that will trade a little liberty for a little order
> >>  will lose both and deserve neither" - T.Jefferson
> >>
> >
> >
> 

Re: Time for 1.3.32 ?

[Rasmus Lerdorf]

We still have that outstanding issue of conn->keepalive being bogus in
ap_die() because it hasn't been set yet and thus we can't discard the
request body in situations where we really need to.  See my previous long
explanation of that problem.

-Rasmus

On Sat, 3 Jul 2004, Jim Jagielski wrote:

> Let's use STATUS :)
>
> =?ISO-8859-15?Q?Andr=E9?= Malo wrote:
> >
> > * Jeff Trawick <[EMAIL PROTECTED]> wrote:
> >
> > > well, if you're going to be that way then consider my simple Win32 patch to
> > > fix reporting of proper error by spawnl(), which needs another +1 :)
> > >
> > > (see thread "[1.3 PATCH] restore failing errno for Win32 spawn errors" on
> > > this list)
> >
> > +1 from me for that errno patch ;)
> >
> > nd
> > --
> > "Umfassendes Werk (auch fuer Umsteiger vom Apache 1.3)"
> >   -- aus einer Rezension
> >
> > 
> >
>
>
> --
> ===
>Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
>   "A society that will trade a little liberty for a little order
>  will lose both and deserve neither" - T.Jefferson
>

Re: 1.3.31 regression affecting Front Page?

[Rasmus Lerdorf]

On Wed, 9 Jun 2004, Jim Jagielski wrote:
> On Jun 9, 2004, at 3:24 PM, Rasmus Lerdorf wrote:
> > I guess what we are agreeing on here is that the logic that sets 
> > keepalive
> > to 0 is faulty and that is probably where the real fix lies.
> yeah... it's pretty inconsistent. Looking at ap_set_keepalive
> even after we know the connection will be closed, we
> set keepalive to 0, for example.

Ok, I had a closer look at the flow.  There are 6 main cases I care about.  
Static, Dynamic and Error requests for configs KeepAlive=Off and 
KeepAlive=On.  Here is what happens:

With KeepAlive On
   
  
For both static and dynamic requests the flow is similar.
  
We start connection->keepalive starts at 0 at the beginning of the request
process_request_internal eventually leads to an ap_send_http_header call 
which calls ap_set_keepalive which determines that the config has 
KeepAlive on and sets connection->keepalive to 1

For an error, like a 404, it is different.  We start off with 
connection->keepalive being set to 0, process_internal_request calls 
ap_die on the error which calls ap_send_error_response which in turn calls  
ap_send_http_header which finally sets connection_keepalive to 1.  But 
this happens after ap_die checks conn->keepalive to determine whether to 
discard the request body or not.

With KeepAlive Off

The picture is as above, except ap_set_keepalive called from 
ap_send_http_header sets connection->keepalive to 0 instead of 1.  So for 
the duration of the request, before and after checking whether we are on a 
keepalive connection, connection->keepalive is 0.

The summary here is that checking connection->keepalive before 
ap_set_keepalive() has been called really doesn't make any sense.  And we 
can't just call ap_set_keepalive in ap_die before the check because it 
would end up getting called twice and there is no guard against that in 
it.  It would double-count the request in the keepalives counter.  We need 
to either call ap_set_keepalive earlier on, like in 
process_request_internal before we hit ap_die, or we need to add a 
double-call guard in it and just add a call in ap_die before the keepalive 
check.

Another alternative would be to clean up this mess which has our undecided
state indistinguishable from our disabled state.  Checking for 0 in ap_die
is only wrong because the check is before the ap_set_keepalive call.  The
meaning of that 0 changes on that call.

-Rasmus

Re: 1.3.31 regression affecting Front Page?

[Rasmus Lerdorf]

On Wed, 9 Jun 2004, Joe Orton wrote:
> When ap_die() is called, ap_set_keepalive() has not been called, and
> r->connection->keepalive is set to 0, as it was initialized so.
> 
> The last thing ap_die does is call ap_send_error_response, which calls
> ap_send_http_header, which calls ap_set_keepalive, which sets
> r->connection->keepalive = 1.
> 
> FWIW, "r->connection->keeaplive == 0" is documented in the header to
> mean "undecided", not specifically "no keep alive", as you seem to want
> to use it.

Well, we have a bunch of cases where we leave it "undecided" when it is 
decidedly not undecided.  For example, when I configure a server to 
exlicitly not use keepalives and restrict Post requests with a Limit Post 
clause, I don't expect a POST request with a huge file upload to hit this 
ap_die code and suck in the entire POST request because something has 
decided that the keepalive state is undecided.  It's not undecided, I have 
told it that it is off and the 403 or 404 is the second big clue.

I guess what we are agreeing on here is that the logic that sets keepalive 
to 0 is faulty and that is probably where the real fix lies.

-Rasmus

Re: 1.3.31 regression affecting Front Page?

[Rasmus Lerdorf]

On Wed, 9 Jun 2004, Joe Orton wrote:

> On Wed, Jun 09, 2004 at 09:21:07AM -0700, Rasmus Lerdorf wrote:
> > Don't see that anywhere.  Either eaten by spam filters or a gerbil.
> > 
> > Anyway, I don't understand why this would have broken mod_dav.  If mod_dav
> > wants a keepalive connection it should determine this prior to the ap_die
> > and set conn->keepalive to 1.  Or am I missing something with respect to
> > what mod_dav is doing here?  I suppose we could add an ugly exception for
> > a PROPFIND here, but I'd like to make sure that is actually needed.
> 
> >From my debugging: mod_dav doesn't actually get involved at all, the
> check_user_id handler from the auth module returns AUTH_REQUIRED for the
> request, and ap_set_keepalive is not called before ap_die is invoked in
> that case. r->connection->keepalive has never been changed from 0 when
> the test in question is reached - it's called later by
> ap_send_error_response, but that's too late.

Just to save me some debugging time, are you saying that on a 401 we call 
ap_set_keepalive after calling ap_die or that we are not calling 
ap_set_keepalive at all?  The former should never happen as far as I am 
concerned, and the latter should be fine as the request simply won't be a 
keepalive request and nothing should expect it to be.

-Rasmus

Re: 1.3.31 regression affecting Front Page?

[Rasmus Lerdorf]

Don't see that anywhere.  Either eaten by spam filters or a gerbil.

Anyway, I don't understand why this would have broken mod_dav.  If mod_dav
wants a keepalive connection it should determine this prior to the ap_die
and set conn->keepalive to 1.  Or am I missing something with respect to
what mod_dav is doing here?  I suppose we could add an ugly exception for
a PROPFIND here, but I'd like to make sure that is actually needed.

Without this patch non-keepalive connections are not being dropped when we
know there is nothing more to do.  For example, on a server that doesn't
allow POST someone can POST to it and it will happily sit there and read
the entire POST request.  This defeats the purpose of adding a Limit POST
and introduces a DoS.  Same for a 404 or any other error handler.  I can
POST to a bogus URL and Apache will read the entire POST request even
though we know it is a 404 at this point and that we can safely discard
the request body.  I don't think releasing .32 without addressing this
issue is a good idea.

-Rasmus

On Wed, 9 Jun 2004, Jim Jagielski wrote:

> I had sent private Email to your @apache.org address
> (since that's the one you use to provide HTTPD related
> patches).
>
> On Jun 8, 2004, at 5:10 PM, Rasmus Lerdorf wrote:
>
> > Uh, I never received anything on this.  Did you actually send me
> > something?  I'll have a look at addressing this issue.  Releasing
> > 1.3.32
> > without this fix would be a nasty backwards step.  The original problem
> > this fixes is serious.
> >
> > -Rasmus
> >
> > On Fri, 28 May 2004, Jim Jagielski wrote:
> >
> >> I've backed out that patch and asked Rasmus to send a replacemnet
> >> which addresses his specific problem but does not cause
> >> the below behavior.
> >>
> >> I'm tempted to release 1.3.32...
> >>
> >> Jeff Trawick wrote:
> >>>
> >>> This patch did it:
> >>>
> >>> http://cvs.apache.org/viewcvs.cgi/apache-1.3/src/main/
> >>> http_request.c?r1=1.173&r2=1.174
> >>>
> >>> See also
> >>>
> >>> http://issues.apache.org/bugzilla/show_bug.cgi?id=29257
> >>> http://www.rtr.com/fp2002disc/_disc2/0a71.htm
> >>>
> >>
> >>
> >> --
> >> ==
> >> =
> >>Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]
> >> http://www.jaguNET.com/
> >>   "A society that will trade a little liberty for a little order
> >>  will lose both and deserve neither" - T.Jefferson
> >>
> >
> >
>

Re: 1.3.31 regression affecting Front Page?

[Rasmus Lerdorf]

Uh, I never received anything on this.  Did you actually send me 
something?  I'll have a look at addressing this issue.  Releasing 1.3.32 
without this fix would be a nasty backwards step.  The original problem 
this fixes is serious.

-Rasmus

On Fri, 28 May 2004, Jim Jagielski wrote:

> I've backed out that patch and asked Rasmus to send a replacemnet
> which addresses his specific problem but does not cause
> the below behavior.
> 
> I'm tempted to release 1.3.32...
> 
> Jeff Trawick wrote:
> > 
> > This patch did it:
> > 
> > http://cvs.apache.org/viewcvs.cgi/apache-1.3/src/main/http_request.c?r1=1.173&r2=1.174
> > 
> > See also
> > 
> > http://issues.apache.org/bugzilla/show_bug.cgi?id=29257
> > http://www.rtr.com/fp2002disc/_disc2/0a71.htm
> > 
> 
> 
> -- 
> ===
>Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
>   "A society that will trade a little liberty for a little order
>  will lose both and deserve neither" - T.Jefferson
> 

Re: Time for 1.3.30?

[Rasmus Lerdorf]

On Tue, 9 Mar 2004, Jim Jagielski wrote:
   
 > There are a few open patches floating about, 
but in general I think
> we're close to a point where we should seriously consider 1.3.30.
> I volunteer to be RM... I'd like to shoot for mid-late next
> week for a release.
>
> Comments?

I'd like to see this patch in 1.3.30:

+++ src/main/http_protocol.c   2004-02-27 20:54:48.0 -0800
@@ -1001,7 +976,7 @@
  */
 ap_bsetflag(conn->client, B_SAFEREAD, 1);
 while ((len = ap_getline(l, sizeof(l), conn->client, 0)) <= 0) {
-if ((len < 0) || ap_bgetflag(conn->client, B_EOF)) {
+if ((len < 0) || ap_bgetflag(conn->client, B_EOF) || !conn->keepalives) {
 ap_bsetflag(conn->client, B_SAFEREAD, 0);
/* this is a hack to make sure that request time is set,
 * it's not perfect, but it's better than nothing

The logic here was only meant to be in effect for pipelined requests. This
patch makes sure it is only applied when we actually have a pipelined
request and it isn't the first request in a series.

-Rasmus

Re: Antw: RE: consider reopening 1.3

[Rasmus Lerdorf]

On Mon, 17 Nov 2003, Jim Jagielski wrote:
> On Nov 17, 2003, at 2:22 PM, Bill Stoddard wrote:
> >
> > In this economic environment (and perhaps this will turn out to be 
> > generally true from now on), companies are not making investments in 
> > IT unless they can get a proven and almost immediate return on that 
> > investment. Making the jump to Apache 2.0 -can- be a big investment 
> > (depending on how many custom/third party modules you use)
> 
> Most people with those big investments are using at least *some* 3rd 
> party
> modules. Having a 1.4 that is not binary compatible with 1.3
> means that those 3rd party modules will need to be (at least)
> re-compiled for 1.4. So they will need to worry about 1.3,
> 1.4 and 2.0 (and potentially 2.2)... That's an *awful* lot
> to have people keep track of. I don't see companies out
> there wanting to do that... they will maintain their 1.3
> modules for awhile, and their 2.x ones, because it *is*
> the next gen, but I think they would avoid 1.4 almost
> totally.
> 
> Having 1.4 not be binary compatible with 1.3 severely limits its
> usefulness to those exact people that it's supposed to be
> helping.

As someone working in a company like that, I can tell you definitively 
that this is not true.  At least not here at the biggest web company in 
the world.

-Rasmus

Re: Antw: RE: consider reopening 1.3

[Rasmus Lerdorf]

On Mon, 17 Nov 2003, Jim Jagielski wrote:

> Glenn wrote:
> > 
> > On Mon, Nov 17, 2003 at 01:31:55PM -0500, Bill Stoddard wrote:
> > > Apache 1.4, an APR'ized version of Apache 1.3 (to pick up IPV6 and 64 bit 
> > > support) with all the Windows specific code stripped out and source 
> > > compatability (to the extent possible) with Apache 1.3 modules would 
> > > probably see rapid uptake. I can't say that thrills me but it's probably 
> > > true...
> > 
> > +1
> > 
> 
> Again, unless there is 100% binary compatibility, which I do NOT
> see with 1.4, then *what* is the driver for 1.4 over 2.x??

Why binary compatibility?  Recompiling a module is a hell of a lot easier 
than rewriting it.  

-Rasmus

Re: Antw: RE: consider reopening 1.3

[Rasmus Lerdorf]

On Mon, 17 Nov 2003, Bill Stoddard wrote:
> Apache 1.4, an APR'ized version of Apache 1.3 (to pick up IPV6 and 64 bit support) 
> with all the Windows 
> specific code stripped out and source compatability (to the extent possible) with 
> Apache 1.3 modules would 
> probably see rapid uptake. I can't say that thrills me but it's probably true...

Not sure why providing useful software wouldn't thrill you.  I think the
point here is that someone is eventually going to do this.  Would be nicer
to have it under our control than have it leave the ASF.

-Rasmus

Re: consider reopening 1.3

[Rasmus Lerdorf]

On Mon, 17 Nov 2003, Ian Holsman wrote:
> I belive 2.0 beats 1.3 on these metrics, but like everyone here, Ihave 
> no more energy proving/disproving which is faster.. 2.0 works for me, 
> and thats all I really care about, not who else is using it.

Do you really believe this to be true for Apache2-prefork as well?

It may be, I just have never seen any evidence of that in my own tests.  
Obviously both servers can easily fill a 100M connection when just serving 
up static files at which point it comes down to what sort of cpu load 
doing so puts on the server.  I really don't see any compelling advantage 
of Apache2-prefork over Apache1 to justify the development costs involved 
in porting hundreds of modules.  And I am not speaking with my PHP hat on 
here, by the way.  And a threaded mpm is just not an option.  Most humans 
are simply not smart enough to write threadsafe code.

-Rasmus

RE: consider reopening 1.3

[Rasmus Lerdorf]

On Sun, 16 Nov 2003, Marc Slemko wrote:
> 3. Threading issues.  This is a red herring; threading issues can be a
> reason why moving to 2.0 wouldn't give someone enough of a reason to make
> it worthwhile, but they do not block anyone moving to 2.0.  if they
> don't want to use threads, they don't have to use threads.

I have always had the feeling that Apache2+prefork was a bit of a 
second-class citizen.  I have tested it periodically over the past 2 years 
and it has never gotten anywhere close to Apache1 in performance.  I ran 
another test of 1.3.29 vs 2.0.48-prefork just now just to make sure I 
wasn't blowing pure smoke here and here is what I got:

Apache-1.3.29

Document Path:  /bench_main.php
Document Length:7832 bytes
 
Concurrency Level:  5
Time taken for tests:   2.038 seconds
Complete requests:  500
Failed requests:0
Broken pipe errors: 0
Total transferred:  4071500 bytes
HTML transferred:   3916000 bytes
Requests per second:245.34 [#/sec] (mean)
Time per request:   20.38 [ms] (mean)
Time per request:   4.08 [ms] (mean, across all concurrent requests)
Transfer rate:  1997.79 [Kbytes/sec] received
 
Apache-2.0.48

Concurrency Level:  5
Time taken for tests:   3.367 seconds
Complete requests:  500
Failed requests:0
Broken pipe errors: 0
Total transferred:  4028508 bytes
HTML transferred:   3931622 bytes
Requests per second:148.50 [#/sec] (mean)
Time per request:   33.67 [ms] (mean)
Time per request:   6.73 [ms] (mean, across all concurrent requests)
Transfer rate:  1196.47 [Kbytes/sec] received

I did try more requests, different concurrency levels and I tweaked the 
config of both to be as close as I could make them.

Looking at the total bytes tranferred, Apache2 is sending less mostly
because I forgot to remove the p3p module from my Apache1 setup which adds
a big long nasty p3p header to every response, but even with that Apache1
was way faster.

The benchmark itself is a standard little PHP thing I use to test stuff, 
so it may be that our Apache2Handler for PHP is crap and we aren't smart 
enough to write a decent Apache2 module.

-Rasmus

Re: 1.3 Wishlist: (Was: Re: consider reopening 1.3)

[Rasmus Lerdorf]

On Sun, 16 Nov 2003, Jim Jagielski wrote:
> So a useful topic is: What is *missing* in 1.3 that needs to be 
> addressed.
> What are the features/additions that the disenfranchised 1.3 developers
> want to add to 1.3...

How about support for chunked compressed responses right in 
src/main/buff.c where it belongs (given we don't have filters in Apache1)

That alone would probably warrant a bump to 1.4.  I think people are 
thinking 1.4 because adding any significant features between 1.3.29 and 
1.3.30 would seem pretty odd.

-Rasmus

Re: consider reopening 1.3

[Rasmus Lerdorf]

On Sun, 16 Nov 2003, Jeff Trawick wrote:
> The point was not to blame anybody.  Instead, I don't believe there are so many 
> people as you imply.  Many of the people who are no longer developing have 
> moved on to other interests/work/etc. and have dropped out of httpd dev because 
> of that.
> 
> If somebody is no longer working on [EMAIL PROTECTED] because they feel 
> disenfranchised, it is fair to point out that their skills are still very 
> useful even if they don't want to mess with 2.x.

But the message they have gotten is that they are useful as bug fixers and 
documenters only.  All the fun stuff is reserved for people working on 
Apache2.  If that is an incorrect impression then we need to shout this 
from the rooftops and put up a big sign inviting people to submit any 
outstanding cool patches for 1.3 along with ideas for things they would 
like to see added.  

Basically I see us back in the NCSA days right now.  The ASF has mostly 
abandoned Apache1 and we are in that transition phase where people are 
looking at each other waiting for someone to step up and continue 
development on the code that everyone is using.  That is how Apache was 
born to begin with.

-Rasmus

Re: consider reopening 1.3

[Rasmus Lerdorf]

On Sun, 16 Nov 2003, Graham Leggett wrote:
> I think the key thing is "bugfixes" compared to "features" and 
> "architecture changes".
> 
> I am +1 on seeing bugfixes go into v1.3 - people are using it, and if it 
> can work better, so be it. But to actively encourage people to add 
> features or architecture changes to v1.3, that simply turns v1.3 into 
> "something else no longer compatible with v1.3", which we already have: 
> v2.0.

We have two completely different pieces of software.  The fact that they 
are both called Apache is just marketing.  By saying bug fixes only in 1.3 
you are effectively telling people who want to work on it to get lost.  
There are plenty of evolutionary things that could be done in it short of 
the revolutionary jump to Apache2.  We know the jump is too big for a big 
chunk of the market but nobody seems to want to address that.  

-Rasmus

Re: consider reopening 1.3

[Rasmus Lerdorf]

On Sun, 16 Nov 2003, Jeff Trawick wrote:
> Too bad all these supposedly-disenfranchised people aren't around to review 1.3 
> fixes.  1.3 would be healthier if they were.

And it is the reason for why they are not around that is in question here.  
Why wouldn't there be plenty of hackers around for a piece of software
used to widely?  Blaming the hackers seems a bit shortsighted.  Have a
look at the environment we have given them to operate in and at some of
the responses over the years people have gotten when they suggested new
features for 1.3.

-Rasmus

Re: consider reopening 1.3

[Rasmus Lerdorf]

On Sun, 16 Nov 2003, Jeff Trawick wrote:
> > *** We need to get back many of the disenfranchised Apache 1.3 developers
> 
> Who are these people?

/me raises a hand

People have suggested that we have fewer developers today because Apache 2 
is too complex.  That the crappy economy has reduced the time people have 
to work on it.  Or that Apache-1.3 is simply good enough for most people.

I don't think it is any of that.  Hackers love to hack, but hackers are 
also practical creatures.  They hack on things that they need and use.  
They need and use Apache1 but we have taken our ball away and gone off to 
play another game.

Just compare the list of contributors today to 4 years ago if you want a 
list.

> I don't expect any of the current Apache developers would be interested in 
> this.  But plenty of people join the development community over time (see 
> previous comments) and theoretically the opinions could change.

I think the key word there is "current"

I also work for a large company with plenty of talented developers and
thousands of production Apache-1.3 servers along with hundreds of custom
Apache-1.3 modules.  It will be years before I can even consider Apache2,
given the architecture and API differences between the two.  If something
could be done in an Apache-1.4 and 2.1/2.2 to help bridge the gap perhaps
one day Apache2 will be an option, but today it is simply too big a gap to
jump and I am pretty sure the bulk of the Apache community is in the same
boat.

-Rasmus

Re: Changing user ID with prefork

[Rasmus Lerdorf]

That's what the perchild mpm is for.  You can't do this with prefork.

-Rasmus

On Sat, 21 Jun 2003, Pablo Yaggi wrote:

> Well, sorry if I'm bodering you with this kind of questions,
>   Using prefork I want to change the user ID that process the request,
> that user id will be based on the request, so I need to change it once It has
> the url (even on an ssl request) and before any lock that couldn't be
> unloked if it is on a dif uid.
>   Is it posibble to do ? is there any hook available in that instance ?
>   I tryied before doing it on the translation hook, but the problem
> was that prefork couldn't destroy the mutex (log said).
>
>   Thank's,
> Pablo
>
>

Small optimization patch for 1.3

[Rasmus Lerdorf]

In doing a bit of performance tweaking on 1.3, I noticed that
ap_send_header_field() does an ap_rvputs() with each little piece of a
header which results in separate ap_bwrite() calls for "Primitive", ":",
"Value", "crlf" for each header line sent.  Rather than having these 1 and
2 character ap_bwrite() calls wouldn't it make more sense to ap_pstrcat()  
these little bits together and do a single ap_bputs()?

Trivial patch follows:

Index: http_protocol.c
===
RCS file: /home/cvs/apache-1.3/src/main/http_protocol.c,v
retrieving revision 1.330
diff -u -r1.330 http_protocol.c
--- http_protocol.c 3 Feb 2003 17:13:22 -   1.330
+++ http_protocol.c 28 Mar 2003 18:27:38 -
@@ -1566,7 +1566,7 @@
 return 1;
 }
 }
-return (0 < ap_rvputs(r, fieldname, ": ", fieldval, CRLF, NULL));
+return (0 < ap_bputs(ap_pstrcat(r->pool, fieldname, ": ", fieldval, CRLF, NULL), 
r->connection->client));
 }

 API_EXPORT(void) ap_basic_http_header(request_rec *r)
@@ -1595,7 +1595,7 @@
 #endif /*CHARSET_EBCDIC*/

 /* output the HTTP/1.x Status-Line */
-ap_rvputs(r, protocol, " ", r->status_line, CRLF, NULL);
+ap_bputs(ap_pstrcat(r->pool, protocol, " ", r->status_line, CRLF, NULL), 
r->connection->client);

 /* output the date header */
 ap_send_header_field(r, "Date", ap_gm_timestr_822(r->pool, r->request_time));

Re: story posted

[Rasmus Lerdorf]

> On to a).  The PHP developers have screamed quite loudly about the
> failings of Apache2, when, in fact, the largest reason why mod_php will
> not work with Apache2 is because it is not a priority for PHP developers.

Nobody is screaming.

> Personally, my servers run PHP as CGI.  Works well.  Thank you.
> Kudos for safe_mode and safe_mode_gid.

??  safe_mode and safe_mode_gid are rather useless when running PHP as a 
CGI.  All you are doing is burning syscalls aimlessly and slowing 
everything down.  If you are running PHP as a CGI anyway, run it under 
suExec.

As for the Apache2+PHP stuff, we are not deflecting blame.  We are simply
not acknowledging that it is somehow our duty to drop everything and
support Apache2 just because someone decided it was a good idea to release
a new web server.  And when attacked repeatedly on this issue we tend to
point out that thread safety issues in commonly used libraries pretty much
nullify any threading gains of the new architecture since we'd have to
mutex them, and also the fact that the filter api's complexity isn't
making life easy for us, nor does the architecture mesh well with how PHP
likes to do things.  As a handler under prefork it may work very well.  
Who knows.  Feel free to apply for a CVS account and write it, but please
stop trying to browbeat us into doing something.  I personally have a
working web server that does what I need.  Show me something irresistible
about this new server and I will be compelled to work on it myself.

-Rasmus

Re: story posted

[Rasmus Lerdorf]

Do you really think we don't know this?

The point is that at this point we would basically have to mutex every 
call to every 3rd-party library function since we simply don't know which 
ones are threadsafe and which ones aren't.  And if we lock everything, 
isn't it simpler to just go prefork and avoid the whole thing?

It boils down to where our time is best spent.  Yes, we can come up with 
all sorts of tricks to sorta-kinda-almost make this work.  But why?  What 
do we gain?  At this point most of us feel that we really don't gain 
anything.

-Rasmus

On Sun, 9 Feb 2003, Glenn wrote:

> At the risk of inflaming an already tense conversation (too late),
> I have to say that third party thread-safety issues are not a very
> good excuse.
> 
> It's trivial to make a library thread safe by serializing access to the
> functions of the non-thread-safe library.  Although scripts that employ
> the non-thread-safe library would suffer a performance penalty, they
> would still work!
> 
> If the core of PHP is thread safe, then this should not be a big deal.
> Also, as time goes on, third party library locks can be removed as
> thread-safe versions of the libraries become available.
> 
> For a well-written and concise explanation, please refer to:
> 
>http://www.unet.univie.ac.at/aix/aixprggd/genprogc/writing_reentrant_thread_safe_code.htm
> 
> Cheers,
> -Glenn
> 
> On Fri, Feb 07, 2003 at 02:02:39AM +0100, Sascha Schumann wrote:
> > > Rather than fix the TS issues in all the 3rd party libraries
> > > that can be used with PHP's and its extension its more
> > > pragmatic to sort out thread/filter issues in PHP core.
> > 
> > The PHP project has been working on thread-safety since 1999.
> > All core components and most extensions can be compiled in
> > thread-safe mode.
> > 
> > The first SAPI module for a threaded UNIX web server was
> > completed in the same year (for AOLserver).
> > 
> > It's safe to assert that we need to primarily worry about 3rd
> > party libraries.
> > 
> > > Also blaming on use of TLS seems a little strange since use of
> > > TLS is supposed to improve re-entrancy isn't it ?
> > 
> > Let me just quickly define what these concepts refer to.
> > 
> > Thread-safe means that you can run code in multiple threads
> > concurrently (e.g. multiple worker threads).
> > 
> > Reentrant means that you can run code in a single thread
> > context in a nested way (e.g. Apache sub-requests).
> > 
> > Thread local storage aids only the former one.
> > 
> > From my point of view we should not worry about reentrancy
> > right now, at least not until we a have firm grasp on the 3rd
> > party TS issues.
> > 
> > - Sascha
> 

Re: story posted

[Rasmus Lerdorf]

> Perhaps it's best for someone to simply port php4apache to 2.0 and start
> moving a little momentum.  The thread-safety arguement is a little bogus,
> until folks have something to start finding thread-safety bugs.  How long
> have Win32 users been doing PHP within threaded servers?

We have gotten a number of TS bug reports, and what does Windows have to
do with it?  Most Windows libraries are threadsafe to begin with.  We are
not talking about TS issues in PHP here, we are talking about them in
libraries that we have no control over.  Sometimes these are even
binary-only libraries.  This is something that will eventually work itself
out as developers become more aware and threaded apps start becoming more
common on UNIX systems.  But that doesn't change the fact that it is a
real problem for us today.

I would love to have a list of the known TS issues in the various
3rd-party libraries out there, and I did try to push that a bit with this:

http://httpd.apache.org/docs-2.0/developer/thread_safety.html

a while ago now.  I was hoping for some more input here.  We should be
able to get to the point where we can tell people that Apache2+PHP+"List
of safe extensions" is production-quality as long as they don't stray into
any of the extensions/3rd party libs on the unsafe/unknown list.  Right
now that list unfortunately includes just about everything you can hook
into PHP.

-Rasmus

Re: story posted

[Rasmus Lerdorf]

> Is it really #1 module??  *sigh*

Yup, by far.

> From my experience and that of some of programmers I know, PHP people are
> very reluctant to admit that they have bugs or fix their bugs.  Usually
> they find it better to argue that you're the idiot and their code works
> (even when it does not).

Most reported bugs are actually user error, and there are volunteers who
are usually not actually hard core PHP developers who go through and clean
up the bug reports and ask for more info when needed.  They don't always
get it right, but they reduce the load on the developers who tackle the
tough problems.

-Rasmus

Re: story posted

[Rasmus Lerdorf]

On Wed, 5 Feb 2003, Joshua Slive wrote:
> On Wed, 5 Feb 2003, Sascha Schumann wrote:
> > Now, we could solve both problems by using a handler and
> > the prefork MPM.  But then, Apache 2.0+PHP is basically
> > Apache 1.3+PHP with a few extra modules thrown in.  That's
> > how it appears to the end-user at least.
> 
> I don't buy that argument.  Are you saying that if Apache 1.4 had been
> released with a couple extra modules but no threading or filters, then PHP
> would have stuck with 1.3 because it works "well enough"?  No, I hope PHP
> would have updated to 1.4 to take advantage of the current development
> efforts.  Those development efforts include not just a couple new modules
> (major ones like mod_ssl, mod_dav, mod_deflate, mod_auth_ldap, etc), but
> lots and lots of other enhancements (IPv6, PCRE, improved negotiation,
> better documentation, better non-unix support, many bug fixes, etc).

The reality is that people can do SSL, DAV, Gzip, auth_ldap and everything 
else with 1.3.  Things are a bit more coherent in 2.0 and from a design 
perspective it all fits together in a nicer way, but an end user just 
installs the appropriate package for 1.3 and it just works.  The fact that 
the guy who rolled the package had a nightmare job to make it seamless is 
of no real concern to the end user.  

Our bread and butter here are the Linux and FreeBSD web servers out there.  
They do not run ipv6, they don't care about Windows and they certainly 
don't read documentation.  What are we offering them?

> So why not just do a handler-based PHP for 2.0, and work on other problems
> in the future.  This is a silly family quarel that is making everyone look
> bad.

Go for it.  You sound motivated.  http://www.php.net/cvs-php.php is where 
you sign up for a CVS account.  I will make sure you, or anyone else here 
interested in helping out gets a CVS account promptly.

-Rasmus

Re: story posted

[Rasmus Lerdorf]

> What I think is useful information to people who want PHP+Apache-2.0 is:
> 
> a) is PHP not production ready with Apache 2.0 because it was not high 
> enough priority for PHP to be tested?

That is a big part of it.  The fact that the thread-safety of many
third-party libraries that can be linked into a PHP (or mod_perl) setup is
questionable at best has not made us rush out to support Apache 2.  If
there was some compelling end-user feature that people clamoured for, it
would be a different story, but as it is, Apache2 is just a headache for
us.  People would have to run in prefork mode so neither the architecture
nor the performance are compelling arguments to switch for the end user.  
>From our perspective, the additional effort required to field all these
bug reports about Apache2+PHP+Worker doing weird and wonderful things
makes it quite a PITA and we thus push people toward 1.3.  Thread-safety
issues are hard to debug and causes a lot of indecipherable and mostly
useless bug reports.  RedHat is currently my least favourite company due 
to the added support email load they have caused me.  And yes, I know, 
"Just tell them to use prefork".  But with Apache-1.3 there is no Worker 
mpm and most of the time when you ask a user which mpm they are using, the 
response is a big fat, "duh, mpm?".

> b) is PHP not production ready with Apache 2.0 because Apache 2.0 must 
> be fixed before such a thing is even possible?
> 
> I see more comments implying b) than a).  As such, it seems that anybody 
> that wants to hack PHP to work better with 2.0 is wasting their time.

I don't think b) outweighs a) here.  The issues in a) is the demotivator 
and b) is just the final nail in the coffin.  If we were motivated enough 
we would work through the API stuff and figure out how to better get PHP 
aligned to the current filter approach.  It's not like the Apache 1.3 API 
is super clean either.  We just know it pretty well and PHP was built up 
around it almost from the beginning.

I guess the real issue here is not even the end users as much as the
developers themselves.  Apache 1.3 was historically written by webmasters
for webmasters running realworld servers.  When/if Apache2 reaches a point
where we, as developers, switch to using it for ourselves, PHP support
will arrive in full force.  As it is, it is just one server on the long
list of servers somewhat supported by PHP.

-Rasmus

Re: story posted

[Rasmus Lerdorf]

> You can also see text in our bug database from a prominent PHP developer 
> saying that the filter API needs to be redone from scratch (my 
> paraphrase).  For the enthusiastic PHP users, such comments carry a lot 
> of weight and imply that PHP isn't production ready with 2.0 not because 
>   nobody has made it a high priority to make PHP production ready but 
> instead because there is something flawed about Apache 2.0.

Well, I stand by that.  The filter API is a bloody mess.  Something which 
should be simple and elegant has turned into this amazingly complex tangle 
of code that is nearly impossible to figure out.

-Rasmus

Re: mod_usertrack patch

[Rasmus Lerdorf]

In cvs?  But no, it has not.  Everyone seems to be ignoring 1.3 these
days.

-Rasmus

On Wed, 27 Nov 2002, Andrei Zmievski wrote:

> Where can I check if it's been committed or not?
>
> On Fri, 22 Nov 2002, Rasmus Lerdorf wrote:
> > +1 on committing this.  I finally got around to testing it.
> >
> > -Rasmus
> >
> > On Thu, 7 Nov 2002, Andrei Zmievski wrote:
> >
> > > Hello,
> > >
> > > I am submitting a patch to mod_usertrack for your consideration.
> > > This patch was developed here at Fast Search & Transfer for
> > > alltheweb.com and provides the following changes:
> > >
> > >   - ability to have a "verbose" and a "compact" version of the cookie
> > >   - ability to set a prefix string for the cookie
> > >
> > > This patch is meant for 1.3.x versions.
> > >
> > > Please copy me on the replies.
> > >
> > > Regards,
> > >
> > > -Andrei
> > >
>
>
>
> -Andrei   http://www.gravitonic.com/
> * Black holes are where God divided by zero. *
>

Re: mod_usertrack patch

[Rasmus Lerdorf]

+1 on committing this.  I finally got around to testing it.

-Rasmus

On Thu, 7 Nov 2002, Andrei Zmievski wrote:

> Hello,
>
> I am submitting a patch to mod_usertrack for your consideration.
> This patch was developed here at Fast Search & Transfer for
> alltheweb.com and provides the following changes:
>
>   - ability to have a "verbose" and a "compact" version of the cookie
>   - ability to set a prefix string for the cookie
>
> This patch is meant for 1.3.x versions.
>
> Please copy me on the replies.
>
> Regards,
>
> -Andrei
>

Re: Apache 1.3 and invalid headers

[Rasmus Lerdorf]

You mean when you send a request header that looks something like this?

~> telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.1
Host: localhost
Connection: foo

HTTP/1.1 200 OK
Date: Wed, 20 Nov 2002 22:52:24 GMT
Server: Apache/1.3.28-dev (Unix) PHP/4.3.0RC1
X-Powered-By: PHP/4.3.0RC1
Last-Modified: Wed, 02 Oct 2002 13:34:42 GMT
Transfer-Encoding: chunked
Content-Type: text/html

99d



...

I don't get a closed connection right away.  I get all my output (chunked
in this case) and then my connection is closed.

-Rasmus

On Tue, 19 Nov 2002, Andy Yang wrote:

> Hello all,
>
> Does anyone know what the behaviour of Apache 1.3 is
> under the circumstances where the HTTP request or
> response contains an invalid request header?
>
> Specifically, when the Connection header contains
> something other than 'close'?
>
> It appears to immediately close the connection - can
> anyone confirm or deny that this is Apache's behaviour
> for both requests and responses?
>
> Thanks,
> Andy
>
>
> __
> Do you Yahoo!?
> Yahoo! Web Hosting - Let the expert host your site
> http://webhosting.yahoo.com
>

Re: PHP POST handling

[Rasmus Lerdorf]

> On Tue, Oct 01, 2002 at 04:27:15PM -0400, Ryan Bloom wrote:
> > On Tue, 1 Oct 2002, William A. Rowe, Jr. wrote:
> >
> > > At 01:12 PM 10/1/2002, Greg Stein wrote:
> > > >For PHP, we said "make it a filter [so the source can come from anywhere]".
> > > >I think we really should have said "for GET requests, allow it to be
> > > >processed by PHP." The POST, PROPFIND, COPY, etc should all be possible to 
>handle by PHP, which means that PHP also needs a handler.
> > >
> > > Agreed, if you write a PHP script we better allow you to PROPFIND
> > > or COPY the puppy, in addition to POST.
> >
> > These are two different statements, if I am reading both
> > correctly.  Please correct me if I am not.  Will, you are saying that if
> > we have a PHP script, then we need to be able to do all DAV operations on
> > the script.  Greg, you are saying that a PHP script needs to be able to
> > satisfy a DAV request (meaning that the PHP code actually copies the
> > resource, or generates the PROPFIND data).
> >
> > Assuming I am reading the two statements correctly, I agree with
> > Will, but not with Greg.
>
> Why couldn't mod_dav be implemented in PHP? I see no particular reason why
> not... Currently, PHP cannot because it is a filter, not a handler.

We have a switch in PHP now to handle mod_dav requests actually (under
1.3.x)  There is no specific DAV support in there, it's just a switch that
allows PHP to be a handler for things other than GET, HEAD and POST so
people can implement whatever DAV stuff they want in userspace.

-Rasmus

Re: PHP and Apache 1.3 issue ...

[Rasmus Lerdorf]

See his domain name.  Novell.com

On Sat, 28 Sep 2002, Jim Jagielski wrote:

> Ramesh Shankar wrote:
> >
>
> Threads? Is this under Win32?
> --
> ===
>Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
>   "A society that will trade a little liberty for a little order
>  will lose both and deserve neither" - T.Jefferson
>

Re: E-Kabong resolution: Re: acceptance of El-Kabong into APR

[Rasmus Lerdorf]

> > While we believe that the El-Kabong codebase is a valuable contribution
> > that we would like to pick up, we also recognize the possibility that
> > you may have a decreased incentive to work on it further if you are
> > initially denied commit access to the repository when it moves to the
> > ASF.  If this is the case, we understand that it may be best for the
> > codebase to live elsewhere if it would be harmed by a lack of
> > contributions from its original author, and we look to you for further
> > guidance on this issue.

I find this rather shitty.  Obviously someone who donates code is going to
want to be able to work on it so this is the same as telling Jon to take a
hike.  It would have been much more forthright and honest to just tell him
to take a hike.

-Rasmus

Re: Thread safety issues

[Rasmus Lerdorf]

Contributions are more than welcome.

On Tue, 10 Sep 2002, Cyrille Artho wrote:

> Hi,
> as someone who works on multi-threaded problems, but not Apache, I ran
> into your page at
> http://httpd.apache.org/docs-2.0/developer/thread_safety.html
>
> I strongly suggest to revise it, because it lacks depth. First of all,
> you probably should start explaining the basic problem in case a
> developer has never used threads: what is a race condition (and what is
> a deadlock), why is it hard to test (because it is scheduling-dependent,
> the scheduling cannot be controlled).
>
> What can be done about it (manual analysis, VisualThreads under Alpha).
> Basically the programmer has to ensure that each access to shared data
> is protected by at least one common lock (the word "common" is important).
>
> When to use apr_atomic_* and apr_thread_mutex_*. How to use them.
>
> Perhaps make a small shell script that parses the ldd output and
> compares each line to a "database" of libraries in it, printing
> problematic libraries.
>
> Maybe you can cover some of these issues by linking to other pages. It
> is important that they are well documented, otherwise people will never
> dare using 2.0 or writing modules for it.
>
> I personally hope 2.0 will be adapted fast, but I know such things take
> time. Good luck, and keep up the good work!
> --
> Regards,
> Cyrille Artho
>

Re: El-Kabong -- HTML Parser

[Rasmus Lerdorf]

> > Which closed doors are those?  There has been discussion on the dev list
> > and on the board list.  Both of which are public lists that you can
> > subscribe to.
>
> All I know of is the PMC list (which is private), but discussion on
> board@ (which is also private) is news to me.

Well, I had assumed you were already an ASF member.  I guess I was wrong.
Yes, the board list is private in the sense that non-ASF members can not
subscribe.  Any ASF member can though.

-Rasmus

Re: El-Kabong -- HTML Parser

[Rasmus Lerdorf]

> The ASF is apparently not about working together, since I (and
> everyone else who is not on the PMC list) have been entirely left
> out of all this conversation which is going on behind closed doors.

Which closed doors are those?  There has been discussion on the dev list
and on the board list.  Both of which are public lists that you can
subscribe to.

> It seems as though that is exactly what you've done -- "ignored it."
> I am constantly probing for information as to where this stands, both
> via these lists and asking on the #apr IRC channel.  You're interpretation
> if "holding this over your head" is fairly outrageous, since Apache
> exists perfectly fine without this contribution.

I agree with you that Greg may have misinterpreted your message somewhat
and come on a little strong, but everything I have seen has pointed at
putting it somewhere, just nobody is sure where.  I have yet to see you
suggest something to help place it properly, but I could easily have
missed that message in my flood of email.

-Rasmus

Re: [VOTE] Location of aaa rewrite

[Rasmus Lerdorf]

> On Tue, 3 Sep 2002, Rasmus Lerdorf wrote:
>
> > > On Tue, 3 Sep 2002, Chris Taylor wrote:
> > >
> > > > -BEGIN PGP SIGNED MESSAGE-
> > > > Hash: SHA1
> > > >
> > > > > [ ] Check in aaa rewrite to 2.0.
> > > > > [X] Check in aaa rewrite to 2.1.
> > > >
> > > > My view is that it's important to keep 2.0 stable to attract new
> > > > users, and breaking things all the time won't help :)
> > >
> > > Can I ask a stupid question?  What have we actually broken since Apache
> > > 2.0 went GA?  Binary compatibility?  How many functions?  How many of
> > > those were APR and not Apache?
> >
> > Sure, both source and binary compatibility were broken numerous times.
> > Check the PHP bug database for umpteen reports on each breakage.
>
> Okay, but how is that different from early releases of 1.3?

I don't know, but you asked the question which sort of implied that
nothing had been broken.

-Rasmus

Re: [VOTE] Location of aaa rewrite

[Rasmus Lerdorf]

> On Tue, 3 Sep 2002, Chris Taylor wrote:
>
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA1
> >
> > > [ ] Check in aaa rewrite to 2.0.
> > > [X] Check in aaa rewrite to 2.1.
> >
> > My view is that it's important to keep 2.0 stable to attract new
> > users, and breaking things all the time won't help :)
>
> Can I ask a stupid question?  What have we actually broken since Apache
> 2.0 went GA?  Binary compatibility?  How many functions?  How many of
> those were APR and not Apache?

Sure, both source and binary compatibility were broken numerous times.
Check the PHP bug database for umpteen reports on each breakage.

-Rasmus

Re: Is anybody getting CVS commit messages?

[Rasmus Lerdorf]

I see them

On Tue, 27 Aug 2002 [EMAIL PROTECTED] wrote:

>
> I realized earlier today that I haven't been seeing commit messages.
>
> Is anybody getting these messages?
>
> Ryan
>
> --
>
> ___
> Ryan Bloom[EMAIL PROTECTED]
> 550 Jean St
> Oakland CA 94610
> ---
>

Re: Thread-unsafe libraries in httpd-2.0

[Rasmus Lerdorf]

> FWIW, I'm +1 on what Rasmus says - at least for widely used libraries.
> Obviously anything internal to PHP is PHP's problem. But anything
> everyone uses is our problem.
>
> However, I would advocate fixing the libraries rather than wrapping them
> where possible.

Yup, definitely.  They aren't always open source or sometimes they are
just architecturally not fixabled.  Step 1 is to gather information.  I
have committed an initial short list to

httpd-2.0/docs/manual/developer/thread_safety.html

where I plan on keeping track of stuff as the PHP bug reports flow in or I
dive into each library to check out the code.

-Rasmus

Re: Thread-unsafe libraries in httpd-2.0

[Rasmus Lerdorf]

That's just crap.  I can say the same thing.  PHP is threadsafe, it's not
my responsibility that libxyz is not.  Any Apache 2.0 module running under
a threaded mpm linked against libxyz that contains non-threadsafe code is
going to have exactly the same problem unless the module sticks in
mutexes.

Right now in the PHP world we simply tell people that they should not
upgrade to Apache 2.0.  If everyone ends up doing that, then I am sorry to
say, but Apache 2.0 is dead and the current Netcraft statement is going to
be a recurring thing.

As a platform there are some cool possibilities with Apache 2.0 and I'd
like to see it do well, but as a module author, being told to piss off and
deal with the problem myself is not any way to encourage 3rd-party module
support for Apache 2.0.  In the end the users out there don't really give
a crap about which web server they use.  They just want something that
works.  Apache was always the web server that just worked.  I strongly
feel it is our job to help make sure that Apache remain the web server
that just works.  Like it or not, but the web server is the foundation for
all the projects under the ASF umbrella and if we fuck up Apache, every
sub-project will be hurt in the process.

-Rasmus


On Thu, 15 Aug 2002, Greg Stein wrote:

> EXACTLY.
>
> Thanks, Justin.
>
> We are not trying to shirk our responsibilities or be lazy about this. But
> you can't say "my module is so popular that you must account for problems
> that I introduce into your environment."
>
> I'm fine with adding something to our document that says something along the
> lines of, "if you choose a threaded MPM such as FOO, BAR, or BAZ, then you
> need to ensure that the third-party modules you choose to use with the web
> server are thread-safe. Please contact your third-party modules' vendors for
> more information on their thread-safety."
>
> -g
>
> On Thu, Aug 15, 2002 at 10:09:27AM -0700, Justin Erenkrantz wrote:
> > On Thu, Aug 15, 2002 at 09:40:06AM -0700, Rasmus Lerdorf wrote:
> > > thttpd/Zeus/boa/Tux/khttpd for that.  All I am after is a simple very
> > > visible addition to the Apache 2 distribution which explains that the
> > > threaded mpms may not be suitable for serving up dynamic content due to
> > > the unknown thread safety of the libraries these dynamic solutions rely
> > > on.
> >
> > And, my point back to you is that should be part of the documentation
> > of the module NOT of httpd-2.0.  Making broad statements that will
> > confuse our users like "threaded MPMs may not be suitable for serving
> > up dynamic content" is a ridiculously overbroad and inaccurate
> > statement.
> >
> > A better statement may be: "Some PHP or Perl modules may not
> > interact well with a threaded MPM in httpd-2.0.  Caution is urged
> > when using a threaded MPM."  To me, that totally belongs in the
> > PHP or Perl documentation.  That is a limitation of PHP and mod_perl
> > not of httpd-2.0.
> >
> > That statement doesn't hold for a mod_jk2 (or whatever the latest
> > httpd-2.0 Tomcat module is).  It totally depends on how the 3rd
> > party module is architected not on the architecture of the web
> > server itself.  -- justin
>
> --
> Greg Stein, http://www.lyra.org/
>

Re: Thread-unsafe libraries in httpd-2.0

[Rasmus Lerdorf]

On Thu, 15 Aug 2002, Justin Erenkrantz wrote:

> [ Moving to dev@httpd where this belongs ]
>
> On Thu, Aug 15, 2002 at 07:10:02AM -0700, Rasmus Lerdorf wrote:
> > Correct.  From the feedback I am getting, users do not adequately
> > understand the implications of choosing a threaded MPM.  We need to do a
> > better job educating them to the fact that we simple cannot guarantee
> > which libraries are threadsafe and which are not.
>
> Precisely which libraries are the core of httpd-2.0 using in a
> thread-unsafe manner?  At least in the core distribution, we
> should have knowledge of which libraries are thread-unsafe and
> prevent this.  To my knowledge, I haven't seen a single problem
> related to this.  If you have actual cases where httpd-2.0 is
> doing something unsafe, please let us know!
>
> I think perhaps your concern is simply to PHP which may bring
> in thread-unsafe libraries not httpd-2.0.  -- justin

As I have stated a dozen times now.  Yes, I am talking about PHP and any
other add-on module people may use.  I think the attitude of most people
here sucks.  "Apache doesn't have a problem, why should we do anything?"

The reality is that a big chunk of our Apache community uses Apache to
serve up complex dynamic content using PHP, mod_perl, other 3rd-party
modules and homegrown modules as well. PHP and mod_perl alone are on well
over 50% of all Apache servers. People don't look to Apache for simply
serving up static content. A lot of sites replaced Apache long ago with
thttpd/Zeus/boa/Tux/khttpd for that.  All I am after is a simple very
visible addition to the Apache 2 distribution which explains that the
threaded mpms may not be suitable for serving up dynamic content due to
the unknown thread safety of the libraries these dynamic solutions rely
on.

-Rasmus

Re: perchild on FreeBSD 5?

[Rasmus Lerdorf]

> >If we end up having to mutex, sure, if every request for that domain needs
> >to access the mutxed library, then yes, you would only be able to serve up
> >one request at a time from that domain.
> >
> >
>
> That's not accurate in general.  The amount of serialization will depend on
> how much time you spend in mutexed operations in that library, relative
> to the entire request processing time.  If the time spent in mutex-protected
> code is 1% of the total request processing time on an otherwise idle server,
> you'll see good concurrency as you add more requests.  In contrast, if the
> time spent in mutex-protected code is 99% of the total request processing
> time, then the server will scale poorly.  The key success factor is to not
> use libraries that require locking for lengthy operations.

If, for example, we have to mutex an entire database library and every
request needs to make an sql query, then effectively we are down to
serializing the requests and only handling one at a time.  And yes, it
goes without saying that it is better to use libraries that don't need to
be locked.  I don't think he was asking for the best-case scenario when he
asked that question.

-Rasmus

Re[2]: perchild on FreeBSD 5?

[Rasmus Lerdorf]

> Hi Rasmus Lerdorf,
> you wrote.
>
> RL> libraries as well.  We are going to have to fix a bunch of them and mutex
> RL> some others before Apache2 with a threaded MPM will be of any use with PHP
> RL> or mod_perl.
>
> Am I correct assuming that when they are mutex'ed that there will be
> one instance of XX_lib PER perchild process, right? So eventually,
> each domain will be able to serve one request at a time?

If we end up having to mutex, sure, if every request for that domain needs
to access the mutxed library, then yes, you would only be able to serve up
one request at a time from that domain.

-Rasmus

RE: perchild on FreeBSD 5?

[Rasmus Lerdorf]

> > Apache 2 is being pitched as a production-quality web server without any
> > mention of the fact that over half of the current Apache sites probably
> > should not upgrade at this point.
>
> Sorry to disagree, Rasmus, but if you're talking about PERCHILD, that's a
> threaded MPM that will not solve your problem with those libraries used by
> PHP which are not thread safe.
>
> If your problem is thread safety, use PREFORK, it works allright, and AFAICS
> (from my tests and my experience, I don't want to push my results as "global",
> PREFORK works at least as good as the old 1.3.
>
> Plus 2.0 has WORKER, that for less-than-half of this planet is, IMVHO, a
> great improvement over the old 1.3 when thread safety is not a problem.
>
> So, what's your point? Apache 2.0 _already_ delivers, for what I can see and
> experience on my silly little websites doing several millions hits/day. It can
> be used AS 1.3, _and_ it has something extra.
>
> I really don't see your point...

My point is that people blindly install the latest stable PHP with the
latest stable Apache and they end up something that is not stable at all.
They don't understand what an MPM is, much less which MPM to use for what.
The troublesome part here is that this is actually more than half of
all the current Apache users that will end up getting bitten if they
blindly upgrade.  Nowhere in the release notes do we warn people that
things are likely to break if they choose a threaded MPM and have PHP
and/or mod_perl.

-Rasmus

RE: perchild on FreeBSD 5?

[Rasmus Lerdorf]

> > I am sure it does, but you also need to recognize that around 40% of all
> > Apache installations have PHP on them.  And 30% or so have mod_perl.  Many
> > actually have both, so there is some overlap, but I would say that over
> > half of all Apache installs have one or both of PHP or mod_perl.
> >
> > Apache 2 is being pitched as a production-quality web server without any
> > mention of the fact that over half of the current Apache sites probably
> > should not upgrade at this point.
>
> This is definitely worth mentioning in the 2.0 Announcements. Have the
> problems in mod_perl and mod_php been identified, so folks can at least make
> an educated guess whether their site will tickle the known bugs?

Nope, we really have no idea which libraries are threadsafe and which ones
aren't.

-Rasmus

RE: perchild on FreeBSD 5?

[Rasmus Lerdorf]

> I agree with Pier, the threaded MPM has been a real lifesaver. Supporting
> 10,000+ concurrent clients with Apache 1.3 (including some complex modules)
> on AIX and Solaris is practically impossible.  Quite doable with Apache 2.0
> and the worker MPM.

I am sure it does, but you also need to recognize that around 40% of all
Apache installations have PHP on them.  And 30% or so have mod_perl.  Many
actually have both, so there is some overlap, but I would say that over
half of all Apache installs have one or both of PHP or mod_perl.

Apache 2 is being pitched as a production-quality web server without any
mention of the fact that over half of the current Apache sites probably
should not upgrade at this point.

-Rasmus

Re: perchild on FreeBSD 5?

[Rasmus Lerdorf]

> Hello,
> I've more or less accepted that perchild on FreeBSD 4.X isn't going to
> happen (as sad as it is, I always considered it to be THE feature [1] in
> 2.0 that would warrant an upgrade for us) but what I'd like to know is
> if there is any chance to see perchild on FreeBSD 5 which gets wholly
> new threading and SMP libs?

I agree, and I have been preaching the same thing for a while.  Almost no
point in releasing Apache2 without a working perchild.  Unfortunately
there are other issues as well.  A lot of the 3rd party libs that
something like PHP or mod_perl depends on are not necessarily threadsafe.
As witnessed by FreeBSD's incredibly buggy threading code there aren't a
lot of things using threads heavily on UNIX.  With some notable
exceptions, of course, but very few try to pull in 30 or 40 3rd-party
libraries as well.  We are going to have to fix a bunch of them and mutex
some others before Apache2 with a threaded MPM will be of any use with PHP
or mod_perl.  And I am not sure how to go about identifying the libraries
that aren't qiute threadsafe.  Problems generally only show up under load
and only in certain circumstances.  Especially for the libraries that
claim to be threadsafe but aren't quite for whatever reason.

-Rasmus

Re: PATH_INFO in A2?

[Rasmus Lerdorf]

> On Sat, Jul 06, 2002 at 03:11:20PM -0400, [EMAIL PROTECTED] wrote:
> > We just added a new function for all input filters to allow this to be
> > done (Justin referenced it in his reply).  However, that function doesn't
> > solve the problem, because there should be an ap_filter_is_dynamic(r) that
> > hides the implementation details for all filters.
>
> I don't believe that mod_include would want AcceptPathInfo on.
> Only PHP would.  So, I don't know what ap_filter_is_dynamic() would
> buy here (other than setting r->no_local_copy to 1).  -- justin

Why wouldn't mod_include want AcceptPathInfo to be on?  I am sure there
are .shtml files out there that are written with the assumption that
PATH_INFO is set correctly.

-Rasmus

RE: PATH_INFO in A2?

[Rasmus Lerdorf]

> > What is a dynamic page if not a PHP page?
>
> Like I said, Apache doesn't know if a file on disk is meant for PHP or
> not.  The best way to fix this would be for mod_php to set the value if
> the filter is added for the request.
>
> I agree, it would be cool if Apache could set this correctly based on
> the filters that have been added for the request.

Seems like there should be an API call where a filter can specify whether
it is a dynamic one or not as opposed to specifically flipping the
acceptpathinfo bit.

-Rasmus

RE: PATH_INFO in A2?

[Rasmus Lerdorf]

> > > Make sure you have AcceptPathInfo On
> >
> > Argh!  Why the heck is that off by default?
>
> It's on by default for dynamic pages, but there is no way that Apache
> can tell that a page is going to be served by PHP, so it is off for what
> Apache thinks are static pages.

What is a dynamic page if not a PHP page?

-Rasmus

Re: PATH_INFO in A2?

[Rasmus Lerdorf]

On Sat, 6 Jul 2002, Dale Ghent wrote:

> On Sat, 6 Jul 2002, Rasmus Lerdorf wrote:
>
> | 2.0.40-dev built on June 23rd
>
> Make sure you have AcceptPathInfo On

Argh!  Why the heck is that off by default?

-Rasmus

Re: PATH_INFO in A2?

[Rasmus Lerdorf]

2.0.40-dev built on June 23rd

On Sat, 6 Jul 2002, Dale Ghent wrote:

> On Sat, 6 Jul 2002, Rasmus Lerdorf wrote:
>
> | The requested URL /index.php/foo was not found on this server.
> |
> | In Apache1 /index.php would still get passed to PHP and /foo would end up
> | in the PATH_INFO server var.
>
> What AP2 code are you using? I'm running .40-dev with a CVS update as of
> last afternoon (right after Bpane's apr_table mods) and PHP 4.2.1 (with
> the set-cookie fix)... everything is fine for me on a site that reqires
> PATH_INFO ( http://playadelfuego.org/ )
>
> /dale
>

PATH_INFO in A2?

[Rasmus Lerdorf]

With this config:

LoadModule php4_modulemodules/libphp4.so
AddType application/x-httpd-php .php

http://localhost/index.php

works fine and parser PHP pages without problems.  (using prefork - but it
shouldn't matter)

However: http://localhost/index.php/foo

Gives a 404 and says:

The requested URL /index.php/foo was not found on this server.

In Apache1 /index.php would still get passed to PHP and /foo would end up
in the PATH_INFO server var.

-Rasmus

Re: Christopher Williamson: URGENT: Bug/compatability issue in Apache 1.3.26

[Rasmus Lerdorf]

http://www.apache.org/dist/httpd/patches/ has patches for every released
version of Apache 1.2.x and 1.3.x

On Wed, 3 Jul 2002, Andrew Ho wrote:

> Hello,
>
> Is there a patch for earlier versions of Apache that fix the chunked
> Transfer-Encoding security hole, but nothing else? I know OpenBSD, for
> example, has an Apache 1.3.24 in ports that has the chunked
> Transfer-Encoding fixed. That would certainly be a good short term
> solution for this guy--it doesn't sound like it's a good idea for him to
> wait for 1.3.27!
>
> Humbly,
>
> Andrew
>
> --
> Andrew Ho   http://www.tellme.com/   [EMAIL PROTECTED]
> Engineer   [EMAIL PROTECTED]  Voice 650-930-9062
> Tellme Networks, Inc.   1-800-555-TELLFax 650-930-9101
> --
>

Keepalives in A2?

[Rasmus Lerdorf]

With Apache 1.3 all you had to do to get a keep-alive was set your
content-length correctly:

HTTP/1.1 200 OK
Date: Mon, 24 Jun 2002 17:05:04 GMT
Server: Apache/1.3.22 (Unix) PHP/4.3.0-dev
X-Powered-By: PHP/4.3.0-dev
Content-length: 1024
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html

One of the benefits with Apache 2's architecture is that it knows the
content length and sets it.  But I never seem to get a keep-alive from it.
Identical request and identical PHP script with Apache 2:

HTTP/1.1 200 OK
Date: Mon, 24 Jun 2002 17:01:17 GMT
Server: Apache/2.0.40-dev (Unix) PHP/4.3.0-dev
Accept-Ranges: bytes
X-Powered-By: PHP/4.3.0-dev
Content-Length: 1024
Connection: close
Content-Type: text/html; charset=ISO-8859-1

Is there something else I need to do in Apache2 to get a keep-alive
response?

-Rasmus

Re: Apache 2.0 Numbers

[Rasmus Lerdorf]

> * Saying "turn on buffering" is, IMHO, a reasonable solution if you
>   can make buffering the default in PHP under httpd-2.0.  Otherwise,
>   you'll surprise a lot of users who have been running with the default
>   non-buffered output using 1.3 and find that all their applications
>   are far slower with 2.0.

We could turn on buffering for 2.0.  I just verified that this does indeed
create a single 1024-byte bucket for my 1024-byte file test case.  And
combined with compiling PHP non-threaded for the prefork mpm the result
is:

Concurrency Level:  5
Time taken for tests:   115.406395 seconds
Complete requests:  5
Failed requests:0
Write errors:   0
Keep-Alive requests:0
Total transferred:  6325 bytes
HTML transferred:   5120 bytes
Requests per second:433.25 [#/sec] (mean)
Time per request:   11.541 [ms] (mean)
Time per request:   2.308 [ms] (mean, across all concurrent requests)
Transfer rate:  535.21 [Kbytes/sec] received

Up from 397 requests/second but still nowhere near the 615 requests/second
for Apache 1.3.  But, doing this buffering internally in PHP and then
again in Apache doesn't seem efficient to me, and the numbers would seem
to reflect this inefficiency.

> * A better solution, though, would be to have the PHP filter generate
>   flush buckets (in nonbuffered mode) only when it reaches a "<%" or
>   "%>".  I.e., if the input file has 20KB of static text before the
>   first embedded script, send that entire 20KB in a bucket, and don't
>   try to split it into 400-byte segments.  If mod_php is in nonbuffered
>   mode, send an apr_bucket_flush right after it.  (There's a precedent
>   for this approach: one of the ways in  which we managed to get good
>   performance from mod_include in 2.0 was to stop trying to split large
>   static blocks into small chunks.  We were originally concerned about
>   the amount of time it would take for the mod_include lexer to run
>   through large blocks of static content, but it hasn't been a problem
>   in practice.)
>
> From a mod_php perspective, would either of those be a viable solution?

I think Andi is working on this.  But, just to test the theory, I modified
the PHP lexer to use larger chunks.  1024 in this case.  So, the 1k.php
test case which looks like this:


Test Document.

Test Document.


aa
bb
cc
dd
ee
ff
gg
hh
ii
jj
kk
ll
mm
nn
oo
pp
qq
rr
ss
tt
uu
vv
ww
xx




Was split up into 3 buckets.

1. 78 bytes containing:


Test Document.

Test Document.


2. 30 bytes containing (because this was dynamically generated)
This is a 1024 byte HTML file.

3. A 916 byte bucket containing the rest of the static text.

Result:

Concurrency Level:  5
Time taken for tests:   124.456357 seconds
Complete requests:  5
Failed requests:0
Write errors:   0
Keep-Alive requests:0
Total transferred:  6325 bytes
HTML transferred:   5120 bytes
Requests per second:401.75 [#/sec] (mean)
Time per request:   12.446 [ms] (mean)
Time per request:   2.489 [ms] (mean, across all concurrent requests)
Transfer rate:  496.29 [Kbytes/sec] received


So slower than the single 1024 byte bucket and actually also slower than
the 400-byte case, so an invalid test.  There are probably some other
memory-allocation changes I would need to make to make this a valid test.

-Rasmus

RE: Apache 2.0 Numbers

[Rasmus Lerdorf]

> As it happens, DONE is defined to be -2.   :-)

Ok, I will use that, but 'DONE' doesn't really give the impression of
being a fatal error return value.

-Rasmus

RE: Apache 2.0 Numbers

[Rasmus Lerdorf]

On Mon, 24 Jun 2002, Cliff Woolley wrote:
> On Mon, 24 Jun 2002, Rasmus Lerdorf wrote:
>
> > Hrm..  Nope.  doing 'return DECLINED' from the post_config phase does not
> > stop the server from starting.  I have this:
>
> I thought you were supposed to return HTTP_INTERNAL_SERVER_ERROR.

In include/http_config.h it says:

/**
 * Run the post_config function for each module
 * @param pconf The config pool
 * @param plog The logging streams pool
 * @param ptemp The temporary pool
 * @param s The list of server_recs
 * @return OK or DECLINED on success anything else is a error
 */

So I guess I need to return 'anything else'

Trying this, ie. returning -2 it does the job.  But this seems a little
vague.  Should we perhaps have a #define FATAL -2 or something similar so
I don't get stepped on later on if someone decides to use -2 for something
else?

And no, I don't think it makes sense to overload INTERNAL_SERVER_ERROR for
this.  To me that is a 500 error which is very much tied to a
request-level error.

-Rasmus

RE: Apache 2.0 Numbers

[Rasmus Lerdorf]

On Mon, 24 Jun 2002, Rasmus Lerdorf wrote:

> > > What is the correct way to fail in a filter post_config?  Do I return
> > -1
> > > from it if my filter finds a fatal error?  I can't use ap_log_rerror()
> > at
> > > this point, right?  How would I log the reason for the failure?
> >
> > I'm confused by the question, but I'll try to answer.  If you mean the
> > post_config phase, then you can use ap_log_error or ap_log_perror.  If
> > you want to stop the server from starting, just return DECLINED.
>
> Right, I found ap_log_error.  It was the return value I was looking for.
> None of the example filter modules had a fatal error check at the config
> phase.  So returning a -1 is the correct way to stop the server from
> starting.  Thanks.

Hrm..  Nope.  doing 'return DECLINED' from the post_config phase does not
stop the server from starting.  I have this:

static int
php_apache_server_startup(apr_pool_t *pconf, apr_pool_t *plog, apr_pool_t *ptemp, 
server_rec *s)
{
void *data = NULL;
const char *userdata_key = "apache2filter_post_config";
#ifndef ZTS
int threaded_mpm;

ap_mpm_query(AP_MPMQ_IS_THREADED, &threaded_mpm);
if(threaded_mpm) {
ap_log_error(APLOG_MARK, APLOG_CRIT, 0, s, "Apache is running a threaded MPM, 
but your PHP Module is not compiled to be threadsafe.  You need to recompile PHP.");
return DECLINED;
}
#endif
...
}

...

ap_hook_pre_config(php_pre_config, NULL, NULL, APR_HOOK_MIDDLE);
ap_hook_post_config(php_apache_server_startup, NULL, NULL, APR_HOOK_MIDDLE);

And in my log I get:

[Mon Jun 24 08:27:23 2002] [crit] Apache is running a threaded MPM, but your PHP 
Module is not compiled to be threadsafe.  You need to recompile PHP.
[Mon Jun 24 08:27:23 2002] [crit] Apache is running a threaded MPM, but your PHP 
Module is not compiled to be threadsafe.  You need to recompile PHP.
[Mon Jun 24 08:27:23 2002] [notice] Apache/2.0.40-dev (Unix) configured -- resuming 
normal operations

RE: Apache 2.0 Numbers

[Rasmus Lerdorf]

> > What is the correct way to fail in a filter post_config?  Do I return
> -1
> > from it if my filter finds a fatal error?  I can't use ap_log_rerror()
> at
> > this point, right?  How would I log the reason for the failure?
>
> I'm confused by the question, but I'll try to answer.  If you mean the
> post_config phase, then you can use ap_log_error or ap_log_perror.  If
> you want to stop the server from starting, just return DECLINED.

Right, I found ap_log_error.  It was the return value I was looking for.
None of the example filter modules had a fatal error check at the config
phase.  So returning a -1 is the correct way to stop the server from
starting.  Thanks.

-Rasmus

RE: Apache 2.0 Numbers

[Rasmus Lerdorf]

> Runtime is harder, but you can just use ap_mpm_query to get the MPMs
> characteristics.  This won't give you the MPM name, but it will let you
> know if the MPM is threaded or not.

What is the correct way to fail in a filter post_config?  Do I return -1
from it if my filter finds a fatal error?  I can't use ap_log_rerror() at
this point, right?  How would I log the reason for the failure?

-Rasmus

RE: Apache 2.0 Numbers

[Rasmus Lerdorf]

> > > It would be nice
> > > if there was an apxs flag that would return the MPM type.
> >
> > +1
>
> There is.  -q will query for any value in config_vars.mk, and MPM_NAME
> is in that file.  So `apxs -q MPM_NAME` will return the configured MPM
> type.

Ah right.  Is there a way to check at runtime as well?  I've added a PHP
configure check now to the apache2filter sapi module so it will come up
non-threaded by default if it sees Apache2-prefork.  Just a bit worried
about someone changing their MPM after the fact, so perhaps a runtime
check is needed as well.

-Rasmus

Apache 2.0 Numbers

[Rasmus Lerdorf]

Someone asked me for numbers when I mentioned the other day that Apache
2-prefork was really not a viable drop-in replacement for Apache 1.3 when
it comes to running a PHP-enabled server.

Apache 1.3 is still significantly faster than Apache2-prefork for both
static and dynamic content.  Now, part of the blame goes to PHP here for
the dynamic case. We are compiling PHP in threadsafe mode when building
the PHP DSO for Apache2-prefork which is not necessary. It would be nice
if there was an apxs flag that would return the MPM type. Right now we
would need to parse the output of httpd -l or -V to figure out which MPM
is being used. Being able to go non-threadsafe in PHP does speed us up a
bit. But none of this has anything to do with the fact that Apache 1.3 is
faster for static files.  It's going to be very hard to convince people to
switch to Apache2-prefork if we can't get it to go at least as fast as 1.3
for simple static files.

Platform: Linux 2.4.19-pre8, glibc 2.2.5, gcc-2.96, P3-800, 128M
Tested using ab from the httpd-2.0 tree with these flags: -c 5 -n 5 -k

1024-byte file which looked like this:


Test Document.

Test Document.

This is a 1024 byte HTML file.
aa
bb
cc
dd
ee
ff
gg
hh
ii
jj
kk
ll
mm
nn
oo
pp
qq
rr
ss
tt
uu
vv
ww
xx




The PHP version was:


Test Document.

Test Document.


aa
bb
cc
dd
ee
ff
gg
hh
ii
jj
kk
ll
mm
nn
oo
pp
qq
rr
ss
tt
uu
vv
ww
xx




Note the fact that the Apache2 static test produced the wrong number of
total bytes.  3072 bytes too many???  Where in the world did they come
from?  The PHP test on Apache2 produced the correct number as did both
static and PHP on Apache1.


Apache 2 PreFork

KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout15
StartServers 5
MinSpareServers  5
MaxSpareServers 10
MaxClients  15
MaxRequestsPerChild  0

STATIC

Concurrency Level:  5
Time taken for tests:   23.793270 seconds
Complete requests:  5
Failed requests:0
Write errors:   0
Keep-Alive requests:49511
Total transferred:  66681859 bytes
HTML transferred:   51203072 bytes <=== Uh?
Requests per second:2101.43 [#/sec] (mean)
Time per request:   2.379 [ms] (mean)
Time per request:   0.476 [ms] (mean, across all concurrent requests)
Transfer rate:  2736.87 [Kbytes/sec] received

PHP

Concurrency Level:  5
Time taken for tests:   125.831896 seconds
Complete requests:  5
Failed requests:0
Write errors:   0
Keep-Alive requests:0
Total transferred:  6325 bytes
HTML transferred:   5120 bytes
Requests per second:397.36 [#/sec] (mean)
Time per request:   12.583 [ms] (mean)
Time per request:   2.517 [ms] (mean, across all concurrent requests)
Transfer rate:  490.87 [Kbytes/sec] received


Apache 1.3
--
Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout   15
MinSpareServers 5
MaxSpareServers10
StartServers5
MaxClients 15
MaxRequestsPerChild 0
---
STATIC
---
Concurrency Level:  5
Time taken for tests:   19.735772 seconds
Complete requests:  5
Failed requests:0
Write errors:   0
Keep-Alive requests:49507
Total transferred:  66127818 bytes
HTML transferred:   5120 bytes
Requests per second:2533.47 [#/sec] (mean)
Time per request:   1.974 [ms] (mean)
Time per request:   0.395 [ms] (mean, across all concurrent requests)
Transfer rate:  3272.08 [Kbytes/sec] received
---
PHP
---
Concurrency Level:   

Karma please

[Rasmus Lerdorf]

Could someone karma me for apache-2.0 please?

-Rasmus

Re: I asked this on the freebsd-general list...

[Rasmus Lerdorf]

On Tue, 18 Jun 2002, Rick Kukiela wrote:

> In the last episode (Jun 17), Rick Kukiela said:
> > I asked on the apache-dev list about when perchild processing will work on
> > freebsd and this was the response i got:
> >
> > "It will never work with FreeBSD 4.6. Perchild requires a good threading
> > library, and FreeBSD doesn't have one."
> >
> > What is being done about this?
>
> THEIR REPLY:
> Unless they provide more details (i.e. proof that it's bad), there's
> not much that can be done.  The userland threads system is good enough
> to run MySQL, mozilla, openoffice, and who knows how many other pieces
> of software.

Well, there are some nasty bugs in FreeBSD's threads.  See
http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc_r/uthread/uthread_sendfile.c

This fix is rather crucial to Apache2 and it is not in any released
version of FreeBSD yet.  So yes, until FreeBSD fixes their threads, and
actually release the fix, Apache2 in threaded mode is not going to work on
that OS.

-Rasmus

Re: Need a new feature: Listing of CGI-enabled directories.

[Rasmus Lerdorf]

mod_info will tell you some of this.  ie. Look for ScriptAlias lines under
mod_alias.c and AddHandler cgi-script lines under mod_mime.c.

But you are fighting a bit of a lost cause here.  If you allow users to
upload arbitrary cgi scripts there really isn't much you can do at the
Apache level.  It becomes a system security issue.  ie. blocking outbound
port 25 connections, for example.

-Rasmus

On Thu, 30 May 2002, Ronald F. Guilmette wrote:

>
> In message <[EMAIL PROTECTED]>,
> Zac Stevens <[EMAIL PROTECTED]> wrote:
>
> >Let me preface this by noting that I agree with your goals, however I
> >believe that you may not have considered the nature of the problem in
> >sufficient depth.
>
> I'll buy that.  I mean it would be arrogant of me... and possibly also
> just plain wrong... not to admit the possibility that I have misjudged
> the true nature of the problem.
>
> >On Thu, May 30, 2002 at 07:45:42PM -0700, Ronald F. Guilmette wrote:
> >> The first step in finding all such scripts however may often be the most
> >> difficult one.  That first step consists of simply gathering into one
> >> big list a list of all of the CGI-enabled directories on the local web
> >> server.  Once such a list has been compiled, other standard UNIX tools
> >> such as `find' and `file' and `grep' and be set to work, plowing through
> >> all of the files in those (CGI) directories and finding all of the bad
> >> FormMail scripts.
> >
> >How are you defining "bad FormMail scripts" here?
>
> Spammer exploitable.
>
> A more detailed elaboration of that term, as I use it, may be found here:
>
> http://www.monkeys.com/anti-spam/formmail-advisory.pdf
>
> >In the simplest case,
> >you could just run 'find' across the filesystem containing the web content
> >looking for "formmail.cgi" or "formmail.pl"...
>
> Hold on there a moment!
>
> The object here is to do the search _efficiently_, i.e. such that it can
> be done even by very large virtual web hosting companies, on all web
> servers, and every night.
>
> Searching the entire filesystem is out of the question.
>
> > and checking those found
> >against a list of known good/bad versions.  This doesn't require any
> >specific knowledge of the Apache configuration in use, and is an entirely
> >viable approach even on filesystems of several hundred gigabytes.
>
> I believe that I disagree.
>
> There are two separate problems with what you proposed.  First is the
> fact that just searching for _filenames_ with the word "formmail" or
> "FormMail" in them is not sufficient to find all of the bad Matt Wright
> FormMail scripts that are installed on a given server.  End-lusers
> often install the scripts using differet names, e.g. "form.pl" or
> "mail.pl" or just "fm.pl".  The second problem is the notion of searching
> several hunderd gigabytes of filesystem.  That just isn't a viable approach,
> especially given that some of the parties I'm dealing with on this issue
> are already balking, even at the notion of merely scanning _just_ their
> CGI-enabled directories.
>
> >A more thorough check would involve testing all executable ascii files,
> >perhaps excluding .html/.php and variants thereof.
>
> Yes, and that is what is needed.
>
> Every plain file that has the execute bit set and that resides in any
> directory for which ExecCGI is enabled must be checked (a) to see if
> it is a Perl script and then (b) to see if it is a Matt Wright Perl
> script.
>
> >> But seriously, is there already a way to do what I need to do with the
> >> Apache server?  Looking at the command line options on the man page for
> >> httpd, there doesn't seem to be an option to request httpd to just parse
> >> the config file, list the CGI directories, and then exit.  But that's
> >> exactly what I need.
> >
> >It isn't possible in the generic case.  Apache allows many aspects of the
> >configuration to be modified by the inclusion of ".htaccess" files beneath
> >the DocumentRoot of the web server.  Unless Apache has been configured not
> >to allow the ExecCGI option to be overridden, you would need to parse both
> >httpd.conf AND every .htaccess file on the filesystem.  Apache itself does
> >not do this at startup - it is done only as required.
>
> Assume the simplifying assumption that enabling ExecCGI via .htaccess files
> has been disallowed within the http.conf file.
>
> _Now_ tell me how to get a list of top-level CGI-enabled directories out
> of httpd... please.
>
> >You also can't assume that only files in designated CGI directories are
> >going to be problematic.
>
> Actually, I believe that I can, under a certain set of conditions.
>
> If you would like to help me flesh out what that exact set of conditions
> is, then pby all means, please do.  I will appreciate it.
>
> Understand that I am _not_ trying to build a solution to this searching
> problem that will cover every possible contingency.  I will be satisfied
> to build a solution that I can offer to web hosting companies and then
> tell them ``T

Re: Odd URL (mis)behavior in 2.0

[Rasmus Lerdorf]

> Right now PATH_INFO is not automatically passed to PHP scripts in the
> Apache 2.0 module for PHP. I have a patch on my plate that enables it
> by default, but for now you can add "AcceptPathInfo On" to the directory
> or location sections where you want it to work.

That should go in.  Why would you not want $PATH_INFO?  I don't think it
should be configurable at all.  It should always be there.

-Rasmus