Building current HEAD on Win32

2003-02-12 Thread Sebastian Bergmann
  ... yields strange error messages for each "Creating Version Resource"
  build step:

16-Bit-MS-DOS-Subsystem:
vcspawn.exe:
The NTVDM-CPU detected an illegal instruction.

  What's this about?

-- 
  Sebastian Bergmann
  http://sebastian-bergmann.de/ http://phpOpenTracker.de/

  Did I help you? Consider a gift: http://wishlist.sebastian-bergmann.de/




Re: mod_authn_mysql

2003-02-12 Thread Paul Querna
> 1. using signal is not good, or portable, or threadsafe ;( Is there a
> way to write a mysql sql connection without using signals at all?
> 
> 2. connection pools would be a good thing IMHO

#1 is already done, and I am working on #2
 
> 3. instead of specifying the userfield/passwordfield/table as directives
> maybe specify a SQL string with token placement
> eg. "select count(*) from usertabels where user=%U and realm=%R and
> password=crypt(%P)"

If anything I will maybe add both.  There are advantages and disadvantages to
both, so the best option is to let the end user decide. However, I want to
have connection pools working first, so this is a secondary priority.

> 4. I'm unsure about what the consequences of including a GPL header is.
> you pointed out in IRC that there is a Public domain version of
> libmysql available (which is what PHP includes) does your module work
> with that?

to clarify libmysqclient is LGPL, not GPL.
This page explains what is required of programs using LGPL code:
http://www.cs.utah.edu/~gk/teem/lgpl.html  (replace "teem" with "libmysqlclient")

My understanding is that dynamicaly linking is not a very big deal.

If linking to LGPL code is too much, I will investigate the public domain PHP
library further, but at this time I have not tested it.

> otherwise i'm +1 on it

thanks

 
> have you recieved any other comments about it?

nope

-chip



[STATUS] (httpd-2.0) Wed Feb 12 23:45:25 EST 2003

2003-02-12 Thread Rodent of Unusual Size
APACHE 2.1 STATUS:  -*-text-*-
Last modified at [$Date: 2003/01/27 17:50:43 $]

Release [NOTE that only Alpha/Beta releases occur in 2.1 development]:

2.1.0   : in development

Please consult the following STATUS files for information
on related projects:

* srclib/apr/STATUS
* srclib/apr-util/STATUS
* docs/STATUS

Contributors looking for a mission:

* just do an egrep on "TODO" or "XXX" and see what's there


CURRENT RELEASE NOTES:


RELEASE SHOWSTOPPERS:


CURRENT VOTES:

* httpd-std.conf and friends

  a) httpd-std.conf should be tailored by install (from src or
 binbuild) even if user has existing httpd.conf
 +1:   trawick, slive, gregames, ianh, Ken, wrowe, jwoolley, jim, nd
   wrowe - prefer httpd.default.conf to avoid ambiguity with cvs

  b) tailored httpd-std.conf should be copied by install to
 sysconfdir/examples
 -0:   striker

  c) tailored httpd-std.conf should be installed to
 sysconfdir/examples or manualdir/exampleconf/
 +1:   slive, trawick, Ken, nd (prefer the latter)

  d) Installing a set of default config files when upgrading a server
 doesn't make ANY sense at all.
 +1:   ianh - medium/big sites don't use 'standard config' anyway, as it
  usually needs major customizations
 -1:   Ken, wrowe, jwoolley, jim, nd
   wrowe - diff is wonderful when comparing old/new default configs,
   even for customized sites that ianh mentions
   jim - ... assuming that the default configs have been updated
 with the required inline docs to explain the
 changes

* If the parent process dies, should the remaining child processes
  "gracefully" self-terminate. Or maybe we should make it a runtime
  option, or have a concept of 2 parent processes (one being a 
  "hot spare").
  See: Message-ID: <[EMAIL PROTECTED]>

  Self-destruct: Ken, Martin
  Not self-destruct: BrianP, Ian, Cliff, BillS
  Make it runtime configurable: Aaron, jim, Justin, wrowe, rederpj, nd

  /* The below was a concept on *how* to handle the problem */
  Have 2 parents: +1: jim
  -1: Justin, wrowe, rederpj, nd
  +0: Martin (while standing by, could it do
  something useful?)

* Make the worker MPM the default MPM for threaded Unix boxes.
  +1:   Justin, Ian, Cliff, BillS, striker, wrowe, nd
  +0:   BrianP, Aaron (mutex contention is looking better with the
latest code, let's continue tuning and testing), rederpj, jim
  -0:   Lars

RELEASE NON-SHOWSTOPPERS BUT WOULD BE REAL NICE TO WRAP THESE UP:

* There is a bug in how we sort some hooks, at least the pre-config
  hook.  The first time we call the hooks, they are in the correct 
  order, but the second time, we don't sort them correctly.  Currently,
  the modules/http/config.m4 file has been renamed to 
  modules/http/config2.m4 to work around this problem, it should moved
  back when this is fixed.

OtherBill offers that this is a SERIOUS problem.  We do not sort
correctly by the ordering arguments passed to the register hook
functions.  This was proven when I reordered the open_logs hook
to attempt to open the error logs prior to the access logs.  Possibly
the entire sorting code needs to be refactored.

* pipes deadlock on all platforms with limited pipe buffers (e.g. both
  Linux and Win32, as opposed to only Win32 on 1.3).  The right solution
  is either GStein's proposal for a "CGI Brigade", or OtherBill's proposal
  for "Poll Buckets" for "Polling Filter Chains".  Or maybe both :-)

* All handlers should always send content down even if r->header_only
  is set.  If not, it means that the HEAD requests don't generate the
  same headers as a GET which is wrong.

* HP/UX 10.20: compile breakage in APR.  Looks like it should be easy
  to fix, probably just some extraneous #include's that are fouling
  things up.
  PR: 9457
  Jeff: See my reply and patch in the PR (and previous commit to
  stop using "pipe" as a field name).  If patch is committed, we
  should be okay.  I'll wait to see if the user tests the patch.
  Update by Jeff 20020722: I got an account on HP 10.20.  It looks
  like some of the APR thread detection is screwed up.  If we find
  pthread.h but we can't compile the pthread test program we still
  think we can use threads.  For that reason, the patch I posted
  to the PR won't work as-is since a failed compile of the test
  program means nothing.

* exec cmd and suexec arg-passing enhancements
  Status: Patches proposed
  Message-ID: <[EMAIL PROTECTED]>
  (see the "proc.patch" and "suexec-shell.patch" links in this mess

[STATUS] (apache-1.3) Wed Feb 12 23:45:13 EST 2003

2003-02-12 Thread Rodent of Unusual Size
APACHE 1.3 STATUS:  -*-text-*-
  Last modified at [$Date: 2003/02/04 19:08:59 $]

Release:

   1.3.28-dev: In development
   1.3.27: Tagged September 30, 2002. Announced Oct 3, 2002.
   1.3.26: Tagged June 18, 2002.
   1.3.25: Tagged June 17, 2002. Not released.
   1.3.24: Tagged Mar 21, 2002. Announced Mar 22, 2002.
   1.3.23: Tagged Jan 21, 2002.
   1.3.22: Tagged Oct 8, 2001.  Announced Oct 12, 2001.
   1.3.21: Not released.
 (Pulled for htdocs/manual config mismatch. t/r Oct 5, 2001)
   1.3.20: Tagged and rolled May 15, 2001. Announced May 21, 2001.
   1.3.19: Tagged and rolled Feb 26, 2001. Announced Mar 01, 2001.
   1.3.18: Tagged and rolled Not released.
 (Pulled because of an incorrect unescaping fix. t/r Feb 19, 2001)
   1.3.17: Tagged and rolled Jan 26, 2001. Announced Jan 29, 2001.
   1.3.16: Not released.
 (Pulled because of vhosting bug. t/r Jan 20, 2001)
   1.3.15: Not released.
 (Pulled due to CVS dumping core during the tagging when it
  reached src/os/win32/)
   1.3.14: Tagged and Rolled Oct 10, 2000.  Released/announced on the 13th.
   1.3.13: Not released.
 (Pulled in the "first minutes" due to a Netware build bug)
   1.3.12: Tagged and rolled Feb. 23, 2000. Released/announced on the 25th.
   1.3.11: Tagged and rolled Jan. 19, 2000. Released/announced on the 21st.
   1.3.10: Not released.
 (Pulled at "last minute" due to a build bug in the MPE port)
1.3.9: Tagged and rolled on Aug. 16, 1999. Released and announced on 19th.
1.3.8: Not released.
1.3.7: Not released.
1.3.6: Tagged and rolled on Mar. 22, 1999. Released and announced on 24th.
1.3.5: Not released.
1.3.4: Tagged and rolled on Jan. 9, 1999.  Released on 11th, announced on 12th.
1.3.3: Tagged and rolled on Oct. 7, 1998.  Released on 9th, announced on 10th.
1.3.2: Tagged and rolled on Sep. 21, 1998. Announced and released on 23rd.
1.3.1: Tagged and rolled on July 19, 1998. Announced and released.
1.3.0: Tagged and rolled on June 1, 1998.  Announced and released on the 6th.
   
2.0  : Available for general use, see httpd-2.0 repository

RELEASE SHOWSTOPPERS:

RELEASE NON-SHOWSTOPPERS BUT WOULD BE REAL NICE TO WRAP THESE UP:

   * Current vote on 2 PRs for inclusion:
  Bugz #9181 (Unable to set headers on non-2XX responses)
+1: Martin, Jim
  Gnats #10246 (Add ProxyConnAllow directive)
+0: Martin (or rather -.5, see dev@ Message
<[EMAIL PROTECTED]>)

* htpasswd.c and htdigest.c use tmpnam()... consider using
  mkstemp() when available.
Message-ID: <[EMAIL PROTECTED]>
Status:

* Dean's "unescaping hell" (unescaping the various URI components
  at the right time and place, esp. unescaping the host name).
Message-ID: <[EMAIL PROTECTED]>
Status:

* Martin observed a core dump because a ipaddr_chain struct contains
  a NULL-"server" pointer when being dereferenced by invoking "httpd -S".
Message-ID: <[EMAIL PROTECTED]>
Status: Workaround enabled. Clean solution can come after 1.3.19

* long pathnames with many components and no AllowOverride None
  Workaround is to define  with AllowOverride None,
  which is something all sites should do in any case.
Status: Marc was looking at it.  (Will asks 'wasn't this patched?')

* Ronald Tschalär's patch to mod_proxy to allow other modules to
  set headers too (needed by mod_auth_digest)
Message-ID: <[EMAIL PROTECTED]>
Status:


Available Patches (Most likely, will be ported to 2.0 as appropriate):

   *  A rewrite of ap_unparse_uri_components() by Jeffrey W. Baker
 <[EMAIL PROTECTED]> to more fully close some segfault potential.
Message-ID: 
Status:  Jim +1 (for 1.3.19), Martin +0

* Andrew Ford's patch (1999/12/05) to add absolute times to mod_expires
Message-ID: <[EMAIL PROTECTED]>
Status: Martin +1, Jim +1, Ken +1 (on concept)

* Raymond S Brand's path to mod_autoindex to fix the header/readme
  include processing so the envariables are correct for the included
  documents.  (Actually, there are two variants in the patch message,
  for two different ways of doing it.)
Message-ID: <[EMAIL PROTECTED]>
Status: Martin +1(concept)

* Jayaram's patch (10/27/99) for bugfix to mod_autoindex
  IndexIgnore  should hide the files with this file-
  extension in directory listings. This was NOT happening because the 
  total filename was being compared with the file-extension.
  Status: Martin +1(untested), Ken +1(untested)
   
* Salvador Ortiz Garcia <[EMAIL PROTECTED]>' patch to allow DirectoryIndex
  to refer to URIs for non-static resources.
MID: <[EMAIL PROTECTED]>
Status: Ken +1 (on concept), Lars +1 (on concept)

* Brian Havard's patch to

SetEnvIf foo reg(ex) var=$1

2003-02-12 Thread André Malo
The attached patch makes subexpression capturing possible. I've missed this 
for a long time ;-)
Any objections to commit it?

Btw: ap_pregsub treats '&' as $0 (whole match). Is it documented somewhere? 
Is it used or at least useful somewhere? It looks for me like a legacy, 
that should be removed (in 2.1), since it may lead to unexpected behaviour. 
(ISTR that it actually did in early 1.3 versions in mod_rewrite). Opinions?

nd
-- 
Wenn nur Ingenieure mit Diplom programmieren würden, hätten wir
wahrscheinlich weniger schlechte Software.
Wir hätten allerdings auch weniger gute Software.
   -- Felix von Leitner in dasr

Index: modules/metadata/mod_setenvif.c
===
RCS file: /home/cvs/httpd-2.0/modules/metadata/mod_setenvif.c,v
retrieving revision 1.41
diff -u -r1.41 mod_setenvif.c
--- modules/metadata/mod_setenvif.c 12 Feb 2003 17:58:09 -  1.41
+++ modules/metadata/mod_setenvif.c 12 Feb 2003 23:42:16 -
@@ -351,8 +351,7 @@
 }
 else {
 new->preg = ap_pregcomp(cmd->pool, regex,
-(REG_EXTENDED | REG_NOSUB
- | (icase ? REG_ICASE : 0)));
+(REG_EXTENDED | (icase ? REG_ICASE : 0)));
 if (new->preg == NULL) {
 return apr_pstrcat(cmd->pool, cmd->cmd->name,
" regex could not be compiled.", NULL);
@@ -490,6 +489,7 @@
 apr_size_t val_len = 0;
 int i, j;
 char *last_name;
+regmatch_t regm[10];
 
 if (!ap_get_module_config(r->request_config, &setenvif_module)) {
 ap_set_module_config(r->request_config, &setenvif_module,
@@ -577,7 +577,8 @@
 }
 
 if ((b->pattern && apr_strmatch(b->pattern, val, val_len)) ||
-(!b->pattern && !ap_regexec(b->preg, val, 0, NULL, 0))) {
+(!b->pattern && !ap_regexec(b->preg, val, b->preg->re_nsub + 1,
+regm, 0))) {
 const apr_array_header_t *arr = apr_table_elts(b->features);
 elts = (const apr_table_entry_t *) arr->elts;
 
@@ -586,7 +587,18 @@
 apr_table_unset(r->subprocess_env, elts[j].key);
 }
 else {
-apr_table_setn(r->subprocess_env, elts[j].key, elts[j].val);
+if (!b->pattern) {
+char *replaced = ap_pregsub(r->pool, elts[j].val, val,
+b->preg->re_nsub + 1, regm);
+if (replaced) {
+apr_table_setn(r->subprocess_env, elts[j].key,
+   replaced);
+}
+}
+else {
+apr_table_setn(r->subprocess_env, elts[j].key,
+   elts[j].val);
+}
 }
 }
 }



Re: Strange Behavior of Apache 2.0.43 on SPARC MP system

2003-02-12 Thread Aaron Bannert

On Wednesday, February 12, 2003, at 01:08  PM, Min Xu wrote:

We will soon have two new 8P Sun servers equipped with
Gigabit ethernet coming to our lab. With that, I should be able to
experiment with separate machines.


I'd be very interested in seeing updated results from a multi-machine
networked test. Feel free to post them here once you have them.

-aaron




Re: [PATCH] Sendfile API compatibility breakage

2003-02-12 Thread William A. Rowe, Jr.
At 02:51 PM 2/12/2003, Greg Ames wrote:

>I think there should be an MMN bump in the 2.1 stream for this, so independent 
>modules can use the new API if they chose to.  In 2.0 stable, IMO we should do our 
>best to make it transparent.

It's a new feature.  +1 on an MMN *minor* bump to 2.0.45-dev and the 2.1 tree.

The new feature doesn't *break* anybody by this schema, correct?  If we (the
APR team) shut down sendfile on some obtuse or buggy platform, that's our
(wise) choice to help users avoid hitting such bugs.

Bill





Re: Strange Behavior of Apache 2.0.43 on SPARC MP system

2003-02-12 Thread Min Xu
On Wed, Feb 12, 2003 at 10:35:18AM -0800, Justin Erenkrantz wrote:
> --On Wednesday, February 12, 2003 11:52 AM -0600 Min Xu 
> <[EMAIL PROTECTED]> wrote:
> 
> >First, I don't think the disk should be bottleneck in any case,
> >this is because the system has 2GB memory, Solaris's file cache is
> >able to cache all the file content. top shows the following stats:
> 
> The size of memory has nothing to do with the available bandwidth 
> that the memory has.

I didn't say it does. I was trying to rule out the possibilities that
the disk being the bottleneck. But you apparently have underestimated
Sun server's memory system. For our system, it uses Sun's gigaplane/Sunfire
system interconnect.

> ...
> 
> All MP Sparcs share the same memory backplane.  That's why you hardly 
> ever see performance improvements past 8x CPUs because the memory 
> bandwidth kills you (the CPUs are starved for memory).  Moving to a 
> NUMA architecture might help, but I think that's not a feature 
> UltraSparc or Solaris support.  (I hear Linux has experimental NUMA 
> support now.)

It is indeed the case apache's performance doesn't go up very much
past 8x CPUs in my experiments. However, whether this is due to limited
memory bandwidth is yet to be tested. Also, I am not aware of any
literature supports that NUMA architecture would have higher memory
bandwidth.

> I'd recommend reading http://www.sunperf.com/perfmontools.html.  You 
> should also experiment with mod_mem_cache and mod_disk_cache.

Thanks for the suggestions, I would like to try mod_mem_cache and
mod_disk_cache.

> >To test the context switching hypothesis and the backplane
> >hypothesis I changed all files in the repository to 2 bytes long,
> >that's an "a" plus an "eof". I rerun the experiment, the
> >performance is poorer!
> 
> There will still be overhead in the OS networking layer.  You are 
> using connection keep-alives and pipelining, right?  The fact that 
> your top output had a lot of kernel time, I'd bet you are spending a 
> lot of time contending on the virtual network (which is usually the 
> case when you are not using connection keep-alives - the TCP stack 
> just gets hammered).  I'd bet the local network is not optimized for 
> performance.  (DMA can't be used and functionality that could be 
> implemented on dedicated hardware must be done on the main CPU.)

Sounds interesting. I'd like to test whether the networking layer
is problem or not somehow.

> Please stop trying to convince us to pay attention to benchmarks 
> where the client and server are on the same machine.  There are just 
> too many variables that will screw things up.  The performance 
> characteristics change dramatically when they are physically separate 
> boxes.  -- justin

I agree. We will soon have two new 8P Sun servers equipped with
Gigabit ethernet coming to our lab. With that, I should be able to
experiment with separate machines.

Thanks for your insightful comments.

-Min

-- 
Rapid keystrokes and painless deletions often leave a writer satisfied with
work that is merely competent.
  -- "Writing Well" Donald Hall and Sven Birkerts



Re: [PATCH] Sendfile API compatibility breakage

2003-02-12 Thread Greg Ames
William A. Rowe, Jr. wrote:


I suggest the following as a friendly compromise to avoid voting... 

this sounds like progress.


introduce DISABLE, don't drop ENABLE (especially since your patch would *reverse* 
the meaning of the EnableSendfile directive for httpd 2.0.44 built against 
a newer apr!!!)  

D'oh!  You're right, good catch.


Respect both flags unless both are passed to apr_file_open.

If neither is specified, let the platform author of the apr_sendfile() code decide
how *safe* it is to enable the feature.  This could include context (such as
the filesystem if they want to query it) or filesize (<2gb) or whatever choices
are wise.

ENABLE and DISABLE would both be explicit overrides to any sanity tests.
Based on current bugzilla reports, httpd EnableSendfile should gain the
'default' state.  So we could turn it on or off, or leave it up to APR.


I think this could be made to work.  I assume the platform author makes the call 
on what to do with sendfile in the default case at apr_file_open time. 
Otherwise there's a lot of code that would have to be churned between httpd and 
apr without much benefit.

I think there should be an MMN bump in the 2.1 stream for this, so independent 
modules can use the new API if they chose to.  In 2.0 stable, IMO we should do 
our best to make it transparent.

Greg



Re: Apache 2.0.45 -- When?

2003-02-12 Thread William A. Rowe, Jr.
At 10:01 AM 2/12/2003, Jess M. Holle wrote:
>There were mutterings that Apache 2.0.45 would be following shortly after 2.0.44. 
>It's now been a bit, and I'm left wondering: what is the release timeframe for 2.0.45?

My goal is to tag ASAP - but I'm just back from CA and had too many
deliverables lately...

Here are my goals (your mileage may vary);

  .Resolve Greg's concerns about ENABLE_SENDFILE, a 2.0.44
   regression.

  .Fix the 2.0.44 regression in OtherChild platform compatibility

  .Call APR 0.9.2 ''done" so they can move on to 9.3 (and soon 1.0.0)

  .Fix my choice of .dbgmark for .dbg file creation timestamps (it turns
   out that *.dbg includes 8.3 names of .dbgmark files, yuck!)

I'm tracking a few higher priority issues at the moment so I will probably
tag and roll a 2.0.45_RC1 tarball candidate late Friday afternoon.

Bill 




Re: mod_authn_mysql

2003-02-12 Thread Ian Holsman
Hi Paul.
some quick comments about your code.

1. using signal is not good, or portable, or threadsafe ;( Is there a 
way to write a mysql sql connection without using signals at all?

2. connection pools would be a good thing IMHO

3. instead of specifying the userfield/passwordfield/table as directives 
maybe specify a SQL string with token placement
eg. "select count(*) from usertabels where user=%U and realm=%R and 
password=crypt(%P)"

4. I'm unsure about what the consequences of including a GPL header is.
   you pointed out in IRC that there is a Public domain version of 
libmysql available (which is what PHP includes) does your module work 
with that?

otherwise i'm +1 on it

have you recieved any other comments about it?

Paul Querna wrote:
It is a simple hack based off of mod_authn_dbm, and mod_digest_mysql using the
new Auth*Providers in Apache 2.1/2.2

It hasn't had much testing except for internaly so far, So I wouldn't be
surprised if there are some bugs.

You can download it from:
http://open.cyanworlds.com/

With the new Auth*Providers, I would like to see mod_authn_mysql added to the
Apache HTTPD source tree.  Many people use MySQL databases for all kinds of
things, and I think including a module that can easily integrate with a common
enviroment is a good idea.

-chip








Re: Strange Behavior of Apache 2.0.43 on SPARC MP system

2003-02-12 Thread Ian Holsman
Hi Min.

I'm not sure if this would make a difference in 2.0.43,
but you might want to configure apache to use non-portable-atomics
  --enable-nonportable-atomics=yes

I think the worker MPM can then take advantage of this, and remove a 
mutex lock (replacing it with a spin) which may give you better 
performance with multiple CPUs.

1 other point.
you might want to disable some CPU's, and re-run your test. (eg.. on a 8 
cpu machine) and see if that helps



Min Xu wrote:
Hi All,

Sorry I am posting this directly to the development list. But
I think this is not a user setup problem and it is so strange
maybe only you guys will have a clue on what's going on.

I am a student of UW-Madison. In order to study computer
architecture of commercial multiprocessor servers, we have used
APACHE as one of our important workloads.

I am the one who setup the workload on a 14-processor Sun
Enterprise server. During setup I found a very strange behavior
of the apache server (running with worker MPM). Essentially the
strange thing is that:

  The server optimal throughput is not achieved by using a
  greedy client, who drive the server with no think time. But
  with tiny amount of think time, much better throughput is
  archievable. Also, with the greedy client, the server's
  performance decreased over time, which seems to be very
  counter-intuitive.

Of course, just give you the short decription above does not
help you to help me. So I will give you the detail problem
description and data in the following. With the understanding
of the source code, maybe you can give me some more hypothesises
to try on.






Re: Strange Behavior of Apache 2.0.43 on SPARC MP system

2003-02-12 Thread Justin Erenkrantz
--On Wednesday, February 12, 2003 11:52 AM -0600 Min Xu 
<[EMAIL PROTECTED]> wrote:

First, I don't think the disk should be bottleneck in any case,
this is because the system has 2GB memory, Solaris's file cache is
able to cache all the file content. top shows the following stats:


The size of memory has nothing to do with the available bandwidth 
that the memory has.  I believe recent Sparcs still only use 133MHz 
RAM (PC133 at best - Sparc's don't yet use DDR, I think).  Since all 
code pages will most likely not fit entirely in CPU cache, some 
section has to be read from main memory.  IIRC, some versions of 
UIIIi's have 4MB CPU cache, but I wouldn't be surprised if that's not 
enough (kernel pages would also have to be counted).  (I don't know 
your specifics here.)

So, if you have 14 processors (I think this is what you said you 
had), they will all be contending on the memory (~133MHz) bus.  The 
effective memory bus for all of the processors will be 133/14 = 
~14MHz.  That's a severe bottleneck if main memory is accessed in a 
critical path for all processes.

All MP Sparcs share the same memory backplane.  That's why you hardly 
ever see performance improvements past 8x CPUs because the memory 
bandwidth kills you (the CPUs are starved for memory).  Moving to a 
NUMA architecture might help, but I think that's not a feature 
UltraSparc or Solaris support.  (I hear Linux has experimental NUMA 
support now.)

I'd recommend reading http://www.sunperf.com/perfmontools.html.  You 
should also experiment with mod_mem_cache and mod_disk_cache.

To test the context switching hypothesis and the backplane
hypothesis I changed all files in the repository to 2 bytes long,
that's an "a" plus an "eof". I rerun the experiment, the
performance is poorer!


There will still be overhead in the OS networking layer.  You are 
using connection keep-alives and pipelining, right?  The fact that 
your top output had a lot of kernel time, I'd bet you are spending a 
lot of time contending on the virtual network (which is usually the 
case when you are not using connection keep-alives - the TCP stack 
just gets hammered).  I'd bet the local network is not optimized for 
performance.  (DMA can't be used and functionality that could be 
implemented on dedicated hardware must be done on the main CPU.)

Please stop trying to convince us to pay attention to benchmarks 
where the client and server are on the same machine.  There are just 
too many variables that will screw things up.  The performance 
characteristics change dramatically when they are physically separate 
boxes.  -- justin


RE: Problem with SSL in 64 bit build on Solaris

2003-02-12 Thread MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1)
Title: Problem with SSL in 64 bit build on Solaris



Question : Did you run into any problems when 
you used Apache (not 2.0.44) + OpenSSL in 64-bit mode 
earlier ?
 
I'm asking this because I remember having run into a 
64-bit porting issue with OpenSSL (long time back, on HP-UX) - like a file not 
including  or something like that - but I'm unable to track it 
down.
 
-Madhu

  -Original Message-From: CASTELLE Thomas 
  [mailto:[EMAIL PROTECTED]]Sent: Wednesday, February 12, 2003 
  1:15 AMTo: [EMAIL PROTECTED]Subject: Problem with SSL 
  in 64 bit build on Solaris
  Hi there ! 
  I'm sorry to come again with this, but it is quite important 
  to us : is someone looking at the bug 16333 I opened a few weeks ago ? I don't 
  need a solution right now, but I would be relieved if I know that someone's 
  looking at it...
  To summarize the problem, mod_ssl doesn't seem to work when 
  Apache 2.0.44 is build in 64 bit on Solaris 8. 
  Thanks for your answer ! 
  Thomas. 


Re: Solaris 8 + gcc 3.2.1 + libtool = fuckup...

2003-02-12 Thread Hacker at Large
Pier, and others:

> Cannot load /opt/apache/modules/mod_status.so into
> server: ld.so.1:
> /opt/apache/bin/httpd: fatal: relocation error: file
> /opt/apache/modules/mod_status.so: symbol
__floatdisf:
> referenced symbol not found

I discovered the above mentioned discussion while I
was plagued with the same problem on Solaris 7. The
bits of information in your discussion led me to what
I believe is a reasonable solution. If this has been
otherwise solved, I apollogise for wasting your time.

I grabbed libgcc from sunfreeware and compiled Apache
2.0.44 as follows:
LDFLAGS="-L/usr/local/lib -lgcc_s" ./configure \
   --enable-mods-shared=most \ 
   --prefix=/usr/local/apache_2.0.44
make

It's working fine so far. It's not ideal due to the
external library dependancy created, but it's better
than a kick in the head.

Cheers!
Stephen

__
Do you Yahoo!?
Yahoo! Shopping - Send Flowers for Valentine's Day
http://shopping.yahoo.com



Re: Strange Behavior of Apache 2.0.43 on SPARC MP system

2003-02-12 Thread Min Xu
On Wed, Feb 12, 2003 at 11:52:03AM -0600, Min Xu wrote:
> with normal file size:
>   0us delay: system xput ~= 2500 pages/sec
>   3us delay: system xput ~= 4700 pages/sec
> 
> with 2byte file size:
>   0us delay: system xput ~= 1900 pages/sec
>   3us delay: system xput ~= 1700 pages/sec
>  10us delay: system xput ~= 1700 pages/sec

also:

  50us delay: system xput ~= 1700 pages/sec

  I ran 100 client threads.

-Min



Re: Strange Behavior of Apache 2.0.43 on SPARC MP system

2003-02-12 Thread Min Xu
On Wed, Feb 12, 2003 at 12:30:28AM -0500, Cliff Woolley wrote:
> Of course the client isn't spinning, but your statement that "the server
> should have done its job" assumes that the entire response is sent at
> once.  Depending on its length, it is entirely possible that the server
> would send the response in multiple packets.  But as soon as *any* packets
> arrive, the client is woken up.  That means that the server and the client
> are vying for timeslices.  Let's say the client gets it.  The client
> receives the partial response and then goes right back to sleep again.
> Sure the server wakes right back up and starts sending more of the
> response, but in the meanwhile, several context switches had to happen.
> 

I am sorry for my bad assumption. The reason I thought that was "mostly"
the case that server has done its job is the following:

I have a file repository contains about 2 files in one directory.
I know all in one directory is bad, but I didn't have time to fix
the client about this. And 15000 out of this 2 files are smaller
than 8K, which is the page size of the system. I thought in terms of DMA,
the file smaller than 8K should be transfered in one shot.

Anyway, I think your point is valid, and Justin's hypothesis about the
backplane/disk being bottleneck is also very interesting. So I did some
more experiments this morning.

First, I don't think the disk should be bottleneck in any case, this is
because the system has 2GB memory, Solaris's file cache is able to cache
all the file content. top shows the following stats:

load averages:  4.05,  4.68,  5.25 10:33:36
47 processes:  45 sleeping, 2 on cpu
CPU states: 51.2% idle, 11.3% user, 27.8% kernel,  9.8% iowait,  0.0% swap
Memory: 2048M real, 1327M free, 508M swap in use, 3256M swap free

The total file size is < 500MB, so there is no reason solaris can't cache
them all.

To test the context switching hypothesis and the backplane hypothesis
I changed all files in the repository to 2 bytes long, that's an "a"
plus an "eof". I rerun the experiment, the performance is poorer!

with normal file size:
  0us delay: system xput ~= 2500 pages/sec
  3us delay: system xput ~= 4700 pages/sec

with 2byte file size:
  0us delay: system xput ~= 1900 pages/sec
  3us delay: system xput ~= 1700 pages/sec
 10us delay: system xput ~= 1700 pages/sec

It seems that the bottleneck is some where else, and the bottleneck is
stressed harder when file size is smaller. But I doubt it is the backplane
or the context switching.

-Min

-- 
Rapid keystrokes and painless deletions often leave a writer satisfied with
work that is merely competent.
  -- "Writing Well" Donald Hall and Sven Birkerts



Apache 2.0.45 -- When?

2003-02-12 Thread Jess M. Holle
There were mutterings that Apache 2.0.45 would be following shortly 
after 2.0.44. It's now been a bit, and I'm left wondering: what is the 
release timeframe for 2.0.45?

--
Jess Holle




Re: Patches and Enhancements for a SSL-Proxy Based on Apache 2.0 (mod_ssl, mod_proxy, mod_headers)

2003-02-12 Thread Erik Abele
Hi Maik,
 
> And here follows the patch (My proposed changes to the HTML docu are now not
> included in the patch. Please advice me if and how to post this changes to
> mod_headers.html.en)

Just post it here or to the docs-list ([EMAIL PROTECTED]) as a unified
diff patch (diff -u). But please be aware, that we are generating the whole
documentation from XML source. Therefore you should patch these instead of
the HTML files.

More information is available at
http://httpd.apache.org/docs-project/docsformat.html

cheers,
Erik




RE: Patches and Enhancements for a SSL-Proxy Based on Apache 2.0 (mod_ssl, mod_proxy, mod_headers)

2003-02-12 Thread Maik Mueller
> Cool..

> Can you please post the patch to the list, so that ppl can review the
> code,
> and give their comments.

> -Madhu
No problem!
Here is my short README describing the patch and its history form Apache
version 2.0.43 to 2.0.44:

Hello!
This is the distribution point for the Apache 2.0 as SSL Intermediary Patch.
Currently you need this patch to use Apache 2.0 as a trusted intermediary
in configuration with the SAP J2EE Engine.
The patch is subject to become part of the standard Apache 2.0 distribution.

Feedback welcome!
Maik ([EMAIL PROTECTED])

INSTRUCTIONS:
- extract the Apache 2.0.43 distribution (httpd-2.0.43.tar.gz)
- change directory to httpd-2.0.43
- apply the patch with -p1 (patch -p1 < Apache-2.0.43-SSLintermediary.patch)
- follow the Apache INSTALL instructions

HISTORY:
02-12-30 initial release (available SAP internal)
03-01-07 httpd-2.0.43-patched-as-SSLintermediary.zip added
 In this ZIP archive the Apache-2.0.43-SSLintermediary.patch is
 already applied. More convenient for users not so familiar with the
 usage of diff & patch.
03-01-08 httpd-2.0.43-win32-src-patched-as-SSLintermediary.zip added
 You cannot use the UNIX source to build the WIN32 binaries.
 This ZIP archive contains the already patched version of
 httpd-2.0.43-win32-src. Use it to build the WIN32 binaries.
 If you want to apply Apache-2.0.43-SSLintermediary.patch to the
 original httpd-2.0.43-win32-src be aware that you have to convert
 CR-LFs in CR before applying the patch. In the successfully patched
 files you can again expand CR to CR-LF.
03-01-20 Bug in base 64 padding found. The calculation of the number of
padding
 characters ('=') needed computes wrong results in some cases.
03-02-07 Apache 2.0.44 Released
 Apache-2.0.44-SSLintermediary.patch corresponds to
httpd-2.0.44.tar.gz
 The documentation changes are NO longer part of the patch.
 Download mod_headers_mai.html.en for proposed documentation
changes.
 SSLproxy.conf is a good example for a proxy's mod_ssl
configuration.
 The SAP proposed header names are use in the example added to the
 mod_headers documentation (see mod_headers_mai.html.en).

And here follows the patch (My proposed changes to the HTML docu are now not
included in the patch. Please advice me if and how to post this changes to
mod_headers.html.en):
--- httpd-2.0.44.ori/modules/metadata/mod_headers.c Mon Nov  4 19:31:57 2002
+++ httpd-2.0.44/modules/metadata/mod_headers.c Fri Feb  7 18:00:18 2003
@@ -109,6 +109,7 @@
 #include "apr_lib.h"
 #include "apr_strings.h"
 #include "apr_buckets.h"
+#include "apr_base64.h"

 #include "apr_hash.h"
 #define APR_WANT_STRFUNC
@@ -198,6 +199,62 @@
 else
 return "(null)";
 }
+
+/* Base 64 encoded ASN.1 data is usually tagged with decorations of
+ * the following style:
+ *   -BEGIN -
+ *   
+ *   -END -
+ * The defines are used to search for such decorations.
+ */
+#define DECORATION_MARKER_BEGIN "-BEGIN"
+#define DECORATION_MARKER_END   "-END"
+#define DECORATION_EOF_MARKER   "-"
+
+static const char *header_request_env_varB64(request_rec *r, char *a)
+{
+  const char *s = apr_table_get(r->subprocess_env,a);
+  char *pStartBody = NULL;
+  char *pBehindBody = NULL;
+  char *ptr;
+
+  if (s) {
+/* search for decorations marking encapsulated base64 encoded data */
+ptr = strstr((char *)s, DECORATION_MARKER_BEGIN);
+if (ptr) {
+  ptr = strstr(ptr + strlen(DECORATION_MARKER_BEGIN),
DECORATION_EOF_MARKER);
+  if (ptr && (ptr + strlen(DECORATION_EOF_MARKER) + 1) != '\0') {
+   /* explicit check that there are sitll chars in the string */
+   pStartBody = ptr + strlen(DECORATION_EOF_MARKER) + 1;
+
+   ptr = strstr(pStartBody, DECORATION_MARKER_END);
+   if (ptr && strstr(ptr, DECORATION_EOF_MARKER))
+ pBehindBody = ptr;
+  }
+}
+
+if (pStartBody && pBehindBody) {
+  /* encapsulated base64 encoded data found */
+  /* all except the body will be skipped */
+  *pBehindBody = '\0';
+  apr_base64_cleanB64(pStartBody);
+  return pStartBody;
+} else {
+  /* call apr_base64_encode() to encode the data */
+  int inlen = strlen(s);
+  int outsize = apr_base64_encode_len(inlen);
+  char *encoded = apr_palloc(r->pool, outsize);
+  int rc = apr_base64_encode(encoded, s, inlen);
+  if (rc > outsize)
+   return "(null)";
+  else
+   return encoded;
+}
+  }
+  else
+return "(null)";
+}
+
 /*
  * Config routines
  */
@@ -407,7 +464,7 @@

 /* Handle the envclause on Header */
 if (envclause != NULL) {
-if (inout != hdr_out) {
+if (inout != hdr_out && inout != hdr_in) {
 return "error: envclause (env=...) only valid on Header
directive";
 }
 if (strncasecmp(envclause, "env=", 4) != 0) {
@@ -448,12 +505,23 @@
 return head

Re: [PATCH] Allow modules to load if they were compiled with AP_DEBUG(--enable-maintainer-mode) although the core was not.

2003-02-12 Thread Jeff Trawick
Sander Striker wrote:


This patch might be incomplete in that I'm not completely sure how to
handle the exports.  Basically ap_[gs]et_module_config should always
be in exports.c.

Secondly, while looking at util_debug.c, I see that we have more
functions we conditionally define.  We may want to define those all
the time aswell.


definitely; I still have teeth marks from this one

+1 (concept)




RE: PR 13211

2003-02-12 Thread Frank Faubert
Hello,

Does anyone have any additional ideas on how this can be fixed?

-Frank

-Original Message-
From: André Malo [mailto:[EMAIL PROTECTED]] 
Sent: Wednesday, January 29, 2003 9:25 PM
To: [EMAIL PROTECTED]
Subject: Re: PR 13211


* Frank Faubert wrote:

> Add "CookieTracking on" to the end of the default httpd.conf file, start
up
> Apache and look at the headers for "/" -- only one cookie.  Now copy
> /tmp/apache/htdocs/index.html.en to /tmp/apache/htdocs/index.html and look
> at the headers for "/".  I get two cookies for every page that does NOT
use
> content negotiation (which unfortunately for me is my entire site...).

*hrm* I can verify that behaviour for directory requests (i.e. 
mod_dir/DirectoryIndex) without negotiation of the index file.

It seems to happen the following (GET / HTTP/1.0) without negotiation:

run_fixup:
->mod_usertrack: spot_cookie
->mod_dir: search and find index.html using sub_req_lookup_uri, which runs 
a fixup itself (->next spot_cookie)
  ->internal_fast_redirect -> apr_table_overlay(r->headers_out, 
rr->headers_out)

   ^^ two cookies here ^^

mod_negotiation instead (in case of index.html.var) does an additional 
normal internal_redirect to the negotiated resource, which drops the old 
stuff and cooks (not only) its own cookie.

right?

Conclusion: are overlay'ed tables in internal_fast_redirect semantically 
intended? Could someone please explain in slow words ;-), why?

> I also ran the same tests with v1.3.27 built as follows:
> And -never- got two cookies.

mod_dir uses a normal internal_redirect in 1.3.

nd
-- 
"Die Untergeschosse der Sempergalerie bleiben währenddessen aus
 statistischen Gründen geflutet." -- Spiegel Online




Re: Patches and Enhancements for a SSL-Proxy Based on Apache 2.0 (mod_ssl, mod_proxy, mod_headers)

2003-02-12 Thread Graham Leggett
Maik Mueller wrote:


1. SSL_CLIENT_CERT produces multi-line output and the RequestHeader
directive isn't able to transfer it into a correct multi-line HTTP header.


As I understand it, headers may span multiple lines (correct me if I am 
wrong). Therefore if RequestHeader isn't able to handle multi line 
headers, then this is a bug which should be fixed.

Regards,
Graham
--
-
[EMAIL PROTECTED]		"There's a moon
	over Bourbon Street
		tonight..."



Patches and Enhancements for a SSL-Proxy Based on Apache 2.0 (mod_ssl, mod_proxy, mod_headers)

2003-02-12 Thread Maik Mueller
Hello All,

I want to provide updated information to my earlier described scenario using
mod_ssl + mod_proxy + mod_headers:
Component:  Web Browser --- Proxy (mod_proxy)   --- Web Server
SSL Role:   SSL Client  --- SSL server | SSL Client --- SSL Server

The following discussion focuses on Apache 2.0.43 and 2.0.44.

I have implemented a solution to transfer the Web browser's client
certificate (and other SSL information) to the backend Web server:
Component:  Web Browser --- Proxy (mod_proxy)   --- Web Server
SSL Role:   SSL Client  --- SSL server | SSL Client --- SSL Server
Client Cert (and other SSL information) --> Transfer 
as HTTP Headers

The problem was that mod_headers' RequestHeader directive didn't really
matched the requirements.
RequestHeader set SSL_CLIENT_CERT %{SSL_CLIENT_CERT}e is not a practical
solution to
forward the client's certificate to the backend server for the
following reasons:
1. SSL_CLIENT_CERT produces multi-line output and the RequestHeader
directive isn't able to transfer it into a correct multi-line HTTP header.
2. The "decorations" (-BEGIN/END CERTIFICATE-) and the multi-line
format are not very useful in this scenario.

Therefore I have introduced the option "E" in addition to "e" for putting
environment variables in headers. The "E" has the following meaning:

%{FOOBAR}E  The base64 encoded content of the environment variable FOOBAR.
If the environment variable already contains a base64 encoded body (e. g.
SSL_CLIENT_CERT) the body will be set as the value of the header variable.

The result is in any case a single line of base64 characters only.

This behavior serves two requirements:
1. There is no problem escaping special characters when putting other SSL
information in HTTP headers. In many cases, SSL_CLIENT_S_DN will probably
contain characters that have to be escaped.
2. Reduces the overhead produced by "decorations" and multi-line format.

Here is an example for forwarding the SSL Client Certificate and other SSL
information:
RequestHeader set SSL_CLIENT_CERT %{SSL_CLIENT_CERT}E
env=SSL_CLIENT_S_DN
RequestHeader set SSL_CLIENT_CERT_CHAIN_0 %{SSL_CLIENT_CERT_CHAIN_0}E
env=SSL_CLIENT_CERT_CHAIN_0
RequestHeader set SSL_CLIENT_CERT_CHAIN_1 %{SSL_CLIENT_CERT_CHAIN_1}E
env=SSL_CLIENT_CERT_CHAIN_1
RequestHeader set SSL_CIPHER_USEKEYSIZE   %{SSL_CIPHER_USEKEYSIZE}e
env=SSL_CIPHER_USEKEYSIZE
RequestHeader set SSL_CIPHER_SUITE%{SSL_CIPHER}e
env=SSL_CIPHER

To make this work I also patched two other things:
1. mod_headers' RequestHeader directive wasn't able to take an env clause as
a forth argument in contrast to the Header directive. I don't know the
reason
for that behavior, but env clause seams to work fine with the SSL
environment
variables for RequestHeaders. This was necessary to avoid an empty header if
the environment variable isn't present.
If there are objections, let me know.
2. SSL_CLIENT_CERT_CHAIN_n is broken. To me it seems that somebody has tried
to change SSL_CLIENT_CERT_CHAINn to SSL_CLIENT_CERT_CHAIN_n. However, the
introduction of the "_" wasn't quite consistent. I patched that and now I
can see the intermediate CAs as SSL_CLIENT_CERT_CHAIN_0 to
SSL_CLIENT_CERT_CHAIN_n in the environment.

Last but not least I have updated the mod_headers documentation with the new
option "E" and an example for forwarding the Web browser's client
certificate and some other SSL information.

I think the described patches and enhancements are quite reasonable and I
would like to make them part of the standard Apache distribution. I have
already produced a patch file that works for Apache 2.0.43 and 2.0.44. I
would appreciate guidance on how to proceed.

Comments welcome!

Regards,
Maik

Maik Mueller
Development Architect
SAP




[PATCH] Allow modules to load if they were compiled with AP_DEBUG (--enable-maintainer-mode) although the core was not.

2003-02-12 Thread Sander Striker
This patch might be incomplete in that I'm not completely sure how to
handle the exports.  Basically ap_[gs]et_module_config should always
be in exports.c.

Secondly, while looking at util_debug.c, I see that we have more
functions we conditionally define.  We may want to define those all
the time aswell.

Sander


Index: server/util_debug.c
===
RCS file: /home/cvs/httpd-2.0/server/util_debug.c,v
retrieving revision 1.9
diff -u -r1.9 util_debug.c
--- server/util_debug.c 3 Feb 2003 17:53:20 -   1.9
+++ server/util_debug.c 12 Feb 2003 10:10:19 -
@@ -94,6 +94,13 @@
 return strstr(s,c);
 }

+#endif /* AP_DEBUG */
+
+#if defined(ap_get_module_config)
+#undef ap_get_module_config
+AP_DECLARE(void *) ap_get_module_config(const ap_conf_vector_t *cv,
+const module *m);
+#endif

 AP_DECLARE(void *) ap_get_module_config(const ap_conf_vector_t *cv,
 const module *m)
@@ -110,11 +117,14 @@
  * @param val The module-specific data to set
  * @deffunc void ap_set_module_config(ap_conf_vector_t *cv, const module *m, void 
*val)
  */
+#if defined(ap_set_module_config)
+#undef ap_set_module_config
+AP_DECLARE(void) ap_set_module_config(ap_conf_vector_t *cv, const module *m,
+  void *val);
+#endif
+
 AP_DECLARE(void) ap_set_module_config(ap_conf_vector_t *cv, const module *m,
   void *val)
 {
 ((void **)cv)[m->module_index] = val;
 }
-
-
-#endif /* AP_DEBUG */




Problem with SSL in 64 bit build on Solaris

2003-02-12 Thread CASTELLE Thomas
Title: Problem with SSL in 64 bit build on Solaris





Hi there !


I'm sorry to come again with this, but it is quite important to us : is someone looking at the bug 16333 I opened a few weeks ago ? I don't need a solution right now, but I would be relieved if I know that someone's looking at it...

To summarize the problem, mod_ssl doesn't seem to work when Apache 2.0.44 is build in 64 bit on Solaris 8.


Thanks for your answer !


Thomas.