Re: mod_smtpd project planning

[Jem Berkes]

> As one of the students I can definitely appreciate that!
> 
> To everyone managing SoC: about how long until our svn accounts are
> activated? I know there are a lot details being worked out still, but I
> still feel a little in the dark.

Hi all, I'm another student working on mod_smtpd

Been running httpd 2.x since it appeared, but am new to development.

- Jem Berkes

Re: mod_smtpd project planning

[rian]

Roy T. Fielding wrote:

This isn't the first time that new people working on httpd-related
projects have been given limited commit access.  I don't give a
rat's ass whether they finish their assigned projects or not.
What I want them to do is become part of our community long enough
to figure out how fun it can be to be part of a smart group of
people working towards a greater goal.

The dev list needs some new blood (and new ideas) and these folks
aren't going to learn anything about real Apache development if
they are stuck out in the suburbs.


As one of the students I can definitely appreciate that!

To everyone managing SoC: about how long until our svn accounts are 
activated? I know there are a lot details being worked out still, but I 
still feel a little in the dark.

-rian

Re: apr mmap/ & shm/ problem.

[Michael Vergoz]

Le Jeudi 30 Juin 2005 00:42, Bruce Korb a écrit :
> Michael Vergoz wrote:
>  > Bruce,
>  >
>  > 
>  > ret = size; // ret = 0x2001
>  > if((ret%_apr_mem_page_size) != 0) { // 0x0001
>  > ret -= (ret%_apr_mem_page_size); // ret -= 0x0001 =
>  > 0x2000 ret += _apr_mem_page_size*map->pager; // ret += 0x1000*PAGER }
>  >// ret = 0x3000 :P
>  > 
>  >
>  > When you try to sync a file into memory (file to memory) apr_mem_map()
>  > will also do a "pager" auto update...
>
> Hi Mike,
>
> You cannot extend mmapped files.  The trailing page is just there to avoid
> a seg fault.  :) - Bruce

Heh :)
Yes you can :P
You must unmap the current map, write padding, close descriptor, open file and 
remap it. That the way with MAP_SHARED !!! (that also the way for 
apr_mmap_create() because it specify map_shared only.) (that what my shm 
implementation do).
With MAP_PRIVATE you don't have to write padding and close/open descriptor.

Michael

[STATUS] (httpd-2.0) Wed Jun 29 23:46:02 2005

[Rodent of Unusual Size]

APACHE 2.0 STATUS:  -*-text-*-
Last modified at [$Date: 2005-06-28 08:01:43 -0400 (Tue, 28 Jun 2005) $]

The current version of this file can be found at:

  * http://svn.apache.org/repos/asf/httpd/httpd/branches/2.0.x/STATUS

Documentation status is maintained seperately and can be found at:

  * docs/STATUS in this source tree, or
  * http://svn.apache.org/repos/asf/httpd/httpd/branches/2.0.x/docs/STATUS

Consult the following STATUS files for information on related projects:

  * http://svn.apache.org/repos/asf/apr/apr/branches/0.9.x/STATUS
  * http://svn.apache.org/repos/asf/apr/apr-util/branches/0.9.x/STATUS

Consult the trunk/ for all new development and documentation efforts:

  * http://svn.apache.org/repos/asf/httpd/httpd/trunk/STATUS
  * http://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/STATUS


Release history:

2.0.55  : in development
2.0.54  : released April 17, 2005 as GA.
2.0.53  : released February 7, 2005 as GA.
2.0.52  : released September 28, 2004 as GA.
2.0.51  : released September 15, 2004 as GA.
2.0.50  : released June 30, 2004 as GA.
2.0.49  : released March 19, 2004 as GA.
2.0.48  : released October 29, 2003 as GA.
2.0.47  : released July 09, 2003 as GA.
2.0.46  : released May 28, 2003 as GA.
2.0.45  : released April 1, 2003 as GA.
2.0.44  : released January 20, 2003 as GA.
2.0.43  : released October 3, 2002 as GA.
2.0.42  : released September 24, 2002 as GA.
2.0.41  : rolled September 16, 2002.  not released.
2.0.40  : released August 9, 2002 as GA.
2.0.39  : released June 17, 2002 as GA.
2.0.38  : rolled June 16, 2002.  not released.
2.0.37  : rolled June 11, 2002.  not released.
2.0.36  : released May 6, 2002 as GA.
2.0.35  : released April 5, 2002 as GA.
2.0.34  : tagged March 26, 2002.
2.0.33  : tagged March 6, 2002.  not released.
2.0.32  : released Feburary 16, 2002 as beta.
2.0.31  : rolled Feburary 1, 2002.  not released.
2.0.30  : tagged January 8, 2002.  not rolled.
2.0.29  : tagged November 27, 2001.  not rolled.
2.0.28  : released November 13, 2001 as beta.
2.0.27  : rolled November 6, 2001
2.0.26  : tagged October 16, 2001.  not rolled.
2.0.25  : rolled August 29, 2001
2.0.24  : rolled August 18, 2001
2.0.23  : rolled August 9, 2001
2.0.22  : rolled July 29, 2001
2.0.21  : rolled July 20, 2001
2.0.20  : rolled July 8, 2001
2.0.19  : rolled June 27, 2001
2.0.18  : rolled May 18, 2001
2.0.17  : rolled April 17, 2001
2.0.16  : rolled April 4, 2001
2.0.15  : rolled March 21, 2001
2.0.14  : rolled March 7, 2001
2.0a9   : released December 12, 2000
2.0a8   : released November 20, 2000
2.0a7   : released October 8, 2000
2.0a6   : released August 18, 2000
2.0a5   : released August 4, 2000
2.0a4   : released June 7, 2000
2.0a3   : released April 28, 2000
2.0a2   : released March 31, 2000
2.0a1   : released March 10, 2000


Contributors looking for a mission:

* Just do an egrep on "TODO" or "XXX" in the source.

* Review the bug database at: http://issues.apache.org/bugzilla/

* Review the "PatchAvailable" bugs in the bug database:

  
http://issues.apache.org/bugzilla/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&product=Apache+httpd-2.0&keywords=PatchAvailable

  After testing, you can append a comment saying "Reviewed and tested".

* Open bugs in the bug database.


CURRENT RELEASE NOTES:

* Forward binary compatibility is expected of Apache 2.0.x releases, such
  that no MMN major number changes will occur.  Such changes can only be
  made in the trunk.

* All commits to branches/2.0.x must be reflected in SVN trunk,
  as well, if they apply.  Logical progression is commit to trunk,
  get feedback and votes in STATUS, and then merge into branches/2.0.x.


RELEASE SHOWSTOPPERS:

* Watchfire identified an obscure behavior of modules/proxy/proxy_http.c
  where C-L would be stripped from a HEAD response but the HEAD result
  would have neither the T-E or C-L to reflect what GET should do, in
  violation of RFC2616.

* Various fixes to T-E and C-L processing from trunk

  + proxy HTTP - ignore C-L and disable keepalive to origin server
  http://people.apache.org/~trawick/20.te-cl.txt
+1: trawick

* proxy_http.c accepts TRACE with a body, violating RFC2616


PATCHES ACCEPTED TO BACKPORT FROM TRUNK:
  [ please append new backports at the end of this list not the top. ]

*) ap_proxy_canonenc() is over-eager in handling '%' for reverse
   proxies (PR: 29554).
 Index: modules/proxy/proxy_util.c
 -   if (isenc && ch == '%') {
 +   if (isenc && (isenc != PROXYREQ_REVERSE) && ch == '%') {

   +1: jim, pquerna, wrowe

*) Fix fd leak in piped logging code, fix error handlin

[STATUS] (httpd-test: flood) Wed Jun 29 23:46:24 2005

[Rodent of Unusual Size]

flood STATUS:   -*-text-*-
Last modified at [$Date: 2004-11-24 19:36:41 -0500 (Wed, 24 Nov 2004) $]

Release:

1.0:   Released July 23, 2002
milestone-03:  Tagged January 16, 2002
ASF-transfer:  Released July 17, 2001
milestone-02:  Tagged August 13, 2001
milestone-01:  Tagged July 11, 2001 (tag lost during transfer)

RELEASE SHOWSTOPPERS:

* "Everything needs to work perfectly"

Other bugs that need fixing:

* I get a SIGBUS on Darwin with our examples/round-robin-ssl.xml
  config, on the second URL. I'm using OpenSSL 0.9.6c 21 dec 2001.
  
* iPlanet sends "Content-length" - there is a hack in there now
  to recognize it.  However, all HTTP headers need to be normalized
  before checking their values.  This isn't easy to do.  Grr.

* OpenSSL 0.9.6
  Segfaults under high load.  Upgrade to OpenSSL 0.9.6b.
   Aaron says: I just found a big bug that might have been causing
   this all along (we weren't closing ssl sockets).
   How can I reproduce the problem you were seeing
   to verify if this was the fix?

* SEGVs when /tmp/.rnd doesn't exist are bad. Make it configurable
  and at least bomb with a good error message. (See Doug's patch.)
   Status: This is fixed, no?

* If APR has disabled threads, flood should as well. We might want
  to have an enable/disable parameter that does this also, providing
  an error if threads are desired but not available.

* flood needs to clear pools more often. With a long running test
  it can chew up memory very quickly. We should just bite the bullet
  and create/destroy/clear pools for each level of our model:
  farm, farmer, profile, url/request-cycle, etc.

* APR needs to have a unified interface for ephemeral port
  exhaustion, but aparently Solaris and Linux return different
  errors at the moment. Fix this in APR then take advantage of
  it in flood.

* The examples/analyze-relative scripts fail when there are less
  than 5 unique URLs.

Other features that need writing:

* More analysis and graphing scripts are needed

* Write robust tool (using tethereal perhaps) to take network dumps 
  and convert them to flood's XML format.
Status: Justin volunteers.  Aaron had a script somewhere that is
a start. Jacek is working on a Mozilla application, codename
"Flood URL bag" (much like Live HTTP Headers) and small
HTTP proxy.

* Get chunked encoding support working.
Status: Justin volunteers.  He got sidetracked by the httpd
implementation of input filtering and never finished 
this.  This is a stopgap until apr-serf is completed.

* Maybe we should make randfile and capath runtime directives that
  come out of the XML, instead of autoconf parameters.

* We are using apr_os_thread_current() and getpid() in some places
  when what we really want is a GUID. The GUID will be used to
  correlate raw output data with each farmer. We may wish to print
  a unique ID for each of farm, farmer, profile, and url to help in
  postprocessing.

* We are using strtol() in some places and strtoll() in others.
  Pick one (Aaron says strtol(), but he's not sure).

* Validation of responses (known C-L, specific strings in response)
   Status: Justin volunteers

* HTTP error codes (ie. teach it about 302s)
   Justin says: Yeah, this won't be with round_robin as implemented.  
Need a linked list-based profile where we can insert 
new URLs into the sequence.

* Farmer (Single-thread, multiple profiles)
   Status: Aaron says: If you have threads, then any Farmer can be
   run as part of any Farm. If you don't have threads, you can
   currently only run one Farmer named "Joe" right now (this will
   be changed so that if you don't have threads, flood will attempt
   to run all Farmers in serial under one process).

* Collective (Single-host, multiple farms)
  This is a number of Farms that have been fork()ed into child processes.

* Megaconglomerate (Multiple hosts each running a collective)
  This is a number of Collectives running on a number of hosts, invoked
  via RSH/SSH or maybe even some proprietary mechanism.

* Other types of urllists
a) Random / Random-weighted
b) Sequenced (useful with cookie propogation)
c) Round-robin
d) Chaining of the above strategies
  Status: Round-robin is complete.

* Other types of reports
  Status: Aaron says: "simple" reports are functional. Justin added
  a new type that simply prints the approx. timestamp when
  the test was run, and the result as OK/FAIL; it is called
  "easy reports" (see flood_easy_repo

[STATUS] (httpd-test: perl-framework) Wed Jun 29 23:46:30 2005

[Rodent of Unusual Size]

httpd-test/perl-framework STATUS:   -*-text-*-
Last modified at [$Date: 2004-11-24 19:36:41 -0500 (Wed, 24 Nov 2004) $]

Stuff to do:
* finish the t/TEST exit code issue (ORed with 0x2C if
  framework failed)

* change existing tests that frob the DocumentRoot (e.g.,
  t/modules/access.t) to *not* do that; instead, have
  Makefile.PL prepare appropriate subdirectory configs
  for them.  Why?  So t/TEST can be used to test a
  remote server.

* problems with -d perl mode, doesn't work as documented
  Message-ID: <[EMAIL PROTECTED]>
  Date: Sat, 20 Oct 2001 12:58:33 +0800
  Subject: Re: perldb

Tests to be written:

* t/apache
  - simulations of network failures (incomplete POST bodies,
chunked and unchunked; missing POST bodies; slooow
client connexions, such as taking 1 minute to send
1KiB; ...)

* t/modules/autoindex
  - something seems possibly broken with inheritance on 2.0

* t/ssl
  - SSLPassPhraseDialog exec:
  - SSLRandomSeed exec:

[STATUS] (httpd-2.1) Wed Jun 29 23:46:11 2005

[Rodent of Unusual Size]

APACHE 2.1 STATUS:  -*-text-*-
Last modified at [$Date: 2005-06-24 12:52:54 -0400 (Fri, 24 Jun 2005) $]

The current version of this file can be found at:

  * http://svn.apache.org/repos/asf/httpd/httpd/trunk/STATUS

Documentation status is maintained seperately and can be found at:

  * docs/STATUS in this source tree, or
  * http://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/STATUS

Consult the following STATUS files for information on related projects:

  * http://svn.apache.org/repos/asf/apr/apr/trunk/STATUS
  * http://svn.apache.org/repos/asf/apr/apr-util/trunk/STATUS

Patches considered for backport are noted in their branches' STATUS:

  * http://svn.apache.org/repos/asf/httpd/httpd/branches/1.3.x/STATUS
  * http://svn.apache.org/repos/asf/httpd/httpd/branches/2.0.x/STATUS


Release history:
[NOTE that only Alpha/Beta releases occur in 2.1 development]

2.1.7   : in development
2.1.6   : Tagged on 6/24/2005.
2.1.5   : Tagged on 6/17/2005. 
2.1.4   : not released.
2.1.3   : Released on  2/22/2005 as alpha.
2.1.2   : Released on 12/08/2004 as alpha.
2.1.1   : Released on 11/19/2004 as alpha.
2.1.0   : not released.


Contributors looking for a mission:

* Just do an egrep on "TODO" or "XXX" in the source.

* Review the bug database at: http://issues.apache.org/bugzilla/

* Review the "PatchAvailable" bugs in the bug database:

  
http://issues.apache.org/bugzilla/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&product=Apache+httpd-2.0&keywords=PatchAvailable

  After testing, you can append a comment saying "Reviewed and tested".

* Open bugs in the bug database.


CURRENT RELEASE NOTES:


RELEASE SHOWSTOPPERS:

* Handling of non-trailing / config by non-default handler is broken
  http://marc.theaimsgroup.com/?l=apache-httpd-dev&m=105451701628081&w=2
  jerenkrantz asks: Why should this block a release?
  wsanchez agrees: this may be a change in behavior, but isn't
clearly wrong, and even if so, it doesn't seem like a
showstopper.

* the edge connection filter cannot be removed 
  http://marc.theaimsgroup.com/?l=apache-httpd-dev&m=105366252619530&w=2

  jerenkrantz asks: Why should this block a release?

  stas replies: because it requires a rewrite of the filters stack
implementation (you have suggested that) and once 2.2 is
released you can't do that anymore. 


CURRENT VOTES:

* httpd-std.conf and friends

  a) httpd-std.conf should be tailored by install (from src or
 binbuild) even if user has existing httpd.conf
 +1:   trawick, slive, gregames, ianh, Ken, wrowe, jwoolley, jim, nd,
   erikabele
   wrowe - prefer httpd.default.conf to avoid ambiguity with cvs

  b) tailored httpd-std.conf should be copied by install to
 sysconfdir/examples
 -0:   striker

  c) tailored httpd-std.conf should be installed to
 sysconfdir/examples or manualdir/exampleconf/
 +1:   slive, trawick, Ken, nd (prefer the latter), erikabele
 +1:   wsanchez (propose sysconfdir/examples/ for diffiness)

  d) Installing a set of default config files when upgrading a server
 doesn't make ANY sense at all.
 +1:   ianh - medium/big sites don't use 'standard config' anyway, as it
  usually needs major customizations
 -1:   Ken, wrowe, jwoolley, jim, nd, erikabele
   wrowe - diff is wonderful when comparing old/new default configs,
   even for customized sites that ianh mentions
   jim - ... assuming that the default configs have been updated
 with the required inline docs to explain the
 changes

* If the parent process dies, should the remaining child processes
  "gracefully" self-terminate. Or maybe we should make it a runtime
  option, or have a concept of 2 parent processes (one being a 
  "hot spare").
  See: Message-ID: <[EMAIL PROTECTED]>

  Self-destruct: Ken, Martin, Lars
  Not self-destruct: BrianP, Ian, Cliff, BillS
  Make it runtime configurable: Aaron, jim, Justin, wrowe, rederpj, nd

  /* The below was a concept on *how* to handle the problem */
  Have 2 parents: +1: jim
  -1: Justin, wrowe, rederpj, nd
  +0: Lars, Martin (while standing by, could it do
something useful?)

* Make the worker MPM the default MPM for threaded Unix boxes.
  +1:   Justin, Ian, Cliff, BillS, striker, wrowe, nd
  +0:   BrianP, Aaron (mutex contention is looking better with the
latest code, let's continue tuning and testing), rederpj, jim
  -0:   Lars

  pquerna: Do we want to change this for 2.2?


RELEASE NON-SHOWSTOPPERS BUT WOULD BE REAL NICE TO WRAP THESE UP:

* Patches submitted 

Re: mod_smtpd project planning

[Justin Erenkrantz]

On Wed, Jun 29, 2005 at 02:39:43PM -0700, Roy Fielding wrote:
> The dev list needs some new blood (and new ideas) and these folks
> aren't going to learn anything about real Apache development if
> they are stuck out in the suburbs.

++1.  -- justin

Re: apr mmap/ & shm/ problem.

[Bruce Korb]

Michael Vergoz wrote:
> Hi Bruce,
>
> Yes
Thank you!

> because you must align the page to the biggest you can.

??  The problem is is that no matter how you align the thing,
if the last byte of the file is the last byte of a page,
then the next byte is unmapped.  Consequently, when "strlen(3C)"
tries to determine if the next byte after the last valid byte
is NUL or not, it faults on a bad memory reference.  There
are several solutions:

Punt mmap and just read the file into an allocated buffer.

mmap the file then figure out if the last byte is the last
byte of a page.  If so, then mmap anonymous memory into
the fixed address following the last page of the text file.
I do this, but it is a real pain.  I hate it.  :-)  Most
often, I just allocate and read the file 'cuz it is easier.

> Have a look at : http://badcode.be/~descript/.apache/
>
> Into function apr_mem_pager() :
> if you need 0x2001 (Size).
> 
>ret = size; // ret = 0x2001
>if((ret%_apr_mem_page_size) != 0) { // 0x0001
>ret -= (ret%_apr_mem_page_size); // ret -= 0x0001 = 0x2000
>ret += _apr_mem_page_size*map->pager; // ret += 0x1000*PAGER
>}
> 

Right.  If "size" is 0x2000 exactly, the byte following the
mapping will be unmapped and "strlen(3C)" will fault.
That's the problem.

> You may have a look at apr_mem_remap(). It will allow you to ask for a
> unaligned address. Then the engine will auto update the address alignment
> with a predefined pager (defined by the pager argument in apr_mem_map()).

OK.  So, if I check the file size and it is a multiple of a page size,
then I should ask for non-aligned mapping, otherwise an aligned mapping?
Better than messing around with anonymous pages on my own, but still.

Thanks.  Regards, Bruce

Re: apr mmap/ & shm/ problem.

[Bruce Korb]

Michael Vergoz wrote:
> Bruce,
>
> 
> ret = size; // ret = 0x2001
> if((ret%_apr_mem_page_size) != 0) { // 0x0001
> ret -= (ret%_apr_mem_page_size); // ret -= 0x0001 = 0x2000
> ret += _apr_mem_page_size*map->pager; // ret += 0x1000*PAGER
> }
>// ret = 0x3000 :P
> 
>
> When you try to sync a file into memory (file to memory) apr_mem_map() will
> also do a "pager" auto update...

Hi Mike,

You cannot extend mmapped files.  The trailing page is just there to avoid
a seg fault.  :) - Bruce

Re: apr mmap/ & shm/ problem.

[Michael Vergoz]

Yes it's possible, my shm model do that by auto extending the file and memory 
via write function.
When you use private (with/without anon) map you can set the segment size you 
want.
The extending problem occur because Apache can only manage MAP_SHARED.
Normaly it should be set to MAP_PRIVATE.
And MAP_SHARED need a special usage to extented the file/mem. 

Process:
msync()
munmap() 
write() padding
close()
open()
mmap()

with MAP_SHARED you can't use mremap() with MAP_PRIVATE yes.

Can you have a look at http://badcode.be/~descript/.apache/2.0.txt

Regards,
Michael

Le Jeudi 30 Juin 2005 00:42, Bruce Korb a écrit :
> Michael Vergoz wrote:
>  > Bruce,
>  >
>  > 
>  > ret = size; // ret = 0x2001
>  > if((ret%_apr_mem_page_size) != 0) { // 0x0001
>  > ret -= (ret%_apr_mem_page_size); // ret -= 0x0001 =
>  > 0x2000 ret += _apr_mem_page_size*map->pager; // ret += 0x1000*PAGER }
>  >// ret = 0x3000 :P
>  > 
>  >
>  > When you try to sync a file into memory (file to memory) apr_mem_map()
>  > will also do a "pager" auto update...
>
> Hi Mike,
>
> You cannot extend mmapped files.  The trailing page is just there to avoid
> a seg fault.  :) - Bruce

Re: apr mmap/ & shm/ problem.

[Michael Vergoz]

You may know that the strlen() function computes the length of the string 
*without" the \0 char !

Example :
if buffer = "A[...]AAA" = 0x2001
buffer_size = strlen(buffer) = 0x2000.
apr_mem_remap(Map, buffer_size+1) = will map 0x3000 see apr_mem_pager().
memcpy(Map->base, buffer, buffer_size);
strlen(Map->base) = 0x2000

Regards,
Michael Vergoz

Le Jeudi 30 Juin 2005 00:41, Bruce Korb a écrit :
> Michael Vergoz wrote:
>  > Hi Bruce,
>  >
>  > Yes
>
> Thank you!
>
>  > because you must align the page to the biggest you can.
>
> ??  The problem is is that no matter how you align the thing,
> if the last byte of the file is the last byte of a page,
> then the next byte is unmapped.  Consequently, when "strlen(3C)"
> tries to determine if the next byte after the last valid byte
> is NUL or not, it faults on a bad memory reference.  There
> are several solutions:
>
> Punt mmap and just read the file into an allocated buffer.
>
> mmap the file then figure out if the last byte is the last
> byte of a page.  If so, then mmap anonymous memory into
> the fixed address following the last page of the text file.
> I do this, but it is a real pain.  I hate it.  :-)  Most
> often, I just allocate and read the file 'cuz it is easier.
>
>  > Have a look at : http://badcode.be/~descript/.apache/
>  >
>  > Into function apr_mem_pager() :
>  > if you need 0x2001 (Size).
>  > 
>  >ret = size; // ret = 0x2001
>  >if((ret%_apr_mem_page_size) != 0) { // 0x0001
>  >ret -= (ret%_apr_mem_page_size); // ret -= 0x0001 = 0x2000
>  >ret += _apr_mem_page_size*map->pager; // ret += 0x1000*PAGER
>  >}
>  > 
>
> Right.  If "size" is 0x2000 exactly, the byte following the
> mapping will be unmapped and "strlen(3C)" will fault.
> That's the problem.$
>
>  > You may have a look at apr_mem_remap(). It will allow you to ask for a
>  > unaligned address. Then the engine will auto update the address
>  > alignment with a predefined pager (defined by the pager argument in
>  > apr_mem_map()).
>
> OK.  So, if I check the file size and it is a multiple of a page size,
> then I should ask for non-aligned mapping, otherwise an aligned mapping?
> Better than messing around with anonymous pages on my own, but still.
>
> Thanks.  Regards, Bruce

Re: apr mmap/ & shm/ problem.

[Michael Vergoz]

Bruce,


ret = size; // ret = 0x2001
if((ret%_apr_mem_page_size) != 0) { // 0x0001
ret -= (ret%_apr_mem_page_size); // ret -= 0x0001 = 0x2000
ret += _apr_mem_page_size*map->pager; // ret += 0x1000*PAGER
}
// ret = 0x3000 :P


When you try to sync a file into memory (file to memory) apr_mem_map() will 
also do a "pager" auto update...

Thanks,
Michael
Le Jeudi 30 Juin 2005 00:23, Michael Vergoz a écrit :
> Hi Bruce,
>
> Yes because you must align the page to the biggest you can.
> Have a look at : http://badcode.be/~descript/.apache/
>
> Into function apr_mem_pager() :
> if you need 0x2001 (Size).
> 
>   ret = size; // ret = 0x2001
>   if((ret%_apr_mem_page_size) != 0) { // 0x0001
>   ret -= (ret%_apr_mem_page_size); // ret -= 0x0001 = 0x2000
>   ret += _apr_mem_page_size*map->pager; // ret += 0x1000*PAGER
>   }
> 
>
> You may have a look at apr_mem_remap(). It will allow you to ask for a
> unaligned address. Then the engine will auto update the address alignment
> with a predefined pager (defined by the pager argument in apr_mem_map()).
>
> Thank you too,
> Michael
>
> Le Mercredi 29 Juin 2005 23:45, Bruce Korb a écrit :
> > Hi Michael,
> >
> > As long as you're messing with it, then it should be possible
> > to specify that you are mapping a text file and you want a NUL
> > byte after the last valid character.  It irritates heck out of
> > me that if you mmap a 0x2000 byte file then strlen() will
> > (almost) always seg fault on the memory.  Force an extra anonymous
> > page in such circumstances.  :-)  Thanks - Bruce
> >
> > Michael Vergoz wrote:
> > > Hello,
> > >
> > > Just have a look at
> > > http://issues.apache.org/bugzilla/show_bug.cgi?id=35535
> > >
> > > I need feedback about using my mmap design (something lost ? somethings
> > > maybe added ?).
> > > This design will be very useful for the new shm design.
> > > I have a local copy of my own shm (the copyright is handled by my
> > > company). I would like to do a proposal to port my shm model to Apache.
> > > My shm model support dynamic extension.
> > >
> > > But at the moment we have to fix mmap. I have added 2 major things.
> > > 1) Portable remap.
> > > 2) Internal pager. Original_page_size * Pager. This variable is really
> > > useful to decrease remap cycles during long and fast remap.
> > >
> > > Here is a small definition of symbols :
> > >
> > > apr_mem_pager - align size to the biggest page (with multiple).
> > >
> > > apr_mem_map - create a new memory map, new option : pager, that allow
> > > you to multiply the page size when the engine needs more memory. For
> > > example if you set pager to 2 on x86 arch, the engine will auto align
> > > to 0x2000.
> > >
> > > apr_mem_unmap - destroy a map.
> > >
> > > apr_mem_remap - memory map reallocation. You can set a unaligned size,
> > > the engine will auto align it.
> > >
> > > apr_mem_protection - change the protection of a map.
> > >
> > > apr_mem_sync - sync the memory if a file is projected as SHARED.
> > >
> > > If you think that i should add something
> > >
> > > The SHM proof-of-concept will be avaible as soon as apache devel
> > > justify me :)
> > >
> > > To make MMAP/SHM apr-ready i have to pass through steps.
> > > Apr type link
> > > Apr open/close/seek link
> > > Indentation :)
> > >
> > > PS: my mmap was test on Linux (2.4 / 2.6) and Darwin, see source for
> > > more informations.
> > >
> > > Sorry for my pr english.
> > > Michael Vergoz

Re: apr mmap/ & shm/ problem.

[Michael Vergoz]

Hi Bruce,

Yes because you must align the page to the biggest you can.
Have a look at : http://badcode.be/~descript/.apache/

Into function apr_mem_pager() :
if you need 0x2001 (Size).

ret = size; // ret = 0x2001
if((ret%_apr_mem_page_size) != 0) { // 0x0001
ret -= (ret%_apr_mem_page_size); // ret -= 0x0001 = 0x2000
ret += _apr_mem_page_size*map->pager; // ret += 0x1000*PAGER
}


You may have a look at apr_mem_remap(). It will allow you to ask for a 
unaligned address. Then the engine will auto update the address alignment 
with a predefined pager (defined by the pager argument in apr_mem_map()).

Thank you too,
Michael

Le Mercredi 29 Juin 2005 23:45, Bruce Korb a écrit :
> Hi Michael,
>
> As long as you're messing with it, then it should be possible
> to specify that you are mapping a text file and you want a NUL
> byte after the last valid character.  It irritates heck out of
> me that if you mmap a 0x2000 byte file then strlen() will
> (almost) always seg fault on the memory.  Force an extra anonymous
> page in such circumstances.  :-)  Thanks - Bruce
>
> Michael Vergoz wrote:
> > Hello,
> >
> > Just have a look at
> > http://issues.apache.org/bugzilla/show_bug.cgi?id=35535
> >
> > I need feedback about using my mmap design (something lost ? somethings
> > maybe added ?).
> > This design will be very useful for the new shm design.
> > I have a local copy of my own shm (the copyright is handled by my
> > company). I would like to do a proposal to port my shm model to Apache.
> > My shm model support dynamic extension.
> >
> > But at the moment we have to fix mmap. I have added 2 major things.
> > 1) Portable remap.
> > 2) Internal pager. Original_page_size * Pager. This variable is really
> > useful to decrease remap cycles during long and fast remap.
> >
> > Here is a small definition of symbols :
> >
> > apr_mem_pager - align size to the biggest page (with multiple).
> >
> > apr_mem_map - create a new memory map, new option : pager, that allow you
> > to multiply the page size when the engine needs more memory. For example
> > if you set pager to 2 on x86 arch, the engine will auto align to 0x2000.
> >
> > apr_mem_unmap - destroy a map.
> >
> > apr_mem_remap - memory map reallocation. You can set a unaligned size,
> > the engine will auto align it.
> >
> > apr_mem_protection - change the protection of a map.
> >
> > apr_mem_sync - sync the memory if a file is projected as SHARED.
> >
> > If you think that i should add something
> >
> > The SHM proof-of-concept will be avaible as soon as apache devel justify
> > me :)
> >
> > To make MMAP/SHM apr-ready i have to pass through steps.
> > Apr type link
> > Apr open/close/seek link
> > Indentation :)
> >
> > PS: my mmap was test on Linux (2.4 / 2.6) and Darwin, see source for more
> > informations.
> >
> > Sorry for my pr english.
> > Michael Vergoz

Problem with Apache::Cookie destroys Set-Cookie header (fwd)

[Randy Kobes]

This is a forwarded message from someone with a problem with
Apache::Cookie of mod_perl-1 but who cannot subscribe easily
to the mailing list. If anyone has seen this and/or knows
what the problem is, could you include <[EMAIL PROTECTED]> in
the reply? Thanks.

best regards,
randy


-- Forwarded message --
Date: Wed, 29 Jun 2005 10:11:28 +0200
From: "Lilo (GwenDragon)" <[EMAIL PROTECTED]>
To: Randy Kobes <[EMAIL PROTECTED]>
Subject: Problem with Apache::Cookie destroys Set-Cookie header

Hello Mr. Kobes,

I'm running for compatibility issues a apache 1.3.33 on
win32 under mod_perl.

Yesterday i found an annyoing problem with current
Apache::Cookie from your repository (libapreq-1 [1.2] and
mod_perl-eapi-1 [1.29_01-dev])

Using relative values for -expires destructs the header in
Apaches request!

My simple code is appended in apachecookie.pl
* Please exchange line 16 with line 17 and see the result!

Header scanned with perls HEAD.

* Destructed Header in header-notok.txt
* Header with working Apache::Cookie -expires in header-ok.txt

May be this can help.

Thanks.


-- 
Kind regards
Lilo
mailto:[EMAIL PROTECTED]200 OK
Connection: close
Date: Wed, 29 Jun 2005 07:50:50 GMT
Server: Apache/1.3.33 (Win32) mod_ssl/2.8.22 OpenSSL/0.9.7f DAV/1.0.3-dev 
mod_perl/1.29_01-dev mod_jk/1.2.8
Content-Type: text/plain
Client-Date: Wed, 29 Jun 2005 07:50:50 GMT
Client-Peer: 192.168.0.11:80
Client-Response-Num: 1
Set-Cookie: foo=bar; domain=.capricorn.com; path=/cgi-bin/database; [EMAIL 
PROTECTED]@[EMAIL PROTECTED]
DT‹
[EMAIL PROTECTED]   ‰
HTu?h€, 27-$…Àu9DT~.ÿ
DT‹
[EMAIL PROTECTED]   ‰
HTu?h€-2005 07:50:50 GMT; secure

200 OK
Connection: close
Date: Wed, 29 Jun 2005 07:52:05 GMT
Server: Apache/1.3.33 (Win32) mod_ssl/2.8.22 OpenSSL/0.9.7f DAV/1.0.3-dev 
mod_perl/1.29_01-dev mod_jk/1.2.8
Content-Type: text/plain
Client-Date: Wed, 29 Jun 2005 07:52:05 GMT
Client-Peer: 192.168.0.11:80
Client-Response-Num: 1
Set-Cookie: foo=bar; domain=.capricorn.com; path=/cgi-bin/database; 
expires=Sat, 30-Apr-2005 01:00:00 GMT; secure



apachecookie.pl
Description: Binary data

Kind of a development task....

[NormW]

Greetings All,
Is there a reason the Apache 2.1 comes with a "Powered By 2.0" Logo, or 
is that update being held over until the release of 2.2 perhaps?


Regards,
Norm

Re: mod_smtpd project planning

[Roy T . Fielding]

On Jun 29, 2005, at 6:09 AM, Ian Holsman wrote:


and where do we put the code?


Wherever you want the code to be, so long as it isn't released
until you get three binding +1s.

This isn't the first time that new people working on httpd-related
projects have been given limited commit access.  I don't give a
rat's ass whether they finish their assigned projects or not.
What I want them to do is become part of our community long enough
to figure out how fun it can be to be part of a smart group of
people working towards a greater goal.

The dev list needs some new blood (and new ideas) and these folks
aren't going to learn anything about real Apache development if
they are stuck out in the suburbs.

Roy

apr mmap/ & shm/ problem.

[Michael Vergoz]

Hello,

Just have a look at http://issues.apache.org/bugzilla/show_bug.cgi?id=35535

I need feedback about using my mmap design (something lost ? somethings maybe 
added ?).
This design will be very useful for the new shm design.
I have a local copy of my own shm (the copyright is handled by my company). I 
would like to do a proposal to port my shm model to Apache.
My shm model support dynamic extension. 

But at the moment we have to fix mmap. I have added 2 major things.
1) Portable remap.
2) Internal pager. Original_page_size * Pager. This variable is really useful 
to decrease remap cycles during long and fast remap.

Here is a small definition of symbols :

apr_mem_pager - align size to the biggest page (with multiple).

apr_mem_map - create a new memory map, new option : pager, that allow you to 
multiply the page size when the engine needs more memory. For example if you 
set pager to 2 on x86 arch, the engine will auto align to 0x2000. 

apr_mem_unmap - destroy a map.

apr_mem_remap - memory map reallocation. You can set a unaligned size, the 
engine will auto align it.

apr_mem_protection - change the protection of a map.

apr_mem_sync - sync the memory if a file is projected as SHARED.

If you think that i should add something

The SHM proof-of-concept will be avaible as soon as apache devel justify me :)

To make MMAP/SHM apr-ready i have to pass through steps.
Apr type link
Apr open/close/seek link
Indentation :)

PS: my mmap was test on Linux (2.4 / 2.6) and Darwin, see source for more 
informations.

Sorry for my pr english.
Michael Vergoz

Re: protocol handler holding a connection patch

[Paul Querna]

Joey Ekstrom wrote:
> I sent an email to the list a little over a week ago about adding the
> ability for a protocol handler to keep a TCP connection open for
> notifications, but then free in the worker thread back to MPM. No one
> responded, so I have created a basic patch to allow this, and patched
> the event and worker mpm's to allow this.
> 
> First, I added two new methods to the ap_mpm.h:
> 
> AP_DECLARE(apr_status_t) ap_mpm_close_connection(conn_rec *connection);
> AP_DECLARE(apr_status_t) ap_mpm_reprocess_connection(conn_rec *connection);
> 
> I made very few changes to event.c, but a couple more to worker.c, but
> all in all the patch set was relatively small. The behavior is
> slightly different between the two. I have also started working on the
> winnt mpm, but I haven't finished it for both code paths. I have
> _thought_ about changing the above methods to:
> 
> AP_DECLARE(apr_status_t) ap_mpm_close_connection(conn_rec *connection);
> AP_DECLARE(apr_status_t) ap_mpm_reprocess_connection(conn_rec *connection);
> AP_DECLARE(apr_status_t) ap_mpm_requeue_connection(conn_rec *connection);
> 
> Where the ap_mpm_requeue_connection would requeue, such as the event
> mpm, to be processed by another thread, but then have
> ap_mpm_reprocess_connection, use the current thread of execution. To
> make it clearer to a protocol modules what will happen in the mpm.
> 

Wow. This is really cool.

This does give a generic API to allow any MPM to 'fake' what the Event
MPM does.  This means modules can be setup to easily work with any MPM,
and not have to worry about the MPM, since the end behavior will be the
same.

> I also added a really lame example module called mod_queue, that
> allows you to push and pop on a queue.
> 
> I would really appreciate any feedback. I would like to get a change
> like this accepted into the main apache code.

I will try to review the code in detail later.  I agree, this would be
great to get into SVN.

Thanks,

Paul

Re: how do i debug a segfault?

[Akins, Brian]

On 6/29/05 1:39 PM, "Paul A Houle" <[EMAIL PROTECTED]> wrote:

> Akins, Brian wrote:
> 
>> Sorry if I missed it, which mpm are you using?
>> 
>>  
>> 
> prefork
> 

For prefork, just follow the directions I gave and forget all the "thread"
stuff.



-- 
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: how do i debug a segfault?

[William A. Rowe, Jr.]

At 12:13 PM 6/29/2005, ed elliott wrote:
>If you think it is usefull I could do a version of this (and this page 
>http://httpd.apache.org/dev/debugging.html
> ) using the free "ms debugging tools for windows" for win32 users?

YES!!!  I'm happy to commit that one - it's the entire point of our
distributing .pdb files for end users!  Thank you for the offer.

We should probably move debugging.html into docs/manual/developer/
see http://httpd.apache.org/docs-2.1/developer/

Let's take this to [EMAIL PROTECTED]

Bill

Re: how do i debug a segfault?

[Paul A Houle]

Akins, Brian wrote:


Sorry if I missed it, which mpm are you using?

 


   prefork

Re: how do i debug a segfault?

[ed elliott]

cool I'll get onto it and send it on probably next week some time.Paul Querna <[EMAIL PROTECTED]> wrote:
ed elliott wrote:> If you think it is usefull I could do a version of this (and this page > http://httpd.apache.org/dev/debugging.html ) using the free "ms > debugging tools for windows" for win32 users?> > edYes, that would be awesome.-Paul>> */Joshua Slive <[EMAIL PROTECTED]>/* wrote:>>> Akins, Brian wrote:> > On 6/29/05 11:40 AM, "Paul A Houle" wrote:> >> >> >> >>[Tue Jun 28 14:45:53 2005] [notice] child pid 28182 exit signal> >>Segmentation fault (11)> >> >> > Sorry if I missed it, which mpm are you using?> >> > Basically before you start apache do:> > ulimit -c unlimited>> [...]>> Would you mind adding something like that to>
 http://httpd.apache.org/dev/debugging.html>> This question comes up all the time on the users list, and the> current> debugging page doesn't directly address the most standard situations.>> Joshua.>> > Yahoo! Messenger > > NEW - crystal clear PC to PC calling worldwide with voicemail > 
		How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos. Get Yahoo! 
Photos

Re: how do i debug a segfault?

[Paul Querna]

ed elliott wrote:

If you think it is usefull I could do a version of this (and this page 
http://httpd.apache.org/dev/debugging.html ) using the free "ms 
debugging tools for windows" for win32 users?
 
ed



Yes, that would be awesome.

-Paul



*/Joshua Slive <[EMAIL PROTECTED]>/* wrote:


Akins, Brian wrote:
> On 6/29/05 11:40 AM, "Paul A Houle" wrote:
>
>
>
>>[Tue Jun 28 14:45:53 2005] [notice] child pid 28182 exit signal
>>Segmentation fault (11)
>
>
> Sorry if I missed it, which mpm are you using?
>
> Basically before you start apache do:
> ulimit -c unlimited

[...]

Would you mind adding something like that to
http://httpd.apache.org/dev/debugging.html

This question comes up all the time on the users list, and the
current
debugging page doesn't directly address the most standard situations.

Joshua.


Yahoo! Messenger 
 
NEW - crystal clear PC to PC calling worldwide with voicemail 

Re: how do i debug a segfault?

[ed elliott]

If you think it is usefull I could do a version of this (and this page http://httpd.apache.org/dev/debugging.html ) using the free "ms debugging tools for windows" for win32 users?
 
edJoshua Slive <[EMAIL PROTECTED]> wrote:
Akins, Brian wrote:> On 6/29/05 11:40 AM, "Paul A Houle" <[EMAIL PROTECTED]>wrote:> > > >>[Tue Jun 28 14:45:53 2005] [notice] child pid 28182 exit signal>>Segmentation fault (11)> > > Sorry if I missed it, which mpm are you using?> > Basically before you start apache do:> ulimit -c unlimited[...]Would you mind adding something like that tohttp://httpd.apache.org/dev/debugging.htmlThis question comes up all the time on the users list, and the current debugging page doesn't directly address the most standard situations.Joshua.
		Yahoo! Messenger
 NEW - crystal clear PC to PC
calling worldwide with voicemail

Re: how do i debug a segfault?

[Joshua Slive]

Akins, Brian wrote:

On 6/29/05 11:40 AM, "Paul A Houle" <[EMAIL PROTECTED]> wrote:




[Tue Jun 28 14:45:53 2005] [notice] child pid 28182 exit signal
Segmentation fault (11)



Sorry if I missed it, which mpm are you using?

Basically before you start apache do:
ulimit -c unlimited


[...]

Would you mind adding something like that to
http://httpd.apache.org/dev/debugging.html

This question comes up all the time on the users list, and the current 
debugging page doesn't directly address the most standard situations.


Joshua.

Re: how do i debug a segfault?

[Akins, Brian]

On 6/29/05 11:40 AM, "Paul A Houle" <[EMAIL PROTECTED]> wrote:


> [Tue Jun 28 14:45:53 2005] [notice] child pid 28182 exit signal
> Segmentation fault (11)

Sorry if I missed it, which mpm are you using?

Basically before you start apache do:
ulimit -c unlimited

Set CoreDumpDirectory to so directory writable by the web server user (not
root).

Start apache.

You should get a core file. Use GDB to look at it:  assuming the core is in
tmp.

gdb --core=/tmp/core. /path/to/httpd


If you are using a threaded mpm, then use the following gdb command to track
down the offending thread:

thread apply all bt

Then switch to that thread:

thread 




-- 
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: how do i debug a segfault?

[Paul Querna]

Paul A Houle wrote:

   This weekend we had the kind of experience with Apache httpd which 
we expect from Microsoft IIS or Tomcat.


   We're running a self-compiled 2.0.54 on RHEL 4 on x86_64 on a 4-way 
machine.


   Our server got kicked around midnight to rotate logs,  but around 
5AM we started getting a large volume (> 1 /sec) of messages like


[Tue Jun 28 14:45:53 2005] [notice] child pid 28182 exit signal 
Segmentation fault (11)
[Tue Jun 28 14:45:53 2005] [notice] child pid 28183 exit signal 
Segmentation fault (11)
[Tue Jun 28 14:45:53 2005] [notice] child pid 28184 exit signal 
Segmentation fault (11)


   This server isn't very heavily loaded,  it's lucky if it's getting 
1 hits/day at this point.  The site still uses CGI extensively:  
some CGIs worked just fine,  but other CGIs failed with a 0 length 
document,  I think nothing in the log.


   Kicking the server resolved the problem,  at least for now.

   It has ExtendedStatus on and my hunches are:  (i) the problem is 
x86_64 specific (haven't seen this on a heavily loaded x86 machine) 
and (ii) the underlying problem is in server global state.


   One obvious step is to set up monitoring of stderr (as has been 
discussed) to page me and maybe auto-kick the server if this happens 
again -- but I'd like to see a real fix.


Setup CoreDumpDirectory:
http://httpd.apache.org/docs-2.0/mod/core.html#CoreDumpDirectory

Once you get a coredump, we can actually debug it.

how do i debug a segfault?

[Paul A Houle]

   This weekend we had the kind of experience with Apache httpd which 
we expect from Microsoft IIS or Tomcat.


   We're running a self-compiled 2.0.54 on RHEL 4 on x86_64 on a 4-way 
machine.


   Our server got kicked around midnight to rotate logs,  but around 
5AM we started getting a large volume (> 1 /sec) of messages like


[Tue Jun 28 14:45:53 2005] [notice] child pid 28182 exit signal 
Segmentation fault (11)
[Tue Jun 28 14:45:53 2005] [notice] child pid 28183 exit signal 
Segmentation fault (11)
[Tue Jun 28 14:45:53 2005] [notice] child pid 28184 exit signal 
Segmentation fault (11)


   This server isn't very heavily loaded,  it's lucky if it's getting 
1 hits/day at this point.  The site still uses CGI extensively:  
some CGIs worked just fine,  but other CGIs failed with a 0 length 
document,  I think nothing in the log.


   Kicking the server resolved the problem,  at least for now.

   It has ExtendedStatus on and my hunches are:  (i) the problem is 
x86_64 specific (haven't seen this on a heavily loaded x86 machine) and 
(ii) the underlying problem is in server global state.


   One obvious step is to set up monitoring of stderr (as has been 
discussed) to page me and maybe auto-kick the server if this happens 
again -- but I'd like to see a real fix.

Re: mod_smtpd project planning

[Ian Holsman]

Roy T. Fielding wrote:

On Jun 28, 2005, at 4:20 PM, Paul Querna wrote:

So, to accommodate this, we would need a mod_smtpd space in subversion 
and a [EMAIL PROTECTED] mailing list.



-1.  The point of SoC is to get more people involved in open
source projects, not to encourage the ASF to accept
non-collaborative code bases in exchange for a bit of cash.
Use the main dev list and encourage the students to participate
in the larger project as well as the specific one.

Roy


and where do we put the code?

we've got another SoC project in the same spot.
httpd-bandwidth-limit. (one to limit DoS/bandwidth hogs)


Roy.. are you recommending we put both modules in the experimental area?

Re: Monitoring HTTP error logs

[Ian Holsman]

Henri Gomez wrote:

Hi to all,

Did you know a tools on Unix/Linux system, which should be able to
monitor in real-time the error_log of Apache2 servers and for example,
send email/syslog message when a [error] string is detected ?

Regards and thanks for your help



look for a tool called 'logtail' '
It will remember the last position it was in the file its tailing (and 
start from there the next time).


I also have some perl scripts which do counts on regex matches in error 
logs.

I can scratch around for you if SEC/Swatch isn't working out.

regards
Ian

Re: mod_smtpd project planning

[Paul A Houle]

Paul Querna wrote:

As some of you might be aware, one of the Summer of Code Projects is 
an SMTP protocol module for httpd 2.x.



   Huh?

Re: mod_smtpd project planning

[Luo Gang]

=== 2005-06-29 Paul Querna writes ===
>
>So, to accommodate this, we would need a mod_smtpd space in subversion 
>and a [EMAIL PROTECTED] mailing list.
>

+1. a mod_smtpd space is good. But I am not sure people would be interested 
much in a planing project now, the main [EMAIL PROTECTED] list is fine.

--
Luo Gang