Re: [RELEASE CANDIDATE] Apache-Test-1.27 RC

2005-10-04 Thread Philip M. Gollucci

Philip M. Gollucci wrote:

A release candidate for Apache-Test 1.27 is now available.

  http://people.apache.org/~pgollucci/at/Apache-Test-1.27-dev.tar.gz
I've noticed we include the RELEASE file in the release tarball.  I don't believe mod_perl or apreq do.  Is this 
intentional ?


--
END

What doesn't kill us can only make us stronger.
Nothing is impossible.

Philip M. Gollucci ([EMAIL PROTECTED]) 301.254.5198
Consultant / http://p6m7g8.net/Resume/
Senior Developer / Liquidity Services, Inc.
  http://www.liquidityservicesinc.com
   http://www.liquidation.com
   http://www.uksurplus.com
   http://www.govliquidation.com
   http://www.gowholesale.com



Re: towards a 2.07 release

2005-10-04 Thread Steve Hay

Joe Schaefer wrote:

Note this is our first non-dev candidate, so please give it the 
extra scrutiny it deserves. Release Candidate #1 -


   http://people.apache.org/~joes/libapreq2-2.07-rc1.tar.gz

This still doesn't build on Win32 with my perl configuration, in which 
PERL_IMPLICIT_SYS is not defined.


I described the problem fully here:

http://marc.theaimsgroup.com/?l=apache-modperl-devm=112626236718924w=2

but I still haven't thought of a good solution.

In brief, the problem is that APR/Request/Param/Param.xs pulls in the 
Perl headers which contain:


#define PerlLIO_link(oldname, newname)link((oldname), (newname))
#define linkwin32_link

and then pulls in apreq_xs_postperl.h, which pulls in mod_perl's 
modperl_perl_unembed.h, which contains:


/* these three clash with apr bucket structure member names */
#undef link
#undef read
#undef free

Thus, the symbol link is left undefined, leading to the linker error:

Param.obj : error LNK2001: unresolved external symbol _link
..\..\..\..\blib\arch\auto\APR\Request\Param\Param.dll : fatal error 
LNK1120: 1 unresolved externals


since Param.xs calls PerlLIO_link() on line 204.

Note that apreq_xs_postperl.h also includes modperl_common_util.h, 
which includes modperl_perl_includes.h, which also includes 
modperl_perl_unembed.h, so link gets #undef'ed again there too.


I'm sure not where this needs fixing -- Perl, mod_perl, or libapreq -- 
but now would be a good time whichever of those it is to be!


I tried simply removing #undef link from modperl_perl_unembed.h to 
see what happens.  Here's what happens:


error C2039: 'win32_link' : is not a member of 'apr_bucket'

so it is definitely required :-(

I can't think how to fix this, other than with some dreadful hack in 
Param.xs.  Any ideas?





Radan Computational Ltd.

The information contained in this message and any files transmitted with it are 
confidential and intended for the addressee(s) only. If you have received this 
message in error or there are any problems, please notify the sender 
immediately. The unauthorized use, disclosure, copying or alteration of this 
message is strictly forbidden. Note that any views or opinions presented in 
this email are solely those of the author and do not necessarily represent 
those of Radan Computational Ltd. The recipient(s) of this message should check 
it and any attached files for viruses: Radan Computational will accept no 
liability for any damage caused by any virus transmitted by this email.


Re: towards a 2.07 release

2005-10-04 Thread Max Kellermann
On 2005/10/03 04:59, Joe Schaefer [EMAIL PROTECTED] wrote:
 Note this is our first non-dev candidate, so please give it the 
 extra scrutiny it deserves. Release Candidate #1 -

All tests pass on Debian Sid/ppc64 and Debian Etch/amd64 (both
pure64).

Max



Can't locate object method FIRSTKEY

2005-10-04 Thread Herve Guillemet


Hi,

I get :

Can't locate object method FIRSTKEY via package APR::Request::Param::Table

when I try something like :

my $req =  APR::Request::Apache2-handle($r);
my $table = $req-param;
my $fk = each %$table;

While using Apache2::Request does work.

How could this be ?

Thanks,

== Hervé Guillemet


Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

2005-10-04 Thread Colm MacCarthaigh
On Mon, Oct 03, 2005 at 08:38:02AM -0400, Jim Jagielski wrote:
 At the very last, if we are assuming behavior which is specifically
 implementation dependent, then a test during configure time that
 ensures sizeof(void *) = sizeof(long) makes sense.
 
 There is no room, IMO, for silent hidden assumptions in httpd.

How about;

Index: configure.in
===
--- configure.in(revision 293579)
+++ configure.in(working copy)
@@ -32,6 +32,9 @@
 dnl export expanded and relative configure argument variables
 APACHE_EXPORT_ARGUMENTS
 
+dnl confirm that a void pointer is large enough to store a long integer
+APACHE_CHECK_VOID_PTR_LEN
+
 dnl Save user-defined environment settings for later restoration
 dnl
 APR_SAVE_THE_ENVIRONMENT(CPPFLAGS)
Index: acinclude.m4
===
--- acinclude.m4(revision 293579)
+++ acinclude.m4(working copy)
@@ -570,3 +570,24 @@
 undefine([ap_ckver_cvar])
 undefine([ap_ckver_name])
 ])
+
+dnl
+dnl APACHE_CHECK_VOID_PTR_LEN
+dnl
+dnl Checks if the size of a void pointer is at least as big as a long 
+dnl integer type.
+dnl
+AC_DEFUN([APACHE_CHECK_VOID_PTR_LEN], [
+
+AC_CACHE_CHECK([for void pointer length], [ap_void_ptr_lt_long],
+[AC_TRY_RUN([
+int main(void)
+{
+return sizeof(void *)  sizeof(long); 
+}], [ap_void_ptr_lt_long=yes], [ap_void_ptr_lt_long=no], 
+[ap_void_ptr_lt_long=no])])
+
+if test $ap_void_ptr_lt_long = no; then
+AC_MSG_ERROR([Size of void * is less than size of int])
+fi
+])
-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]


Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

2005-10-04 Thread Joe Orton
On Tue, Oct 04, 2005 at 11:03:31AM +0100, Colm MacCarthaigh wrote:
 On Mon, Oct 03, 2005 at 08:38:02AM -0400, Jim Jagielski wrote:
  At the very last, if we are assuming behavior which is specifically
  implementation dependent, then a test during configure time that
  ensures sizeof(void *) = sizeof(long) makes sense.
  
  There is no room, IMO, for silent hidden assumptions in httpd.
 
 How about;

-0.5, benefit is nil.

I don't think it's a good idea to clutter up configure with checks for 
hypothetical platforms since the scope is unlimited and the build system 
is fragile enough already.  We could go through this forever; hey, will 
httpd work if sizeof(char) != 1?  hmm, doubt it, lets add a configure 
check for that too etc.

joe


Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

2005-10-04 Thread André Malo
* Joe Orton [EMAIL PROTECTED] wrote:

 On Tue, Oct 04, 2005 at 11:03:31AM +0100, Colm MacCarthaigh wrote:
  On Mon, Oct 03, 2005 at 08:38:02AM -0400, Jim Jagielski wrote:
   At the very last, if we are assuming behavior which is specifically
   implementation dependent, then a test during configure time that
   ensures sizeof(void *) = sizeof(long) makes sense.
   
   There is no room, IMO, for silent hidden assumptions in httpd.
  
  How about;
 
 -0.5, benefit is nil.
 
 I don't think it's a good idea to clutter up configure with checks for 
 hypothetical platforms since the scope is unlimited and the build system 
 is fragile enough already.

Then let's write valid ANSI C plus POSIX. I don't see a benefit in
writing fragile code as well.

(In this particular example I don't see a valid reason to store boolean
stuff in a pointer at all. This is just bad. But perhaps I'm overlooking
something.)

 We could go through this forever; hey, will 
 httpd work if sizeof(char) != 1?  hmm, doubt it, lets add a configure 
 check for that too etc.

Huh? The standard specifies that sizeof(char) is always 1. So that is the
point to stop.

Just my EUR 0.02,
nd


Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

2005-10-04 Thread Jim Jagielski
Joe Orton wrote:
 
 On Tue, Oct 04, 2005 at 11:03:31AM +0100, Colm MacCarthaigh wrote:
  On Mon, Oct 03, 2005 at 08:38:02AM -0400, Jim Jagielski wrote:
   At the very last, if we are assuming behavior which is specifically
   implementation dependent, then a test during configure time that
   ensures sizeof(void *) = sizeof(long) makes sense.
   
   There is no room, IMO, for silent hidden assumptions in httpd.
  
  How about;
 
 -0.5, benefit is nil.
 
 I don't think it's a good idea to clutter up configure with checks for 
 hypothetical platforms since the scope is unlimited and the build system 
 is fragile enough already.  We could go through this forever; hey, will 
 httpd work if sizeof(char) != 1?  hmm, doubt it, lets add a configure 
 check for that too etc.
 

It is bad coding practise to make assumptions on impl dependent
code, period. Although conversion to/from (void *) to (unsigned long)
is likely safe, it should not be silently assumed with no comments,
etc. And yes, if we started making assumptions on sizeof(char),
say with alignment rules or whatever, then we *should* test that.
Simply saying Ah, for all systems we care about this is safe
enough isn't enough. A configure test is cheap insurance.

-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   If you can dodge a wrench, you can dodge a ball.


Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

2005-10-04 Thread Jim Jagielski
One comment: I am curious if we should force the casts to
unsigned longs, instead of signed longs. This would be more
logical. Ideally, a typedef would also make things self-documenting
(ap_ptrint_conv_t or whatever), but that's nit picking. I do think
the error message is wrong though :)

In any case, the below is enough for me to withhold my veto, so +1

Colm MacCarthaigh wrote:
 
 On Mon, Oct 03, 2005 at 08:38:02AM -0400, Jim Jagielski wrote:
  At the very last, if we are assuming behavior which is specifically
  implementation dependent, then a test during configure time that
  ensures sizeof(void *) = sizeof(long) makes sense.
  
  There is no room, IMO, for silent hidden assumptions in httpd.
 
 How about;
 
 Index: configure.in
 ===
 --- configure.in  (revision 293579)
 +++ configure.in  (working copy)
 @@ -32,6 +32,9 @@
  dnl export expanded and relative configure argument variables
  APACHE_EXPORT_ARGUMENTS
  
 +dnl confirm that a void pointer is large enough to store a long integer
 +APACHE_CHECK_VOID_PTR_LEN
 +
  dnl Save user-defined environment settings for later restoration
  dnl
  APR_SAVE_THE_ENVIRONMENT(CPPFLAGS)
 Index: acinclude.m4
 ===
 --- acinclude.m4  (revision 293579)
 +++ acinclude.m4  (working copy)
 @@ -570,3 +570,24 @@
  undefine([ap_ckver_cvar])
  undefine([ap_ckver_name])
  ])
 +
 +dnl
 +dnl APACHE_CHECK_VOID_PTR_LEN
 +dnl
 +dnl Checks if the size of a void pointer is at least as big as a long 
 +dnl integer type.
 +dnl
 +AC_DEFUN([APACHE_CHECK_VOID_PTR_LEN], [
 +
 +AC_CACHE_CHECK([for void pointer length], [ap_void_ptr_lt_long],
 +[AC_TRY_RUN([
 +int main(void)
 +{
 +return sizeof(void *)  sizeof(long); 
 +}], [ap_void_ptr_lt_long=yes], [ap_void_ptr_lt_long=no], 
 +[ap_void_ptr_lt_long=no])])
 +
 +if test $ap_void_ptr_lt_long = no; then
 +AC_MSG_ERROR([Size of void * is less than size of int])
 +fi
 +])
 -- 
 Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
 


-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   If you can dodge a wrench, you can dodge a ball.


Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

2005-10-04 Thread Colm MacCarthaigh
On Tue, Oct 04, 2005 at 08:11:46AM -0400, Jim Jagielski wrote:
 I do think the error message is wrong though :)

*bangs head off of table* 

Thanks :)

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]


Re: [PATCH] Re: Pluggable mod_log_config

2005-10-04 Thread Brian Akins

Colm MacCarthaigh wrote:


But in order to save overhead, this would require some intelligence, it
would not make much sense for the pluggable logger to re-parse this
string everytime, to figure out what it should be doing. And where does
it get its database, username and host information from?  Do we require
per-provider directives? Do we hack the format more to get them in
somehow?



In this case, from my patches:

LogFormat INSERT INTO foo VALUES ('%h', '%l'); foo-sql

CustomLog mysql://user:[EMAIL PROTECTED]/database foo-sql

and the mysql module would get the arrays of strings and lengths.  at 
init time, it would have prepared the format sql.  At log time, it would 
bind and execute.




And after all of this, what if any, are the compelling reasons to
implement this in httpd at all? Why can't all of this be moved into
piped loggers? 



Try pipe loggers with 60 or so virtual hosts.  It doesn't scale well, as 
we open a pipe for each virtual that defines custom log.



Why can't they just parse the logs as they get them and

do whatever whoever wants with it after that? After all, many people use
logger to log to syslog. The whole mod_log_spread architecture with
two netcat commands, and that can even be done with privilege
seperation, which an in-httpd module never could.



What I did was something similar, although without pipes and not 
portably (at least not to windows).  I wrote a very simple log module 
that logs to Unix domain sockets:


CustomLog /logs/site.sock common

And the log server does whatever with it -- it's pluggable.  I have 
a spread and an asynchronous disk one.



On ftp.heanet.ie, we've been using piped loggers forever, our logs are
damn busy, and we've seen our share of crashes, and these issues just
never arise for us.


define damn busy.  We may have a different scale of busy.  I'v found 
pipe logs with lots of virtuals to be less than spectacular.  In Apache 
logging will be the best performance, but it lacks some flexibility. 
That's why I wrote my domain socket stuff.  However, I think the patches 
I've submitted allow for in-Apache logging to be very flexible with no 
additional overhead of pipes.


--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies


Re: [PATCH] Re: Pluggable mod_log_config

2005-10-04 Thread Jeffrey Burgoyne
I run a system with about 250 virtual hosts averaging around 100 hits per
second with little problem. Rather than having a piped log for each
virtual host though, we only use one piped log for every virtual host, and
add in the host name into the log. Your logging program then has ot have
the smarts to figure out which virtual host the log is for, but that is
relatively easy and inexpensive.

Jeffrey Burgoyne

Chief Technology Architect
KCSI Keenuh Consulting Services Inc
[EMAIL PROTECTED]

On Tue, 4 Oct 2005, Brian Akins wrote:

 Colm MacCarthaigh wrote:


 Try pipe loggers with 60 or so virtual hosts.  It doesn't scale well, as
 we open a pipe for each virtual that defines custom log.





Re: [PATCH] Re: Pluggable mod_log_config

2005-10-04 Thread Brian Akins

Jeffrey Burgoyne wrote:

I run a system with about 250 virtual hosts averaging around 100 hits per
second with little problem.


100 hits per second is not that much relative to what we do.  We may be 
more of an extreme case, though.



 Rather than having a piped log for each

virtual host though, we only use one piped log for every virtual host, and
add in the host name into the log. Your logging program then has ot have
the smarts to figure out which virtual host the log is for, but that is
relatively easy and inexpensive.



Yes, but Apache already knows what virtual host a log line, so it seems 
redundant to do it again in the external program.


I would agree, for 90+% of users, piped logs are probably fine.  But, 
for the rest of us, it seems like we ought to be able to come up with 
some standard way of doing high-performance flexible logging.  If 
nothing else but to have more eyes on the code and techniques.




--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies


Re: [PATCH] Re: Pluggable mod_log_config

2005-10-04 Thread Colm MacCarthaigh
On Tue, Oct 04, 2005 at 08:37:08AM -0400, Brian Akins wrote:
 In this case, from my patches:
 
 LogFormat INSERT INTO foo VALUES ('%h', '%l'); foo-sql
 
 CustomLog mysql://user:[EMAIL PROTECTED]/database foo-sql
 
 and the mysql module would get the arrays of strings and lengths.  at 
 init time, it would have prepared the format sql.  At log time, it would 
 bind and execute.

No, I don't think it's so simple at all. Although it would have to parse
the SQL at init time (how does LogFormat know to ask the MySQL provider
to do this?) that can't be simply untied from the actions at log-time;
in the case of SQL for example you have to be very pedantic about the
escaping, so that a request for /foo\'; nasty sql; doesn't kill us. 

And we still need additional per-provider directives to provide the
database, hostname, username and so on information. If such directives
are required anyway, what effort are we saving? Why not just replace
mod_log_config rather than plug into it. 

 And after all of this, what if any, are the compelling reasons to
 implement this in httpd at all? Why can't all of this be moved into
 piped loggers? 
 
 Try pipe loggers with 60 or so virtual hosts.  It doesn't scale well, as 
 we open a pipe for each virtual that defines custom log.

Ahh, that's a different problem; it's trivial to have a piped logger do
the splitting (it's what we do). 

Though if all of this text parsing is getting expensive, I wonder would
anyone be interested in a protocol for binary logging from httpd?

 CustomLog /logs/site.sock common

You re-implemented syslog :-) I've done the same myself for our hosting
service, we use syslog-ng to do all sorts of weird things with the info. 

 define damn busy. 

We frequently see over 4000 requests per second being logged. Almost any
time there's a major security update for fedora really. 

 We may have a different scale of busy.  I'v found pipe logs with
 lots of virtuals to be less than spectacular.  In Apache logging will
 be the best performance, but it lacks some flexibility.  That's why I
 wrote my domain socket stuff.  However, I think the patches I've
 submitted allow for in-Apache logging to be very flexible with no
 additional overhead of pipes.

Pipes themselves have little overhead, it's basically shared memory with
a standard IO interface and automatic mutexing. The processes on the
other side certainly need not be cumbersome - and I really like that you
can run a pipe-logger as a different uid, in a chroot and so on, it's a
nice place to sandbox all of that icky SQL parsing and that kind of
thing.

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]


Re: [PATCH] Re: Pluggable mod_log_config

2005-10-04 Thread Ondrej Sury
On Mon, 2005-10-03 at 23:28 +0100, Colm MacCarthaigh wrote:
 And after all of this, what if any, are the compelling reasons to
 implement this in httpd at all? Why can't all of this be moved into
 piped loggers? Why can't they just parse the logs as they get them and
 do whatever whoever wants with it after that? After all, many people use
 logger to log to syslog. The whole mod_log_spread architecture with
 two netcat commands, and that can even be done with privilege
 seperation, which an in-httpd module never could.

No, it cannot be implemented with two netcat commands (just tried it).

Purpose of mod_log_spread is to unify several balanced frontends logging
to one common log server.  With netcat you can only do one to one.

Second problem that restart of backend server listener cannot cause
piped loggers to fail (and thus force you to restart all frontends).
Which is f.e. true for netcat.

Ondrej.
-- 
Ondrej Sury [EMAIL PROTECTED]


signature.asc
Description: This is a digitally signed message part


Re: [PATCH] Re: Pluggable mod_log_config

2005-10-04 Thread Colm MacCarthaigh
On Tue, Oct 04, 2005 at 02:59:40PM +0200, Ondrej Sury wrote:
 No, it cannot be implemented with two netcat commands (just tried it).

Sure, it can, use my Multicast Netcat;

http://people.heanet.ie/~colmmacc/mnc/

(version 1.3 is experimental and not working right now, avoid that, and
I'm about the public the APR version I've been working on for a while). 

 Purpose of mod_log_spread is to unify several balanced frontends logging
 to one common log server.  With netcat you can only do one to one.

On the clients;

CustomLog |mnc group-id

(you can use ordinary netcat for that part if you want). 

On the collector;

mnc -l group-id  logfile

 Second problem that restart of backend server listener cannot cause
 piped loggers to fail (and thus force you to restart all frontends).
 Which is f.e. true for netcat.

I know about the restart problem, but I don't know what you mean about
this forcing you to restart frontends, or how it affects the
architecture.  It wouldn't matter using the above commands.

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]


Re: [PATCH] Re: Pluggable mod_log_config

2005-10-04 Thread Ondrej Sury
On Tue, 2005-10-04 at 13:55 +0100, Colm MacCarthaigh wrote:
 On Tue, Oct 04, 2005 at 08:37:08AM -0400, Brian Akins wrote:
  In this case, from my patches:
  
  LogFormat INSERT INTO foo VALUES ('%h', '%l'); foo-sql
  
  CustomLog mysql://user:[EMAIL PROTECTED]/database foo-sql
  
  and the mysql module would get the arrays of strings and lengths.  at 
  init time, it would have prepared the format sql.  At log time, it would 
  bind and execute.
 
 No, I don't think it's so simple at all. Although it would have to parse
 the SQL at init time (how does LogFormat know to ask the MySQL provider
 to do this?) that can't be simply untied from the actions at log-time;
 in the case of SQL for example you have to be very pedantic about the
 escaping, so that a request for /foo\'; nasty sql; doesn't kill us. 

Well, I think you can do something like this:

LogFormat %h %l foo
MySQLFormat INSERT INTO bar SET hostname = '$1', logname = '$2' bar

MySQLHost mysql0 localhost

CustomLog mysql://mysql0/bar foo

Then ap_mysql_log_writer would do it's own processing of strs/strl
combo.

 And we still need additional per-provider directives to provide the
 database, hostname, username and so on information.

That's true.

 If such directives are required anyway, what effort are we saving?
 Why not just replace mod_log_config rather than plug into it. 

Because there is a lot more stuff which needs to be reimplemented in
mod_log_config and which can be reused in other modules.

Ondrej.
-- 
Ondrej Sury [EMAIL PROTECTED]


signature.asc
Description: This is a digitally signed message part


Re: [PATCH] Re: Pluggable mod_log_config

2005-10-04 Thread Ondrej Sury
On Tue, 2005-10-04 at 14:06 +0100, Colm MacCarthaigh wrote:
 On Tue, Oct 04, 2005 at 02:59:40PM +0200, Ondrej Sury wrote:
  No, it cannot be implemented with two netcat commands (just tried it).
 
 Sure, it can, use my Multicast Netcat;

Didn't know that anything like that exists.  My comment about restarting
frontends was based on fact, that I didn't realized that apache2
restarts piped logger automaticaly.

Ondrej.
-- 
Ondrej Sury [EMAIL PROTECTED]


signature.asc
Description: This is a digitally signed message part


Re: [PATCH] Re: Pluggable mod_log_config

2005-10-04 Thread Brian Akins

Colm MacCarthaigh wrote:

On Tue, Oct 04, 2005 at 08:37:08AM -0400, Brian Akins wrote:


No, I don't think it's so simple at all. Although it would have to parse
the SQL at init time (how does LogFormat know to ask the MySQL provider
to do this?) 


LogFormat doesn't, but the init of the provider could.


that can't be simply untied from the actions at log-time;

in the case of SQL for example you have to be very pedantic about the
escaping, so that a request for /foo\'; nasty sql; doesn't kill us. 



That what the binding ensures.



And we still need additional per-provider directives to provide the
database, hostname, username and so on information. 



No, the init of the provider worries about the parameters.  In fact, the 
mysql modules may just have seperate directives to set this:


SqlLoggerUsername User
SqlLoggerPassword password



If such directives

are required anyway, what effort are we saving? Why not just replace
mod_log_config rather than plug into it. 


Because mod_log_config handles a bunch of stuff for us.  May be a better 
solution would be to use the standard log_config way of replacing init 
and writer and replace them with a pluggable one.




Though if all of this text parsing is getting expensive, I wonder would
anyone be interested in a protocol for binary logging from httpd?


Hmmm. interesting. Especially for things like spread, this would make 
alot of sense.





CustomLog /logs/site.sock common



You re-implemented syslog :-) I've done the same myself for our hosting
service, we use syslog-ng to do all sorts of weird things with the info. 


but syslog can't, on it's own, determine between virtuals (same problems 
as piped loggers).  I may have to look at syslog-ng.  Can you maybe 
off-list share some of your techniques?





define damn busy. 



We frequently see over 4000 requests per second being logged. Almost any
time there's a major security update for fedora really. 


Okay, you qualify :)




Pipes themselves have little overhead, it's basically shared memory with
a standard IO interface and automatic mutexing. The processes on the
other side certainly need not be cumbersome - and I really like that you
can run a pipe-logger as a different uid, in a chroot and so on, it's a
nice place to sandbox all of that icky SQL parsing and that kind of
thing.


True.  The test parsing just seems so icky to determine the virtual 
host since Apache already knows that.



--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies


Re: [PATCH] Re: Pluggable mod_log_config

2005-10-04 Thread Colm MacCarthaigh
On Tue, Oct 04, 2005 at 09:15:32AM -0400, Brian Akins wrote:
 Why not just replace mod_log_config rather than plug into it. 
 
 Because mod_log_config handles a bunch of stuff for us.  May be a
 better solution would be to use the standard log_config way of
 replacing init and writer and replace them with a pluggable one.

O.k., well if people really want to do this, using schemes and providers
is an elegant way, we would just have to signpost the road to handling
the formats. 

 Though if all of this text parsing is getting expensive, I wonder would
 anyone be interested in a protocol for binary logging from httpd?
 
 Hmmm. interesting. Especially for things like spread, this would make 
 alot of sense.

Whether it represents even more encoding or not is hard to tell though
:)

 CustomLog /logs/site.sock common
 
 You re-implemented syslog :-) I've done the same myself for our hosting
 service, we use syslog-ng to do all sorts of weird things with the info. 
 
 but syslog can't, on it's own, determine between virtuals (same problems 
 as piped loggers).  I may have to look at syslog-ng.  Can you maybe 
 off-list share some of your techniques?

Once you see syslog-ng's way of doing things, it becomes fairly
self-explanatory; We use its regular expression engine for logging
vhosts seperately and so on, but we also have a centralised log file
which logs everything we consider important (which is basically errors
we have not explicitly filtered out) accross all hosts. 

We also use its mysql output features to put this later class of logging
information in a transient database, so that we can integrate and
monitor the important, potentially critical, events within our NMS more
easily.

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]


mod_smtpd: handling rctp addresses

2005-10-04 Thread Brian J. France
I would like to propose a change on how rcpt to addresses are  
validated and messages are handled (expanded) and how queue modules  
know it needs to process this message.


I would like to change rcpt_to in smtpd_trans_rec to apr_table_t.  The  
key for the table will be the address sent to rcpt to command and the  
value will be a pointer to a apr_array_header_t list of pointers to the  
following structure:


typedef struct smtpd_rctp_rec {
apr_pool_t *p;
const char *mode;
int validated;
const char *address;
apr_table_t info;
const char *expanded;
}

mode: what queue handler should handle this message.
Example would be local, relay, relay_to, forward, postfix.

validated: if the rctp has been validated and ready to process

address: what address to send to

info: extra data is stored that the queue handler might needed

expanded: log of how the address was expanded

mod_smtpd would create a hook for expanding rctp address which other  
modules will use to enable different ways to validate rcpt address and  
setting what queue module should process a message.  It would take a  
smtpd_rctp_rec * and a apr_array_header_t * of things to add to add and  
return decline, ok or delete.


HANDLER_DECLARE(rcpt) will create a smtpd_rctp_rec and run the  
expanding hook and handle the return value and also the things to be  
add.


Here are a few example of how I think it would work. This will be using  
some modules that do not exists yet and some that do:


mod_smtpd_user_pwd - handle validating local users and setting info on  
them


mod_smtpd_alias - enables aliases (forwarding email)

mod_smtpd_queue_local - handles the local mode
Handles deliver of mail to local users

mod_smtpd_queue_relay - handles the relay and forward modes
This module would hook the expanding hook using
APR_HOOK_LAST since it will allow forwarding and
relaying for authenticated user, but wants to make
sure all other modules have a chance to change the
address if needed.

mod_smtpd_queue_smtp - handles the relay_to mode
Sends all mail to another server.
This module would hook the expanding hook using
APR_HOOK_FIRST since it want so to set every
smtpd_rctp_rec mode to replay_to and doesn't
care what other modules have to say.

Examples:

 



[EMAIL PROTECTED] - user

expand hook:
{ mode: NULL
  address: [EMAIL PROTECTED]
  validate: 0
  info: { }
  expanded: NULL
}

mod_smtpd_user_pwd handles request and returns ok.
It changes the struct to look like this:

{ mode: local
  address: [EMAIL PROTECTED]
  info: { mail_box - username }
  validate: 1
  expanded: [EMAIL PROTECTED] - local user username
}

mod_smtpd_queue_local would handle queuing

 



auth'ed user relaying mail to [EMAIL PROTECTED]

expand hook:
{ mode: NULL
  address: [EMAIL PROTECTED]
  validate: 0
  info: { }
  expanded: NULL
}

mod_smtpd_auth will handle this and returns ok.
It changes the struct to look like this:

{ mode: relay
  address: [EMAIL PROTECTED]
  info: { }
  validate: 1
  expanded: authenticated user user relaying [EMAIL PROTECTED]
}

mod_smtpd_queue_relay would handle queuing

 



[EMAIL PROTECTED] - [EMAIL PROTECTED]

expand hook:
{ mode: NULL
  address: [EMAIL PROTECTED]
  info: { }
  validate: 0
  expanded: NULL
}

mod_smtpd_alias would handle the request and return delete.
It adds a new entry to the list that looks like this:

{ mode: forward
  address: [EMAIL PROTECTED]
  info: { }
  validate: 0
  expanded : [EMAIL PROTECTED] - [EMAIL PROTECTED]
}

expand hook call on the new entry:

{ mode: forward
  address: [EMAIL PROTECTED]
  info: { }
  validate: 0
  expanded : [EMAIL PROTECTED] - [EMAIL PROTECTED]
}

mod_smtpd_forward would handle the request and return ok.
It would set the struct like this:

{ mode: forward
  address: [EMAIL PROTECTED]
  info: { }
  validate: 1
  expanded : [EMAIL PROTECTED] - [EMAIL PROTECTED]
}

mod_smtpd_queue_relay would handle queuing

 



[EMAIL PROTECTED]
- [EMAIL PROTECTED]
- [EMAIL 

Re: mod_smtpd: handling rctp addresses

2005-10-04 Thread Nick Kew
On Tuesday 04 October 2005 15:23, Brian J. France wrote:
 I would like to propose a change on how rcpt to addresses are
 validated and messages are handled (expanded) and how queue modules
 know it needs to process this message.

How long have you been following mod_smtpd discussion?  The topic of
rcpt to has come up before, with reference to dealing with multiple
recipients.  In my view, we need to be able to treat recipients as
mutually independent - e.g. local or relayed, filtered or not against
spam, viruses, etc.  That implies either running each recipient as a
separate (sub)request or reinventing quite a lot of wheel.

 I would like to change rcpt_to in smtpd_trans_rec to apr_table_t.  The
 key for the table will be the address sent to rcpt to command and the
 value will be a pointer to a apr_array_header_t list of pointers to the
 following structure:

 typedef struct smtpd_rctp_rec {
   apr_pool_t *p;
   const char *mode;
   int validated;
   const char *address;
   apr_table_t info;
   const char *expanded;
 }

Yow!  That is a new wheel (though a table isn't what you want -
an array with the name as an additional data member would
surely do the job).

Your proposal goes some way towards decoupling different
recipients, but I'm not really convinced.  The basic issue it
doesn't really address is dynamic configuration.  For example,
I want a whitelist or blacklist that may remove all complex
filtering (like mod_spamd) from the chain.  That works fine
at the request level, but in your model it would appear to
imply moving the input filter chain to the smtp_rcpt_rec.

 mode: what queue handler should handle this message.
   Example would be local, relay, relay_to, forward, postfix.

 validated: if the rctp has been validated and ready to process

 address: what address to send to

 info: extra data is stored that the queue handler might needed

 expanded: log of how the address was expanded

This does look a little like a request, but is not really sufficient.

 Examples:

This is useful thinking-it-through: what you're proposing looks a little
like a subrequest, and might ultimately make more sense.  But it'll
need at least that move of the input filter chain!

-- 
Nick Kew


Flexible Handlers

2005-10-04 Thread Nick Kew
The current implementation of handlers is a little bizarre, with each
handler both having to register itself and check its own name.
I'm not sure where that comes from, but I can't help thinking the
ap_provider API might be a more rational way to declare handlers.

We have one frequently-requested feature that IMO we should support:
cascaded/fallback handlers.  The request for this typically says if a
document isn't there, then use my handler as backup.  ErrorDocument
offers a limited workaround, but
  (a) This shouldn't have to be an error path
  (b) This should be able to cascade

So, how about AddHandler  friends taking a *list* of handlers as argument
(let's call the new version AddHandlerByExt)?  Then we can run each handler
in turn until one returns a value other than DECLINED.   That would be coupled 
with the default handler being assigned a name and being configurable to 
appear anywhere in the list, so fall back to my CGI if there's no file of 
that name becomes AddHandlerByExt .html default cgi-script.
ErrorDocuments are invoked when we fall off the end of the list.

For a slightly bigger change, we can use ap_provider on top of that to
declare and reference handlers.

-- 
Nick Kew


Re: Flexible Handlers

2005-10-04 Thread Nick Kew
On Tuesday 04 October 2005 16:25, Brian Akins wrote:
 Nick Kew wrote:
  The current implementation of handlers is a little bizarre, with each
  handler both having to register itself and check its own name.

 Some people, myself included, have handlers that run even when they
 aren't set, ithey never check r-handler and always returns DECLINED.
   Yes, I know this is bad and should be done in fixups probably, but I'm
 sure this may break some things.

Sounds contrary to the API.  Flexible configuration would presumably enable
you to insert them legitimately?

 That being said.  I have always found it strange that everything has to
 check r-handler.

Exactly.

 What about cases like this?

 Location /stuff
   SetHandler stuff-handler
 /Location

That's easy: no change as far as stuff-handler is concerned.  SetHandler
gets to take extra args, but only if we want it (i.e. not in cases like that).


  For a slightly bigger change, we can use ap_provider on top of that to
  declare and reference handlers.

 This is interesting. But what if we had a handler chain much like a
 filter chain?  First one to return OK stops the chain.

Indeed.  But that's first one NOT to return DECLINED.

 Chains could be 
 built up as one shots (like SetHandler) or more elaborate like
 mod_filter does with filters.

Even SetHandler could take more than one argument.  In fact, that
should serve to make SetHandler compatible with Directory Indexes -
which people sometimes want in a cgi-bin type of situation.

 the default_handler in core could always 
 be added to the end.

I'd prefer not.  It can be added anywhere, but needn't be implied except
as a default (no explicit handler at all in effect).

-- 
Nick Kew


Re: Flexible Handlers

2005-10-04 Thread Brian Akins

Nick Kew wrote:


Indeed.  But that's first one NOT to return DECLINED.


That's what I meant to say.



I'd prefer not.  It can be added anywhere, but needn't be implied except
as a default (no explicit handler at all in effect).



sure. agreed.


--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies


Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules:

2005-10-04 Thread Jim Jagielski
Brian Candler wrote:
 
 Hmm. Given the example
 
 pid = (pid_t)((long)apr_hash_get(script_hash, cgid_req.conn_id, 
 sizeof(cgid_req.conn_id)));
 
 then it's no good declaring variable 'pid' to be of type apr_intptr_t rather
 than pid_t, because somewhere down the line it will be passed to a function
 which expects a pid_t argument, and you'll get a warning there instead. Are
 you saying it should be written as
 

The above line just confuses me, but I haven't taken the time
to try to understand the rationale for why it's written the way it is.

-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   If you can dodge a wrench, you can dodge a ball.


Re: mod_smtpd: handling rctp addresses

2005-10-04 Thread rian
As far as an abstracted mechanism for informing queue modules what
recipients they are queue I like this approach, but before any major
changes are made to mod_smtpd I think this and other problems should be
concretely resolved.

Originally the idea was to treat each transaction as an atomic entity,
for simplicity and strictness to how the SMTP protocol. Also this is how
qpsmtpd works, and seems to work fine.

Now the idea is handling each recipient differently and subrequesting by
recipient. Either way, I think it's important to first look at how
mod_smtpd will be used and then create it's plugin architecture around
that.

For example it should be possible to run one queue module on a SMTP
server, and then arbitrarily add another as time goes on.

For instance you have a local delivery server, but then later you want
to forward mail for authenticated users. In the conf you should first
you should have:

---

LoadModule smtpd_module *.so
LoadModule smtpd_queue_local *.so

# Conf directive handled in mod_smtpd_queue_local
SmtpdQueueLocalRcpts [A-Za-z0-9]+([EMAIL PROTECTED])? 

---

Later you shoudl just be able to add:

---

LoadModule smtpd_queue_relay *.so

---

And the relay module would queue every other address not handled by
relay, assuming the connection was authenticated.

This implies that if all the queue modules are called for each of the
recipients, even after they are handled, there should be some metadata
coming with the recipient that says it has already been handled so relay
knows not to relay a message that has already been handled.

Or that once a address is handled, the hooks are stopped (RUN_ONCE) and
the next address is called on the queue hooks. Of course relay should be
a RUN_LAST hook.

What's more powerful? I think probably the first, because maybe a server
admin will want a recipient to be queued by two modules. This means
there should be some metadata with each recipient.

But wait! Earlier I gace an example about how a module should know what
recipients it's dealing with using a module specific directive. But what
if we did this:

---

SmtpdQueueHandler regex:[EMAIL PROTECTED] regex:[EMAIL PROTECTED], postfix relay
local
SmtpdQueueHandler mysql:/rcpttable.mysqlCONF, relay

-

These directives impose a recipient policy on all of mod_smtpd's
submodules. For something like this to work, it would require that each
rcpt is subrequested and has metadata associated with it (in this case,
queue handler name). I feel something like this is a lot better because
it minimizes configuration directives needed to be learned by server
admins and encourages code reuse (at least in recipient lists). In this
scheme mod_smtpd is responsible for assigning queue handlers to
recipients and all the queue handlers have to do is check whether or not
they are handlers of the recipient object.

So more and more Brian's proposal seems to make a lot of sense for
mod_smtpd, except maybe adding a 'queued' member.

I think the ultimate goal for mod_smtpd is as much code reuse as
possible so your proposal seems to fit in with that goal.

After lots of thought it seems the only criteria queueing will be based
on is recipients. Queuing based on other message characteristics (like
subject, who the message is from (ip and mail from)) or headers seem
more and more rare.
-rian

On Tue, 2005-10-04 at 09:23 -0500, Brian J. France wrote:
 I would like to propose a change on how rcpt to addresses are  
 validated and messages are handled (expanded) and how queue modules  
 know it needs to process this message.
 
 I would like to change rcpt_to in smtpd_trans_rec to apr_table_t.  The  
 key for the table will be the address sent to rcpt to command and the  
 value will be a pointer to a apr_array_header_t list of pointers to the  
 following structure:
 
 typedef struct smtpd_rctp_rec {
   apr_pool_t *p;
   const char *mode;
   int validated;
   const char *address;
   apr_table_t info;
   const char *expanded;
 }
 
 mode: what queue handler should handle this message.
   Example would be local, relay, relay_to, forward, postfix.
 
 validated: if the rctp has been validated and ready to process
 
 address: what address to send to
 
 info: extra data is stored that the queue handler might needed
 
 expanded: log of how the address was expanded
 
 mod_smtpd would create a hook for expanding rctp address which other  
 modules will use to enable different ways to validate rcpt address and  
 setting what queue module should process a message.  It would take a  
 smtpd_rctp_rec * and a apr_array_header_t * of things to add to add and  
 return decline, ok or delete.
 
 HANDLER_DECLARE(rcpt) will create a smtpd_rctp_rec and run the  
 expanding hook and handle the return value and also the things to be  
 add.
 
 Here are a few example of how I think it would work. This will be using  
 some modules that do not exists yet and some that do:
 
 

Re: svn commit: r294809 - in /httpd/httpd/trunk: acinclude.m4 configure.in

2005-10-04 Thread Rüdiger Plüm


On 10/04/2005 06:18 PM, [EMAIL PROTECTED] wrote:
 Author: colm
 Date: Tue Oct  4 09:18:24 2005
 New Revision: 294809

[..cut..]

 
 +
 +AC_CACHE_CHECK([for void pointer length], [ap_void_ptr_lt_long],
 +[AC_TRY_RUN([
 +int main(void)
 +{
 +return sizeof(void *)  sizeof(long); 
 +}], [ap_void_ptr_lt_long=yes], [ap_void_ptr_lt_long=no], 
 +[ap_void_ptr_lt_long=no])])
 +
 +if test $ap_void_ptr_lt_long = no; then
 +AC_MSG_ERROR([Size of void * is less than size of long])
 +fi
 +])
 

[..cut..]

Sorry for being confused by this, but I read your variable name
ap_void_ptr_lt_long as (size of) void ptr is less than (size of) long,
but the assignment of yes and no is exactly the opposite. What about

AC_CACHE_CHECK([for void pointer length], [ap_void_ptr_lt_long],
[AC_TRY_RUN([
int main(void)
{
return sizeof(void *)  sizeof(long);
}], [ap_void_ptr_lt_long=no], [ap_void_ptr_lt_long=yes],
[ap_void_ptr_lt_long=yes])])

if test $ap_void_ptr_lt_long = yes; then
AC_MSG_ERROR([Size of void * is less than size of long])
fi
])

No functional change, but seems clearer to me.

Regards

Rüdiger





Re: svn commit: r294809 - in /httpd/httpd/trunk: acinclude.m4 configure.in

2005-10-04 Thread William A. Rowe, Jr.

Rüdiger Plüm wrote:


On 10/04/2005 06:18 PM, [EMAIL PROTECTED] wrote:


Author: colm
Date: Tue Oct  4 09:18:24 2005
New Revision: 294809




+AC_CACHE_CHECK([for void pointer length], [ap_void_ptr_lt_long],


Sorry for being confused by this, but I read your variable name
ap_void_ptr_lt_long as (size of) void ptr is less than (size of) long,
but the assignment of yes and no is exactly the opposite. What about

AC_CACHE_CHECK([for void pointer length], [ap_void_ptr_lt_long],


Actually I'm also confused, not by the fact that we resolve this test,
but exactly what it nets us.

Much more useful would be, as I mentioned before, a test with results
if sizeof int == sizeof void*  then typedef ap_intptr_t int,
if sizeof long == sizeof void* then typedef ap_intptr_t long,
and we could have unsigned (ap_uintptr_t) of the same if anyone
supposes that's useful.





Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules:

2005-10-04 Thread William A. Rowe, Jr.

Brian Candler wrote:


Hmm. Given the example

pid = (pid_t)((long)apr_hash_get(script_hash, cgid_req.conn_id, 
sizeof(cgid_req.conn_id)));

then it's no good declaring variable 'pid' to be of type apr_intptr_t rather
than pid_t, because somewhere down the line it will be passed to a function
which expects a pid_t argument, and you'll get a warning there instead.


pid = (pid_t)apr_hash_get(script_hash, cgid_req.conn_id,
  sizeof(cgid_req.conn_id));

would be correct in -this- case.  We have to make a small concession
to the platform architects, that there's no sense in more pid's than
you could count with a void*, so no matter if pid_t is an int or long
(or a handle) this simply works.  Now if the line above produces our
compile warning, of truncation, there's no issue I suppose with

 pid = (pid_t)((ap_ptrint_t)apr_hash_get(script_hash,
cgid_req.conn_id,
sizeof(cgid_req.conn_id)));

or perhaps we simplify, with AP_INTPTR(n) and AP_PTRINT(p) macros?

 pid = (pid_t)AP_PTRINT(apr_hash_get(script_hash,
cgid_req.conn_id,
sizeof(cgid_req.conn_id));

e.g.
#define AP_INTPTR(n) ((void*)((ap_ptrint_t)(n)))
#define AP_PTRINT(n) ((ap_ptrint_t)(n))

I suppose I understand the truncation warning but...


Now, Apache has lots of cases where (void *)'s are cast to integral types
(possibly smaller than a void *) and back again. Since this is intentional,
I think the real solution is to turn off the compiler warning for this case,
not to bastardise the code with lots of double-casts. The more unnecessary
casts you put into your code, the more likely you are to make type errors
which the compiler can't check.



IMO, if gcc can't disable this warning, then we need to file a bug report.
Or else just pipe gcc stderr through

egrep -v warning: cast (to|from) pointer (to|from) integer of different 
size

and forget about it.


Now that's not a healthy situation, -1 on the gcc pipe :)


P.S. The reverse case is sillier, given that the value is moving to a
*larger* type and therefore no data loss can occur:

short a;
long b = a;  // (7) no warning

short a;
void *b = a; // (8) warns pointer from integer without a cast (ok)
void *c = (void *)a; // (9) warns cast to pointer from integer of different 
size
void *d = (void *)(long)a;  // (10) even more stupid!!!

Note that having a 'large enough' integral type in case (10) is not good
enough. We need to cast 'a' to an integral type which is *exactly* the same
size as a void *, to silence this particular warning.


I agree with your assertion that the code example above IS a gcc bug.
This is expected in C++, but the waning (9) above is completely bogus,
as you illustrate with line (7) above.

What's your though on AP_PTRINT(n) / AP_INTPTR(p)?

Bill


Re: svn commit: r294902 - /httpd/httpd/trunk/acinclude.m4

2005-10-04 Thread Jim Jagielski
[EMAIL PROTECTED] wrote:
 
 Author: colm
 Date: Tue Oct  4 12:25:51 2005
 New Revision: 294902
 
 URL: http://svn.apache.org/viewcvs?rev=294902view=rev
 Log:
 Invert the yes and no values for $ap_void_ptr_lt_long, which
 as Rudiger points out; are exactly wrong.
 
 -if test $ap_void_ptr_lt_long = no; then
 +if test $ap_void_ptr_lt_long = yes; then
  AC_MSG_ERROR([Size of void * is less than size of long])
 

Don't shoot me :)

The problem is whether or not a pointer will fit in a long.
Is the site of void* is smaller than a long, that's
OK (if it fits in an int and sizeof(int)sizeof(long)
that is perfectly OK).

Bill suggested figuring out what the smallest int that would
safely hold a void* and making that a typedef. And pointers
are always unsigned, iirc.

-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   If you can dodge a wrench, you can dodge a ball.


Re: svn commit: r294809 - in /httpd/httpd/trunk: acinclude.m4 configure.in

2005-10-04 Thread Colm MacCarthaigh
On Tue, Oct 04, 2005 at 02:16:16PM -0500, William A. Rowe, Jr. wrote:
 Actually I'm also confused, not by the fact that we resolve this test,
 but exactly what it nets us.

It makes sure that a void * container can store the largest of the types
we ocasionally place in such a container. It validates the assumption
made by code which is in httpd right now.

It should be taken out if/when the code making those assumptions are
removed. 

In the meantime; if someone somehow managed to wheel out a rusty copy of
TurboC for Dos, with the forever-brilliant near, far and huge
pointer-model, and then somehow again managed to get a configure script
to use this compiler, it would fail here (a 16-bit pointer would be less
than the 32-bit long) instead of probably about 20 lines further along
the configure script ;)

 Much more useful would be, as I mentioned before, a test with results
 if sizeof int == sizeof void*  then typedef ap_intptr_t int,
 if sizeof long == sizeof void* then typedef ap_intptr_t long,
 and we could have unsigned (ap_uintptr_t) of the same if anyone
 supposes that's useful.

This would be much better, though it's not just longs that we place in
these pointer types, chars and ints get placed in them too. It's just
all a lot more work, tracking down all of the instances, and adding new
conditional typedef's for each one.

Adreas' original patch just fixed the compiler warnings, these
assignments have been in quite a while, and there are more to be found, no
doubt. Does anyone have a tool which will

But it is an utterly tedious near-pointless task, so it's kind of hard to
be motivated to do it ;) 

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]


Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules:

2005-10-04 Thread Brian Candler
On Tue, Oct 04, 2005 at 12:12:14PM -0400, Jim Jagielski wrote:
 Brian Candler wrote:
  
  Hmm. Given the example
  
  pid = (pid_t)((long)apr_hash_get(script_hash, cgid_req.conn_id, 
  sizeof(cgid_req.conn_id)));
  
  then it's no good declaring variable 'pid' to be of type apr_intptr_t rather
  than pid_t, because somewhere down the line it will be passed to a function
  which expects a pid_t argument, and you'll get a warning there instead. Are
  you saying it should be written as
  
 
 The above line just confuses me, but I haven't taken the time
 to try to understand the rationale for why it's written the way it is.

That's what I was hoping the little example with shorts, longs and void *'s
would explain :-)

In the above code:

- apr_hash_get returns void *
- if you assign it directly to 'pid', you get a compiler warning saying
  assigning integer from pointer without a cast. That's fair enough.
- however, if you cast it to pid_t, you get a compiler warning saying
  cast from pointer to integer of different size, if pid_t is not
  actually the same size as a void *

So stupidly you have to cast it to an integer of the same size as a pointer,
*then* cast it to pid_t or whatever you wanted in the first place, to
silence the warnings.

I think this is the compiler grumbling about something which it shouldn't,
at least for Apache's way of working: the code explicitly casts void * to a
smaller integral type, because it was being used to hold a value of a
smaller integral type in the first place. IMO the cast is the programmer's
way of saying yes, I know this is a pointer, and yes I want you treat it as
a pid_t (or whatever). I guess the warning is for the benefit of people who
use an integral type for temporarily holding a pointer, rather than a
pointer type for temporarily holding an integer.

I believe the solution is to silence the compiler warnings, rather than
corrupt the code with superfluous casts. A union might be cleaner, in which
case you'd write something like

pid = apr_hash_get(...).num;

I would hope that most modern compilers are able to cope with passing and
returning unions by value (especially ones which fit within a machine word),
but maybe that doesn't apply to all the platforms Apache is supposed to
compile on.

Regards,

Brian.

// Proof of concept
#include stdio.h

typedef union {
  long long_v;
  void *void_p;
} ap_union;

ap_union foo(ap_union bar)
{
return bar;
}

int main(void)
{
ap_union x, y;
x.void_p = hello;

y = foo(x);
printf(%s\n, (char *)y.void_p);
printf(%08lx\n, y.long_v);
return 0;
}


Re: svn commit: r294902 - /httpd/httpd/trunk/acinclude.m4

2005-10-04 Thread William A. Rowe, Jr.

Jim Jagielski wrote:

Bill suggested figuring out what the smallest int that would
safely hold a void* and making that a typedef. And pointers
are always unsigned, iirc.


Yes, but there should be no issue with caching an int (signed)
in a void*.  They are unsigned in that comparing void *p to 0,
you will never find p  0.  Once it's cast to an int, of course,
it's sign can be restored.

Bill




Re: svn commit: r294902 - /httpd/httpd/trunk/acinclude.m4

2005-10-04 Thread Colm MacCarthaigh
On Tue, Oct 04, 2005 at 03:35:29PM -0400, Jim Jagielski wrote:
  -if test $ap_void_ptr_lt_long = no; then
  +if test $ap_void_ptr_lt_long = yes; then
   AC_MSG_ERROR([Size of void * is less than size of long])
  
 
 Don't shoot me :)
 
 The problem is whether or not a pointer will fit in a long.
 Is the site of void* is smaller than a long, that's
 OK (if it fits in an int and sizeof(int)sizeof(long)
 that is perfectly OK).

I don't see how that could be o.k.. when we have code like;

*context = (void *)((long)(value == 'T')); 

Where *context is a void * (context itself is defined void **).

I'll have to revert the compiler-warning-fixes entirely to remove that
assumption. I'm o.k. with that, we'll then have slightly different
assumptions I think, but they can be tested for too.

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]


Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules:

2005-10-04 Thread Jim Jagielski
Brian Candler wrote:
 
  The above line just confuses me, but I haven't taken the time
  to try to understand the rationale for why it's written the way it is.
 
 That's what I was hoping the little example with shorts, longs and void *'s
 would explain :-)
 

I understand *what* I just don't understand *why* :)

-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   If you can dodge a wrench, you can dodge a ball.


Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules:

2005-10-04 Thread Rüdiger Plüm


On 10/04/2005 09:32 PM, William A. Rowe, Jr. wrote:
 Brian Candler wrote:
 

[..cut..]

 
 P.S. The reverse case is sillier, given that the value is moving to a
 *larger* type and therefore no data loss can occur:

 short a;
 long b = a;  // (7) no warning

 short a;
 void *b = a; // (8) warns pointer from integer without a
 cast (ok)
 void *c = (void *)a; // (9) warns cast to pointer from integer of
 different size
 void *d = (void *)(long)a;  // (10) even more stupid!!!

 Note that having a 'large enough' integral type in case (10) is not good
 enough. We need to cast 'a' to an integral type which is *exactly* the
 same
 size as a void *, to silence this particular warning.
 
 
 I agree with your assertion that the code example above IS a gcc bug.
 This is expected in C++, but the waning (9) above is completely bogus,

I do not think so. While a does fit in c from the storage point of view
converting c to a different pointer type e.g. (char *) and dereferencing it
afterwards most likely leads to SIGBUS or SIGSEGV. So I think a warning is
justified here.

 as you illustrate with line (7) above.
 

This is different as no harm can be expected here like in (9).

[..cut..]

Regards

Rüdiger


Re: svn commit: r294809 - in /httpd/httpd/trunk: acinclude.m4 configure.in

2005-10-04 Thread Jim Jagielski
Colm MacCarthaigh wrote:
 
 It makes sure that a void * container can store the largest of the types
 we ocasionally place in such a container. It validates the assumption
 made by code which is in httpd right now.
 

I thought the containers were integral types, not void*. If the containers
themselves are void*, then the problem is reversed from what I thought,
but still a valid problem. 
-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   If you can dodge a wrench, you can dodge a ball.


Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules:

2005-10-04 Thread Jim Jagielski
=?UTF-8?B?UsO8ZGlnZXIgUGzDvG0=?= wrote:
 
 I do not think so. While a does fit in c from the storage point of view
 converting c to a different pointer type e.g. (char *) and dereferencing it
 afterwards most likely leads to SIGBUS or SIGSEGV. So I think a warning is
 justified here.
 

Yes, alignment rules.
-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   If you can dodge a wrench, you can dodge a ball.


Re: svn commit: r294809 - in /httpd/httpd/trunk: acinclude.m4 configure.in

2005-10-04 Thread Rüdiger Plüm


On 10/04/2005 10:06 PM, Jim Jagielski wrote:
 Colm MacCarthaigh wrote:
 

[..cut..]

 
 I thought the containers were integral types, not void*. If the containers
 themselves are void*, then the problem is reversed from what I thought,
 but still a valid problem. 

I think we have both problems in the code as we cast void* to integral types
and vice versa.

Regards

Rüdiger