Re: Urp(!) - indeterminant connection SO_NONBLOCK state?

[Joe Orton]

On Fri, Sep 30, 2005 at 01:19:54PM -0500, William Rowe wrote:
> Linux, Win32 and Netware begin their new socket's life in a blocking
> state, while Solaris and AIX begin their life in a non-blocking state.

The socket returned by apr_socket_accept() will be non-blocking on all 
Unix platforms in multi-listener configurations.

> You can observe the discrepancy (in 2.0.x, which happens to be what
> I'm testing) by simply loading mod_echo.so, and setting ProtocolEcho on
> in the default SSL context.
> 
> Linux, Win32 and Netware successfully handshake an SSL echo port, while
> Solaris and AIX read 'some bytes' and then die with a failed handshake.

I'm not sure I understand the issue.  mod_ssl should already propagate 
the APR_BLOCK_READ down so that blocking ap_get_brigade calls are made 
during the SSL_accept() as far as I can see?  Does this fix help?

http://svn.apache.org/viewcvs.cgi?rev=105768&view=rev

joe

[PATCH] PR36507: mod_proxy_balancer does not handle sticky sessions with tomcat correctly

[Ruediger Pluem]

Hi,

any objections / comments to the following patch for PR36507?
If not I would commit it to the trunk in 24 hours.

Regards

Rüdiger


Index: modules/proxy/mod_proxy_balancer.c
===
--- modules/proxy/mod_proxy_balancer.c  (Revision 292553)
+++ modules/proxy/mod_proxy_balancer.c  (Arbeitskopie)
@@ -193,7 +193,18 @@
 *route = get_path_param(r->pool, *url, balancer->sticky);
 if (!*route)
 *route = get_cookie_param(r, balancer->sticky);
-if (*route) {
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r->server,
+"proxy: BALANCER: Found value %s for "
+"stickysession %s", *route, balancer->sticky);
+/*
+ * If we found a value for sticksession, find the first '.' within.
+ * Everything after '.' (if present) is our route.
+ */
+if ((*route) && ((*route = strchr(*route, '.')) != NULL ))
+(*route)++;
+if ((*route) && (**route)) {
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r->server,
+  "proxy: BALANCER: Found route %s", *route);
 /* We have a route in path or in cookie
  * Find the worker that has this route defined.
  */

Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

[Jim Jagielski]

Just some lines that caught my eye:

>  
> -*context = (void *)(value == 'T');
> +*context = (void *)((long)(value == 'T'));
>  
> -int value = context != NULL;
> +long value = context != NULL;
>  apr_fileperms_t perms = resource->info->finfo.protection;
> -int old_value = (perms & APR_UEXECUTE) != 0;
> +long old_value = (perms & APR_UEXECUTE) != 0;
>  

Huh? Whenever I see conditionals cast to (long) I get
suspicious. 

>  
> -pid = (pid_t)apr_hash_get(script_hash, &cgid_req.conn_id, 
> sizeof(cgid_req.conn_id));
> +pid = (pid_t)((long)apr_hash_get(script_hash, &cgid_req.conn_id, 
> sizeof(cgid_req.conn_id)));

>  apr_hash_set(script_hash, key, sizeof(cgid_req.conn_id),
> - (void *)procnew->pid);
> + (void *)((long)procnew->pid));
>  }
>  {
> -int fd = (int)thefd;
> +int fd = (int)((long)thefd);
>  
>  else {
> -apr_pool_cleanup_register(r->pool, (void *)sd, close_unix_socket,
> -  apr_pool_cleanup_null);
> +apr_pool_cleanup_register(r->pool, (void *)((long)sd),
> +  close_unix_socket, 
> apr_pool_cleanup_null);
>  break; /* we got connected! */
>  }

Same when I see sequential casts... What's the rationale for them?
-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   "If you can dodge a wrench, you can dodge a ball."

Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

[Joe Orton]

On Mon, Oct 03, 2005 at 07:50:39AM -0400, Jim Jagielski wrote:
> Just some lines that caught my eye:
> 
> >  
> > -*context = (void *)(value == 'T');
> > +*context = (void *)((long)(value == 'T'));
> >  
> > -int value = context != NULL;
> > +long value = context != NULL;
> >  apr_fileperms_t perms = resource->info->finfo.protection;
> > -int old_value = (perms & APR_UEXECUTE) != 0;
> > +long old_value = (perms & APR_UEXECUTE) != 0;
> >  
> 
> Huh? Whenever I see conditionals cast to (long) I get
> suspicious. 

These are all cases where an integer is stored in a pointer; it's safe 
to assume that a long will fit in a pointer on all platforms which httpd 
runs on as a practical consideration, and using a cast to long rather 
than a cast to int will avoid compiler warnings on LP64 platforms.

joe

Re: Pluggable mod_log_config

[Brian Akins]

mod_log_config is "pluggable."   You can "replace" the normal logging 
functions quite easily.



Just do something like:


static apr_status_t my_logger_writer(request_rec *r,
   void *handle,
  const char **strs,
  int *strl,
  int nelts,
  apr_size_t len)

{
...

}

static void *my_logger_init(apr_pool_t *p, server_rec *s,
const char* path)
{
/*return a log "handle"*/
}

static void register_hooks(apr_pool_t * p)
{

set_writer_init = APR_RETRIEVE_OPTIONAL_FN(ap_log_set_writer_init);
set_writer = APR_RETRIEVE_OPTIONAL_FN(ap_log_set_writer);


set_writer_init(my_logger_init);
set_writer(my_logger_writer);

}



--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Release plans?

[Oden Eriksson]

Hi.

I don't know if this is the right forum for this question but I found no info 
elsewhere. I simply wonder when 2.1.x is going to be dubbed stable?

-- 
Regards // Oden Eriksson
Mandriva: http://www.mandriva.com
NUX: http://nux.se

Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

[Jim Jagielski]

Joe Orton wrote:
> 
> On Mon, Oct 03, 2005 at 07:50:39AM -0400, Jim Jagielski wrote:
> > Just some lines that caught my eye:
> > 
> > >  
> > > -*context = (void *)(value == 'T');
> > > +*context = (void *)((long)(value == 'T'));
> > >  
> > > -int value = context != NULL;
> > > +long value = context != NULL;
> > >  apr_fileperms_t perms = resource->info->finfo.protection;
> > > -int old_value = (perms & APR_UEXECUTE) != 0;
> > > +long old_value = (perms & APR_UEXECUTE) != 0;
> > >  
> > 
> > Huh? Whenever I see conditionals cast to (long) I get
> > suspicious. 
> 
> These are all cases where an integer is stored in a pointer; it's safe 
> to assume that a long will fit in a pointer on all platforms which httpd 
> runs on as a practical consideration, and using a cast to long rather 
> than a cast to int will avoid compiler warnings on LP64 platforms.
> 

The above explanation worries me to no end. :) Certainly a union could be
used to avoid this.

-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   "If you can dodge a wrench, you can dodge a ball."

Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

[Joe Orton]

On Mon, Oct 03, 2005 at 08:11:44AM -0400, Jim Jagielski wrote:
> Joe Orton wrote:
> > 
> > On Mon, Oct 03, 2005 at 07:50:39AM -0400, Jim Jagielski wrote:
> > > Just some lines that caught my eye:
> > > 
> > > >  
> > > > -*context = (void *)(value == 'T');
> > > > +*context = (void *)((long)(value == 'T'));
> > > >  
> > > > -int value = context != NULL;
> > > > +long value = context != NULL;
> > > >  apr_fileperms_t perms = resource->info->finfo.protection;
> > > > -int old_value = (perms & APR_UEXECUTE) != 0;
> > > > +long old_value = (perms & APR_UEXECUTE) != 0;
> > > >  
> > > 
> > > Huh? Whenever I see conditionals cast to (long) I get
> > > suspicious. 
> > 
> > These are all cases where an integer is stored in a pointer; it's safe 
> > to assume that a long will fit in a pointer on all platforms which httpd 
> > runs on as a practical consideration, and using a cast to long rather 
> > than a cast to int will avoid compiler warnings on LP64 platforms.
> > 
> 
> The above explanation worries me to no end. :) Certainly a union could be
> used to avoid this.

Well there are solutions, sure, not sure how a union would help here.  
Try it and see... I usually end up deciding these issues are so marginal 
that there's some better way to spend time fixing real bugs than to try 
and break code which works perfectly well ;)

joe

[PATCH] worker status change during creation of backend proxy connection

[Ruediger Pluem]

Hi,

I noticed that the status of a connection (or better for a worker score)
is changed back to SERVER_BUSY_READ from SERVER_BUSY_WRITE, if this connection
requires that mod_proxy creates a new connection to a backend. This does not 
happen
if mod_proxy can pick up a connection to this backend from the connection pool.
This is a hint to me that the behaviour in the situation where a new connection 
gets
created is not correct.

I see two problems with the current behaviour:

1. mod_status shows the "wrong" status.
2. It is hard for functions in the pre_connection hook to decide if they are 
called
   for an outgoing proxy connection or for an incoming client connection.
   For sure for some this does not matter, but for others it does.
   If the status of the connection would not be changed they could determine 
this
   by checking if the status is SERVER_BUSY_READ (client connection) or
   SERVER_BUSY_WRITE (proxy connection to backend).

The reason for the status change is the execution of the create_connection hook
in ap_proxy_connection_create in proxy_util.c and thus the execution of
core_create_conn in core.c.

So my idea is to save the scoreboard entry for this slot before calling the
create_connection hook and restore it afterwards. If this is is not a solution
for the problem I fear that some refactoring of the connection creation process 
in
mod_proxy is needed.

Below there is a patch which does what I described. As struct ap_sb_handle_t is
private data to scoreboard.c, I created a small function that allows me to get 
the
pointer to worker_score based on the scoreboard handle.
I am not totally happy with the name of this function 
(ap_get_scoreboard_worker_sbh),
so any proposals for a different name are welcome.

Comments / thoughts?


Regards

Rüdiger
Index: server/scoreboard.c
===
--- server/scoreboard.c»(Revision 292553)
+++ server/scoreboard.c»(Arbeitskopie)
@@ -476,6 +476,13 @@
 return &ap_scoreboard_image->servers[x][y];
 }
·
+AP_DECLARE(worker_score *) ap_get_scoreboard_worker_sbh(ap_sb_handle_t *sbh)
+{
+if (!sbh)
+return(NULL);
+return ap_get_scoreboard_worker(sbh->child_num, sbh->thread_num);
+}
+
 AP_DECLARE(process_score *) ap_get_scoreboard_process(int x)
 {
 if ((x < 0) || (server_limit < x)) {
Index: modules/proxy/proxy_util.c
===
--- modules/proxy/proxy_util.c»·(Revision 292553)
+++ modules/proxy/proxy_util.c»·(Arbeitskopie)
@@ -1995,13 +1995,35 @@
 {
 apr_sockaddr_t *backend_addr = conn->addr;
 int rc;
+/* Pointer to scoreboard entry of current worker */
+worker_score *current_worker_score;
+/* Saves scoreboard entry of current worker temporarily */
+worker_score current_worker_score_save;
·
+/*
+ * As ap_run_create_connection modifies the scoreboard entry for the
+ * current worker, save the whole scoreboard entry for this worker
+ * and restore it after ap_run_create_connection has been run.
+ * Apart from the fact that mod_status will display correct information
+ * again it makes it possible for functions in the
+ * ap_run_pre_connection to check whether they are called for
+ * a mod_proxy connection or for an incoming connection by just checking
+ * the status of the current worker (whether its SERVER_BUSY_READ
+ * or SERVER_BUSY_WRITE).
+ */
+current_worker_score = ap_get_scoreboard_worker_sbh(c->sbh);
+memcpy(¤t_worker_score_save, current_worker_score,
+   sizeof(worker_score));
+
 /* The socket is now open, create a new backend server connection·
 *·
 */
 conn->connection = ap_run_create_connection(c->pool, s, conn->sock,
 c->id, c->sbh,
 c->bucket_alloc);
+/* Restore scoreboard entry for current worker */
+memcpy(current_worker_score, ¤t_worker_score_save,
+   sizeof(worker_score));
·
 if (!conn->connection) {
 /* the peer reset the connection already; ap_run_create_connection()·
Index: include/scoreboard.h
===
--- include/scoreboard.h»···(Revision 292553)
+++ include/scoreboard.h»···(Arbeitskopie)
@@ -190,6 +190,7 @@
 void ap_time_process_request(ap_sb_handle_t *sbh, int status);
·
 AP_DECLARE(worker_score *) ap_get_scoreboard_worker(int x, int y);
+AP_DECLARE(worker_score *) ap_get_scoreboard_worker_sbh(ap_sb_handle_t *sbh);
 AP_DECLARE(process_score *) ap_get_scoreboard_process(int x);
 AP_DECLARE(global_score *) ap_get_scoreboard_global(void);
 AP_DECLARE(lb_score *) ap_get_scoreboard_lb(int lb_num);

Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

[Jim Jagielski]

Joe Orton wrote:
> 
> On Mon, Oct 03, 2005 at 08:11:44AM -0400, Jim Jagielski wrote:
> > Joe Orton wrote:
> > > 
> > > On Mon, Oct 03, 2005 at 07:50:39AM -0400, Jim Jagielski wrote:
> > > > Just some lines that caught my eye:
> > > > 
> > > > >  
> > > > > -*context = (void *)(value == 'T');
> > > > > +*context = (void *)((long)(value == 'T'));
> > > > >  
> > > > > -int value = context != NULL;
> > > > > +long value = context != NULL;
> > > > >  apr_fileperms_t perms = resource->info->finfo.protection;
> > > > > -int old_value = (perms & APR_UEXECUTE) != 0;
> > > > > +long old_value = (perms & APR_UEXECUTE) != 0;
> > > > >  
> > > > 
> > > > Huh? Whenever I see conditionals cast to (long) I get
> > > > suspicious. 
> > > 
> > > These are all cases where an integer is stored in a pointer; it's safe 
> > > to assume that a long will fit in a pointer on all platforms which httpd 
> > > runs on as a practical consideration, and using a cast to long rather 
> > > than a cast to int will avoid compiler warnings on LP64 platforms.
> > > 
> > 
> > The above explanation worries me to no end. :) Certainly a union could be
> > used to avoid this.
> 
> Well there are solutions, sure, not sure how a union would help here.  
> Try it and see... I usually end up deciding these issues are so marginal 
> that there's some better way to spend time fixing real bugs than to try 
> and break code which works perfectly well ;)
> 

At the very last, if we are "assuming" behavior which is
specifically implementation dependent, then a test during
configure time that ensures sizeof(void *) <= sizeof(long)
makes sense.

There is no room, IMO, for silent hidden assumptions
in httpd.

-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   "If you can dodge a wrench, you can dodge a ball."

Re: Pluggable mod_log_config

[Ondrej Sury]

On Mon, 2005-10-03 at 07:58 -0400, Brian Akins wrote:
> 
> mod_log_config is "pluggable."   You can "replace" the normal logging 
> functions quite easily.

The word here is _replace_.  If you "replace" ap_log_writer and
ap_log_writer_init you need to implement all types of logging inside
ap_log_writer.

Imagine this type of config:
--cut here--

  [...]
  CustomLog mysql://blah/data common




  [...]
  CustomLog $apache#1 combined # mod_log_spread syntax




  [...]
  CustomLog /var/log/apache2/access.log


--cut here--

Then your replacement function would have to implement all those types
of log backends (ie. logging to mysql, spread, and files).  So you
cannot just easily add support for just another log backend.

Even now when you want to make some log files buffered and some not you
are out of luck and this could also be solved by making plugging more
"general".

Ondrej.
-- 
Ondrej Sury <[EMAIL PROTECTED]>


signature.asc
Description: This is a digitally signed message part

Re: Pluggable mod_log_config

[Brian Akins]

Ondrej Sury wrote:


Even now when you want to make some log files buffered and some not you
are out of luck and this could also be solved by making plugging more
"general".



Ok I see what you mean and I agree.  Maybe something stupid like a 
SetLogHandler similar to SetHandler?  or as an argument to CustomLog?


Hmmm... Maybe custom loggers could be providers that have an init and a 
writer function.


--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: Pluggable mod_log_config

[Ondrej Sury]

On Mon, 2005-10-03 at 09:19 -0400, Brian Akins wrote:
> Ondrej Sury wrote:
> 
> > Even now when you want to make some log files buffered and some not you
> > are out of luck and this could also be solved by making plugging more
> > "general".
> 
> 
> Ok I see what you mean and I agree.  Maybe something stupid like a 
> SetLogHandler similar to SetHandler?  or as an argument to CustomLog?

I think that argument to CustomLog is ok...  you just need to check if
logger was already initialized or not (ie. return NULL or data structure
from ap_*_logger_init).  Each ap_*_logger_init function will be run in
some order (AP_HOOK_FIRST, etc.) and it will stop after logger will get
initialized.  Should not be that hard to implement.

> Hmmm... Maybe custom loggers could be providers that have an init and a 
> writer function.

Yep, something like that.

You also need to store which writer to use in each config_log_state.

I'll think about it more and came with some proposal (or patch :-).

Ondrej.
-- 
Ondrej Sury <[EMAIL PROTECTED]>


signature.asc
Description: This is a digitally signed message part

Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules: dav/fs/dbm.c

[William A. Rowe, Jr.]

Joe Orton wrote:

On Mon, Oct 03, 2005 at 08:11:44AM -0400, Jim Jagielski wrote:


Joe Orton wrote:


On Mon, Oct 03, 2005 at 07:50:39AM -0400, Jim Jagielski wrote:


Just some lines that caught my eye:
... Whenever I see conditionals cast to (long) I get
suspicious. 


These are all cases where an integer is stored in a pointer; it's safe 
to assume that a long will fit in a pointer on all platforms which httpd 
runs on as a practical consideration, and using a cast to long rather 
than a cast to int will avoid compiler warnings on LP64 platforms.


... Certainly a union could be
used to avoid this.


I usually end up deciding these issues are so marginal 
that there's some better way to spend time fixing real bugs than to try 
and break code which works perfectly well ;)


Joe's correct that this code change works on ILP32, ILP64 and LP64, 
platforms - but I concur with Jim that for the casual developer, the

purpose is hard to glean...

...perhaps we need an apr_intptr_t type which is a best-fit int for any
arbitrary void* storage class?

Bill

Re: Pluggable mod_log_config

[Brian Akins]

Akins, Brian wrote:


Hmmm... Maybe custom loggers could be providers that have an init and a
writer function.



Maybe have customLog be something like this:

CustomLog mysql://something common env=images
CustomLog file:///logs/my.log combined
CustomLog spread://somegroup refere
CustomLog buffer:///logs/other.log common

with file:// being the default.

Then to create a new log handler, just do a provider:

/*define init and writer function like normal*/

Then register your provider:

static const log_provider_t my_provider = {
  &my_init,
  &my_writer
};

static void register_hooks(apr_pool_t * p)
{
ap_register_provider(p, LOG_PROVIDER_GROUP, "mysql", 
LOG_PROVIDER_VERSION,

 &my_provider);
}


Thoughts?

--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: svn commit: r293305 - in /httpd/httpd/branches/2.2.x/modules:

[Jim Jagielski]

William A. Rowe, Jr. wrote:
> 
> Joe Orton wrote:
> > On Mon, Oct 03, 2005 at 08:11:44AM -0400, Jim Jagielski wrote:
> > 
> >>Joe Orton wrote:
> >>
> >>>On Mon, Oct 03, 2005 at 07:50:39AM -0400, Jim Jagielski wrote:
> >>>
> Just some lines that caught my eye:
> ... Whenever I see conditionals cast to (long) I get
> suspicious. 
> >>>
> >>>These are all cases where an integer is stored in a pointer; it's safe 
> >>>to assume that a long will fit in a pointer on all platforms which httpd 
> >>>runs on as a practical consideration, and using a cast to long rather 
> >>>than a cast to int will avoid compiler warnings on LP64 platforms.
> >>
> >>... Certainly a union could be
> >>used to avoid this.
> > 
> > I usually end up deciding these issues are so marginal 
> > that there's some better way to spend time fixing real bugs than to try 
> > and break code which works perfectly well ;)
> 
> Joe's correct that this code change works on ILP32, ILP64 and LP64, 
> platforms - but I concur with Jim that for the casual developer, the
> purpose is hard to glean...
> 
> ...perhaps we need an apr_intptr_t type which is a best-fit int for any
> arbitrary void* storage class?
> 

ANSI C specifically allows for pointers to be converted to
a "large enough" integral type. For safety, I would suggest unsigned
long, but whenever we have impl dependent code, we should make
sure it's well documented, abstract it out if possible, or, at
least, confirm during configuration that our assumptions are
correct.

-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   "If you can dodge a wrench, you can dodge a ball."

Re: Pluggable mod_log_config

[Brian Akins]

Ondrej Sury wrote:


You also need to store which writer to use in each config_log_state.


This is already done.

Also, init is called for each CustomLog config entry.

--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: dbd connections tied to conn_rec

[Brian J. France]

Not sure why yet, but for some reason I now have to do start/end 
transaction around around my sql calls (postgres).  Otherwise I will 
get a error "another command is already in progress" after the first 
connection, which I wasn't getting before.  After doing this all is 
working good (both with and without APR_HAS_THREADS set in mod_dbd.c).


While trying to debug this I found my original thought:


I think it might have to do with allocating a dbd from the connection
pool when it needs to be from the server pool for persist connections.
non-persist connection can use the connection pool.


was wrong as because it does the right thing inside of ap_dbd_open, but 
I do have a question about code in ap_dbd_open.  If the connection is 
not persist, should it use the passed in pool (the connection pool, 
request pool for acquire functions or temp pool if called directly) 
instead of the server pool?  Currently every thing with the DB handle 
will be closed correctly, but I think it will leak sizeof(ap_dbd_t) for 
each call of ap_dbd_open because that memory is cleaned up from the 
server pool (instead of connect, request or temp pool).


Brian


On Oct 1, 2005, at 4:33 AM, Nick Kew wrote:

On Saturday 01 October 2005 01:26, Brian J. France wrote:

Thanks for the work!

I am having problems with it on the second connection, first 
connection

works with multiple calls to ap_dbd_cacquire.

I think it might have to do with allocating a dbd from the connection
pool when it needs to be from the server pool for persist connections.
non-persist connection can use the connection pool.

I don't have time to dig into right now, but will try this weekend.


Thanks for the feedback.  Are you running the patch against current SVN
(i.e. after the fixes we made yesterday), or something older?

Re: Pluggable mod_log_config

[Ondrej Sury]

On Mon, 2005-10-03 at 09:35 -0400, Brian Akins wrote:
> Ondrej Sury wrote:
> 
> > You also need to store which writer to use in each config_log_state.
> 
> This is already done.

Nope, (void *)cls->log_writer is pointer to log file descriptor.

> Also, init is called for each CustomLog config entry.

Yep, but only ap_log_writer_init is called.  You need to do something
like that:

foreach provider in ap_log_writer_providers {
cls->log_writer = provider->init(, &cls->log_data);

if (cls->log_writer) {
break;
}
}

Then later when writing log you need to call

cls->log_writer(..., cls->log_data, ...)

instead of just

log_writer(...)

Ondrej.
-- 
Ondrej Sury <[EMAIL PROTECTED]>


signature.asc
Description: This is a digitally signed message part

Re: Pluggable mod_log_config

[Brian Akins]

Ondrej Sury wrote:



Nope, (void *)cls->log_writer is pointer to log file descriptor.


this can be anything. (it's a void).  In some modules I have written, 
it's a wrapper around a socket. See, open_config_log


cls->log_writer = log_writer_init(p, s, cls->fname);

log_writer_init retures whatever you function returns.  In 
mod_log_config, it's an apr_file_t *.


Currently, however, log_writer_init is global.  It should be per cls, or 
per server.






Also, init is called for each CustomLog config entry.



Yep, but only ap_log_writer_init is called.  You need to do something
like that:

foreach provider in ap_log_writer_providers {
cls->log_writer = provider->init(, &cls->log_data);

if (cls->log_writer) {
break;
}
}

Then later when writing log you need to call

cls->log_writer(..., cls->log_data, ...)

instead of just

log_writer(...)


yes. basically.

I figure you could change cls to:

typedef struct {
const char *fname;
const char *format_string;
apr_array_header_t *format;
void *log_writer;
void *log_init; /*added*/
char *condition_var;
} config_log_state;


and log_init and log_writer would be set at config time.  log_init would 
get ran in open_logs.



--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: svn commit: r291914 - /httpd/httpd/trunk/server/request.c

[Joe Orton]

On Tue, Sep 27, 2005 at 11:52:16AM -, [EMAIL PROTECTED] wrote:
> Author: colm
> Date: Tue Sep 27 04:52:14 2005
> New Revision: 291914
> 
> URL: http://svn.apache.org/viewcvs?rev=291914&view=rev
> Log:
> When doing a subrequest lookup, do not lookup using the quickhandler if
> next_filter is NULL, since this implies that the subrequest will be
> internal_fast_redirect'ed or will never be called. See line request.c:1513
> onwards (in make_sub_request) also.  

gcc < 4 is giving a warning:

request.c: In function `ap_sub_req_method_uri':
request.c:1584: warning: 'res' might be used uninitialized in this function

Re: Pluggable mod_log_config

[Ondrej Sury]

On Mon, 2005-10-03 at 10:26 -0400, Brian Akins wrote:
> Ondrej Sury wrote:
> 
> > 
> > Nope, (void *)cls->log_writer is pointer to log file descriptor.
> 
> this can be anything. (it's a void).  In some modules I have written, 
> it's a wrapper around a socket. See, open_config_log
> 
> cls->log_writer = log_writer_init(p, s, cls->fname);
> 
> log_writer_init retures whatever you function returns.  In 
> mod_log_config, it's an apr_file_t *.

Sorry for not being accurate.  My point was that cls->log_writer hold
logger *data/configuration* and not logging *function*.

> >>Also, init is called for each CustomLog config entry.
> > 
> > 
> > Yep, but only ap_log_writer_init is called.  You need to do something
> > like that:
> > 
> > foreach provider in ap_log_writer_providers {
> > cls->log_writer = provider->init(, &cls->log_data);
> > 
> > if (cls->log_writer) {
> > break;
> > }
> > }
> > 
> > Then later when writing log you need to call
> > 
> > cls->log_writer(..., cls->log_data, ...)
> > 
> > instead of just
> > 
> > log_writer(...)
> 
> yes. basically.
> 
> I figure you could change cls to:
> 
> typedef struct {
>  const char *fname;
>  const char *format_string;
>  apr_array_header_t *format;
>  void *log_writer;
>  void *log_init; /*added*/
>  char *condition_var;
> } config_log_state;

What about:

typedef struct {
const char *fname;
const char *format_string;
apr_array_header_t *format;
ap_log_writer *log_writer; /* changed */
ap_log_writer_init *log_writer_init; /* added */
void *log_writer_config; /* changed from log_writer */
char *condition_var;
} config_log_state;

If you set log writing function to log_writer then you are missing
config for this writer (ie. socket or apr_file_t * or mysql connection
or whatever else).

> and log_init and log_writer would be set at config time.  log_init would 
> get ran in open_logs.

Agree.

Ondrej.
-- 
Ondrej Sury <[EMAIL PROTECTED]>


signature.asc
Description: This is a digitally signed message part

Re: svn commit: r291914 - /httpd/httpd/trunk/server/request.c

[Colm MacCarthaigh]

On Mon, Oct 03, 2005 at 03:50:34PM +0100, Joe Orton wrote:
> gcc < 4 is giving a warning:
> 
> request.c: In function `ap_sub_req_method_uri':
> request.c:1584: warning: 'res' might be used uninitialized in this function

Bah, stupid compilers and their invalid warnings :) I'll fix that.

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]

Re: Urp(!) - indeterminant connection SO_NONBLOCK state?

[William A. Rowe, Jr.]

Joe Orton wrote:


I'm not sure I understand the issue.  mod_ssl should already propagate 
the APR_BLOCK_READ down so that blocking ap_get_brigade calls are made 
during the SSL_accept() as far as I can see?  Does this fix help?


http://svn.apache.org/viewcvs.cgi?rev=105768&view=rev


Nope, we go from...

[Mon Oct 03 07:56:25 2005] [info] Connection to child 0 established 
(server www.example.com:8022, client 10.0.8.7)

[Mon Oct 03 07:56:25 2005] [info] Seeding PRNG with 136 bytes of entropy
[Mon Oct 03 07:56:25 2005] [info] SSL handshake interrupted by system 
[Hint: Stop button pressed in browser?!]
[Mon Oct 03 07:56:25 2005] [info] Connection to child 0 closed with 
abortive shutdown(server www.example.com:8022, client 10.0.8.7)


to...

[Mon Oct 03 08:28:50 2005] [info] Connection to child 0 established 
(server www.example.com:8022, client 10.0.8.7)

[Mon Oct 03 08:28:50 2005] [info] Seeding PRNG with 136 bytes of entropy

However the client dies immediately.

Reproducing is easy, simply load mod_echo.c, add 'ProtocolEcho on' to
the SSL VHost section, then openssl s_client --crlf --connect foo:8443
... it's really that easy.

I actually set up a second host on another port.  http through mod_ssl
does work, mod_echo does not.

Bill

[PATCH] Re: Pluggable mod_log_config

[Brian Akins]

Akins, Brian wrote:


CustomLog mysql://something common env=images
CustomLog file:///logs/my.log combined
CustomLog spread://somegroup refere
CustomLog buffer:///logs/other.log common



This patch implements the above.  Within mod_log_config two providers
are provided: file and buffer.  If no "scheme" is given, file is assumed.

I have tested and preliminarily it works for both file and buffer.  I
had to rearrange some of the buffer code to get it to work "non-globally."

It should be just as easy to write custom log handles. With this patch,
different custom loggers could handle different log "files."

Comments?


--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

diff -ur httpd-2.1.8-beta/modules/loggers/mod_log_config.c 
httpd-2.1.8-beta.new/modules/loggers/mod_log_config.c
--- httpd-2.1.8-beta/modules/loggers/mod_log_config.c   2005-09-18 
16:39:34.0 -0400
+++ httpd-2.1.8-beta.new/modules/loggers/mod_log_config.c   2005-10-03 
11:10:48.677989129 -0400
@@ -159,6 +159,7 @@
 #include "http_protocol.h"
 #include "util_time.h"
 #include "ap_mpm.h"
+#include "ap_provider.h"
 
 #if APR_HAVE_UNISTD_H
 #include 
@@ -167,6 +168,8 @@
 #include 
 #endif
 
+#define DEFAULT_LOG_PROVIDER "file"
+
 #define DEFAULT_LOG_FORMAT "%h %l %u %t \"%r\" %>s %b"
 
 module AP_MODULE_DECLARE_DATA log_config_module;
@@ -188,15 +191,9 @@
int nelts,
apr_size_t len);
 static void *ap_default_log_writer_init(apr_pool_t *p, server_rec *s, 
-const char* name);
+const apr_uri_t *uri);
 static void *ap_buffered_log_writer_init(apr_pool_t *p, server_rec *s, 
-const char* name);
-
-static ap_log_writer_init* ap_log_set_writer_init(ap_log_writer_init *handle);
-static ap_log_writer* ap_log_set_writer(ap_log_writer *handle);
-static ap_log_writer *log_writer = ap_default_log_writer;
-static ap_log_writer_init *log_writer_init = ap_default_log_writer_init;
-static int buffered_logs = 0; /* default unbuffered */
+ const apr_uri_t *uri);
 static apr_array_header_t *all_buffered_logs = NULL;
 
 /* POSIX.1 defines PIPE_BUF as the maximum number of bytes that is
@@ -255,10 +252,12 @@
 } buffered_log;
 
 typedef struct {
-const char *fname;
+apr_uri_t uri;
+const char *fname; /*raw file name*/
 const char *format_string;
 apr_array_header_t *format;
-void *log_writer;
+void *handle;
+log_provider_t *provider;
 char *condition_var;
 } config_log_state;
 
@@ -954,12 +953,7 @@
 for (i = 0; i < format->nelts; ++i) {
 len += strl[i] = strlen(strs[i]);
 }
-if (!log_writer) {
-ap_log_rerror(APLOG_MARK, APLOG_ERR, APR_EGENERAL, r,
-"log writer isn't correctly setup");
- return HTTP_INTERNAL_SERVER_ERROR;
-}
-rv = log_writer(r, cls->log_writer, strs, strl, format->nelts, len);
+rv = cls->provider->writer(r, cls->handle, strs, strl, format->nelts, len);
 /* xxx: do we return an error on log_writer? */
 return OK;
 }
@@ -1093,8 +1087,22 @@
 else {
 cls->format = parse_log_string(cmd->pool, fmt, &err_string);
 }
-cls->log_writer = NULL;
+
+if(apr_uri_parse(cmd->pool, cls->fname, &cls->uri) != APR_SUCCESS) {
+return "failed to parse uri";
+}
 
+if(!cls->uri.scheme) {
+cls->uri.scheme = DEFAULT_LOG_PROVIDER;
+}
+
+if((cls->provider = ap_lookup_provider(LOG_PROVIDER_GROUP,
+   cls->uri.scheme, 
LOG_PROVIDER_VERSION))
+   == NULL) {
+return apr_psprintf(cmd->pool, "unable to file handler for: \"%s\" in 
%s",
+cls->uri.scheme, cls->fname);
+}
+
 return err_string;
 }
 
@@ -1109,15 +1117,6 @@
 return add_custom_log(cmd, dummy, fn, "%{Cookie}n \"%r\" %t", NULL);
 }
 
-static const char *set_buffered_logs_on(cmd_parms *parms, void *dummy, int 
flag)
-{
-buffered_logs = flag;
-if (buffered_logs) {
-ap_log_set_writer_init(ap_buffered_log_writer_init);
-ap_log_set_writer(ap_buffered_log_writer);
-}
-return NULL;
-}
 static const command_rec config_log_cmds[] =
 {
 AP_INIT_TAKE23("CustomLog", add_custom_log, NULL, RSRC_CONF,
@@ -1129,8 +1128,6 @@
  "a log format string (see docs) and an optional format name"),
 AP_INIT_TAKE1("CookieLog", set_cookie_log, NULL, RSRC_CONF,
  "the filename of the cookie log"),
-AP_INIT_FLAG("BufferedLogs", set_buffered_logs_on, NULL, RSRC_CONF,
- "Enable Buffered Logging (experimental)"),
 {NULL}
 };
 
@@ -1138,7 +1135,7 @@
  config_log_state *cls,
  apr_array_header_t *default_format)
 {
-if (cls->log_writer != NULL) {
+if (cls->handle != NULL) {
 return cls; /* virtual config s

Re: Release plans?

[Paul Querna]

Oden Eriksson wrote:
> Hi.
> 
> I don't know if this is the right forum for this question but I found no info 
> elsewhere. I simply wonder when 2.1.x is going to be dubbed stable?
> 

See our versioning file:
http://svn.apache.org/repos/asf/httpd/httpd/trunk/VERSIONING

2.1.x will never be 'stable'.  Your question should be, 'I simply wonder
when 2.2.0 is going to be released?'

There is no simple answer.  Mine is simply 'soon'.  The more people that
test the 2.1.x betas on more sites give us more confidence in calling it
2.2.0.  We want both positive and negative reports.  So, my question is
to you, did you test 2.1.8-beta?  Did you try some of the new features?
Did it work how you expected? Do you need more documentation on how to
use the new features?

Thanks,

-Paul

Re: Release plans?

[Brian Akins]

Paul Querna wrote:
 Do you need more documentation on how to

use the new features?



Do we have some examples of mod_proxy_balancer and sticky sessions?



--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: patch to mod_arm4

[Bill Stoddard]

Kai Engels wrote:

HI,
just wanted to move this thread to the top, since there was no reaction 
to my patches

(I have splitted the one to 4 different) ?
Are they too bad :-) ? Or was it just forgotten ?



I've not forgotten, just really busy.  One of the patches I can commit no 
problem. Need to look at the other 3.

Bill

Re: patch to mod_arm4

[Kai Engels]

Bill Stoddard wrote:


Kai Engels wrote:


HI,
just wanted to move this thread to the top, since there was no 
reaction to my patches

(I have splitted the one to 4 different) ?
Are they too bad :-) ? Or was it just forgotten ?



I've not forgotten, just really busy.  One of the patches I can commit 
no problem. Need to look at the other 3.


Bill


Hi Bill,
ok, no problem. I can wait.
Was just wondering...
:-)
Greetings
Kai

Re: [PATCH] ProxyRemote + ProxyBlock oddness

[Eric Covener]

On 6/15/05, Jeff Trawick <[EMAIL PROTECTED]> wrote:
> On 4/25/05, Eric Covener <[EMAIL PROTECTED]> wrote:
>
> > I've attached a patch that resolves the hostname in the URI and hands
> > that off separately to ap_proxy_checkproxyblock().

> Any comments from the peanut gallery, particularly the proxy portion?

Just revisiting this issue that still appears in 2.1.8...when proxying
by way of another proxy (ProxyRemote), httpd will compare that
ProxyRemote backend address to the list of ProxyBlocks. It should
compare the address in the URI.


ProxyRequests on
ProxyRemote * http://nextproxy:8081
   ProxyBlock blockedhost



When a browser requests something from the server 'blockedhost':

[debug] proxy_util.c(872): proxy: checking remote machine [nextproxy]
against [blockedhost]

Would expect to see a check of blockedhost against blockedhost, as in 2.0

--
Eric Covener
[EMAIL PROTECTED]
Index: proxy/proxy_util.c
===
--- proxy/proxy_util.c	(revision 159730)
+++ proxy/proxy_util.c	(working copy)
@@ -1743,6 +1743,7 @@
 {
 int server_port;
 apr_status_t err = APR_SUCCESS;
+apr_sockaddr_t *uri_addr = NULL;
 
 /*
  * Break up the URL to determine the host to connect to
@@ -1806,6 +1807,16 @@
 apr_pool_cleanup_kill(conn->connection->pool, conn, connection_cleanup);
 conn->connection = NULL;
 }
+if (conf->noproxies->nelts > 0)  {
+err = apr_sockaddr_info_get(&uri_addr, 
+apr_pstrdup(conn->pool, uri->hostname), APR_UNSPEC, 
+uri->port, 0, 
+conn->pool);
+if (err != APR_SUCCESS)
+return ap_proxyerror(r, HTTP_BAD_GATEWAY,
+ apr_pstrcat(p, "DNS lookup failure for: ",
+ uri->hostname, NULL));
+}
 err = apr_sockaddr_info_get(&(conn->addr),
 conn->hostname, APR_UNSPEC,
 conn->port, 0,
@@ -1850,7 +1861,10 @@
 }
 }
 /* check if ProxyBlock directive on this host */
-if (OK != ap_proxy_checkproxyblock(r, conf, conn->addr)) {
+
+if (!proxyname) uri_addr = conn->addr;
+
+if (OK != ap_proxy_checkproxyblock(r, conf, uri_addr)) {
 return ap_proxyerror(r, HTTP_FORBIDDEN,
  "Connect to remote machine blocked");
 }

Re: Urp(!) - indeterminant connection SO_NONBLOCK state?

[Joe Orton]

On Mon, Oct 03, 2005 at 10:34:28AM -0500, William Rowe wrote:
> Joe Orton wrote:
> >
> >I'm not sure I understand the issue.  mod_ssl should already propagate 
> >the APR_BLOCK_READ down so that blocking ap_get_brigade calls are made 
> >during the SSL_accept() as far as I can see?  Does this fix help?
> >
> >http://svn.apache.org/viewcvs.cgi?rev=105768&view=rev
> 
> Nope, we go from...
> 
> [Mon Oct 03 07:56:25 2005] [info] Connection to child 0 established 
> (server www.example.com:8022, client 10.0.8.7)
> [Mon Oct 03 07:56:25 2005] [info] Seeding PRNG with 136 bytes of entropy
> [Mon Oct 03 07:56:25 2005] [info] SSL handshake interrupted by system 
> [Hint: Stop button pressed in browser?!]
> [Mon Oct 03 07:56:25 2005] [info] Connection to child 0 closed with 
> abortive shutdown(server www.example.com:8022, client 10.0.8.7)
> 
> to...
> 
> [Mon Oct 03 08:28:50 2005] [info] Connection to child 0 established 
> (server www.example.com:8022, client 10.0.8.7)
> [Mon Oct 03 08:28:50 2005] [info] Seeding PRNG with 136 bytes of entropy
> 
> However the client dies immediately.

Dies how, what error, what is the server left doing?  This needs further 
diagnosis; truss the server, loglevel debug output, etc.  Does it work 
on the trunk?

> Reproducing is easy, simply load mod_echo.c, add 'ProtocolEcho on' to
> the SSL VHost section, then openssl s_client --crlf --connect foo:8443
> ... it's really that easy.

Works fine for me here on Linux as you say.  You're right that there is 
platform-specific behaviour here of course, I always get confused about 
SO_NONBLOCK.

joe

Re: [PATCH] Re: Pluggable mod_log_config

[Colm MacCarthaigh]

On Mon, Oct 03, 2005 at 11:40:27AM -0400, Brian Akins wrote:
> >CustomLog mysql://something common env=images
> >CustomLog file:///logs/my.log combined
> >CustomLog spread://somegroup refere
> >CustomLog buffer:///logs/other.log common
> 
> This patch implements the above.  Within mod_log_config two providers
> are provided: file and buffer.  If no "scheme" is given, file is assumed.

Looks useful, but file://|/bin/foo would be very non-intuitive for piped
loggers, balancing the backwards compatibility might need a bit more, I
guess "pipe://" or "cmd://" schemes might make sense also. 

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]

Re: [PATCH] Re: Pluggable mod_log_config

[Brian Akins]

Colm MacCarthaigh wrote:


Looks useful, but file://|/bin/foo would be very non-intuitive for piped
loggers, balancing the backwards compatibility might need a bit more, I
guess "pipe://" or "cmd://" schemes might make sense also. 



True.  I just used uri's as they seemed to just "fit."  I'm not 
particularly tied to the configuration interface.  Any interest in me 
changing it to use "cmd://"?



Also, since the default is file://, |/some/command would behave just as 
it does now.


I would like to be able to use multiple custom loggers, rather than only 
get one.


--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: [PATCH] Re: Pluggable mod_log_config

[Jim Jagielski]

Brian Akins wrote:
> 
> Colm MacCarthaigh wrote:
> 
> > Looks useful, but file://|/bin/foo would be very non-intuitive for piped
> > loggers, balancing the backwards compatibility might need a bit more, I
> > guess "pipe://" or "cmd://" schemes might make sense also. 
> 
> 
> True.  I just used uri's as they seemed to just "fit."  I'm not 
> particularly tied to the configuration interface.  Any interest in me 
> changing it to use "cmd://"?
> 

'pipe' is clearer I think...

-- 
===
 Jim Jagielski   [|]   [EMAIL PROTECTED]   [|]   http://www.jaguNET.com/
   "If you can dodge a wrench, you can dodge a ball."

Re: [PATCH] Re: Pluggable mod_log_config

[Brian Akins]

Jim Jagielski wrote:



'pipe' is clearer I think...



sure.  Of course I just discovered that pip logs don't work.  I think 
it's the spaces in the command, ie "|bin/rotatelogs /var/logs/logfile 
5M". Hmmm.. looking into it...




--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: [PATCH] Re: Pluggable mod_log_config

[Rüdiger Plüm]

On 10/03/2005 08:49 PM, Colm MacCarthaigh wrote:
> On Mon, Oct 03, 2005 at 11:40:27AM -0400, Brian Akins wrote:
> 

[..cut..]
> 
> Looks useful, but file://|/bin/foo would be very non-intuitive for piped

I guess you still can use |/bin/foo because file is the default provider.

> loggers, balancing the backwards compatibility might need a bit more, I
> guess "pipe://" or "cmd://" schemes might make sense also. 
> 

Makes also sense to me since it seems to me that piped logging does not really 
play
well with other things like spread or mysql. On the other side, what about the
buffered logging? Would it make sense to make it possible to turn this on
and off with each provider? If yes, we might should have schemes that say e.g.:

mysql:// for unbuffered mysql backend
mysqlb:// for buffered mysql backend

file:// for unbuffered file backend
fileb:// for buffered file backend

Regards

Rüdiger

Re: [PATCH] Re: Pluggable mod_log_config

[Brian Akins]

Rüdiger Plüm wrote:


Makes also sense to me since it seems to me that piped logging does not really 
play
well with other things like spread or mysql. On the other side, what about the
buffered logging? Would it make sense to make it possible to turn this on
and off with each provider? If yes, we might should have schemes that say e.g.:


It seems to me this would be provider specific.



mysql:// for unbuffered mysql backend
mysqlb:// for buffered mysql backend


Probably, a mysql module would want each line individual, rather than a 
large buffer.



file:// for unbuffered file backend
fileb:// for buffered file backend



Or, sticking with the uri methods:

file:///some/log/path?buffered

or maybe:

file://buffered@/some/log/path


Interesting comments.

--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: [PATCH] Re: Pluggable mod_log_config

[Brian Akins]

Jim Jagielski wrote:
True.  I just used uri's as they seemed to just "fit."  I'm not 
particularly tied to the configuration interface.  Any interest in me 
changing it to use "cmd://"?





'pipe' is clearer I think...




This patch just adds pipe:// as a "log uri"

--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies
diff -ur httpd-2.1.8-beta/modules/loggers/mod_log_config.c 
httpd-2.1.8-beta.new/modules/loggers/mod_log_config.c
--- httpd-2.1.8-beta/modules/loggers/mod_log_config.c   2005-09-18 
16:39:34.0 -0400
+++ httpd-2.1.8-beta.new/modules/loggers/mod_log_config.c   2005-10-03 
15:19:55.176764353 -0400
@@ -159,6 +159,7 @@
 #include "http_protocol.h"
 #include "util_time.h"
 #include "ap_mpm.h"
+#include "ap_provider.h"
 
 #if APR_HAVE_UNISTD_H
 #include 
@@ -167,6 +168,8 @@
 #include 
 #endif
 
+#define DEFAULT_LOG_PROVIDER "file"
+
 #define DEFAULT_LOG_FORMAT "%h %l %u %t \"%r\" %>s %b"
 
 module AP_MODULE_DECLARE_DATA log_config_module;
@@ -188,15 +191,9 @@
int nelts,
apr_size_t len);
 static void *ap_default_log_writer_init(apr_pool_t *p, server_rec *s, 
-const char* name);
+const apr_uri_t *uri);
 static void *ap_buffered_log_writer_init(apr_pool_t *p, server_rec *s, 
-const char* name);
-
-static ap_log_writer_init* ap_log_set_writer_init(ap_log_writer_init *handle);
-static ap_log_writer* ap_log_set_writer(ap_log_writer *handle);
-static ap_log_writer *log_writer = ap_default_log_writer;
-static ap_log_writer_init *log_writer_init = ap_default_log_writer_init;
-static int buffered_logs = 0; /* default unbuffered */
+ const apr_uri_t *uri);
 static apr_array_header_t *all_buffered_logs = NULL;
 
 /* POSIX.1 defines PIPE_BUF as the maximum number of bytes that is
@@ -255,10 +252,12 @@
 } buffered_log;
 
 typedef struct {
-const char *fname;
+apr_uri_t uri;
+const char *fname; /*raw file name*/
 const char *format_string;
 apr_array_header_t *format;
-void *log_writer;
+void *handle;
+log_provider_t *provider;
 char *condition_var;
 } config_log_state;
 
@@ -954,12 +953,7 @@
 for (i = 0; i < format->nelts; ++i) {
 len += strl[i] = strlen(strs[i]);
 }
-if (!log_writer) {
-ap_log_rerror(APLOG_MARK, APLOG_ERR, APR_EGENERAL, r,
-"log writer isn't correctly setup");
- return HTTP_INTERNAL_SERVER_ERROR;
-}
-rv = log_writer(r, cls->log_writer, strs, strl, format->nelts, len);
+rv = cls->provider->writer(r, cls->handle, strs, strl, format->nelts, len);
 /* xxx: do we return an error on log_writer? */
 return OK;
 }
@@ -1093,8 +1087,27 @@
 else {
 cls->format = parse_log_string(cmd->pool, fmt, &err_string);
 }
-cls->log_writer = NULL;
+
+if(apr_uri_parse(cmd->pool, cls->fname, &cls->uri) != APR_SUCCESS) {
+return "failed to parse uri";
+}
+
+/*special case, backaward compatible pipes*/
+if(*fn == '|') {
+cls->uri.scheme = "pipe";
+}
+
+if(!cls->uri.scheme) {
+cls->uri.scheme = DEFAULT_LOG_PROVIDER;
+}
 
+if((cls->provider = ap_lookup_provider(LOG_PROVIDER_GROUP,
+   cls->uri.scheme, 
LOG_PROVIDER_VERSION))
+   == NULL) {
+return apr_psprintf(cmd->pool, "unable to file handler for: \"%s\" in 
%s",
+cls->uri.scheme, cls->fname);
+}
+
 return err_string;
 }
 
@@ -1109,15 +1122,6 @@
 return add_custom_log(cmd, dummy, fn, "%{Cookie}n \"%r\" %t", NULL);
 }
 
-static const char *set_buffered_logs_on(cmd_parms *parms, void *dummy, int 
flag)
-{
-buffered_logs = flag;
-if (buffered_logs) {
-ap_log_set_writer_init(ap_buffered_log_writer_init);
-ap_log_set_writer(ap_buffered_log_writer);
-}
-return NULL;
-}
 static const command_rec config_log_cmds[] =
 {
 AP_INIT_TAKE23("CustomLog", add_custom_log, NULL, RSRC_CONF,
@@ -1129,8 +1133,6 @@
  "a log format string (see docs) and an optional format name"),
 AP_INIT_TAKE1("CookieLog", set_cookie_log, NULL, RSRC_CONF,
  "the filename of the cookie log"),
-AP_INIT_FLAG("BufferedLogs", set_buffered_logs_on, NULL, RSRC_CONF,
- "Enable Buffered Logging (experimental)"),
 {NULL}
 };
 
@@ -1138,7 +1140,7 @@
  config_log_state *cls,
  apr_array_header_t *default_format)
 {
-if (cls->log_writer != NULL) {
+if (cls->handle != NULL) {
 return cls; /* virtual config shared w/main server */
 }
 
@@ -1146,8 +1148,8 @@
 return cls; /* Leave it NULL to decline.  */
 }
 
-cls->log_writer = log_writer_init(p, s, cls->fname);
-if (cls->log_writer == NULL)
+cls->han

Re: [PATCH] Re: Pluggable mod_log_config

[Ondrej Sury]

On Mon, 2005-10-03 at 11:40 -0400, Brian Akins wrote:
> Akins, Brian wrote:
> 
> > CustomLog mysql://something common env=images
> > CustomLog file:///logs/my.log combined
> > CustomLog spread://somegroup refere
> > CustomLog buffer:///logs/other.log common
> 
> 
> This patch implements the above.  Within mod_log_config two providers
> are provided: file and buffer.  If no "scheme" is given, file is assumed.
> 
> I have tested and preliminarily it works for both file and buffer.  I
> had to rearrange some of the buffer code to get it to work "non-globally."
> 
> It should be just as easy to write custom log handles. With this patch,
> different custom loggers could handle different log "files."
> 
> Comments?

Wow, you're fast :-).

Just quick thought, maybe we should add:

ap_log_writer_close *close;

to struct log_provider_t.  It's not absolutely necessary, because you
can use apr_pool_cleanup_register(...), but it will make writing addon
modules much cleaner.

Ondrej.
-- 
Ondrej Sury <[EMAIL PROTECTED]>


signature.asc
Description: This is a digitally signed message part

Re: [PATCH] Re: Pluggable mod_log_config

[Brian Akins]

Ondrej Sury wrote:


Just quick thought, maybe we should add:

ap_log_writer_close *close;

to struct log_provider_t.  It's not absolutely necessary, because you
can use apr_pool_cleanup_register(...), but it will make writing addon
modules much cleaner.


That's debatable.  I already use apr_pool_cleanup_register in a lot of 
stuff, so it seems "natural" to me.  I guess we could register a cleanup 
 on each cls that called close or we could let the module developer do 
it...  I'm leaning toward just letting others register the cleanup on 
there own like buffered logs does now.



--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: [PATCH] Re: Pluggable mod_log_config

[Rüdiger Plüm]

On 10/03/2005 09:11 PM, Brian Akins wrote:
> Rüdiger Plüm wrote:
> 
>> Makes also sense to me since it seems to me that piped logging does
>> not really play
>> well with other things like spread or mysql. On the other side, what
>> about the
>> buffered logging? Would it make sense to make it possible to turn this on
>> and off with each provider? If yes, we might should have schemes that
>> say e.g.:
> 
> 
> It seems to me this would be provider specific.

I think buffering is on a higher level and thus it might not be needed
nor useful to reimplement this in every provider. But if we want to
follow this way more strictly we might end up having something like a filter
chain before the provider actually writes the data to its target.
A buffer filter would be the first implementation, but there might be others
like compression or embedded monitoring filters.

> 
> 
>> mysql:// for unbuffered mysql backend
>> mysqlb:// for buffered mysql backend
> 
> 
> Probably, a mysql module would want each line individual, rather than a
> large buffer.

Or it likes to read many lines in one block as it can commit such things
as a batch rather than as single commits per line. Of course it is not required
to use DB transactions on mysql. But for Oracle this might improve performance.

> 
>> file:// for unbuffered file backend
>> fileb:// for buffered file backend
> 
> 
> 
> Or, sticking with the uri methods:
> 
> file:///some/log/path?buffered
> 
> or maybe:
> 
> file://buffered@/some/log/path
> 

Hm, the question is how to get all the parameters set with the URL approach 
(thinking
of piped loggers, which you said currently do not seem to work because of the 
spaces).
Maybe as named parameters in the args?
If one likes to follow a filter approach I think URL's would not be useful, but 
to be
honest I currently would have no other idea in this case as the shell approach 
e.g.


Customlog "|monitor param=somevalue|buffer size=1024|compress |pipe /bin/foo" 
combined

where the last member needs to be a backend provider like pipe, file, mysql.

Regards

Rüdiger

Re: [PATCH] Re: Pluggable mod_log_config

[Brian Akins]

Rüdiger Plüm wrote:



Or it likes to read many lines in one block as it can commit such things
as a batch rather than as single commits per line. Of course it is not required
to use DB transactions on mysql. But for Oracle this might improve performance.



but that would not be buffering in the sense that mod_log_config does 
it.  mod_log_config just keeps appending data to a buffer.  in an sql 
logger, to "buffer" you would keep a list/array/ring of log lines and 
wrap them in a transaction.  This cannot be "buffered" in mod_log_config




Hm, the question is how to get all the parameters set with the URL approach 
(thinking
of piped loggers, which you said currently do not seem to work because of the 
spaces).


See latest patch :)



Maybe as named parameters in the args?
If one likes to follow a filter approach I think URL's would not be useful, but 
to be
honest I currently would have no other idea in this case as the shell approach 
e.g.


Customlog "|monitor param=somevalue|buffer size=1024|compress |pipe /bin/foo" 
combined



where the last member needs to be a backend provider like pipe, file, mysql.



interesting.  You could have a "linebuffer" that buffered in a way I 
describe above and a "buffer" that does it the current way.  This may be 
 a larger change that we first started.  Of course, you could define a 
log filter chain:


LogFilter example "monitor param=somevalue|buffer size=1024|compress|file"


and then apply the filter in Custom Log:

CustomeLog filter://example/logs/some.log common

filter could supply "/logs/some.log" as user_data to each filter.

Of course, this is a much larger change than I think we are ready to 
tackle now.


--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

Re: [PATCH] Re: Pluggable mod_log_config

[Joshua Kogut]

On 10/3/05, Brian Akins <[EMAIL PROTECTED]> wrote:
Rüdiger Plüm wrote:>> Or it likes to read many lines in one block as it can commit such things> as a batch rather than as single commits per line. Of course it is not required> to use DB transactions on mysql. But for Oracle this might improve performance.

Hey! I resent that. I am sure that this topic has been the starter of
many a flame war, so lets not start another one here, but Mysql is
surely faster that Oracle!
-- ||  jmkogut  ||email: [EMAIL PROTECTED]

Re: [PATCH] Re: Pluggable mod_log_config

[Rüdiger Plüm]

On 10/03/2005 09:53 PM, Brian Akins wrote:
> Rüdiger Plüm wrote:
> 

[..cut..]

> 
> but that would not be buffering in the sense that mod_log_config does
> it.  mod_log_config just keeps appending data to a buffer.  in an sql
> logger, to "buffer" you would keep a list/array/ring of log lines and
> wrap them in a transaction.  This cannot be "buffered" in mod_log_config

Agreed.

[..cut..]


> See latest patch :)

To be honest I haven't tried it yet, but does apr_uri_parse handle the spaces
between the piped log parameters correctly?
Furthermore I guess you should use

pl = ap_open_piped_log(p, name);

instead of

pl = ap_open_piped_log(p, name + 1);

The "|" does not prefix name any longer :-).

[..cut..]

> interesting.  You could have a "linebuffer" that buffered in a way I
> describe above and a "buffer" that does it the current way.  This may be
>  a larger change that we first started.  Of course, you could define a
> log filter chain:
> 
> LogFilter example "monitor param=somevalue|buffer size=1024|compress|file"
> 

Sounds like a good idea

> 
> and then apply the filter in Custom Log:
> 
> CustomeLog filter://example/logs/some.log common
> 
> filter could supply "/logs/some.log" as user_data to each filter.

I guess /logs/some.log is only interesting for the backend. We also
need to consider that backend providers might need totally different (compared 
to filenames)
parameter types. Maybe URL's with args.
I currently found no solution that makes me really happy in conjunction with 
logfilters but
I will keep on thinking on this.

> 
> Of course, this is a much larger change than I think we are ready to
> tackle now.
>

Yes of course. This would require much changes and will not be done in a quick
patch, but it just came up my mind during this discussion and it might be a 
feature to think of.

Regards

Rüdiger

how to query server addresses and ports from a running Apache server using C?

[Yulin Dong]

Is there a way to query what server addresses and ports a running
Apache server listening on and what the secure level of each port
programmatically using C?

I am trying to write a module to get a running Apache server's
addresses and port numbers and also the secure level of the each port ,
such as whether it is HTTP or HTTPS (and then the encryption level).
However, I am having a hard time to find a solution using Apache API by
reading documentation and Apache source code.

Can anyone please point me to a right direction or give me some sample code?

Thanks a lot.

Yulin Dong

Re: Urp(!) - indeterminant connection SO_NONBLOCK state?

[William A. Rowe, Jr.]

Joe Orton wrote:


Dies how, what error, what is the server left doing?  This needs further 
diagnosis; truss the server, loglevel debug output, etc.  Does it work 
on the trunk?


Ok - I added the same disallow-empty brigade logic that's in the core
ap_rgetline_core() - just to prove up/prove down that this is a problem.

I also am trying to enable the score for mod_echo (man - at least in 2.0
the score interface really sucks for non-http, non-req based proto's.)

We immediately crash out with an empty brigade from the core, (no errno)
while on Linux, as you say, things worked fine.

Digging deeper.

Bill

Re: [PATCH] Re: Pluggable mod_log_config

[Ondrej Sury]

On Mon, 2005-10-03 at 15:40 -0400, Brian Akins wrote:
> Ondrej Sury wrote:
> 
> > Just quick thought, maybe we should add:
> > 
> > ap_log_writer_close *close;
> > 
> > to struct log_provider_t.  It's not absolutely necessary, because you
> > can use apr_pool_cleanup_register(...), but it will make writing addon
> > modules much cleaner.
> 
> That's debatable.  I already use apr_pool_cleanup_register in a lot of 
> stuff, so it seems "natural" to me.  I guess we could register a cleanup 
>   on each cls that called close or we could let the module developer do 
> it...  I'm leaning toward just letting others register the cleanup on 
> there own like buffered logs does now.

Disadvantage of having it outside cls->provider is that you have to keep
list of opened connections/files/sockets/etc. somewhere and you will be
duplicating this list if you keep it outside of multi_log_state.

Anyway it's not a big deal since connections to server will be propably
opened in init_child.

Ondrej.
-- 
Ondrej Sury <[EMAIL PROTECTED]>


signature.asc
Description: This is a digitally signed message part

Re: [PATCH] Re: Pluggable mod_log_config

[Colm MacCarthaigh]

On Mon, Oct 03, 2005 at 11:40:27AM -0400, Brian Akins wrote:
> Akins, Brian wrote:
> 
> >CustomLog mysql://something common env=images
> >CustomLog file:///logs/my.log combined
> >CustomLog spread://somegroup refere
> >CustomLog buffer:///logs/other.log common

I've been looking more at this, and I'm kind of confused as to what the
aim is here. The line;

"CustomLog mysql://something common env=images"

doesn't make a lot of sense to me. The "common" LogFormat is
line-oriented, but a database-driven logformat should be anything but
line-oriented. If it's to be of any real use it should have its own
schema and so on, each nugget of log information would probably be its
own field within a record, defined by a table, and so on. 

I think there's a lot more to think about, and I think these kind of
changes to CustomLog may hint at a need for slightly wider-ranging
changes, including to LogFormat. Or maybe strings will do, would we be
happy to see;

LogFormat "INSERT INTO foo VALUES ('%h', '%l');" foo-sql

But in order to save overhead, this would require some intelligence, it
would not make much sense for the pluggable logger to re-parse this
string everytime, to figure out what it should be doing. And where does
it get its database, username and host information from?  Do we require
per-provider directives? Do we hack the format more to get them in
somehow? 

And after all of this, what if any, are the compelling reasons to
implement this in httpd at all? Why can't all of this be moved into
piped loggers? Why can't they just parse the logs as they get them and
do whatever whoever wants with it after that? After all, many people use
"logger" to log to syslog. The whole mod_log_spread architecture with
two netcat commands, and that can even be done with privilege
seperation, which an in-httpd module never could.

I can understand that it is desirable to have non-blocking logging, but
there are means to deal with this, I've been testing some buffered (on
the piped logger side) piped logging that is I think more than good
enough in that regard. I can also understand that it's desirable to have
reliable logs from a httpd crash, or to guard against a piped logger
crash - but these are a question of code quality (of the piped logger)
more than anything else.

On ftp.heanet.ie, we've been using piped loggers forever, our logs are
damn busy, and we've seen our share of crashes, and these issues just
never arise for us.

What am I not getting?

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]

apache/sitemaps?

[Shiva Shivakumar]

Hello folks,



We (@Google) launched Sitemaps to optimize how crawlers work with
webservers, from a hit-or-miss approach to something more directed.
Currently, webcrawlers (including ours) do not know about all pages on
a webserver, or when they change. (A simple "ls -lR" in the ftp-world,
that we dont have in the web-world). Instead, our crawlers crawl pages
that are linked to from other pages and periodically check if they
change, like a random web surfer.
Some of the key aspects of our proposal include (a) a simple XML
protocol we released under Creative Commons 2.0 license so all
webservers, webmasters and search engines could benefit from a common
approach, and (b) an open-source sitemap generator in Python
(@sourceforge) that
produces Sitemaps automatically for some common use cases.

It's been about 4 months since we launched, and webmasters have been
using the Sitemaps protocol (and client) to give us URLs for both small (e.g, 100
urls) to large sites (e.g., 10M+ urls), so we figured it is time to
ping you guys. How do the Apache webserver folks react to something
like Sitemaps protocol being supported in Apache "out of the box" (e.g., as a
mod_sitemap) or shipping the sitemap_gen.py tool (or some variant) thro
http://httpd.apache.org/docs/2.1/programs/

as a support program (similar to htdigest or htdbm)? And in general,
offering additional mechanisms for webservers to help webcrawlers (an
increasing fraction of webserver activity) much more directly?thanks,
- shiva

-
Some links...1. About Sitemaps -- 
http://www.google.com/webmasters/sitemaps/docs/en/about.html2. Sitemaps protocol -- 

http://www.google.com/webmasters/sitemaps/docs/en/protocol.html3. Google released open source sitemap_gen.py -- 
http://www.google.com/webmasters/sitemaps/docs/en/sitemap-generator.html
4. Third party sitemap generators for webservers/CMS that currently support Sitemaps: 
http://code.google.com/sm_thirdparty.html

RE: apache/sitemaps?

[R, Rajesh (STSD)]

 
May be we need a poll on this ??
 
+1
 
--Rajesh R 
 


From: Shiva Shivakumar [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, October 04, 2005 9:39 AMTo: 
dev@httpd.apache.orgCc: Greg SteinSubject: 
apache/sitemaps?
Hello folks,We (@Google) launched Sitemaps to optimize 
how crawlers work with webservers, from a hit-or-miss approach to something more 
directed. Currently, webcrawlers (including ours) do not know about all pages on 
a webserver, or when they change. (A simple "ls -lR" in the ftp-world, that we 
dont have in the web-world). Instead, our crawlers crawl pages that are linked 
to from other pages and periodically check if they change, like a random web 
surfer.Some of the key aspects of our proposal include (a) 
a simple XML protocol we released under Creative Commons 2.0 license so all 
webservers, webmasters and search engines could benefit from a common approach, 
and (b) an open-source sitemap generator in Python (@sourceforge) that produces 
Sitemaps automatically for some common use cases.It's been about 
4 months since we launched, and webmasters have been using the Sitemaps protocol 
(and client) to give us URLs for both small (e.g, 100 urls) to large sites 
(e.g., 10M+ urls), so we figured it is time to ping you guys. How do the Apache 
webserver folks react to something like Sitemaps protocol being supported in 
Apache "out of the box" (e.g., as a mod_sitemap) or shipping the sitemap_gen.py 
tool (or some variant) thro http://httpd.apache.org/docs/2.1/programs/ as a support 
program (similar to htdigest or htdbm)? And in general, offering additional 
mechanisms for webservers to help webcrawlers (an increasing fraction of 
webserver activity) much more directly?thanks,- 
shiva-Some 
links...1. About Sitemaps -- http://www.google.com/webmasters/sitemaps/docs/en/about.html2. 
Sitemaps protocol -- http://www.google.com/webmasters/sitemaps/docs/en/protocol.html3. 
Google released open source sitemap_gen.py -- http://www.google.com/webmasters/sitemaps/docs/en/sitemap-generator.html 
4. Third party sitemap generators for webservers/CMS that 
currently support Sitemaps: http://code.google.com/sm_thirdparty.html

Re: apache/sitemaps?

[Paul Querna]

Shiva Shivakumar wrote:
> Some of the key aspects of our proposal include (a) a simple XML
> protocol we released under Creative Commons 2.0 license so all
> webservers, webmasters and search engines could benefit from a common
> approach, and (b) an open-source sitemap generator in Python
> (@sourceforge) that produces Sitemaps automatically for some common use
> cases.
> 
> It's been about 4 months since we launched, and webmasters have been
> using the Sitemaps protocol (and client) to give us URLs for both small
> (e.g, 100 urls) to large sites (e.g., 10M+ urls), so we figured it is
> time to ping you guys. How do the Apache webserver folks react to
> something like Sitemaps protocol being supported in Apache "out of the
> box" (e.g., as a mod_sitemap)

Write the code. Send Patches.  Then we have something to work with.  If
mod_sitemap turns into anything, it might be possible to find it a place
inside the ASF.

> or shipping the sitemap_gen.py tool (or
> some variant) thro http://httpd.apache.org/docs/2.1/programs/
> 
> as a support program (similar to htdigest or htdbm)? And in general,
> offering additional mechanisms for webservers to help webcrawlers (an
> increasing fraction of webserver activity) much more directly?

I am sort of -0.5 on including something like sitemap_gen.py with our
programs. Frankly content generation has always been kept separate from
the core of httpd.  Yes, there is a shift with helping automated tools
like a webcrawler find content in an automated way, but I am not sure if
httpd should play an active role in promoting it.

-Paul