RE: piped log files
Hi, Maybe this is something to do with a bug that I just reported. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=25103 The bug I reported is actually in the APR, it seems to cause very strange things to happen. What's happening, it seems, is that Apache calls apr_proc_create() to spawn off a new process, and the caller doesn't want to send a list of environment variables (char *env[]). But, apr_proc_create() still ends up calling execve() and provides a NULL for the char *env[] argument. That seems to cause weird things to happen, including segmentation faults when Apache tries to spawn off rotatelogs. Here's a simple diff that I did for a simple patch I've done: diff ./httpd-2.0.48/srclib/apr/threadproc/unix/proc.c ./httpd-2.0.48.fix/srclib/apr/threadproc/unix/proc.c 482c482 < execve(progname, (char * const *)args, (char * const *)env); --- > execv(progname, (char * const *)args); 489c489 < execv(progname, (char * const *)args); --- > execve(progname, (char * const *)args, (char * const *)env); Maybe it'll help. Thanks, Mike C. -Original Message- From: Bastiaan van der Put [mailto:[EMAIL PROTECTED] Sent: Sunday, November 30, 2003 6:07 AM To: [EMAIL PROTECTED] Subject: Re: piped log files At 14:50 11/30/2003, you wrote: >Bastiaan van der Put wrote: > >>Been using the patch for a while now. >>But it seems not only some notice/error in the error logs >>piped log program '/usr/local/apache2/bin/logresolve >> >>/home/accounts/x/xx/logs/access_log' failed unexpectedly > >Does that happen when you stop/restart apache, or can it happen at any >time (steady state)? No only when stopping/starting that is offcource. >>But also the program keeps running after stopping/starting apache. >>So got a load of them. > >Hopefully this undesired restarting is due to PR 24805 > >http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24805 > >which has a patch too. (groan) It looks indeed that bug to me, I use prefork also. I will test the patch later on. Greetings, Bas
Re: piped log files
At 14:50 11/30/2003, you wrote: Bastiaan van der Put wrote: Been using the patch for a while now. But it seems not only some notice/error in the error logs piped log program '/usr/local/apache2/bin/logresolve >> /home/accounts/x/xx/logs/access_log' failed unexpectedly Does that happen when you stop/restart apache, or can it happen at any time (steady state)? No only when stopping/starting that is offcource. But also the program keeps running after stopping/starting apache. So got a load of them. Hopefully this undesired restarting is due to PR 24805 http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24805 which has a patch too. (groan) It looks indeed that bug to me, I use prefork also. I will test the patch later on. Greetings, Bas
Re: piped log files
At 14:50 11/30/2003, you wrote: Bastiaan van der Put wrote: Been using the patch for a while now. But it seems not only some notice/error in the error logs piped log program '/usr/local/apache2/bin/logresolve >> /home/accounts/x/xx/logs/access_log' failed unexpectedly Does that happen when you stop/restart apache, or can it happen at any time (steady state)? No only when starting/stopping. But also the program keeps running after stopping/starting apache. So got a load of them. Hopefully this undesired restarting is due to PR 24805 http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24805 which has a patch too. (groan) It looks indeed that bug to me, I use prefork also. I will test the patch later on. Greetings, Bas
Re: piped log files
Bastiaan van der Put wrote: Been using the patch for a while now. But it seems not only some notice/error in the error logs piped log program '/usr/local/apache2/bin/logresolve >> /home/accounts/x/xx/logs/access_log' failed unexpectedly Does that happen when you stop/restart apache, or can it happen at any time (steady state)? But also the program keeps running after stopping/starting apache. So got a load of them. Hopefully this undesired restarting is due to PR 24805 http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24805 which has a patch too. (groan)
Re: piped log files
Been using the patch for a while now on 2.048 The patch was posted on the list on 2003-11-01 by Jeff Trawick. Index: server/log.c === RCS file: /home/cvs/httpd-2.0/server/log.c,v retrieving revision 1.135 diff -u -r1.135 log.c --- server/log.c14 Jul 2003 14:48:40 - 1.135 +++ server/log.c1 Nov 2003 15:13:35 - @@ -743,6 +743,8 @@ apr_status_t status; if (((status = apr_procattr_create(&procattr, pl->p)) != APR_SUCCESS) || +((status = apr_procattr_cmdtype_set(procattr, APR_SHELLCMD)) + != APR_SUCCESS) || ((status = apr_procattr_child_in_set(procattr, ap_piped_log_read_fd(pl), ap_piped_log_write_fd(pl))) But it seems not only some notice/error in the error logs piped log program '/usr/local/apache2/bin/logresolve >> /home/accounts/x/xx/logs/access_log' failed unexpectedly But also the program keeps running after stopping/starting apache. So got a load of them. Greetings, Bas
Re: piped log files
Been using the patch for a while now. But it seems not only some notice/error in the error logs piped log program '/usr/local/apache2/bin/logresolve >> /home/accounts/x/xx/logs/access_log' failed unexpectedly But also the program keeps running after stopping/starting apache. So got a load of them. Greetings, Bas At 20:23 11/7/2003, you wrote: Glenn wrote: On Fri, Nov 07, 2003 at 05:57:25AM -0500, Jeff Trawick wrote: Glenn wrote: Maybe out in left field, but could ErrorLog be moved from the core and into mod_log_config? Then, if mod_log_config is not present, or no ErrorLog directive is specified, then stderr can be used. Any special reason to keep it in the core? because it is a critical function? If it is able to format write to stderr if no module has taken over the function, most of the existing support is required anyway. And if core code does the work of formatting, then there is no use in an external module to have to do that. Not so. You said yourself that the code that handles piped logs is in two places, once in the core and once in mod_log_config, and this prevented you from committing a patch which only fixed a problem in one of those places. no, I was trying to simplify this situation since I didn't figure anybody cared about the detail :) it would have been helpful if I had been more explicit both sets of code to handle piped loggers are in core... one set is *on behalf of* plug-ins such as mod_log_config >I'm still trying to get a 4 line patch applied to Apache 1.3 > error logging code. what/where is that? repost to [EMAIL PROTECTED] >I was thinking along the lines of mod_log_config exporting a function >to handle piped logs. what if somebody doesn't want the mainline function of mod_log_config and instead has a different logging module? does that different logging module have to implement the piped log feature also? > Main point: duplicated code == BAD. understood, it just happens that it isn't a duplication between mod_log_config and core, but a duplication of core and core... certainly combining as much as possible is a good thing to do... personally I won't be doing any such thing until I know why the broken flavor is broken ;)
Re: piped log files
Glenn wrote: On Fri, Nov 07, 2003 at 05:57:25AM -0500, Jeff Trawick wrote: Glenn wrote: Maybe out in left field, but could ErrorLog be moved from the core and into mod_log_config? Then, if mod_log_config is not present, or no ErrorLog directive is specified, then stderr can be used. Any special reason to keep it in the core? because it is a critical function? If it is able to format write to stderr if no module has taken over the function, most of the existing support is required anyway. And if core code does the work of formatting, then there is no use in an external module to have to do that. Not so. You said yourself that the code that handles piped logs is in two places, once in the core and once in mod_log_config, and this prevented you from committing a patch which only fixed a problem in one of those places. no, I was trying to simplify this situation since I didn't figure anybody cared about the detail :) it would have been helpful if I had been more explicit both sets of code to handle piped loggers are in core... one set is *on behalf of* plug-ins such as mod_log_config >I'm still trying to get a 4 line patch applied to Apache 1.3 > error logging code. what/where is that? repost to [EMAIL PROTECTED] >I was thinking along the lines of mod_log_config exporting a function >to handle piped logs. what if somebody doesn't want the mainline function of mod_log_config and instead has a different logging module? does that different logging module have to implement the piped log feature also? > Main point: duplicated code == BAD. understood, it just happens that it isn't a duplication between mod_log_config and core, but a duplication of core and core... certainly combining as much as possible is a good thing to do... personally I won't be doing any such thing until I know why the broken flavor is broken ;)
Re: piped log files
On Fri, Nov 07, 2003 at 05:57:25AM -0500, Jeff Trawick wrote: > Glenn wrote: > >Maybe out in left field, but could ErrorLog be moved from the core > >and into mod_log_config? Then, if mod_log_config is not present, > >or no ErrorLog directive is specified, then stderr can be used. > >Any special reason to keep it in the core? > > because it is a critical function? > > If it is able to format write to stderr if no module has taken over the > function, most of the existing support is required anyway. And if core > code does the work of formatting, then there is no use in an external > module to have to do that. Not so. You said yourself that the code that handles piped logs is in two places, once in the core and once in mod_log_config, and this prevented you from committing a patch which only fixed a problem in one of those places. The point of my message was to suggest getting rid of the duplicated code and moving the ErrorLog directive to mod_log_config (or elsewhere). I am not suggesting moving all error logging out of the core since I agree that it is critical, just moving the interface to direct log output elsewhere (other than stderr). > Apache 2.0 did add an error log hook so that modules can grab error log > entries and log them however they want. So some mod_log_error (why clutter > up mod_log_config) could handle error log; we'd just need a way to turn off > the default error log. I haven't looked at the Apache2 error logging code carefully -- I'm still trying to get a 4 line patch applied to Apache 1.3 error logging code. I was thinking along the lines of mod_log_config exporting a function to handle piped logs. If it exists, the core would send the formatted error message to that filter instead of to stderr. Of course, this function could even be abstracted from mod_log_config into a "mod_log_gw" log gateway which would take a formatted string and send it to a file, a pipe, redirected >> output, a database, etc. Main point: duplicated code == BAD. Cheers, Glenn
Re: piped log files
Glenn wrote: On Thu, Nov 06, 2003 at 11:02:16AM -0500, Jeff Trawick wrote: Jeff Trawick wrote: Bastiaan van der Put wrote: CustomLog "|/usr/local/apache2/bin/logresolve >> /home/accounts/x/x/logs/access_log" combined unless somebody speaks up soon, I'll commit the patch to Apache 2.1-dev status: I hit a bit of a hangup that must be debugged :( there are two pieces of code handling piped loggers: error log and mod_log_config... in 1.3, both handled shell operations... in 2.0, as you discovered, neither handles shell operations the patch you are using is fine AFAICT, but trying the same change to the code to handle piped error log results in quite a bit of unhappiness (no logging at all)... that needs to be tracked down before trying to fix half the problem Maybe out in left field, but could ErrorLog be moved from the core and into mod_log_config? Then, if mod_log_config is not present, or no ErrorLog directive is specified, then stderr can be used. Any special reason to keep it in the core? because it is a critical function? If it is able to format write to stderr if no module has taken over the function, most of the existing support is required anyway. And if core code does the work of formatting, then there is no use in an external module to have to do that. Apache 2.0 did add an error log hook so that modules can grab error log entries and log them however they want. So some mod_log_error (why clutter up mod_log_config) could handle error log; we'd just need a way to turn off the default error log.
Re: piped log files
On Thu, Nov 06, 2003 at 11:02:16AM -0500, Jeff Trawick wrote: > Jeff Trawick wrote: > >Bastiaan van der Put wrote: > > > >>CustomLog "|/usr/local/apache2/bin/logresolve >> > >>/home/accounts/x/x/logs/access_log" combined > > > > > >unless somebody speaks up soon, I'll commit the patch to Apache 2.1-dev > > status: I hit a bit of a hangup that must be debugged :( > > there are two pieces of code handling piped loggers: error log and > mod_log_config... in 1.3, both handled shell operations... in 2.0, as you > discovered, neither handles shell operations > > the patch you are using is fine AFAICT, but trying the same change to the > code to handle piped error log results in quite a bit of unhappiness (no > logging at all)... that needs to be tracked down before trying to fix half > the problem Maybe out in left field, but could ErrorLog be moved from the core and into mod_log_config? Then, if mod_log_config is not present, or no ErrorLog directive is specified, then stderr can be used. Any special reason to keep it in the core? Thanks. Glenn
Re: piped log files
Jeff Trawick wrote: Bastiaan van der Put wrote: CustomLog "|/usr/local/apache2/bin/logresolve >> /home/accounts/x/x/logs/access_log" combined unless somebody speaks up soon, I'll commit the patch to Apache 2.1-dev status: I hit a bit of a hangup that must be debugged :( there are two pieces of code handling piped loggers: error log and mod_log_config... in 1.3, both handled shell operations... in 2.0, as you discovered, neither handles shell operations the patch you are using is fine AFAICT, but trying the same change to the code to handle piped error log results in quite a bit of unhappiness (no logging at all)... that needs to be tracked down before trying to fix half the problem
Re: piped log files
Bastiaan van der Put wrote: CustomLog "|/usr/local/apache2/bin/logresolve >> /home/accounts/x/x/logs/access_log" combined unless somebody speaks up soon, I'll commit the patch to Apache 2.1-dev Works now, When stopping apache i still get piped log program '/usr/local/apache2/bin/logresolve >> /home/accounts/x/xx/logs/access_log' failed unexpectedly right, due to unrelated problem
Re: piped log files
Hi, CustomLog "|/usr/local/apache2/bin/logresolve >> /home/accounts/x/x/logs/access_log" combined Works now, When stopping apache i still get piped log program '/usr/local/apache2/bin/logresolve >> /home/accounts/x/xx/logs/access_log' failed unexpectedly Greetings, Bas At 16:17 11/1/2003, you wrote: Bastiaan van der Put wrote: Piped log files still dont work with apache 2.0.48 piped log program '/usr/local/apache2/bin/logresolve >> /home/accounts/x//logs/access_log' failed unexpectedly So i am using a script as suggested before , called it apacheresolve : #!/bin/sh exec /usr/local/apache2/bin/logresolve >>/tmp/lr.out I still have this patch in my tree for this feature. ISTR wondering what other people thought about it, and forgetting about the issue until you just brought it up again. Any concerns out there? Index: server/log.c === RCS file: /home/cvs/httpd-2.0/server/log.c,v retrieving revision 1.135 diff -u -r1.135 log.c --- server/log.c14 Jul 2003 14:48:40 - 1.135 +++ server/log.c1 Nov 2003 15:13:35 - @@ -743,6 +743,8 @@ apr_status_t status; if (((status = apr_procattr_create(&procattr, pl->p)) != APR_SUCCESS) || +((status = apr_procattr_cmdtype_set(procattr, APR_SHELLCMD)) + != APR_SUCCESS) || ((status = apr_procattr_child_in_set(procattr, ap_piped_log_read_fd(pl), ap_piped_log_write_fd(pl))) Any comments from the audience? When stopping apache I also get : piped log program '/usr/local/apache2/bin/apacheresolve >> /home/accounts/x/x/logs/access_log' failed unexpectedly There is a PR for this issue. It is perhaps timing related, as it does not affect everybody.
Re: piped log files
Bastiaan van der Put wrote: Piped log files still dont work with apache 2.0.48 piped log program '/usr/local/apache2/bin/logresolve >> /home/accounts/x//logs/access_log' failed unexpectedly So i am using a script as suggested before , called it apacheresolve : #!/bin/sh exec /usr/local/apache2/bin/logresolve >>/tmp/lr.out I still have this patch in my tree for this feature. ISTR wondering what other people thought about it, and forgetting about the issue until you just brought it up again. Any concerns out there? Index: server/log.c === RCS file: /home/cvs/httpd-2.0/server/log.c,v retrieving revision 1.135 diff -u -r1.135 log.c --- server/log.c14 Jul 2003 14:48:40 - 1.135 +++ server/log.c1 Nov 2003 15:13:35 - @@ -743,6 +743,8 @@ apr_status_t status; if (((status = apr_procattr_create(&procattr, pl->p)) != APR_SUCCESS) || +((status = apr_procattr_cmdtype_set(procattr, APR_SHELLCMD)) + != APR_SUCCESS) || ((status = apr_procattr_child_in_set(procattr, ap_piped_log_read_fd(pl), ap_piped_log_write_fd(pl))) Any comments from the audience? When stopping apache I also get : piped log program '/usr/local/apache2/bin/apacheresolve >> /home/accounts/x/x/logs/access_log' failed unexpectedly There is a PR for this issue. It is perhaps timing related, as it does not affect everybody.
