Re: error log providers, multiple vhosts, mod_syslog
Am Dienstag, 12. November 2013, 13:33:23 schrieb Jan Kaluža: I think LDAPLibraryDebug is one user of stderr going to the error log. As the logging is done by the ldap library, there is really no way to change it. But I guess it would be acceptable if that works only if logging to a file (as long as logging to a file can be enabled even if the normal logging goes somewhere else via an error log provider). Hm, it does not work like that currently. You can have just one active error log provider (so you log only to file or only using provider). However, current situation is not a regression. Stderr output done by LDAPLibraryDebug has been lost already in httpd-2.2 if the admin used ErrorLog syslog. LDAPLibraryDebug is new in 2.4. The solution could be forking another process which would read from stderr and send it to error log provider, but I personally think it's not worth doing it. Another solution could be logging stderr to file and everything else using error log provider, but this way looks little bit messy to me... I think the StderrLog thing suggested by Jeff may be good idea, if is not too complex to implement. I don't have any other ideas.
Re: error log providers, multiple vhosts, mod_syslog
On 10/15/2013 05:27 PM, Jeff Trawick wrote:
Does this patch/commit look okay? It works for me with a simple
provider in different scenarios (vhost that inherits provider setup from
s_main, vhost that has its own setup).
I suppose mod_syslog needs to disallow any attempts to configure it in a
vhost with different settings since openlog() is process-wide. Jan, can
you look at that by chance?
I've done that in r1541029.
Jan Kaluza
-- Forwarded message --
From: ** traw...@apache.org mailto:traw...@apache.org
Date: Tue, Oct 15, 2013 at 10:09 AM
Subject: svn commit: r1532344 - /httpd/httpd/trunk/server/log.c
To: c...@httpd.apache.org mailto:c...@httpd.apache.org
Author: trawick
Date: Tue Oct 15 14:09:29 2013
New Revision: 1532344
URL: http://svn.apache.org/r1532344
Log:
Follow-up to r1525597:
Initialize error log providers in vhosts, solving crashes
when logging from those vhosts as well as allowing a different
provider (or provider configuration) for vhosts.
Modified:
httpd/httpd/trunk/server/log.c
Modified: httpd/httpd/trunk/server/log.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/trunk/server/log.c?rev=1532344r1=1532343r2=1532344view=diff
==
--- httpd/httpd/trunk/server/log.c (original)
+++ httpd/httpd/trunk/server/log.c Tue Oct 15 14:09:29 2013
@@ -458,6 +458,18 @@ int ap_open_logs(apr_pool_t *pconf, apr_
virt-error_log = q-error_log;
}
}
+else if (virt-errorlog_provider) {
+/* separately-configured vhost-specific provider */
+if (open_error_log(virt, 0, p) != OK) {
+return DONE;
+}
+}
+else if (s_main-errorlog_provider) {
+/* inherit provider from s_main */
+virt-errorlog_provider = s_main-errorlog_provider;
+virt-errorlog_provider_handle =
s_main-errorlog_provider_handle;
+virt-error_log = NULL;
+}
else {
virt-error_log = s_main-error_log;
}
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Re: error log providers, multiple vhosts, mod_syslog
On 11/11/2013 10:50 AM, Stefan Fritsch wrote: On Thu, 7 Nov 2013, Joe Orton wrote: On Thu, Oct 17, 2013 at 12:33:50PM +, Plüm, Rüdiger, Vodafone Group wrote: Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog_provider-writer to the log? Jumping in here... Actually there should be few cases where modules write to stderr, no? I'm not sure we should even consider writes to stderr get logged a valid part of the module API. If you're not using ap_log_*error, you're out of luck is not totally unreasonable. I think LDAPLibraryDebug is one user of stderr going to the error log. As the logging is done by the ldap library, there is really no way to change it. But I guess it would be acceptable if that works only if logging to a file (as long as logging to a file can be enabled even if the normal logging goes somewhere else via an error log provider). Hm, it does not work like that currently. You can have just one active error log provider (so you log only to file or only using provider). However, current situation is not a regression. Stderr output done by LDAPLibraryDebug has been lost already in httpd-2.2 if the admin used ErrorLog syslog. The solution could be forking another process which would read from stderr and send it to error log provider, but I personally think it's not worth doing it. Another solution could be logging stderr to file and everything else using error log provider, but this way looks little bit messy to me... Regards, Jan Kaluza
Re: error log providers, multiple vhosts, mod_syslog
On Tue, Nov 12, 2013 at 7:33 AM, Jan Kaluža jkal...@redhat.com wrote: On 11/11/2013 10:50 AM, Stefan Fritsch wrote: On Thu, 7 Nov 2013, Joe Orton wrote: On Thu, Oct 17, 2013 at 12:33:50PM +, Plüm, Rüdiger, Vodafone Group wrote: Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog_provider-writer to the log? Jumping in here... Actually there should be few cases where modules write to stderr, no? I'm not sure we should even consider writes to stderr get logged a valid part of the module API. If you're not using ap_log_*error, you're out of luck is not totally unreasonable. I think LDAPLibraryDebug is one user of stderr going to the error log. As the logging is done by the ldap library, there is really no way to change it. But I guess it would be acceptable if that works only if logging to a file (as long as logging to a file can be enabled even if the normal logging goes somewhere else via an error log provider). Hm, it does not work like that currently. You can have just one active error log provider (so you log only to file or only using provider). However, current situation is not a regression. Stderr output done by LDAPLibraryDebug has been lost already in httpd-2.2 if the admin used ErrorLog syslog. The solution could be forking another process which would read from stderr and send it to error log provider, but I personally think it's not worth doing it. Another solution could be logging stderr to file and everything else using error log provider, but this way looks little bit messy to me... I was thinking about adding StderrLog filename|none to be able to separate or discard the stuff that goes to stderr, whether or not a provider is being used. I think it would be useful in a more than one type of configuration. Provider in place? Acts like StderrLog none unless StderrLog is coded. No provider in place? stderr goes to the error log unless StderrLog is coded. A warning about StderrLog filename is that there is no built-in support for rotation (until/unless someone adds piped log support) and as such it is intended only to be active while collecting data for a problem, after which it should be disabled. Regards, Jan Kaluza -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: error log providers, multiple vhosts, mod_syslog
On Thu, 7 Nov 2013, Joe Orton wrote: On Thu, Oct 17, 2013 at 12:33:50PM +, Plüm, Rüdiger, Vodafone Group wrote: Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog_provider-writer to the log? Jumping in here... Actually there should be few cases where modules write to stderr, no? I'm not sure we should even consider writes to stderr get logged a valid part of the module API. If you're not using ap_log_*error, you're out of luck is not totally unreasonable. I think LDAPLibraryDebug is one user of stderr going to the error log. As the logging is done by the ldap library, there is really no way to change it. But I guess it would be acceptable if that works only if logging to a file (as long as logging to a file can be enabled even if the normal logging goes somewhere else via an error log provider).
Re: error log providers, multiple vhosts, mod_syslog
On Thu, Oct 17, 2013 at 12:33:50PM +, Plüm, Rüdiger, Vodafone Group wrote: Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog_provider-writer to the log? Jumping in here... Actually there should be few cases where modules write to stderr, no? I'm not sure we should even consider writes to stderr get logged a valid part of the module API. If you're not using ap_log_*error, you're out of luck is not totally unreasonable. mod_cgi does intercept stderr and logs it properly via ap_log_rerror, and mod_cgid does stderr logging differently anyway. One case which is common is the use of apr_procattr_child_errfn_set() to write exec() errors to stderr after forking a child. But perhaps we could and should provide a single common implementation of that which does something more useful. With ErrorLog syslog in current 2.x (and 1.3?), stderr is /dev/null already, so the status quo is that we lose stuff, though that is kind of ugly. Regards, Joe
Re: error log providers, multiple vhosts, mod_syslog
On 11/07/2013 11:11 AM, Joe Orton wrote:
On Thu, Oct 17, 2013 at 12:33:50PM +, Plüm, Rüdiger, Vodafone Group wrote:
Hmm. This points out another issue when using an error log provider for the
main server log:
We lose everything that the server or other programs like CGI-scripts write to
the stderr FD as it
is simply written to /dev/null. Don't we need to have a separate process in
this case that
like a piped logger reads from the reading end of the stderr pipe and writes
it
via ap_server_conf-errorlog_provider-writer to the log?
Jumping in here...
Actually there should be few cases where modules write to stderr, no?
I'm not sure we should even consider writes to stderr get logged a
valid part of the module API. If you're not using ap_log_*error,
you're out of luck is not totally unreasonable.
mod_cgi does intercept stderr and logs it properly via ap_log_rerror,
and mod_cgid does stderr logging differently anyway.
One case which is common is the use of apr_procattr_child_errfn_set() to
write exec() errors to stderr after forking a child. But perhaps we
could and should provide a single common implementation of that which
does something more useful.
With ErrorLog syslog in current 2.x (and 1.3?), stderr is /dev/null
already, so the status quo is that we lose stuff, though that is kind of
ugly.
Reading this I think I could commit my httpd-trunk-stderr-provider.patch
from older email in this thread. It does not fix stderr logging, but it
fixes httpd-trunk problem where messages with NULL server_rec* are not
logged.
I'm attaching better version of that patch here (it fixes potential
crash pointed out by Rüdiger when both logf and error_log_provider are
NULL).
If anyone is against it or has better idea how to fix logging of
messages with NULL server_rec* when log providers are used, tell me.
Regards, Joe
Regards,
Jan Kaluza
Index: server/log.c
===
--- server/log.c (revision 1539566)
+++ server/log.c (working copy)
@@ -437,9 +437,12 @@
#define NULL_DEVICE /dev/null
#endif
-if (replace_stderr freopen(NULL_DEVICE, w, stderr) == NULL) {
-ap_log_error(APLOG_MARK, APLOG_CRIT, errno, s_main, APLOGNO(00093)
- unable to replace stderr with %s, NULL_DEVICE);
+if (replace_stderr) {
+if (freopen(NULL_DEVICE, w, stderr) == NULL) {
+ap_log_error(APLOG_MARK, APLOG_CRIT, errno, s_main, APLOGNO(00093)
+unable to replace stderr with %s, NULL_DEVICE);
+}
+stderr_log = NULL;
}
for (virt = s_main-next; virt; virt = virt-next) {
@@ -1034,6 +1037,8 @@
const request_rec *rmain = NULL;
core_server_config *sconf = NULL;
ap_errorlog_info info;
+ap_errorlog_provider *errorlog_provider = NULL;
+void *errorlog_provider_handle = NULL;
/* do we need to log once-per-req or once-per-conn info? */
int log_conn_info = 0, log_req_info = 0;
@@ -1060,6 +1065,10 @@
#endif
logf = stderr_log;
+if (!logf ap_server_conf ap_server_conf-errorlog_provider) {
+errorlog_provider = ap_server_conf-errorlog_provider;
+errorlog_provider_handle = ap_server_conf-errorlog_provider_handle;
+}
}
else {
int configured_level = r ? ap_get_request_module_loglevel(r, module_index):
@@ -1078,6 +1087,9 @@
logf = s-error_log;
}
+errorlog_provider = s-errorlog_provider;
+errorlog_provider_handle = s-errorlog_provider_handle;
+
/* the faked server_rec from mod_cgid does not have s-module_config */
if (s-module_config) {
sconf = ap_get_core_module_config(s-module_config);
@@ -1106,6 +1118,14 @@
}
}
+if (!logf !errorlog_provider) {
+/* There is no file to send the log message to (or it is
+ * redirected to /dev/null and therefore any formating done below
+ * would be lost anyway) and there is no log provider available, so
+ * we just return here. */
+return;
+}
+
info.s = s;
info.c = c;
info.pool = pool;
@@ -1191,7 +1211,7 @@
continue;
}
-if (logf || (s-errorlog_provider-flags
+if (logf || (errorlog_provider-flags
AP_ERRORLOG_PROVIDER_ADD_EOL_STR)) {
/* Truncate for the terminator (as apr_snprintf does) */
if (len MAX_STRING_LEN - sizeof(APR_EOL_STR)) {
@@ -1205,8 +1225,8 @@
write_logline(errstr, len, logf, level_and_mask);
}
else {
-s-errorlog_provider-writer(info, s-errorlog_provider_handle,
- errstr, len);
+errorlog_provider-writer(info, errorlog_provider_handle,
+ errstr, len);
}
if (done) {
RE: error log providers, multiple vhosts, mod_syslog
Seems fine. Regards Rüdiger -Original Message- From: Jan Kaluža Sent: Donnerstag, 7. November 2013 13:09 To: dev@httpd.apache.org Subject: Re: error log providers, multiple vhosts, mod_syslog On 11/07/2013 11:11 AM, Joe Orton wrote: On Thu, Oct 17, 2013 at 12:33:50PM +, Plüm, Rüdiger, Vodafone Group wrote: Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog_provider-writer to the log? Jumping in here... Actually there should be few cases where modules write to stderr, no? I'm not sure we should even consider writes to stderr get logged a valid part of the module API. If you're not using ap_log_*error, you're out of luck is not totally unreasonable. mod_cgi does intercept stderr and logs it properly via ap_log_rerror, and mod_cgid does stderr logging differently anyway. One case which is common is the use of apr_procattr_child_errfn_set() to write exec() errors to stderr after forking a child. But perhaps we could and should provide a single common implementation of that which does something more useful. With ErrorLog syslog in current 2.x (and 1.3?), stderr is /dev/null already, so the status quo is that we lose stuff, though that is kind of ugly. Reading this I think I could commit my httpd-trunk-stderr-provider.patch from older email in this thread. It does not fix stderr logging, but it fixes httpd-trunk problem where messages with NULL server_rec* are not logged. I'm attaching better version of that patch here (it fixes potential crash pointed out by Rüdiger when both logf and error_log_provider are NULL). If anyone is against it or has better idea how to fix logging of messages with NULL server_rec* when log providers are used, tell me. Regards, Joe Regards, Jan Kaluza
Re: error log providers, multiple vhosts, mod_syslog
On Thu, Nov 7, 2013 at 7:09 AM, Jan Kaluža jkal...@redhat.com wrote: On 11/07/2013 11:11 AM, Joe Orton wrote: On Thu, Oct 17, 2013 at 12:33:50PM +, Plüm, Rüdiger, Vodafone Group wrote: Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog_provider-writer to the log? Jumping in here... Actually there should be few cases where modules write to stderr, no? I'm not sure we should even consider writes to stderr get logged a valid part of the module API. If you're not using ap_log_*error, you're out of luck is not totally unreasonable. mod_cgi does intercept stderr and logs it properly via ap_log_rerror, and mod_cgid does stderr logging differently anyway. One case which is common is the use of apr_procattr_child_errfn_set() to write exec() errors to stderr after forking a child. But perhaps we could and should provide a single common implementation of that which does something more useful. With ErrorLog syslog in current 2.x (and 1.3?), stderr is /dev/null already, so the status quo is that we lose stuff, though that is kind of ugly. Reading this I think I could commit my httpd-trunk-stderr-provider.patch from older email in this thread. It does not fix stderr logging, but it fixes httpd-trunk problem where messages with NULL server_rec* are not logged. I'm attaching better version of that patch here (it fixes potential crash pointed out by Rüdiger when both logf and error_log_provider are NULL). If anyone is against it or has better idea how to fix logging of messages with NULL server_rec* when log providers are used, tell me. Is the observation about NULL server_rec for modules that weren't updated properly to pass ap_server_conf when no server_rec was available, or for some window where ap_server_conf is NULL but server_rec is needed? Regards, Joe Regards, Jan Kaluza -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: error log providers, multiple vhosts, mod_syslog
On 11/07/2013 07:07 PM, Jeff Trawick wrote: On Thu, Nov 7, 2013 at 7:09 AM, Jan Kaluža jkal...@redhat.com mailto:jkal...@redhat.com wrote: On 11/07/2013 11:11 AM, Joe Orton wrote: On Thu, Oct 17, 2013 at 12:33:50PM +, Plüm, Rüdiger, Vodafone Group wrote: Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog___provider-writer to the log? Jumping in here... Actually there should be few cases where modules write to stderr, no? I'm not sure we should even consider writes to stderr get logged a valid part of the module API. If you're not using ap_log_*error, you're out of luck is not totally unreasonable. mod_cgi does intercept stderr and logs it properly via ap_log_rerror, and mod_cgid does stderr logging differently anyway. One case which is common is the use of apr_procattr_child_errfn_set() to write exec() errors to stderr after forking a child. But perhaps we could and should provide a single common implementation of that which does something more useful. With ErrorLog syslog in current 2.x (and 1.3?), stderr is /dev/null already, so the status quo is that we lose stuff, though that is kind of ugly. Reading this I think I could commit my httpd-trunk-stderr-provider.__patch from older email in this thread. It does not fix stderr logging, but it fixes httpd-trunk problem where messages with NULL server_rec* are not logged. I'm attaching better version of that patch here (it fixes potential crash pointed out by Rüdiger when both logf and error_log_provider are NULL). If anyone is against it or has better idea how to fix logging of messages with NULL server_rec* when log providers are used, tell me. Is the observation about NULL server_rec for modules that weren't updated properly to pass ap_server_conf when no server_rec was available, or for some window where ap_server_conf is NULL but server_rec is needed? I've originally observed that right in log.c in open_error_log(). ap_log_error calls there have NULL server_rec and in case something goes wrong during vhost logger initialization, the error messages about that are lost in case you use error log provider in ap_server_conf. That's what I'm trying to fix by that patch. I haven't checked if there are other places where NULL server_rec is used, but I would say it's pretty common during httpd startup. Jan Kaluza Regards, Joe Regards, Jan Kaluza -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: error log providers, multiple vhosts, mod_syslog
On Tue, Oct 29, 2013 at 7:04 AM, Jeff Trawick traw...@gmail.com wrote: On Tue, Oct 29, 2013 at 5:30 AM, Jan Kaluža jkal...@redhat.com wrote: On 10/25/2013 03:37 PM, Jeff Trawick wrote: I'm hoping to get back to this discussion soon. I also have a 2.4.x patch I've played with that is roughly in sync with trunk (though syslog logging support isn't a provider in the 2.4.x version). Does your patch fix stderr logging? I wanted to write proof of concept logger as described by Rüdiger below, but I didn't manage to find a time for it recently :(. If you have working solution for that, I wouldn't waste my time :). No, I just have a backport to 2.4.x with accommodation for syslog. What are the thoughts around handling stderr? In some cases it is just junk if not wrapped in a proper message (i.e., there can be a requirement that error log be completely parseable and/or a requirement that stderr has timestamps or other info). But it seems to require a thread of execution to implement that in order to avoid blocking the writer. (Recently I encountered a situation where a third-party module called some library during httpd init that wrote an initialization message directly to stderr, and the message was stuck in the stdio buffer and forked into every child process for later, multiple appearances :) ) I guess an error log provider could be given the opportunity to route stderr wherever it wants, but that doesn't handle the existing suckage of random, un-timestamped stuff in the log for normal configurations. I guess you could have mod_stderr that may or may not be enabled if you wanted stderr handled more cleanly (groan), whether or not you had a provider for ap_log calls. Thanks, Jan Kaluza On Thu, Oct 17, 2013 at 8:33 AM, Plüm, Rüdiger, Vodafone Group ruediger.pl...@vodafone.com mailto:ruediger.pl...@vodafone.com wrote: -Original Message- From: Jan Kaluža Sent: Donnerstag, 17. Oktober 2013 13:21 To: dev@httpd.apache.org mailto:dev@httpd.apache.org Subject: Re: error log providers, multiple vhosts, mod_syslog There's another problem with log providers and vhosts and I think I have no idea how to fix it without doing dirty hacks... The problem is with ap_open_logs function, which does following: 1. Main server log is opened (open_error_log()). If this log uses error log provider, s_main-error_log is set to NULL. 2. When there is no s_main-error_log, stderr is redirected to /dev/null. Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog_provider-writer to the log? 3. Error logs for vhosts are opened (another open_error_log()). If there is some problem when opening these logs, any ap_log_error() call is sent to /dev/null. Wouldn't it try to call s-errorlog_provider-writer in line 1196 and Segfault if s-errorlog_provider is NULL because of some bad classic configuration that simply failed? Regards Rüdiger -- Born in Roswell... married an alien... http://emptyhammock.com/ -- Born in Roswell... married an alien... http://emptyhammock.com/ -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: error log providers, multiple vhosts, mod_syslog
On Wed, 6 Nov 2013 10:05:25 -0500 Jeff Trawick traw...@gmail.com wrote: What are the thoughts around handling stderr? In some cases it is just junk if not wrapped in a proper message (i.e., there can be a requirement that error log be completely parseable and/or a requirement that stderr has timestamps or other info). But it seems to require a thread of execution to implement that in order to avoid blocking the writer. (Recently I encountered a situation where a third-party module called some library during httpd init that wrote an initialization message directly to stderr, and the message was stuck in the stdio buffer and forked into every child process for later, multiple appearances :) ) I guess an error log provider could be given the opportunity to route stderr wherever it wants, but that doesn't handle the existing suckage of random, un-timestamped stuff in the log for normal configurations. I guess you could have mod_stderr that may or may not be enabled if you wanted stderr handled more cleanly (groan), whether or not you had a provider for ap_log calls. The windows nt event logging logic faced the same issue (and was the reason for the early _commit() call :), so stderr is shifted to a pipe to 'decorate' it into the event log queue, which would look very similar for timestamps/other annotations into a syslog provider.
Re: error log providers, multiple vhosts, mod_syslog
On 10/25/2013 03:37 PM, Jeff Trawick wrote: I'm hoping to get back to this discussion soon. I also have a 2.4.x patch I've played with that is roughly in sync with trunk (though syslog logging support isn't a provider in the 2.4.x version). Does your patch fix stderr logging? I wanted to write proof of concept logger as described by Rüdiger below, but I didn't manage to find a time for it recently :(. If you have working solution for that, I wouldn't waste my time :). Thanks, Jan Kaluza On Thu, Oct 17, 2013 at 8:33 AM, Plüm, Rüdiger, Vodafone Group ruediger.pl...@vodafone.com mailto:ruediger.pl...@vodafone.com wrote: -Original Message- From: Jan Kaluža Sent: Donnerstag, 17. Oktober 2013 13:21 To: dev@httpd.apache.org mailto:dev@httpd.apache.org Subject: Re: error log providers, multiple vhosts, mod_syslog There's another problem with log providers and vhosts and I think I have no idea how to fix it without doing dirty hacks... The problem is with ap_open_logs function, which does following: 1. Main server log is opened (open_error_log()). If this log uses error log provider, s_main-error_log is set to NULL. 2. When there is no s_main-error_log, stderr is redirected to /dev/null. Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog_provider-writer to the log? 3. Error logs for vhosts are opened (another open_error_log()). If there is some problem when opening these logs, any ap_log_error() call is sent to /dev/null. Wouldn't it try to call s-errorlog_provider-writer in line 1196 and Segfault if s-errorlog_provider is NULL because of some bad classic configuration that simply failed? Regards Rüdiger -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: error log providers, multiple vhosts, mod_syslog
On Tue, Oct 29, 2013 at 5:30 AM, Jan Kaluža jkal...@redhat.com wrote: On 10/25/2013 03:37 PM, Jeff Trawick wrote: I'm hoping to get back to this discussion soon. I also have a 2.4.x patch I've played with that is roughly in sync with trunk (though syslog logging support isn't a provider in the 2.4.x version). Does your patch fix stderr logging? I wanted to write proof of concept logger as described by Rüdiger below, but I didn't manage to find a time for it recently :(. If you have working solution for that, I wouldn't waste my time :). No, I just have a backport to 2.4.x with accommodation for syslog. Thanks, Jan Kaluza On Thu, Oct 17, 2013 at 8:33 AM, Plüm, Rüdiger, Vodafone Group ruediger.pl...@vodafone.com mailto:ruediger.pluem@**vodafone.comruediger.pl...@vodafone.com wrote: -Original Message- From: Jan Kaluža Sent: Donnerstag, 17. Oktober 2013 13:21 To: dev@httpd.apache.org mailto:dev@httpd.apache.org Subject: Re: error log providers, multiple vhosts, mod_syslog There's another problem with log providers and vhosts and I think I have no idea how to fix it without doing dirty hacks... The problem is with ap_open_logs function, which does following: 1. Main server log is opened (open_error_log()). If this log uses error log provider, s_main-error_log is set to NULL. 2. When there is no s_main-error_log, stderr is redirected to /dev/null. Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog_**provider-writer to the log? 3. Error logs for vhosts are opened (another open_error_log()). If there is some problem when opening these logs, any ap_log_error() call is sent to /dev/null. Wouldn't it try to call s-errorlog_provider-writer in line 1196 and Segfault if s-errorlog_provider is NULL because of some bad classic configuration that simply failed? Regards Rüdiger -- Born in Roswell... married an alien... http://emptyhammock.com/ -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: error log providers, multiple vhosts, mod_syslog
I'm hoping to get back to this discussion soon. I also have a 2.4.x patch I've played with that is roughly in sync with trunk (though syslog logging support isn't a provider in the 2.4.x version). On Thu, Oct 17, 2013 at 8:33 AM, Plüm, Rüdiger, Vodafone Group ruediger.pl...@vodafone.com wrote: -Original Message- From: Jan Kaluža Sent: Donnerstag, 17. Oktober 2013 13:21 To: dev@httpd.apache.org Subject: Re: error log providers, multiple vhosts, mod_syslog There's another problem with log providers and vhosts and I think I have no idea how to fix it without doing dirty hacks... The problem is with ap_open_logs function, which does following: 1. Main server log is opened (open_error_log()). If this log uses error log provider, s_main-error_log is set to NULL. 2. When there is no s_main-error_log, stderr is redirected to /dev/null. Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog_provider-writer to the log? 3. Error logs for vhosts are opened (another open_error_log()). If there is some problem when opening these logs, any ap_log_error() call is sent to /dev/null. Wouldn't it try to call s-errorlog_provider-writer in line 1196 and Segfault if s-errorlog_provider is NULL because of some bad classic configuration that simply failed? Regards Rüdiger -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: error log providers, multiple vhosts, mod_syslog
There's another problem with log providers and vhosts and I think I have
no idea how to fix it without doing dirty hacks...
The problem is with ap_open_logs function, which does following:
1. Main server log is opened (open_error_log()). If this log uses error
log provider, s_main-error_log is set to NULL.
2. When there is no s_main-error_log, stderr is redirected to /dev/null.
3. Error logs for vhosts are opened (another open_error_log()). If there
is some problem when opening these logs, any ap_log_error() call is sent
to /dev/null.
The same happens for any further error message logged without server_rec *.
I think that what we need is to log to stderr_log only when it points to
valid location (real stderr or some real file), but when it's set to
/dev/null, we should use main server's error log provider.
I have created patch (attached) which does that, but I'm not satisfied
with the way how it works... Maybe someone has better idea how to fix
this issue?
Thanks,
Jan Kaluza
On 10/15/2013 05:27 PM, Jeff Trawick wrote:
Does this patch/commit look okay? It works for me with a simple
provider in different scenarios (vhost that inherits provider setup from
s_main, vhost that has its own setup).
I suppose mod_syslog needs to disallow any attempts to configure it in a
vhost with different settings since openlog() is process-wide. Jan, can
you look at that by chance?
-- Forwarded message --
From: ** traw...@apache.org mailto:traw...@apache.org
Date: Tue, Oct 15, 2013 at 10:09 AM
Subject: svn commit: r1532344 - /httpd/httpd/trunk/server/log.c
To: c...@httpd.apache.org mailto:c...@httpd.apache.org
Author: trawick
Date: Tue Oct 15 14:09:29 2013
New Revision: 1532344
URL: http://svn.apache.org/r1532344
Log:
Follow-up to r1525597:
Initialize error log providers in vhosts, solving crashes
when logging from those vhosts as well as allowing a different
provider (or provider configuration) for vhosts.
Modified:
httpd/httpd/trunk/server/log.c
Modified: httpd/httpd/trunk/server/log.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/trunk/server/log.c?rev=1532344r1=1532343r2=1532344view=diff
==
--- httpd/httpd/trunk/server/log.c (original)
+++ httpd/httpd/trunk/server/log.c Tue Oct 15 14:09:29 2013
@@ -458,6 +458,18 @@ int ap_open_logs(apr_pool_t *pconf, apr_
virt-error_log = q-error_log;
}
}
+else if (virt-errorlog_provider) {
+/* separately-configured vhost-specific provider */
+if (open_error_log(virt, 0, p) != OK) {
+return DONE;
+}
+}
+else if (s_main-errorlog_provider) {
+/* inherit provider from s_main */
+virt-errorlog_provider = s_main-errorlog_provider;
+virt-errorlog_provider_handle =
s_main-errorlog_provider_handle;
+virt-error_log = NULL;
+}
else {
virt-error_log = s_main-error_log;
}
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Index: server/log.c
===
--- server/log.c (revision 1532978)
+++ server/log.c (working copy)
@@ -435,9 +435,12 @@
#define NULL_DEVICE /dev/null
#endif
-if (replace_stderr freopen(NULL_DEVICE, w, stderr) == NULL) {
-ap_log_error(APLOG_MARK, APLOG_CRIT, errno, s_main, APLOGNO(00093)
- unable to replace stderr with %s, NULL_DEVICE);
+if (replace_stderr) {
+if (freopen(NULL_DEVICE, w, stderr) == NULL) {
+ap_log_error(APLOG_MARK, APLOG_CRIT, errno, s_main, APLOGNO(00093)
+unable to replace stderr with %s, NULL_DEVICE);
+}
+stderr_log = NULL;
}
for (virt = s_main-next; virt; virt = virt-next) {
@@ -1032,6 +1035,8 @@
const request_rec *rmain = NULL;
core_server_config *sconf = NULL;
ap_errorlog_info info;
+ap_errorlog_provider *errorlog_provider = NULL;
+void *errorlog_provider_handle = NULL;
/* do we need to log once-per-req or once-per-conn info? */
int log_conn_info = 0, log_req_info = 0;
@@ -1058,6 +1063,10 @@
#endif
logf = stderr_log;
+if (!logf ap_server_conf ap_server_conf-errorlog_provider) {
+errorlog_provider = ap_server_conf-errorlog_provider;
+errorlog_provider_handle = ap_server_conf-errorlog_provider_handle;
+}
}
else {
int configured_level = r ? ap_get_request_module_loglevel(r, module_index):
@@ -1076,6 +1085,9 @@
logf = s-error_log;
}
+errorlog_provider = s-errorlog_provider;
+errorlog_provider_handle = s-errorlog_provider_handle;
+
/* the faked server_rec from mod_cgid does not have s-module_config */
if (s-module_config) {
sconf =
RE: error log providers, multiple vhosts, mod_syslog
-Original Message- From: Jan Kaluža Sent: Donnerstag, 17. Oktober 2013 13:21 To: dev@httpd.apache.org Subject: Re: error log providers, multiple vhosts, mod_syslog There's another problem with log providers and vhosts and I think I have no idea how to fix it without doing dirty hacks... The problem is with ap_open_logs function, which does following: 1. Main server log is opened (open_error_log()). If this log uses error log provider, s_main-error_log is set to NULL. 2. When there is no s_main-error_log, stderr is redirected to /dev/null. Hmm. This points out another issue when using an error log provider for the main server log: We lose everything that the server or other programs like CGI-scripts write to the stderr FD as it is simply written to /dev/null. Don't we need to have a separate process in this case that like a piped logger reads from the reading end of the stderr pipe and writes it via ap_server_conf-errorlog_provider-writer to the log? 3. Error logs for vhosts are opened (another open_error_log()). If there is some problem when opening these logs, any ap_log_error() call is sent to /dev/null. Wouldn't it try to call s-errorlog_provider-writer in line 1196 and Segfault if s-errorlog_provider is NULL because of some bad classic configuration that simply failed? Regards Rüdiger
Re: error log providers, multiple vhosts, mod_syslog
On 10/15/2013 05:27 PM, Jeff Trawick wrote:
Does this patch/commit look okay? It works for me with a simple
provider in different scenarios (vhost that inherits provider setup from
s_main, vhost that has its own setup).
The patch looks OK to me. Thanks for it.
I suppose mod_syslog needs to disallow any attempts to configure it in a
vhost with different settings since openlog() is process-wide. Jan, can
you look at that by chance?
Yes, I will do this later today.
Regards,
Jan Kaluza
-- Forwarded message --
From: ** traw...@apache.org mailto:traw...@apache.org
Date: Tue, Oct 15, 2013 at 10:09 AM
Subject: svn commit: r1532344 - /httpd/httpd/trunk/server/log.c
To: c...@httpd.apache.org mailto:c...@httpd.apache.org
Author: trawick
Date: Tue Oct 15 14:09:29 2013
New Revision: 1532344
URL: http://svn.apache.org/r1532344
Log:
Follow-up to r1525597:
Initialize error log providers in vhosts, solving crashes
when logging from those vhosts as well as allowing a different
provider (or provider configuration) for vhosts.
Modified:
httpd/httpd/trunk/server/log.c
Modified: httpd/httpd/trunk/server/log.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/trunk/server/log.c?rev=1532344r1=1532343r2=1532344view=diff
==
--- httpd/httpd/trunk/server/log.c (original)
+++ httpd/httpd/trunk/server/log.c Tue Oct 15 14:09:29 2013
@@ -458,6 +458,18 @@ int ap_open_logs(apr_pool_t *pconf, apr_
virt-error_log = q-error_log;
}
}
+else if (virt-errorlog_provider) {
+/* separately-configured vhost-specific provider */
+if (open_error_log(virt, 0, p) != OK) {
+return DONE;
+}
+}
+else if (s_main-errorlog_provider) {
+/* inherit provider from s_main */
+virt-errorlog_provider = s_main-errorlog_provider;
+virt-errorlog_provider_handle =
s_main-errorlog_provider_handle;
+virt-error_log = NULL;
+}
else {
virt-error_log = s_main-error_log;
}
--
Born in Roswell... married an alien...
http://emptyhammock.com/
