Re: [PATCH] Make error logging modular
- Original Message - Hello Jan, Is there any reason we shouldn't do this in trunk? I don't see any reason. This patch was intended for trunk, but I don't have svn commit access, so I'm sending patches to this list :). It's also better that someone reviews my code, because I don't have so long experience with httpd development. Regards, Jan Kaluza The patches and features seem generally correct to me with a cursory review. Thanks, Paul On Mon, May 27, 2013 at 3:23 AM, Jan Kaluža jkal...@redhat.com wrote: Hi, last week I was trying to write my own module to log error_log to systemd-journal [1] and I've found out that with the current error_log code, it's not possible to do that properly. I was able to use error_log hook, but there is no way to disable creation of ErrorLog file (One can set it to /dev/null, but httpd will still write data to it without any reason). Syslog logger fixes that by hardcoding syslog methods in log.c/core.c, but I don't think that's the right thing to do with journald. Therefore, I've created following patches: http://people.apache.org/~jkaluza/patches/logging/ Their descriptions should be clear from their names, but I will describe them briefly here too. Patch 0001 declares ap_errorlog_provider which can be implemented by module providing error_log logger. Admin can later define ErrorLog provider arg to choose particular errorlog provider. Old syntax still works and the change is backward compatible. This patch also removes syslog logging from log.c (it is moved to newly created mod_syslog.c in next patch) Patch 0002 creates mod_syslog.c which uses the new API to implement syslog logging. It works the same way as the version in log.c I removed in previous patch, but it's in separate module. Patch 0003 shows how mod_journald.c can use the existing API. This module works well with systemd-journal, but unfortunately the performance of systemd-journal daemon is poor so far [2], but I presume it will be fixed and the module will be usable for general use in the future. There is probably no real benefit in accepting this last patch right now. It's here to only show why the previous two patches are useful. Note that this is my first bigger patch touching httpd core, so feel free to correct my possible mistakes... :) [1] http://0pointer.de/blog/projects/journalctl.html [2] https://bugzilla.redhat.com/show_bug.cgi?id=963620 [patches] http://people.apache.org/~jkaluza/patches/logging/ Regards, Jan Kaluza
Wanted: Orientation on interaction between filters and subrequests
Hello,
Apologies in advance for the potential repeat; I initially posted to
modules-dev but realized this is more of an internals problem.
I'm trying to write a module that does what one would intuitively
expect from the following mod_rewrite incantation:
RewriteCond %{REQUEST_URI} !-U
RewriteRule (.*) http://another.host$1 [P,NS]
In other words, If the resource (powered by an arbitrary back-end)
can't be found on this server, reverse-proxy the request to
another.host. —which due to its design, can't be done by mod_rewrite.
What I want this functionality for is a sort of scaffolding through
which a website can be replaced incrementally, resource by resource,
without having to know anything about the middleware(s) of either the
old or the new server (see
http://doriantaylor.com/the-redesign-dissolved for context).
I'm hoping somebody can spell out for me, or point me in the direction
of resources on how to better understand the design of the bucket
brigade I/O and how it interacts with subrequests.
Here is my strategy so far:
1) Start with a fixup handler rigged to run only on main requests
2) The fixup handler performs a subrequest on the same URI as the main request
3) If the lookup response is 404, set the response handler to
proxy-handler and r-filename to proxy:... and return OK
4) Otherwise, attach an output filter to the subrequest that (somehow)
blocks writes to the network and run it
5) Repeat step 3.
6) If we're still here, set the response handler to a dummy handler
with its own output filter that unblocks the output and return OK.
(Note, for simplicity's sake I am leaving out dealing with request
bodies for the moment, but the plan is tentatively to 'tee' them to a
temporary file and then replay them into the proxy request if
applicable.)
(I should also note that unless there is a particularly compelling
alternative I have overlooked, I have good reason to be confident that
this conditional reverse proxy should manifest as an Apache module.)
Where I'm fuzzy is the somehow of pausing the output to the
network from the subrequest and resuming it in the response handler in
the main request. I have a prototype which currently runs the local
request twice: first in the subrequest which I discard, and again
(notwithstanding being diverted to the reverse proxy) in the main
request which I leave untouched (recall, I am performing this
operation in the fixup phase). Inefficiency aside, this solution is
inadequate because of its potential to corrupt application state in
dynamic resources. And of course, the only reason why I have to run
the subrequest at all, is because it appears the 404 status is more
often than not set by a module's response handler.
(I am also not clear about what happens under the hood with respect to
the subrequest's header set/protocol data, and whether or I have to
manually pull it up into the main request.)
I considered putting an EOS bucket at the front of the subrequest's
output brigade, and then popping it off in an output filter in the
main request, but I'm not sure how nasty a hack that would be and/or
what side effects might arise from a trick like that. Alternatively, I
suppose I could write the subrequest's content to another temporary
file and then just replay it in the response handler, but I'd prefer
to avoid creating any unnecessary I/O.
Thanks in advance for any orientation, existing code that behaves
similarly, or any other advice anybody can share.
Regards,
--
Dorian Taylor
http://doriantaylor.com/
[ANNOUNCEMENT] Apache HTTP Server (httpd) 2.4.6 Released
Apache HTTP Server 2.4.6 Released The Apache Software Foundation and the Apache HTTP Server Project are pleased to announce the release of version 2.4.6 of the Apache HTTP Server (Apache). This version of Apache is our latest GA release of the new generation 2.4.x branch of Apache HTTPD and represents fifteen years of innovation by the project, and is recommended over all previous releases. This release of Apache is principally a security and bug fix release, including the following 2 security fixes: *) SECURITY: CVE-2013-1896 (cve.mitre.org) Sending a MERGE request against a URI handled by mod_dav_svn with the source href (sent as part of the request body as XML) pointing to a URI that is not configured for DAV will trigger a segfault. *) SECURITY: CVE-2013-2249 (cve.mitre.org) mod_session_dbd: Make sure that dirty flag is respected when saving sessions, and ensure the session ID is changed each time the session changes. This changes the format of the updatesession SQL statement. Existing configurations must be changed. Also in this release are some exciting new features including: *) Major updates to mod_lua *) Support for proxying websocket requests *) Higher performant shm-based cache implementation *) Addition of mod_macro for easier configuration management As well as several exciting fixes, especially those related to RFC edge cases in mod_cache and mod_proxy. NOTE: Apache httpd 2.4.5 was not released. We consider this release to be the best version of Apache available, and encourage users of all prior versions to upgrade. Apache HTTP Server 2.4.6 is available for download from: http://httpd.apache.org/download.cgi Apache 2.4 offers numerous enhancements, improvements, and performance boosts over the 2.2 codebase. For an overview of new features introduced since 2.4 please see: http://httpd.apache.org/docs/trunk/new_features_2_4.html Please see the CHANGES_2.4 file, linked from the download page, for a full list of changes. A condensed list, CHANGES_2.4.6 includes only those changes introduced since the prior 2.4 release. A summary of all of the security vulnerabilities addressed in this and earlier releases is available: http://httpd.apache.org/security/vulnerabilities_24.html This release requires the Apache Portable Runtime (APR) version 1.4.x and APR-Util version 1.4.x. The APR libraries must be upgraded for all features of httpd to operate correctly. This release builds on and extends the Apache 2.2 API. Modules written for Apache 2.2 will need to be recompiled in order to run with Apache 2.4, and require minimal or no source code changes. http://svn.apache.org/repos/asf/httpd/httpd/trunk/VERSIONING When upgrading or installing this version of Apache, please bear in mind that if you intend to use Apache with one of the threaded MPMs (other than the Prefork MPM), you must ensure that any modules you will be using (and the libraries they depend on) are thread-safe.
Re: Anyone interested in a patch to mod_fcgid(with pay)
On Sat, 20 Jul 2013 10:39:20 +0800 (CST) Pqf 潘庆峰 p...@mailtech.cn wrote: Hi, guys A company need a TCP/IP patch of mod_fcgid or alternative, and will pay for it, anyone interested? I really like to take it but I don't have too much time... Anyone interested please reply to me and I will forward the email address of them. ... Neti only listens on TCP/IP socket, it assumes both an authorizer role and a responder role in the Fast CGI request. Here's the 3 candidate Apache modules to interface Neti: ... 2. Mod_proxy_fcgi: this module supports TCP socket, it can connect to Neti, but it doesn't support authorizer role. So in the first FCGI request, it forwards the request to Neti as a responder instead of an authorizer, Neti cannot simply let it through without properly authorizing it first, thus the request fails; 3. Mod_fcgid: this module supports authorizer role while doesn't support TCP connection. We cannot confirm its authorizer role since it doesn't even connect to Neti due to lack of TCP; So our choice is between either adding authorizer role to mod_proxy_fcgi or adding TCP/IP to mod_fcgid. We’re really willing to pay to have this project done, I mean either adding proxy to mod_fcgid or adding authorizer to mod_proxy_fcgi. Are you willing to work on this with reward or do you know anyone else who’s interested in doing so with pay, for example, author of mod_proxy_fcgi? (I cannot find his name) No cycles myself at this instant, but it seems overtime to break apart the mod_fcgid process-control so that it can then stack on top of a single mod_proxy_fcgi communications pipe, dispatched through different fcgi-stream methods (including child process stdio), including the various authn/authz roles. It would take more time to refactor it in this way, but would get rid of any discrepancies between proxy_fcgi and fcgid and serve as a good excuse to draw the remaining mod_fcgid development back into trunk/, now that fcgid is generally sufficient for 2.2 users.
Re: Linux: CAP_DAC_OVERRIDE needed - why?
On Sun, 21 Jul 2013 00:15:45 +0200 Reindl Harald h.rei...@thelounge.net wrote: but why does httpd need CAP_DAC_OVERRIDE while starting initially as root? CapabilityBoundingSet=CAP_IPC_LOCK CAP_NET_BIND_SERVICE CAP_SETGID CAP_SETUID Jul 21 00:04:01 srv-rhsoft httpd[8813]: AH00112: Warning: DocumentRoot [/mnt/data/www/www] does not exist Jul 21 00:04:01 srv-rhsoft httpd[8813]: AH00112: Warning: DocumentRoot [/mnt/data/www/private] does not exist Could one of the parents /mnt .../data .../www offer no other-traverse (x) access? If so, these need to be both root and switch-to-user traversable and perhaps readable.
Re: Hey Steinar... Re: Revisiting the pre_htaccess hook
On Wed, Jul 10, 2013 at 03:07:56PM -0400, Jeff Trawick wrote: I guess it seems to work in the earlier e-mail is the validation that the API is sufficient for MPM-ITK. Hi, I see that 2.4.6 has been released, with no mention of open_htaccess in the source code. Was this reverted after 2.4.5? Or was it simply never applied to the 2.4.x branch? /* Steinar */ -- Homepage: http://www.sesse.net/
Re: Linux: CAP_DAC_OVERRIDE needed - why?
Am 22.07.2013 17:01, schrieb William A. Rowe Jr.: On Sun, 21 Jul 2013 00:15:45 +0200 Reindl Harald h.rei...@thelounge.net wrote: but why does httpd need CAP_DAC_OVERRIDE while starting initially as root? CapabilityBoundingSet=CAP_IPC_LOCK CAP_NET_BIND_SERVICE CAP_SETGID CAP_SETUID Jul 21 00:04:01 srv-rhsoft httpd[8813]: AH00112: Warning: DocumentRoot [/mnt/data/www/www] does not exist Jul 21 00:04:01 srv-rhsoft httpd[8813]: AH00112: Warning: DocumentRoot [/mnt/data/www/private] does not exist Could one of the parents /mnt .../data .../www offer no other-traverse (x) access? If so, these need to be both root and switch-to-user traversable and perhaps readable *bingo* not that way - some had 770 while owner/group apache:apache so at least questionable why the warning happens anyways but after change to 775 it is gone signature.asc Description: OpenPGP digital signature
Re: Linux: CAP_DAC_OVERRIDE needed - why?
If it was 770 apache:apache, then root had no access, and root (before processing the User directive) was 'unable' to verify the existence of the child directory without violating the apparent access control (not traditional access control, of course). On Mon, Jul 22, 2013 at 1:08 PM, Reindl Harald h.rei...@thelounge.netwrote: Am 22.07.2013 17:01, schrieb William A. Rowe Jr.: On Sun, 21 Jul 2013 00:15:45 +0200 Reindl Harald h.rei...@thelounge.net wrote: but why does httpd need CAP_DAC_OVERRIDE while starting initially as root? CapabilityBoundingSet=CAP_IPC_LOCK CAP_NET_BIND_SERVICE CAP_SETGID CAP_SETUID Jul 21 00:04:01 srv-rhsoft httpd[8813]: AH00112: Warning: DocumentRoot [/mnt/data/www/www] does not exist Jul 21 00:04:01 srv-rhsoft httpd[8813]: AH00112: Warning: DocumentRoot [/mnt/data/www/private] does not exist Could one of the parents /mnt .../data .../www offer no other-traverse (x) access? If so, these need to be both root and switch-to-user traversable and perhaps readable *bingo* not that way - some had 770 while owner/group apache:apache so at least questionable why the warning happens anyways but after change to 775 it is gone
Re: Linux: CAP_DAC_OVERRIDE needed - why?
thank you - learned another lesson! Am 22.07.2013 20:37, schrieb William A. Rowe Jr.: If it was 770 apache:apache, then root had no access, and root (before processing the User directive) was 'unable' to verify the existence of the child directory without violating the apparent access control (not traditional access control, of course). On Mon, Jul 22, 2013 at 1:08 PM, Reindl Harald h.rei...@thelounge.net mailto:h.rei...@thelounge.net wrote: Am 22.07.2013 17:01, schrieb William A. Rowe Jr.: On Sun, 21 Jul 2013 00:15:45 +0200 Reindl Harald h.rei...@thelounge.net mailto:h.rei...@thelounge.net wrote: but why does httpd need CAP_DAC_OVERRIDE while starting initially as root? CapabilityBoundingSet=CAP_IPC_LOCK CAP_NET_BIND_SERVICE CAP_SETGID CAP_SETUID Jul 21 00:04:01 srv-rhsoft httpd[8813]: AH00112: Warning: DocumentRoot [/mnt/data/www/www] does not exist Jul 21 00:04:01 srv-rhsoft httpd[8813]: AH00112: Warning: DocumentRoot [/mnt/data/www/private] does not exist Could one of the parents /mnt .../data .../www offer no other-traverse (x) access? If so, these need to be both root and switch-to-user traversable and perhaps readable *bingo* not that way - some had 770 while owner/group apache:apache so at least questionable why the warning happens anyways but after change to 775 it is gone signature.asc Description: OpenPGP digital signature
Re: Re: Anyone interested in a patch to mod_fcgid(with pay)
Yes, split process control from mod_fcgid, merge proxy_fcgi(with load balance) and mod_fcgid(with authXX support) is a good idea, admins can use httpd as process manager, or 3rd party process managers as they like. But don't just make a patch to make mod_fcgid support TCP/IP, it's ugly... Hi, guys A company need a TCP/IP patch of mod_fcgid or alternative, and will pay for it, anyone interested? I really like to take it but I don't have too much time... Anyone interested please reply to me and I will forward the email address of them. ... Neti only listens on TCP/IP socket, it assumes both an authorizer role and a responder role in the Fast CGI request. Here's the 3 candidate Apache modules to interface Neti: ... 2. Mod_proxy_fcgi: this module supports TCP socket, it can connect to Neti, but it doesn't support authorizer role. So in the first FCGI request, it forwards the request to Neti as a responder instead of an authorizer, Neti cannot simply let it through without properly authorizing it first, thus the request fails; 3. Mod_fcgid: this module supports authorizer role while doesn't support TCP connection. We cannot confirm its authorizer role since it doesn't even connect to Neti due to lack of TCP; So our choice is between either adding authorizer role to mod_proxy_fcgi or adding TCP/IP to mod_fcgid. We’re really willing to pay to have this project done, I mean either adding proxy to mod_fcgid or adding authorizer to mod_proxy_fcgi. Are you willing to work on this with reward or do you know anyone else who’s interested in doing so with pay, for example, author of mod_proxy_fcgi? (I cannot find his name) No cycles myself at this instant, but it seems overtime to break apart the mod_fcgid process-control so that it can then stack on top of a single mod_proxy_fcgi communications pipe, dispatched through different fcgi-stream methods (including child process stdio), including the various authn/authz roles. It would take more time to refactor it in this way, but would get rid of any discrepancies between proxy_fcgi and fcgid and serve as a good excuse to draw the remaining mod_fcgid development back into trunk/, now that fcgid is generally sufficient for 2.2 users.
Re: Anyone interested in a patch to mod_fcgid(with pay)
On Tue, 23 Jul 2013 09:46:58 +0800 (CST) Pqf 潘庆峰 p...@mailtech.cn wrote: Yes, split process control from mod_fcgid, merge proxy_fcgi(with load balance) and mod_fcgid(with authXX support) is a good idea, admins can use httpd as process manager, or 3rd party process managers as they like. But don't just make a patch to make mod_fcgid support TCP/IP, it's ugly... Nice synopsis :)
