[PATCH] ap_rgetline() -- API clarification
Hello, Apparently the second argument to ap_rgetline serves as an upper bound for the returned line, if the first argument points to NULL. This tripped me up (I expected behaviour like ap_get_brigade(mode=AP_MODE_GETLINE,readbytes=0)), so I thought it deserved mention in the header. Jim --- include/http_protocol.h.orig2003-03-14 00:28:04.0 -0800 +++ include/http_protocol.h 2003-03-14 00:29:05.0 -0800 @@ -577,7 +577,8 @@ * @param s Pointer to the pointer to the buffer into which the line * should be read; if *s==NULL, a buffer of the necessary size * to hold the data will be allocated from the request pool - * @param n The size of the buffer + * @param n The size of the buffer; if *s==NULL, the maximum size of the + * buffer to be allocated * @param read The length of the line. * @param r The request * @param fold Whether to merge continuation lines
ap_get_brigade question
Howdy, I have a question in the same vein as my last post. Can I rely on ap_get_brigade(mode=AP_MODE_READBYTES, len=X) to return a brigade of length no greater than X? It would appear not, which means I need to be prepared to set aside extra bytes... But I thought this was exactly what the supplying filter was supposed to do! Confirmation requested. Thanks! Jim
Re: Advanced Mass Hosting Module
On Thu, Mar 13, 2003 at 04:55:19PM -0800, David Burry wrote: These are neat ideas. At a few companies I've worked for we already do similar things but we have scripts that generate the httpd.conf files and distribute them out to the web servers and gracefully restart. Adding a new web server machine to the mix is as simple as adding the host name to the distribution script. This only works when you have a limited number of vhosts - if you were to run thousands of vhosts on each machine, then mod_vhost_alias (or mod_rewrite) is currently the only way to go. A module like this could provide a nice compromise between the flexibility of using httpd.conf to specify each vhost and the speed of vhost_alias. vh Mads Toftum -- `Darn it, who spiked my coffee with water?!' - lwall
Re: Advanced Mass Hosting Module
You and someone else said the same thing. I currently have a setup where we run several hundred vhosts (all individually specified) without issue, I'll have to remember this if it ever grows to thousands. Thanks. With the lack of a more powerful vhost-alias type thing, I'll probably have to vhost-alias all the standard bare bones configs, and list out the anomalies separately Dave - Original Message - From: Mads Toftum [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, March 14, 2003 12:55 AM Subject: Re: Advanced Mass Hosting Module On Thu, Mar 13, 2003 at 04:55:19PM -0800, David Burry wrote: These are neat ideas. At a few companies I've worked for we already do similar things but we have scripts that generate the httpd.conf files and distribute them out to the web servers and gracefully restart. Adding a new web server machine to the mix is as simple as adding the host name to the distribution script. This only works when you have a limited number of vhosts - if you were to run thousands of vhosts on each machine, then mod_vhost_alias (or mod_rewrite) is currently the only way to go. A module like this could provide a nice compromise between the flexibility of using httpd.conf to specify each vhost and the speed of vhost_alias. vh Mads Toftum -- `Darn it, who spiked my coffee with water?!' - lwall
Re: Advanced Mass Hosting Module
On Thu, Mar 13, 2003 at 08:27:30PM +0900, Nathan Ollerenshaw wrote: Resending this to this list as I got no response on users list. Currently, we are using flat config files generated by our website provisioning software to support our mass hosted customers. The reason for doing it this way, and not using the mod_vhost_alias module is because we need to be able to turn on/off CGI, PHP, Java, shtml etc on a per vhost basis. We need the power that having a distinct VirtualHost directive for each site gives you. Is there a better way? I once started a project to do this from a database, I eventually stopped as I couldn't figure out a nice way to enable/disable php,cgi,whatever on demand. Serving virtualhosts from documentroots you pull out of a database is no big deal.
RE: ap2 , parent process in worker mpm dies under load
The problem also occurs in 32-bit mode. Here are the backtraces from
pstack and dbx.
-- core.httpd.25359.u0 (pstack) --
core '/var/core/core.httpd.25359.u0' of 25359:
/usr/local/apache2/bin/httpd -f /etc/httpd2.conF
- lwp# 1 / thread# 1
00060408 server_main_loop (0, 6c8, 7, 0, 119af0, df6a0) + 338
000609d8 ap_mpm_run (117d00, 141da8, 119af0, 119af0, 0, 0) + 580
0006f1c0 main (3, ffbefd6c, ffbefd7c, fd000, 0, 0) + e98
00048d88 _start (0, 0, 0, 0, 0, 0) + 108
- lwp# 2 / thread# 2
fefced24 sigaddset (ff0eb1c0, a, 0, 0, 0, 0) + 4c
ff0be9b4 _sigredirect (a, ff0def98, 3, ff0e5930, ff0e5948, ff01f7b4) + 34
ff0bee80 _dynamiclwps (ff0de000, ff3d19b4, ff3a0dd8, ff3e66c0, ff29e381, 0)
+ 148
ff0c2030 thr_yield (0, 0, 0, 0, 0, 0) + 8c
- lwp# 4
ff0c9774 private___lwp_cond_wait (4, ff0ded9c, ff0de000, 0, 0, ff01f7b4) +
8
ff0c6bc4 _sc_door_func (, ff0df688, ff0df6a0, 3, ff0de000, 1) + 74
ff0ba740 _lwp_start (fef65d70, 0, 6000, ffbef98c, 0, 0) + 18
ff0c2030 thr_yield (0, 0, 0, 0, 0, 0) + 8c
-- thread# 3
ff0bddbc _reap_wait (ff0e29e0, 20520, 0, ff0de000, 0, 0) + 38
ff0bdb14 _reaper (ff0dee30, ff0e4740, ff0e29e0, ff0dee08, 1, fe40) +
38
ff0cb728 _thread_start (0, 0, 0, 0, 0, 0) + 40
-- core.httpd.25359.u0 (pstack) --
-- core.httpd.25359.u0 (dbx) --
[EMAIL PROTECTED] ([EMAIL PROTECTED]) terminated by signal BUS (invalid address
alignment)
Current function is server_main_loop
1645 perform_idle_server_maintenance();
(/tool/lang9.1/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) where
current thread: [EMAIL PROTECTED]
=[1] server_main_loop(remaining_children_to_start = 0), line 1645 in
worker.c
[2] ap_mpm_run(_pconf = 0x117d00, plog = 0x141da8, s = 0x119af0), line
1745 in worker.c
[3] main(argc = 3, argv = 0xffbefd6c), line 644 in main.c
-- core.httpd.25359.u0 (dbx) --
-- core.httpd.8656.u0 (pstack) --
core '/var/core/core.httpd.8656.u0' of 8656:
/usr/local/apache2/bin/httpd -f /etc/httpd2.conf
- lwp# 1 / thread# 1
ff2b8974 apr_proc_wait_all_procs (31, 45, 49, 1, 117d00, df46c) + 44
0007be2c ap_wait_or_timeout (49, 45, 31, 117d00, 0, 0) + ac
0006015c server_main_loop (0, 6c8, 7, 0, 119af0, df6a0) + 8c
- lwp# 2 / thread# 2
fefced24 sigaddset (ff0eb1c0, a, 0, 0, 0, 0) + 4c
ff0be9b4 _sigredirect (a, ff0def98, 3, ff0e5930, ff0e5948, ff01f7b4) + 34
ff0bee80 _dynamiclwps (ff0de000, ff3d19b4, ff3a0dd8, ff3e66c0, ff29e381, 0)
+ 148
ff0c2030 thr_yield (0, 0, 0, 0, 0, 0) + 8c
- lwp# 4
ff0c9774 private___lwp_cond_wait (4, ff0ded9c, ff0de000, 0, 0, ff01f7b4) +
8
ff0c6bc4 _sc_door_func (, ff0df688, ff0df6a0, 3, ff0de000, 1) + 74
ff0ba740 _lwp_start (fef65d70, 0, 6000, ffbef9e4, 0, 0) + 18
ff0c2030 thr_yield (0, 0, 0, 0, 0, 0) + 8c
-- thread# 3
ff0bddbc _reap_wait (ff0e29e0, 20520, 0, ff0de000, 0, 0) + 38
ff0bdb14 _reaper (ff0dee30, ff0e4740, ff0e29e0, ff0dee08, 1, fe40) +
38
ff0cb728 _thread_start (0, 0, 0, 0, 0, 0) + 40
-- core.httpd.8656.u0 (pstack) --
-- core.httpd.8656.u0 (dbx) --
[EMAIL PROTECTED] ([EMAIL PROTECTED]) terminated by signal BUS (invalid address
alignment)
0xff2b8974: bad address 0xff2b8974
Current function is ap_wait_or_timeout
223 if (APR_STATUS_IS_EINTR(rv)) {
(/tool/lang9.1/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) where
current thread: [EMAIL PROTECTED]
[1] 0xff2b8974(0x31, 0x45, 0x49, 0x1, 0x117d00, 0xdf46c), at 0xff2b8973
=[2] ap_wait_or_timeout(status = 0x49, exitcode = 0x45, ret = 0x31, p =
0x117d00), line 223 in mpm_common.c
dbx: warning: invalid frame pointer
-- core.httpd.8656.u0 (dbx) --
Regards,
Steve
--
Steve Sabljak | BBC Technology - Internet Operations
Tel: +44 (0) 1628 407708 | Maiden House, Vanwell Road
URL: http://support.bbc.co.uk | Maidenhead, SL6 4UB
-Original Message-
From: Jeff Trawick [SMTP:[EMAIL PROTECTED]
Sent: Monday, March 03, 2003 2:04 PM
To: [EMAIL PROTECTED]
Subject: Re: ap2 , parent process in worker mpm dies under load
Andre Breiler wrote:
Hi,
On Sun, 2 Mar 2003, Jeff Trawick wrote:
Andre Breiler wrote:
the ap2 (2.0.43) parent process dies (but childs arn't) under load.
This is with worker mpm on solaris 8 (multiprocessor).
...
--- snip 1 ---
program terminated by signal BUS (Bus Error)
0x7ee2c880:
Current function is ap_wait_or_timeout
222 rv = apr_proc_wait_all_procs(ret, exitcode, status,
APR_NOWAIT, p);
(/tool/lang9.1/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx) where
[1] 0x7ee2c880(0x080e, 0x0836,
0x083a, 0x1, 0x100133e58, 0x0), at 0x7ee2c87f
=[2]
Re: Advanced Mass Hosting Module
On Saturday, March 15, 2003, at 12:02 AM, Thomas Eibner wrote: On Thu, Mar 13, 2003 at 08:27:30PM +0900, Nathan Ollerenshaw wrote: Resending this to this list as I got no response on users list. Currently, we are using flat config files generated by our website provisioning software to support our mass hosted customers. The reason for doing it this way, and not using the mod_vhost_alias module is because we need to be able to turn on/off CGI, PHP, Java, shtml etc on a per vhost basis. We need the power that having a distinct VirtualHost directive for each site gives you. Is there a better way? I once started a project to do this from a database, I eventually stopped as I couldn't figure out a nice way to enable/disable php,cgi,whatever on demand. Serving virtualhosts from documentroots you pull out of a database is no big deal. I wasn't thinking of anything radical. Just have a hook to set the handler for a particular document (if it matches .php or .php4) to the PHP module if it's allowed to, and serve it as a normal document if not. Etc. I've not had a great delve in the hooks but nothing has suggested in what I've looked at that it's not possible. I really need to get a proof-of-concept working; maybe this weekend if my other half gives me a 'allowed to use computer' note for the teacher. Nathan. -- Nathan Ollerenshaw - Systems Engineer - Shared Hosting ValueCommerce Japan - http://www.valuecommerce.ne.jp You can't be a Real Country unless you have a BEER and an airline - it helps if you have some kind of a football team or some nuclear weapons, but at the very least you need a BEER. - Frank Zappa
Re: apr 0.9.2 release?
Craig... we are waiting on only one issue; addressing the inherited apr handles vulnerability discussed for httpd 2.0 cgi scripting on vuln-dev. Bjoern Zeeb has spearheaded the effort for the Apache and APR projects to adopt appropriate patches... and as soon as those are evaluated and committed you can expect APR 0.9.2 and very soon after, Apache 2.0.45 built on that tag. It's unclear to me (but I'm starting to get a handle on it) if it's entirely Apache's issue (unlikely) or if we have things to change in apr_file_inherit_set (likely). My confusion comes from the fact that I'm still wrapping my brain around when FD_CLOEXEC actually is triggered, and how to safely assure we close what we intend, and leave open the handles that the author desires. Doesn't give you a definitive date, but I hope this helps explain where we sit right now. More eyeballs on Bjoern's patches will definitely speed this along ;-) Bill At 09:38 AM 3/14/2003, you wrote: Hi, When will apr 0.9.2 be released? I am the FreeBSD maintainer of the apr port, and several users are asking me about this. Thanks. -- Craig Rodrigues http://home.attbi.com/~rodrigc [EMAIL PROTECTED]
Re: RSA private key attack [CERT VU#997481] Apache
Another citation; http://marc.theaimsgroup.com/?l=apache-modsslm=104760046402468w=2 Bill
and the httpd-2.0.45 release...
At 10:28 AM 3/14/2003, William A. Rowe, Jr. wrote: Bjoern Zeeb has spearheaded the effort for the Apache and APR projects to adopt appropriate patches... and as soon as those are evaluated and committed you can expect APR 0.9.2 and very soon after, Apache 2.0.45 built on that tag. It's unclear to me (but I'm starting to get a handle on it) if it's entirely Apache's issue (unlikely) or if we have things to change in apr_file_inherit_set (likely). Of course that's only once issue that applies to httpd 2.0.45 - we also have the ssl timing issue to address before the httpd release. Bill
Re: Advanced Mass Hosting Module
Thomas Eibner wrote: On Sat, Mar 15, 2003 at 01:00:18AM +0900, Nathan Ollerenshaw wrote: On Saturday, March 15, 2003, at 12:02 AM, Thomas Eibner wrote: On Thu, Mar 13, 2003 at 08:27:30PM +0900, Nathan Ollerenshaw wrote: Resending this to this list as I got no response on users list. Currently, we are using flat config files generated by our website provisioning software to support our mass hosted customers. The reason for doing it this way, and not using the mod_vhost_alias module is because we need to be able to turn on/off CGI, PHP, Java, shtml etc on a per vhost basis. We need the power that having a distinct VirtualHost directive for each site gives you. Is there a better way? I don't know of a specific virtual host hook, but if there isn't there might be a need for it. I guess you need to have someplace which calls your module's hook *before* the server definition gets set, and allows you to run a pre-config/post-config followup merge for all the modules currently loaded on the first time the server-name is loaded into memory, and then pass the resulting server-config down to the rest of the hooks. this should make it possible to allow you to do anything in your module that the plaintext v-host one could do. --Ian
Re: Advanced Mass Hosting Module
On Saturday, March 15, 2003, at 01:13 AM, Thomas Eibner wrote: On Sat, Mar 15, 2003 at 01:00:18AM +0900, Nathan Ollerenshaw wrote: I wasn't thinking of anything radical. Just have a hook to set the handler for a particular document (if it matches .php or .php4) to the PHP module if it's allowed to, and serve it as a normal document if not. Etc. I've not had a great delve in the hooks but nothing has suggested in what I've looked at that it's not possible. I'm not sure if it's as simple as you describe. What is to stop a user from placing a .htaccess file in a directory giving himself ability to give the right content type to execute a php script for instance? If you want suexec to work too, there might be further complications. (Just thinking out loud here) :) You bring up a valid point, but I was thinking more of sbox. Thats what use use currently (because suexec didn't fit our model) and it works great. Though, there seems to be a bug where it's poisoning the environment ... At any rate, if I'm interfering around the URI-to-filename translation phase first, I should be able to minimise any problems with .htaccess files. But, I don't know, I don't fully understand all the phases that I can interfere with just yet :) There are other phases I've not really looked at as well which I could hook into to do extra sanity checks, I guess. But, I think, get the thing basically working, then narrow down all the annoying security holes it will make, eh? I really need to get a proof-of-concept working; maybe this weekend if my other half gives me a 'allowed to use computer' note for the teacher. What would you consider a proof-of-concept? I have my code lurking on some machine in cvs if you want to take a look at it. If my feeble coding skills are up to it :) I've requested a new sf.net project, so in a couple of days I should be able to put up my hacky bits of code. Really, I only started programming C with a vengeance about a week ago. I'm an old perl hacker, and never felt a need to use C. So fear my code. Expect apache to segfault. ;) Nathan. -- Nathan Ollerenshaw - Systems Engineer - Shared Hosting ValueCommerce Japan - http://www.valuecommerce.ne.jp I'm your blubber boy you should rub me The sun beat me down too viciously I fell into the ground to what I used to be I've melted away I'm nothing again
discussion on fd leak problematic
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, this is a summary for further discussion on the fd leak problematic in httpd-2.0 and related apr (inherit) code. History: - Christian Kratzer noticed a lot of open fds in http-2.0.44 and we started to dig into this[1]. Some other people noticed those too which we got to know after a posting to vuln-dev[2]. The Problem has existed for a along time in httpd code but came not up before 2.0.40 because of another bug in apr which has been fixed before 2.0.40 release[3]. - From then on APR_IMPLEMENT_INHERIT_SET and APR_IMPLEMENT_INHERIT_UNSET worked the way they should but they had already been misused the wrong way for unknown reasons in httpd code. See cvs commits around the versions of diff in [4]. Our first patch intended as a starting for a complete fix is also in bugzilla [1]. After further discussions with Steve Grubb I also traced the open pipes identified as pod (pipe of death) and after about a week later also provided a diff. There also had been some mails to security. Steve Grubb worked/works together with some people at redhat which started intensively working on the problem: [5] Joe Orton then came up with two patches on [EMAIL PROTECTED] and [EMAIL PROTECTED] I also had some private mails with William A. Rowe, Jr. Problems needing discussion: - APR - - - - - - - - - - - - - - - - - - - - - with APR we already had one commit[8]. The main question here is from what I can see: is it possible to use FD_CLOEXEC somewhere in the API of APR_IMPLEMENT_INHERIT_(UN)SET so that we might have some double checks for closing file descriptors on execs. Another question from me is if it would be possible to always register a child chlean_fn not apr_pool_cleanup_null for pipes - resp. s.th. like in my diff[9]. Last point on this would be that developers that use APR need to somehow recognnize that they can change the behaviour for pipes like they can for files with apr_file_inherit_(un)set. not having apr_pipe_inherit_(un)set seems to also cause irritations amog apr/httpd-developers. For open there is the (yet) unused(?) flag APR_FILE_NOCLEANUP. Can some have a look at this and either document it as yet unused or remove it from the code or correct me if I am wrong. [file_io/unix/open.c] HTTPD - - - - - - - - - - - - - - - - - - - - - because of the missuse of apr_file_inherit_set and not registered child cleanup_fn we have various fd leaks for pipes and logs. After my initial patch [1][10] Joe Orton came up with another patch to [EMAIL PROTECTED] My problem with this is: is it good enough to simply remove the apr_file_inherit_set() calls or explicitly use apr_file_inherit_unset() so that we can be sure that there is always a child cleanup_fn registered. In apr there still is the flag APR_FILE_NOCLEANUP that might prevent registering a child cleanup_fn as the deault for now is. This flag seems to be unused at the moment for open() calls. Joe Orton does this in his patch for pipes (as they do not have a child cleanup_fn registered by default). Another thing I lately discovered after some discussion with Steve Grubb on lseek()ing on open file descriptors was that for me only error logs had been readable but not access logs. I think we do not need to open error_logs for reading. A fix for at least one place has been posted to [EMAIL PROTECTED] There might be other places where files are opened with more priviledges than needed. Please look at this. Why ? - Some might say that CGIs are only as secure as one trusts or verifies the code that runs on a webserver. Heard that multiple times. This is correct. But there are other reasons why these things are security relevant: a) as stated by multiple persons reading and writing on open file descriptors is no good for multiple reasons. See p.ex. posting to vuln-dev by Steve Grubb[2]. As stated by some persons: one may open those files in any case. That is not fully true. I am using suexec and the CGI is not able to open the log files (even not the ones from the virtual host it is running on). It for sure can as long as the fds are leaked to it. b) the open pipes can be abused for a partly DoS attack or one can at least cause performance impacts when running apache with mpm=worker or mpm=threadpool by simply writing the correct 'restart' or 'graceful' char to them which causes a drop/restart on some of the processes/threads from what I could see on linux. There is at least one more reason why this needs to be fixed: a lot of open file descriptors are problems for resource limits. Small hosters or people with few virtual hosts will perhaps not log to pipes but have an error and an access log open for each virtual host. If they have set resource limits by p.ex. patching suexec they run into problems. This is btw. why this had been discovered by Christian Kratzer from
[Win32] compiling Apache 2.1.0-dev: missing sslc.h
Hi, I cannot compile the latest snapshots (with OpenSSL 0.9.7a) without errors: -Configuration: mod_ssl - Win32 Release Generating ssl_expr_parse.c/.h from ssl_expr_parse.y Compiling... ssl_expr_parse.c F:\Projects\MSVC\httpd-2.0\modules\ssl\ssl_toolkit_compat.h(139) : fatal error C1083: Cannot open include file: 'sslc.h': No such file or directory Where should I search for this file? Regards -- Juergen Heckel
