Re: Help: Apache Input filter
On Mon, 1 Oct 2007 14:55:57 +0530 Mohit_Garg01 [EMAIL PROTECTED] wrote: Hi, My application has an apache input filter which reads the large amount of POST data at one go and writes on to the back end server. We want that reads are maintained in chunks and then those chunks are written onto the backend server. the backend server? If your application involves a backend server, it should be managed by a handler, not an input filter. You should probably use mod_proxy, with you own protocol handler if necessary. How can the filter read the data in chunks and write it? Lets say it reads one brigade at a time. The filter doesn't write data. It returns data to its caller. That normally happens in chunks. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Bug report for Apache httpd-1.3 [2007/09/30]
+---+ | Bugzilla Bug ID | | +-+ | | Status: UNC=Unconfirmed NEW=New ASS=Assigned| | | OPN=ReopenedVER=Verified(Skipped Closed/Resolved) | | | +-+ | | | Severity: BLK=Blocker CRI=CriticalMAJ=Major | | | | MIN=Minor NOR=Normal ENH=Enhancement | | | | +-+ | | | | Date Posted | | | | | +--+ | | | | | Description | | | | | | | |10038|New|Min|2002-06-20|ab benchmaker hangs on 10K https URLs with keepali| |10744|New|Nor|2002-07-12|suexec might fail to open log file| |10747|New|Maj|2002-07-12|ftp SIZE command and 'smart' ftp servers results i| |10760|New|Maj|2002-07-12|empty ftp directory listings from cached ftp direc| |14518|Opn|Nor|2002-11-13|QUERY_STRING parts not incorporated by mod_rewrite| |16013|Opn|Nor|2003-01-13|Fooling mod_autoindex + IndexIgnore | |16631|Inf|Min|2003-01-31|.htaccess errors logged outside the virtual host l| |17318|Inf|Cri|2003-02-23|Abend on deleting a temporary cache file if proxy | |19279|Inf|Min|2003-04-24|Invalid chmod options in solaris build| |21637|Inf|Nor|2003-07-16|Timeout causes a status code of 200 to be logged | |21777|Inf|Min|2003-07-21|mod_mime_magic doesn't handle little gif files| |22618|New|Maj|2003-08-21|MultiViews invalidates PATH_TRANSLATED if cgi-wrap| |25057|Inf|Maj|2003-11-27|Empty PUT access control in .htaccess overrides co| |26126|New|Nor|2004-01-14|mod_include hangs with request body | |26152|Ass|Nor|2004-01-15|Apache 1.3.29 and below directory traversal vulner| |26790|New|Maj|2004-02-09|error deleting old cache file | |29257|Opn|Nor|2004-05-27|Problem with apache-1.3.31 and mod_frontpage (dso,| |29498|New|Maj|2004-06-10|non-anonymous ftp broken in mod_proxy | |29538|Ass|Enh|2004-06-12|No facility used in ErrorLog to syslog| |30207|New|Nor|2004-07-20|Piped logs don't close read end of pipe | |30877|New|Nor|2004-08-26|htpasswd clears passwd file on Sun when /var/tmp i| |30909|New|Cri|2004-08-28|sporadic segfault resulting in broken connections | |31975|New|Nor|2004-10-29|httpd-1.3.33: buffer overflow in htpasswd if calle| |32078|New|Enh|2004-11-05|clean up some compiler warnings | |32539|New| |2004-12-06|[PATCH] configure --enable-shared= brocken on SuSE| |32974|Inf|Maj|2005-01-06|Client IP not set | |33086|New|Nor|2005-01-13|unconsistency betwen 404 displayed path and server| |33495|Inf|Cri|2005-02-10|Apache crashes with WSADuplicateSocket failed for| |33772|New|Nor|2005-02-28|inconsistency in manual and error reporting by sue| |33875|New|Enh|2005-03-07|Apache processes consuming CPU| |34108|New|Nor|2005-03-21|mod_negotiation changes mtime to mtime of Document| |34114|New|Nor|2005-03-21|Apache could interleave log entries when writing t| |34404|Inf|Blk|2005-04-11|RewriteMap prg can not handle fpout | |34571|Inf|Maj|2005-04-22|Apache 1.3.33 stops logging vhost| |34573|Inf|Maj|2005-04-22|.htaccess not working / mod_auth_mysql| |35424|New|Nor|2005-06-20|httpd disconnect in Timeout on CGI| |35439|New|Nor|2005-06-21|Problem with remove /../ in util.c and mod_rewri| |35547|Inf|Maj|2005-06-29|Problems with libapreq 1.2 and Apache::Cookie | |3|New|Nor|2005-06-30|Can't find DBM on Debian Sarge| |36375|Opn|Nor|2005-08-26|Cannot include http_config.h from C++ file| |37166|New|Nor|2005-10-19|Under certain conditions, mod_cgi delivers an empt| |37185|New|Enh|2005-10-20|AddIcon, AddIconByType for OpenDocument format| |37252|New| |2005-10-26|gen_test_char reject NLS string | |38989|New|Nor|2006-03-15|restart + piped logs stalls httpd for 24 minutes (| |39104|New|Enh|2006-03-25|[FR] fix build with -Wl,--as-needed | |39287|New|Nor|2006-04-12|Incorrect If-Modified-Since validation (due to syn| |39937|New|Nor|2006-06-30|Garbage output if README.html is gzipped or compre| |40176|New|Nor|2006-08-03|magic and mime| |40224|Ver|Nor|2006-08-10|System time crashes Apache @year 2038 (win32 only?| |41279|New|Nor|2007-01-02|Apache 1.3.37 htpasswd is vulnerable to buffer ove| |42355|New|Maj|2007-05-08|Apache 1.3 permits non-rfc HTTP error code = 600 |
2.0.54 unstable, requests time-out, NO warnings in logs
We are running a busy Apache/2.0.54 server on 2.6.9 kernel, that suddenly becomes very slow- requests either time out, or it takes 10-20sec to serve a 1K thumbnail. It is somewhat correlated with load spikes, but not perfectly (by looking at the bandwidth graph, it never happens during the low bandwidth periods at night, but it does not coincide with peaks of b/w) When we initially encountered an apache overload, it was always accompanied with [error] server reached MaxClients setting, consider raising the MaxClients setting in the apache error log. We also got kernel: possible SYN flooding on port 80. Sending cookies. in /var/log/messages system log. After that I raised MaxClients from 200 to 300. The problem initially disappeared, but after our bandwidth grew a bit more, we got this behavior again. Now apache crashes (becomes very slow) silently, with no warning in apache error logs at all (although we still get SYN flood message in the system log) When apache is this 'slow' regime, /server-status still shows available slots, i.e. MaxClients is not reached. This is the relevant part of httpd.conf: ServerLimit 300 # we are using prefork MPM StartServers 10 MinSpareServers 5 MaxSpareServers 20 MaxClients 300 MaxRequestsPerChild 1 MaxMemFree 2500 The server has 4GB of physical RAM and 4GB of swap. During these apache “slowdowns, the swap size is still 0 and vmstat shows no swapping at all. I suspect the problem may be in MaxMemFree 2500 but then I would expect some kind of “out of memory” errors in the logs? I am posting it on this list since I have not gotten a response in the users list, and I think it's a bug for two reasons: 1) When apache is in this slow degraded regime, I would expect a log message in the apache error log, with an explanation why. 3) If this is related to resource exhaustion, I would expect the server to recover from this regime by itself when the load subsides, but this is not the case. Only apachectl start/stop recovers the server.
Time to chop exports.c in half?
server/Makefile.in; export_files: tmp=export_files_unsorted.txt; \ rm -f $$tmp touch $$tmp; \ for dir in $(EXPORT_DIRS); do \ ls $$dir/*.h $$tmp; \ done; \ for dir in $(EXPORT_DIRS_APR); do \ (ls $$dir/ap[ru].h $$dir/ap[ru]_*.h $$tmp 2/dev/null); \ done; \ sort -u $$tmp $@; \ rm -f $$tmp Isn't it time, already, do do away with everything related to EXPORT_DIRS_APR in httpd 2.3-dev? (Obviously I wouldn't suggest changing anything for 2.2). It seems every modern OS should do a perfectly respectible job of binding dynamic libraries and their symbols without this extra, leftover cruft. Bill
Adding timestamp to apache releases?
Greetings, To-do list item #1 for this week is upgrade to 2.2.6. When I was waiting for the tar-ball to download, it occurred to me that it isn't blindingly obvious *when* the update was published. There's no date on the homepage (http://httpd.apache.org/) or on the download page (http://httpd.apache.org/download.cgi) or on the announcement page (http://www.apache.org/dist/httpd/Announcement2.2.html) or even on the changes log (http://apache.mirror.testserver.li/httpd/CHANGES_2.2)... It's just that when I announce the upgrade internally, managers like to see something like a date rather than just an arbitrary version number. Is there a reason for the coyness or is it just an oversight, like people who send out invites to parties with elaborate directions and clip-art but forget to put the date? Might it be an idea for 2.2.7? Rgds, Owen Boyle Disclaimer: Any disclaimer attached to this message may be ignored. This message is for the named person's use only. It may contain confidential, proprietary or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please notify the sender urgently and then immediately delete the message and any copies of it from your system. Please also immediately destroy any hardcopies of the message. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. The sender's company reserves the right to monitor all e-mail communications through their networks. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorised to state them to be the views of the sender's company.
Re: Adding timestamp to apache releases?
Boyle Owen wrote: Might it be an idea for 2.2.7? I like the idea of adding a date to each news item, be it on httpd.a.o, or our www.apache.org. +1. (Especially since the datestamps of our tarballs are several days prior to each release).
Re: Adding timestamp to apache releases?
On 10/1/07, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: Boyle Owen wrote: Might it be an idea for 2.2.7? I like the idea of adding a date to each news item, be it on httpd.a.o, or our www.apache.org. +1. (Especially since the datestamps of our tarballs are several days prior to each release). I like that idea too! +1 -- ~Jorge
Re: 2.0.54 unstable, requests time-out, NO warnings in logs
On 10/01/2007 08:32 AM, Alec Matusis wrote: We are running a busy Apache/2.0.54 server on 2.6.9 kernel, that suddenly becomes very slow- requests either time out, or it takes 10-20sec to serve a 1K thumbnail. It is somewhat correlated with load spikes, but not perfectly (by looking at the bandwidth graph, it never happens during the low bandwidth periods at night, but it does not coincide with peaks of b/w) When we initially encountered an apache overload, it was always accompanied with [error] server reached MaxClients setting, consider raising the MaxClients setting in the apache error log. We also got kernel: possible SYN flooding on port 80. Sending cookies. in /var/log/messages system log. After that I raised MaxClients from 200 to 300. The problem initially disappeared, but after our bandwidth grew a bit more, we got this behavior again. Now apache crashes (becomes very slow) silently, with no warning in apache error logs at all (although we still get SYN flood message in the system log) When apache is this 'slow' regime, /server-status still shows available slots, i.e. MaxClients is not reached. This is the relevant part of httpd.conf: ServerLimit 300 # we are using prefork MPM StartServers 10 MinSpareServers 5 MaxSpareServers 20 MaxClients 300 MaxRequestsPerChild 1 MaxMemFree 2500 The server has 4GB of physical RAM and 4GB of swap. During these apache “slowdowns, the swap size is still 0 and vmstat shows no swapping at all. I suspect the problem may be in MaxMemFree 2500 Have you checked without the MaxMemFree setting? Why do you use MaxMemFree with such a small value at all? Regards Rüdiger
RE: 2.0.54 unstable, requests time-out, NO warnings in logs
Have you checked without the MaxMemFree setting? I raised MaxMemFree to 3100, we will have to wait for a few days, since it does not happen every day. Why do you use MaxMemFree with such a small value at all? We are also running 2 twistd server processes on this machine that can take up to 256MB each, so we wanted to leave some room for them. As I understand from the manual, MaxMemFree is not an absolute limit on the memory available to Apache anyway, it just asks it to release *unused* memory with free() call after it reaches MaxMemFree value? -Original Message- From: Ruediger Pluem [mailto:[EMAIL PROTECTED] Sent: Monday, October 01, 2007 1:23 AM To: dev@httpd.apache.org Subject: Re: 2.0.54 unstable, requests time-out, NO warnings in logs On 10/01/2007 08:32 AM, Alec Matusis wrote: We are running a busy Apache/2.0.54 server on 2.6.9 kernel, that suddenly becomes very slow- requests either time out, or it takes 10- 20sec to serve a 1K thumbnail. It is somewhat correlated with load spikes, but not perfectly (by looking at the bandwidth graph, it never happens during the low bandwidth periods at night, but it does not coincide with peaks of b/w) When we initially encountered an apache overload, it was always accompanied with [error] server reached MaxClients setting, consider raising the MaxClients setting in the apache error log. We also got kernel: possible SYN flooding on port 80. Sending cookies. in /var/log/messages system log. After that I raised MaxClients from 200 to 300. The problem initially disappeared, but after our bandwidth grew a bit more, we got this behavior again. Now apache crashes (becomes very slow) silently, with no warning in apache error logs at all (although we still get SYN flood message in the system log) When apache is this 'slow' regime, /server-status still shows available slots, i.e. MaxClients is not reached. This is the relevant part of httpd.conf: ServerLimit 300 # we are using prefork MPM StartServers 10 MinSpareServers 5 MaxSpareServers 20 MaxClients 300 MaxRequestsPerChild 1 MaxMemFree 2500 The server has 4GB of physical RAM and 4GB of swap. During these apache slowdowns, the swap size is still 0 and vmstat shows no swapping at all. I suspect the problem may be in MaxMemFree 2500 Have you checked without the MaxMemFree setting? Why do you use MaxMemFree with such a small value at all? Regards Rüdiger
Re: Adding timestamp to apache releases?
On 01.10.2007, at 09:58, William A. Rowe, Jr. wrote: Boyle Owen wrote: Might it be an idea for 2.2.7? You can also get it from here for now: http://projects.apache.org/projects/http_server.html or as a feed: http://projects.apache.org/feeds/rss/http_server.xml I like the idea of adding a date to each news item, be it on httpd.a.o, or our www.apache.org. +1. +1, see attached patch which adds dates to the index and download pages (see changes to site.vsl which add a 2nd column to the relevant section headers as soon as a new date= attribute is present). Please test and give your opinion, works for me on Safari and Firefox... Cheers, Erik dates.patch Description: Binary data
Re: Proxying OPTIONS *
On Mon, Oct 01, 2007 at 12:05:58AM +0100, Nick Kew wrote: RFC2616 is clear that: 1. OPTIONS * is allowed. 2. OPTIONS can be proxied. However, it's not clear that OPTIONS * can be proxied, given that there's no natural URL representation of it (* != /*). The Co-Advisor suite has a test case to proxy OPTIONS * using: OPTIONS * HTTP/1.1\r\n Host: [remote target host]\r\n \r\n Unfortunately PR#43519 is obscuring the Co-Advisor test case (which purports to be testing our handline of Max-Forwards) by returning 403. It's not at all clear to me whether that syntax should be supported. Anyone? I know Roy's already reported the proxy error as bogus, but I think the OPTIONS * BUGZ report is also bogus. As a test, I assumed that both www.apache.org and apache.webthing.com are reasonably configured servers: $ telnet apache.webthing.com 80 Trying 195.50.92.131... Connected to www.webthing.com. Escape character is '^]'. OPTIONS * HTTP/1.1 Host: apache.webthing.com HTTP/1.1 200 OK Date: Mon, 01 Oct 2007 12:58:45 GMT Server: Apache/2.2.5 (Unix) DAV/2 mod_ssl/2.2.5 OpenSSL/0.9.8a SVN/1.2.3 Allow: GET,HEAD,POST,OPTIONS,TRACE Content-Length: 0 Content-Type: text/plain; charset=UTF-8 --- $ telnet apache.webthing.com 80 Trying 195.50.92.131... Connected to www.webthing.com. Escape character is '^]'. OPTIONS * HTTP/1.0 HTTP/1.1 200 OK Date: Mon, 01 Oct 2007 13:01:32 GMT Server: Apache/2.2.5 (Unix) DAV/2 mod_ssl/2.2.5 OpenSSL/0.9.8a SVN/1.2.3 Allow: GET,HEAD,POST,OPTIONS,TRACE Content-Length: 0 Connection: close Content-Type: text/plain; charset=UTF-8 Can anything confirm that OPTIONS * is as hosed as the BUGZ report claim? I haven't had time to actually trace the internals yet...
Re: Proxying OPTIONS *
On 10/1/07, Jim Jagielski [EMAIL PROTECTED] wrote: I know Roy's already reported the proxy error as bogus, but I think the OPTIONS * BUGZ report is also bogus. As a test, I assumed that both www.apache.org and apache.webthing.com are reasonably configured servers: www.apache.org is using a config built from the 2.0 default, where Directory / was not restricted. To hit the (alleged) bug, you'd need to test on a server using the 2.2 default: Directory / Order deny,allow Deny from all /Directory (I haven't done this testing myself, so I have nothing else to contribute on the issue.) Joshua.
Re: Adding timestamp to apache releases?
On Oct 1, 2007, at 12:34 AM, Boyle Owen wrote: Is there a reason for the coyness or is it just an oversight, like people who send out invites to parties with elaborate directions and clip-art but forget to put the date? PGP to the rescue! Just downloaded the release, and Safari preserves the modification time: [EMAIL PROTECTED] downloads $ curl -I http://mirrors.sirium.net/ pub/apache/httpd/httpd-2.2.6.tar.gz HTTP/1.1 200 OK Date: Mon, 01 Oct 2007 13:51:22 GMT Server: Apache Last-Modified: Thu, 06 Sep 2007 19:31:02 GMT ETag: 547541-5bfe97-46e05576 Accept-Ranges: bytes Content-Length: 6028951 Content-Type: application/x-gzip [EMAIL PROTECTED] downloads $ ls -lt httpd-2.2.6.tar.gz* -rw-r--r-- 1 sctemme admin 53 Sep 6 12:31 httpd-2.2.6.tar.gz.md5 -rw-r--r-- 1 sctemme admin 6028951 Sep 6 12:31 httpd-2.2.6.tar.gz -rw-r--r-- 1 sctemme admin 186 Sep 6 12:31 httpd-2.2.6.tar.gz.asc Now when I verify the PGP signature: [EMAIL PROTECTED] downloads $ gpg --verify httpd-2.2.6.tar.gz.asc gpg: Signature made Tue Sep 4 13:09:41 2007 PDT using DSA key ID 08C975E5 gpg: Good signature from Jim Jagielski [EMAIL PROTECTED] gpg: aka Jim Jagielski [EMAIL PROTECTED] gpg: aka Jim Jagielski [EMAIL PROTECTED] gpg: aka Jim Jagielski [EMAIL PROTECTED] gpg: aka Jim Jagielski [EMAIL PROTECTED] Note the time stamp on the signature. Of course this is the time of the clock on Jim's computer: I don't think GPG can get a trusted timestamp for signatures. I looked through the options and saw none. Perhaps that's something to look into, but for now there is a timestamp on the signature. S. -- Sander Temme [EMAIL PROTECTED] PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF smime.p7s Description: S/MIME cryptographic signature
Re: Proxying OPTIONS *
On 10/01/2007 03:30 PM, Joshua Slive wrote: On 10/1/07, Jim Jagielski [EMAIL PROTECTED] wrote: I know Roy's already reported the proxy error as bogus, but I think the OPTIONS * BUGZ report is also bogus. As a test, I assumed that both www.apache.org and apache.webthing.com are reasonably configured servers: www.apache.org is using a config built from the 2.0 default, where Directory / was not restricted. To hit the (alleged) bug, you'd need to test on a server using the 2.2 default: Directory / Order deny,allow Deny from all /Directory I have done a test on 2.2.x with the above setting: telnet 192.168.2.4 80 Trying 192.168.2.4... Connected to 192.168.2.4. Escape character is '^]'. OPTIONS * HTTP/1.1 Host: 192.168.2.4 HTTP/1.1 200 OK Date: Mon, 01 Oct 2007 14:43:11 GMT Server: Apache/2.2.7-dev (Unix) mod_ssl/2.2.7-dev OpenSSL/0.9.8d DAV/2 Allow: GET,HEAD,POST,OPTIONS,TRACE Content-Length: 0 Content-Type: text/plain No problem. Regards Rüdiger
Re: Jose's recent location test/failures
Hello, Here's my second take at submitting these tests. Following Bill's comments, I did some changes to remove the ambiguity. These tests check that the directives inside LocationMatch, Directory sections as well as .htaccess are taken into account when processing internal subrequests. The test case consists in using AddCharset to add a bogus charset in the configuration and using conneg to trigger a subrequest. e.g., LocationMatch ^/apache/subrequests/location_override/ Options +MultiViews AddCharset bogus .bc /LocationMatch Although there are six tests, only test four fails when PR 41960 is not applied. I added the extra five just to make sure the flaw won't appear in the other sections. You will find here attached the subrequests.t test file as well as a tgz archive with the tests. Before running the tests, you need to patch extra.conf.in using the included extra.conf.in.patch diff file. This is a diff from svn trunk. Feel free to request further changes. -jose subrequests.t Description: Troff document subrequests-20061001.tgz Description: GNU Unix tar archive
Re: Proxying OPTIONS *
On Mon, 01 Oct 2007 16:43:57 +0200 Ruediger Pluem [EMAIL PROTECTED] wrote: On 10/01/2007 03:30 PM, Joshua Slive wrote: On 10/1/07, Jim Jagielski [EMAIL PROTECTED] wrote: [summary of everyone] No problem. OK, it's actually applying the permissions of DocumentRoot. It's also ignoring the permissions on Location / So my report was wrong, but we still have a bug: we shouldn't be mapping OPTIONS * to the filesystem. You can reproduce the 403 with: Directory / DENY /Directory DocumentRoot /usr/local/apache/htdocs Directory /usr/local/apache/htdocs # no access/authnz directives at all here /Directory Location / ALLOW /Location RFC2616 tells us OPTIONS * is basically a simple HTTP ping, which suggests it could be at a 'lower' level than authconfig and always be allowed. If there is a reason to deny it, that could be by means of something analagous to TraceEnable. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Proxying OPTIONS *
Nick Kew wrote: RFC2616 tells us OPTIONS * is basically a simple HTTP ping, which suggests it could be at a 'lower' level than authconfig and always be allowed. If there is a reason to deny it, that could be by means of something analagous to TraceEnable. Insufficient. If we configure server-forced connection: upgrade/TLS we had better do so in the OPTIONS phase. So I agree that files don't apply. VirtualHost would. Location * should (and I'm not stating LocationMatch .* or Location /, but an explicit case which handles only OPTIONS). But I'm rather against breaking this in 2.2 to solve (what are, today) configuration quirks. Let's get this right for 2.4 and call out the change very clearly in (our overlong) CHANGES? I'm thinking of a new second-priority category after SECURITY:, e.g. CONFIG: or MUSTNOTE: so administrators who migrate aren't surprised. Bill
Re: Proxying OPTIONS *
On Mon, 1 Oct 2007 16:14:14 +0100 Nick Kew [EMAIL PROTECTED] wrote: RFC2616 tells us OPTIONS * is basically a simple HTTP ping, which suggests it could be at a 'lower' level than authconfig and always be allowed. If there is a reason to deny it, that could be by means of something analagous to TraceEnable. An option that fixes this in httpd.conf would be: --- docs/conf/httpd.conf.in (revision 580782) +++ docs/conf/httpd.conf.in (working copy) @@ -113,6 +113,12 @@ Options FollowSymLinks AllowOverride None Require all denied + +# Allow OPTIONS * (simple HTTP ping) +Limit OPTIONS +Order Allow,Deny +Allow from all +/Limit /Directory # Otherwise a simple function running REALLY_FIRST on the access hook could check for OPTIONS. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Proxying OPTIONS *
On Oct 1, 2007, at 12:02 PM, Nick Kew wrote: On Mon, 1 Oct 2007 16:14:14 +0100 Nick Kew [EMAIL PROTECTED] wrote: RFC2616 tells us OPTIONS * is basically a simple HTTP ping, which suggests it could be at a 'lower' level than authconfig and always be allowed. If there is a reason to deny it, that could be by means of something analagous to TraceEnable. An option that fixes this in httpd.conf would be: --- docs/conf/httpd.conf.in (revision 580782) +++ docs/conf/httpd.conf.in (working copy) @@ -113,6 +113,12 @@ Options FollowSymLinks AllowOverride None Require all denied + +# Allow OPTIONS * (simple HTTP ping) +Limit OPTIONS +Order Allow,Deny +Allow from all +/Limit /Directory # Otherwise a simple function running REALLY_FIRST on the access hook could check for OPTIONS. Why not use a quick_handler for the OPTIONS * case?
Re: Proxying OPTIONS *
On Oct 1, 2007, at 11:14 AM, Nick Kew wrote: On Mon, 01 Oct 2007 16:43:57 +0200 Ruediger Pluem [EMAIL PROTECTED] wrote: On 10/01/2007 03:30 PM, Joshua Slive wrote: On 10/1/07, Jim Jagielski [EMAIL PROTECTED] wrote: [summary of everyone] No problem. OK, it's actually applying the permissions of DocumentRoot. It's also ignoring the permissions on Location / So my report was wrong, but we still have a bug: we shouldn't be mapping OPTIONS * to the filesystem. TRACE also does not/should not trace to the filesystem. So, I think what we should do is use the existing architecture and have a quick_handler that checks for the OPTIONS * case and, if so, return DONE. I am not sure, to be honest, what we should do for OPTIONS /foo if /foo is a protected entity... Reading 9.2: communication options available on the request/response chain... without implying a resource action or initiating a resource retrieval implies to me that ACL shouldn't even enter into it and should never return a 403... Which also implies that we should not honor any Limit for Options either... Before I work on the fix (http://issues.apache.org/bugzilla/ attachment.cgi?id=20902 seems just plain wrong to me), I'd like to see what Roy thinks about the above compliance points...
Re: svn commit: r581030 - /httpd/httpd/trunk/modules/proxy/mod_proxy_http.c
On Mon, Oct 01, 2007 at 06:08:13PM -, [EMAIL PROTECTED] wrote: Author: niq Date: Mon Oct 1 11:08:13 2007 New Revision: 581030 URL: http://svn.apache.org/viewvc?rev=581030view=rev Log: No change, but they won't let me have foo (and ... this is the module with a function addit_dammit !!!) Well, at least addit_dammit is descriptive :) -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ If you can dodge a wrench, you can dodge a ball.
Re: Proxying OPTIONS *
Jim Jagielski wrote: TRACE also does not/should not trace to the filesystem. So, I think what we should do is use the existing architecture and have a quick_handler that checks for the OPTIONS * case and, if so, return DONE. You can't ignore the vhost, and preferably would handle the Location * as well in replacement for what Directory /docroot offered before. OPTIONS is a standard mechanism for handling the cart-before-the-horse problems of things like POST with ssl renegotiation. If we can correctly respond that you must Upgrade to SSL, or rehandshake now upon the initial OPTIONS query, their next POST won't fall into that trap. I am not sure, to be honest, what we should do for OPTIONS /foo if /foo is a protected entity... Reading 9.2: communication options available on the request/response chain... without implying a resource action or initiating a resource retrieval implies to me that ACL shouldn't even enter into it and should never return a 403... Which also implies that we should not honor any Limit for Options either... But if OPTIONS /uploads is a directory, while /uploads/ is a PUT-enabled web space, shouldn't we distinguish? w.r.t. auth, if they aren't logged in, /uploads/ doesn't include PUT. Now I'd totally agree that we want a smarter API for OPTIONS to allow resources to look at the auth results to decide 'yea, PUT's in that list' or 'nope, axe PUT'. Before I work on the fix (http://issues.apache.org/bugzilla/attachment.cgi?id=20902 seems just plain wrong to me), I'd like to see what Roy thinks about the above compliance points... Agreed.
Re: Proxying OPTIONS *
On Oct 1, 2007, at 2:17 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: TRACE also does not/should not trace to the filesystem. So, I think what we should do is use the existing architecture and have a quick_handler that checks for the OPTIONS * case and, if so, return DONE. You can't ignore the vhost, and preferably would handle the Location * as well in replacement for what Directory /docroot offered before. OPTIONS is a standard mechanism for handling the cart-before-the-horse problems of things like POST with ssl renegotiation. If we can correctly respond that you must Upgrade to SSL, or rehandshake now upon the initial OPTIONS query, their next POST won't fall into that trap. But all this is still valid at the quick_handler phase... I am not sure, to be honest, what we should do for OPTIONS /foo if /foo is a protected entity... Reading 9.2: communication options available on the request/response chain... without implying a resource action or initiating a resource retrieval implies to me that ACL shouldn't even enter into it and should never return a 403... Which also implies that we should not honor any Limit for Options either... But if OPTIONS /uploads is a directory, while /uploads/ is a PUT- enabled web space, shouldn't we distinguish? w.r.t. auth, if they aren't logged in, /uploads/ doesn't include PUT. That's what I want Roy to clear up... Certainly PUT is a valid communication option, right, it's just that when they do that they get a Auth Required response? You can *do* a PUT, you just may not be *authorized* for the resource, which I think are 2 distinct things.
Re: svn commit: r581030 - /httpd/httpd/trunk/modules/proxy/mod_proxy_http.c
On Oct 1, 2007, at 2:18 PM, Nick Kew wrote: On Mon, 1 Oct 2007 14:12:44 -0400 Jim Jagielski [EMAIL PROTECTED] wrote: Well, at least addit_dammit is descriptive :) Aha, so the struct should've been called holdit_dammit! :)
Re: Proxying OPTIONS *
On Oct 1, 2007, at 2:33 PM, Jim Jagielski wrote: On Oct 1, 2007, at 2:17 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: TRACE also does not/should not trace to the filesystem. So, I think what we should do is use the existing architecture and have a quick_handler that checks for the OPTIONS * case and, if so, return DONE. You can't ignore the vhost, and preferably would handle the Location * as well in replacement for what Directory /docroot offered before. OPTIONS is a standard mechanism for handling the cart-before-the- horse problems of things like POST with ssl renegotiation. If we can correctly respond that you must Upgrade to SSL, or rehandshake now upon the initial OPTIONS query, their next POST won't fall into that trap. But all this is still valid at the quick_handler phase... Hmmm on 2nd thought, map_to_storage is likely the more logical place.
Re: Proxying OPTIONS *
Jim Jagielski wrote: Hmmm on 2nd thought, map_to_storage is likely the more logical place. The answer, of course, is with the next version of apache, to finish abstracting out the filesystem at map_to_storage; where there is no DocumentRoot / FilePathAlias (e.g. alias) to force some other provider to serve the request, or fail :) httpd 2.2 remains far too filesystem-centric. Bill
Re: Proxying OPTIONS *
On 10/1/07, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: But I'm rather against breaking this in 2.2 to solve (what are, today) configuration quirks. Let's get this right for 2.4 and call out the change very clearly in (our overlong) CHANGES? I'm thinking of a new second-priority category after SECURITY:, e.g. CONFIG: or MUSTNOTE: so administrators who migrate aren't surprised. Should be in this, rather sparse file: http://httpd.apache.org/docs/trunk/new_features_2_4.html Joshua.
Re: Proxying OPTIONS *
Joshua Slive wrote: On 10/1/07, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: But I'm rather against breaking this in 2.2 to solve (what are, today) configuration quirks. Let's get this right for 2.4 and call out the change very clearly in (our overlong) CHANGES? I'm thinking of a new second-priority category after SECURITY:, e.g. CONFIG: or MUSTNOTE: so administrators who migrate aren't surprised. Should be in this, rather sparse file: http://httpd.apache.org/docs/trunk/new_features_2_4.html But it's not a feature-per say. It's a bugfix, so the name new_features doesn't tell admins they have to adopt a change (new feature implies there's a goodie I can exploit if I choose to)... ...and hiding in docs isn't really the best place for major config-changing bullet points that will break their previously working, 2.2 server in some unexpected way. Bill
Re: Proxying OPTIONS *
On Mon, Oct 01, 2007 at 12:05:41PM -0700, Roy T. Fielding wrote: On Oct 1, 2007, at 11:02 AM, Jim Jagielski wrote: TRACE also does not/should not trace to the filesystem. So, I think what we should do is use the existing architecture and have a quick_handler that checks for the OPTIONS * case and, if so, return DONE. fine. I am not sure, to be honest, what we should do for OPTIONS /foo if /foo is a protected entity... Reading 9.2: communication options available on the request/response chain... without implying a resource action or initiating a resource retrieval implies to me that ACL shouldn't even enter into it and should never return a 403... Which also implies that we should not honor any Limit for Options either... No, what the client wants are the communication options. It is commonly used to find out what is required for a PUT before the request with big body is sent. We want to return 401, 403, ... Great! That's exactly what I needed to know. So it seems to me that a map_to_storage to check for the special case of '*' whereas present action for all other URIs is the best course of action. -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ If you can dodge a wrench, you can dodge a ball.
Re: Proxying OPTIONS *
On 10/1/07, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: Joshua Slive wrote: Should be in this, rather sparse file: http://httpd.apache.org/docs/trunk/new_features_2_4.html But it's not a feature-per say. It's a bugfix, so the name new_features doesn't tell admins they have to adopt a change (new feature implies there's a goodie I can exploit if I choose to)... Sorry, my little tiny contribution to this thread was less than useful. I meant the even more sparse: http://httpd.apache.org/docs/trunk/upgrading.html ...and hiding in docs isn't really the best place for major config-changing bullet points that will break their previously working, 2.2 server in some unexpected way. Hmmm... Hiding in an enormous, mostly-indecipherable CHANGES file is better? I think the upgrading guide is exactly where that stuff belongs. Joshua.
Re: Proxying OPTIONS *
Jim Jagielski wrote: Great! That's exactly what I needed to know. So it seems to me that a map_to_storage to check for the special case of '*' whereas present action for all other URIs is the best course of action. Provided it's vetted against the vhost (it is) and against location * then ++1, sounds great! (Note we could even shortcut everything after one location * mapping and not do the followup location * remapping - which occurs on all other patterns such as directory or proxy blocks!) Bill
Re: Proxying OPTIONS *
Joshua Slive wrote: On 10/1/07, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: Joshua Slive wrote: Should be in this, rather sparse file: http://httpd.apache.org/docs/trunk/new_features_2_4.html But it's not a feature-per say. It's a bugfix, so the name new_features doesn't tell admins they have to adopt a change (new feature implies there's a goodie I can exploit if I choose to)... Sorry, my little tiny contribution to this thread was less than useful. I meant the even more sparse: http://httpd.apache.org/docs/trunk/upgrading.html Woot :) Thanks.
Re: Proxying OPTIONS *
On Mon, Oct 01, 2007 at 03:22:34PM -0400, Jim Jagielski wrote: On Mon, Oct 01, 2007 at 12:05:41PM -0700, Roy T. Fielding wrote: On Oct 1, 2007, at 11:02 AM, Jim Jagielski wrote: TRACE also does not/should not trace to the filesystem. So, I think what we should do is use the existing architecture and have a quick_handler that checks for the OPTIONS * case and, if so, return DONE. fine. I am not sure, to be honest, what we should do for OPTIONS /foo if /foo is a protected entity... Reading 9.2: communication options available on the request/response chain... without implying a resource action or initiating a resource retrieval implies to me that ACL shouldn't even enter into it and should never return a 403... Which also implies that we should not honor any Limit for Options either... No, what the client wants are the communication options. It is commonly used to find out what is required for a PUT before the request with big body is sent. We want to return 401, 403, ... Great! That's exactly what I needed to know. So it seems to me that a map_to_storage to check for the special case of '*' whereas present action for all other URIs is the best course of action. oops... one other thing. Should we allow Limit to restrict OPTIONS? Or should Limit not affect OPTIONS as an allowed method...? -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ If you can dodge a wrench, you can dodge a ball.
Re: Proxying OPTIONS *
On Mon, Oct 01, 2007 at 02:30:30PM -0500, William A. Rowe, Jr. wrote: Jim Jagielski wrote: Great! That's exactly what I needed to know. So it seems to me that a map_to_storage to check for the special case of '*' whereas present action for all other URIs is the best course of action. Provided it's vetted against the vhost (it is) and against location * then ++1, sounds great! But, as I read it, the '*' in OPTIONS * does not really mean a Location *... in other words, it's not a URI per se. OPTIONS * asks for the capabilities of the server itself, independent of URI... At least, that's how I read it. -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ If you can dodge a wrench, you can dodge a ball.
Re: Proxying OPTIONS *
Jim Jagielski wrote: But, as I read it, the '*' in OPTIONS * does not really mean a Location *... in other words, it's not a URI per se. OPTIONS * asks for the capabilities of the server itself, independent of URI... At least, that's how I read it. There is no 'real' Location * There's a Location /*, or a LocationMatch .* But since Location is segment-delimited, Location * would only affect OPTIONS *. Bill
Re: Proxying OPTIONS *
On sön, 2007-09-30 at 16:54 -0700, Roy T. Fielding wrote: On Sep 30, 2007, at 4:05 PM, Nick Kew wrote: RFC2616 is clear that: 1. OPTIONS * is allowed. 2. OPTIONS can be proxied. However, it's not clear that OPTIONS * can be proxied, given that there's no natural URL representation of it (* != /*). An absolute http request-URI with no path. In RFC2068 yes, but not RFC2616.. Regards Henrik signature.asc Description: This is a digitally signed message part
As we contemplate what to fix, and how to roll out 2.4 and 3.0
this is something really worth pondering; http://www.securityspace.com/s_survey/server_graph.html?type=httpdomaindir=month=200709servbase=YToyOntpOjA7czoxMzoiQXBhY2hlLzIuMC41OSI7aToxO3M6MTM6IkFwYWNoZS8xLjMuMzciO30=serv1=QXBhY2hlLzIuMi40 Give that some thought :) Bill
Re: As we contemplate what to fix, and how to roll out 2.4 and 3.0
William A. Rowe, Jr. wrote: Give that some thought :) One thing I'm pondering is a 2.3.0 alpha in the near future. If only to give the we stay back at version n.x-1 crowd something to chew on. Not to mention that it would be good for folks to start exploring what needs to be fixed in the API, etc. Bill
RE: 2.0.54 unstable, requests time-out, NO warnings in logs
Have you checked without the MaxMemFree setting? Why do you use MaxMemFree with such a small value at all? I finally removed MaxMemFree altogether, and it crashed again. Nothing in the apache error logs, but this is how /server-status looks like during the crash: 300 requests currently being processed, 0 idle workers WCRRCCCRRRCCRWWRRRCRWRRCRRCRWRRWRRRC RRCCRCRRWWWCRRRWRRRWRCRRRCRCRRCRRRCCCRRCCCWR RCWWRRRWRWRWCCCWWWRCRCRCCWRRWRCRCRWC WRRRWRRRCRRCRRCRRRCCCCRWCRRRCRRR RRCCRWCRCRRRWWCWRRCWRRRCRRCRRRCR Immediately after I restarted the apache after the crash, I did get [Mon Oct 01 15:20:49 2007] [notice] mod_python: Creating 32 session mutexes based on 300 max processes and 0 max threads. [Mon Oct 01 15:20:49 2007] [notice] Apache/2.0.54 (Unix) mod_python/3.1.4 Python/2.4.1 configured -- resuming normal operations [Mon Oct 01 15:21:25 2007] [error] server reached MaxClients setting, consider raising the MaxClients setting*** but it's strange that this message was not written before or during the crash, even though /server-status shows no available free child processes. -Original Message- From: Ruediger Pluem [mailto:[EMAIL PROTECTED] Sent: Monday, October 01, 2007 1:23 AM To: dev@httpd.apache.org Subject: Re: 2.0.54 unstable, requests time-out, NO warnings in logs On 10/01/2007 08:32 AM, Alec Matusis wrote: We are running a busy Apache/2.0.54 server on 2.6.9 kernel, that suddenly becomes very slow- requests either time out, or it takes 10- 20sec to serve a 1K thumbnail. It is somewhat correlated with load spikes, but not perfectly (by looking at the bandwidth graph, it never happens during the low bandwidth periods at night, but it does not coincide with peaks of b/w) When we initially encountered an apache overload, it was always accompanied with [error] server reached MaxClients setting, consider raising the MaxClients setting in the apache error log. We also got kernel: possible SYN flooding on port 80. Sending cookies. in /var/log/messages system log. After that I raised MaxClients from 200 to 300. The problem initially disappeared, but after our bandwidth grew a bit more, we got this behavior again. Now apache crashes (becomes very slow) silently, with no warning in apache error logs at all (although we still get SYN flood message in the system log) When apache is this 'slow' regime, /server-status still shows available slots, i.e. MaxClients is not reached. This is the relevant part of httpd.conf: ServerLimit 300 # we are using prefork MPM StartServers 10 MinSpareServers 5 MaxSpareServers 20 MaxClients 300 MaxRequestsPerChild 1 MaxMemFree 2500 The server has 4GB of physical RAM and 4GB of swap. During these apache slowdowns, the swap size is still 0 and vmstat shows no swapping at all. I suspect the problem may be in MaxMemFree 2500 Have you checked without the MaxMemFree setting? Why do you use MaxMemFree with such a small value at all? Regards Rüdiger
Re: As we contemplate what to fix, and how to roll out 2.4 and 3.0
William A. Rowe, Jr. wrote: this is something really worth pondering; http://www.securityspace.com/s_survey/server_graph.html?type=httpdomaindir=month=200709servbase=YToyOntpOjA7czoxMzoiQXBhY2hlLzIuMC41OSI7aToxO3M6MTM6IkFwYWNoZS8xLjMuMzciO30=serv1=QXBhY2hlLzIuMi40 Give that some thought :) I'm not sure what we are supposed to think about? Lots of people still use 1.3 for their own reasons, its not going to hold me back on things I would like to do in 2.4 or 3.0. -Paul
Re: As we contemplate what to fix, and how to roll out 2.4 and 3.0
Paul Querna wrote: I'm not sure what we are supposed to think about? Lots of people still use 1.3 for their own reasons, its not going to hold me back on things I would like to do in 2.4 or 3.0. Good point; maybe this is part of the issue, what we *already* do today in 2.2 etc, vs. what users are aware of? And of course, making them aware of what we do introduce to 2.4 or 3.0.
Re: As we contemplate what to fix, and how to roll out 2.4 and 3.0
On 10/1/2007 at 4:52 PM, in message [EMAIL PROTECTED], William A. Rowe, Jr. [EMAIL PROTECTED] wrote: William A. Rowe, Jr. wrote: Give that some thought :) One thing I'm pondering is a 2.3.0 alpha in the near future. If only to give the we stay back at version n.x-1 crowd something to chew on. Not to mention that it would be good for folks to start exploring what needs to be fixed in the API, etc. +1, It's been almost 2 years since the new provider based authorization code was added to 2.3. I would really like to see how it stands up. Brad
Proxy: Handling Interim Responses
RFC2616 mandates that a proxy MUST return interim (1xx) responses to an HTTP/1.1 client, except where the proxy itself requested the interim response. I'd interpret that slightly liberally, to mean we MUST return an interim response if the Client has asked for one. Our proxy currently eats all 1xx responses. That's broken. It could possibly have some bearing on that elusive PR 37770. As I see it: (1) 100-Continue should be forwarded to the client if there's an Expect header asking for it. If there isn't, then it really doesn't matter. (2) 101 Switching Protocols needs a framework to plug in a provider for switched protocols. In the absence of one, we should instead return 502. However, since we strip out any Upgrade request header, the question is only one of error-correction, and current behaviour is probably no worse. (3) Any other 1xx is unrecognised, and it's not clear to me whether it's best to return 502 or to forward the interim response. I'm not sure how I should forward an interim response to the client: suggestions welcome. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
