RE: svn commit: r1302444 - in /httpd/httpd/trunk: CHANGES modules/proxy/mod_proxy.c
-Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Montag, 19. März 2012 15:31 To: dev@httpd.apache.org Subject: Re: svn commit: r1302444 - in /httpd/httpd/trunk: CHANGES modules/proxy/mod_proxy.c On Mar 19, 2012, at 9:53 AM, rpl...@apache.org wrote: Modified: httpd/httpd/trunk/modules/proxy/mod_proxy.c URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/mod_proxy. c?rev=1302444r1=1302443r2=1302444view=diff === === --- httpd/httpd/trunk/modules/proxy/mod_proxy.c (original) +++ httpd/httpd/trunk/modules/proxy/mod_proxy.c Mon Mar 19 13:53:28 2012 @@ -2461,11 +2461,11 @@ static void child_init(apr_pool_t *p, se ap_proxy_hashfunc(reverse-s-name, PROXY_HASHFUNC_FNV); /* Do not disable worker in case of errors */ reverse-s-status |= PROXY_WORKER_IGNORE_ERRORS; -conf-reverse = reverse; ap_proxy_initialize_worker(conf-reverse, s, conf-pool); /* Disable address cache for generic reverse worker */ reverse-s-is_address_reusable = 0; } +conf-reverse = reverse; s = s-next; } } Is that right? Doesn't that mean that the ap_proxy_initialize_worker() call gets an unknown/undefined 1st arg (conf-reverse)?? Yes :-). The original reporter already pointed that out. Fixed in r1302483. Regards Rüdiger
RE: printing r-filename for access denied errors
-Original Message- From: Eric Covener [mailto:cove...@gmail.com] Sent: Freitag, 16. März 2012 12:55 To: dev@httpd.apache.org Subject: printing r-filename for access denied errors Seems like IRC users are often confused that permission denied errors include the URI only and not the filesystem path. (They're convinced it's failing because httpd is looking in the wrong place for /index.html, or they think we forgot to add a documentroot, or have no idea where /foo/bar/baz is supposed to be in the filesystem) Is there any harm in adding it? This is the rv from a stat in the directory walk. +1 Regards Rüdiger
RE: printing r-filename for access denied errors
-Original Message- From: Nick Kew Sent: Freitag, 16. März 2012 14:50 To: dev@httpd.apache.org Subject: Re: printing r-filename for access denied errors On Fri, 16 Mar 2012 07:54:37 -0400 Eric Covener cove...@gmail.com wrote: Seems like IRC users are often confused that permission denied errors include the URI only and not the filesystem path. (They're convinced it's failing because httpd is looking in the wrong place for /index.html, or they think we forgot to add a documentroot, or have no idea where /foo/bar/baz is supposed to be in the filesystem) Is there any harm in adding it? This is the rv from a stat in the directory walk. Yes, there is harm. Exposing filesystem information will bring in a flood of vulnerability reports. Remember the kerfuffle we had about inodes appearing in etags? The vulenerability report about inodes in etags was because a HTTP client could read the inode information (Do not want to rehash the discussion here if this is really a vulnerability if a HTTP client retrieves this information). In this case the information is kept on the server and only written to the logfile. I see no vulnerability here and IMHO vulnerability reports on this should be easy to fend off. Regards Rüdiger
RE: httpd 2.4.1 and mod_slotmem_shm / mod_proxy_balancer (AH01179)
Sounds reasonable. -Original Message- From: Jeff Trawick [mailto:traw...@gmail.com] Sent: Dienstag, 6. März 2012 14:37 To: dev@httpd.apache.org Subject: Re: httpd 2.4.1 and mod_slotmem_shm / mod_proxy_balancer (AH01179) On Tue, Mar 6, 2012 at 7:56 AM, Jim Jagielski j...@jagunet.com wrote: OK... What I'll do is add a directive which provides a default location for slotmem file... Uhh, that seems as endless as per-mutex directives. Is slotmem not using DEFAULT_REL_RUNTIMEDIR already? (not perfect, but a good start) Directive to specify runtime directory (API returns serverroot + DEFAULT_REL_RUNTIMEDIR if not configured). Directive like Mutex but for shmem?
RE: httpd 2.4.1 and mod_slotmem_shm / mod_proxy_balancer (AH01179)
The files that you see in strace are not mutex files. Hence the mutex directive
cannot
work here. The correct fix would be IMHO another directive (either for
mod_proxy or better
for mod_proxy_balancer) to allow defining a directory where these shared memory
files should be created.
Regards
Rüdiger
-Original Message-
From: Zisis Lianas Sent: Montag, 5. März 2012 16:47
To: dev@httpd.apache.org
Subject: httpd 2.4.1 and mod_slotmem_shm / mod_proxy_balancer (AH01179)
Hi,
I think there is an issue in mod_slotmem_shm / mod_proxy_balancer
with httpd 2.4.x when building and installing as root, but trying
to run httpd as standard unix-user.
Scenario:
my httpd is installed as 'root' in /root/httpd-2.4.1/, permissions
root:root/0755. When I create a 'user' httpd.conf and load
slotmem_shm_module,
proxy_module, proxy_http_module and proxy_balancer_module and do some
balancer configuration, the httpd doesn't come up. Error log (trace8):
[Mon Mar 05 11:36:40.739013 2012] [proxy_balancer:debug] [pid 27793:tid
140642808817440] mod_proxy_balancer.c(751): AH01178: Doing balancers
create: 544, 1 (6)
[Mon Mar 05 11:36:40.739054 2012] [proxy_balancer:emerg] [pid 27793:tid
140642808817440] (13)Permission denied: AH01179: balancer
slotmem_create failed
[Mon Mar 05 11:36:40.739080 2012] [:emerg] [pid 27793:tid
140642808817440] AH00020: Configuration Failed, exiting
In strace you can see that httpd is trying to create mutex(?) files in
installation root directory, which belongs to user 'root' and is not
writeable for other users:
open(/root/httpd-2.4.1/s29df2056, O_RDONLY|O_CLOEXEC) = -1 ENOENT (No
such file or directory)
open(/root/httpd-2.4.1/s29df2056, O_WRONLY|O_CLOEXEC) = -1 ENOENT (No
such file or directory)
open(/root/httpd-2.4.1/s29df2056, O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC,
0666) = -1 EACCES (Permission denied)
I tried to change the mutex location with the mutex directive and
proxy-balancer-shm, but that doesn't work:
Mutex file:/tmp/ proxy-balancer-shm
Syntax seems to be OK, but this configuration item is ignored
completely.
config extraction:
ServerRoot /root/httpd-2.4.1
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
LoadModule lbmethod_byrequests_module
modules/mod_lbmethod_byrequests.so
Mutex file:/tmp/ proxy-balancer-shm
Proxy balancer://default
BalancerMember http://appserver1.localhost:3001 route=0
BalancerMember http://appserver2.localhost:3001 route=1
ProxySet lbmethod=byrequests stickysession=JSESSIONID|jsessionid
/Proxy
ProxyPassMatch /servlet/ balancer://default/
ProxyPassReverse / balancer://default/
My quick hack to get my apache instance started:
--- httpd-2.4.1/modules/slotmem/mod_slotmem_shm.c
+++ httpd-2.4.1/modules/slotmem/mod_slotmem_shm.c
@@ -269,7 +269,10 @@
}
if (name) {
if (name[0] != '/') {
-fname = ap_server_root_relative(pool, name);
+char file_name[100];
+strcpy(file_name, /tmp/);
+strcat(file_name, name);
+fname = file_name;
}
else {
fname = name;
I think it would make sense to check why the mutex configuration of
proxy-balancer-shm is ignored.
Best regards,
Zisis
RE: Intent to TR 2.4.1
:-) -Original Message- From: Rainer Jung [mailto:rainer.j...@kippdata.de] Sent: Montag, 13. Februar 2012 08:20 To: dev@httpd.apache.org Subject: Re: Intent to TR 2.4.1 Lame mathematician joke: 2.4 at 2^4 (16th Apache birthday). Regards, Rainer
RE: [PATCH] AIX configure options
--enable-mpms-shared=all \ will not work with 2.2. Only 2.4 and up. With 2.2 you still need - --with-mpm=worker Regards Rüdiger From: Michael Felt [mailto:mamf...@gmail.com] Sent: Mittwoch, 8. Februar 2012 15:08 To: dev@httpd.apache.org; packag...@httpd.apache.org Subject: Re: [PATCH] AIX configure options Well, it was building something. I'll try the proposed change and make sure it is still giving me all the mods. FYI - I have added --enable-ssl (so that the same script works for both 2.2.22 and 2.4.0 which have different ideas of all. And I hope configure will still leave AIX modules as shared (as it used to make them all static by default - long ago). I shall try both 2.4.0 and 2.2.22 (without --enable-ssl) - and I am continuing to work towards to havin g APR external to httpd. On Tue, Feb 7, 2012 at 8:09 PM, Jeff Trawick traw...@gmail.com wrote: The buildaix.ksh script committed recently had some outdated options or requirements, and didn't build loadable MPMs. Suggested changes are below. Do or do not fold, spindle, or mutilate; commit with or without testing; etc. Index: build/aix/buildaix.ksh === --- build/aix/buildaix.ksh (revision 1241549) +++ build/aix/buildaix.ksh (working copy) @@ -30,11 +30,7 @@ nohup.out ./configure \ --enable-layout=$LAYOUT \ - --enable-module=so \ - --enable-proxy \ - --enable-cache \ - --enable-disk-cache \ - --with-mpm=worker \ + --enable-mpms-shared=all \ --enable-mods-shared=all | tee nohup.out make | tee -a nohup.out -- Born in Roswell... married an alien...
RE: OpenSSL configuration and mod_ssl
-Original Message- From: Dr Stephen Henson [mailto:shen...@opensslfoundation.com] Sent: Donnerstag, 2. Februar 2012 15:14 To: dev@httpd.apache.org Subject: OpenSSL configuration and mod_ssl Guys, It has been apparent for some time that mod_ssl (and other applications) require a considerable effort to support new features in OpenSSL. A simple example is when a new flag is added which some, but not all, users may want to set. Once this flag appears in an OpenSSL release every OpenSSL based application needs to be modified to support and document it. Specification of this option might be via a command line option or (in the case of mod_ssl and others) a configuration file. It would IMHO be far better if a mechanism existed to support automatic configuration of some options by conforming applications. There is a current example where this works well: the cipher string. With the inclusion of TLS v1.2 in the upcoming OpenSSL 1.0.1 release several new ciphersuites based on SHA256 and GCM have appeared. An application generally doesn't need to know or care what these are. A user can enable or disable them by just using the cipher string: it is passed as an opaque string which OpenSSL interprets. So my thoughts are that this concept could be generalised. A simple answer is to add new string setting options. For example: int SSL_CTX_set_options_string(SSL_CTX *ctx, const char *str); +1 in principle. Could be handy for mod_ssl. This works for existing simple configuration but a new string (for example TLS 1.2 supported signature algorithms) might be added in the future so then we're back to having to explicitly add support to all applications for each new string configuration option. So perhaps: int SSL_CTX_set_config_string(SSL_CTX *ctx, const char *name, const char *value); Where the values of name can expand over time. +1 same as above. I'm not completely sure that this could be handled by the mod_ssl configuration routines, perhaps someone could comment on that? A third method is to delegate the configuration completely to OpenSSL using a separate configuration file. So, we'd have an option to set the configuration file to use and then something like: int SSL_CTX_config(SSL_CTX *ctx, const char *config_name); -0 from mod_ssl perspective. How do you configure which configuration file to use in this case? If it is the system wide one I don't regard this as beneficial as a web server operator might not have write access to it. Regards Rüdiger
RE: [VOTE] Bundle apr/apu with 2.4.x
-Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Donnerstag, 2. Februar 2012 15:54 To: dev@httpd.apache.org Subject: [VOTE] Bundle apr/apu with 2.4.x I'm calling a vote to get consensus on whether we should continue to bundle apr/apu with httpd 2.4.x. The proposal is that at the time we TR 2.4.x, we also bundle that latest, released versions of apr/apu with the httpd tarball. How we bundle it (eg: sep tarball or have it part of the httpd tarball ala 2.2.x) and what we call it are *not* part of the vote, the idea being that if we do wish to bundle it, we can decide the best way to do so. If we don't wish to bundle, the issue is moot. Let's give it the normal 72 hours: [X] +1: Bundle apr/apu w/ Apache httpd 2.4.x [ ] +0: I don't care [ ] -1: Do not bundle apr/apu with Apache httpd 2.4.x Cheers! Regards Rüdiger
RE: svn commit: r1238824 - in /httpd/httpd/branches/2.4.x: ./ CHANGES server/core.c
Never mind. I just saw that it was done in r1238833. Regards Rüdiger -Original Message- From: Rüdiger Plüm [mailto:ruediger.pl...@vodafone.com] Sent: Mittwoch, 1. Februar 2012 09:43 To: dev@httpd.apache.org Subject: Fwd: svn commit: r1238824 - in /httpd/httpd/branches/2.4.x: ./ CHANGES server/core.c Original Message Subject: svn commit: r1238824 - in /httpd/httpd/branches/2.4.x: ./ CHANGES server/core.c Date: Tue, 31 Jan 2012 21:50:03 GMT From: s...@apache.org Author: sf Date: Tue Jan 31 21:50:03 2012 New Revision: 1238824 URL: http://svn.apache.org/viewvc?rev=1238824view=rev Log: Merge r1225199: Check during configtest that the directories for error logs exist PR: 29941 Don't we need to backport r1225223 as well? Regards Rüdiger
RE: Questions on open ports between trunk and 2.4.x
-Original Message- From: Rainer Jung [mailto:rainer.j...@kippdata.de] Sent: Dienstag, 31. Januar 2012 17:37 To: dev@httpd.apache.org Subject: Re: Questions on open ports between trunk and 2.4.x On 31.01.2012 17:17, Graham Leggett wrote: On 31 Jan 2012, at 5:43 PM, Rainer Jung wrote: All the items listed are actual code differences. The log info was the shortest way to explain the differences. I tried hard to verify, that the differences I've seen are actually the ones generatde by these missing commits. In this case check modules/cache/cache_storage.c modules/cache/cache_storage.h modules/cache/mod_cache.c for differences. The code in v2.4 is correct. The code on trunk still has work outstanding to address cache invalidation, an issue that is an edge case in HTTP and needs more work on it. So: [ X ] leave trunk as is, i.e. do not forward port the changes that are in 2.4 but not in trunk I follow Grahams arguments here. Regards Rüdiger
RE: [RFC] further proxy/rewrite URL validation security issue (CVE-2011-4317)
-Original Message- From: Joe Orton [mailto:jor...@redhat.com] Sent: Mittwoch, 23. November 2011 15:23 To: dev@httpd.apache.org Subject: [RFC] further proxy/rewrite URL validation security issue (CVE-2011-4317) Prutha Parikh from Qualys reported a variant on the CVE-2011-3368 attack against certain mod_proxy/mod_rewrite configurations. A new CVE name, CVE-2011-4317, has been assigned to this variant. The configurations in question are the same as affected by -3368, e.g.: RewriteRule ^(.*) http://www.example.com$1 [P] and the equivalent ProxyPassMatch. Request examples are: GET @localhost::8880 HTTP/1.0\r\n\r\n GET qualys:@qqq.qq.qualys.com HTTP/1.0\r\n\r\n These unfortunately do not get trapped in the request parsing trap added in r1179239, so result in an input to rewrite rule processing which does not match the URL-path grammar (i.e. does not start with /). We could try improve that fix, but I think it would be simpler to change the translate_name hooks in mod_proxy and mod_rewrite to enforce the requirement in the right place. Other translate_name hooks do this already. I propose the patch below. Any comments? +1. Go ahead with the patch. One comment though: Shouldn't we check r-unparsed_uri as well (at least in the proxy case, as it may be used by ap_proxy_trans_match instead of r-uri)? Regards Rüdiger
RE: [Vote] .htaccess logic abuse
-Original Message- From: Stefan Fritsch [mailto:s...@sfritsch.de] Sent: Samstag, 19. November 2011 03:37 To: dev@httpd.apache.org Subject: Re: [Vote] .htaccess logic abuse On Friday 18 November 2011, William A. Rowe Jr. wrote: Resource abuse of an .htaccess config in the form of cpu/memory/bandwidth; [ ] Represents a security defect [X] Is not a security defect This would obviously need to be clarified in the associated .htaccess documentation, be associated with an advisory and affect the conclusion of several recent defect reports, both embargoed and discussed plainly here on this list. We should not make any promises we won't be able to keep. There are countless ways to cause a DoS from .htaccess. The .htaccess mechanism has not been designed with resource limitation in mind. Changing that will be a lot of work and will likely break ABI/API, i.e. the fixes won't be backportable to stable releases. We should treat those issues as regular bugs and make DoS safe .htaccess a goal. But we should make it clear that this goal likely won't be reached in 2.4.x and earlier. +1. Well put. Regards Rüdiger
RE: Win 2.3.15 :: The timeout specified has expired
-Original Message- From: Steffen [mailto:i...@apachelounge.com] Sent: Montag, 21. November 2011 11:50 To: dev@httpd.apache.org Subject: Win 2.3.15 :: The timeout specified has expired Observing that the error.log is filling with [http:error] lines, never seen with 2.2: [http:error] [pid 3244:tid 2656] (70007)The timeout specified has expired: [client 114.79.60.32:11091] Timeout while writing data for URI /download/binaries/httpd-2.2.21-win32-x86-ssl.zip to the client. New feature ? Happens when client is downloading partial (206). What is the timeout specified in the line and can it changed ? Should be the normal Timeout parameter. Regards Rüdiger
RE: CVE-2011-3607, int overflow ap_pregsub()
The patch is fine on trunk because the affected code is not within
AP_DECLARE(char *) ap_pregsub(...)
but within
static apr_status_t regsub_core(apr_pool_t *p, char **result,
struct ap_varbuf *vb, const char *input,
const char *source, size_t nmatch,
ap_regmatch_t pmatch[], apr_size_t maxlen)
but there is no regsub_core in 2.2.x. So the patch needs to be adjusted for
backport
to 2.2.x. But returning NULL in the 2.2.x case looks to be the correct thing to
do
as this is how trunk behaves now.
OTOH there was some discussion on this list whether it is correct to backport
this trunk
behaviour to 2.2.x.
Regards
Rüdiger
-Original Message-
From: Roman Drahtmueller [mailto:dr...@suse.de]
Sent: Dienstag, 15. November 2011 15:13
To: dev@httpd.apache.org
Subject: CVE-2011-3607, int overflow ap_pregsub()
Hi there,
Revision 1198940 attempts to fix an integer overflow in
ap_pregsub() in
server/util.c:394. The patch is:
--- httpd/httpd/trunk/server/util.c 2011/11/07 21:09:41 1198939
+++ httpd/httpd/trunk/server/util.c 2011/11/07 21:13:40 1198940
@@ -411,6 +411,8 @@
len++;
}
else if (no nmatch pmatch[no].rm_so
pmatch[no].rm_eo) {
+if (APR_SIZE_MAX - len = pmatch[no].rm_eo -
pmatch[no].rm_so)
+return APR_ENOMEM;
len += pmatch[no].rm_eo - pmatch[no].rm_so;
}
, and appears wrong, because, ap_pregsub() is
AP_DECLARE(char *) ap_pregsub(...)
This would require something along the lines of (proposal):
}
else if (no nmatch pmatch[no].rm_so
pmatch[no].rm_eo) {
+if (APR_SIZE_MAX - len = pmatch[no].rm_eo -
pmatch[no].rm_so) {
+ ap_log_error(APLOG_MARK, APLOG_WARNING,
APR_ENOMEM, NULL,
+ integer overflow or out of memory
condition. );
+return NULL;
+ }
len += pmatch[no].rm_eo - pmatch[no].rm_so;
}
}
dest = dst = apr_pcalloc(p, len + 1);
+if(!dest)
+ return NULL;
+
+
/* Now actually fill in the string */
...or simply without the error logging.
Thoughts?
Thanks,
Roman.
RE: Changes in mod_ssl
mod_ssl is part of Apache Http Server 2.0.x and up. Just open a report in bugzilla and attach the patch as a proposed enhancement. Further discussion on this patch might happen there or here (depending on the contents of the discussion). Regards Rüdiger From: Moran Jacuel [mailto:mor...@arx.com] Sent: Montag, 14. November 2011 15:00 To: dev@httpd.apache.org Cc: Moshe Harel Subject: Changes in mod_ssl Hello, Our company is an HSM manufacturer (See link for http://www.arx.com/products/private-server-hsm PrivateServer product) We wanted to connect apache server with SSL using our HSM to hold the Private RSA and Certificate. We downloaded apache httpd-2.2.20 and modified the module mod_ssl that came with the package in a generic way to work with OpenSSL PKCS#11 engine. Now we want to add the small code changes we made to the open source code. It is not clear to us if the mod_ssl is part of the Apache project or not. If so, can you please explain us who we need to contact and what is the procedure we need to follow. Regards Moran Jacuel | Software Engineer | ARX phone: +972.3.9279512 | email: mor...@arx.com blocked::mailto:rot...@arx.com | www.arx.com blocked::blocked::http://www.arx.com/n
RE: [VOTE] Formal deprecation of 2.0.x branch
+1 Regards Rüdiger -Original Message- From: William A. Rowe Jr. Sent: Freitag, 11. November 2011 17:49 To: dev@httpd.apache.org Subject: [VOTE] Formal deprecation of 2.0.x branch Stealing a plan executed by Colm for 1.3, I'd like to propose that we set a two week window following committers' return-from-ApacheCon to execute any backports of general interest and apply important fixes/backports to pregsub allocation and non-absolute uri parsing. On approval of this plan, I would offer to introduce the EOL notices (as we ultimately committed to 1.3), tag and roll 2.0.65 on Nov 26th and we would potentially approve and release 2.0 'final' this month. And as we did with 1.3, we would start a 12 month clock to removing 2.0.x pretty much in its entirety from the live httpd.apache.org site and /dist/ mirrors (although still available from archive.a.o/dist/). Otherwise we will be developing towards 4 releases even beyond this month, which seems excessive when the current stable has rendered the prior version obsolete six years ago, on December 1, 2005.
RE: Memory Leak when working with AliasMatch and Cache
I don't think that this has something to do with AliasMatch. Most likely this is caused by reading the file that should be delivered into some request pool backed memory in order to write it to the cache. If you don't cache the file it is probably not read by httpd but transported via a sendmail like mechanism (transferfile on Windows?). Regards Rüdiger From: Ofer Israeli [mailto:of...@checkpoint.com] Sent: Mittwoch, 9. November 2011 14:38 To: dev@httpd.apache.org Subject: Memory Leak when working with AliasMatch and Cache Hi all, We have just witnessed a memory leak in the case of Apache 2.2.16 with WinOpenSSL running on Windows when downloading a file that is served from the disk via the AliasMatch directive and does not reside in the cache but is configured as cacheable. In this case, we see that the child process memory grows by the same size as the size of the file being downloaded and the memory usage does not decline. After this initial download, the file is cached and when downloading the file again, there is no memory leak. We have also sent that when Apache is configured not to cache this file and it is served via AliasMatch there is no leak. Anyone seen this before? Thanks, Ofer Israeli Team Leader, Endpoint Security Management Check Point Software Technologies http://www.checkpoint.com/ * +972-3-753-4715 | M +972-54-749-0320
RE: Fwd: svn commit: r1197405 - in /httpd/httpd/trunk: CHANGES docs/manual/upgrading.xml modules/filters/mod_substitute.c
-Original Message- From: Stefan Fritsch [mailto:s...@sfritsch.de] Sent: Dienstag, 8. November 2011 03:15 To: dev@httpd.apache.org Subject: Re: Fwd: svn commit: r1197405 - in /httpd/httpd/trunk: CHANGES docs/manual/upgrading.xml modules/filters/mod_substitute.c On Sun, 6 Nov 2011, Stefan Fritsch wrote: Isn't it sufficient to reduce space_left only by repl_len? IMHO we only allocate repl_len new memory in ap_pregsub_ex and not len + repl_len or am I wrong here? Any comment on that? That was also supposed to cause the line length to be limited instead of the length of the replacement. But you are right, this is not correct. I will look at it during the hackathon. I have looked more closely, and the code is actually correct. It's purpose is not to limit new memory allocation but the line length. I have added a few comments in r1199060 which hopefully make the intentions clear. len = (apr_size_t) (regm[0].rm_eo - regm[0].rm_so); +repl_len = strlen(repl); +space_left -= len + repl_len; But len is the length of the string we remove correct? So the limit will be imposed on the sum of the length of the strings to replace plus the sum of all replacements for this line. So actually the length of the original and the resulting line can be much shorter than the allowed limit. Example: Lets say the limit is 10 chars per line. The source line is: 12345 I do a s/12345/123456/. This would fail because the length of the string to replace was 5 and that of the replacement was 6. But the resulting line only would have a length of 6. But s/12345/123456/n would work (so not using regex). Does this approach really make sense and is comprehensible for users? Or do I simply fail to correctly understand the code? Maybe space_left above is meant to be space_left -= repl_len + (apr_size_t) (regm[0].rm_so - pos); Regards Rüdiger
RE: prefetch proxy
-Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Donnerstag, 3. November 2011 12:53 To: dev@httpd.apache.org Subject: Re: prefetch proxy On Nov 2, 2011, at 7:40 AM, Plüm, Rüdiger, VF-Group wrote: I think a timeout should be handled like it is now as failing on a slow client is IMHO a desired action by the admin. If he wants to give the client more time he should configure a higher timeout. For other errors like 'Resource temporarily unavailable' we should log and possibly retry. But the question is: Why does this error happen in the first place? What is the root cause of it and what can be done to remove it? Is this error something that can just happen and we should gracefully live with by retrying? And if we are retrying how often do we do this to avoid an endless loop? I'm fine with with having a set number of retries with EAGAIN and treating a timeout as an error. If we exhaust the retries, we simply break out of the prefetch loop and continue on, and let Continue without prefetch in the EAGAIN case and hope for better times when we need the body really later? a continued Resource temporarily unavailable be handled as it normally would. So as an error? Sound OK? In general yes. See the questions for details above :-) Regards Rüdiger
RE: prefetch proxy
-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Mittwoch, 2. November 2011 12:22
To: dev@httpd.apache.org
Subject: Re: prefetch proxy
On Nov 2, 2011, at 5:44 AM, Rüdiger Plüm wrote:
Am 01.11.2011 21:23, schrieb Jim Jagielski:
In mod_proxy_http we have:
/* Prefetch MAX_MEM_SPOOL bytes
*
* This helps us avoid any election of C-L v.s. T-E
* request bodies, since we are willing to keep in
* memory this much data, in any case. This gives
* us an instant C-L election if the body is of some
* reasonable size.
*/
temp_brigade = apr_brigade_create(p, bucket_alloc);
do {
status = ap_get_brigade(r-input_filters, temp_brigade,
AP_MODE_READBYTES, APR_BLOCK_READ,
MAX_MEM_SPOOL - bytes_read);
if (status != APR_SUCCESS) {
ap_log_error(APLOG_MARK, APLOG_ERR, status, r-server,
proxy: prefetch request body
failed to %pI (%s)
from %s (%s),
p_conn-addr, p_conn-hostname ?
p_conn-hostname: ,
c-remote_ip, c-remote_host ?
c-remote_host: );
return HTTP_BAD_REQUEST;
}
apr_brigade_length(temp_brigade, 1,bytes);
bytes_read += bytes;
However, I see times when status could be APR_EAGAIN. IMO,
it doesn't
make sense to error out here in that case. Right?
Why should it be EAGAIN with a blocking read?
Or a Timeout... With Fed14, I'm seeing occasional 'Resource
temporarily unavailable'
errors, which I think we should be able to handle.
I think a timeout should be handled like it is now as failing on
a slow client is IMHO a desired action by the admin. If he wants to give
the client more time he should configure a higher timeout.
For other errors like 'Resource temporarily unavailable' we should log and
possibly
retry. But the question is: Why does this error happen in the first place?
What is the root cause of it and what can be done to remove it?
Is this error something that can just happen and we should gracefully live
with by retrying? And if we are retrying how often do we do this to avoid an
endless loop?
Regards
Rüdiger
RE: CVE-2011-3368 not fully fixed?
Can you please check if the following patch fixes this issue?
Index: protocol.c
===
--- protocol.c (revision 1181036)
+++ protocol.c (working copy)
@@ -672,6 +672,7 @@
r-hostname = NULL;
r-status = HTTP_BAD_REQUEST;
r-uri = apr_pstrdup(r-pool, uri);
+return 0;
}
if (ll[0]) {
@@ -960,13 +961,13 @@
if (!read_request_line(r, tmp_bb)) {
if (r-status == HTTP_REQUEST_URI_TOO_LARGE
|| r-status == HTTP_BAD_REQUEST) {
-if (r-status == HTTP_BAD_REQUEST) {
+if (r-status == HTTP_REQUEST_URI_TOO_LARGE) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
- request failed: invalid characters in URI);
+ request failed: URI too long (longer than %d),
r-server-limit_req_line);
}
-else {
+else if (r-method == NULL) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
- request failed: URI too long (longer than %d),
r-server-limit_req_line);
+ request failed: invalid characters in URI);
}
ap_send_error_response(r, 0);
ap_update_child_status(conn-sbh, SERVER_BUSY_LOG, r);
Regards
Rüdiger
-Original Message-
From: Marcus Meissner [mailto:meiss...@suse.de]
Sent: Dienstag, 25. Oktober 2011 14:29
To: dev@httpd.apache.org
Subject: CVE-2011-3368 not fully fixed?
Hi,
I probably have overlooked something, but while QAing our
Apache (2.2.12 based)
updates it seems CVE-2011-3368 is not fully fixed by the
patch referenced.
With the RewriteRule within the VirtualHost *:80 section,
RewriteEngine on
RewriteRule (.*)\.(ico|jpg|gif|png) http://leo.suse.de$1.$2 [P]
$ telnet teshost 80
GET @www.suse.de/foo.png
...gives me the 404 page of www.suse.de, which is not intended
I get in the error log:
[Tue Oct 25 14:10:50 2011] [error] [client 10.10.0.233]
invalid request-URI @www.suse.de/foo.png
and in access.log
10.10.0.233 - - [25/Oct/2011:14:10:50 +0200] GET
@www.suse.de/foo.png 404 16006 - -
which seems to me like it is half working.
The error.log has the invalid request-URI message from the
patched part
of the code, but the 404 is from www.suse.de/foo.png.
= I think the 0.9 protocol method is not falling out of the
uri handling correctly.
It seems on reading ap_read_request() the 0.9 assbackwards
case handling
does not error out on r-status set but proceeds and sets
r-status to HTTP_OK and
goes on.
Any ideas? Am I doing stuff wrong?
Ciao, Marcus
RE: CVE-2011-3368 not fully fixed?
Thanks for testing. Applied to trunk as r1188745.
Regards
Rüdiger
-Original Message-
From: Marcus Meissner [mailto:meiss...@suse.de]
Sent: Dienstag, 25. Oktober 2011 17:48
To: dev@httpd.apache.org
Subject: Re: CVE-2011-3368 not fully fixed?
Hi Rüdiger,
I ported your patch to our 2.2.12 codebase (attached) and my testcase
now correctly reports a 400 from the testserver when
doing
GET @www.suse.de/foo.png
as request.
Ciao, Marcus
On Tue, Oct 25, 2011 at 02:49:08PM +0200, Plüm, Rüdiger,
VF-Group wrote:
Can you please check if the following patch fixes this issue?
Index: protocol.c
===
--- protocol.c (revision 1181036)
+++ protocol.c (working copy)
@@ -672,6 +672,7 @@
r-hostname = NULL;
r-status = HTTP_BAD_REQUEST;
r-uri = apr_pstrdup(r-pool, uri);
+return 0;
}
if (ll[0]) {
@@ -960,13 +961,13 @@
if (!read_request_line(r, tmp_bb)) {
if (r-status == HTTP_REQUEST_URI_TOO_LARGE
|| r-status == HTTP_BAD_REQUEST) {
-if (r-status == HTTP_BAD_REQUEST) {
+if (r-status == HTTP_REQUEST_URI_TOO_LARGE) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
- request failed: invalid
characters in URI);
+ request failed: URI too
long (longer than %d), r-server-limit_req_line);
}
-else {
+else if (r-method == NULL) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
- request failed: URI too
long (longer than %d), r-server-limit_req_line);
+ request failed: invalid
characters in URI);
}
ap_send_error_response(r, 0);
ap_update_child_status(conn-sbh, SERVER_BUSY_LOG, r);
Regards
Rüdiger
-Original Message-
From: Marcus Meissner [mailto:meiss...@suse.de]
Sent: Dienstag, 25. Oktober 2011 14:29
To: dev@httpd.apache.org
Subject: CVE-2011-3368 not fully fixed?
Hi,
I probably have overlooked something, but while QAing our
Apache (2.2.12 based)
updates it seems CVE-2011-3368 is not fully fixed by the
patch referenced.
With the RewriteRule within the VirtualHost *:80 section,
RewriteEngine on
RewriteRule (.*)\.(ico|jpg|gif|png) http://leo.suse.de$1.$2 [P]
$ telnet teshost 80
GET @www.suse.de/foo.png
...gives me the 404 page of www.suse.de, which is not intended
I get in the error log:
[Tue Oct 25 14:10:50 2011] [error] [client 10.10.0.233]
invalid request-URI @www.suse.de/foo.png
and in access.log
10.10.0.233 - - [25/Oct/2011:14:10:50 +0200] GET
@www.suse.de/foo.png 404 16006 - -
which seems to me like it is half working.
The error.log has the invalid request-URI message from the
patched part
of the code, but the 404 is from www.suse.de/foo.png.
= I think the 0.9 protocol method is not falling out of the
uri handling correctly.
It seems on reading ap_read_request() the 0.9 assbackwards
case handling
does not error out on r-status set but proceeds and sets
r-status to HTTP_OK and
goes on.
Any ideas? Am I doing stuff wrong?
Ciao, Marcus
--
Working, but not speaking, for the following german company:
SUSE LINUX Products GmbH, HRB 16746 (AG Nuernberg)
Geschaeftsfuehrer: Jeff Hawn, Jennifer Guild, Felix Imendoerffer
RE: CVE-2011-3368 not fully fixed?
I did some further analysis. While the patch for trunk is still fine
as it shortens the path for bailing out the behaviour was already correct
with trunk and 2.2.21. So the HTTP/0.9 behaviour you see does NOT happen with
2.2.x = 2.2.18 (plus patch) and trunk. You are affected by an old logic that
was changed in r1100200 and hence changed since 2.2.18.
Regards
Rüdiger
-Original Message-
From: Plüm, Rüdiger, VF-Group [mailto:ruediger.pl...@vodafone.com]
Sent: Dienstag, 25. Oktober 2011 17:58
To: dev@httpd.apache.org
Subject: RE: CVE-2011-3368 not fully fixed?
Thanks for testing. Applied to trunk as r1188745.
Regards
Rüdiger
-Original Message-
From: Marcus Meissner [mailto:meiss...@suse.de]
Sent: Dienstag, 25. Oktober 2011 17:48
To: dev@httpd.apache.org
Subject: Re: CVE-2011-3368 not fully fixed?
Hi Rüdiger,
I ported your patch to our 2.2.12 codebase (attached) and
my testcase
now correctly reports a 400 from the testserver when
doing
GET @www.suse.de/foo.png
as request.
Ciao, Marcus
On Tue, Oct 25, 2011 at 02:49:08PM +0200, Plüm, Rüdiger,
VF-Group wrote:
Can you please check if the following patch fixes this issue?
Index: protocol.c
===
--- protocol.c (revision 1181036)
+++ protocol.c (working copy)
@@ -672,6 +672,7 @@
r-hostname = NULL;
r-status = HTTP_BAD_REQUEST;
r-uri = apr_pstrdup(r-pool, uri);
+return 0;
}
if (ll[0]) {
@@ -960,13 +961,13 @@
if (!read_request_line(r, tmp_bb)) {
if (r-status == HTTP_REQUEST_URI_TOO_LARGE
|| r-status == HTTP_BAD_REQUEST) {
-if (r-status == HTTP_BAD_REQUEST) {
+if (r-status == HTTP_REQUEST_URI_TOO_LARGE) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
- request failed: invalid
characters in URI);
+ request failed: URI too
long (longer than %d), r-server-limit_req_line);
}
-else {
+else if (r-method == NULL) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
- request failed: URI too
long (longer than %d), r-server-limit_req_line);
+ request failed: invalid
characters in URI);
}
ap_send_error_response(r, 0);
ap_update_child_status(conn-sbh,
SERVER_BUSY_LOG, r);
Regards
Rüdiger
-Original Message-
From: Marcus Meissner [mailto:meiss...@suse.de]
Sent: Dienstag, 25. Oktober 2011 14:29
To: dev@httpd.apache.org
Subject: CVE-2011-3368 not fully fixed?
Hi,
I probably have overlooked something, but while QAing our
Apache (2.2.12 based)
updates it seems CVE-2011-3368 is not fully fixed by the
patch referenced.
With the RewriteRule within the VirtualHost *:80 section,
RewriteEngine on
RewriteRule (.*)\.(ico|jpg|gif|png)
http://leo.suse.de$1.$2 [P]
$ telnet teshost 80
GET @www.suse.de/foo.png
...gives me the 404 page of www.suse.de, which is not
intended
I get in the error log:
[Tue Oct 25 14:10:50 2011] [error] [client 10.10.0.233]
invalid request-URI @www.suse.de/foo.png
and in access.log
10.10.0.233 - - [25/Oct/2011:14:10:50 +0200] GET
@www.suse.de/foo.png 404 16006 - -
which seems to me like it is half working.
The error.log has the invalid request-URI message from the
patched part
of the code, but the 404 is from www.suse.de/foo.png.
= I think the 0.9 protocol method is not falling out of the
uri handling correctly.
It seems on reading ap_read_request() the 0.9 assbackwards
case handling
does not error out on r-status set but proceeds and sets
r-status to HTTP_OK and
goes on.
Any ideas? Am I doing stuff wrong?
Ciao, Marcus
--
Working, but not speaking, for the following german company:
SUSE LINUX Products GmbH, HRB 16746 (AG Nuernberg)
Geschaeftsfuehrer: Jeff Hawn, Jennifer Guild, Felix Imendoerffer
RE: CVE-2011-3368 not fully fixed?
-Original Message- From: William A. Rowe Jr. [mailto:wr...@rowe-clan.net] Sent: Dienstag, 25. Oktober 2011 18:44 To: dev@httpd.apache.org Subject: Re: CVE-2011-3368 not fully fixed? On 10/25/2011 11:21 AM, Plüm, Rüdiger, VF-Group wrote: I did some further analysis. While the patch for trunk is still fine as it shortens the path for bailing out the behaviour was already correct with trunk and 2.2.21. So the HTTP/0.9 behaviour you see does NOT happen with 2.2.x = 2.2.18 (plus patch) and trunk. You are affected by an old logic that was changed in r1100200 and hence changed since 2.2.18. Should the contents of www.a.o/dist/httpd/patches/apply_to_2.2... be updated? No. We only supply this patch for 2.2.21 and everything later then 2.2.18 does not show the behaviour mentioned here. Or to put it the other way around 2.2.21 plus the patch we supply behaves well with HTTP/0.9 requests. Regards Rüdiger
RE: CVE-2011-3368 not fully fixed?
-Original Message- From: Plüm, Rüdiger, VF-Group [mailto:ruediger.pl...@vodafone.com] Sent: Dienstag, 25. Oktober 2011 18:48 To: dev@httpd.apache.org Subject: RE: CVE-2011-3368 not fully fixed? -Original Message- From: William A. Rowe Jr. [mailto:wr...@rowe-clan.net] Sent: Dienstag, 25. Oktober 2011 18:44 To: dev@httpd.apache.org Subject: Re: CVE-2011-3368 not fully fixed? On 10/25/2011 11:21 AM, Plüm, Rüdiger, VF-Group wrote: I did some further analysis. While the patch for trunk is still fine as it shortens the path for bailing out the behaviour was already correct with trunk and 2.2.21. So the HTTP/0.9 behaviour you see does NOT happen with 2.2.x = 2.2.18 (plus patch) and trunk. You are affected by an old logic that was changed in r1100200 and hence changed since 2.2.18. Should the contents of www.a.o/dist/httpd/patches/apply_to_2.2... be updated? No. We only supply this patch for 2.2.21 and everything later then 2.2.18 does not show the behaviour mentioned here. Or to put it the other way around 2.2.21 plus the patch we supply behaves well with HTTP/0.9 requests. Or further put: My patch on trunk is an optimization, but not a change in behaviour. I do not even consider it worth backporting to 2.2.x. Regards Rüdiger
RE: mod_proxy_html
-Original Message- From: Nick Kew [mailto:n...@webthing.com] Sent: Freitag, 14. Oktober 2011 10:45 To: dev@httpd.apache.org Subject: Re: mod_proxy_html On 14 Oct 2011, at 07:55, jean-frederic clere wrote: I think I prefer it in 2.4 than in a separate module. OK, there seem to be a few folks saying that. If we include it in 2.4 then we're making libxml2 a dependency. I don't think that this is a real issue. As said we need to check this dependency and prevent building it if the dependency is not there. I guess things like lua are even more special as libxml2 and mod_lua is included in httpd as well. That's of concern primarily to packagers, but it's a decision we should at least take consciously. That's why my initial proposal made it a separate module! Adding libxml2 could also open the way to bundling some other great modules, such as XSLT transformations. But only if we make it a module independent dependency. (note: it is now possible to use libxml2 as the the backend for APR's XML module, though expat remains the default there). IMHO that would create an APR dependency and not primarily a httpd dependency. So this should be discussed over at dev@apr and is IMHO unrelated to this topic. Regards Rüdiger
RE: ap_log_error wants a string literal in mod_lua
-Original Message- From: Igor Galić [mailto:i.ga...@brainsware.org] Sent: Mittwoch, 5. Oktober 2011 16:51 To: dev Subject: ap_log_error wants a string literal in mod_lua Hey folks, When trunk with (hardened) gcc 4.6 on Ubuntu beta, I get: lua_config.c: In function 'cmd_log_at': lua_config.c:177:5: error: format not a string literal and no format arguments [-Werror=format-security] cc1: some warnings being treated as errors The appropriate source: lua_Debug dbg; lua_getstack(L, 1, dbg); lua_getinfo(L, Sl, dbg); msg = luaL_checkstring(L, 2); ap_log_error(dbg.source, dbg.currentline, APLOG_MODULE_INDEX, level, 0, cmd-server, msg); Is there anything we can do to fix this so it builds again with paranoia options? How about: ap_log_error(dbg.source, dbg.currentline, APLOG_MODULE_INDEX, level, 0, cmd-server, %s, msg); Regards Rüdiger
RE: Child process coring
Basicly this behaviour is a design decision. If we run out of memory, then:
SEGFAULT, when such a NULL pointer is used.
Regards
Rüdiger
From: narayana.na...@thomsonreuters.com
[mailto:narayana.na...@thomsonreuters.com]
Sent: Freitag, 30. September 2011 12:25
To: dev@httpd.apache.org
Subject: Child process coring
Hello,
We have been experiencing child process core occasionally when system
memory runs low on SCO UNIX system. Debugging the core with gdb indicates that
the core is generated in the following function:
APR_DECLARE(void) apr_pool_tag(apr_pool_t *pool, const char *tag)
{
pool-tag = tag;
}
When the memory is low, passed pool parameter becomes null and causing
this issue. I am thinking of adding some checks before the assignment, however
does it guarantee it will not core further down in the execution path? There
are multiple places where apr_pool_create is being called which is returning
NULL pool structure. In proxy_util.c apr_pool_tag is called at many places
without checking the return value of apr_pool_create function.
This issue is seen in apache 2.2.11. Is this fixed in any higher
version?
I appreciate your quick response.
Thanks,
N.Nayak
RE: svn commit: r1176019 - in /httpd/httpd/trunk: CHANGES modules/filters/mod_substitute.c
Anyone time for remote eyes if my findings are correct or wrong?
Regards
Rüdiger
-Original Message-
From: Ruediger Pluem
Sent: Mittwoch, 28. September 2011 08:29
To: dev@httpd.apache.org
Subject: Re: svn commit: r1176019 - in /httpd/httpd/trunk:
CHANGES modules/filters/mod_substitute.c
On 09/26/2011 10:09 PM, s...@apache.org wrote:
Author: sf
Date: Mon Sep 26 20:09:06 2011
New Revision: 1176019
URL: http://svn.apache.org/viewvc?rev=1176019view=rev
Log:
Make mod_substitute more efficient:
- Use varbuf resizable buffer instead of constantly allocating pool
memory and copying data around. This changes the memory
requirement from
quadratic in ((number of substitutions in line) * (length
of line)) to
linear in (length of line).
- Instead of copying buckets just to append a \0, use new
ap_regexec_len()
function
PR: 50559
Modified:
httpd/httpd/trunk/CHANGES
httpd/httpd/trunk/modules/filters/mod_substitute.c
Modified: httpd/httpd/trunk/modules/filters/mod_substitute.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/filters
/mod_substitute.c?rev=1176019r1=1176018r2=1176019view=diff
==
--- httpd/httpd/trunk/modules/filters/mod_substitute.c (original)
+++ httpd/httpd/trunk/modules/filters/mod_substitute.c Mon
Sep 26 20:09:06 2011
/*
@@ -189,82 +174,70 @@ static void do_pattmatch(ap_filter_t *f,
bytes -= len;
buff += len;
}
-if (script-flatten s1 !force_quick) {
-/*
- * we've finished looking at the
bucket, so remove the
- * old one and add in our new one
- */
-s2 = apr_pstrmemdup(tmp_pool, buff, bytes);
-s1 = apr_pstrcat(tmp_pool, s1, s2, NULL);
-tmp_b =
apr_bucket_transient_create(s1, strlen(s1),
-
f-r-connection-bucket_alloc);
+if (have_match script-flatten
!force_quick) {
+char *copy = ap_varbuf_pdup(pool,
vb, NULL, 0,
+buff,
bytes, len);
+tmp_b =
apr_bucket_pool_create(copy, len, pool,
+
f-r-connection-bucket_alloc);
APR_BUCKET_INSERT_BEFORE(b, tmp_b);
apr_bucket_delete(b);
b = tmp_b;
}
-
}
else if (script-regexp) {
-/*
- * we need a null terminated string
here :(. To hopefully
- * save time and memory, we don't
alloc for each run
- * through, but only if we need to
have a larger chunk
- * to save the string to. So we keep
track of how much
- * we've allocated and only re-alloc
when we need it.
- * NOTE: this screams for a macro.
- */
-if (!scratch || (bytes (fbytes + 1))) {
Not that it matters on trunk now any longer, but on 2.2.x:
Don't we have an off by two here?
If scratch was already allocated it is fbytes large.
If bytes is e.g. equal to fbytes or bytes == fbytes +1
the above won't be true, but we will
copy fbytes + 1 (2) data (trailing \0) to the buffer.
Shouldn't the check be
bytes + 1 fbytes
instead?
-fbytes = bytes + 1;
-scratch = apr_palloc(tpool, fbytes);
-}
-/* reset pointer to the scratch space */
-p = scratch;
-memcpy(p, buff, bytes);
-p[bytes] = '\0';
-while (!ap_regexec(script-regexp, p,
+int left = bytes;
+const char *pos = buff;
+while (!ap_regexec_len(script-regexp,
pos, left,
AP_MAX_REG_MATCH,
regm, 0)) {
-/* first, grab the replacement string */
-repl = ap_pregsub(tmp_pool,
script-replacement, p,
- AP_MAX_REG_MATCH, regm);
+have_match = 1;
if (script-flatten !force_quick) {
-SEDSCAT(s1, s2, tmp_pool, p,
regm[0].rm_so, repl);
+/* copy bytes before the match */
+if (regm[0].rm_so 0)
+ap_varbuf_strmemcat(vb,
pos,
RE: svn commit: r1176019 - in /httpd/httpd/trunk: CHANGES modules/filters/mod_substitute.c
Thanks. Will review and vote.
Regards
Rüdiger
-Original Message-
From: Rainer Jung
Sent: Donnerstag, 29. September 2011 14:36
To: dev@httpd.apache.org
Subject: Re: svn commit: r1176019 - in /httpd/httpd/trunk:
CHANGES modules/filters/mod_substitute.c
On 29.09.2011 13:09, Plüm, Rüdiger, VF-Group wrote:
Anyone time for remote eyes if my findings are correct or wrong?
I did only locally check the scratch and fbytes stuff, but I agree, it
must be
Index: modules/filters/mod_substitute.c
===
--- modules/filters/mod_substitute.c(revision 1177244)
+++ modules/filters/mod_substitute.c(working copy)
@@ -213,7 +213,7 @@
* we've allocated and only re-alloc
when we need it.
* NOTE: this screams for a macro.
*/
-if (!scratch || (bytes (fbytes + 1))) {
+if (!scratch || (bytes + 1 fbytes)) {
fbytes = bytes + 1;
scratch = apr_palloc(tpool, fbytes);
}
Will propose for 2.2.x.
Regards,
Rainer
RE: Improving SSL config
-Original Message- From: Rainer Jung Sent: Donnerstag, 29. September 2011 16:32 To: dev@httpd.apache.org Subject: Improving SSL config In light of the TLS 1.0 CBC attack (aka BEAST, CVE-2011-3389) I suggest we update our SSL configuration analogous to what's in trunk. - Choose a better default SSLCipherSuite - Add SSLHonorCipherOrder - restrict MSIE exceptions to MSIE 2-5 The patch looks like this: svn diff docs/conf/extra/httpd-ssl.conf.in Index: docs/conf/extra/httpd-ssl.conf.in === --- docs/conf/extra/httpd-ssl.conf.in (revision 1177244) +++ docs/conf/extra/httpd-ssl.conf.in (working copy) @@ -87,8 +87,14 @@ # SSL Cipher Suite: # List the ciphers that the client is permitted to negotiate. # See the mod_ssl documentation for a complete list. -SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL +SSLCipherSuite RC4-SHA:AES128-SHA:ALL:!aNULL:!EXP:!LOW:!MD5:!SSLV2:!NULL +# SSL Cipher Honor Order: +# On a busy HTTPS server you may want to enable this directive +# to force clients to use one of the faster ciphers like RC4-SHA +# or AES128-SHA in the order defined by SSLCipherSuite. +#SSLHonorCipherOrder on· + # Server Certificate: # Point SSLCertificateFile at a PEM encoded certificate. If # the certificate is encrypted, then you will be prompted for a @@ -218,7 +224,7 @@ # Similarly, one has to force some clients to use HTTP/1.0 to workaround # their broken HTTP/1.1 implementation. Use variables downgrade-1.0 and # force-response-1.0 for this. -BrowserMatch .*MSIE.* \ +BrowserMatch MSIE [2-5] \ nokeepalive ssl-unclean-shutdown \ downgrade-1.0 force-response-1.0 Furthermore I wonder whether we should activate the SSLHonorCipherOrder in this config by default - at least for trunk. At the moment it is commented out. For 2.2.x it is possible people use OpenSSL older than 0.9.6 and the directive will not work then. We might even backport the change to SSLCipherSuite and the MSIE exceptions to 2.0. Any comments on: - Updating 2.2? +1 - Activating SSLHonorCipher in trunk? -0: IMHO this is something the local admin should decide. - Updating 2.0? -0.5: As users of 2.0 are unlikely to do new deployments they will use their existing configs and not use our examples. Furthermore doing stuff like this on 2.0 could set false expectations to users especially as we discussed on putting it EOL at a reasonable point in the future. Regards Rüdiger
RE: svn commit: r1176254 - /httpd/httpd/branches/2.2.x/STATUS
-Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Dienstag, 27. September 2011 13:21 To: dev@httpd.apache.org Cc: c...@httpd.apache.org Subject: Re: svn commit: r1176254 - /httpd/httpd/branches/2.2.x/STATUS Fixed... Thx! Hm. http://people.apache.org/~jim/patches/byterange0-.txt still looks the same. Or is this some kind of caching issue on my side? Regards Rüdiger
RE: svn commit: r1176254 - /httpd/httpd/branches/2.2.x/STATUS
Ok. Looks like the cache issue fixed itself now. Regards Rüdiger -Original Message- From: Plüm, Rüdiger, VF-Group [mailto:ruediger.pl...@vodafone.com] Sent: Dienstag, 27. September 2011 14:04 To: dev@httpd.apache.org Cc: c...@httpd.apache.org Subject: RE: svn commit: r1176254 - /httpd/httpd/branches/2.2.x/STATUS -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Dienstag, 27. September 2011 13:21 To: dev@httpd.apache.org Cc: c...@httpd.apache.org Subject: Re: svn commit: r1176254 - /httpd/httpd/branches/2.2.x/STATUS Fixed... Thx! Hm. http://people.apache.org/~jim/patches/byterange0-.txt still looks the same. Or is this some kind of caching issue on my side? Regards Rüdiger
RE: svn commit: r1176351 - /httpd/httpd/branches/2.2.x/STATUS
-Original Message- From: Jim Jagielski Sent: Dienstag, 27. September 2011 16:48 To: dev@httpd.apache.org Cc: c...@httpd.apache.org Subject: Re: svn commit: r1176351 - /httpd/httpd/branches/2.2.x/STATUS done and done and done :) Really :-)? I just downloaded the proposed backport and it still does not change the error message. Regards Rüdiger
RE: httpd 2.0.65 - when?
-Original Message- From: Rainer Jung [mailto:rainer.j...@kippdata.de] Sent: Montag, 26. September 2011 17:47 To: dev@httpd.apache.org Subject: Re: httpd 2.0.65 - when? On 26.09.2011 17:35, Jim Jagielski wrote: All looks good... testing passes w/ no regressions so I'll likely tag and roll tomorrow AM. Is there consensus how to handle the range 0- returns 200 problem? It looks like the discussion for 2.2 is still open, but I haven't checked whether that influences the 2.0 patch. I think it does. So good point. So IMHO it makes sense to discuss this first and patch afterwards. Regards Rüdiger
RE: httpd 2.0.65 - when?
-Original Message- From: William A. Rowe Jr. Sent: Montag, 26. September 2011 18:13 To: dev@httpd.apache.org Subject: Re: httpd 2.0.65 - when? On 9/26/2011 10:46 AM, Rainer Jung wrote: On 26.09.2011 17:35, Jim Jagielski wrote: All looks good... testing passes w/ no regressions so I'll likely tag and roll tomorrow AM. Is there consensus how to handle the range 0- returns 200 problem? It looks like the discussion for 2.2 is still open, but I haven't checked whether that influences the 2.0 patch. Agreed, if people decide our handling of range 0- is not desirable, this would seem to be a showstopper on all three branches. Personally, I find the current behavior acceptable by the spec and per the underlying errata Roy has suggested. Clients should not be able to shift trivial processing (which the client is perfectly capable of performing) to the server in ways that increase network traffic or server load. HTTP/1.1 conversations must be designed to efficiently utilize network bandwidth, and these particular clients did not do that. I'm on the fence whether we should restore such abuse. I agree with you, but I am leaning towards to revert this behaviour, because there are too much stupid clients out there. So it looks like the smarter party has to give in :-). Sigh. Regards Rüdiger
RE: httpd 2.0.65 - when?
-Original Message- From: Stefan Fritsch [mailto:s...@sfritsch.de] Sent: Montag, 26. September 2011 18:30 To: dev@httpd.apache.org Subject: Re: httpd 2.0.65 - when? On Monday 26 September 2011, Plüm, Rüdiger, VF-Group wrote: Agreed, if people decide our handling of range 0- is not desirable, this would seem to be a showstopper on all three branches. Personally, I find the current behavior acceptable by the spec and per the underlying errata Roy has suggested. Clients should not be able to shift trivial processing (which the client is perfectly capable of performing) to the server in ways that increase network traffic or server load. HTTP/1.1 conversations must be designed to efficiently utilize network bandwidth, and these particular clients did not do that. I'm on the fence whether we should restore such abuse. I agree with you, but I am leaning towards to revert this behaviour, because there are too much stupid clients out there. So it looks like the smarter party has to give in :-). Sigh. +1 Also, as documented in the Mozilla Bugzilla, there have also been dumb servers which send Accept-Ranges: bytes but don't support ranges. So it's somewhat understandable that clients try different methods to determine range support. They should have used a partial range request instead of requesting the whole file, though :-( Like 0-0 or -1 Regards Rüdiger
RE: svn commit: r1172010 - /httpd/httpd/trunk/modules/ssl/ssl_engine_init.c
-Original Message- From: Kaspar Brand Sent: Freitag, 23. September 2011 17:07 To: dev@httpd.apache.org Subject: Re: svn commit: r1172010 - /httpd/httpd/trunk/modules/ssl/ssl_engine_init.c Maybe I'm somewhat confused by what Apache Group is actually referring to here - I read that to be the PMC... but I'll gladly stand corrected. Can someone clarify? It is correct that only PMC members have binding votes. But this is only critical for releases. For backports any committer can vote and I have never seen a vote on a backport to be discarded just because the voter was not on the PMC. Even on releases you can and should vote (even non comitters can do this), but finally the vote needs 3 +1's from PMC members to pass. Regards Rüdiger
RE: With IP address in Host: header ServerName/ServerAlias doesn't work
Please update the bug report: 1. Declare the unneeded patch obsolete. 2. Update the needed patch to be against trunk. Regards Rüdiger -Original Message- From: Micha Lenk Sent: Freitag, 9. September 2011 11:30 To: dev@httpd.apache.org Subject: Re: With IP address in Host: header ServerName/ServerAlias doesn't work Hi Rüdiger, On 08/23/2011 12:25 PM CEST +02:00, Plüm, Rüdiger, VF-Group wrote: IMHO the patch does not solve the issue mentioned in the comment and is not needed. Keep in mind the difference between ap_matches_request_vhost and check_host_alias: ap_matches_request_vhost checks only the data of *one* server_rec and hence only *one* Servername and ServerAlias settings as set for this vhost, whereas check_host_alias scans over *multiple* server_recs with *multiple* Servername and ServerAlias settings from different vhosts. Okay, good point. Then the first patch should indeed be sufficient. What is needed to get it applied to trunk? Regards, Micha
RE: svn commit: r1166551 - /httpd/httpd/trunk/modules/proxy/mod_proxy_ajp.c
-Original Message-
From: Joe Orton
Sent: Donnerstag, 8. September 2011 14:16
To: dev@httpd.apache.org
Subject: Re: svn commit: r1166551 -
/httpd/httpd/trunk/modules/proxy/mod_proxy_ajp.c
On Thu, Sep 08, 2011 at 07:45:40AM -, Jean-Frederic Clere wrote:
--- httpd/httpd/trunk/modules/proxy/mod_proxy_ajp.c (original)
+++ httpd/httpd/trunk/modules/proxy/mod_proxy_ajp.c Thu Sep
8 07:45:40 2011
@@ -214,7 +214,7 @@ static int ap_proxy_ajp_request(apr_pool
proxy: AJP: request failed to %pI (%s),
conn-worker-cp-addr,
conn-worker-s-hostname);
-if (status == AJP_EOVERFLOW)
+if (status == AJP_EOVERFLOW || status == AJP_EBAD_METHOD)
return HTTP_BAD_REQUEST;
else {
/*
An unrecognized method from the client does not imply a syntactically
invalid request, so it does not look like 400 is an appropriate
response.
501 would be normal here - if I'm reading the proxy logic correctly,
only 500 and 503 have special semantics, so it should be fine to do
this?
Index: modules/proxy/mod_proxy_ajp.c
===
--- modules/proxy/mod_proxy_ajp.c (revision 1166642)
+++ modules/proxy/mod_proxy_ajp.c (working copy)
@@ -214,8 +214,10 @@
proxy: AJP: request failed to %pI (%s),
conn-worker-cp-addr,
conn-worker-s-hostname);
-if (status == AJP_EOVERFLOW || status == AJP_EBAD_METHOD)
+if (status == AJP_EOVERFLOW)
return HTTP_BAD_REQUEST;
+else if (status == AJP_EBAD_METHOD)
+return HTTP_NOT_IMPLEMENTED;
else {
/*
* This is only non fatal when the method is
idempotent. In this
+1
Regards
Rüdiger
RE: MaxRanges
-Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Dienstag, 6. September 2011 13:34 To: dev@httpd.apache.org Subject: MaxRanges From the code, MaxRanges 0 means unlimited... Is that what we want? I can envision some use-cases where an admin may want to disable ranges totally and MaxRanges is really the place to do that. How about a setting 0 means unlimited? (yes, this involves some code and logic changes)... In general yes. But this would also require that we do not sent Accept-Ranges any longer if MaxRanges is set to 0. Regards Rüdiger
RE: MaxRanges
-Original Message- From: Eric Covener [mailto:cove...@gmail.com] Sent: Dienstag, 6. September 2011 14:09 To: dev@httpd.apache.org Subject: Re: MaxRanges On Tue, Sep 6, 2011 at 7:34 AM, Jim Jagielski j...@jagunet.com wrote: From the code, MaxRanges 0 means unlimited... Is that what we want? I can envision some use-cases where an admin may want to disable ranges totally and MaxRanges is really the place to do that. How about a setting 0 means unlimited? (yes, this involves some code and logic changes)... No magic reasons for current state here, I'd suggest teaching it to take strings as well as numbers so users don't need to know 0/-1 significance e.g. MaxRanges none | 0 (none) | unlimited | n=1 +1 Regards Rüdiger
RE: next steps for range fix in 2.2.x
-Original Message- From: Jeff Trawick [mailto:traw...@gmail.com] Sent: Sonntag, 4. September 2011 17:30 To: Apache HTTP Server Development List Subject: next steps for range fix in 2.2.x Can anyone fill in any details for the following? 1. Any known regressions not yet fixed in trunk? 2. Any other issues not yet fixed in trunk? a. r1163833 needs to be modified/reverted so that 200 is sent instead of 206 (see discussion for that commit) 3. Need patch for 2.2.x to review/test; needs r1163833 (or modification) and following from http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http/by terange_filter.c?sortby=dateview=log 4. Wait to see if more regressions pop up in the next couple of days I guess we should wait until mid week for further regression. After that we should create a backport starting from r1163197 until current of byterange_filter.c Regards Rüdiger
RE: CVE-2003-1418 - still affects apache 2 current
-Original Message- From: Joe Orton [mailto:jor...@redhat.com] Sent: Montag, 5. September 2011 15:21 To: dev@httpd.apache.org Cc: tho...@redhat.com Subject: Re: CVE-2003-1418 - still affects apache 2 current On Thu, Sep 01, 2011 at 06:27:35PM +0200, Plüm, Rüdiger, VF-Group wrote: Can't find the discussion either, but I remember that it was not seen as a security issue. For those still concerned about this, the advice was as you said FileETag -INode. So IMHO no need for a patch here except for documentation and default config Ah - I found the discussion, it was on security@. Tomas (CC'ed) pointed out that CVE-2003-1418 also covers the fact that the byterange filter leaks pids. I don't think that is worth treating as a vulnerability, either; but I changed it in r1165268 anyway - that is still leaking some MPM-specific data, but it doesn't seem worth going to any more effort. = http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2003-1418 Is there consensus to treat the issues described there as not being security-sensitive? If so we can probably put tihs on the vulnerability list is as a not-a-bug as an official statement. +1 Regards Rüdiger
RE: svn commit: r1163833 - /httpd/httpd/trunk/modules/http/byterange_filter.c
-Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Freitag, 2. September 2011 15:43 To: dev@httpd.apache.org Subject: Re: svn commit: r1163833 - /httpd/httpd/trunk/modules/http/byterange_filter.c On Sep 1, 2011, at 2:44 PM, Roy T. Fielding wrote: On Sep 1, 2011, at 1:11 AM, Tim Bannister wrote: On Wed, Aug 31, 2011 at 6:28 PM, Roy T. Fielding wrote: On Aug 31, 2011, at 6:10 PM, William A. Rowe Jr. wrote: The presumption here is that the client requests bytes=0- to begin the transmission, and provided it sees a 206, restarting somewhere in the stream results in aborting the connection and streaming bytes=n- from the restart point. Further testing should determine if this was the broken assumption. Do we send the Accept-Ranges header field? http://tools.ietf.org/html/rfc2616#page-105 Apache httpd 2.2.9 is sending this header in the Debian bug report at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=639825 Then the client is broken and they also break intermediary caches (most of which don't cache 206 responses) by performing this stunt. We could add a version-specific browsermatch to do the 206, but I'd prefer to just tell the client developers to fix their code. +1 from me. I tend to +1 this as well. Regards Rüdiger
RE: non-splittable buckets (was: Regression with range fix)
-Original Message- From: Stefan Fritsch [mailto:s...@sfritsch.de] Sent: Mittwoch, 31. August 2011 23:09 To: dev@httpd.apache.org Subject: non-splittable buckets (was: Regression with range fix) On Wednesday 31 August 2011, Jim Jagielski wrote: Looking at the patch in 2.2.x; there is a lot of effort expended deadling with apr_bucket_split() returning ENOTIMPL - that looks unnecessary; the filter will only handle brigades containing buckets with known length, and all such buckets must be _split-able. So you think we can rip out the whole if (rv == APR_ENOTIMPL) blocks? Belt and suspenders? If we rip it out, we should replace it with ap_assert()s. And maybe only do it in 2.3? I guess asserts are fine. IMHO we should do it in 2.3 and 2.2 if we do it. Nothing is different between 2.2 and 2.3 in this respect and we keep the diff between 2.2 and 2.3 small. From what I can see Joe is correct with his assumption that this case cannot happen there. A different idea I had: If apr_bucket_split() returns ENOTIMPL, we could call apr_brigade_partition on the copied brigade. apr_brigade_partition() would then do all the complex handling of apr_bucket_read(), etc. It is only slightly less efficient because it has to traverse the brigade again. But the memory usage would still stay low because the original brigade would remain untouched. And we would get rid of much of the complexity in copy_brigade_range(). Would be an approach, but I would like to see Joe's feedback on this as I don't see value in reconstructing this part if ripping out is fine as well. Regards Rüdiger
RE: Appropriate patches for 2.2.19 and 2.0.64?
-Original Message- From: William A. Rowe Jr. [mailto:wr...@rowe-clan.net] Sent: Donnerstag, 1. September 2011 03:51 To: dev@httpd.apache.org Subject: Re: Appropriate patches for 2.2.19 and 2.0.64? On 8/31/2011 4:16 PM, William A. Rowe Jr. wrote: I've attempted to simply substitute the 2.2.19 filter code into the 2.0.64 http_protocol.c sources, and am unsure how far off these patches are from what they need to be; there's been a significant amount of drift and refactoring in the interim. Still looking for feedback, but the attached applies and corresponds to 2.2.20 with the exception of atoi rather than strtoi semantics, and without the no DefaultType exception.. Thanks for taking care. It applies cleanly, but most byterange tests from the framework fail: Failed Test Stat Wstat Total Fail Failed List of Failed --- t/apache/byterange.t 136 97 71.32% 2-9 11 16-17 19-20 25-26 28 33-35 37-39 42 44-47 49-61 63-68 70-73 77-79 83-88 90-96 98-99 101-103 105-113 115-123 126-127 130 132-136 t/apache/byterange5.t54 80.00% 1 3-5 t/apache/byterange7.t 135 38.46% 2-5 12 This might be because I use a 64 bit build of 2.0.x (no other test environment at hand at the moment) and 2.0.x is not really good at 64 bit (lots of compiler warnings). Regards Rüdiger
Another regression regarding byteranges
PR 51748 (https://issues.apache.org/bugzilla/show_bug.cgi?id=51748) is an IMHO valid regression in range behaviour (from the report): Request and response sample in each versions. = version 2.2.20 GET / HTTP/1.1 Host: localhost Range: bytes=-1 HTTP/1.1 206 Partial Content Server: Apache/2.2.20 (Unix) Accept-Ranges: bytes Content-Length: 2 Content-Range: bytes 0-1/10240 Content-Type: text/html = version 2.2.19 GET / HTTP/1.1 Host: localhost Range: bytes=-1 HTTP/1.1 206 Partial Content Server: Apache/2.2.19 (Unix) Accept-Ranges: bytes Content-Length: 1 Content-Range: bytes 10239-10239/10240 Content-Type: text/html = I already fixed that in trunk. I think this regression justifies another release for 2.2.x. But IMHO we should wait at least until mid next week to see if other regressions come thru and hit them all with a 2.2.21. Regards Rüdiger
RE: non-splittable buckets (was: Regression with range fix)
-Original Message- From: Joe Orton [mailto:jor...@redhat.com] Sent: Donnerstag, 1. September 2011 14:39 To: dev@httpd.apache.org Subject: Re: non-splittable buckets (was: Regression with range fix) On Wed, Aug 31, 2011 at 11:08:51PM +0200, Stefan Fritsch wrote: On Wednesday 31 August 2011, Jim Jagielski wrote: Looking at the patch in 2.2.x; there is a lot of effort expended deadling with apr_bucket_split() returning ENOTIMPL - that looks unnecessary; the filter will only handle brigades containing buckets with known length, and all such buckets must be _split-able. So you think we can rip out the whole if (rv == APR_ENOTIMPL) blocks? Yes. This code could only get exercised with a custom bucket type which has fixed-length buckets but no -split implementation. Belt and suspenders? If we rip it out, we should replace it with ap_assert()s. And maybe only do it in 2.3? It would seem odd to have ENOTIMPL as a fatal error but other real errors non-fatal. *No* error should occur here with unless somebody has cooked up a really whacky bucket type. So you would follow the rip-out-and-replace-with-asserts approach? Regards Rüdiger
RE: CVE-2003-1418 - still affects apache 2 current
-Original Message- From: Joe Orton [mailto:jor...@redhat.com] Sent: Donnerstag, 1. September 2011 16:46 To: Marcus Meissner Cc: dev@httpd.apache.org Subject: Re: CVE-2003-1418 - still affects apache 2 current On Thu, Sep 01, 2011 at 02:39:11PM +0200, Marcus Meissner wrote: Hi, CVE-2003-1418, a minor security issue, is still affecting the current codebase. someone opened a tracker bug a year ago without feedback: https://issues.apache.org/bugzilla/show_bug.cgi?id=49623 Do you have a statement? Use FileETag -INode if you care about leaking inode numbers. I think there was consensus that the default should be changed to that, but I can't find the discussion. Can't find the discussion either, but I remember that it was not seen as a security issue. For those still concerned about this, the advice was as you said FileETag -INode. So IMHO no need for a patch here except for documentation and default config Regards Rüdiger
RE: svn commit: r1163918 - /httpd/httpd/trunk/modules/http/byterange_filter.c
-Original Message-
From: William A. Rowe Jr. [mailto:wr...@rowe-clan.net]
Sent: Donnerstag, 1. September 2011 18:38
To: dev@httpd.apache.org
Subject: Re: svn commit: r1163918 -
/httpd/httpd/trunk/modules/http/byterange_filter.c
On 9/1/2011 1:30 AM, rpl...@apache.org wrote:
Author: rpluem
Date: Thu Sep 1 06:30:02 2011
New Revision: 1163918
URL: http://svn.apache.org/viewvc?rev=1163918view=rev
Log:
* Fix error message
--- httpd/httpd/trunk/modules/http/byterange_filter.c (original)
+++ httpd/httpd/trunk/modules/http/byterange_filter.c Thu
Sep 1 06:30:02 2011
@@ -640,7 +640,9 @@ static int ap_set_byterange(request_rec
}
else if (num_ranges == 0 unsatisfiable) {
/* If all ranges are unsatisfiable, we should return 416 */
- ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r, 0U);
+ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
+ All ranges in Range header (%s) are
unsatisfiable,
+ it);
I know this isn't your change, but just noticed it. APLOG_ERR?
At its highest shouldn't that be APLOG_INFO?
APLOG_ERR and APLOG_WARN should be reserved for server-side errors
that we have some control over, not bogus input :)
See r1163920 (http://svn.apache.org/viewvc?rev=1163920view=rev) :-)
Regards
Rüdiger
RE: Regression with range fix
-Original Message- From: Joe Orton [mailto:jor...@redhat.com] Sent: Mittwoch, 31. August 2011 11:13 To: dev@httpd.apache.org Subject: Re: Regression with range fix On Tue, Aug 30, 2011 at 08:51:55PM +0200, Stefan Fritsch wrote: The first regression report, though slightly too late for the vote: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=639825 The byterange_filter.c in the Debian update is exactly the one from 2.2.20. I will keep you updated. Hi; I'm just back from holiday and catching up. The behaviour changes in the patch which could feasibly break non-compliant clients are: a) using 200 in some cases where a 206 response would end up being larger b) using a chunked response where previously C-L was always used, in cases where =32 ranges are being returned Anything else to watch out for? Looking at the patch in 2.2.x; there is a lot of effort expended deadling with apr_bucket_split() returning ENOTIMPL - that looks unnecessary; the filter will only handle brigades containing buckets with known length, and all such buckets must be _split-able. So you think we can rip out the whole if (rv == APR_ENOTIMPL) blocks? Regards Rüdiger
RE: [VOTE] httpd-2.2.20 tarballs
-Original Message- From: Steffen [mailto:i...@apachelounge.com] Sent: Tuesday, August 30, 2011 2:57 PM To: dev@httpd.apache.org Subject: Re: [VOTE] httpd-2.2.20 tarballs All looks fine on Windows +1 Download for testing available www.apachelounge.com btw. warning C4244: 'function' : conversion from 'apr_uint64_t' to 'apr_size_t', possible loss of data byterange_filter.c 213 I have fixed this in trunk as r1163197. Anyone opposed to do the same in 2.2.x? Regards Rüdiger - Original Message - From: Jim Jagielski j...@jagunet.com Newsgroups: gmane.comp.apache.devel To: dev@httpd.apache.org Sent: Tuesday, August 30, 2011 2:17 AM Subject: [VOTE] httpd-2.2.20 tarballs Are available on httpd.apache.org/dev http://httpd.apache.org/dev/dist/ Vote on release as 2.2.20-GA
RE: svn commit: r1162579 - /httpd/httpd/trunk/modules/http/byterange_filter.c
-Original Message-
From: Stefan Fritsch [mailto:s...@sfritsch.de]
Sent: Sonntag, 28. August 2011 23:36
To: dev@httpd.apache.org
Subject: Re: svn commit: r1162579 -
/httpd/httpd/trunk/modules/http/byterange_filter.c
On Sun, 28 Aug 2011, s...@apache.org wrote:
Author: sf
Date: Sun Aug 28 19:45:21 2011
New Revision: 1162579
URL: http://svn.apache.org/viewvc?rev=1162579view=rev
Log:
Every 32 ranges, pass the prepared ranges down the filter chain.
Modified:
httpd/httpd/trunk/modules/http/byterange_filter.c
Modified: httpd/httpd/trunk/modules/http/byterange_filter.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http/by
terange_filter.c?rev=1162579r1=1162578r2=1162579view=diff
==
--- httpd/httpd/trunk/modules/http/byterange_filter.c (original)
+++ httpd/httpd/trunk/modules/http/byterange_filter.c Sun
Aug 28 19:45:21 2011
@@ -378,6 +378,12 @@ AP_CORE_DECLARE_NONSTD(apr_status_t) ap_
}
APR_BRIGADE_CONCAT(bsend, tmpbb);
+if (i i % 32 == 0) {
+/* Every now and then, pass what we have down
the filter chain */
+if ((rv = ap_pass_brigade(f-next, bsend)) !=
APR_SUCCESS)
+return rv;
+apr_brigade_cleanup(bsend);
+}
}
This is broken. It causes the Content-Length header to
contain the size of
the original file instead of the response body. Is the
correct fix to add
apr_table_unset(r-headers_out, Content-Length) ?
This looks correct. Maybe to be extra save we should also set r-clength to
0 as well.
Regards
Rüdiger
RE: svn commit: r1162579 - /httpd/httpd/trunk/modules/http/byterange_filter.c
-Original Message- From: Stefan Fritsch [mailto:s...@sfritsch.de] Sent: Montag, 29. August 2011 09:45 To: dev@httpd.apache.org Subject: Re: svn commit: r1162579 - /httpd/httpd/trunk/modules/http/byterange_filter.c On Sunday 28 August 2011, Stefan Fritsch wrote: This is broken. It causes the Content-Length header to contain the size of the original file instead of the response body. Is the correct fix to add apr_table_unset(r-headers_out, Content-Length) ? Committed that to trunk and updated http://people.apache.org/~sf/byterange-no-merge.2.2.diff to include it and a change to reset the status to 200 if the range header is invalid. The latter issue was fixed in trunk by Eric's MaxRanges change. Patch looks good, but a few comments: 1. r1162669 is missing (provided this was really a good idea from me :-)). 2. I adjusted trunk code to drop the copying of the original range header to or as well as this does not seem to be needed (r1162687). Regards Rüdiger
RE: 2.2 approach for byterange; please review
-Original Message- From: Stefan Fritsch [mailto:s...@sfritsch.de] Sent: Montag, 29. August 2011 17:03 To: dev@httpd.apache.org Subject: Re: 2.2 approach for byterange; please review On Monday 29 August 2011, Jim Jagielski wrote: I propose we add the cheap brigade copy + sum_len clen failsafe to 2.2 and push it out today (I can RM) and we save the other enhancements and cool-to-do's for trunk post 2.2 release... OK, I have just proposed a patch for backport. Please review. Do we want 2.2.20 to be a security fix only release? Otherwise, there are a few proposals with 2 or 3 votes waiting to be voted/committed. IMHO we should limit it to the security fix. Some time has already passed since the security issue is known and adding only the security fix speeds up the release process: 1. Fewer possible regressions. 2. No waiting for votes on the other patches. Regards Rüdiger
RE: 2.2 approach for byterange?
From: Greg Ames [mailto:ames.g...@gmail.com]
Sent: Montag, 29. August 2011 17:32
To: dev@httpd.apache.org
Subject: Re: 2.2 approach for byterange?
On Sun, Aug 28, 2011 at 4:22 PM, Stefan Fritsch s...@sfritsch.de
wrote:
looks good overall.
+while (start64 - off_first
(apr_uint64_t)copy-length) {
+apr_bucket *tmp;
+int i = 0;
+if (i++ = 9)
+return APR_EINVAL;
I assume you meant to initialize i before the while() loop.
Greg
I guess yes. The question is if we should keep that in the backport at
all, as we only do it in the first location
and not in the second location and 9 looks like a rather high
number without any comment and documention.
IMHO even arbitrary numbers deserve that.
Regards
Rüdiger
RE: 2.2 approach for byterange?
-Original Message-
From: Stefan Fritsch [mailto:s...@sfritsch.de]
Sent: Montag, 29. August 2011 17:43
To: dev@httpd.apache.org
Subject: RE: 2.2 approach for byterange?
On Mon, 29 Aug 2011, Plüm, Rüdiger, VF-Group wrote:
Sent: Montag, 29. August 2011 17:32
To: dev@httpd.apache.org
Subject: Re: 2.2 approach for byterange?
On Sun, Aug 28, 2011 at 4:22 PM, Stefan Fritsch
s...@sfritsch.de wrote:
looks good overall.
+while (start64 - off_first
(apr_uint64_t)copy-length) {
+apr_bucket *tmp;
+int i = 0;
+if (i++ = 9)
+return APR_EINVAL;
I assume you meant to initialize i before the while() loop.
Greg
I guess yes. The question is if we should keep that in
the backport at all, as we only do it in the first location
and not in the second location and 9 looks like a
rather high number without any comment and documention.
IMHO even arbitrary numbers deserve that.
Looks like an accidental commit or merge error in r1162131. I
think we
should remove that block both from trunk and from the backport.
+1
Regards
Rüdiger
RE: svn commit: r1162881 - /httpd/httpd/branches/2.2.x/modules/http/byterange_filter.c
-Original Message-
From: Stefan Fritsch [mailto:s...@sfritsch.de]
Sent: Montag, 29. August 2011 18:00
To: dev@httpd.apache.org
Subject: Re: svn commit: r1162881 -
/httpd/httpd/branches/2.2.x/modules/http/byterange_filter.c
On Mon, 29 Aug 2011, j...@apache.org wrote:
Author: jim
Date: Mon Aug 29 15:53:52 2011
New Revision: 1162881
URL: http://svn.apache.org/viewvc?rev=1162881view=rev
Log:
Allow for actual counting...
Modified:
httpd/httpd/branches/2.2.x/modules/http/byterange_filter.c
Modified: httpd/httpd/branches/2.2.x/modules/http/byterange_filter.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/module
s/http/byterange_filter.c?rev=1162881r1=1162880r2=1162881view=diff
==
---
httpd/httpd/branches/2.2.x/modules/http/byterange_filter.c (original)
+++
httpd/httpd/branches/2.2.x/modules/http/byterange_filter.c
Mon Aug 29 15:53:52 2011
@@ -137,6 +137,7 @@ static apr_status_t copy_brigade_range(a
if (off_first != start64) {
rv = apr_bucket_split(copy,
(apr_size_t)(start64 - off_first));
if (rv == APR_ENOTIMPL) {
+int i;
rv = apr_bucket_read(copy, s, len,
APR_BLOCK_READ);
if (rv != APR_SUCCESS) {
apr_brigade_cleanup(bbout);
@@ -147,9 +148,10 @@ static apr_status_t copy_brigade_range(a
* of shorter length. So read and
delete until we reached
* the correct bucket for splitting.
*/
+i = 0;
while (start64 - off_first
(apr_uint64_t)copy-length) {
apr_bucket *tmp;
-int i = 0;
+/* don't allow inf. spin */
if (i++ = 9)
return APR_EINVAL;
IMNSHO such changes need to be voted upon before commiting to
branches/2.2.x. When can this case happen? And why do it for
the start
bucket but not for the end bucket?
Agreed. Please let us bring this in shape in trunk and backport a voted
solution later on
to 2.2.x.
Regards
Rüdiger
RE: svn commit: r1162881 - /httpd/httpd/branches/2.2.x/modules/http/byterange_filter.c
I am fine with this on 2.2.x. +1. Regards Rüdiger -Original Message- From: Jim Jagielski [mailto:j...@apache.org] Sent: Montag, 29. August 2011 18:22 To: dev@httpd.apache.org Subject: Re: svn commit: r1162881 - /httpd/httpd/branches/2.2.x/modules/http/byterange_filter.c Do we need to vote on this: http://svn.apache.org/viewvc?rev=1162885view=rev
RE: Advisory improvement
Below comments make sense to me.
We should pick this up.
Regards
Rüdiger
-Original Message-
From: Dirk-Willem van Gulik
Sent: Freitag, 26. August 2011 13:35
To: dev@httpd.apache.org
Subject: Advisory improvement
From the Full Disclosure list. Does anyone have time to
confirm this improvement.
On 26 Aug 2011, at 12:09, Carlos Alberto Lopez Perez wrote:
RewriteEngine on
RewriteCond %{HTTP:range} !(^bytes=[^,]+(,[^,]+){0,4}$|^$) [NC,OR]
RewriteCond %{HTTP:request-range}
!(^bytes=[^,]+(,[^,]+){0,4}$|^$) [NC]
RewriteRule .* - [F]
Because if you don't specify the [OR] apache will combine the rules
making an AND (and you don't want this!).
Also use NC=(nocase) to prevent the attacker upper casing bytes=
(don't know if it will work.. but just to prevent)
Pretty Please !
Thanks,
Dw.
RE: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c
-Original Message- From: Jim Jagielski Sent: Freitag, 26. August 2011 13:38 To: dev@httpd.apache.org Subject: Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c I still think that your version is wrong wrong wrong and am tempted to veto it. It completely invalidates what ap_set_byterange() is designed to do (set r-range) as well as removes the ability to count overlaps, non-ascends, etc... The question is: Does r-range need to reflect the range definition that is actually delivered by the byte range filter? The comment in httpd.h only says: /** The Range: header */ So you could put it vice versa and say r-range is not allowed to contain an adjusted / optimized version of the Range header, but must contain the original Range header. To be honest I would not follow that, because the original Range header is still in r-headers_out and it makes sense to store an optimized version of the Range header in r-range. Nevertheless it is my opinion that it doesn't make sense to parse the Range header twice. If we need a proper r-range and the other statistics, we should have a function that parses the Range header and outputs 1. The number of ranges in the original header 2. The number of ranges after the merges. 3. Whatever else statistics we need. 4. The merged ranges as an array (like the one Stefans patch creates) 5. A proper r-range created from 4. Although ap_set_byterange is in the ap_ namespace it is a static function in byterange_filter.c. So we are free to adjust it as we like without the need for any bumps and possibly creating code that cannot be backported. Regards Rüdiger
RE: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c
-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Freitag, 26. August 2011 13:49
To: dev@httpd.apache.org
Subject: Re: svn commit: r1161661 -
/httpd/httpd/trunk/modules/http/byterange_filter.c
On Aug 26, 2011, at 2:50 AM, Ruediger Pluem wrote:
I followed through the above algorithm with a range of
0-1,1000-1001 and the
result was a range of 0-1001 which feels wrong. Culprit
seems to be the ((end-1) = oend) check.
Shouldn't that be oend = (start-1)?
Yeah, that is wrong. Investigating. That part was to catch
things like 10-20,21-100
+} else {
+char *nr = apr_psprintf(r-pool, %
APR_OFF_T_FMT -% APR_OFF_T_FMT,
+ostart, oend);
+merged = apr_pstrcat(r-pool, merged,
(num_ranges++ ? , : ), nr, NULL);
Doing this in a loop feels bad as it creates new buffers
over and over.
Shouldn't we create a buffer of the size of strlen(range)
and add to this subsequently
(we know that the length of the merged range = length of range)?
My plan is to put each range into an array and at the
end flatten it via apr_array_pstrcat()
I thought about that as well, but I think a combination of a preallocated
buffer and apr_snprintf using a moving pointer in this buffer could
save even more memory in the typical use case.
Of course this changes if you remove a lot of ranges by merging.
Regards
Rüdiger
RE: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c
-Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Freitag, 26. August 2011 14:38 To: dev@httpd.apache.org Subject: Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c On Aug 26, 2011, at 8:31 AM, Jim Jagielski wrote: Agreed... The issue is that to merge, you need to parse... We could have ap_set_byterange() also return a set of start/stop points (in an array), so that the filter can work with that and not bother with rereading r-range. In fact, I think that makes the most sense... That is, adjust ap_set_byterange() to create the modified, parsed r-range, a return status and the start/stop array. The filter then goes thru that array. So ap_set_byterange(request_rec *r, apr_off_t clen, apr_array_header_t *indexes) What is clen? Regards Rüdiger
RE: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c
-Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Freitag, 26. August 2011 14:56 To: dev@httpd.apache.org Subject: Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c On Aug 26, 2011, at 8:46 AM, Plüm, Rüdiger, VF-Group wrote: -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Freitag, 26. August 2011 14:38 To: dev@httpd.apache.org Subject: Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c On Aug 26, 2011, at 8:31 AM, Jim Jagielski wrote: Agreed... The issue is that to merge, you need to parse... We could have ap_set_byterange() also return a set of start/stop points (in an array), so that the filter can work with that and not bother with rereading r-range. In fact, I think that makes the most sense... That is, adjust ap_set_byterange() to create the modified, parsed r-range, a return status and the start/stop array. The filter then goes thru that array. So ap_set_byterange(request_rec *r, apr_off_t clen, apr_array_header_t *indexes) What is clen? The content length... we need to know for ranges like 100- and -99 Thanks. Where in these parameters / return values do we find the statistics (if needed), e.g. number of ranges, number of overlaps, number of merges, etc. Regards Rüdiger
RE: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c
-Original Message- From: Stefan Fritsch [mailto:s...@sfritsch.de] Sent: Freitag, 26. August 2011 15:41 To: dev@httpd.apache.org Subject: Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c On Fri, August 26, 2011 14:38, Jim Jagielski wrote: On Aug 26, 2011, at 8:31 AM, Jim Jagielski wrote: Agreed... The issue is that to merge, you need to parse... We could have ap_set_byterange() also return a set of start/stop points (in an array), so that the filter can work with that and not bother with rereading r-range. In fact, I think that makes the most sense... That is, adjust ap_set_byterange() to create the modified, parsed r-range, a return status and the start/stop array. The filter then goes thru that array. So ap_set_byterange(request_rec *r, apr_off_t clen, apr_array_header_t *indexes) will exist and parse_byterange() will go away... Sound OK? Sounds OK to me. But I would prefer that the array is allocated with the right size right from the start instead of growing an apr_array which would again involve lots of copying. I guess we can do both: Count the ',' and give the number to apr_array_make Regards Rüdiger
RE: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c
-Original Message- From: Jim Jagielski [mailto:j...@apache.org] Sent: Freitag, 26. August 2011 16:27 To: dev@httpd.apache.org Subject: Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c I guess we can do both: Count the ',' and give the number to apr_array_make Doesn't that mean that someone can craft a nasty Range (e.g: 0-0,1-1,2-2, 3-3,- and cause us to preallocate a bunch of memory when at the end we'll get 0- ??? In principal yes. Two things can happen: 1. The ranges are valid and do not overlap or are not mergable. In this case we need to allocate that memory anyway. 2. The ranges are mergable. In this case we allocate too much memory for the array. But this effect is limited by the maximum length a header field can have. And if this is not enough do a sane cut for the preallocation: MIN(number of ranges, MAX_PREALLOCATED_ARRAY_MEMBERS) This should work fine for the typical use case where we can't merge anything and avoid running in a DoS trap if we have a large number of mergable ranges. Regards Rüdiger
RE: PoC ready
IMHO commit and let it be fixed in trunk. -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Freitag, 26. August 2011 16:34 To: dev@httpd.apache.org Subject: PoC ready Should I commit or post?
RE: PoC ready
-Original Message- From: Plüm, Rüdiger, VF-Group [mailto:ruediger.pl...@vodafone.com] Sent: Freitag, 26. August 2011 16:38 To: dev@httpd.apache.org Subject: RE: PoC ready IMHO commit and let it be fixed in trunk. I mean improved :-). Not to imply your code has errors, but there is always room for improvement :-) Regards Rüdiger -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Freitag, 26. August 2011 16:34 To: dev@httpd.apache.org Subject: PoC ready Should I commit or post?
RE: svn commit: r1161791 - /httpd/httpd/trunk/modules/http/byterange_filter.c
I guess this should catch cases where you have overlapping or adjacent ranges next to each other but in the wrong order, so e.g. 2000-3000,1000-2000 or 2000-3000,1500-2500 But I think this check is currently wrong as you point out. Currently not sure how to fix it. Regards Rüdiger From: Greg Ames [mailto:ames.g...@gmail.com] Sent: Freitag, 26. August 2011 16:58 To: dev@httpd.apache.org Subject: Re: svn commit: r1161791 - /httpd/httpd/trunk/modules/http/byterange_filter.c On Thu, Aug 25, 2011 at 7:02 PM, s...@apache.org wrote: +if (!(ranges[i].start ranges[i-1].end + 1 + ranges[i].endranges[i-1].start - 1)) and the if condition looks similar. the first test is fine, but why is ranges[i].end ranges[i-1].start a good thing? since we have the ! in front, we are testing for the normal ascending case. Greg
RE: PoC ready
I think the
+if (in_merge) {
+overlaps++;
+continue;
+} else {
+new = (char **)apr_array_push(merged);
+*new = apr_psprintf(r-pool, % APR_OFF_T_FMT -% APR_OFF_T_FMT,
+ostart, oend);
+idx = (indexes_t *)apr_array_push(indexes);
+idx-start = ostart;
+idx-end = oend;
num_ranges++;
-range++;
+}
should be really
+if (in_merge) {
+overlaps++;
+continue;
+} else {
+new = (char **)apr_array_push(merged);
+*new = apr_psprintf(r-pool, % APR_OFF_T_FMT -% APR_OFF_T_FMT,
+ostart, oend);
+idx = (indexes_t *)apr_array_push(indexes);
+idx-start = ostart;
+idx-end = oend;
+ostart = start;
+oend = end;
+in_merge = 1;
num_ranges++;
-range++;
+}
Otherwise I think 0-1,1000-1001
will result in
0-1
Regards
Rüdiger
-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Freitag, 26. August 2011 17:19
To: dev@httpd.apache.org
Subject: Re: PoC ready
Committed... r 1162131
RE: PoC ready
@@ -252,6 +205,9 @@
off_last += start64 - off_first;
copy = out_first;
}
+else {
+APR_BRIGADE_INSERT_TAIL(bbout, copy);
+}
if (end64 - off_last != (apr_uint64_t)e-length) {
rv = apr_bucket_split(copy, (apr_size_t)(end64 + 1 -
off_last));
if (rv == APR_ENOTIMPL) {
This one seems to be a merge error in your working copy and was fixed by Stefan
in r1161791
Regards
Rüdiger
-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Freitag, 26. August 2011 17:19
To: dev@httpd.apache.org
Subject: Re: PoC ready
Committed... r 1162131
RE: DoS with mod_deflate range requests
-Original Message- From: Stefan Fritsch Sent: Donnerstag, 25. August 2011 08:21 To: dev@httpd.apache.org Subject: Re: DoS with mod_deflate range requests On Thursday 25 August 2011, Jim Jagielski wrote: OK then... we seem to be coalescing into some consensus here... basically, if the client sends stuff which is brain-dead stupid, we simply 2000 and send the whole kit-and-kaboodle. I'd like to propose that we update the byterange filter to perform the following: o coalesce all adjacent ranges, whether overlapping or not. (eg: 200-250,251-300 200-250,220-300 both merge to 200-300) This may still confuse a broken client. Maybe we could omit that from the 2.2 patch for now and only commit to 2.3. Sounds like a plan. Or make it configurable with a default of off in 2.2.x and on in 2.3.x o We count: the number of times a gap between ranges is 80bytes the number of times we hit a descendent range (eg: 200-1000,2000-3000,1200-1500,4000-5000 would count as 1) the number of ranges total (post ascending merge) If any = some config-time limit, we send a 200 This is a start and was chosen simply for ease of implementation... We can then expand it to be more functional... Comments? Looks good. Plus we should implement the patch from Stefan below and then we should be good. Please also look at the patch at http://mail-archives.apache.org/mod_mbox/httpd- dev/201108.mbox/%3c201108250138.49474...@sfritsch.de%3E which greatly reduces the memory needed for the range requests. BTW, I won't have time to beat that into shape today. If anyone else has, please go ahead. Regards Rüdiger
RE: Fixing Ranges
-Original Message- From: Stefan Fritsch Sent: Donnerstag, 25. August 2011 01:39 To: dev@httpd.apache.org Subject: Re: Fixing Ranges On Thursday 25 August 2011, Greg Ames wrote: On Wed, Aug 24, 2011 at 5:16 PM, Stefan Fritsch s...@sfritsch.de wrote: I have another idea: Instead of using apr_brigade_partition write a new function ap_brigade_copy_part that leaves the original brigade untouched. It would copy the necessary buckets to a new brigade and then split the first and last of those copied buckets as necessary and destroy the excess buckets. AFAICS, this would reduce the quadratic growth into linear. Do you think that would solve our problems? How does apr_brigade_partition contribute to quadratic growth? Does the original brigade end up with a lot of one byte buckets? Yes, it splits the buckets in the original brigade, creating up to two new buckets for every range. These split one-byte buckets are then copied again for each of the subsequent ranges. The attached PoC patch does not change the original brigade and seems to fix the DoS for me. It needs some more work and some review for integer overflows, though. (apr_brigade_partition does some interesting things there). I guess the following should be adjusted with this patch: 1. Do the same integer casting thing apr_brigade_partition does. So convert everything to apr_uint64_t and work with this. 2. Doing a read on a bucket can change its length. So we need to check afterwards if we still can split the bucket and if not might do a read on the next bucket until we are fine. 3. We should do the read bucket thing for the last bucket as well. I think it would be best if you could commit this PoC patch as is (or with adjustments according to the comments above) to trunk and we polish it up in trunk until it is fine. Regards Rüdiger
RE: Next update on CVE-2011-3192
+1 Regards Rüdiger -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Donnerstag, 25. August 2011 14:13 To: dev@httpd.apache.org Subject: Re: Next update on CVE-2011-3192 I have a feeling that we could push this out today... I'm going to fold Stefan's path into trunk, and we should use trunk (CTR) to polish up the patch as well as add whatever other features we need. From there, backporting to 2.2/2.0 will be trivial. On Aug 25, 2011, at 4:18 AM, Dirk-Willem van Gulik wrote: I am keeping a draft at http://people.apache.org/~dirkx/CVE-2011-3192.txt Changes since last are: - version ranges more specific - vendor information added - backgrounder on relation to 2007 issues (see below to ensure I got this right). I suggest we sent this out late Z time today (i.e. end of working day US) _if_ 1) it is likely that we do not have a firm timeline for the full fix and 2) we have a bit more to add. Otherwise we skip to a final update with the fixing instructions for 2.0 and 2.2 Feedback welcome, Thanks, Dw.
RE: Fixing Ranges
+1 for 2.3, -0 for 2.2. I guess for 2.2 we should only detect misuse (the definition of misuse needs to configurable) and reply with a 200 if misuse is detected. misuse would be - too much ranges [Let the number be configurable with a sane default e.g. 100 and a possible setting of 0 for unlimited] - overlapping ranges (after sorting) [Let this detector be configurable with a default to on] For 2.3 the last one could be 3 state: off - Don't do anything about that on - reply with 200 if misuse is detected. optimize - Do sorts and merges and fill too small chunks between the ranges. Default for 2.3 would be optimize. Regards Rüdiger -Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Donnerstag, 25. August 2011 16:25 To: dev@httpd.apache.org Subject: Re: Fixing Ranges Now that the memory utilz is being fixed, we need to determine what sort of usage we want to allow... I'm guessing that people are ok with http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311 as the guiding spec?
RE: DoS with mod_deflate range requests
-Original Message- From: Stefan Fritsch Sent: Mittwoch, 24. August 2011 00:28 To: dev@httpd.apache.org Subject: Re: DoS with mod_deflate range requests On Tuesday 23 August 2011, William A. Rowe Jr. wrote: On 8/23/2011 4:00 PM, Greg Ames wrote: On Tue, Aug 23, 2011 at 3:32 PM, William A. Rowe Jr. wrote: I suggest we should be parsing and reassembling the list before we start the bucket logic. I propose we satisfy range requests in the only sensible manner, returning the ranges in sequence, yeah, overlapping ranges should be merged up front. That ought to completely fix the issue. So the only remaining question; are we free to reorder them into sequence? Good point. I haven't seen anything in the RFC about that. I guess that there are at least some clients that will be broken by that. Nevertheless, I have done a first try at a patch. The necessary modification to only merge and not reorder should be small. I have done only limited testing, so there are probably some bugs. There are no tests with multiple ranges in the test-framework, yet. Even in the most pedantic case, I believe that the total array shouldn't ever exceed 1024, because in those cases a large number of the acceptable expected ranges should be in the nnn-nnn, format, or 8 characters long, out of our MAX_LINE_LENGTH of some 8190. If we argue that asking for single bytes is simply wrong, we should probably allocate some 16 ranges and grow the list by a power of four, resulting in a max of some 4 allocs and maximum memory consumption of less than 64k per request. Just counting the commas in the header line seems acceptable to me. In any case, single byte ranges are explicitly mentioned in the RFC as example, so we probably should not disallow those. Patch looks good, but some comments: As far as I can see the following range request would not get merged: Range: bytes=0-0,1-1,2-2 into a 0-2 range as need_sort would remain 0. OTOH Range: bytes=0-0,0-1,1-2 would get get merged into a 0-2 range. Using boundary and !boundary in the later if statements to decide whether a request is multi range or single range is IMHO bad as boundary is set based on the old number ranges and not based on the number of merged ranges. So multiple ranges in the beginning might get merged to a single range in the end. Regards Rüdiger
RE: CT oops?
-Original Message- From: NormW [mailto:no...@gknw.net] Sent: Mittwoch, 24. August 2011 10:12 To: dev@httpd.apache.org Subject: CT oops? G/E, httpd-trunk\modules\ssl\ssl_engine_config.c (164): mctx-pkp-cert_file = NULL; mctx-pkp-cert_path = NULL; mctx-pkp-ca_cert_file = NULL mctx-pkp-certs = NULL; mctx-pkp-ca_certs = NULL; Seems like there is a need of punctuation... Fixed in r1161002. This one is trickier: httpd-trunk\modules\ssl\mod_ssl.c (175): ### mwccnlm Compiler: #File: mod_ssl.c # -- # 175: ProxyMachineCertificateChainFile, ssl_cmd_SSLProxyMachineCertificateChainFile, ((void *) 0), 128, TAKE1, SSL Proxy: file # Error: ^ # undefined identifier 'ssl_cmd_SSLProxyMachineCertificateChainFile' an extern perhaps? Fixed in r1161005 Regards Rüdiger
RE: Mitigation Range header (Was: DoS with mod_deflate range requests)
-Original Message-
From: Dirk-Willem van Gulik
Sent: Mittwoch, 24. August 2011 13:33
To: dev@httpd.apache.org
Subject: Mitigation Range header (Was: DoS with mod_deflate
range requests)
Folks,
This issue is now active in the wild. So some unified/simple
comms is needed.
What is the wisdom on mitigation advise/briefing until a
proper fix it out - in order of ease:
-Where possible - disable mod_deflate
= we sure this covers all cases - or this is a good stopgap ?
As said this has *nothing* to do with mod_deflate. This was IMHO just
a guess by the original author of the tool.
-Where possible - set LimitRequestFieldSize to a small value
- Suggesting of 128 fine ?
-Where this is not possible (e.g. long cookies, auth
headers of serious size) consider using
mod_rewrite to not accept more than a few commas
= anyone a config snipped for this ?
How about the following (untested) rewrite rule. It should only allow 5
ranges at max.
RewriteCond %{HTTP:range} ^bytes=[^,]+(,[^,]+){0,4}$
RewriteRule .* - [F]
Regards
Rüdiger
RE: Mitigation Range header (Was: DoS with mod_deflate range requests)
-Original Message-
From: Eric Covener [mailto:cove...@gmail.com]
Sent: Mittwoch, 24. August 2011 14:05
To: dev@httpd.apache.org
Subject: Re: Mitigation Range header (Was: DoS with
mod_deflate range requests)
On Wed, Aug 24, 2011 at 7:57 AM, Plüm, Rüdiger, VF-Group
ruediger.pl...@vodafone.com wrote:
-Original Message-
From: Dirk-Willem van Gulik
Sent: Mittwoch, 24. August 2011 13:33
To: dev@httpd.apache.org
Subject: Mitigation Range header (Was: DoS with mod_deflate
range requests)
Folks,
This issue is now active in the wild. So some unified/simple
comms is needed.
What is the wisdom on mitigation advise/briefing until a
proper fix it out - in order of ease:
- Where possible - disable mod_deflate
= we sure this covers all cases - or this is a good
stopgap ?
As said this has *nothing* to do with mod_deflate. This was
IMHO just
a guess by the original author of the tool.
- Where possible - set LimitRequestFieldSize to a small value
- Suggesting of 128 fine ?
- Where this is not possible (e.g. long cookies, auth
headers of serious size) consider using
mod_rewrite to not accept more than a few commas
= anyone a config snipped for this ?
How about the following (untested) rewrite rule. It should
only allow 5
ranges at max.
RewriteCond %{HTTP:range} ^bytes=[^,]+(,[^,]+){0,4}$
RewriteRule .* - [F]
Is [E=no-gzip] enough to avoid the downward spiral, for the sake of
false positives?
As said it has IMHO nothing to do with mod_deflate. It is an issue of the byte
range filter.
But your regex matches when there's just a couple of ranges -- maybe
{4} and no $?
Of course it should have been:
RewriteCond %{HTTP:range} !^bytes=[^,]+(,[^,]+){0,4}$
RewriteRule .* - [F]
As said untested. Thanks for remote eyes.
Regards
Rüdiger
RE: Mitigation Range header (Was: DoS with mod_deflate range requests)
-Original Message-
From: Dirk-Willem van Gulik [mailto:di...@webweaving.org]
Sent: Mittwoch, 24. August 2011 14:14
To: dev@httpd.apache.org
Subject: Re: Mitigation Range header (Was: DoS with
mod_deflate range requests)
On 24 Aug 2011, at 12:57, Plüm, Rüdiger, VF-Group wrote:
- Where possible - disable mod_deflate
= we sure this covers all cases - or this is a good stopgap ?
As said this has *nothing* to do with mod_deflate. This was
IMHO just
a guess by the original author of the tool.
Ok - but when I try it on my servers (with the check of the
tool removed) - it seems quite impotent unless mod_deflate
is in the wire.
Hm, weird. I would guess that mod_deflate could even mitigate this attack as
the byterange
filter only does something if it sees the whole response in the brigade the
first time it
is called. Having mod_deflate compressing larger chunks of data causes the
byterange filter
to be called multiple times with only parts of the response in the brigade.
So the byte range filter should only be applied with responses whose compressed
response
fits into the zlibs output filter.
Depending on the size of the input and the number of parallel requests it might
be possible
that a lot of memory is consumed by mod_deflate anyway. But I would expect the
same behviour
without range requests as well.
And it seems a bit more potent when there is other 'keep in
the air' modules around.
So I guess mod_deflate is right now the largest 'plug' we
have in the server which can cause this backup ?
Or is that totally wrong. Happy to stand correctede !
- Where possible - set LimitRequestFieldSize to a small value
- Suggesting of 128 fine ?
- Where this is not possible (e.g. long cookies, auth
headers of serious size) consider using
mod_rewrite to not accept more than a few commas
= anyone a config snipped for this ?
How about the following (untested) rewrite rule. It should
only allow 5
ranges at max.
RewriteCond %{HTTP:range} ^bytes=[^,]+(,[^,]+){0,4}$
RewriteRule .* - [F]
Sounds like a plan ! This mail crossed one I just sent out -
lemme update that too.
Please see my response to Eric. He detected an error in the above.
Regards
Rüdiger
RE: Mitigation Range header
-Original Message- From: Dirk-WIllem van Gulik [mailto:di...@webweaving.org] Sent: Mittwoch, 24. August 2011 14:40 To: dev@httpd.apache.org Cc: Plüm, Rüdiger, VF-Group Subject: Re: Mitigation Range header On 24 Aug 2011, at 13:22, Florian Weimer wrote: * Plüm, Rüdiger, VF-Group: As said this has *nothing* to do with mod_deflate. This was IMHO just a guess by the original author of the tool. This matches my testing, too. I see a significant peak in RAM usage on a server where apachectl -M does not print anything with the string deflate (so I assume that mod_deflate is not enabled). This is with 2.2.9-10+lenny9 on Debian. If it is more difficult to check if mod_deflate is enabled, the advisory should tell how to check your server. If the method I used is the correct one, I don't think it's reasonable to suggest disabling mod_deflate as a mitigation because it does not seem to make much of a difference. Hmm - when I remove mod_deflate (i.e. explicitly as it is the default in all our installs) and test on a / entry which is a static file which is large (100k)* - then I cannot get apache on its knees on a freebsd machine - saturating the 1Gbit connection it has (Note: the attack machines *are* getting saturated). The moment i put in mod_deflate, mod_external filter, etc - it is much easier to get deplete enough resources to notice. Dw. *: as I cannot reproduce the issue with very small index.html files. Have you tried if the same happens with mod_deflate, but with one of the the proposed mitigations in place? As said my guess is that this might be an issue with mod_deflate that is unrelated to the Range request issue. Regards Rüdiger
RE: Mitigation Range header (Was: DoS with mod_deflate range requests)
-Original Message-
From: Eric Covener [mailto:cove...@gmail.com]
Sent: Mittwoch, 24. August 2011 14:59
To: dev@httpd.apache.org
Subject: Re: Mitigation Range header (Was: DoS with
mod_deflate range requests)
Of course it should have been:
RewriteCond %{HTTP:range} !^bytes=[^,]+(,[^,]+){0,4}$
RewriteRule .* - [F]
The problem with the negation is you need an addl rule to handle an
empty range header, this would forbid normal non-range requests.
Damn it. Got me again. How about this:
RewriteCond %{HTTP:range} !(^bytes=[^,]+(,[^,]+){0,4}$|^$)
RewriteRule .* - [F]
Regards
Rüdiger
RE: CVE-2011-3192: Range header DoS vulnerability in Apache 1.3 and Apache 2 (DRAFT-3)
Reverse the order a litte bit:
2) , 3), 1) (as 1) is likely to break the most things compared to 2) and 3))
Regarding 2) see the ongoing discussion between Eric and me to find the correct
expression.
Regards
Rüdiger
-Original Message-
From: Dirk-WIllem van Gulik
Sent: Mittwoch, 24. August 2011 15:08
To: Dirk-Willem van Gulik
Cc: dev@httpd.apache.org; secur...@httpd.apache.org
Subject: Re: CVE-2011-3192: Range header DoS vulnerability in
Apache 1.3 and Apache 2 (DRAFT-3)
* Folks - do we also need to add Request-Range ?
* Updated with Rudigers comments., Eric, Florians
* Consensus that the deflate stuff needs to go out reflected.
* More Comments please. Esp. on the quality and
realisticness of the mitigtions.
* Is this the right list (and order) of the mitigations -
or should ReWrite be first ?
* Timeline mentioning fine (we've never done that before)
-- or best avoided ?
My plan is to wait for the US to fully wake up - and then
call for a few quick +1's to get this out - ideally before 1600 zulu.
Thanks,
Dw.
Title:CVE-2011-3192: Range header DoS
vulnerability in Apache 1.3 and Apache 2
Date: 20110824 1600Z
# Last Updated: 20110824 1600Z
Product: Apache Web Server
Versions: Apache 1.3 all versions, Apache 2 all versions
Description:
A denial of service vulnerability has been found in the way
the multiple overlapping ranges are handled by apache
(http://seclists.org/fulldisclosure/2011/Aug/175). An attack
tool is circulating in the wild. Active use of this tools has
been observed.
The attack can be done remotely and with a modest number of
requests leads to very significant memory and CPU usage.
The default apache installation is vulnerable.
There is currently no patch/new version of apache which fixes
this vulnerability. This advisory will be updated when a long
term fix is available. A fix is expected in the next 96 hours.
Mitigation:
However are several immediate options to mitigate this issue
until that time:
1)Use mod_headers to dis-allow the use of Range headers:
RequestHeader unset Range
Note that this may break certain clients - such as
those used for
e-Readers and progressive/http-streaming video.
2)Use mod_rewrite to limit the number of ranges:
RewriteCond %{HTTP:range} !^bytes=[^,]+(,[^,]+){0,4}$
RewriteRule .* - [F]
3)Limit the size of the request field to a few hundred
bytes. Note that while this
keeps the offending Range header short - it may break
other headers; such as sizable
cookies or security fields.
LimitRequestFieldSize 200
Note that as the attack evolves in the field you are
likely to have
to further limit this and/or impose other
LimitRequestFields limits.
See:
http://httpd.apache.org/docs/2.2/mod/core.html#limitrequestfieldsize
3)Deploy a Range header count module as a temporary
stopgap measure:
http://people.apache.org/~dirkx/mod_rangecnt.c
5)Apply any of the current patches under discussion - such as:
http://mail-archives.apache.org/mod_mbox/httpd-dev/201108.mbox
/%3cCAAPSnn2PO-d-C4nQt_TES2RRWiZr7urefhTKPWBC1b+K1Dqc7g@mail.g
mail.com%3e
Actions:
---
Apache HTTPD users are advised to investigate wether they are
vulnerable (e.g. allow use of the Range header )and consider
implementing any of the above mitigations immediately.
When using a third party attack tool to verify vulnerability
- know that most of the versions in the wild currently check
for the presence of mod_deflate; and will (mis)report that
your server is not vulnerable if this module is not present.
This vulnerability is not dependent on presence or absence of
that module.
Planning:
-
This advisory will be updated when a fix/patch or new release
is available. A patch or new apache release for Apache 2.0
and 2.2 is expected in the next 96 hours. Note that, while
popular, Apache 1.3 is deprecated.
RE: CVE-2011-3192: Range header DoS vulnerability in Apache 1.3 and Apache 2 (DRAFT-3)
-Original Message-
From: Eric Covener [mailto:cove...@gmail.com]
Sent: Mittwoch, 24. August 2011 15:29
To: dev@httpd.apache.org
Subject: Re: CVE-2011-3192: Range header DoS vulnerability in
Apache 1.3 and Apache 2 (DRAFT-3)
On Wed, Aug 24, 2011 at 9:17 AM, Eric Covener
cove...@gmail.com wrote:
* Is this the right list (and order) of the
mitigations - or should ReWrite be first ?
FWIW I don't like rewrite first because it's so unruly with being
defined once per vhost + main server + RewriteEngine on.
I like RequestHeader simplicity, and could be combined with SetEnvIf
to only zap long malicious looking headers.
e.g.
SetEnvIf Range (,.*?){5,} bad-range=1
RequestHeader unset Range env=bad-range
Nice one as well. Might be even better then the rewrite rule.
Regards
Rüdiger
RE: Mitigation Range header
From: Greg Ames Sent: Mittwoch, 24. August 2011 16:05 To: dev@httpd.apache.org Subject: Re: Mitigation Range header On Wed, Aug 24, 2011 at 9:01 AM, Plüm, Rüdiger, VF-Group ruediger.pl...@vodafone.com wrote: Hmm - when I remove mod_deflate (i.e. explicitly as it is the default in all our installs) and test on a / entry which is a static file which is large (100k)* - then I cannot get apache on its knees on a freebsd machine - saturating the 1Gbit connection it has (Note: the attack machines *are* getting saturated). The moment i put in mod_deflate, mod_external filter, etc - it is much easier to get deplete enough resources to notice. Dw. *: as I cannot reproduce the issue with very small index.html files. Have you tried if the same happens with mod_deflate, but with one of the the proposed mitigations in place? As said my guess is that this might be an issue with mod_deflate that is unrelated to the Range request issue. I think mod_deflate is just the tool to convert an O(N^2) data size problem into an O(N^2) CPU usage problem, where N is some function of LimitRequestLine. If the file size is smaller than the largest range end used in the attack, it may reduce the amount of data actually going down the filter chain. Greg I don't think so. The compression happens before the byterange filter and the byterange filter just hacks the already compressed brigade into more buckets and rearranges them. mod_deflate does not do more work if it is a range request. It does the same amount of work as for the non range request. Regards Rüdiger
RE: VOTES please -- CVE-2011-3192: Range header DoS vulnerability in Apache 1.3 and Apache 2 (Final-5)
-Original Message-
From: Dirk-Willem van Gulik [mailto:di...@webweaving.org]
Sent: Mittwoch, 24. August 2011 16:36
To: dev@httpd.apache.org
Subject: VOTES please -- CVE-2011-3192: Range header DoS
vulnerability in Apache 1.3 and Apache 2 (Final-5)
Folks,
Can I have a few +1's on below - or feedback on what we'd
like to have changed ?
* Would like to get this out in an hour or so ?
* FIne with the 48 hours commitment of an update ?
Dw.
Title:CVE-2011-3192: Range header DoS vulnerability
Apache HTTPD 1.3/2.x
Date: 20110824 1600Z
Product: Apache HTTPD Web Server
Versions: Apache 1.3 all versions, Apache 2 all versions
Description:
A denial of service vulnerability has been found in the way
the multiple overlapping ranges are handled by the Apache
HTTPD server:
http://seclists.org/fulldisclosure/2011/Aug/175
An attack tool is circulating in the wild. Active use of this
tools has been observed.
The attack can be done remotely and with a modest number of
requests can cause very significant memory and CPU usage on
the server.
The default Apache HTTPD installation is vulnerable.
There is currently no patch/new version of Apache HTTPD which
fixes this vulnerability. This advisory will be updated when
a long term fix is available.
A full fix is expected in the next 48 hours.
Mitigation:
However there are several immediate options to mitigate this
issue until that time:
1) Use mod_rewrite to limit the number of ranges:
Option 1L
RewriteCond %{HTTP:range} !(^bytes=[^,]+(,[^,]+){0,4}$|^$)
RewriteRule .* - [F]
Option 2:
SetEnvIf Range (,.*?){5,} bad-range=1
RequestHeader unset Range env=bad-range
# optional logging.
CustomLog logs/range.log %r %{Range}i %{bad-range}e
Shouldn't it be a conditional logging?
CustomLog logs/range.log %r %{Range}i env=bad-range
Otherwise looks good. +1.
Regards
Rüdiger
RE: DoS with mod_deflate range requests
-Original Message- From: Dirk-Willem van Gulik [mailto:dirk-willem.van.gu...@bbc.co.uk] Sent: Mittwoch, 24. August 2011 17:46 To: dev@httpd.apache.org Subject: Re: DoS with mod_deflate range requests On 24 Aug 2011, at 16:35, Tim Bannister wrote: On Tue, Aug 23, 2011, Roy T. Fielding wrote: And the spec says ... When a client requests multiple ranges in one request, the server SHOULD return them in the order that they appeared in the request. My suggestion is to reject any request with overlapping ranges or more than five ranges with a 416, and to send 200 for any request with 4-5 ranges. There is simply no need to support random access in HTTP. Deshpande Zeng in http://dx.doi.org/10.1145/500141.500197 describe a method for streaming JPEG 2000 documents over HTTP, using many more than 5 ranges in a single request. A client that knows about any server-side limit could make multiple requests each with a small number of ranges, but discovering that limit will add latency and take more code. Agreed - I've seen many 10's of ranges in a single request for things like clever PDF pagination (or tiny TIFF quicklooks for the pages), clever http fake streaming and clever use of jumping to i-Frames. I think we just need to sit this out - and accept up to RequestFieldSize limit bytes on that line - and then do a sort merge overlaps as needed. Hm. If I got it right what Roy says above about the spec sorting and merging is not an option as we need to stick to the order and number of ranges the client requested. But we can deny overlapping with a 416. Or we do a 416 as well if merging would change something. Regards Rüdiger
RE: DoS with mod_deflate range requests
-Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Mittwoch, 24. August 2011 17:48 To: dev@httpd.apache.org Subject: Re: DoS with mod_deflate range requests On Aug 24, 2011, at 4:05 AM, Plüm, Rüdiger, VF-Group wrote: Patch looks good, but some comments: As far as I can see the following range request would not get merged: Range: bytes=0-0,1-1,2-2 into a 0-2 range as need_sort would remain 0. OTOH Range: bytes=0-0,0-1,1-2 would get get merged into a 0-2 range. Using boundary and !boundary in the later if statements to decide whether a request is multi range or single range is IMHO bad as boundary is set based on the old number ranges and not based on the number of merged ranges. So multiple ranges in the beginning might get merged to a single range in the end. +1... Suggestion: Let's fold the patch, as-is, into trunk, tune it there and then backport to 2.x... Based on Roy's comment about the spec I think we cannot optimize this way. I think we can only detect if something weird goes on (overlapping, merging would result in smaller number of ranges, excessive number of ranges, whereas excessive needs to be configurable with a sane default) and reply with a 416 then. Regards Rüdiger
RE: DoS with mod_deflate range requests
-Original Message- From: Jim Jagielski [mailto:j...@jagunet.com] Sent: Mittwoch, 24. August 2011 18:02 To: dev@httpd.apache.org Subject: Re: DoS with mod_deflate range requests Sorting isn't allowed but I get the impression that merging is OK... Roy can confirm... But merging might require sorting... If not, then some sort of runtime limit on the number of allowable ranges plus a 416 w/ overlapping ranges makes the most sense. On Aug 24, 2011, at 11:55 AM, Plüm, Rüdiger, VF-Group wrote: Hm. If I got it right what Roy says above about the spec sorting and merging is not an option as we need to stick to the order and number of ranges the client requested. But we can deny overlapping with a 416. Or we do a 416 as well if merging would change something. Regards Rüdiger Regards Rüdiger
RE: Mitigation Range header
From: Greg Ames [mailto:ames.g...@gmail.com] Sent: Mittwoch, 24. August 2011 18:20 To: dev@httpd.apache.org Subject: Re: Mitigation Range header On Wed, Aug 24, 2011 at 10:33 AM, Plüm, Rüdiger, VF-Group ruediger.pl...@vodafone.com wrote: I think mod_deflate is just the tool to convert an O(N^2) data size problem into an O(N^2) CPU usage problem, where N is some function of LimitRequestLine. If the file size is smaller than the largest range end used in the attack, it may reduce the amount of data actually going down the filter chain. Greg I don't think so. The compression happens before the byterange filter and the byterange filter just hacks the already compressed brigade into more buckets and rearranges them. mod_deflate does not do more work if it is a range request. It does the same amount of work as for the non range request. OK, thanks for the clarification, Rüdiger. Then I don't understand why mod_deflate seems to be an important factor in killing the server. If the DEFLATE filter runs first, can you do anything useful with a subrange of its output? i.e., could a client decompress a subrange that starts in the middle of the compressed version and get a subrange of the original uncompressed data? Greg It depends. Think of a download that was broken in the middle and that was compressed by mod_deflate on the fly. A client could just request the missing bits and decompress the whole thing afterwards. I guess in general it only works if the client has all the missing pieces (or at least the ones before the contents of the partial response) that are not sent via the partial response at hand to decompress the whole stream (or decompress at least until the last part of the partial response). Regards Rüdiger
RE: With IP address in Host: header ServerName/ServerAlias doesn't work
-Original Message- From: Micha Lenk [mailto:mi...@lenk.info] Sent: Dienstag, 23. August 2011 12:08 To: dev@httpd.apache.org Subject: Re: With IP address in Host: header ServerName/ServerAlias doesn't work Hi, On 08/23/2011 10:42 AM CEST +02:00, Micha Lenk wrote: However, I believe the fix is yet incomplete. The function ap_matches_request_vhost() used by modules like mod_proxy seems to implement the virtual host check also in the wrong order. [...] I'll follow up with a patch for this function later. Here you go. Attached is the suggested patch for the function ap_matches_request_vhost(). Anything else missing? IMHO the patch does not solve the issue mentioned in the comment and is not needed. Keep in mind the difference between ap_matches_request_vhost and check_host_alias: ap_matches_request_vhost checks only the data of *one* server_rec and hence only *one* Servername and ServerAlias settings as set for this vhost, whereas check_host_alias scans over *multiple* server_recs with *multiple* Servername and ServerAlias settings from different vhosts. Regards Rüdiger
RE: DoS with mod_deflate range requests
-Original Message- From: Stefan Fritsch [mailto:s...@sfritsch.de] Sent: Dienstag, 23. August 2011 13:09 To: dev@httpd.apache.org Subject: DoS with mod_deflate range requests http://seclists.org/fulldisclosure/2011/Aug/175 I haven't looked into it so far. And I am not sure I will have time today. After checking the attack script and the code this has IMHO nothing to do with mod_deflate but only with the byterange filter. But I admit that haven't run the script to check. The host is seen as vulnerable if it replies to a range request that requests the whole entity via a range 0- with a partial response. A possible problem is that the output bucket brigade gets transformed in a one bucket per byte brigade and thus into a brigade with many buckets. Futhermore the created range response has a lot of buckets with boundaries, strings allocated from r-pool. So it might be advisable if we limit the number of ranges we accept contained in a Range header. As a further optimization we could check for 0- ranges and once we hit one just reply with the full response instead of a partial response. Regards Rüdiger
RE: [PATCH 51489] ProxyPassReverse issue + patch
-Original Message- From: Micha Lenk [mailto:mi...@lenk.info] Sent: Montag, 22. August 2011 14:19 To: dev@httpd.apache.org Subject: Re: [PATCH 51489] ProxyPassReverse issue + patch Hi Ruediger, sorry, things piled up recently... On 07/13/2011 06:36 PM CEST +02:00, Ruediger Pluem wrote: Try ProxyPassReverse balancer://196f045aca6adc82a0b6eea93ed286a1 instead. This results in an Internal Server Error: [warn] proxy: No protocol handler was valid for the URL request URL. If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule. Doesn't seem to be a solution... :( This means that your config is wrong and you have not loaded the required protocol handlers like mod_proxy_http or mod_proxy_ajp. Or your config does not load mod_proxy_balancer. Regards Rüdiger
RE: With IP address in Host: header ServerName/ServerAlias doesn't work
-Original Message- From: Micha Lenk Sent: Montag, 22. August 2011 18:10 To: dev@httpd.apache.org Subject: With IP address in Host: header ServerName/ServerAlias doesn't work Do you agree that this is something that needs to be fixed? If yes I could start to work on a patch... No, this works as designed and documented for ages. I guess your question is better suited for the us...@httpd.apache.org list. Regards Rüdiger
RE: With IP address in Host: header ServerName/ServerAlias doesn't work
-Original Message- From: Micha Lenk [mailto:mi...@lenk.info] Sent: Montag, 22. August 2011 18:27 To: dev@httpd.apache.org Subject: Re: With IP address in Host: header ServerName/ServerAlias doesn't work Hi Ruediger, On 08/22/2011 06:16 PM CEST +02:00, Plüm, Rüdiger, VF-Group wrote: No, this works as designed and documented for ages. I guess your question is better suited for the us...@httpd.apache.org list. Sorry, I disagree. The documentation reads: Now when a request arrives, the server will first check if it is using an IP address that matches the NameVirtualHost. If it is, then it will look at each VirtualHost section with a matching IP address and try to find one where the ServerName or ServerAlias matches the requested hostname. If it finds one, then it uses the configuration for that server. If no matching virtual host is found, then the first listed virtual host that matches the IP address will be used. Source: http://httpd.apache.org/docs/2.2/en/vhosts/name-based.html In the case I described earlier, *only* the virtual server v2 has a ServerName that matches the requested hostname. The last sentence from the cited documentation doesn't apply here. Sorry, I missed the ServerAlias for the IP in the second virtual host. So, yes in general the second virtual host should be hit. But using IP addresses as Serveralias is quite unusual and in this case the solution is to just reverse the definition of the virtual hosts. Regards Rüdiger
RE: DO NOT REPLY [Bug 51679] New: Code signature key expired
IMHO it can be resigned in place since we do not touch the release artifacts itself. But as Bill did the release IMHO he should resign the release to be consistent with the other metadata of this release (e.g. the creator of the 2.2.19) tag. Regards Rüdiger -Original Message- From: Eric Covener Sent: Donnerstag, 18. August 2011 17:29 To: dev@httpd.apache.org Subject: Fwd: DO NOT REPLY [Bug 51679] New: Code signature key expired CHANGES says that currently nothing is backported to 2.2.x since 2.2.19 -- should we burn a release # to replace? Can the existing release be re-signed in-place? -- Forwarded message -- From: bugzi...@apache.org Date: Thu, Aug 18, 2011 at 9:29 AM Subject: DO NOT REPLY [Bug 51679] New: Code signature key expired To: b...@httpd.apache.org https://issues.apache.org/bugzilla/show_bug.cgi?id=51679 Bug #: 51679 Summary: Code signature key expired Product: Apache httpd-2 Version: 2.2.19 Platform: All OS/Version: All Status: NEW Severity: minor Priority: P2 Component: All AssignedTo: b...@httpd.apache.org ReportedBy: brunz...@dimdi.de Classification: Unclassified The signing key for httpd-2.2.19.tar.gz.asc seems to have expired: gpg -v --verify httpd-2.2.19.tar.gz.asc gpg: armor header: Version: GnuPG v1.4.9 (GNU/Linux) gpg: assuming signed data in `httpd-2.2.19.tar.gz' gpg: Signature made Fri 20 May 2011 07:02:24 PM CEST using RSA key ID 7F7214A7 gpg: NOTE: signature key CB9B9EC5 expired Fri 03 Jul 2009 05:21:46 PM CEST gpg: NOTE: signature key 7F7214A7 expired Sat 09 Jul 2011 02:54:10 AM CEST ^^ gpg: using subkey 7F7214A7 instead of primary key B55D9977 gpg: NOTE: signature key 7F7214A7 expired Sat 09 Jul 2011 02:54:10 AM CEST gpg: NOTE: signature key CB9B9EC5 expired Fri 03 Jul 2009 05:21:46 PM CEST gpg: NOTE: signature key 7F7214A7 expired Sat 09 Jul 2011 02:54:10 AM CEST gpg: using PGP trust model gpg: Good signature from William A. Rowe, Jr. wr...@rowe-clan.net gpg: aka William A. Rowe, Jr. wr...@apache.org gpg: aka William A. Rowe, Jr. wr...@vmware.com gpg: aka William A. Rowe, Jr. william.r...@springsource.com gpg: NOTE: signature key CB9B9EC5 expired Fri 03 Jul 2009 05:21:46 PM CEST gpg: NOTE: signature key 7F7214A7 expired Sat 09 Jul 2011 02:54:10 AM CEST gpg: Note: This key has expired! Primary key fingerprint: B1B9 6F45 DFBD CCF9 7401 9235 193F 180A B55D 9977 Subkey fingerprint: 4962 0827 E32B C882 DC6B EF54 A348 B984 7F72 14A7 gpg: binary signature, digest algorithm SHA1 -- Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug. - To unsubscribe, e-mail: bugs-unsubscr...@httpd.apache.org For additional commands, e-mail: bugs-h...@httpd.apache.org -- Eric Covener cove...@gmail.com
RE: Half-baked subprojects
-Original Message- From: Eric Covener [mailto:cove...@gmail.com] Sent: Donnerstag, 11. August 2011 15:19 To: dev@httpd.apache.org Subject: Re: Half-baked subprojects Would I be right in assuming apreq maintenance straddles httpd and mod_perl folks, and you are around on this list? I guess an apreq subcategory under bugzilla/httpd would fix this. How many of our subprojects have dangling loose ends like this? Maybe no others -- fcgid and ftp look like normal mods, and flood lives under httpd-test (from a purely bugzilla POV). +1 to component of libapreq2 under apache-httpd2. Is that an infra ticket or do some folks here have access? Done. IMHO the PMC chair should have the Bugzilla karma to do this. If you don't have it, please ask infra to grant it. Regards Rüdiger
RE: mod_proxy_ajp: ignoring flush before headers (again)
Can you please try if the following patch fixes your issue?
Index: mod_proxy_ajp.c
===
--- mod_proxy_ajp.c (revision 1150558)
+++ mod_proxy_ajp.c (working copy)
@@ -506,16 +506,18 @@
if (bb_len != -1)
conn-worker-s-read += bb_len;
}
-if (ap_pass_brigade(r-output_filters,
-output_brigade) != APR_SUCCESS) {
-ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
- proxy: error processing body.%s,
- r-connection-aborted ?
- Client aborted connection. : );
-output_failed = 1;
+if (headers_sent) {
+if (ap_pass_brigade(r-output_filters,
+output_brigade) !=
APR_SUCCESS) {
+ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
+ proxy: error processing
body.%s,
+ r-connection-aborted ?
+ Client aborted connection. :
);
+output_failed = 1;
+}
+data_sent = 1;
+apr_brigade_cleanup(output_brigade);
}
-data_sent = 1;
-apr_brigade_cleanup(output_brigade);
}
}
else {
Currently the code sends an empty brigade in your case which also triggers the
sending of headers by httpd.
Regards
Rüdiger
-Original Message-
From: Jim Riggs
Sent: Dienstag, 2. August 2011 18:03
To: dev@httpd.apache.org
Subject: mod_proxy_ajp: ignoring flush before headers (again)
For some (old 2007) context, see:
http://markmail.org/message/btwcnbl2i7ftwj4n
https://community.jivesoftware.com/message/201787
I am proxying an app via AJP to Tomcat 6/7. In certain
circumstances, it appears that the app (or possibly Tomcat)
is erroneously sending a flush before the headers have been
sent. In r57, Jim added an exception to handle this
situation with the intention of ignoring the flush. I'm not
sure it's working quite right, though, as the brigade is
still getting passed through the filter chain. So,
ap_headers_output_filter() is getting called too soon, I
think. (I am no expert in the httpd code, so I'm not sure
this is really the problem.)
Can any of you who ARE experts in the code tell me what you
think of the issue and how we can fix it? I'm thinking that
if we are ignoring a flush at mod_proxy_ajp.c:448 (in 2.2.x),
we should not be calling ap_pass_brigade() at line 472, but I
don't know if there are any ramifications of that.
The symptom is that when this issue happens, the user gets
prompted to save a file (Content-Type returned by httpd is
'text/plain' even though Tomcat is returning
'text/html;charset=utf-8'). Below is some debug output
showing correct and incorrect behavior:
Correct:
[Tue Aug 02 09:34:50 2011] [debug] mod_proxy_ajp.c(266):
proxy: APR_BUCKET_IS_EOS
[Tue Aug 02 09:34:50 2011] [debug] mod_proxy_ajp.c(271):
proxy: data to read (max 8186 at 4)
[Tue Aug 02 09:34:50 2011] [debug] mod_proxy_ajp.c(286):
proxy: got 0 bytes of data
[Tue Aug 02 09:34:50 2011] [debug] ajp_header.c(687):
ajp_read_header: ajp_ilink_received 04
[Tue Aug 02 09:34:50 2011] [debug] ajp_header.c(697):
ajp_parse_type: got 04
[Tue Aug 02 09:34:50 2011] [debug] ajp_header.c(516):
ajp_unmarshal_response: status = 200
[Tue Aug 02 09:34:50 2011] [debug] ajp_header.c(537):
ajp_unmarshal_response: Number of headers is = 5
[Tue Aug 02 09:34:50 2011] [debug] ajp_header.c(599):
ajp_unmarshal_response: Header[0] [Pragma] = [No-cache]
[Tue Aug 02 09:34:50 2011] [debug] ajp_header.c(599):
ajp_unmarshal_response: Header[1] [Cache-Control] = [no-cache]
[Tue Aug 02 09:34:50 2011] [debug] ajp_header.c(599):
ajp_unmarshal_response: Header[2] [Expires] = [Wed, 31 Dec
1969 18:00:00 CST]
[Tue Aug 02 09:34:50 2011] [debug] ajp_header.c(599):
ajp_unmarshal_response: Header[4] [Content-Type] =
[text/html;charset=utf-8]
[Tue Aug 02 09:34:50 2011] [debug] ajp_header.c(609):
ajp_unmarshal_response: ap_set_content_type done
[Tue Aug 02 09:34:50 2011] [debug] ajp_header.c(687):
ajp_read_header: ajp_ilink_received 03
[Tue Aug 02 09:34:50 2011] [debug] ajp_header.c(697):
ajp_parse_type: got 03
[Tue Aug 02 09:34:50 2011] [debug] mod_headers.c(756):
headers: ap_headers_output_filter()
Incorrect (notice how ap_headers_output_filter() is called
before the headers are received):
