Re: Memory Pool
On 11/10/2010 20:48, Nick Kew wrote: On Mon, 11 Oct 2010 15:14:02 +0100 Martin Townsendmartin.towns...@power-oasis.com wrote: Hi, I have created a pool from the child pool for storing warning messages that can live across requests, the final request will insert the warnings into the response. How do I ensure that this pool is cleared at the end of the final request? That doesn't really make sense. What is the final request? If it's requests in a connection, use the connection pool. Otherwise, you're looking at a time-based solution such as garbage collection. Thanks for the reply and Apologies, I should have put a bit more detail in. I'm working on a module for an embedded platform where we can use the web server to configure the firmware. The interface between the module and firmware allows the firmware to send back warning messages for side effects of configuration changes. The time taken for the configuration change will likely outlive the connection timeout. I could increase this timeout but I would like to keep it short as you can only connect to the web server from a local network and then I can keep the number of child processes low. These warnings are inserted into a response as a table within a DIV HTML element by my output filter. So what usually happens is that the first request is a POST with the configure command, an input filter parses this and sends the configure to the firmware, the firmware then may send back warning messages. Theses are collected and stored in a list. The next request (which I wrongly referred to as the final request) will then have the warning table inserted into the response to which the client can then use javascript to display as an overlay. I've implemented a solution where I use a sub pool from the child pool which is now cleared using a flag which I set when the warnings are processed by the output filter. The next (initial) request will then pick up the flag and clear the pool. What I was hoping for, is a hook at a point after the data had been sent but the flag works well. -- Martin Townsend Power*Oasis* Suite 1, Unit 41 Shrivenham Hundred Business Park Majors Road Watchfield Swindon Wilts SN6 8TZ Mobile: 07884 444658 Skype: mtownsend1973
Re: Memory Pool
Martin, if you are working in a constrained environment, then you are probably better off using something like libmicrohttpd[1] or libevent's evhttp interface[2]. Apache has a rather heavy resource footprint. [1] http://www.gnu.org/software/libmicrohttpd/ [2] http://monkey.org/~provos/libevent/doxygen/evhttp_8h.html
Re: svn commit: r1021546 - in /httpd/httpd/trunk: CHANGES docs/manual/mod/mod_cache.xml include/ap_mmn.h modules/cache/cache_util.h modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mo
On 10/12/2010 01:32 AM, minf...@apache.org wrote:
Author: minfrin
Date: Mon Oct 11 23:32:56 2010
New Revision: 1021546
URL: http://svn.apache.org/viewvc?rev=1021546view=rev
Log:
mod_cache: Optionally serve stale data when a revalidation returns a
5xx response, controlled by the CacheStaleOnError directive.
Modified:
httpd/httpd/trunk/CHANGES
httpd/httpd/trunk/docs/manual/mod/mod_cache.xml
httpd/httpd/trunk/include/ap_mmn.h
httpd/httpd/trunk/modules/cache/cache_util.h
httpd/httpd/trunk/modules/cache/mod_cache.c
httpd/httpd/trunk/modules/cache/mod_cache.h
httpd/httpd/trunk/modules/cache/mod_disk_cache.c
Modified: httpd/httpd/trunk/modules/cache/mod_disk_cache.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/cache/mod_disk_cache.c?rev=1021546r1=1021545r2=1021546view=diff
==
--- httpd/httpd/trunk/modules/cache/mod_disk_cache.c (original)
+++ httpd/httpd/trunk/modules/cache/mod_disk_cache.c Mon Oct 11 23:32:56 2010
@@ -860,16 +860,14 @@ static apr_status_t recall_headers(cache
static apr_status_t recall_body(cache_handle_t *h, apr_pool_t *p,
apr_bucket_brigade *bb)
{
-apr_bucket *e;
disk_cache_object_t *dobj = (disk_cache_object_t*) h-cache_obj-vobj;
if (dobj-data.fd) {
-apr_brigade_insert_file(bb, dobj-data.fd, 0, dobj-file_size, p);
+apr_bucket *e = apr_bucket_file_create(dobj-data.fd, 0,
+dobj-file_size, p, bb-bucket_alloc);
+APR_BRIGADE_INSERT_HEAD(bb, e);
}
-e = apr_bucket_eos_create(bb-bucket_alloc);
-APR_BRIGADE_INSERT_TAIL(bb, e);
-
return APR_SUCCESS;
}
What is the purpose of these changes? What makes the need for
apr_brigade_insert_file
go away and why don't we add an eos any longer?
Regards
Rüdiger
Re: svn commit: r1021546 - in /httpd/httpd/trunk: CHANGES docs/manual/mod/mod_cache.xml include/ap_mmn.h modules/cache/cache_util.h modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mo
On 12 Oct 2010, at 8:16 AM, Ruediger Pluem wrote:
Modified: httpd/httpd/trunk/modules/cache/mod_disk_cache.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/cache/mod_disk_cache.c?rev=1021546r1=1021545r2=1021546view=diff
=
=
=
=
=
=
=
=
=
=
--- httpd/httpd/trunk/modules/cache/mod_disk_cache.c (original)
+++ httpd/httpd/trunk/modules/cache/mod_disk_cache.c Mon Oct 11
23:32:56 2010
@@ -860,16 +860,14 @@ static apr_status_t recall_headers(cache
static apr_status_t recall_body(cache_handle_t *h, apr_pool_t *p,
apr_bucket_brigade *bb)
{
-apr_bucket *e;
disk_cache_object_t *dobj = (disk_cache_object_t*) h-cache_obj-
vobj;
if (dobj-data.fd) {
-apr_brigade_insert_file(bb, dobj-data.fd, 0, dobj-
file_size, p);
+apr_bucket *e = apr_bucket_file_create(dobj-data.fd, 0,
+dobj-file_size, p, bb-bucket_alloc);
+APR_BRIGADE_INSERT_HEAD(bb, e);
}
-e = apr_bucket_eos_create(bb-bucket_alloc);
-APR_BRIGADE_INSERT_TAIL(bb, e);
-
return APR_SUCCESS;
}
What is the purpose of these changes? What makes the need for
apr_brigade_insert_file
go away and why don't we add an eos any longer?
The changes to cache_out_filter() didn't come out cleanly in the
patch, it would have made it clearer if it had.
In the past, cache_out_filter() made the bogus assumption that it
would always receive an empty brigade from cache_[quick_]handler(),
and that it was always safe to just add the file, followed by an eos
to the end of this empty brigade.
Now, in the failure case, we already have buckets in the brigade
representing the error, which we need to replace with our stale cached
file, so our assumption that the brigade is empty is no longer valid.
What we do now, is have the cache_out_filter() delete existing buckets
properly until it finds eos, and then asks recall_body() to prepend
the body to the front of the brigade, instead of appending it to at
the end and creating it's own eos, as before.
Because an eos bucket is already present in the output brigade, it is
no longer valid to add a second eos bucket inside recall_body(), and
for this reason, it was removed.
Looking deeper at the code for apr_brigade_insert_file(), it looks
like it does more than just add a single file bucket, but rather adds
multiple smaller file buckets if the file exceeds a certain size. What
we may need to do is switch back to using apr_brigade_insert_file(),
but then prepend the brigade returned to the brigade destined for the
client.
Regards,
Graham
--
Re: Planning some tags, and more tags
On 11.10.2010 20:35, William A. Rowe Jr. wrote: ... and tagged 0.9. Note 0.9 -util tag already failed on Darwin and we are holding 2.0 tag for a bundled expat for -util fix. There are four backport bullets in 2.0 STATUS with two signoffs already, if anyone has eyeballs to invest between now and when -util has been retagged. I'd hope this is the final 2.0 tag, so if any attract your attention as 'really should fix', this would be the day. I presume the bundled expat flaw did not apply to 2.2. Jeff was going to look at the last 2.2 patch per Joe's comments. Not sure if his travels interfere with this, so if he's unable I'm game to apply this with the additional footnotes before tagging. The build system is substantially different in 1.3.x. So neither is there the big difference in linker flags used for Darwin, AIX etc. and other *nix platforms, nor the different code path between implicit bundled and explicit builtin. In short: 1.3.10 looks good. I also retested 1.3.10 using implicit bundled expat, which was fine on Solaris. Couldn't do testing on Darwin or AIX though. Regards, Rainer
[PATCH] mod_cgi: Mitigating some header injections by dropping invalid headers?
Hi folks,
a few days ago on a bored afternoon thumbed through the Unusual Web Bugs
presentation [1] from 24C3. On slide 19/20 the author shows a way to
inject otherwise filtered headers from Flash into CGI scripts. This is
caused by sloppy filtering on the client side and the simple translation
to environment variables (essentially y/a-z/A-Z/;s/[^A-Z]/_/g) on the
server side. That way you can set eg. the HTTP_USER_AGENT environment
variable by sending a User!Agent:foo header.
I had a quick look at the Apache source and the solution was simple: Just
drop headers which contain any character outside the range [a-zA-Z0-9-].
The patch against trunk is attached.
Now, my next task was to imagine any way how this could be used for XSS
attacks as claimed in the presentation. You'd need a header which
contains a dash to do so; the only relevant one I could think of was X-
Requested-With but maybe there are others I don't know of. So, is this
really needed? Dunno.
On the other hand, would it hurt to be a little less forgiving when
parsing headers? I mean, this is the 21st century, HTTP is around for
almost 20 years, by now everybody who has to write a client should know
how to format headers. RFC3875 section 4.1.18 doesn't complain either.
Cheers,
Malte
P.S.: I couldn't find anything like apr_pfree to get rid of the memory
allocated for bad headers, but if I grokked the APR docs correctly, we've
got to wait until the pool is emptied to reclaim our memory, right?
Shouldn't hurt to keep those few unused bytes around then.
[1]http://events.ccc.de/congress/2007/Fahrplan/events/2212.en.html
Index: server/util_script.c
===
--- server/util_script.c (revision 1006168)
+++ server/util_script.c (working copy)
@@ -67,11 +67,14 @@
*cp++ = '_';
while ((c = *w++) != 0) {
-if (!apr_isalnum(c)) {
+if (apr_isalnum(c)) {
+*cp++ = apr_toupper(c);
+}
+else if (c == '-') {
*cp++ = '_';
}
else {
-*cp++ = apr_toupper(c);
+return NULL;
}
}
*cp = 0;
@@ -175,8 +178,8 @@
continue;
}
#endif
-else {
-apr_table_addn(e, http2env(r-pool, hdrs[i].key), hdrs[i].val);
+else if ((env_temp = http2env(r-pool, hdrs[i].key)) != NULL) {
+apr_table_addn(e, env_temp, hdrs[i].val);
}
}
Re: [PATCH] mod_cgi: Mitigating some header injections by dropping invalid headers?
On 12 Oct 2010, at 15:30, Malte S. Stretz wrote: I had a quick look at the Apache source and the solution was simple: Just drop headers which contain any character outside the range [a-zA-Z0-9-]. The patch against trunk is attached. This made me think of something we had a while ago; and after checking the logs - big +1 from me! Dw.
Problem with building httpd without /dev/random, link of htpasswd fails: ERROR: Undefined symbol: .apr_generate_random_bytes
Error message during make all in support: ld: 0711-317 ERROR: Undefined symbol: .apr_generate_random_bytes .apr_generate_random_bytes [46]ER PR htpasswd.c(.libs/htpasswd.o) 001c .textR_RBR[521] .seed_rand Using gcc on AIX 4.3.3 with following configure command: ./configure \ --prefix=/usr/local/apache2 \ --enable-layout=Apache \ --enable-module=so \ --enable-mods-shared=all \ --disable-auth_digest \ --with-z=/usr/local I have the disable of auth_digest because configue complained that it could not complete without /dev/random or egb installed. I prefer not to install egb. Suggestions for how I can 'neatly' satisfy htpasswd need for apr_generate_random_bytes() are appreciated! HTTP 2.2.13 code btw. Many thanks.
Re: [PATCH] mod_cgi: Mitigating some header injections by dropping invalid headers?
On 10/12/2010 10:06 AM, Dirk-Willem van Gulik wrote: On 12 Oct 2010, at 15:30, Malte S. Stretz wrote: I had a quick look at the Apache source and the solution was simple: Just drop headers which contain any character outside the range [a-zA-Z0-9-]. The patch against trunk is attached. This made me think of something we had a while ago; and after checking the logs - big +1 from me! Agreed, with a caviat... we aught to be able to toggle this for the rare but significant legacy app that requires it... which implies a per-dir flag that can override just one CGI script out of an entire server.
Re: Problem with building httpd without /dev/random, link of htpasswd fails: ERROR: Undefined symbol: .apr_generate_random_bytes
On 10/12/2010 10:47 AM, Michael Felt wrote: I have the disable of auth_digest because configue complained that it could not complete without /dev/random or egb installed. I prefer not to install egb. Suggestions for how I can 'neatly' satisfy htpasswd need for apr_generate_random_bytes() are appreciated! You can't, you need entropy, and apr does not offer a pseudo-random generator of its own.
Re: Problem with building httpd without /dev/random, link of htpasswd fails: ERROR: Undefined symbol: .apr_generate_random_bytes
i wonder how i did this before on AIX before it had /dev/random. Ok. I'll look for egb then, or something similar. Thanks. On Tue, Oct 12, 2010 at 6:15 PM, William A. Rowe Jr. wr...@rowe-clan.netwrote: On 10/12/2010 10:47 AM, Michael Felt wrote: I have the disable of auth_digest because configue complained that it could not complete without /dev/random or egb installed. I prefer not to install egb. Suggestions for how I can 'neatly' satisfy htpasswd need for apr_generate_random_bytes() are appreciated! You can't, you need entropy, and apr does not offer a pseudo-random generator of its own.
Re: Problem with building httpd without /dev/random, link of htpasswd fails: ERROR: Undefined symbol: .apr_generate_random_bytes
- Michael Felt mamf...@gmail.com wrote: Error message during make all in support: ld: 0711-317 ERROR: Undefined symbol: .apr_generate_random_bytes .apr_generate_random_bytes [46] ER PR htpasswd.c(.libs/htpasswd.o) 001c .text R_RBR [521] .seed_rand Using gcc on AIX 4.3.3 with following configure command: According to: http://en.wikipedia.org/wiki/AIX#Versions AIX 4.3.3 is 11 years old now. ./configure \ --prefix=/usr/local/apache2 \ --enable-layout=Apache \ --enable-module=so \ --enable-mods-shared=all \ --disable-auth_digest \ --with-z=/usr/local I have the disable of auth_digest because configue complained that it could not complete without /dev/random or egb installed. I prefer not to install egb. Suggestions for how I can 'neatly' satisfy htpasswd need for apr_generate_random_bytes() are appreciated! HTTP 2.2.13 code btw. Many thanks. -- Igor Galić Tel: +43 (0) 664 886 22 883 Mail: i.ga...@brainsware.org URL: http://brainsware.org/
Re: Problem with building httpd without /dev/random, link of htpasswd fails: ERROR: Undefined symbol: .apr_generate_random_bytes
That's right, and still being used. Amazing how fast it boots, even on old hardware. Compiling, however, takes it's time :) Using gcc on AIX 4.3.3 with following configure command: According to: http://en.wikipedia.org/wiki/AIX#Versions AIX 4.3.3 is 11 years old now. -- Igor Galić Tel: +43 (0) 664 886 22 883 Mail: i.ga...@brainsware.org URL: http://brainsware.org/
Re: [PATCH] mod_cgi: Mitigating some header injections by dropping invalid headers?
On Tuesday 12 October 2010 18:13:46 William A. Rowe Jr. wrote:
On 10/12/2010 10:06 AM, Dirk-Willem van Gulik wrote:
On 12 Oct 2010, at 15:30, Malte S. Stretz wrote:
I had a quick look at the Apache source and the solution was simple:
Just drop headers which contain any character outside the range
[a-zA-Z0-9-]. The patch against trunk is attached.
This made me think of something we had a while ago; and after
checking the logs - big +1 from me!
Agreed, with a caviat... we aught to be able to toggle this for the
rare but significant legacy app that requires it... which implies a
per-dir flag that can override just one CGI script out of an entire
server.
I think an option is not needed as there is a workaround. Eg. to make an
Accept_Encoding header work:
SetEnvIfNoCase ^Accept.Encoding$ ^(.*)$ fix_header=$1
RequestHeader set Accept-Encoding %{fix_header}e env=fix_header
(I had to use a regexp in SetEnvIf since for some reason comparing to an
invalid header doesn't work.)
Cheers,
Malte
Re: logging patch bug
On Monday 11 October 2010, William A. Rowe Jr. wrote: Why were ap_errorlog_format_item-min_loglevel and ap_errorlog_info-level declared with two different types? ap_errorlog_info-level is -1 if no log level is available, i.e. when logging per-request/per-conn info. However, ap_errorlog_format_item- min_loglevel always contains a genuine loglevel (which is =0). Therefore I have declared the former as signed int and the latter as unsigned int. But the comparison (item-min_loglevel info-level) in log.c does cause a warning with -Wextra, even if it is correct. What do you suggest? Adding a cast or changing the declaration of min_loglevel to signed int?
Re: svn commit: r1000814 - in /httpd/httpd/trunk: CHANGES server/mpm/event/event.c
On Fri, Sep 24, 2010 at 7:25 AM, traw...@apache.org wrote: Author: trawick Date: Fri Sep 24 11:25:25 2010 New Revision: 1000814 URL: http://svn.apache.org/viewvc?rev=1000814view=rev Log: Fix crash accessing pollset on worker thread when child process is exiting. The timeout mutex and pollset were allocated from the listener thread pool. During child process shutdown, the listener thread exits first while any outstanding requests finish. These objects need to be allocated from pchild since the lifetime extends until the last worker thread has finished. Switch to pchild, and move init of these objects to the same place as other thread-independent objects. Someone mentioned using 2.2 event in production on the list,today or yesterday, so I peeked at 2.2 and this bug appears to affect it. Any interest out there in seeing what it takes to backport? (away from working env until tonight; dunno when wrowe is going to TR; dunno if anyone cares)
Re: svn commit: r1021924 - in /httpd/httpd/trunk: ./ docs/conf/extra/ docs/manual/ docs/manual/misc/ docs/manual/mod/ docs/manual/platform/ server/ server/mpm/event/ server/mpm/winnt/ server/mpm/worke
On 10/12/2010 4:12 PM, s...@apache.org wrote: URL: http://svn.apache.org/viewvc?rev=1021924view=rev Log: Rename MaxRequestsPerChild to MaxConnectionsPerChild, which describes more accurately what the directive does. Doesn't this actually mean MaxConnectionsPerWorker?
Re: svn commit: r1021924 - in /httpd/httpd/trunk: ./ docs/conf/extra/ docs/manual/ docs/manual/misc/ docs/manual/mod/ docs/manual/platform/ server/ server/mpm/event/ server/mpm/winnt/ server/mpm/worke
On 10/12/2010 4:55 PM, William A. Rowe Jr. wrote: On 10/12/2010 4:12 PM, s...@apache.org wrote: URL: http://svn.apache.org/viewvc?rev=1021924view=rev Log: Rename MaxRequestsPerChild to MaxConnectionsPerChild, which describes more accurately what the directive does. Doesn't this actually mean MaxConnectionsPerWorker? Whoops, I was thinking of MaxConnectionsPerProcess, which means something different than Child, depending on how you interpret it.
Re: svn commit: r1000814 - in /httpd/httpd/trunk: CHANGES server/mpm/event/event.c
On 10/12/2010 4:22 PM, Jeff Trawick wrote: Someone mentioned using 2.2 event in production on the list,today or yesterday, so I peeked at 2.2 and this bug appears to affect it. Any interest out there in seeing what it takes to backport? (away from working env until tonight; dunno when wrowe is going to TR; dunno if anyone cares) 2.0 docs/manual/ build.sh all (and build.bat all) is still giving me hassle, so let me know what you want to do with this.
