Re: Careful meets checking of PR#40169
> -Ursprüngliche Nachricht- > Von: Dirk-Willem van Gulik > Gesendet: Mittwoch, 7. Mai 2008 22:53 > An: dev@httpd.apache.org > Betreff: Careful meets checking of PR#40169 > > Any reason why we never applied > > https://issues.apache.org/bugzilla/show_bug.cgi?id=40169 > > It looks sensible to me - and would, as a side effect fix a simple > edge case. But I am not sure I get all its ramfications. I do not know which patch you mean exactly in this report. The ideas of https://issues.apache.org/bugzilla/attachment.cgi?id=18889 look good, but the coding style is very strange. So please post your version of the patch here or just commit it to trunk and I will do further comments. Regards Rüdiger
mod_mbox compile issue on SuSE Enterprise 10
Hello, I'm currently looking to deploy mod_mbox on a SuSE 10 server, and I'm having some issues getting the package to compile. I'm hoping that someone might be able to shed some light on the issue. mod_mbox_util-mod-mbox-util.o: In function `scan_dir': /root/mod_mbox/module-2.0/mod-mbox-util.c:162: undefined reference to `ap_strstr' /root/mod_mbox/module-2.0/mod-mbox-util.c:207: undefined reference to `ap_strchr' /root/mod_mbox/module-2.0/mod-mbox-util.c:217: undefined reference to `ap_strchr' /root/mod_mbox/module-2.0/mod-mbox-util.c:226: undefined reference to `ap_strrchr' ./.libs/libmboxutil.a(libmboxutil_la-mbox_parse.o): In function `mbox_parse_cte_header': /root/mod_mbox/module-2.0/mbox_parse.c:1190: undefined reference to `ap_strstr' /root/mod_mbox/module-2.0/mbox_parse.c:1201: undefined reference to `ap_strchr' /root/mod_mbox/module-2.0/mbox_parse.c:1204: undefined reference to `ap_strchr' /root/mod_mbox/module-2.0/mbox_parse.c:1207: undefined reference to `ap_strchr' /root/mod_mbox/module-2.0/mbox_parse.c:1191: undefined reference to `ap_strchr' /root/mod_mbox/module-2.0/mbox_parse.c:1194: undefined reference to `ap_strchr' ./.libs/libmboxutil.a(libmboxutil_la-mbox_parse.o):/root/mod_mbox/module-2.0/mbox_parse.c:1197: more undefined re ferences to `ap_strchr' follow ./.libs/libmboxutil.a(libmboxutil_la-mbox_parse.o): In function `parse_from': /root/mod_mbox/module-2.0/mbox_parse.c:470: undefined reference to `ap_strrchr' /root/mod_mbox/module-2.0/mbox_parse.c:462: undefined reference to `ap_strchr' /root/mod_mbox/module-2.0/mbox_parse.c:477: undefined reference to `ap_strchr' /root/mod_mbox/module-2.0/mbox_parse.c:481: undefined reference to `ap_strchr' ./.libs/libmboxutil.a(libmboxutil_la-mbox_parse.o): In function `mbox_fillbuf': /root/mod_mbox/module-2.0/mbox_parse.c:103: undefined reference to `ap_strchr' ./.libs/libmboxutil.a(libmboxutil_la-mbox_parse.o): In function `mbox_bgets': /root/mod_mbox/module-2.0/mbox_parse.c:144: undefined reference to `ap_strchr' ./.libs/libmboxutil.a(libmboxutil_la-mbox_parse.o):/root/mod_mbox/module-2.0/mbox_parse.c:152: more undefined ref erences to `ap_strchr' follow ./.libs/libmboxutil.a(libmboxutil_la-mbox_parse.o): In function `mbox_generate_index': /root/mod_mbox/module-2.0/mbox_parse.c:855: undefined reference to `ap_strstr' /root/mod_mbox/module-2.0/mbox_parse.c:860: undefined reference to `ap_strstr' /root/mod_mbox/module-2.0/mbox_parse.c:850: undefined reference to `ap_strstr' collect2: ld returned 1 exit status make[2]: *** [mod-mbox-util] Error 1 make[2]: Leaving directory `/root/mod_mbox/module-2.0' make[1]: *** [all] Error 2 make[1]: Leaving directory `/root/mod_mbox/module-2.0' make: *** [all-recursive] Error 1 Those functions appear to be defined within httpd.h of the apache2-devel package, which is installed. The system is running the suse version of Apache 2.2.3. I previously compiled mod_mbox on a Debian Etch system with the same apache version with no issues. Do you have any suggestions for things to check for? -- Greg
Re: High security
hi all, sorry to enter so later in the discution... i'm ok with you torsten (+1), i think is the good place to do it, think adding hook like pre_childinit that occure before unixd_setup_child remove the root rigth (in child_init you lost the root right juste before) can be more generic way. and after use this hook to do implemente the chroot option as loadable module. and i think can be usefull for other module to have chance to use root right juste before is removed in child_init... existe other implementation of chroot in apache and i think many interesting hints to use chroot in apache are in the page of mod_chroot: http://core.segfault.pl/~hobbit/mod_chroot/caveats.html i think is interesting to add some of this in the documentation also... and loadfile of libgcc_s.so.1 is only needed in threaded mpm version. because the thread lib use some symbole of libgcc_s.so.1 that are autoloaded in child when create thread after unixd_setup_child occure... i've made modified version (at 02/08) of mod_chroot that work like your modification (chrooting juste before seting up setuid in the child) but without modifying apache code and working with ap 2.0 and 2.2... it work like that, i save the unixd_config.user_id in pre_mpm hook, and force it to 0 (root), in that way in child_init root your are root (unixd_setup_child skip changing user_id)! like i can do the chroot in child_init hook, and restore unixd_config.user_id and redo unixd_setup_child to remove root right. i think an other modification must be done to be complete... is to modify the way DocumentRoot global context cmd check if the path is directory or not, will be very good, to check document root path in chroot directory relative in place of server_root if chroot option is activated. without that modification you must have DocumentRoot in the chroot path and outside the chroot path like that : if chroot dir is /var/chroot and globale document root to /var/chroot/www in httpd.conf your globale document root point to /var/chroot/www/var/chroot/www. with that modification you can set globale document root in the jail path without problem... a work around exist... using documentroot in virtualhost... in vhost you can use path in the jail without problem because documentroot cmd in the jail are executed when vhost are checked... far later from the point the chroot occure (juste before unixd_setup_child)... in that way can be possible to set globale DocumentRoot to same dir of chroot, and use only vhost to specify Document root directory... but is not completely transparent because you must set a fake globale document root that exist ouside the jail and you are obliged to use vhost... in my mod_chroot modification in correcte all path transparantly on the fly (in translate hook, and map_to_storage hook) to work around this (to do it without modification in apache code!)...but is more havy modification... and more risky... Regards, Mathieu *From:* Torsten Foertsch *Sent:* Sat, 26 Jan 2008 08:44:17 +0100 On Fri 25 Jan 2008, Nick Kew wrote: > > A > > compromise might be to create a chroot hook and allow module > > developers to use it. This would shift the support burden somewhat > > from the core Apache team to those willing to engage the users > > providing support. > > Isn't that basically the status quo (mod_security presumably hooks it > in at post_config?) Sometimes I have missed a ChildPrivilegedInit hook that is run between fork() and dropping privileges in the worker. That would be the right place to chroot() I think. Torsten
Careful meets checking of PR#40169
Any reason why we never applied https://issues.apache.org/bugzilla/show_bug.cgi?id=40169 It looks sensible to me - and would, as a side effect fix a simple edge case. But I am not sure I get all its ramfications. Thanks, Dw.
jumbo patch from 39380 - and moving things 'up' to mod_cache itself
Niklas, There is a lot of valuable stuff in your jumbo patch - but I am not sure what the best approach is to fold it in. Could you have have a look at the rough patch I posted earlier today - and let me know if you have any thoughts as to which parts should be moved 'up' -- and hence be of use to other caching backends as well - and which parts are pure disk optimized/specific ? Or perhaps form an option if we need multiple disk cache modules - each optimized differently (e.g. for large files on multiple spindles; versus very 'hot' cache which is virtually living on a meory disk). Thanks, Dw
Re: very rough caching change patch - centralize vary/http knowledge
On 05/07/2008 06:25 PM, Dirk-Willem van Gulik wrote: Apologies for the size - but about as minimal as I can get it :) So the problem I see is: -current caching modules should understand things such as Vary and negotiation. And we're bound to get more. -currently only mod_disk_cache does so. There are some 6 or 7 other modules which ought to get this capability too. -we can probably improve the current vary and header understanding to get better caching. Now we could go all out and reduce the mod_disk/mem/distcache/memcached/et.al. modules to a pure get/set/put API (e.g. see the API's of distcached and memcached at http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt). But to me allowing caching modules some knowledge of HTTP is probably good - as optimizing the generic case is simply not worthwhile - the very point of caches is that they understand something of the biz-processes to go beyond what a the operating system and what not can yield. So my suggestion is to: -Fundamentally expect modules to understand Vary. -Fundamentally assume that HTTP headers and similar caching info is serializable in a few k's to 10's of k's. -But strip everything out but for the get header data for some Key deserialize something.. if (vary_key returned) get header data for vary key deserialize something .. and then work on the body I am not quite sure if every cache module needs to know this logic. What about a very thin vary layer that can be moved between the caching client user and the cache provider itself. Some sort of a meta provider. I think there is also reasonable use for caches in the code that follow the pure key/value get/set/put/del as depending on their implementation these caches can deliver services that the OS maybe cannot out of the box (like distcache for a common cache across a cluster) or simply ease IPC a lot. -Move all VARY trickery into cache_util et.al. I agree to move more of the VARY trickery to common cache code like cache_util to avoid code duplications and reinvention of the wheel in various cache modules. As opposed to going the pure key/value get/set/put/del route and layering something on top of that. Thoughts - below is some very rough yet functioningish code. I haven't had a very close look yet, but the principal ideas sound good. Regards Rüdiger
Re: very rough caching change patch - centralize vary/http knowledge
On May 7, 2008, at 7:01 PM, Roy T. Fielding wrote: On May 7, 2008, at 9:25 AM, Dirk-Willem van Gulik wrote: But to me allowing caching modules some knowledge of HTTP is probably good - as optimizing the generic case is simply not worthwhile - the very point of caches is that they understand something of the biz-processes to go beyond what a the operating system and what not can yield. Please keep in mind that Vary is an instruction from the origin server to external caches that it does not control. A cache within an origin server or reverse proxy/gateway is free to ignore Vary if so configured. Thus, an internal cache may have two sets of instructions -- one "internal vary" that guides its operation and another Vary header field that is intended for external recipients of a message. Which is why I intentionally stopped short of hiding this completely in the cache_utils - but instead let the caching modules 'choose' if they follow up, change, have their own and/or use (or totally ignore) vary. Or am I misreading you - and are you suggesting that we should have an extra layer in there - i.e. bury the vary aspect for the storage part of the cache - and have a separate layer for the 'is this equal' logic (rather than having them bundled in one, as it is now) ? Dw
Re: sendfile in darwin
On May 7, 2008, at 9:31 AM, Jim Jagielski wrote: On May 7, 2008, at 9:25 AM, Jim Jagielski wrote: On May 7, 2008, at 9:14 AM, Jim Jagielski wrote: Just a quick FYI: Simply using the FreeBSD logic flow for sendfile (ie: adding || defined(DARWIN) and pulling out the existing DARWIN-specific section) clears up the problems I saw with the getfile tests... So I guess I need to dig deeper on this... Hold on... this may not be correct... double checking Yeah, ignore the above for now... Squashed... Spent a LOT of time on this (even ported the Linux impl to Darwin, which actually is nicer since it uses apr_socket_sendv() and does things like checking partial header/footer stuff, which I may eventually commit to trunk) and the fix was simple...
Re: svn commit: r647263 - in /httpd/httpd/trunk: ./ docs/manual/mod/ include/ modules/aaa/ modules/filters/ modules/http/ server/
On May 2, 2008, at 11:28 AM, Roy T. Fielding wrote: Hah, I was reading it like an equation from my discrete math days. I guess ^ really is less readable. ;-) low & hi would be my preference. Er, duh, (hi | low) would be my preference. Or just leave it as is. Roy
Re: very rough caching change patch - centralize vary/http knowledge
On May 7, 2008, at 9:25 AM, Dirk-Willem van Gulik wrote: But to me allowing caching modules some knowledge of HTTP is probably good - as optimizing the generic case is simply not worthwhile - the very point of caches is that they understand something of the biz-processes to go beyond what a the operating system and what not can yield. Please keep in mind that Vary is an instruction from the origin server to external caches that it does not control. A cache within an origin server or reverse proxy/gateway is free to ignore Vary if so configured. Thus, an internal cache may have two sets of instructions -- one "internal vary" that guides its operation and another Vary header field that is intended for external recipients of a message. Roy
very rough caching change patch - centralize vary/http knowledge
Apologies for the size - but about as minimal as I can get it :)
So the problem I see is:
- current caching modules should understand things such as
Vary and negotiation. And we're bound to get more.
- currently only mod_disk_cache does so. There are some 6 or 7
other modules which ought to get this capability too.
- we can probably improve the current vary and header
understanding to get better caching.
Now we could go all out and reduce the mod_disk/mem/distcache/
memcached/et.al.
modules to a pure get/set/put API (e.g. see the API's of distcached and
memcached at http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt)
.
But to me allowing caching modules some knowledge of HTTP is probably
good - as
optimizing the generic case is simply not worthwhile - the very point
of caches
is that they understand something of the biz-processes to go beyond
what a the
operating system and what not can yield.
So my suggestion is to:
- Fundamentally expect modules to understand Vary.
- Fundamentally assume that HTTP headers and similar
caching info is serializable in a few k's to 10's
of k's.
- But strip everything out but for the
get header data for some Key
deserialize something..
if (vary_key returned)
get header data for vary key
deserialize something
.. and then work on the body
- Move all VARY trickery into cache_util et.al.
As opposed to going the pure key/value get/set/put/del route and
layering something on top of that.
Thoughts - below is some very rough yet functioningish code.
Dw.
Index: mod_cache.h
===
--- mod_cache.h (revision 651547)
+++ mod_cache.h (working copy)
@@ -311,6 +311,74 @@
apr_table_t
*t,
server_rec
*s);
+/* Serialize a table into a bucked brigade. Assumes that an already
+ * created bucked brigade. Passing an empty table (i.e. null elements)
+ * or a null pointer is treated equally (and the deserialization
+ * will not notice the difference). The table is assumed to contain
+ * textual key and value pairs ('\0' are used as termination tokens).
+ */
+CACHE_DECLARE(apr_status_t)
+ap_brigade_puttable(apr_bucket_brigade * bb, apr_table_t * table);
+
+/* Serialize a table into a bucked brigade. Assumes that an already
+ * created bucked brigade. Passing an empty array (i.e. null elements)
+ * or a null pointer is treated equally (and the deserialization
+ * will not notice the difference). The array is assumed to contain
+ * textual value's ('\0' is used as termination tokens).
+ */
+CACHE_DECLARE(apr_status_t)
+ap_brigade_putarray(apr_bucket_brigade * bb, apr_array_header_t * a);
+
+/* Deserialize a char buffer into an table. Or return a table with
+ * zero element when empty (rather than a null pointer). A table is
+ * created when none is passed; any key value pairs are 'add'-ed
+ * to (any existing) table.
+ */
+CACHE_DECLARE(apr_size_t)
+ap_deserialize_table(apr_pool_t * pool, apr_table_t ** tablep, char
*in, apr_size_t at, apr_size_t len);
+
+/* Deserialize a char buffer into an array. Or return an arrya with
+ * zero elements when empty (rather than a null pointer). An array is
+ * created when none is passed; any value pairs are 'push'-ed to
+ * the end of (any existing) table.
+ */
+CACHE_DECLARE(apr_size_t)
+ap_deserialize_array(apr_pool_t * pool, apr_array_header_t ** arr,
char *in, apr_size_t at, apr_size_t len);
+
+CACHE_DECLARE(apr_status_t)ap_serialize_cache_object(
+apr_pool_t * pool, apr_bucket_alloc_t * bucket_alloc,
+const char * key,
+
+apr_bucket_brigade ** serialized_bb,
+
+const char ** vary_key,
+apr_bucket_brigade ** serialized_vary_bb,
+
+cache_info *info,
+
+apr_table_t * headers_in,
+apr_table_t * headers_out,
+
+void * private_block,
+apr_size_t private_len
+);
+CACHE_DECLARE(apr_status_t)ap_deserialize_cache_object(
+apr_pool_t * pool,
+const char * key,
+const char ** vary_key,
+
+char * buffp, apr_size_t len,
+
+char ** urip,
+
+cache_info * info,
+apr_table_t ** headers_inp,
+apr_table_t ** headers_outp,
+
+void ** private_blockp,
+apr_size_t * private_lenp
+);
+
/**
* cache_storage.c
*/
Index: cache_util.c
===
--- cache_util.c(revision 651547)
+++ cache_util.c(working copy)
@@ -20,6 +20,42 @@
/* -- */
+#ifndef AP_CACHE_SERIAL_VERSION
+#define AP_CACHE_SERIAL_VERSION (1)
+#endif
+
+/* Trick the version a littl
Re: sendfile in darwin
On May 7, 2008, at 9:25 AM, Jim Jagielski wrote: On May 7, 2008, at 9:14 AM, Jim Jagielski wrote: Just a quick FYI: Simply using the FreeBSD logic flow for sendfile (ie: adding || defined(DARWIN) and pulling out the existing DARWIN-specific section) clears up the problems I saw with the getfile tests... So I guess I need to dig deeper on this... Hold on... this may not be correct... double checking Yeah, ignore the above for now...
Re: sendfile in darwin
On May 7, 2008, at 9:14 AM, Jim Jagielski wrote: Just a quick FYI: Simply using the FreeBSD logic flow for sendfile (ie: adding || defined(DARWIN) and pulling out the existing DARWIN-specific section) clears up the problems I saw with the getfile tests... So I guess I need to dig deeper on this... Hold on... this may not be correct... double checking
Re: sendfile in darwin
Just a quick FYI: Simply using the FreeBSD logic flow for sendfile (ie: adding || defined(DARWIN) and pulling out the existing DARWIN-specific section) clears up the problems I saw with the getfile tests... So I guess I need to dig deeper on this... BTW: This is on OS X 10.5.2 (Darwin 9.2.2) On May 6, 2008, at 4:20 PM, Jim Jagielski wrote: I should have some cycles to dig deeper on this tomorrow...
Re: 2.2.9
> -Ursprüngliche Nachricht- > Von: Jim Jagielski > Gesendet: Mittwoch, 7. Mai 2008 14:26 > An: dev@httpd.apache.org > Betreff: Re: 2.2.9 > > > On May 7, 2008, at 6:51 AM, Niklas Edmundsson wrote: > > > On Tue, 6 May 2008, Nick Kew wrote: > > > >>>2. Consensus on whether we ship with APR 1.2.x or 1.3.x... > >>> My pref would be 1.3. > >> > >> -1. > > > > I tend to agree with NOT shipping 1.3. > > > > This goes w/o saying, but I'll do so anyway. > > If 1.3.0 is not ready for prime time, then (1) we assume that > APR will not tag and release it and (2) we do not bundle httpd 2.2.9 > with it. This does not mean that later versions of 2.2.x won't > ship with it. > > With our httpd hat on, we do what's best for httpd, even if that > means we don't "push" APR ;) Keep in mind that if we do not bundle 1.3 we need to do a new release of 1.2.x as it contains a regression. Regards Rüdiger
Re: 2.2.9
On May 7, 2008, at 6:51 AM, Niklas Edmundsson wrote: On Tue, 6 May 2008, Nick Kew wrote: 2. Consensus on whether we ship with APR 1.2.x or 1.3.x... My pref would be 1.3. -1. I tend to agree with NOT shipping 1.3. This goes w/o saying, but I'll do so anyway. If 1.3.0 is not ready for prime time, then (1) we assume that APR will not tag and release it and (2) we do not bundle httpd 2.2.9 with it. This does not mean that later versions of 2.2.x won't ship with it. With our httpd hat on, we do what's best for httpd, even if that means we don't "push" APR ;)
Re: 2.2.9
On Tue, 6 May 2008, Nick Kew wrote:
2. Consensus on whether we ship with APR 1.2.x or 1.3.x...
My pref would be 1.3.
-1.
I tend to agree with NOT shipping 1.3.
My issue is mainly that the stable branch should be just that - stable
and boring. Fix bugs, add new features that doesn't conflict with
existing ones, but try the damnest to avoid changes that might mess
upp things more than needed.
If we want to be adventurous I'd prefer wrapping up what's in httpd
trunk as 2.4 and ship that with 1.3 so we have a separate bucket of
worms to fight with.
That said, we have the issue with people thinking that all major
releases will be perpetually supported and thus avoids upgrading to a
new major version. I recall this was discussed rather recently, but my
personal view is that 1.3 and 2.0 really should be in
securityfix-only-mode by now.
Looking at 2.0 and 2.2 I think that there tends to be too much that
changes in those stable releases at times, getting us in the not so
funny "every release is broken somehow" cycle...
/Nikke - opening another can of worms...
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | [EMAIL PROTECTED]
---
"I love unicorns. What, dad? Girls are crazy about unicorns!" - Veronica
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Re: High security
On May 7, 2008, at 11:00 AM, Nick Gearls wrote: I propose to add the following: In the usage: All config files, logs, etc. are used by the main process and should thus not be stored in the chroot. Only files used by children listeners must be present in the chroot. Content of the chroot The following files must be present in the chroot: /lib/libgcc_s.so.1 (Linux) if bind (DNS) is used: /etc/resolv.conf & /lib/ libnss_dns.so.2 (Linux) if a hosts file is used: /etc/hosts if both a hosts file and bind (DNS) are used: /etc/ hosts.conf HTML files (htdocs/ files) Temporary files used by modules (ex: ModSecurity temp files) When using additional modules, other files may be needed Remark: shared object can also be loaded explicitely in httpd.conf, instead of copying them into the chroot. When using Apache as a reverse proxy, the chroot could thus potentially be totally empty. I was sort of hoping for a separate how-to page; with the exact 'chmod/ own' settings, groups you need to create, information about the log file locations/ ownership, the ownership of the cache directories and so on. Dw,.
Re: High security
I propose to add the following: In the usage: All config files, logs, etc. are used by the main process and should thus not be stored in the chroot. Only files used by children listeners must be present in the chroot. Content of the chroot The following files must be present in the chroot: /lib/libgcc_s.so.1 (Linux) if bind (DNS) is used: /etc/resolv.conf & /lib/libnss_dns.so.2 (Linux) if a hosts file is used: /etc/hosts if both a hosts file and bind (DNS) are used: /etc/hosts.conf HTML files (htdocs/ files) Temporary files used by modules (ex: ModSecurity temp files) When using additional modules, other files may be needed Remark: shared object can also be loaded explicitely in httpd.conf, instead of copying them into the chroot. When using Apache as a reverse proxy, the chroot could thus potentially be totally empty. Regards, Nick Dirk-Willem van Gulik wrote: On May 6, 2008, at 5:03 PM, Plüm, Rüdiger, VF-Group wrote: -Ursprüngliche Nachricht- Von: Dirk-Willem van Gulik Gesendet: Dienstag, 6. Mai 2008 17:00 An: dev@httpd.apache.org Betreff: Re: High security On May 6, 2008, at 4:12 PM, Nick Gearls wrote: If there's a chance to add it, I'm ready to write the doc patch I did below a while ago. May be useful as a start. There is already a documentation in trunk for this: http://svn.apache.org/viewvc?view=rev&revision=639005 Aye - I edited on top of that version. Dw.
