Re: Solaris, prefork, accept mutex and mod_ext_filter (Was: Prefork MPM and mod_watchdog)
Am 30.03.2020 um 11:28 schrieb Joe Orton: On Mon, Mar 30, 2020 at 12:31:05AM +0200, Rainer Jung wrote: I can now see the same problem on Linux (eg. RHEL 7, SLES 12 and SLES 15) when doing testing for 2.4.43. I think it is not a regression and for me it is not a showstopper, but something that would be nice to fix. Symptom is [Sat Mar 28 15:53:21.121461 2020] [mpm_prefork:debug] [pid 4574] prefork.c(919): AH00165: Accept mutex: pthread (default: pthread) [Sat Mar 28 15:58:19.251858 2020] [mpm_prefork:emerg] [pid 6509] (22)Invalid argument: AH00144: couldn't grab the accept mutex [Sat Mar 28 15:58:20.624995 2020] [mpm_prefork:emerg] [pid 4902] (22)Invalid argument: AH00146: couldn't release the accept mutex [Sat Mar 28 15:58:21.410552 2020] [:emerg] [pid 4574] AH02818: MPM run failed, exiting happening during t/modules/ext_filter.t and as a result all httpd processes are gone. Although it happens only rarely, I think it does not happen when using APR 1.6, but only for 1.7 (1.7.0 and also for latest head). The accept mutex is pthread based. There are changes in APR 1.7 for those. This is interesting, I saw some similar mutex state snafu (Fedora 31) but assumed it was some kind of PEBKAC issue and moved on. You have only seen this with prefork? Yes, only with prefork. If the root cause would also apply for multi-threaded MPMs, then the escalation might be different, ie. not leading to all child processes exiting. I can not rule out this. May be completely unrelated but since I added a grep for child segfaults in error_log that has triggered with prefork at least once: https://travis-ci.org/github/apache/httpd/jobs/665874906 That looks more like my rarely observed prefork shutdown crashes when mod_watchdog shuts down after some other deinitialization that happened before. Regards, Rainer Regards, Joe Since I only observe it for mod_ext_filter, there should be some dependency with the forked perl process. I didn't yet have the opportunity to check, how much of the follwing description for Solaris also holds for Linux. Regards, Rainer Am 03.02.2019 um 13:30 schrieb Rainer Jung: I can now frequently reproduce running t/modules/ext_filter.t only. I stripped the reproducer down to the part of t/modules/ext_filter.t where it runs POST "/apache/extfilter/out-limit/modules/cgi/perl_echo.pl", content => "foo and bar\n" 10 times in a row. If I increase the iteration count slightly to 20, the problem happens nearly every time. I also replaced perl_echo.pl and eval-cmd.pl by small C programs doing the echo and the s/foo/bar/ and can still reproduce. This test involved mod_ext_filter and LimitRequestBody. It seems I can not reproduce, if I shorten the POST body, so that it no longer hits the LimitRequestBody 6 configured for /apache/extfilter/out-limit/. What happens, but I do not understand is: - the first few requests are handled by two children in an alternating way. I can see the accept mutex being passed between these two children using lwp_mutex_timedlock(ADDRESS, 0x) and lwp_mutex_unlock(ADDRESS) using truss (Solaris variant of strace). So Solaris seems to implement the pthread mutexes via these lwp mutexes. - after a few requests alternating between the two children, something strange happens: - child A returns from port_getn() for 22 (probably part of the accept() impl) - child A calls accept() - child A unlocks the accept mutex using lwp_mutex_unlock() - child B locks the accept mutex using lwp_mutex_timedlock() - child B calls port_associate for 22 (probably part of accept() impl) - child A handles the request ! - child A also calls port_associate for 22 (no sign of lock acquire) ! - child A returns from port_getn() for 22 - child A calls accept() and starts to handle the connection ! - child B also returns from port_getn() for 22 ! - child B gets EAGAIN for the accept() - child B calls port_associate for 22 - child A handles the request ! - child A gets EDEADLK for pthread_mutex_lock() (this is from the error_log; there's no system call for this instance of pthread_mutex_lock() in the truss output). EDEADLK for pthread_mutex_lock() means that this process already has that lock. - child B returns from port_getn() for 22 - child B calls accept() and starts to handle the connection - child A exits (now B is the only child left) - child B proceeds request handling - child B does all further port_associate(), port_getn(), and accept() plus connection and request handling. No more lwp_mutex_timedlock() or lwp_mutex_unlock() for B, maybe due to some optimization when only one process for a lock is left. It is quite possible, that there is an underlying lwp_mutex or portfs bug here. But it is strange, that this only comes up when used with mod_ext_filter in combination with LimitRequestBody. There was the fix https://bz.apache.org/bugzilla/show_bug.cgi?id=60375 bu
Re: Solaris, prefork, accept mutex and mod_ext_filter (Was: Prefork MPM and mod_watchdog)
On Mon, Mar 30, 2020 at 12:31:05AM +0200, Rainer Jung wrote: > I can now see the same problem on Linux (eg. RHEL 7, SLES 12 and SLES 15) > when doing testing for 2.4.43. I think it is not a regression and for me it > is not a showstopper, but something that would be nice to fix. Symptom is > > [Sat Mar 28 15:53:21.121461 2020] [mpm_prefork:debug] [pid 4574] > prefork.c(919): AH00165: Accept mutex: pthread (default: pthread) > [Sat Mar 28 15:58:19.251858 2020] [mpm_prefork:emerg] [pid 6509] (22)Invalid > argument: AH00144: couldn't grab the accept mutex > [Sat Mar 28 15:58:20.624995 2020] [mpm_prefork:emerg] [pid 4902] (22)Invalid > argument: AH00146: couldn't release the accept mutex > [Sat Mar 28 15:58:21.410552 2020] [:emerg] [pid 4574] AH02818: MPM run > failed, exiting > > happening during t/modules/ext_filter.t and as a result all httpd processes > are gone. > > Although it happens only rarely, I think it does not happen when using APR > 1.6, but only for 1.7 (1.7.0 and also for latest head). The accept mutex is > pthread based. There are changes in APR 1.7 for those. This is interesting, I saw some similar mutex state snafu (Fedora 31) but assumed it was some kind of PEBKAC issue and moved on. You have only seen this with prefork? May be completely unrelated but since I added a grep for child segfaults in error_log that has triggered with prefork at least once: https://travis-ci.org/github/apache/httpd/jobs/665874906 Regards, Joe > Since I only observe it for mod_ext_filter, there should be some dependency > with the forked perl process. > > I didn't yet have the opportunity to check, how much of the follwing > description for Solaris also holds for Linux. > > Regards, > > Rainer > > Am 03.02.2019 um 13:30 schrieb Rainer Jung: > > I can now frequently reproduce running t/modules/ext_filter.t only. I > > stripped the reproducer down to the part of t/modules/ext_filter.t where > > it runs > > > > POST "/apache/extfilter/out-limit/modules/cgi/perl_echo.pl", content => > > "foo and bar\n" > > > > 10 times in a row. If I increase the iteration count slightly to 20, the > > problem happens nearly every time. I also replaced perl_echo.pl and > > eval-cmd.pl by small C programs doing the echo and the s/foo/bar/ and > > can still reproduce. > > > > This test involved mod_ext_filter and LimitRequestBody. > > > > It seems I can not reproduce, if I shorten the POST body, so that it no > > longer hits the LimitRequestBody 6 configured for > > /apache/extfilter/out-limit/. > > > > What happens, but I do not understand is: > > > > - the first few requests are handled by two children in an alternating > > way. I can see the accept mutex being passed between these two children > > using lwp_mutex_timedlock(ADDRESS, 0x) and > > lwp_mutex_unlock(ADDRESS) using truss (Solaris variant of strace). So > > Solaris seems to implement the pthread mutexes via these lwp mutexes. > > > > - after a few requests alternating between the two children, something > > strange happens: > > > > - child A returns from port_getn() for 22 (probably part of the > > accept() impl) > > - child A calls accept() > > - child A unlocks the accept mutex using lwp_mutex_unlock() > > - child B locks the accept mutex using lwp_mutex_timedlock() > > - child B calls port_associate for 22 (probably part of accept() impl) > > - child A handles the request > > ! - child A also calls port_associate for 22 (no sign of lock acquire) > > ! - child A returns from port_getn() for 22 > > - child A calls accept() and starts to handle the connection > > ! - child B also returns from port_getn() for 22 > > ! - child B gets EAGAIN for the accept() > > - child B calls port_associate for 22 > > - child A handles the request > > ! - child A gets EDEADLK for pthread_mutex_lock() (this is from the > > error_log; there's no system call for this instance of > > pthread_mutex_lock() in the truss output). EDEADLK for > > pthread_mutex_lock() means that this process already has that lock. > > - child B returns from port_getn() for 22 > > - child B calls accept() and starts to handle the connection > > - child A exits (now B is the only child left) > > - child B proceeds request handling > > - child B does all further port_associate(), port_getn(), and > > accept() plus connection and request handling. No more > > lwp_mutex_timedlock() or lwp_mutex_unlock() for B, maybe due to some > > optimization when only one process for a lock is left. > >
Re: Solaris, prefork, accept mutex and mod_ext_filter (Was: Prefork MPM and mod_watchdog)
I can now see the same problem on Linux (eg. RHEL 7, SLES 12 and SLES 15) when doing testing for 2.4.43. I think it is not a regression and for me it is not a showstopper, but something that would be nice to fix. Symptom is [Sat Mar 28 15:53:21.121461 2020] [mpm_prefork:debug] [pid 4574] prefork.c(919): AH00165: Accept mutex: pthread (default: pthread) [Sat Mar 28 15:58:19.251858 2020] [mpm_prefork:emerg] [pid 6509] (22)Invalid argument: AH00144: couldn't grab the accept mutex [Sat Mar 28 15:58:20.624995 2020] [mpm_prefork:emerg] [pid 4902] (22)Invalid argument: AH00146: couldn't release the accept mutex [Sat Mar 28 15:58:21.410552 2020] [:emerg] [pid 4574] AH02818: MPM run failed, exiting happening during t/modules/ext_filter.t and as a result all httpd processes are gone. Although it happens only rarely, I think it does not happen when using APR 1.6, but only for 1.7 (1.7.0 and also for latest head). The accept mutex is pthread based. There are changes in APR 1.7 for those. Since I only observe it for mod_ext_filter, there should be some dependency with the forked perl process. I didn't yet have the opportunity to check, how much of the follwing description for Solaris also holds for Linux. Regards, Rainer Am 03.02.2019 um 13:30 schrieb Rainer Jung: I can now frequently reproduce running t/modules/ext_filter.t only. I stripped the reproducer down to the part of t/modules/ext_filter.t where it runs POST "/apache/extfilter/out-limit/modules/cgi/perl_echo.pl", content => "foo and bar\n" 10 times in a row. If I increase the iteration count slightly to 20, the problem happens nearly every time. I also replaced perl_echo.pl and eval-cmd.pl by small C programs doing the echo and the s/foo/bar/ and can still reproduce. This test involved mod_ext_filter and LimitRequestBody. It seems I can not reproduce, if I shorten the POST body, so that it no longer hits the LimitRequestBody 6 configured for /apache/extfilter/out-limit/. What happens, but I do not understand is: - the first few requests are handled by two children in an alternating way. I can see the accept mutex being passed between these two children using lwp_mutex_timedlock(ADDRESS, 0x) and lwp_mutex_unlock(ADDRESS) using truss (Solaris variant of strace). So Solaris seems to implement the pthread mutexes via these lwp mutexes. - after a few requests alternating between the two children, something strange happens: - child A returns from port_getn() for 22 (probably part of the accept() impl) - child A calls accept() - child A unlocks the accept mutex using lwp_mutex_unlock() - child B locks the accept mutex using lwp_mutex_timedlock() - child B calls port_associate for 22 (probably part of accept() impl) - child A handles the request ! - child A also calls port_associate for 22 (no sign of lock acquire) ! - child A returns from port_getn() for 22 - child A calls accept() and starts to handle the connection ! - child B also returns from port_getn() for 22 ! - child B gets EAGAIN for the accept() - child B calls port_associate for 22 - child A handles the request ! - child A gets EDEADLK for pthread_mutex_lock() (this is from the error_log; there's no system call for this instance of pthread_mutex_lock() in the truss output). EDEADLK for pthread_mutex_lock() means that this process already has that lock. - child B returns from port_getn() for 22 - child B calls accept() and starts to handle the connection - child A exits (now B is the only child left) - child B proceeds request handling - child B does all further port_associate(), port_getn(), and accept() plus connection and request handling. No more lwp_mutex_timedlock() or lwp_mutex_unlock() for B, maybe due to some optimization when only one process for a lock is left. It is quite possible, that there is an underlying lwp_mutex or portfs bug here. But it is strange, that this only comes up when used with mod_ext_filter in combination with LimitRequestBody. There was the fix https://bz.apache.org/bugzilla/show_bug.cgi?id=60375 but I don't see, how this would influence the above list. I can try to further narrow down (using static content to eliminate one forked child; using LimitRequestBody without mod_exit-filter etc.), but maybe someone already has an idea? Regards, Rainer
Solaris, prefork, accept mutex and mod_ext_filter (Was: Prefork MPM and mod_watchdog)
Am 31.01.2019 um 10:31 schrieb Stefan Eissing: Am 27.01.2019 um 14:40 schrieb Rainer Jung : - as soon as I enable mod_watchdog only (but not the above modules that would use it), the hangs start to happen every now and then. Hmm, that sounds strange. As I understood the code, none of the parent/child watchdogs would be active then. So, its child_init should not do anything either. But, if one of the proxy monitors runs against the server itself, could it connect against the child process it (the watchdog) is running on? mod_watchdog was a red herring. I can now frequently reproduce running t/modules/ext_filter.t only. I stripped the reproducer down to the part of t/modules/ext_filter.t where it runs POST "/apache/extfilter/out-limit/modules/cgi/perl_echo.pl", content => "foo and bar\n" 10 times in a row. If I increase the iteration count slightly to 20, the problem happens nearly every time. I also replaced perl_echo.pl and eval-cmd.pl by small C programs doing the echo and the s/foo/bar/ and can still reproduce. This test incolved mod_ext_filter and LimitRequestBody. It seems I can not reproduce, if I shorten the POST body, so that it no longer hits the LimitRequestBody 6 configured for /apache/extfilter/out-limit/. What happens, but I do not understand is: - the first few requests are handled by two children in an alternating way. I can see the accept mutex being passed between these two children using lwp_mutex_timedlock(ADDRESS, 0x) and lwp_mutex_unlock(ADDRESS) using truss (Solaris variant of strace). So Solaris seems to implement the pthread mutexes via these lwp mutexes. - after a few requests alternating between the two children, something strange happens: - child A returns from port_getn() for 22 (probably part of the accept() impl) - child A calls accept() - child A unlocks the accept mutex using lwp_mutex_unlock() - child B locks the accept mutex using lwp_mutex_timedlock() - child B calls port_associate for 22 (probably part of accept() impl) - child A handles the request ! - child A also calls port_associate for 22 (no sign of lock acquire) ! - child A returns from port_getn() for 22 - child A calls accept() and starts to handle the connection ! - child B also returns from port_getn() for 22 ! - child B gets EAGAIN for the accept() - child B calls port_associate for 22 - child A handles the request ! - child A gets EDEADLK for pthread_mutex_lock() (this is from the error_log; there's no system call for this instance of pthread_mutex_lock() in the truss output). EDEADLK for pthread_mutex_lock() means that this process already has that lock. - child B returns from port_getn() for 22 - child B calls accept() and starts to handle the connection - child A exits (now B is the only child left) - child B proceeds request handling - child B does all further port_associate(), port_getn(), and accept() plus connection and request handling. No more lwp_mutex_timedlock() or lwp_mutex_unlock() for B, maybe due to some optimization when only one process for a lock is left. It is quite possible, that there is an underlying lwp_mutex or portfs bug here. But it is strange, that this only comes up when used with mod_ext_filter in combination with LimitRequestBody. There was the fix https://bz.apache.org/bugzilla/show_bug.cgi?id=60375 but I don't see, how this would influence the above list. I can try to further narrow down (using static content to eliminate one forked child; using LimitRequestBody without mod_exit-filter etc.), but maybe someone already has an idea? Regards, Rainer
mod_ext_filter and HTTP status codes
Is there any way to have an external program (called via mod_ext_filter) tell the server to return a different status code (other than 200). Specifically, I want to be able to set a 500 error. Also, a "nice to have" would be a way to set other HTTP headers (like Last-Modified). Conceptually, I'm not even sure how that would work... maybe via the ext program's exit codes? Any ideas would be appreciated. TIA Craig
Re: mod_ext_filter and last-modified header
Quoting Geoffrey Young ([EMAIL PROTECTED]): > Craig Sebenik wrote: > > It looks like mod_ext_filter sets the last-modified http header based on > > the mod time of the *filter* file and not the actual file represented by > > the URL. > > > the Last-Modified HTTP header seems to be set based on the timestamp on > > "/web_home/filters/filter.pl" and NOT on "/web_home/htdocs/file.html" > > I would hope it would be the newer of both, since both contribute to the > freshness of the content, which is what is important. if you touch > file.html do you get the newer timestamp? my mistake. it looks like it works just the opposite of what i said. if i change the *code*, and NOT the file, then the last-modified is unchanged. i agree, it should be the newer of both. (fyi, the machines are in the pacific time zone.) > ll file.html -rw-rw-r-- 1 craighttp2898 Apr 13 19:03 file.html > ll ../../public/filters/filter.pl -rwxrwxr-x 1 craighttp 368 Apr 9 14:39 ../../public/filters/filter.pl* > HEAD server/form11/file.html 200 OK Cache-Control: max-age=0, public, must-revalidate Connection: close Date: Wed, 14 Apr 2004 02:07:53 GMT Accept-Ranges: bytes ETag: "13ec08a-b52-6312b4c0" Server: Apache/2.0.39 (Unix) Content-Length: 3178 Content-Type: text/html; charset=ISO-8859-1 Expires: Wed, 14 Apr 2004 02:07:53 GMT Last-Modified: Wed, 14 Apr 2004 02:03:39 GMT Client-Date: Wed, 14 Apr 2004 02:04:22 GMT Client-Response-Num: 1 > touch ../../public/filters/filter.pl > ll file.html -rw-rw-r-- 1 craighttp2898 Apr 13 19:03 file.html > ll ../../public/filters/filter.pl -rwxrwxr-x 1 craighttp 368 Apr 13 19:08 ../../public/filters/filter.pl* > HEAD server/form11/file.html 200 OK Cache-Control: max-age=0, public, must-revalidate Connection: close Date: Wed, 14 Apr 2004 02:08:28 GMT Accept-Ranges: bytes ETag: "13ec08a-b52-6312b4c0" Server: Apache/2.0.39 (Unix) Content-Length: 3178 Content-Type: text/html; charset=ISO-8859-1 Expires: Wed, 14 Apr 2004 02:08:28 GMT Last-Modified: Wed, 14 Apr 2004 02:03:39 GMT Client-Date: Wed, 14 Apr 2004 02:05:00 GMT Client-Response-Num: 1 > touch file.html > ll file.html -rw-rw-r-- 1 craighttp2898 Apr 13 19:15 file.html > ll ../../public/filters/filter.pl -rwxrwxr-x 1 craighttp 368 Apr 13 19:08 ../../public/filters/filter.pl* > HEAD craig-www/form11/file.html 200 OK Cache-Control: max-age=0, public, must-revalidate Connection: close Date: Wed, 14 Apr 2004 02:15:11 GMT Accept-Ranges: bytes ETag: "13ec08a-b52-8c057e80" Server: Apache/2.0.39 (Unix) Content-Length: 3178 Content-Type: text/html; charset=ISO-8859-1 Expires: Wed, 14 Apr 2004 02:15:11 GMT Last-Modified: Wed, 14 Apr 2004 02:15:06 GMT Client-Date: Wed, 14 Apr 2004 02:11:43 GMT Client-Response-Num: 1
Re: mod_ext_filter and last-modified header
Craig Sebenik wrote: > It looks like mod_ext_filter sets the last-modified http header based on > the mod time of the *filter* file and not the actual file represented by > the URL. > the Last-Modified HTTP header seems to be set based on the timestamp on > "/web_home/filters/filter.pl" and NOT on "/web_home/htdocs/file.html" I would hope it would be the newer of both, since both contribute to the freshness of the content, which is what is important. if you touch file.html do you get the newer timestamp? --Geoff
mod_ext_filter and last-modified header
It looks like mod_ext_filter sets the last-modified http header based on the mod time of the *filter* file and not the actual file represented by the URL. eg. the following filter is configured in httpd.conf ExtFilterDefine filter-name mode=output intype=text/html outtype=text/html \ cmd="/web_home/filters/filter.pl" SetOutputFilter filter-name [etc...] now, if you try to get the following url: http://server/file.html the Last-Modified HTTP header seems to be set based on the timestamp on "/web_home/filters/filter.pl" and NOT on "/web_home/htdocs/file.html" i was hoping that this would be an easy fix for someone a lot more familiar with the code. i do understand that this may change the current behavior of existing apps that use mod_ext_filter... so, comments are welcomed... tia! craig
Re: HTTP_REFERER in mod_ext_filter
Andrew Ho wrote: There is another function, ap_add_cgi_vars(), in server/util_script.c which sets REQUEST_URI, SCRIPT_NAME, and all the other CGI variables we are accustomed to. It looks like mod_ext_filter calls ap_add_cgi_vars() but not ap_add_common_vars(). see attached patch :) Index: modules/filters/mod_ext_filter.c === RCS file: /home/cvs/httpd-2.0/modules/filters/mod_ext_filter.c,v retrieving revision 1.6 diff -u -r1.6 mod_ext_filter.c --- modules/filters/mod_ext_filter.c1 Jul 2003 11:21:00 - 1.6 +++ modules/filters/mod_ext_filter.c11 Jul 2003 22:43:34 - @@ -490,6 +490,7 @@ * and QUERY_STRING_UNESCAPED */ ap_add_cgi_vars(f->r); +ap_add_common_vars(f->r); apr_table_setn(f->r->subprocess_env, "DOCUMENT_URI", f->r->uri); apr_table_setn(f->r->subprocess_env, "DOCUMENT_PATH_INFO", f->r->path_info); if (f->r->args) {
Re: HTTP_REFERER in mod_ext_filter
Hello, CS>Currently, mod_ext_filter doesn't seem to grab all of the environment CS>variables. One that's notably missing is HTTP_REFERER. But, I can't CS>figure out where this env variable is set. I searched thru the source CS>files (*.c and *.h) and the only reference I found was in CS>modules/ssl/ssl_engine_vars.c and modules/mappers/mod_rewrite.c. The various HTTP_* environment variables that CGI scripts see are part of the CGI specification; they correspond specifically to any HTTP request headers. These are set by ap_add_common_vars() in server/util_script.c. You didn't find HTTP_REFERER specifically, because ap_add_common_vars() just cycles through all request headers (except Authorization and Proxy-Authorization, for security reasons) and sets appropriately named environment variables. There is another function, ap_add_cgi_vars(), in server/util_script.c which sets REQUEST_URI, SCRIPT_NAME, and all the other CGI variables we are accustomed to. It looks like mod_ext_filter calls ap_add_cgi_vars() but not ap_add_common_vars(). Humbly, Andrew -- Andrew Ho http://www.tellme.com/ [EMAIL PROTECTED] Engineer1-800-555-TELL Voice 650-930-9062 Tellme Networks, Inc. Fax 650-930-9101 --
HTTP_REFERER in mod_ext_filter
Currently, mod_ext_filter doesn't seem to grab all of the environment variables. One that's notably missing is HTTP_REFERER. But, I can't figure out where this env variable is set. I searched thru the source files (*.c and *.h) and the only reference I found was in modules/ssl/ssl_engine_vars.c and modules/mappers/mod_rewrite.c. I was hoping someone could point me to how/where this env var is set. I am pretty new to the apace src, so I apologize if this is an "obvious" (aka stupid :> ) question. TIA Craig
Re: input filters with mod_ext_filter [patch]
Hi Jeff, I recognized now that there were still some deficiencies in the patch you applied. Where can I get the (your) current version of that file, so that I can send you the next patch. -- Of course with detailed description what I needed to modify. -Philipp Am Dienstag, 1. Juli 2003 13:27 schrieb Jeff Trawick: > finally committed, sorry for the delay, thanks again for your contribution! > > I made some minor formatting changes to your final patch as well as a > code change to an area you were concerned about. > > Philipp Reisner wrote: > > Am Montag, 23. Juni 2003 03:56 schrieb Jeff Trawick: > > > > BTW, Please have a close look to the part where I look at the > > "Content-Type" header. The old code examien r->content_type. I had to > > find an other way, one that is also possible for request filtering... > > r->content_type is set in cases where the Content-Type header in > r->headers_out is not accurate > > For input filters I left the checking for Content-Type alone, but for > output filters I changed it back to the way it worked previously. The > code is not as pretty as yours, but as ap_set_content_type() does not > bother to fill in r->headers_out, the difference is necessary. > > from your patch: > > -if (!f->r->content_type) { > > -ctx->noop = 1; > > -} > > -else { > > -const char *ctypes = f->r->content_type; > > -const char *ctype = ap_getword(f->r->pool, &ctypes, ';'); > > - > > -if (strcasecmp(ctx->filter->intype, ctype)) { > > -/* wrong IMT for us; don't mess with the output */ > > -ctx->noop = 1; > > -} > > -} > > +const char *ctypes = apr_table_get(ctx->filter->mode == > > INPUT_FILTER ? + > > f->r->headers_in : f->r->headers_out, + > > "Content-Type"); -- : Dipl-Ing Philipp Reisner Tel +43-1-8178292-50 : : LINBIT Information Technologies GmbH Fax +43-1-8178292-82 : : Schönbrunnerstr 244, 1120 Vienna, Austriahttp://www.linbit.com :
Re: input filters with mod_ext_filter [patch]
finally committed, sorry for the delay, thanks again for your contribution! I made some minor formatting changes to your final patch as well as a code change to an area you were concerned about. Philipp Reisner wrote: Am Montag, 23. Juni 2003 03:56 schrieb Jeff Trawick: BTW, Please have a close look to the part where I look at the "Content-Type" header. The old code examien r->content_type. I had to find an other way, one that is also possible for request filtering... r->content_type is set in cases where the Content-Type header in r->headers_out is not accurate For input filters I left the checking for Content-Type alone, but for output filters I changed it back to the way it worked previously. The code is not as pretty as yours, but as ap_set_content_type() does not bother to fill in r->headers_out, the difference is necessary. from your patch: -if (!f->r->content_type) { -ctx->noop = 1; -} -else { -const char *ctypes = f->r->content_type; -const char *ctype = ap_getword(f->r->pool, &ctypes, ';'); - -if (strcasecmp(ctx->filter->intype, ctype)) { -/* wrong IMT for us; don't mess with the output */ -ctx->noop = 1; -} -} +const char *ctypes = apr_table_get(ctx->filter->mode == INPUT_FILTER ? + f->r->headers_in : f->r->headers_out, + "Content-Type");
Re: input filters with mod_ext_filter [patch]
Am Montag, 23. Juni 2003 19:02 schrieben Sie: > On Mon, Jun 23, 2003 at 10:26:55AM +0200, Philipp Reisner wrote: > > Am Montag, 23. Juni 2003 03:56 schrieb Jeff Trawick: > > > (apologies for missing the right mail reference and nasty quoting... > > > e-mail problems, and luckily I was browsing the archives on daedalus > > > during my ISP-imposed silence) > > > > > > Philip, > > > > > > Thanks for submitting the patch. I hope you will fix minor issues and > > > resubmit for further review? > > > > Hi Jeff, > > > > Of course! I am happy that you are willing to help me to get it in. > > > > Please find the new version of the patch attached. > > > > BTW, Please have a close look to the part where I look at the > > "Content-Type" header. The old code examien r->content_type. I had to > > find an other way, one that is also possible for request filtering... > > You removed > -apr_bucket_brigade *bb; > so the later > APR_BRIGADE_INSERT_TAIL(bb, b); > is going to fail, won't it? > > Cheers, > Glenn > > > @@ -658,7 +662,6 @@ > apr_size_t len; > char buf[4096]; > apr_status_t rv; > -apr_bucket_brigade *bb; > apr_bucket *b; > > while (1) { > @@ -675,14 +678,9 @@ > if (rv != APR_SUCCESS) { > return rv; > } > -bb = apr_brigade_create(r->pool, c->bucket_alloc); > -b = apr_bucket_transient_create(buf, len, c->bucket_alloc); > +b = apr_bucket_heap_create(buf, len, NULL, c->bucket_alloc); > APR_BRIGADE_INSERT_TAIL(bb, b); > -if ((rv = ap_pass_brigade(f->next, bb)) != APR_SUCCESS) { > -ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r, > - "ap_pass_brigade()"); > -return rv; > -} > +return APR_SUCCESS; > } > /* we should never get here; if we do, a bogus error message would be > * the least of our problems Have a look at the chunk before, there the function gets a new parameter called "bb". -Philipp -- : Dipl-Ing Philipp Reisner Tel +43-1-8178292-50 : : LINBIT Information Technologies GmbH Fax +43-1-8178292-82 : : Schönbrunnerstr 244, 1120 Vienna, Austriahttp://www.linbit.com :
Re: input filters with mod_ext_filter [patch]
Am Montag, 23. Juni 2003 03:56 schrieb Jeff Trawick: > (apologies for missing the right mail reference and nasty quoting... > e-mail problems, and luckily I was browsing the archives on daedalus > during my ISP-imposed silence) > > Philip, > > Thanks for submitting the patch. I hope you will fix minor issues and > resubmit for further review? Hi Jeff, Of course! I am happy that you are willing to help me to get it in. Please find the new version of the patch attached. BTW, Please have a close look to the part where I look at the "Content-Type" header. The old code examien r->content_type. I had to find an other way, one that is also possible for request filtering... -Philipp -- : Dipl-Ing Philipp Reisner Tel +43-1-8178292-50 : : LINBIT Information Technologies GmbH Fax +43-1-8178292-82 : : Schönbrunnerstr 244, 1120 Vienna, Austriahttp://www.linbit.com : --- mod_ext_filter.c_orig 2003-02-27 13:33:07.0 +0100 +++ mod_ext_filter.c 2003-06-23 10:22:07.0 +0200 @@ -117,6 +117,9 @@ static const server_rec *main_server; static apr_status_t ef_output_filter(ap_filter_t *, apr_bucket_brigade *); +static apr_status_t ef_input_filter(ap_filter_t *, apr_bucket_brigade *, +ap_input_mode_t, apr_read_type_e, +apr_off_t); #define DBGLVL_SHOWOPTIONS 1 #define DBGLVL_ERRORCHECK 2 @@ -351,12 +354,10 @@ /* XXX need a way to ensure uniqueness among all filters */ ap_register_output_filter(filter->name, ef_output_filter, NULL, filter->ftype); } -#if 0 /* no input filters yet */ else if (filter->mode == INPUT_FILTER) { /* XXX need a way to ensure uniqueness among all filters */ -ap_register_input_filter(filter->name, ef_input_filter, NULL, AP_FTYPE_RESOURCE); +ap_register_input_filter(filter->name, ef_input_filter, NULL, filter->ftype); } -#endif else { ap_assert(1 != 1); /* we set the field wrong somehow */ } @@ -592,18 +593,20 @@ ctx->p = f->r->pool; if (ctx->filter->intype && ctx->filter->intype != INTYPE_ALL) { -if (!f->r->content_type) { -ctx->noop = 1; -} -else { -const char *ctypes = f->r->content_type; -const char *ctype = ap_getword(f->r->pool, &ctypes, ';'); - -if (strcasecmp(ctx->filter->intype, ctype)) { -/* wrong IMT for us; don't mess with the output */ -ctx->noop = 1; -} -} +const char *ctypes = apr_table_get(ctx->filter->mode == INPUT_FILTER ? + f->r->headers_in : f->r->headers_out, + "Content-Type"); + if(ctypes) { + const char *ctype = ap_getword(f->r->pool, &ctypes, ';'); + + if (strcasecmp(ctx->filter->intype, ctype)) { + /* wrong IMT for us; don't mess with the output */ + ctx->noop = 1; + } + } + else { + ctx->noop = 1; + } } if (ctx->filter->enable_env && !apr_table_get(f->r->subprocess_env, ctx->filter->enable_env)) { @@ -646,10 +649,11 @@ /* drain_available_output(): * - * if any data is available from the filter, read it and pass it - * to the next filter + * if any data is available from the filter, read it and append it + * to the the bucket brigade */ -static apr_status_t drain_available_output(ap_filter_t *f) +static apr_status_t drain_available_output(ap_filter_t *f, + apr_bucket_brigade *bb) { request_rec *r = f->r; conn_rec *c = r->connection; @@ -658,7 +662,6 @@ apr_size_t len; char buf[4096]; apr_status_t rv; -apr_bucket_brigade *bb; apr_bucket *b; while (1) { @@ -675,14 +678,9 @@ if (rv != APR_SUCCESS) { return rv; } -bb = apr_brigade_create(r->pool, c->bucket_alloc); -b = apr_bucket_transient_create(buf, len, c->bucket_alloc); +b = apr_bucket_heap_create(buf, len, NULL, c->bucket_alloc); APR_BRIGADE_INSERT_TAIL(bb, b); -if ((rv = ap_pass_brigade(f->next, bb)) != APR_SUCCESS) { -ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r, - "ap_pass_brigade()"); -return rv; -} +return APR_SUCCESS; } /* we should never get here; if we do, a bogus error message would be * the least of our problems @@ -691,7 +689,7 @@ } static apr_status_t pass_data_to_filter(ap_filter_t *f, const char *data, -apr_size_t len) +apr_size_t len, apr_bucket_brigade *bb) { ef_ctx_t *ctx = f->ctx; ef_dir_t *dc = ctx->dc; @@ -716,7 +714,7 @@ * to read data from the child process and pass it down to the * next filter! */ -rv = drain_a
input filters with mod_ext_filter [patch]
(apologies for missing the right mail reference and nasty quoting... e-mail problems, and luckily I was browsing the archives on daedalus during my ISP-imposed silence) Philip, Thanks for submitting the patch. I hope you will fix minor issues and resubmit for further review? + * + * Support for input filters added by Philipp Reisner in June 2003. I don't know that there is a formal policy regarding this sort of attribution, but imagine what the source tree would look like if everybody did this. That's what CHANGES is for. (It won't affect whether or not I commit the final patch, but it is something to think about.) + * + * apxs2 -Wc,-Wall,-g -Wl,-g -c -i mod_ext_filter.c axe +static int ef_input_filter(ap_filter_t *f, apr_bucket_brigade *bb, + ap_input_mode_t mode, apr_read_type_e block, + apr_off_t readbytes); shouldn't this forward decl go next to the one for ef_output_filter()? -#if 0 /* no input filters yet */ + /* no input filters yet */ uhh, that whole line needs to go else if (filter->mode == INPUT_FILTER) { /* XXX need a way to ensure uniqueness among all filters */ ap_register_input_filter(filter->name, ef_input_filter, NULL, AP_FTYPE_RESOURCE); why not filter->ftype instead of AP_FTYPE_RESOURCE? all sorts of tricks can be done with ext output filters by allowing the astute user to override the default filter type @@ -590,20 +599,24 @@ return APR_EINVAL; } ctx->p = f->r->pool; + if (ctx->filter->intype && patches are easier to review when there are no unnecessary changes... if you want to post another patch later for whitespace changes, that's fine but each patch should implement one feature/fix only + const char *ctypes = apr_table_get(ctx->filter->mode == INPUT_FILTER ? + f->r->headers_in : f->r->headers_out, + "Content-Type"); + const char *ctype = ap_getword(f->r->pool, &ctypes, ';'); + there are tab characters here and elsewhere... you need to get rid of them (possibly use an editor setting to encode tabs as spaces) + ap_log_rerror(APLOG_MARK, APLOG_NOERRNO|APLOG_DEBUG, 0, f->r, + "filter '%s': Content-Type='%s' => %s", + ctx->filter->name, ctype, + ctx->noop == 1 ? "noop" : "apply"); + 1) the trace doesn't make sense here since we haven't gotten to the last possible reason to no-op 2) APLOG_NOERRNO is for backwards compatibility only; you don't need it, since passing 0 in for the apr_status_t provides the required information 3) there is already a trace for this info and more anyway, at the end of the function + * if any data is available from the filter, read it and append it + * to the the bucket brigade duplicate duplicate word +static apr_status_t drain_available_output(ap_filter_t \ *f,apr_bucket_brigade *bb) there should be a space before "apr_bucket_brigade" (apply comment globally) @@ -716,7 +723,7 @@ * to read data from the child process and pass it down to the * next filter! */ -rv = drain_available_output(f); +rv = drain_available_output(f,bb); there should be a space before "bb" (apply comment globally) @@ -750,7 +757,14 @@ return rv; } -static apr_status_t ef_output_filter(ap_filter_t *f, apr_bucket_brigade *bb) +/* ef_unified_filter: + * + * runs the bucket brigade bb through the filter and returns the + * and puts the result into bb. Dropping the previous content + * of bb (the input) did you mean "runs the bucket brigade bb through the filter and puts the result into bb, dropping the previous content of bb (the input)" -if (APR_BUCKET_IS_EOS(b)) { + if (APR_BUCKET_IS_EOS(b)) { eos = b; break; } this is another hint that something changed in the patch that didn't need to Thanks!
input filters with mod_ext_filter [patch]
Hi, I guess I do not need to tell you that input filters with mod_ext_filter were not implemented. Since I needed this feature I implemented it. It would be a great honour for me if you would apply this patch to the main tree. -Philipp--- mod_ext_filter.c Thu Feb 27 13:33:07 2003 +++ mod_ext_filter.c_enh Sun Jun 22 10:30:45 2003 @@ -58,6 +58,10 @@ /* * mod_ext_filter allows Unix-style filters to filter http content. + * + * Support for input filters added by Philipp Reisner in June 2003. + * + * apxs2 -Wc,-Wall,-g -Wl,-g -c -i mod_ext_filter.c */ #include "httpd.h" @@ -124,6 +128,11 @@ #define ERRFN_USERDATA_KEY "EXTFILTCHILDERRFN" +static int ef_input_filter(ap_filter_t *f, apr_bucket_brigade *bb, + ap_input_mode_t mode, apr_read_type_e block, + apr_off_t readbytes); + + static void *create_ef_dir_conf(apr_pool_t *p, char *dummy) { ef_dir_t *dc = (ef_dir_t *)apr_pcalloc(p, sizeof(ef_dir_t)); @@ -351,12 +360,12 @@ /* XXX need a way to ensure uniqueness among all filters */ ap_register_output_filter(filter->name, ef_output_filter, NULL, filter->ftype); } -#if 0 /* no input filters yet */ + /* no input filters yet */ else if (filter->mode == INPUT_FILTER) { /* XXX need a way to ensure uniqueness among all filters */ ap_register_input_filter(filter->name, ef_input_filter, NULL, AP_FTYPE_RESOURCE); } -#endif + else { ap_assert(1 != 1); /* we set the field wrong somehow */ } @@ -590,20 +599,24 @@ return APR_EINVAL; } ctx->p = f->r->pool; + if (ctx->filter->intype && ctx->filter->intype != INTYPE_ALL) { -if (!f->r->content_type) { -ctx->noop = 1; -} -else { -const char *ctypes = f->r->content_type; -const char *ctype = ap_getword(f->r->pool, &ctypes, ';'); - -if (strcasecmp(ctx->filter->intype, ctype)) { -/* wrong IMT for us; don't mess with the output */ -ctx->noop = 1; -} -} + const char *ctypes = apr_table_get(ctx->filter->mode == INPUT_FILTER ? + f->r->headers_in : f->r->headers_out, + "Content-Type"); + const char *ctype = ap_getword(f->r->pool, &ctypes, ';'); + + if (strcasecmp(ctx->filter->intype, ctype)) { + /* wrong IMT for us; don't mess with the output */ + ctx->noop = 1; + } + + ap_log_rerror(APLOG_MARK, APLOG_NOERRNO|APLOG_DEBUG, 0, f->r, + "filter '%s': Content-Type='%s' => %s", + ctx->filter->name, ctype, + ctx->noop == 1 ? "noop" : "apply"); + } if (ctx->filter->enable_env && !apr_table_get(f->r->subprocess_env, ctx->filter->enable_env)) { @@ -646,10 +659,10 @@ /* drain_available_output(): * - * if any data is available from the filter, read it and pass it - * to the next filter + * if any data is available from the filter, read it and append it + * to the the bucket brigade */ -static apr_status_t drain_available_output(ap_filter_t *f) +static apr_status_t drain_available_output(ap_filter_t *f,apr_bucket_brigade *bb) { request_rec *r = f->r; conn_rec *c = r->connection; @@ -658,7 +671,6 @@ apr_size_t len; char buf[4096]; apr_status_t rv; -apr_bucket_brigade *bb; apr_bucket *b; while (1) { @@ -675,14 +687,9 @@ if (rv != APR_SUCCESS) { return rv; } -bb = apr_brigade_create(r->pool, c->bucket_alloc); -b = apr_bucket_transient_create(buf, len, c->bucket_alloc); +b = apr_bucket_heap_create(buf, len, NULL, c->bucket_alloc); APR_BRIGADE_INSERT_TAIL(bb, b); -if ((rv = ap_pass_brigade(f->next, bb)) != APR_SUCCESS) { -ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r, - "ap_pass_brigade()"); -return rv; -} + return APR_SUCCESS; } /* we should never get here; if we do, a bogus error message would be * the least of our problems @@ -691,7 +698,7 @@ } static apr_status_t pass_data_to_filter(ap_filter_t *f, const char *data, -apr_size_t len) +apr_size_t len,apr_bucket_brigade *bb) { ef_ctx_t *ctx = f->ctx; ef_dir_t *dc = ctx->dc; @@ -716,7 +723,7 @@ * to read data from the child process and pass it down to the * next filter! */ -rv = drain_available_output(f); +rv = drain_available_output(f,bb); if (APR_STATUS_IS_EAGAIN(rv)) { #if APR_FILES_AS_S
Tagging, WAS: RE: RFC: move mod_ext_filter to modules/filters now*
> From: William A. Rowe, Jr. [mailto:wrowe@;apache.org] > Sent: 14 November 2002 18:36 > At 10:17 AM 11/14/2002, you wrote: > >*or after tagging 2.0.44; that distinction isn't important to me; I > >just want to get the ball rolling to move it out of experimental; > >OtherBill, if a tag is still imminent, then it will definitely move > >*AFTER* 2.0.44; there is no need to complicate matters; > > Well, Sander is thinking about doing this tag. However, Jeff, > go ahead and move 'em ... +1 here! I would have done it already if my entire day hadn't gone to waste... I'll retry tomorrow. Sander
Re: RFC: move mod_ext_filter to modules/filters now*
At 10:17 AM 11/14/2002, you wrote: >*or after tagging 2.0.44; that distinction isn't important to me; I >just want to get the ball rolling to move it out of experimental; >OtherBill, if a tag is still imminent, then it will definitely move >*AFTER* 2.0.44; there is no need to complicate matters; Well, Sander is thinking about doing this tag. However, Jeff, go ahead and move 'em ... +1 here! >This module has been used successfully by a number of people in the >time that it has been available in experimental. I think it is time >to make it supported and available for --enable-modules=most so that >it is present in a binbuild for Unix. > >Currently there are two open PRs for mod_ext_filter: > > http://nagoya.apache.org/bugzilla/show_bug.cgi?id=11978 > (PDF generated in a filter not handled properly by IE; probably an > IE issue, since we would chunk the response and a MS KnowledgeBase > article says that IE requires a Content-Length header to deal with > PDFs properly) > > http://nagoya.apache.org/bugzilla/show_bug.cgi?id=14016 > (ActiveState Perl CGI not seeing eof on stdin; whether or not this > is a mod_ext_filter problem, we build mod_ext_filter by default on > Win32 anyway) > >Is anyone against moving it to modules/filters for 2.0.stable? Absolutely no objection. I'll take a look at that later bug today [since it impacts things several ways.]
Re: RFC: move mod_ext_filter to modules/filters now*
At 11:17 AM -0500 11/14/02, Jeff Trawick wrote: > >Is anyone against moving it to modules/filters for 2.0.stable? > I'm +1 for it being "upgraded" to non-experimental status -- === Jim Jagielski [|] [EMAIL PROTECTED] [|] http://www.jaguNET.com/ "A society that will trade a little liberty for a little order will lose both and deserve neither" - T.Jefferson
RFC: move mod_ext_filter to modules/filters now*
*or after tagging 2.0.44; that distinction isn't important to me; I just want to get the ball rolling to move it out of experimental; OtherBill, if a tag is still imminent, then it will definitely move *AFTER* 2.0.44; there is no need to complicate matters; This module has been used successfully by a number of people in the time that it has been available in experimental. I think it is time to make it supported and available for --enable-modules=most so that it is present in a binbuild for Unix. Currently there are two open PRs for mod_ext_filter: http://nagoya.apache.org/bugzilla/show_bug.cgi?id=11978 (PDF generated in a filter not handled properly by IE; probably an IE issue, since we would chunk the response and a MS KnowledgeBase article says that IE requires a Content-Length header to deal with PDFs properly) http://nagoya.apache.org/bugzilla/show_bug.cgi?id=14016 (ActiveState Perl CGI not seeing eof on stdin; whether or not this is a mod_ext_filter problem, we build mod_ext_filter by default on Win32 anyway) Is anyone against moving it to modules/filters for 2.0.stable? -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: Question about command parsing in mod_ext_filter.
"William A. Rowe, Jr." wrote: > Don't we -already- have some argv parsing code in either proc.c or the mod_cgi > that could be used for this purpose??? Let's make that exported, accessible > code from apr itself. My thoughts exactly, but then I'm the moron who put something different in mod_ext_filter :) -- Jeff Trawick
Re: Question about command parsing in mod_ext_filter.
Will do. [EMAIL PROTECTED] wrote: > On Mon, 19 Aug 2002, William A. Rowe, Jr. wrote: > > >>Don't we -already- have some argv parsing code in either proc.c or the mod_cgi >>that could be used for this purpose??? Let's make that exported, accessible >>code from apr itself. >> > > We already have it, and it is exported from APR. Look in > strings/apr_cpystrn.c for apr_tokenize_to_argv. > > Ryan > > >>Bill >> >>At 03:31 PM 8/19/2002, Paul J. Reder wrote: >> >>>After fixing the parse_cmd code to be able to handle escaped quotes (so that >>>it would process the full >>> <"/bin/awk '{print NR\": \" $N}'"> >>>instead of just >>> <"/bin/awk '{print NR\"> >>>(and then an error for the <: \" $N}'"> part)) I ran into what appears to be >>>the next problem. >>> >>>In parse_cmd it runs >>> >>>/* parms now has the command-line to parse */ >>>while (filter->numArgs < 30 && >>> strlen(filter->args[filter->numArgs] = >>>ap_getword_white_nc(p, &parms))) { >>>++filter->numArgs; >>>} >>> >>>to break the command string into an args list of tokens. This code uses >>>*only* white >>>space as delimeters. Thus the input command string >>> "/bin/awk '{print NR\": \" $N}'" >>> >>>gets broken into >>> /bin/awk >>> '{print >>> NR\": >>> \" >>> $N}' >>> >>>The question I have is, shouldn't this break down into either >>> /bin/awk >>> '{print NR\": \" $N}' >>>or >>> /bin/awk >>> '{print NR\ >>> ": \" >>> $N}' >>> >>>The above listed pieces end up as the args list passed on the apr_proc_create >>>call. >>> >>> >>>Thanks, >>> >>>-- >>>Paul J. Reder >>>--- >>>"The strength of the Constitution lies entirely in the determination of each >>>citizen to defend it. Only if every single citizen feels duty bound to do >>>his share in this defense are the constitutional rights secure." >>>-- Albert Einstein >>> >>> >> > -- Paul J. Reder --- "The strength of the Constitution lies entirely in the determination of each citizen to defend it. Only if every single citizen feels duty bound to do his share in this defense are the constitutional rights secure." -- Albert Einstein
Re: Question about command parsing in mod_ext_filter.
On Mon, 19 Aug 2002, William A. Rowe, Jr. wrote: > Don't we -already- have some argv parsing code in either proc.c or the mod_cgi > that could be used for this purpose??? Let's make that exported, accessible > code from apr itself. We already have it, and it is exported from APR. Look in strings/apr_cpystrn.c for apr_tokenize_to_argv. Ryan > > Bill > > At 03:31 PM 8/19/2002, Paul J. Reder wrote: > >After fixing the parse_cmd code to be able to handle escaped quotes (so that > >it would process the full > > <"/bin/awk '{print NR\": \" $N}'"> > >instead of just > > <"/bin/awk '{print NR\"> > >(and then an error for the <: \" $N}'"> part)) I ran into what appears to be > >the next problem. > > > >In parse_cmd it runs > > > > /* parms now has the command-line to parse */ > > while (filter->numArgs < 30 && > >strlen(filter->args[filter->numArgs] = > > ap_getword_white_nc(p, &parms))) { > > ++filter->numArgs; > > } > > > >to break the command string into an args list of tokens. This code uses > >*only* white > >space as delimeters. Thus the input command string > > "/bin/awk '{print NR\": \" $N}'" > > > >gets broken into > > /bin/awk > > '{print > > NR\": > > \" > > $N}' > > > >The question I have is, shouldn't this break down into either > > /bin/awk > > '{print NR\": \" $N}' > >or > > /bin/awk > > '{print NR\ > > ": \" > > $N}' > > > >The above listed pieces end up as the args list passed on the apr_proc_create > >call. > > > > > >Thanks, > > > >-- > >Paul J. Reder > >--- > >"The strength of the Constitution lies entirely in the determination of each > >citizen to defend it. Only if every single citizen feels duty bound to do > >his share in this defense are the constitutional rights secure." > >-- Albert Einstein > > > > -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: Question about command parsing in mod_ext_filter.
Don't we -already- have some argv parsing code in either proc.c or the mod_cgi that could be used for this purpose??? Let's make that exported, accessible code from apr itself. Bill At 03:31 PM 8/19/2002, Paul J. Reder wrote: >After fixing the parse_cmd code to be able to handle escaped quotes (so that >it would process the full > <"/bin/awk '{print NR\": \" $N}'"> >instead of just > <"/bin/awk '{print NR\"> >(and then an error for the <: \" $N}'"> part)) I ran into what appears to be >the next problem. > >In parse_cmd it runs > > /* parms now has the command-line to parse */ > while (filter->numArgs < 30 && >strlen(filter->args[filter->numArgs] = > ap_getword_white_nc(p, &parms))) { > ++filter->numArgs; > } > >to break the command string into an args list of tokens. This code uses >*only* white >space as delimeters. Thus the input command string > "/bin/awk '{print NR\": \" $N}'" > >gets broken into > /bin/awk > '{print > NR\": > \" > $N}' > >The question I have is, shouldn't this break down into either > /bin/awk > '{print NR\": \" $N}' >or > /bin/awk > '{print NR\ > ": \" > $N}' > >The above listed pieces end up as the args list passed on the apr_proc_create >call. > > >Thanks, > >-- >Paul J. Reder >--- >"The strength of the Constitution lies entirely in the determination of each >citizen to defend it. Only if every single citizen feels duty bound to do >his share in this defense are the constitutional rights secure." >-- Albert Einstein >
Question about command parsing in mod_ext_filter.
After fixing the parse_cmd code to be able to handle escaped quotes (so that it would process the full <"/bin/awk '{print NR\": \" $N}'"> instead of just <"/bin/awk '{print NR\"> (and then an error for the <: \" $N}'"> part)) I ran into what appears to be the next problem. In parse_cmd it runs /* parms now has the command-line to parse */ while (filter->numArgs < 30 && strlen(filter->args[filter->numArgs] = ap_getword_white_nc(p, &parms))) { ++filter->numArgs; } to break the command string into an args list of tokens. This code uses *only* white space as delimeters. Thus the input command string "/bin/awk '{print NR\": \" $N}'" gets broken into /bin/awk '{print NR\": \" $N}' The question I have is, shouldn't this break down into either /bin/awk '{print NR\": \" $N}' or /bin/awk '{print NR\ ": \" $N}' The above listed pieces end up as the args list passed on the apr_proc_create call. Thanks, -- Paul J. Reder --- "The strength of the Constitution lies entirely in the determination of each citizen to defend it. Only if every single citizen feels duty bound to do his share in this defense are the constitutional rights secure." -- Albert Einstein
Re: environment in mod_ext_filter
Craig Sebenik <[EMAIL PROTECTED]> writes: > I put it in as a bug: > > http://nagoya.apache.org/bugzilla/show_bug.cgi?id=10925 code is committed to 2.0.40-dev with very minor tweaks to clean up warnings > The dev guidelines say that patches can be submitted either via an email > (to this list) or via the bug database. But, it doesn't say which is > preferred... I guess different people would have different preferences... > > Anyway, it's there... I could send it to the list also, if you > prefer. Either is a workable mechanism. If you're willing to follow dev@httpd for a while, posting there will get you a wider audience. If you're not willing to follow dev@httpd, posting to the bug db is still fine. Either way, it is your responsibility to remind us after a while if none of the unherded cats around here takes any interest in your patch :) > You'll notice that I put in a call to "ap_add_cgi_vars", which may be a > little excessive (or down right wrong). So, that is "up for discussion". Other than GATEWAY_INTERFACE, the other cgi vars seem reasonable to make available... I left that logic as-is. -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: environment in mod_ext_filter
Quoting Jeff Trawick ([EMAIL PROTECTED]): > Craig Sebenik <[EMAIL PROTECTED]> writes: > > Is there a reason why the environment is null? > probably because I'm a moron :) :) *Probably*??? You're not sure? :) > > Am I missing something? Is there a reason I shouldn't submit this patch? > > no, no I put it in as a bug: http://nagoya.apache.org/bugzilla/show_bug.cgi?id=10925 The dev guidelines say that patches can be submitted either via an email (to this list) or via the bug database. But, it doesn't say which is preferred... I guess different people would have different preferences... Anyway, it's there... I could send it to the list also, if you prefer. You'll notice that I put in a call to "ap_add_cgi_vars", which may be a little excessive (or down right wrong). So, that is "up for discussion". Craig
Re: environment in mod_ext_filter
Craig Sebenik <[EMAIL PROTECTED]> writes: > In modules/experimental/mod_ext_filter.c there's the following: > > rc = apr_proc_create(ctx->proc, > ctx->filter->command, > (const char * const *)ctx->filter->args, > NULL, /* environment */ > ctx->procattr, > ctx->p); > > Is there a reason why the environment is null? probably because I'm a moron :) > Am I missing something? Is there a reason I shouldn't submit this patch? no, no -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
environment in mod_ext_filter
Hi, In modules/experimental/mod_ext_filter.c there's the following: rc = apr_proc_create(ctx->proc, ctx->filter->command, (const char * const *)ctx->filter->args, NULL, /* environment */ ctx->procattr, ctx->p); Is there a reason why the environment is null? I was able to hack in the env in a few simples lines. Am I missing something? Is there a reason I shouldn't submit this patch? Thanx, Craig
Re: mod_ext_filter and windows binaries info needed, please (again)
At 12:06 PM 5/28/2002, you wrote: > >We are working hard to pull the release together since > >so many bug fixes have gone in since .36. >Bill, > >There was a ScriptInterpreterSource config bug in the .35 and .36 >Apache for Windows releases, and I don't know if it will be corrected >in .37 release... > >Any info? Yes... please query the new bug tracking system at http://nagoya.apache.org/bugzilla/ for status of any bugs with respect to their current status, and feel free to annotate [one thing I really like about the change to the new bug tracking system.] Bill
Re: mod_ext_filter and windows binaries info needed, please (again)
>We are working hard to pull the release together since >so many bug fixes have gone in since .36. Bill, There was a ScriptInterpreterSource config bug in the .35 and .36 Apache for Windows releases, and I don't know if it will be corrected in .37 release... Any info? Thanks, Webmaster33 *** REPLY SEPARATOR *** On 2002.05.28 at 11:26 William A. Rowe, Jr. wrote: >At 10:09 AM 5/28/2002, Igor Leturia wrote: >> Hi! >> >> I'm writing this message again, as no one has answered the other one. >> >> I'm interested in the module mod_ext_filter. As it is in experimental >>phase, it is not included in the windows binaries of 2.0.36. > >Sorry I didn't answer your question the first go around. > >Both mod_ext_filter and mod_deflate were -supposed- to be packaged >with the 2.0.36 installer. We built them ... apparently we didn't get all >of the bits toggled correctly to include them in the distro. > >I'll make sure this is fixed for 2.0.37 which should be out in the next >week or so. We are working hard to pull the release together since >so many bug fixes have gone in since .36. > >Bill *** END REPLIED MESSAGE ***
Re: mod_ext_filter and windows binaries info needed, please (again)
At 10:09 AM 5/28/2002, Igor Leturia wrote: > Hi! > > I'm writing this message again, as no one has answered the other one. > > I'm interested in the module mod_ext_filter. As it is in experimental >phase, it is not included in the windows binaries of 2.0.36. Sorry I didn't answer your question the first go around. Both mod_ext_filter and mod_deflate were -supposed- to be packaged with the 2.0.36 installer. We built them ... apparently we didn't get all of the bits toggled correctly to include them in the distro. I'll make sure this is fixed for 2.0.37 which should be out in the next week or so. We are working hard to pull the release together since so many bug fixes have gone in since .36. Bill
mod_ext_filter and windows binaries info needed, please (again)
Hi! I'm writing this message again, as no one has answered the other one. I'm interested in the module mod_ext_filter. As it is in experimental phase, it is not included in the windows binaries of 2.0.36. I would like to ask the people that builds the windows binaries if it will be in the next version. But, as the next version might not be out for some time, I would be very grateful if the windows binaries builders would compile the module and send it to me, if it's not too much asking. Its author, Jeff Trawick, told me that they won't move the module out of experimental until a number of users have tried and used it. He says there aren't many people using it and that he is interested in any feedback about it. Well, I'm ready and willing to try it, because I think it might solve a problem I have, but I need the module compiled for windows. Anyway, thanks in advance and sorry for the intrusion. Igor Leturia
mod_ext_filter
Hi! I'm interested in the module mod_ext_filter. As it is in experimental phase, it is not included in the windows binaries of 2.0.36. I would like to ask the people that builds the windows binaries if it will be in the next version. But, as the next version might not be out for some time, I would be very grateful if the windows binaries builders would compile the module and send it to me, if it's not too much asking. Its author, Jeff Trawick, told me that they won't move the module out of experimental until a number of users have tried and used it. He says there aren't many people using it and that he is interested in any feedback about it. Well, I'm ready and willing to try it, because I think it might solve a problem I have, but I need the module compiled for windows. Anyway, thanks in advance and sorry for the intrusion. Igor Leturia
Re: mod_ext_filter
"Igor Leturia" <[EMAIL PROTECTED]> writes: > First of all, I must apologize as I am not exactly an Apache > developper, though I would like to help in some way. I am writing to > this list just to point out an error in Apache 2.0.35. The problem is > that the Windows distribution of Apache 2.0.35 doesn't include the > mod_ext_filter.so file, although the docs say that the module exists in > that version. FYI... binary distributions don't contain experimental modules, and for good reason... experimental modules typically require more skill on the part of the user in order to make effective use of them... if they are prebuilt for the masses, more hand-holding will be required... by the same token, somebody who has gone to the trouble to build Apache themselves has a greater understanding of how the pieces fit together and can solve some of the problems they encounter (such as the fact that there is no project file for mod_ext_filter) this is not to say that you won't get 3 copies of mod_ext_filter in your inbox, most of which won't contain viruses; I have no idea if anybody has ever built this for Windows, much less if somebody has a copy handy which corresponds to Apache 2.0.35 -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
mod_ext_filter
Hi! First of all, I must apologize as I am not exactly an Apache developper, though I would like to help in some way. I am writing to this list just to point out an error in Apache 2.0.35. The problem is that the Windows distribution of Apache 2.0.35 doesn't include the mod_ext_filter.so file, although the docs say that the module exists in that version. Besides, I am interested in that module, so if someone could please send me the mod_ext_filter.so file for Windows, I would try to help in its development by trying it (I heard it is in an experimental phase yet, isn't it?) and sending the results to this list. Thanks in advance, Igor Leturia