Re: notes on filters in 2.2.x
On Tue, Feb 24, 2009 at 10:27 PM, Roy T. Fielding wrote: > I spent a while looking at mod_deflate and various filter related > issues in 2.2.x/trunk, but I had to context switch away before > I could create such a large fix. This message is to write > down my conclusions so that I can remember them and maybe fix > the silly thing when I get a chance, or maybe encourage someone > else to do it in the meantime. > > The current state is broken. There is no other way to describe it. > mod_deflate is adding -gzip to the end of etags in order to be > correct in regard to variants, but ap_meets_conditions() is not > aware of the -gzip addition; in fact, etags are compared before > the output filter is even initialized for a given response, so > there is no way for ap_meets_conditions() to work without resorting > to stupid hacks like always checking for "foo" and "foo-gzip" in > the core (even the hack is difficult without completely rewriting > how etags are compared). > > Roy, For what its worth (maybe to confirm your observations?), while trying to track down a memory leak with mod_perl, I noted that Apache started using a little more memory for every request only when mod_deflate was enabled. http://www.docunext.com/blog/2008/06/24/even-more-perl-notes/ And to sing the praise of filters, I believe them to be a very good thing for dynamic content generation. And, even if its not the most efficient, mod_ext_filter is a sweet one. - Albert
Re: patch for handling headers_in and headers_out as tables in mod_lua
On Thu, Feb 5, 2009 at 11:14 PM, Brian Akins wrote:
> On 2/5/09 1:51 PM, "Brian McCallister" wrote:
>
>> Yep, Paul changed the internal impl to be less gross, but in doing so
>> changed the API, i changed the impl to be not gross and restored old
>> API.
>
> Okay I see it now.
>
> I may take a crack at a little performance tuning. Setting up the dispatch
> table every single time doesn't seem necessary. We should be able to just
> do that once at httpd start time.
>
>
> So, if I'm reading that correctly, this should work now as well??
>
> r.content_type
>
> And we would just hack up req_newindex to be able to do
> r.content_type = "application/bakins"
>
> I like the dispatch idea. I'll think about the same for newindex as well...
By the way, for content-type, the dispatch function at the moment is:
if (0 == apr_strnatcmp("content_type", key)) {
const char *value = luaL_checkstring(L, 3);
r->content_type = apr_pstrdup(r->pool, value);
luaL_getmetatable(L, "Apache2.Request");
lua_pushstring(L, value);
lua_setfield(L, -2, "content_type");
lua_pop(L, 1);
return 0;
}
Shouldn't it use ap_set_content_type() instead ?
And I find it weird that, now, to get 'document root', you have to
call r:document_root() instead of r.document_root as it was before.
That's because the dispatch is set like this:
apr_hash_set(dispatch, "document_root", APR_HASH_KEY_STRING,
makefun(&req_document_root, APL_REQ_FUNTYPE_LUACFUN, p));
It could be just:
apr_hash_set(dispatch, "document_root", APR_HASH_KEY_STRING,
makefun(&req_content_encoding_field, APL_REQ_FUNTYPE_STRING,
p));
--
Bertrand Mansion
Mamasam
Re: regex-related segfault in mod_include
Ruediger Pluem wrote: > What are the values of > > idx > re->match[idx].rm_so > re->match[idx].rm_eo > re->source > > and what is the string re->source is pointing to when the crash happens? idx is 1 and re->source points to an empty string which is fine. However, re->match[idx].rm_so and re->match[idx].rm_eo are random numbers, i.e., a garbage value (I guess they should be 0 if there was no match?). Thus the argument "re->source + re->match[idx].rm_so" ends up pointing to an out of band location (and a memcpy() for that location results in the segfault). I just don't really get why this happens in some cases (like 1 out of 10 requests). BTW, I can reproduce this on Solaris and Linux (worker and prefork) with 2.2.11. With 2.0 this works fine. ciao... -- Lars Eilebrecht l...@eilebrecht.net
AW: regex-related segfault in mod_include
> -Ursprüngliche Nachricht-
> Von: Lars Eilebrecht [mailto:l...@eilebrecht.net]
> Gesendet: Freitag, 27. Februar 2009 12:54
> An: dev@httpd.apache.org
> Betreff: Re: regex-related segfault in mod_include
>
> Ruediger Pluem wrote:
>
> > What are the values of
> >
> > idx
> > re->match[idx].rm_so
> > re->match[idx].rm_eo
> > re->source
> >
> > and what is the string re->source is pointing to when the
> crash happens?
>
> idx is 1 and re->source points to an empty string which is fine.
> However, re->match[idx].rm_so and re->match[idx].rm_eo are
> random numbers,
> i.e., a garbage value (I guess they should be 0 if there was
> no match?).
IMHO they should be -1.
> Thus the argument "re->source + re->match[idx].rm_so" ends up
> pointing to
> an out of band location (and a memcpy() for that location results in
> the segfault).
>
> I just don't really get why this happens in some cases (like 1 out of
> 10 requests).
>
> BTW, I can reproduce this on Solaris and Linux (worker and prefork)
> with 2.2.11. With 2.0 this works fine.
We use different PCRE versions in both (and maybe mod_include changed too).
I suspect that if ap_regexec in re_check does not detect a match
re->match[idx].rm_so is not setup correctly (maybe this changed between the
different PCRE versions) and as we do not check in get_include_var if we had
a match at all we fall over. So we should either memorize in the re struct
if we matched or not by an additional flag, so something like (untested)
Index: mod_include.c
===
--- mod_include.c (revision 748051)
+++ mod_include.c (working copy)
@@ -158,6 +158,7 @@
const char *rexp;
apr_size_t nsub;
ap_regmatch_t match[AP_MAX_REG_MATCH];
+int matched;
} backref_t;
typedef struct {
@@ -672,7 +673,8 @@
return NULL;
}
-if (re->match[idx].rm_so < 0 || re->match[idx].rm_eo < 0) {
+if (re->match[idx].rm_so < 0 || re->match[idx].rm_eo < 0
+|| !re->matched) {
return NULL;
}
@@ -923,7 +925,6 @@
{
ap_regex_t *compiled;
backref_t *re = ctx->intern->re;
-int rc;
compiled = ap_pregcomp(ctx->dpool, rexp, AP_REG_EXTENDED);
if (!compiled) {
@@ -939,10 +940,10 @@
re->source = apr_pstrdup(ctx->pool, string);
re->rexp = apr_pstrdup(ctx->pool, rexp);
re->nsub = compiled->re_nsub;
-rc = !ap_regexec(compiled, string, AP_MAX_REG_MATCH, re->match, 0);
+re->matched = !ap_regexec(compiled, string, AP_MAX_REG_MATCH, re->match,
0);
ap_pregfree(ctx->dpool, compiled);
-return rc;
+return re->matched;
}
static int get_ptoken(include_ctx_t *ctx, const char **parse, token_t *token,
token_t *previous)
or we should initialize the match array correctly:
Index: mod_include.c
===
--- mod_include.c (revision 748051)
+++ mod_include.c (working copy)
@@ -924,6 +924,7 @@
ap_regex_t *compiled;
backref_t *re = ctx->intern->re;
int rc;
+int i;
compiled = ap_pregcomp(ctx->dpool, rexp, AP_REG_EXTENDED);
if (!compiled) {
@@ -939,6 +940,10 @@
re->source = apr_pstrdup(ctx->pool, string);
re->rexp = apr_pstrdup(ctx->pool, rexp);
re->nsub = compiled->re_nsub;
+for (i = 0; i < AP_MAX_REG_MATCH; i++) {
+re->match[i].rm_so = -1;
+re->match[i].rm_eo = -1;
+}
rc = !ap_regexec(compiled, string, AP_MAX_REG_MATCH, re->match, 0);
ap_pregfree(ctx->dpool, compiled);
Regards
Rüdiger
Re: regex-related segfault in mod_include
"Plüm, Rüdiger, VF-Group" wrote: > > However, re->match[idx].rm_so and re->match[idx].rm_eo are > > random numbers, > > i.e., a garbage value (I guess they should be 0 if there was > > no match?). > > IMHO they should be -1. Right, that actually makes more sense ... > We use different PCRE versions in both (and maybe mod_include changed too). > I suspect that if ap_regexec in re_check does not detect a match > re->match[idx].rm_so is not setup correctly (maybe this changed between the > different PCRE versions) and as we do not check in get_include_var if we had > a match at all we fall over. So we should either memorize in the re struct > if we matched or not by an additional flag, so something like (untested) OK, nice ... I was trying to figure out if such a flag/value exists in ap_regmatch_t, but that didn't got me very far as re->match is basically pointing to garbage data. So initializing that actually prevents the segfault as it hits the if statement for "re->match[idx].rm_so < 0" (I did a quick test with your second patch). However, for performance reasons I think fixing this with an additional flag would be the best. I'll do some more testing and will come up with a final patch for this. Thanks Ruediger, that was very helpful. :) cheers... -- Lars Eilebrecht l...@eilebrecht.net
RE: Mod dumpio module
Hi Nick, becouse in the module mod_usertrack use the next line: apr_table_setn(r->notes, "cookie", cookieval); /* Set the cookie in a note, for logging */ then I suppose that is possible to get the value "cookie" into the mod_dumpio using the line: const char *cookieval = apr_table_get(f->r->notes, "cookie"); There are any form to get the the info set by mod_usertrack in mod_dumpio?? Thaks in advance. -Original Message- From: Nick Kew [mailto:n...@webthing.com] Sent: Jueves, 26 de Febrero de 2009 06:38 p.m. To: dev@httpd.apache.org Subject: Re: Mod dumpio module On Thu, 26 Feb 2009 18:30:03 -0600 David Martínez Albiter wrote: > request_rec *r = f->r; Dumpio is a connection-level filter. There is no request. (and why do you suppose there'd be a cookie in r->notes?) -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: mod_wombat and mod_lua
On Wed, Feb 25, 2009 at 5:50 PM, Paul Querna wrote: > 23-27 March 2009, Amsterdam: > http://www.eu.apachecon.com/c/aceu2009/ > > I'll be there at least Ditto. -- justin
Re: mod_wombat and mod_lua
Justin Erenkrantz wrote: On Wed, Feb 25, 2009 at 5:50 PM, Paul Querna wrote: 23-27 March 2009, Amsterdam: http://www.eu.apachecon.com/c/aceu2009/ I'll be there at least Ditto. -- justin Same here. Except for a drop-in appearance at Jim and Rich's training, I even expect to be hacking on httpd for this time, maybe some apr.
Re: mod_wombat and mod_lua
On Fri, Feb 27, 2009 at 11:16 AM, Justin Erenkrantz wrote: > On Wed, Feb 25, 2009 at 5:50 PM, Paul Querna wrote: > > 23-27 March 2009, Amsterdam: > > http://www.eu.apachecon.com/c/aceu2009/ > > > > I'll be there at least > > Ditto. -- justin > Hoping to be...
Re: mod_wombat and mod_lua
Wish i could be, but urgent family business will keep me away :-( Maybe virtually joining :-) On Fri, Feb 27, 2009 at 11:50 AM, Jeff Trawick wrote: > On Fri, Feb 27, 2009 at 11:16 AM, Justin Erenkrantz > wrote: >> >> On Wed, Feb 25, 2009 at 5:50 PM, Paul Querna wrote: >> > 23-27 March 2009, Amsterdam: >> > http://www.eu.apachecon.com/c/aceu2009/ >> > >> > I'll be there at least >> >> Ditto. -- justin > > Hoping to be... > >
Re: Serving filenames with wildcards using mod_proxy_ftp
On Tue, Feb 24, 2009 at 10:29 AM, Dan Poirier wrote:
> The ProxyFtpListOnWildcard directive controls whether wildcard
> characters ("*?[{~") in requested filenames cause mod_proxy_ftp to
> return a listing of files instead of downloading a file. By default
> (value on), they do. Set to "off" to allow downloading files even
> if they have wildcard characters in their names. (directory
> context)
Looks reasonable
>
> I'd appreciate comments on this approach, and my proposed
> implementation (patch against trunk is attached).
>
> Also, in testing that change, I found that mod_proxy_ftp escapes
> wildcards in filenames using backslashes when sending them to the FTP
> server, which none of the FTP servers I was testing with understood.
> To continue with my testing, I added another directive to turn that
> behavior off, but I'd like to better understand why that behavior is
> there, since it appears to assume a behavior that my FTP servers don't
> have, and doesn't seem to be mentioned in RFC 959.
>
What backends/results did you try? Does anyone have any anecdotal
experience about what backends might require escaping of the glob
characters on a retrieve?
I'm wondering if we add the feature, should we flip the default in trunk?
--
Eric Covener
cove...@gmail.com
Re: Serving filenames with wildcards using mod_proxy_ftp
Eric Covener writes: > On Tue, Feb 24, 2009 at 10:29 AM, Dan Poirier wrote: >> Also, in testing that change, I found that mod_proxy_ftp escapes >> wildcards in filenames using backslashes when sending them to the FTP >> server, which none of the FTP servers I was testing with understood. >> To continue with my testing, I added another directive to turn that >> behavior off, but I'd like to better understand why that behavior is >> there, since it appears to assume a behavior that my FTP servers don't >> have, and doesn't seem to be mentioned in RFC 959. >> > > What backends/results did you try? Does anyone have any anecdotal > experience about what backends might require escaping of the glob > characters on a retrieve? As I recall, I tried vsftpd, proftpd, and Apache FTP server. > I'm wondering if we add the feature, should we flip the default in trunk? I would say yes. I don't think this escaping could ever have been used before, since any paths with globbing characters in them would have triggered a file listing, which doesn't do the escaping. So until now, apache would never have tried to retrieve any file with globbing characters in its name. There's not really any reason to make this an option, now that I think about it, unless someone comes up with an example of an ftp server that needs it. We might just want to remove the escaping code. -- Dan Poirier
Re: mod_wombat and mod_lua
I will be there too, but I probably won't be able to help much because my knowledge of Apache is too limited at the moment. Still, it will be a pleasure to meet you. And maybe I can write the unit tests in Lua :) On Fri, Feb 27, 2009 at 8:56 PM, Brian McCallister wrote: > Wish i could be, but urgent family business will keep me away :-( > > Maybe virtually joining :-) > > On Fri, Feb 27, 2009 at 11:50 AM, Jeff Trawick wrote: >> On Fri, Feb 27, 2009 at 11:16 AM, Justin Erenkrantz >> wrote: >>> >>> On Wed, Feb 25, 2009 at 5:50 PM, Paul Querna wrote: >>> > 23-27 March 2009, Amsterdam: >>> > http://www.eu.apachecon.com/c/aceu2009/ >>> > >>> > I'll be there at least >>> >>> Ditto. -- justin >> >> Hoping to be... >> >> > -- Bertrand Mansion Mamasam
Re: svn commit: r748396 - in /httpd/httpd/trunk: modules/ssl/ssl_engine_init.c modules/ssl/ssl_engine_kernel.c modules/ssl/ssl_engine_vars.c modules/ssl/ssl_util_ssl.c support/ab.c
On 02/27/2009 06:16 AM, scte...@apache.org wrote: > Author: sctemme > Date: Fri Feb 27 05:16:18 2009 > New Revision: 748396 > > URL: http://svn.apache.org/viewvc?rev=748396&view=rev > Log: > The development trunk of OpenSSL has tightened up the type safety of the > STACK construct > and the functions that manipulate it. Make httpd trunk compile against > OpenSSL HEAD > as well as OpenSSL 0.9.8j. Also, get rid of some warnings. > > Modified: > httpd/httpd/trunk/modules/ssl/ssl_engine_init.c > httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c > httpd/httpd/trunk/modules/ssl/ssl_engine_vars.c > httpd/httpd/trunk/modules/ssl/ssl_util_ssl.c > httpd/httpd/trunk/support/ab.c > > Modified: httpd/httpd/trunk/support/ab.c > URL: > http://svn.apache.org/viewvc/httpd/httpd/trunk/support/ab.c?rev=748396&r1=748395&r2=748396&view=diff > == > --- httpd/httpd/trunk/support/ab.c (original) > +++ httpd/httpd/trunk/support/ab.c Fri Feb 27 05:16:18 2009 > @@ -1979,7 +1979,7 @@ > const char *optarg; > char c; > #ifdef USE_SSL > -SSL_METHOD *meth = SSLv23_client_method(); > +const SSL_METHOD *meth = SSLv23_client_method(); > #endif Hm. Now I get the following warning with openssl-0.9.8d: ab.c: In function 'main': ab.c:2230: warning: passing argument 1 of 'SSL_CTX_new' discards qualifiers from pointer target type Regards Rüdiger
Re: mod_wombat and mod_lua
On Fri, Feb 27, 2009 at 2:11 PM, Bertrand Mansion wrote: > I will be there too, but I probably won't be able to help much because > my knowledge of Apache is too limited at the moment. Still, it will be > a pleasure to meet you. And maybe I can write the unit tests in Lua :) Don't underestimate yourself, mod_wombat got started because I pestered Paul to help me learn how to write apache modules at an apachecon! > > On Fri, Feb 27, 2009 at 8:56 PM, Brian McCallister wrote: >> Wish i could be, but urgent family business will keep me away :-( >> >> Maybe virtually joining :-) >> >> On Fri, Feb 27, 2009 at 11:50 AM, Jeff Trawick wrote: >>> On Fri, Feb 27, 2009 at 11:16 AM, Justin Erenkrantz >>> wrote: On Wed, Feb 25, 2009 at 5:50 PM, Paul Querna wrote: > 23-27 March 2009, Amsterdam: > http://www.eu.apachecon.com/c/aceu2009/ > > I'll be there at least Ditto. -- justin >>> >>> Hoping to be... >>> >>> >> > > > > -- > Bertrand Mansion > Mamasam >
Re: svn commit: r748396 - in /httpd/httpd/trunk: modules/ssl/ssl_engine_init.c modules/ssl/ssl_engine_kernel.c modules/ssl/ssl_engine_vars.c modules/ssl/ssl_util_ssl.c support/ab.c
Ruediger Pluem wrote: > > On 02/27/2009 06:16 AM, scte...@apache.org wrote: >> Author: sctemme >> Date: Fri Feb 27 05:16:18 2009 >> New Revision: 748396 >> >> URL: http://svn.apache.org/viewvc?rev=748396&view=rev >> Log: >> The development trunk of OpenSSL has tightened up the type safety of the >> STACK construct >> and the functions that manipulate it. Make httpd trunk compile against >> OpenSSL HEAD >> as well as OpenSSL 0.9.8j. Also, get rid of some warnings. I filed a bug (+ patch) about this in August last year: https://issues.apache.org/bugzilla/show_bug.cgi?id=45521 ... and find it rather irritating that the required modifications now appear to have been redone from scratch (not that I'm particularly keen on getting my specific code into the tree, but two persons doing the same within a few months is pretty needless). > Hm. Now I get the following warning with openssl-0.9.8d: > > ab.c: In function 'main': > ab.c:2230: warning: passing argument 1 of 'SSL_CTX_new' discards qualifiers > from pointer target type It seems to me that the changes which constify SSL_CIPHER and SSL_METHOD are not really related to the modifications for additional STACK type safety, or am I missing something? Kaspar
