Re: svn commit: r1725149 - in /httpd/httpd/trunk: CHANGES docs/manual/mod/core.xml include/ap_mmn.h include/http_core.h server/core.c
On 18 Jan 2016, at 1:40 AM, cove...@apache.org wrote: > Author: covener > Date: Sun Jan 17 23:40:09 2016 > New Revision: 1725149 > > URL: http://svn.apache.org/viewvc?rev=1725149&view=rev > Log: > allow expressions to be used in SetHandler. Opt-in with expr= prefix. I am -0.9 on the opt-in with expr= syntax. Almost all the directives with expression support simply support expressions. Except for the one or two that don’t, which forces our end users to either know or constantly check the manual to work out which is which. We can’t even cleanly detect a possible error - if the person puts in an expression by accident where an expr= is needed there is no way the server can detect that. This is really nasty. If we want opt-in expression support we need some kind of Expression +Foo statement to control this. Regards, Graham —
Re: mod_mime_magic, gzipped tarballs and Docker
On 01/08/2016 07:44 PM, William A Rowe Jr wrote:
Do we have to repeat the softmagic call if checkzmagic resolves to
x-gzip/x-deflate and the internal content type needs to be deciphered?
That's true.
I think that Yann's patch moving the zmagic call after the softmagic
call would just mean that zmagic won't be called at all if the softmagic
recognizes the file format.
We would have to check the softmagic result by something similar to
"magic_rsl_to_request" and if it's type we want to decompress, we would
have to run zmagic.
Before really trying to do so, I want to ask if I understand the
reasoning right. Do we consider this way because users can then remove
x-gzip from mime magic and will be able to use it to disable the
mod_mime_magic behaviour discussed in this thread?
Regards,
Jan Kaluza
If so, a tweak to this patch sounds like a winner, and could selectively
recognize different inflation streams including bzip2 etc.
On Fri, Jan 8, 2016 at 10:57 AM, Yann Ylavic mailto:ylavic@gmail.com>> wrote:
On Fri, Jan 8, 2016 at 5:30 PM, Yann Ylavic mailto:ylavic@gmail.com>> wrote:
> On Fri, Jan 8, 2016 at 3:17 PM, William A Rowe Jr mailto:wr...@rowe-clan.net>> wrote:
>>
>> Agreed it is configuration, but cant we simply tweak our recommended
>> conf/magic
>> file???
>>
>> # standard unix compress
>> # Enable the alternate line below to present gzip content as a transfer
>> encoded
>> # stream of the underlying content;
>> #0 string \037\235application/octet-stream
>> x-compress
>> 0 string \037\235application/octet-stream
>>
>> # gzip (GNU zip, not to be confused with [Info-ZIP/PKWARE] zip archiver)
>> # Enable the alternate line below to present gzip content as a transfer
>> encoded
>> # stream of the underlying content;
>> #0 string \037\213application/octet-stream
>> x-gzip
>> 0 string \037\213application/octet-stream
>>
>> WDYT?
>
> I wasn't aware of conf/magic file, but it seems not used by zmagic()
> which hardcodes its types.
> I may be missing something though...
Looks like it is already like this in conf/magic of 2.4.x and 2.2.x.
My bet is that zmagic() is run before softmagic(), so the magic file
is of no help.
Maybe a simpler patch could be:
Index: modules/metadata/mod_mime_magic.c
===
--- modules/metadata/mod_mime_magic.c(revision 1723283)
+++ modules/metadata/mod_mime_magic.c(working copy)
@@ -880,14 +885,6 @@ static int tryit(request_rec *r, unsigned char *bu
int checkzmagic)
{
/*
- * Try compression stuff
- */
-if (checkzmagic == 1) {
-if (zmagic(r, buf, nb) == 1)
-return OK;
-}
-
-/*
* try tests in /etc/magic (or surrogate magic file)
*/
if (softmagic(r, buf, nb) == 1)
@@ -900,6 +897,14 @@ static int tryit(request_rec *r, unsigned char *bu
return OK;
/*
+ * Try compression stuff
+ */
+if (checkzmagic == 1) {
+if (zmagic(r, buf, nb) == 1)
+return OK;
+}
+
+/*
* abandon hope, all ye who remain here
*/
return DECLINED;
--
Re: mod_mime_magic, gzipped tarballs and Docker
Clash of clans
On Jan 8, 2016 8:43 PM, "Jim Jagielski" wrote:
> -0.9...
>
> This seems a very heavy solution to a specific one-off problem.
>
> > On Jan 8, 2016, at 4:27 AM, Yann Ylavic wrote:
> >
> > Hi,
> >
> > On Fri, Jan 8, 2016 at 8:49 AM, Jan Kaluža wrote:
> >>
> >> Content-Type: application/x-tar
> >> Content-Encoding: x-gzip
> > []
> >>
> >> So, the mod_mime_magic is saying here that the body is tarball encoded
> by
> >> gzip.
> >
> > AIUI, mod_mime_magic does indeed try to uncompress the tar.gz and adds
> > the above headers.
> >
> >>
> >> But I think even we are right here, the Content-Encoding should be used
> only
> >> when httpd itself encodes the original content, otherwise it confuses
> >> clients (I've asked few people and it confuses some web browsers too -
> I can
> >> get more info if needed.).
> >
> > Agreed, this looks wrong, the browsers will likely use the .tar file
> > instead (this adds unnecessary cycles to both the server and client).
> >
> >>
> >> Maybe we could stop setting Content-Encoding in mod_mime_magic and just
> use
> >> Content-Type?
> >
> > Since it's always been there, we probably should add a new
> > mod_mime_magic directive to control the behaviour and avoid breaking
> > cases.
> >
> > Something along:
> >
> > Index: modules/metadata/mod_mime_magic.c
> > ===
> > --- modules/metadata/mod_mime_magic.c(revision 1723283)
> > +++ modules/metadata/mod_mime_magic.c(working copy)
> > @@ -456,6 +456,7 @@ typedef struct {
> > const char *magicfile;/* where magic be found */
> > struct magic *magic; /* head of magic config list */
> > struct magic *last;
> > +int uncompress;
> > } magic_server_config_rec;
> >
> > /* per-request info */
> > @@ -511,6 +512,10 @@ static const command_rec mime_magic_cmds[] =
> > {
> > AP_INIT_TAKE1("MimeMagicFile", set_magicfile, NULL, RSRC_CONF,
> > "Path to MIME Magic file (in file(1) format)"),
> > +AP_INIT_FLAG("MimeMagicUncompress", ap_set_flag_slot,
> > + (void *)APR_OFFSETOF(magic_server_config_rec, uncompress),
> RSRC_CONF,
> > + "Whether MIME should try to render uncompressed content by
> recognizing "
> > + "the underlying type, or not (default)"),
> > {NULL}
> > };
> >
> > @@ -2081,6 +2086,9 @@ static int ncompr = sizeof(compr) / sizeof(compr[0
> >
> > static int zmagic(request_rec *r, unsigned char *buf, apr_size_t nbytes)
> > {
> > +magic_server_config_rec *conf = (magic_server_config_rec *)
> > +ap_get_module_config(r->server->module_config,
> > + &mime_magic_module);
> > unsigned char *newbuf;
> > int newsize;
> > int i;
> > @@ -2095,15 +2103,20 @@ static int zmagic(request_rec *r, unsigned char
> *b
> > if (i == ncompr)
> > return 0;
> >
> > -if ((newsize = uncompress(r, i, &newbuf, HOWMANY)) > 0) {
> > -/* set encoding type in the request record */
> > -r->content_encoding = compr[i].encoding;
> > -
> > +if (!conf->uncompress) {
> > +magic_rsl_puts(r, apr_pstrcat(r->pool, "application/",
> > + compr[i].encoding, NULL));
> > +}
> > +else if ((newsize = uncompress(r, i, &newbuf, HOWMANY)) > 0) {
> > newbuf[newsize-1] = '\0'; /* null-terminate uncompressed data */
> > /* Try to detect the content type of the uncompressed data */
> > if (tryit(r, newbuf, newsize, 0) != OK) {
> > +magic_rsl_puts(r, apr_pstrcat(r->pool, "application/",
> > + compr[i].encoding, NULL));
> > return 0;
> > }
> > +/* set encoding type in the request record */
> > +r->content_encoding = compr[i].encoding;
> > }
> > return 1;
> > }
> > --
> >
> > Regards,
> > Yann.
>
>
Re: svn commit: r1725018 - /httpd/httpd/trunk/modules/proxy/mod_proxy_hcheck.c
On 01/16/2016 08:07 PM, j...@apache.org wrote:
> Author: jim
> Date: Sat Jan 16 19:07:54 2016
> New Revision: 1725018
>
> URL: http://svn.apache.org/viewvc?rev=1725018&view=rev
> Log:
> Move to a set of health check workers, mapping to
> each worker itself, instead of a single reused generic
> worker.
>
> Modified:
> httpd/httpd/trunk/modules/proxy/mod_proxy_hcheck.c
>
> Modified: httpd/httpd/trunk/modules/proxy/mod_proxy_hcheck.c
> URL:
> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/mod_proxy_hcheck.c?rev=1725018&r1=1725017&r2=1725018&view=diff
> ==
> --- httpd/httpd/trunk/modules/proxy/mod_proxy_hcheck.c (original)
> +++ httpd/httpd/trunk/modules/proxy/mod_proxy_hcheck.c Sat Jan 16 19:07:54
> 2016
> @@ -57,43 +57,81 @@ static void *hc_create_config(apr_pool_t
> apr_pool_create(&ctx->p, p);
> ctx->templates = apr_array_make(ctx->p, 10, sizeof(hc_template_t));
> ctx->conditions = apr_array_make(ctx->p, 10, sizeof(hc_condition_t));
> -ctx->hc = NULL;
> +ctx->hcworkers = apr_hash_make(ctx->p);
> ctx->s = s;
>
> return ctx;
> }
>
> -static void hc_child_init(apr_pool_t *p, server_rec *s)
> +static proxy_worker *hc_get_hcworker(sctx_t *ctx, proxy_worker *worker)
> {
> proxy_worker *hc = NULL;
> +const char* wptr;
>
> -/* TODO */
> -while (s) {
> -sctx_t *ctx = (sctx_t *) ap_get_module_config(s->module_config,
> - &proxy_hcheck_module);
> -if (!hc) {
> -ap_proxy_define_worker(ctx->p, &hc, NULL, NULL,
> "http://www.apache.org";, 0);
> -PROXY_STRNCPY(hc->s->name, "proxy:hcheck");
> -PROXY_STRNCPY(hc->s->hostname, "*");
> -PROXY_STRNCPY(hc->s->scheme, "*");
> -hc->hash.def = hc->s->hash.def =
> -ap_proxy_hashfunc(hc->s->name, PROXY_HASHFUNC_DEFAULT);
> -hc->hash.fnv = hc->s->hash.fnv =
> -ap_proxy_hashfunc(hc->s->name, PROXY_HASHFUNC_FNV);
> -/* Do not disable worker in case of errors */
> -hc->s->status |= PROXY_WORKER_IGNORE_ERRORS;
> -/* Mark as the "generic" worker */
> -hc->s->status |= PROXY_WORKER_GENERIC;
> -ctx->hc = hc;
> -ap_proxy_initialize_worker(ctx->hc, s, ctx->p);
> -/* Enable address cache for generic reverse worker */
> -hc->s->is_address_reusable = 1;
> +wptr = apr_psprintf(ctx->p, "%pp", worker);
> +hc = (proxy_worker *)apr_hash_get(ctx->hcworkers, wptr,
> APR_HASH_KEY_STRING);
> +if (!hc) {
> +ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, ctx->s, APLOGNO()
> + "Creating hc worker %s for %s://%s:%d",
> + wptr, worker->s->scheme, worker->s->hostname,
> + (int)worker->s->port);
> +
> +ap_proxy_define_worker(ctx->p, &hc, NULL, NULL, worker->s->name, 0);
> +PROXY_STRNCPY(hc->s->name, wptr);
> +PROXY_STRNCPY(hc->s->hostname, worker->s->hostname);
> +PROXY_STRNCPY(hc->s->scheme, worker->s->scheme);
> +hc->hash.def = hc->s->hash.def = ap_proxy_hashfunc(hc->s->name,
> PROXY_HASHFUNC_DEFAULT);
> +hc->hash.fnv = hc->s->hash.fnv = ap_proxy_hashfunc(hc->s->name,
> PROXY_HASHFUNC_FNV);
> +hc->s->port = worker->s->port;
> +/* Do not disable worker in case of errors */
> +hc->s->status |= PROXY_WORKER_IGNORE_ERRORS;
> +/* Mark as the "generic" worker */
> +hc->s->status |= PROXY_WORKER_GENERIC;
> +ap_proxy_initialize_worker(hc, ctx->s, ctx->p);
> +/* Enable address cache for generic reverse worker */
> +hc->s->is_address_reusable = 1;
Hm, shouldn't the HC worker use the same setting as the corresponding worker?
If the workers address is not reusable then worker address might resolve to a
different IP
each time it resolves which could cause the HC worker to probe the wrong
backend.
> +/* tuck away since we need the preparsed address */
> +hc->cp->addr = worker->cp->addr;
> +apr_hash_set(ctx->hcworkers, wptr, APR_HASH_KEY_STRING, hc);
> +}
> +return hc;
> +}
> +
Regards
Rüdiger
Re: svn commit: r1725018 - /httpd/httpd/trunk/modules/proxy/mod_proxy_hcheck.c
> > Hm, shouldn't the HC worker use the same setting as the corresponding worker? > If the workers address is not reusable then worker address might resolve to a > different IP > each time it resolves which could cause the HC worker to probe the wrong > backend. > That is a good point... I was trying to reduce the overhead, but yeah, that makes sense.
Re: mod_mime_magic, gzipped tarballs and Docker
On Mon, Jan 18, 2016 at 5:13 AM, Jan Kaluža wrote: > On 01/08/2016 07:44 PM, William A Rowe Jr wrote: > >> Do we have to repeat the softmagic call if checkzmagic resolves to >> x-gzip/x-deflate and the internal content type needs to be deciphered? >> > > That's true. > > I think that Yann's patch moving the zmagic call after the softmagic call > would just mean that zmagic won't be called at all if the softmagic > recognizes the file format. > > We would have to check the softmagic result by something similar to > "magic_rsl_to_request" and if it's type we want to decompress, we would > have to run zmagic. > > Before really trying to do so, I want to ask if I understand the reasoning > right. Do we consider this way because users can then remove x-gzip from > mime magic and will be able to use it to disable the mod_mime_magic > behaviour discussed in this thread? > Yes... and in a more flexible manner that allows us to override any compression mode, not only deflate, by simply tweaking the magic entries. Putting magic file entry overrides into the mod_mime_magic directives is a lot more interesting than simply toggling compression recognition.
Re: mod_fcgid and broken doc links
On Fri, Jan 15, 2016 at 7:44 AM, Jim Jagielski wrote:
>
> > On Jan 14, 2016, at 5:19 PM, William A Rowe Jr
> wrote:
> >
> > Good point with your example, this is something that should
> > be benchmarked and the winner-take-all, loser bumped from the
> > trunk/ copy of httpd.
>
> -1
>
> You are implying that one would be a winner in all cases. The
> whole idea is that there are cases where one is better than
> the other. We provide both.
>
You might have made that inference, but I'm going to assert that
for this one module, for s/{literal}/{repl}, mod_sed is going to
outperform mod_substitute /if/ we wrote the code correctly. Whas
is missing is a mod_sed directive for adding literal s/{orig}/{repl}/
patterns that follows the substitute syntax and that admins don't
have to wrap their heads around the special character escaping.
At that point, there is no reason for the redundant module, but
we aren't at that point.
Would you also recommend such a scenario with MPMs, and
> having the "winner" take all? I certainly hope not!!!
>
No, I generally agree with TMTOWTDI. Glad you raised the MPMs
issue, because my exact complaint about folding in mod_fcgid
"as is" and overlapping with mod_proxy_fcgi was addressed by the
MPM community in pulling out and sharing the mpm_common.c
logic. Great example, thanks Jim!
Re: svn commit: r1725301 [1/6] - in /httpd/httpd/branches/2.4.x: ./ docs/manual/mod/ modules/http2/
Le 18/01/2016 17:22, ic...@apache.org a écrit : Author: icing Date: Mon Jan 18 16:22:57 2016 New Revision: 1725301 URL: http://svn.apache.org/viewvc?rev=1725301&view=rev Log: mod_http2 v1.2.2 from trunk I guess it is just a typo, but 2.4.x is version 1.2.2 but trunk is 1.2.1. CJ
Re: mod_fcgid and broken doc links
> On Jan 18, 2016, at 3:28 PM, William A Rowe Jr wrote:
>
> On Fri, Jan 15, 2016 at 7:44 AM, Jim Jagielski wrote:
>
> > On Jan 14, 2016, at 5:19 PM, William A Rowe Jr wrote:
> >
> > Good point with your example, this is something that should
> > be benchmarked and the winner-take-all, loser bumped from the
> > trunk/ copy of httpd.
>
> -1
>
> You are implying that one would be a winner in all cases. The
> whole idea is that there are cases where one is better than
> the other. We provide both.
>
> You might have made that inference, but I'm going to assert that
> for this one module, for s/{literal}/{repl}, mod_sed is going to
> outperform mod_substitute /if/ we wrote the code correctly.
I disagree... it's kind of obvious by simple inspection that
mod_substitute has fast paths that mod_sed lacks and, as thus,
can be quite performant and the "better" choice in numerous cases
where that fast path is used.
Me, I don't tend to think of myself as "smarter" than all of
our users, nor do I try to act as Big Brother and remove choices
from people in cases and situations where they are using them.
The ASF itself doesn't do that, nor do projects... so it seems
kinda weird that you would want the httpd project to all of
a sudden start removing choice and options for end users instead
of helping them out and trusting them to know which impl is
best for them.
Re: mod_fcgid and broken doc links
On Mon, Jan 18, 2016 at 3:29 PM, Jim Jagielski wrote:
>
> > On Jan 18, 2016, at 3:28 PM, William A Rowe Jr
> wrote:
> >
> > On Fri, Jan 15, 2016 at 7:44 AM, Jim Jagielski wrote:
> >
> > > On Jan 14, 2016, at 5:19 PM, William A Rowe Jr
> wrote:
> > >
> > > Good point with your example, this is something that should
> > > be benchmarked and the winner-take-all, loser bumped from the
> > > trunk/ copy of httpd.
> >
> > -1
> >
> > You are implying that one would be a winner in all cases. The
> > whole idea is that there are cases where one is better than
> > the other. We provide both.
> >
> > You might have made that inference, but I'm going to assert that
> > for this one module, for s/{literal}/{repl}, mod_sed is going to
> > outperform mod_substitute /if/ we wrote the code correctly.
>
> I disagree... it's kind of obvious by simple inspection that
> mod_substitute has fast paths that mod_sed lacks and, as thus,
> can be quite performant and the "better" choice in numerous cases
> where that fast path is used.
>
Well, only benchmarking is going to prove that one way or the other,
something I don't have free cycles for right now (committed to way too
many other project priorities.) And it perhaps opens up opportunities
to optimize mod_sed in ways that might have been initially overlooked
when the code was thrown into trunk ;-)
> Me, I don't tend to think of myself as "smarter" than all of
> our users, nor do I try to act as Big Brother and remove choices
> from people in cases and situations where they are using them.
> The ASF itself doesn't do that, nor do projects... so it seems
> kinda weird that you would want the httpd project to all of
> a sudden start removing choice and options for end users instead
> of helping them out and trusting them to know which impl is
> best for them.
>
Thankfully, we aren't, but our users do look at us as experts in the
code they consume, considering that we collectively have authored
and maintain the stuff. So they do look to us to make the best
choices they don't have the individual time to compare and elect.
Why would I want us to collectively (and for me specifically) to
propose the best solutions to the common use cases, and to
depreciate less maintained code in favor of maintained code?
Can't imagine why I or other project members would be possessed
to do that. If you figure it out, please share :)
