Re: strcasecmp raises its...
> On 18 May 2022, at 16:34, Ruediger Pluem wrote: > > Rüdiger What locale are YOU in there? Any attempt at locale is going to have to draw lines: what are the rules for when Ruediger == Rüdiger? In a WWW (and hence httpd) context, internationalised domain names raise all kinds of issues, including for us potentially breaking case-insensitivity rules in matching hostnames, and perhaps other configuration matters. What happens if we make locale a configurable parameter for hostnames and use strcasecmp_l? -- Nick Kew
Re: svn commit: r1884505 - in /httpd/httpd/trunk: changes-entries/pr64339.txt modules/filters/mod_xml2enc.c
On Thu, 17 Dec 2020 17:31:20 +
Nick Kew wrote:
> > On 17 Dec 2020, at 16:22, Joe Orton wrote:
> [chop]
> Thanks for prompting me to take a proper look at where this thread
> started.
Further thought: anything hardwired will come up against future
use cases where it acts contrary to expectations and indeed
requirements. A future-proof solution is to make it configurable.
I attach a patch I've just hacked (untested): if folks are happy
with the approach I'll flesh it out and commit.
--
Nick Kew
Index: mod_xml2enc.c
===
--- mod_xml2enc.c (revision 1884632)
+++ mod_xml2enc.c (working copy)
@@ -84,12 +84,19 @@
apr_bucket_brigade* bbnext;
apr_bucket_brigade* bbsave;
const char* encoding;
+int checkedmime;
} xml2ctx;
typedef struct {
+ int onoff;
+ const char *ctype;
+} mimetype;
+
+typedef struct {
const char* default_charset;
xmlCharEncoding default_encoding;
apr_array_header_t* skipto;
+apr_array_header_t* mimetypes;
} xml2cfg;
typedef struct {
@@ -331,6 +338,7 @@
int pending_meta = 0;
char *ctype;
char *p;
+xml2cfg *cfg;
if (!ctx || !f->r->content_type) {
/* log error about configuring this */
@@ -338,15 +346,38 @@
return ap_pass_brigade(f->next, bb) ;
}
-ctype = apr_pstrdup(f->r->pool, f->r->content_type);
-for (p = ctype; *p; ++p)
-if (isupper(*p))
-*p = tolower(*p);
+if (!ctx->checkedmime) {
+cfg = ap_get_module_config(f->r->per_dir_config, _module);
+ctype = apr_pstrdup(f->r->pool, f->r->content_type);
+for (p = ctype; *p; ++p)
+if (isupper(*p))
+*p = tolower(*p);
-/* only act if starts-with "text/" or contains "+xml" */
-if (strncmp(ctype, "text/", 5) && !strstr(ctype, "+xml")) {
-ap_remove_output_filter(f);
-return ap_pass_brigade(f->next, bb) ;
+/* Check whether we're configured to act on this MIME type */
+if (cfg->mimetypes != NULL) {
+mimetype *conftypes = (mimetype*)cfg->mimetypes->elts;
+int i;
+for (i = 0; i < cfg->mimetypes->nelts; ++i) {
+if (!strncmp(conftypes[i].ctype, ctype,
+strlen(conftypes[i].ctype))) {
+if (conftypes[i].onoff == 0) {
+ap_remove_output_filter(f);
+return ap_pass_brigade(f->next, bb) ;
+}
+ctx->checkedmime = 1;
+break;
+}
+}
+}
+
+/* else only act if starts-with "text/" or contains "xml" */
+if (!ctx->checkedmime) {
+if (strncmp(ctype, "text/", 5) && !strstr(ctype, "xml")) {
+ap_remove_output_filter(f);
+return ap_pass_brigade(f->next, bb) ;
+}
+ctx->checkedmime = 1;
+}
}
if (ctx->bbsave == NULL) {
@@ -637,6 +668,27 @@
attr->val = arg;
return NULL;
}
+static const char* set_mimetype(cmd_parms* cmd, void* CFG, const char* arg)
+{
+mimetype* type;
+xml2cfg* cfg = CFG;
+if (cfg->mimetypes == NULL)
+cfg->mimetypes = apr_array_make(cmd->pool, 6, sizeof(mimetype));
+type = apr_array_push(cfg->mimetypes);
+if (arg[0] == '-') {
+type->onoff = 0;
+type->ctype = arg+1;
+}
+else if (arg[0] == '+') {
+type->onoff = 1;
+type->ctype = arg+1;
+}
+else {
+type->onoff = 1;
+type->ctype = arg;
+}
+return NULL;
+}
static const command_rec xml2enc_cmds[] = {
AP_INIT_TAKE1("xml2EncDefault", set_default, NULL, OR_ALL,
@@ -645,6 +697,8 @@
"EncodingAlias charset alias [more aliases]"),
AP_INIT_ITERATE("xml2StartParse", set_skipto, NULL, OR_ALL,
"Ignore anything in front of the first of these elements"),
+AP_INIT_ITERATE("xml2MimeType", set_mimetype, NULL, OR_ALL,
+"List MIME types to process or leave untouched"),
{ NULL }
};
static void* xml2enc_config(apr_pool_t* pool, char* x)
@@ -664,6 +718,7 @@
ret->default_charset = add->default_charset
? add->default_charset : base->default_charset;
ret->skipto = add->skipto ? add->skipto : base->skipto;
+ret->mimetypes = add->mimetypes ? add->mimetypes : base->mimetypes;
return ret;
}
Re: svn commit: r1884505 - in /httpd/httpd/trunk: changes-entries/pr64339.txt modules/filters/mod_xml2enc.c
> On 17 Dec 2020, at 16:22, Joe Orton wrote:
>
> On Wed, Dec 16, 2020 at 07:41:59PM +0000, Nick Kew wrote:
>>> On 16 Dec 2020, at 17:47, Yann Ylavic wrote:
>>>> Wouldn't this stop matching "application/xml" for instance?
>>>>
>>>> Possibly this test instead:
>>>> if (strncmp(ctype, "text/", 5)
>>>> && (!(x = strstr(ctype, "xml"))
>>>> || x == ctype || !strchr("/+", x[-1]))) {
>>>> ?
>>>
>>> I would even remove the "text/" check (why act on "text/plain" for
>>> instance), so maybe:
>>> if (!(x = strstr(ctype, "xml"))
>>> || x == ctype || !strchr("/+", x[-1])
>>> || apr_isalnum(x[3])) {
>>> ?
>>
>> Be liberal in what you accept. You can limit it further in configuration,
>> but you can't override a hardwired check.
>>
>> It certainly needs to operate on text/html for mod_proxy_html, and users
>> might
>> find reasons for running it on other text types as an alternative to an iconv
>> filter like mod_charset(_lite).
>
> I'm not sure if you are agreeing with Yann or not. You wrote the code,
> how do you think we should you resolve PR 64339, NOTABUG & revert
> r1884505 or something else?
Thanks for prompting me to take a proper look at where this thread started.
I started to compose a reply here, bug got bogged down. So I've gone to the
PR instead. I see there's a patch submitted by Giovanni Bechis which looks
more-or-less acceptable, though it does raise further questions.
I've marked the bug NEEDINFO and asked further questions there.
--
Nick Kew
Re: svn commit: r1884505 - in /httpd/httpd/trunk: changes-entries/pr64339.txt modules/filters/mod_xml2enc.c
> On 16 Dec 2020, at 17:47, Yann Ylavic wrote:
>
> On Wed, Dec 16, 2020 at 6:36 PM Yann Ylavic wrote:
>>
>> On Wed, Dec 16, 2020 at 5:23 PM wrote:
>>>
>>> -/* only act if starts-with "text/" or contains "xml" */
>>> -if (strncmp(ctype, "text/", 5) && !strstr(ctype, "xml")) {
>>> +/* only act if starts-with "text/" or contains "+xml" */
>>> +if (strncmp(ctype, "text/", 5) && !strstr(ctype, "+xml")) {
>>> ap_remove_output_filter(f);
>>> return ap_pass_brigade(f->next, bb) ;
>>> }
>>
>> Wouldn't this stop matching "application/xml" for instance?
>>
>> Possibly this test instead:
>>if (strncmp(ctype, "text/", 5)
>>&& (!(x = strstr(ctype, "xml"))
>>|| x == ctype || !strchr("/+", x[-1]))) {
>> ?
>
> I would even remove the "text/" check (why act on "text/plain" for
> instance), so maybe:
>if (!(x = strstr(ctype, "xml"))
>|| x == ctype || !strchr("/+", x[-1])
> || apr_isalnum(x[3])) {
> ?
Be liberal in what you accept. You can limit it further in configuration,
but you can't override a hardwired check.
It certainly needs to operate on text/html for mod_proxy_html, and users might
find reasons for running it on other text types as an alternative to an iconv
filter like mod_charset(_lite).
--
Nick Kew
Re: APLOGNO number range for vendors?
On Tue, 1 Dec 2020 14:33:29 + Joe Orton wrote: > Very occasionally we backport patches to RHEL's httpd package in a > way that introduces new or different logging output from 2.4/trunk. > I'm wondering if there is any opinion about vendors asking for for a > small (say, 100?) reserved range of APLOGNO() space to use for such > cases? Basically I'd just commit "next-number += 100" and use that > range within downstream patches since they are then reserved upstream. Is there any possibility that a vendor-RH message leaking upstream to us - i.e. you introduce something RH and later commit it to trunk? If so, would you want to preserve the vendor number or just reassign a new one? No strong views, but if pressed I'd +1 to Stefan's suggestion. Simple, and something you can do without touching our repos. -- Nick Kew
Re: Concurrent access to request body by multiple modules
> On 1 Dec 2020, at 14:29, Tim Wetzel wrote: > > Is there any possibility of having the module use, for example, a bucket > brigade or anything I am not aware of that is already in use by WebDAV and > thus operate on the request body of a file from a PUT request in > parallel/concurrently to WebDAV? How does mod_request relate to your needs? If you can't use it (e.g. because you have to deal with arbitrary-sized requests), your choices would then be to adapt it (e.g. provide for it to use a file bucket to cache requests) or to go a level down and write your own input filter module. That is, if I've understood you aright? -- Nick Kew
Re: mod_proxy_spawn: Review request
> On 29 Nov 2020, at 14:48, Florian Wagner wrote: > > Hi everyone! > > I was wondering if someone with a better understanding of httpd and > mod_proxy could review my module idea and prototype implementation and > warn me of any unforeseen pitfalls looming ahead before I commit to > implementing this fully. The main thing that springs to mind is security when you spawn a process. CGI and fastcgi are battle-hardened by decades of operational use. > Wanting to switch to mod_proxy_http for deploying backend applications > (and opening the way for WebSockets through mod_proxy_wstunnel) I'm > missing the process management provided by mod_fastcgi [1]. fastcgi is an unusual model, with a very specific purpose. It's more usual for backends to be self-managing. > While fully understanding that one can be of the opinion that process > management is best kept out of httpd, I personally like the convenience > and more importantly clarity offered by having the complete command, > arguments and environment required to run the backend application in > the httpd configuration. Authentication, URL rewriting and whatelse > will already be setup there, anyway. If it's in the config then at least it's [probably] coming from a trusted source, but then why run per-request? > So I took a shot at seeing if I could implement a module to do just > that. My current idea/prototype: > > 1. Register a hook to run before mod_proxy.c:proxy_handler and have a >look at the request filename and handler to see if they start with >"proxy:spawn://". Big red flag for security there. Hope you're paying very careful attention to your input: there's nothing to that effect in what you attached. Also I'd consider hooking it earlier in the request cycle, or into mod_proxy instead. How does mod_proxy_fcgi fit your vision? -- Nick Kew
Re: Which programming language should be used for newly developed modules?
On Thu, 20 Aug 2020 21:59:50 +0200 Simon Studer wrote: > Hi everyone, > > Recently, I was wondering which programming language should be used > for new Apache httpd modules. Whatever suits the module's developer and task. The C API has a stability promise: if you use it, your module will continue to work with (at least) future 2.4.x releases. That give you C or any language with C linkage. If you deviate from the API, you're on your own. Alternatives that (broadly speaking) wrap the C API are also possible: see for example mod_perl and mod_lua. -- Nick Kew
Re: httpd @ ApacheCon
> On 5 Aug 2020, at 20:06, Rich Bowen wrote: > > Hi, folks, > > I don't know if Jim is on vacation, or otherwise offline, but he's not > responding to my pings, so I was hoping someone else here could step up for > something. Shouldn't this at least have been on pmc@ ? > We have 8 submissions for a httpd track at ApacheCon. It would be cool to > have someone to review these and select some or all of them to run. Some of > them seem like clearly things we want. Others I am not familiar enough with > the topic to say. I may or may not have any insights you don't, but I'll be happy to take a look. That could be this evening. -- Nick Kew
Re: RFC: Documenting changes in the CHANGES file
> On 1 Jun 2020, at 13:33, Graham Leggett wrote: > > On 29 May 2020, at 21:30, Ruediger Pluem wrote: > >> changes-fragments/ >>2.4.41/ >>2.4.42/ >>2.4.43/ >>2.4.44/ And a current/ as symlink? > I’m keen for a simpler version of this that doesn't create additional steps > for people. Not sure that's simpler, though it too has potential. Delete-after-append seems tidier than keeping forever-fragments. I wonder if this could be hooked into SVN? Something like an @CHANGES tag in a svn commit message? -- Nick Kew
Re: Can github activity (new PRs, comments) be forwarded to dev@ ?
> On 28 Apr 2020, at 12:49, Yann Ylavic wrote: > > That'd help being more reactive there, avoid having multiple places to > follow, avoid tenchnical discussions taking place outside our mailing > lists... > > Probably more a question for infra, but what does the team think? Please, no! We have commits arriving at cvs@httpd (dammit, how long ago was it we moved from cvs to svn?), bugs, and some less-used lists. If you like to see them all in one place, get your procmail (or whatever) to sort it locally into the same folder! A mailbox full of messages "from" Gitbox and without meaningful subject lines just provokes [select all] --> delete. -- Nick Kew
Re: "Forbid" directive in core?
> On 27 Apr 2020, at 16:37, Eric Covener wrote: > > > Bumping a very old thread. tl;dr people are often surprised that when > Location sections have access control directives and overlap with the > filesystem it undoes the default > >Require all denied > We always warn against mixing with filesystem. That's just one of many gotchas that may arise from it. I wonder if a better solution would be to be much stricter about the division? Ideally make it a config error (switchable to a warning so as not to break too many old configs) to overlap them? A first-pass implementation might be for the Location directive to issue a warning if it matches a directory in the filesystem. -- Nick Kew
Re: Trying to compile httpd trunk
> On 17 Dec 2019, at 17:46, Stéphane Blondon wrote: > > Hello, > > I fail to compile apache2 from trunk on Debian testing. I take it you have a good reason to want that rather than a packaged version such as Debian's? What did buildconf tell you? > According to the INSTALL file, I check outed `apr` trunk in > srclib/apr. After adding several lacking packages, it seems > ./configure is ok. (The full output is in attachment.) Attachment TLDR. Was that apr configure or httpd configure? > $ ./configure --prefix=/home/stephane/bin/http --with-included-apr Not something I'd try with a non-release version. Decide what apr you want (either trunk or 1.x should work), install that first, then use that to build your httpd. -- Nick Kew
Re: Migrate to git?
> On 7 Oct 2019, at 15:06, Daniel Gruno wrote: > > On 06/10/2019 17.59, Nick Kew wrote: >> >> OK, I've just dug up an example in an Apache/Github project. A simple >> renaming >> of a source file, that with "svn mv" would have preserved history, seems to >> have >> essentially wiped its past. 'History' is highly misleading, 'Blame' is 100% >> wrong! > > It would be 100% wrong in svn as well if the same operation had been > performed there, as it wasn't a move - the number of lines don't match up. > There is a `git mv` just like `svn mv` that preserves history, AIUI. A file > where `svn mv` was actually used [1] shows that the history is preserved > through the mv operation and blame works as intended, even in git. Which points to another problem. The committer concerned wasn't a git newbie, he's been happily using it for (I know not how long) before the project ever came to Apache. And I can't believe the renaming was malicious. So why did he get it wrong in this manner? Perhaps it points to the real-life complexity of git, and its real-life consequences? I've got caught on that myself (in a non-Apache context) before: things like git merge of a branch having spectacularly unintended consequences for what history subsequently looks like. Indeed, perhaps the cause of the screw-up wasn't really the renaming, but some subsequent change whose author had never even looked at the source file in question? If you look at history, many of the changes there seem to be mass formatting-exercises. The line count appears to be an artifact of the committer's toolset, which includes some auto-formatting such as -extern "C" void TSPluginInit(int argc, const char *argv[]) +extern "C" void +TSPluginInit(int argc, const char *argv[]) > As Eric alluded to, it's much less about svn versus git than it is about > tapping into the community on GitHub. If there was an svnhub, I'd be all for > that as an easier way to do this, but alas no. So the Apache community isn't good enough and we need Microsoft? OK, that's not quite fair. But isn't that what the github mirror is for? -- Nick Kew
Re: Migrate to git?
> On 6 Oct 2019, at 04:06, Daniel Gruno wrote: > > On 05/10/2019 19.30, Nick Kew wrote: >>> >> If it moves to github, how and at what level is history preserved? Github >> can do >> alarming things with history even for a project that's always been there! > > We would have the exact same level of history as before (one might even say > we'll get more history, as you can specify committer and author separately in > git). If you look at https://github.com/apache/httpd which is our current git > mirror, it should have the exact same commits going back to 1996 as the > subversion repository. There is a bit of a lag on the mirror right now, but > that is a separate issue that will be fixed on October 12th. OK, I've just dug up an example in an Apache/Github project. A simple renaming of a source file, that with "svn mv" would have preserved history, seems to have essentially wiped its past. 'History' is highly misleading, 'Blame' is 100% wrong! https://github.com/apache/trafficserver/blob/master/plugins/experimental/stream_editor/stream_editor.cc And that's within git: no actual change-of-repos involved. Regarding httpd, we have the git mirror, so access is available through whatever a contributor prefers. How is that not best-of-both-worlds? -- Nick Kew
Re: Migrate to git?
> On 5 Oct 2019, at 21:09, Jim Jagielski wrote: > > Various PMCs have made their default/de-facto SCM git and have seen an > increase in contributions and contributors... > > Is this something the httpd project should consider? Especially w/ the > foundation officially supporting Github, it seems like time to have a > discussion about it, especially as we start thinking about the next 25 years > of this project :) > > Cheers! [apologies if this appears twice. Just sent with wrong from: address so I expect apache to bounce it. I'm still on limited 'net connectivity since my house move - ISP due on Oct 14th to install proper connection]. If it moves to github, how and at what level is history preserved? Github can do alarming things with history even for a project that's always been there! Don't we have an svn-git gateway? If that's not best-of-both-worlds, why not? -- Nick Kew
Re: ACME support in Apache web server
> On 13 Jun 2019, at 16:49, Abul Salek wrote: > > Hello, > > Is there a plan to support the “external account binding” feature of ACME > (RFC 8555) natively in the Apache web server? I'm not familiar with the RFC in question, so I may be missing something. But in general terms, that sounds like the kind of thing that could comfortably be implemented as a module. Is that your plan if you don't find an existing implementation? -- Nick Kew
Re: http://svn.apache.org/r1850745
> On 13 Mar 2019, at 12:43, Jim Jagielski wrote: > > Is there anyone else building 2.4 on macOS under maintainer-mode > who is also being affected by the above? The fact that I seem to > be the anyone "complaining" :) seems weird... > > Thx! Just out of interest, is that with a libxml2-enabled APR version? Guess I need to test-drive that on Mac/latest, which has bitten me on similar platform issues before now! -- Nick Kew
Re: Patches related to HTML output by Apache httpd itself
> On 6 Dec 2018, at 11:34, Andras Farkas wrote: > > Ping. > On Mon, Nov 26, 2018 at 4:08 AM Andras Farkas > wrote: >> >> Evening! >> >> I noticed that most of the time, when Apache httpd itself generates >> HTML output (like for 404 pages and autoindex pages) it uses ancient >> HTML 2.0 and HTML 3.2 doctypes. >> These 11 attached diffs update those. The most important of the diffs >> is httpdh.diff These would be better submitted to bugzilla at issues.apache.org, where someone might adopt them. However, I should point out that there's nothing wrong with the existing stuff. If I were to refactor the error messages, I'd be looking to take all remaining actual HTML out of the server itself, and into documents (or templates) under the control of the sysop. (And I wouldn't touch your indexhtml with a bargepole)! -- Nick Kew
Re: [VOTE] Release httpd-2.4.35
On Mon, 17 Sep 2018 19:56:52 -0500 Daniel Ruggeri wrote: > Hi, all; > Please find below the proposed release tarball and signatures: > https://dist.apache.org/repos/dist/dev/httpd/ > Builds fine (once I'd found a keyserver that would talk to my gpg today), but the tests bombed on me. Unlikely to have time to dig into that before at least Friday, so no vote likely unless you leave it open all week or thereabouts. -- Nick Kew
Re: Write a module to overwrite HTTP methods handling
> On 19 Sep 2018, at 13:57, Danesh Daroui wrote: > > Hi all, > > I am sorry if my question is a bit off! I am wondering whether it is > possible to write a server module that can handle different HTTP > methods in a customized way? Of course it's possible, and indeed common: see for example mod_dav, or even the oldest application module of all, mod_cgi. However, your question leads me to wonder if what you're looking for might not be better accomplished in configuration. If you need functionality that isn't already supported, that would call for a new module (or equivalent), but you'd use configuration to determine when it should or shouldn't be invoked to process a request. -- Nick Kew
Re: How to read data in a request handler and then return DECLINED without consuming the data in the bucket brigade?
On Mon, 4 Jun 2018 10:23:59 -0700 Paul Callahan wrote: > Thank you for your replies. > > I did try with input filters. The reason I'm trying to do this in a > handler is because I want to return 403 to the browser if the request > body has something unsavory in it. With reverse proxied requests, > it appears the input filter fires too late and if I try to send a > bucket with 403 in it, it is ignored and a 400 goes back to the > browser. In the debugger, I see the fixup call back being hit > before my input filter. If I could get the input filter to trip > sooner without consuming the request, I could go with that. If I > call ap_get_brigade() in an earlier handler to trip the input filter, > it appears the request body is consumed. OK, that's actually quite a complex task, especially if you need to deal with larger requests. It is, however, as task that's been done in open source code you can look at, or perhaps use instead of reinventing their wheel. Either Ironbee or mod_security will scan a request body for you. > btw, Nick I bought your book - it was a great help :) Thanks :) -- Nick Kew
Re: How to read data in a request handler and then return DECLINED without consuming the data in the bucket brigade?
> On 4 Jun 2018, at 08:55, Sorin Manolache wrote:
>
> On 2018-06-04 08:27, Paul Callahan wrote:
>> In apache modules, my understanding is if a handler declines a request, the
>> request is passed on to the next suitable handler. I'm finding though if
>> I read the bucket_brigade/request body, and then decline the request, the
>> subsequent handler doesn't get any data. It is like the act of reading the
>> bucket brigade consumes it.
>> I would like to have a request handler read the data, do some task (in this
>> case just count bytes), and decline the request without consuming the data
>> for the next handler.
Why is that a handler? An input filter could count the data and pass them
straight down the chain, avoiding any such problem. At a glance, your code
would need very little modification to work as a filter.
Exhortation: work with the server architecture, not against it!
> Hello,
>
> As far as I know, there is no simple way to do that.
This is true, in the sense that data get consumed. To do otherwise would be
monstrously inefficient, and turn a big request body straight into a DoS.
This is why we have mod_request. It provides an input filter you can use
explicitly to buffer data which then remain available to the next module to
read them.
Use with caution: for example, if you fail to limit request size, you could find
yourself trying to buffer gigabytes of request data.
>> } while (!end && (status == APR_SUCCESS));
>> if (status == APR_SUCCESS) {
>> return DECLINED;
>> } else {
>> return HTTP_INTERNAL_SERVER_ERROR;
>> }
>> }
Minor tip there: you're turning EAGAIN into a fatal error.
--
Nick Kew
Re: mod_proxy_html and special characters
>> ctx->buf = http://internal/!%22%23$/ >> m->from.c = http://internal/!"#$/ A further thought arising from that. Just as strcasecmp is case-independent, the world could no doubt use a standard library function that would treat the above as equal. Something like int stringcmp(const char *a, const char *b, unsigned int flags) where flags would control behaviour such as case-independence, and equivalence over URLencoding, HTML encoding, HTML entities, and whatever else someone might like to support (maybe integrate with locale too?). Anyone know of such a thing? -- Nick Kew
Re: mod_proxy_html and special characters
> On 28 May 2018, at 08:50, Micha Lenk <mi...@lenk.info> wrote: > > The reason I am asking this is, because for Location matching, Apache httpd > apparently does map a request with a URL encoded path to the non-encoded > configured path. For example, if I have configured in a virtual host: Yes of course httpd deals with encoding, as it must, in processing a request URL. > >ProxyPass "http://internal/!\"#$/; >ProxyHTMLURLMap "http://internal/!\"#$/; "http://external/!\"#$/; >... > mod_proxy_html is not processing a request URL, it's processing contents in the response. Contents destined, and encoded, for a HTTP Client. The resemblence is entirely coincidental. To align the behaviour on grounds of consistency would seem to me misleading! -- Nick Kew
Re: mod_proxy_html and special characters
> On 25 May 2018, at 17:43, Micha Lenk <mi...@lenk.info> wrote:
>
> 524 s_from = strlen(m->from.c);
> 525 if (!strncasecmp(ctx->buf, m->from.c, s_from)) {
> ... ... do the string replacement ...
>
>
> ... where ctx->buf is the URL found in the HTML document, and m->from.c is
> the first configured argument of ProxyHTMLURLMap. So, if the latter is a
> prefix of the first, this condition should be true and the string replacement
> should happen. When the expected string replacement doesn't happen, the
> condition is false and the values of the variables are:
>
> ctx->buf = http://internal/!%22%23$/
> m->from.c = http://internal/!"#$/
>
> So, the strings don't match and are not replaced for that reason.
Yep. mod_proxy_html takes what it sees. That's why it relies on another module
(mod_xml2enc) for i18n, which is kind-of what I expected to see from your
subject line!
> Going forward I am not interested in finding a work around for this, but more
> how to approach a fix (if this is a bug at all).
>
> Is it reasonable to expect mod_proxy_html to rewrite URL encoded URLs as well?
I think it's reasonable to use the escaped html in your ProxyHTMLURLMap.
If we have mod_proxy_html unescape characters, it adds complexity to the code,
and (perhaps more to the point) presents a mirror-image of your problem to
anyone with the opposite expectations.
> Let's assume this needs to be fixed. To make the strings match, we could
> either URL escape the value from the Apache directive ProxyHTMLURLMap, or URL
> temporarily URL-decode the string found in the HTML document just for the
> purpose of the string comparison. What is the right thing to do?
I prefer to leave it to server admins to find the match that works for them.
I don't recollect this particular question ever arising in 15 years, which
kind-of
suggests users are not confused by it!
--
Nick Kew
Re: [VOTE] Allow for defect fix releases at httpd
> On 2 May 2018, at 15:45, Micha Lenk <mi...@lenk.info> wrote: > > Hi Graham, > > On 05/01/2018 04:33 PM, Graham Leggett wrote: >> What has been missing is input from the major distributors of our >> software (Fedora, Ubuntu, Redhat, Debian, Apple, Windows, Linux from >> Scratch, etc), who I believe are probably going “httpd is a mature >> project, we have nothing to worry about”. I would recommend against >> making changes to our approach without soliciting the views of these >> people and making sure they’re all catered for. > Why would you make a proposed change dependent on the (almost necessarily > contradicting) views of external entities? Is the feedback from the major > distributors through existing channels really so bad that the httpd project > can't get to an opinion of what it would like to accomplish on its own? What > exactly are you afraid of? +1 to Jim’s reply to that (insofar as it addresses your point). I’d also add: this mailinglist is open to our downstream packagers. Some of them contribute actively, and one would hope they all at least keep an eye on this list and would speak up if a proposal seemed likely to cause them real difficulties. So open discussion here *should* provide a reasonable level of engagement with our distributors. — Nick Kew
Re: [VOTE] Allow for defect fix releases at httpd
> That's not quite fair. > > For me, to be honest, I couldn't quite understand the question at > all... I had a real hard time parsing it. It looked like, by voting +1, > I would also be agreeing to other things (like disallowing > any new features or enhancements to any release) which > would be unacceptable. +1. I’d be uneasy about that clause without a much more in-depth review of its context, which isn’t going to work as a mailinglist discussion (too confusing; TL;DR). At the same time, I applaud what Bill is trying to do. We have a problem, we discuss it, the discussion goes nowhere, Bill makes a valiant effort to take it somewhere. But the context is complex: an existing process, multiple overlapping mailinglist discussion threads, multiple candidate ideas. And I’m not convinced the proposed clause actually resolves the issues: it may just leave us with a more complex process. Sorry if the above is negative. I promise to try and contribute a positive suggestion! — Nick Kew
Re: "Most Popular Web Server?"
> On 19 Apr 2018, at 10:14, Luca Toscano <toscano.l...@gmail.com> wrote: > > Hi Nick, [chop] Thanks. Good reply. Your suggestions make a lot of sense to me: I just wouldn’t have put them in the context of marketing or evangelism. Trouble is, it’s relatively few of us who ever get inspired to write about things. Honourable exception being Rich writing docs and books for longer than anyone can remember! I think the last person to write serious developer documentation was Humbedooh, whose work you deservedly praise: > Ideally the quality bar that I would love to have across our dev docs is this > one http://httpd.apache.org/docs/trunk/developer/modguide.html, but in my > opinion there is still a a lot of work to do :) If you want to get writing at a serious level, that’ll be great! I might even contribute if you can get some momentum going, but I’d never attempt to take a lead, not least because potential conflict-of-interest with my publisher’s copyright. — Nick Kew
Re: "Most Popular Web Server?"
> On 18 Apr 2018, at 20:00, Luca Toscano <toscano.l...@gmail.com> wrote: > > Before joining the httpd project as contributor I struggled to find good > technical sources about how the httpd internals work, Likewise. That’s kind-of what motivated me to start writing about it. But that’s not to say it’s any worse than other software projects I’ve encountered over the years. There’s always a learning curve, and a struggle to find relevant docs. OK, things have improved a lot since “just google it” became an option, but information still needs unearthing. Are you suggesting httpd is somehow *worse* than other software you’ve hacked in terms of developer documentation? In my experience it’s actually a lot better than most, due primarily to the high standard of API docs in /include/ and in APR, and of course open and searchable source. The contrast is closed source software, where docs inevitably diverge badly from reality. I’ve mused about this in the past: for example https://bahumbug.wordpress.com/2006/11/06/the-documentation-gap/ https://bahumbug.wordpress.com/2008/09/16/security-by-cookery/ > My point is: blogging is fine, but before even starting that I'd focus on > dumping everybody's knowledge in sections of the docs like > http://httpd.apache.org/docs/trunk/developer. It is boring and less fun than > writing C code for sure, but I bet that a ton of people would enjoy details > about how things work. It will be easier for people to spot "liars" in the > web that focus their marketing strategy only on how httpd is "old" and not > performant too.. I’ve called out “liars” once or twice. Or more usually, purveyors of “cargo-cult” whose idea of Apache is rooted in how things haven’t been since 1997 or so. But I’m not sure they’re really the issue. nginx has risen primarily because it’s a genuine solution, and secondarily because it’s had the evangelical community that goes with a challenger against an incumbent. Now that it’s risen to be a competitor on more equal terms, the evangelism still has momentum. Insofar as we care about market share, we could respond in kind, preferably avoiding the wilder fringe. — Nick Kew
Re: Expanding httpd adoption internationally
> I suspect the straightforward way to do this, in 2.6/3.0, will be to add an > i18n table of the error log strings extracted from and indexed by those > APLOGNO() entries. No match? Default English message. Please, not without an overhaul of APLOGNO to automate it a lot better! The time to devise your error message is when coding an original ap_log_*error, not retrospectively once a number has been generated! In fact, it would call for a refactoring of the entire ap_log framework away from printf-style format strings and vars, that’ll inevitably give rise to a whole new level of awkwardness. Which seems like a very bad effort-to-gain ratio. -1 to embarking on this, unless you want to create a new experimental branch to play in. BTW, my previous reply in this thread was predicated on your reference to “specific nginx FUD”. I misread your post as describing some FUD. Come to think of it, from memory of developing nginx modules, their error message framework looks a lot like ours. It's in english too. Though I never delved into its internals. — Nick Kew
Re: Expanding httpd adoption internationally
> On 18 Apr 2018, at 17:55, William A Rowe Jr <wr...@rowe-clan.net> wrote: > > So I'll start with this; Erm, would you like to cite a source for that claim? I confess it’s not one I’ve seen. I don’t follow either nginx marketing nor any fan club they might have. If what you refer to is the latter, it’s natural for any incumbent market-leader to feature in such unflattering comparisons, while challenger communities have more tendency to be evangelical. Though nowadays nginx should be up there with us on the wrong side of challenger comparisons! — Nick Kew
Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)
> On 14 Apr 2018, at 14:48, Jim Jagielski <j...@jagunet.com> wrote: > > IMO, the below ignores the impacts on OS distributors who > provide httpd. We have seen how long it takes for them > to go from 2.2 to 2.4... I can't imagine the impact for our > end user community if "new features" cause a minor > bump all the time and we "force" distributions for > 2.4->2.6->2.8->2.10… Chicken httpd version numbers creep in a petty pace from year to year, and packagers do likewise. Contrast a product like, say, Firefox, where major versions just whoosh by, and distros increment theirs every few months. > Just my 2c Indeed, a change needs to be a considered thing, and there are issues. >> On Apr 13, 2018, at 2:28 PM, David Zuelke <dzue...@salesforce.com> wrote: >> >> Remember the thread I started on that quite a while ago? ;) Nope. >> - x.y.0 for new features >> - x.y.z for bugfixes only >> - stop the endless backporting >> - make x.y.0 releases more often An issue there is the API/ABI promise. We are a stable product, and one of our virtues is the guarantee that a third-party module written for x.y.z will continue to work at both source and binary level for x.y.(z+n). Frequent x.y.0 releases devalue that promise unless we extend it to x.(y+m).*, which would in turn push us into new x.0.0 releases, and raise new questions over the whole organisation of our repos. I’m not saying you’re wrong: in fact I think there’s merit in the proposal. But it would need a considered roadmap from here to there. — Nick Kew
Re: URL's in error pages
> On 12 Apr 2018, at 12:46, Eric Covener <cove...@gmail.com> wrote: > > Scanners at $dayjob (and reports on security@) frequently report that > built-in error documents suffer from non-xss HTML injection from the > request URL. Deja vu there. I’m sure we’ve fixed some such, and done a grep on the errordocs repo. I guess the continuing flow comes from the multiplicity of ways we might generate an error page. > Here are a few options to silencing these scans/reports: One more: insert an output filter when generating an error page. Escape URLs and scripts to plain text and highlight them. OK, it’s an overhead, but error pages are small. A sysop could of course have the option to disable it. — Nick Kew
Re: svn commit: r1796352
On Wed, 4 Apr 2018 10:46:31 -0400 Eric Covener <cove...@gmail.com> wrote: > What is the correct configuration that doesn't cause htaccess to be > visited? If it's trunk only, I think it should be an alternate config > mechanism rather than making it incompatible with any other setting in > htaccess. Anything equivalent to the "canonical" example in both the mod_status docs and the shipped httpd-info.conf.in . Note that the latter also implies it can be restricted to an access list, which is misleading if any "Require" can be bypassed through .htaccess. # Allow server status reports generated by mod_status, # with the URL of http://servername/server-status # Change the ".example.com" to match your domain to enable. SetHandler server-status Require host .example.com Require ip 127 > >> Comments? > > RewriteRule [P] in htaccess isn't anywhere near "screwed up". I disagree. .htaccess has no business enabling a user to access server resources outside his/her own directories. AllowOverride Fileinfo is a mess of largely-unrelated stuff, as pointed out by (IIRC) Jacob in the earlier discussion. -- Nick Kew
Re: svn commit: r1796352
Sorry, missed it at the time, but this is nonsense: > remove r1792169 taint checks from proxy and status modules > > Both of these checks are problematic without further work. > > status: even a .htaccess with no SetHandler blocks the handler. The status handler doesn't live in the filesystem. If it's correctly configured, the filesystem won't be visited, so of course no .htaccess will be processed. > proxy: RewriteRule ... ... [P] in htaccess is blocked. As it should be: for .htaccess to run resources outside its own directories is a long-standing design bug, and leads to security issues. Discussed with reference to mod_proxy and mod_status in, for example https://mail-search.apache.org/members/private-arch/httpd-security/201701.mbox/%3c63b4f81e-f742-563c-d4e4-99c4a50a7...@gmail.com%3E https://mail-search.apache.org/members/private-arch/httpd-security/201701.mbox/%3CCALK=yjn55j31eyfmle1bvtgy-9--9ftk2yfjzsumrlql+dk...@mail.gmail.com%3E https://mail-search.apache.org/members/private-arch/httpd-security/201701.mbox/%3c6e96a31c-c4f8-36b8-ea94-8f77a2680...@gmail.com%3E https://mail-search.apache.org/members/private-arch/httpd-security/201701.mbox/%3CCALK=yjnwr3cncercis4icqvs_wmj-exvddxlsntrplp5qoh...@mail.gmail.com%3E Leading to the patch committed in r1792169: > This is for trunk. I'd be more cautious about 2.4 (or 2.2) > because it could break screwed-up-but-not-dangerous configs > in production by refusing unexpectedly to run. For those > I'd suggest moving the check from proxy_handler into scheme > handlers. > > Comments? https://mail-search.apache.org/members/private-arch/httpd-security/201702.mbox/%3c20170208154128.69d12...@bifrost.webthing.com%3E (discussion was originally on security@, but it was suggested and the reporter agreed that it could be brought to dev@). -- Nick Kew
Re: svn commit: r1796352
Sorry, missed it at the time, but this is nonsense: > remove r1792169 taint checks from proxy and status modules > > Both of these checks are problematic without further work. > > status: even a .htaccess with no SetHandler blocks the handler. The status handler doesn't live in the filesystem. If it's correctly configured, the filesystem won't be visited, so of course no .htaccess will be processed. > proxy: RewriteRule ... ... [P] in htaccess is blocked. As it should be: for .htaccess to run resources outside its own directories is a long-standing design bug, and leads to security issues. Discussed with reference to mod_proxy and mod_status in, for example https://mail-search.apache.org/members/private-arch/httpd-security/201701.mbox/%3c63b4f81e-f742-563c-d4e4-99c4a50a7...@gmail.com%3E https://mail-search.apache.org/members/private-arch/httpd-security/201701.mbox/%3CCALK=yjn55j31eyfmle1bvtgy-9--9ftk2yfjzsumrlql+dk...@mail.gmail.com%3E https://mail-search.apache.org/members/private-arch/httpd-security/201701.mbox/%3c6e96a31c-c4f8-36b8-ea94-8f77a2680...@gmail.com%3E https://mail-search.apache.org/members/private-arch/httpd-security/201701.mbox/%3CCALK=yjnwr3cncercis4icqvs_wmj-exvddxlsntrplp5qoh...@mail.gmail.com%3E Leading to the patch committed in r1792169: > This is for trunk. I'd be more cautious about 2.4 (or 2.2) > because it could break screwed-up-but-not-dangerous configs > in production by refusing unexpectedly to run. For those > I'd suggest moving the check from proxy_handler into scheme > handlers. > > Comments? https://mail-search.apache.org/members/private-arch/httpd-security/201702.mbox/%3c20170208154128.69d12...@bifrost.webthing.com%3E (discussion was originally on security@, but it was suggested and the reporter agreed that it could be brought to dev@). -- Nick Kew
Re: Bugzilla open tasks by year
> As hackathon project it could be good to review some of those older-than-2011 > tasks and see which ones are good to keep and which ones can be closed for > no-activity/stale/not-valid-anymore/etc.. Good idea. Deal collectively with some of those judgement-calls that stump a solo bug-blitz. What we perhaps also need is a review of our bugzilla categories and workflow. For example, sometimes a PR is submitted with a proposed patch likely to be useful for some but not appropriate for inclusion in standard HTTPD. I’ve always left those open, which leaves them as not-bugs in the bugzilla count. Maybe we could deal with those with a new RESOLVED category (RESOLVED-PATCH?) and update the docs to invite users to search patch-bugs? — Nick Kew
Re: Poll: increase OpenSSL version requirement for trunk?
On Fri, 16 Mar 2018 09:06:53 -0400 Eric Covener <cove...@gmail.com> wrote: > I think bump trunk now, but not rip out any compat code for ease of > backport. +1. 2.4 is a stable branch: we can't go making changes that would disrupt existing users. What we could do is make it an annoying (even scary) Warning in 2.4, and see what reactions that brings. -- Nick Kew
Re: open tags - minimal example
On Sun, 2018-01-28 at 08:31 -0500, Yehuda Katz wrote: > HTTPD doesn't see the tags in the file at all. The way the file is > processed is determined by which Handler you set in the > configuration: > https://httpd.apache.org/docs/2.4/handler.html > > > To have your probably use your own file extension, for example index.mystuff, and > in your configuration, add AddHandler mystuff-handler .mystuff A handler to parse file contents is actually a poor choice. You use an output filter. Relevant examples in the current codebase include mod_includes, which parses tags in a manner similar to what the OP seems to envisage, and mod_proxy_html which uses a markup-aware parser that feeds each <...> as an event to your registered callback. Either of those modules would be a startingpoint to look at. -- Nick Kew
Re: SSLSrvConfigRec shared
On Sat, 2017-12-23 at 08:20 +0100, Stefan Eissing wrote: > > Ugh. Fine for trunk, but that's a lot of complexity to foist on > > a stable branch. A module would not only need to check MMN, > > but also implement fallback behaviour if there are no flags. > > So why not KISS and stick with that fallback for all 2.4? > > Not sure, I parse that. Any module that does nothing continues to see the > same behaviour than before. Where does the complexity arrive? But a module cannot ever *use* it without checking MMN *AND* implementing fallback behaviour for being loaded into an httpd built with the old struct - and consequently old API and ABI. That's bad enough to work through once, let alone maintain longer-term! Whereas the fallback, by definition, works in all cases. Agreed, post-config per-server stuff is clumsy: I have a distant memory from 2.0 days of hacking some ugly workaround, though the details elude me. But wouldn't it make more sense to review that in 2.5/trunk rather than the stable branch? -- Nick Kew
Re: SSLSrvConfigRec shared
On Thu, 21 Sep 2017 08:11:17 -0400 Eric Covener <cove...@gmail.com> wrote: > IIUC it should be safe to extend module_struct with a minor bump to > add 'int flags' to the bottom, but when you check the value you'd need > to check the MMN first. In the module you'd then just have some flags > or'ed together after register_hooks. Sorry to come to this late: arises from reviewing STATUS. Ugh. Fine for trunk, but that's a lot of complexity to foist on a stable branch. A module would not only need to check MMN, but also implement fallback behaviour if there are no flags. So why not KISS and stick with that fallback for all 2.4? -- Nick Kew
Re: Discard a brigade from filter
On Thu, 19 Oct 2017 16:30:27 +0200 m...@netbsd.org (Emmanuel Dreyfus) wrote: > Hello > > Is there a way to completely discard a brigade in an input filter, and > not pass it through filter chain? This doesn't quite make sense. An input filter pulls data into the brigade supplied by its caller, so you are in control all the time. > Removing all buckets cause an empty brigade to be sent to next filter > and that causes trouble. You mean, returned to the caller? That shouldn't matter (indeed it should be an expected behaviour in a non-blocking filter). If it is a problem (perhaps due to a bug outside your control), issue a blocking call to your own upstream and don't return anything until you have data (or EOS). Or if I were working around a bug in closed source, I might try inserting a placeholder such as an empty data bucket. -- Nick Kew
Re: Drop HttpProtocolOptions Unsafe from 2.later/3.0 httpd releases?
On Wed, 13 Sep 2017 08:29:44 -0500 William A Rowe Jr <wr...@rowe-clan.net> wrote: > So moving forwards, can we stop accepting stuff that isn't HTTP/1.1 in > our HTTP/1.1 server? Do we really want people to configure their > server to speak "other"? Did you mean to say "stop accepting ..."? > I'm starting to collect https://wiki.apache.org/httpd/Applications > based on searching google for instances where users have toggled > HttpProtocolOptions Unsafe, in response to specific application > behavior. Perhaps a useful exercise, but could take us in to a bad cycle of application workarounds that long-outlive the application being fixed. > From this list, we might decide to allow non-HTTP/1.1 input in > specific cases, and perhaps have multiple grades of protocol > correctness, as I first proposed. You mean something like Options or AllowOverride? Things that looked like a good idea at the time but led to all sorts of issues as the server evolved! OK, perhaps that's unduly harsh: this will be less problematic to maintain. Are you enumerating cases? -- Nick Kew
Re: mod_proxy_fcgi and flush
On Thu, 2017-07-06 at 14:08 -0400, Helmut K. C. Tessarek wrote: > One of the comments on the documentation page of mod_proxy_fcgi > (http://httpd.apache.org/docs/2.4/mod/mod_proxy_fcgi.html) mentions an > issue with flush: > > There is just no flush support it seems. I attempt to use PHP flush() > and it won't work until you fill up a buffer first, rendering Server > Sent Events impossible with proxy_fcgi. Do we have any idea what (if anything) proxy_fcgi receives from a PHP flush()? Your fix could be as simple as propagating an event. > > In a perfect world, I'm right there with you, but we've seen (as long as > > technology has existed) that people twist the way how technology is > > applied and used. It's hard to convince those people otherwise, > > especially when most of the time it has been possible to get it to work That takes me back ... https://www.theregister.co.uk/2007/08/24/everything_over_http/ > > > It probably makes sense to work on a nonblocking architecture for > > > proxied responses in general. Is that really the issue in the first place? We have a concept of flushing, and can implement more finely-tuned throughput than merely blocking vs non-blocking. The point at issue is for PHP to communicate a flush with proxy_fcgi, and for proxy_fcgi then to honour it. It seems one or both of those things isn't happening. > > I'm not familiar with that particular code, but would be interested in > > looking into it. Does anybody volunteer as a mentor? Best mentor is this list. Or #httpd-dev on Freenode (IRC), if someone's paying attention there. > lookup https://sks-keyservers.net/i for KeyID 0xC11F128D Please update that: it's too easy to spoof. See https://evil32.com/ , or my blog article at https://bahumbug.wordpress.com/2017/04/27/pretty-good-phishing/ -- Nick Kew
Re: [VOTE] Release httpd-2.2.33
On Tue, 2017-06-27 at 21:11 -0500, William A Rowe Jr wrote: > I'd like to know that there is another binding vote before bothering > to reroll. If the interest disappated faster than expected, that's > fine. Assuming your request was an implicit intent to vote on the > requested reroll. Sorry, interest may be slow rather than absent altogether. It's on my to-do list to check this out, paying more attention than usual to build defaults that might live on for a long time. -- Nick Kew
Re: TTLimit directive
On Tue, 2017-06-13 at 11:41 +0300, Donatas Abraitis wrote: > I would like to propose this patchset allowing to set maximum TTL value for > incoming requests. This is not a usual use case, but I'm interested (maybe > others too) to have this in place. The real use case would be like this one > http://blog.donatas.net/blog/2017/04/20/http-request-validation/. Thanks! I'm not sure I follow your exact scenario, but it looks like a modest enhancement at very low cost or risk! > TL;DR: if you want to deny requests bypassing proxy layer (in this case > Apache operates as a backend). Hence set TTLimit to 1 and Apache will be able > to handle requests coming almost from the local network, because packets with > TTL usually come from local networks. > > > I don't know which place is the right place to put patches, but > original patch is here: > https://bz.apache.org/bugzilla/show_bug.cgi?id=61179 > https://bz.apache.org/bugzilla/attachment.cgi?id=35048 That's exactly the right place. At first glance, patch looks interesting, and I'm minded to adopt (some version of) it for trunk. Though I think I'd default it to 0 (off) rather than your 255. Any other views? -- Nick Kew
Re: drop experimental from http2 for 2.4.next?
On Sat, 2017-04-15 at 17:02 -0400, Eric Covener wrote: > Hi everyone, shall we drop experimental from mod_http2 for 2.4.next? > > We could drop it and keep CTR. > Why would it be good for a stable (i.e. non-experimental) component of httpd to have an entirely different commit policy to the project as a whole? Surely the CTR is in recognition of its experimental status, to lubricate the process of hacking it into shape. -- Nick Kew
Re: APr Utils and PostgreSQL
On Sat, 2017-04-08 at 16:43 -0500, Tom Browder wrote: > config.log > > https://gist.github.com/tbrowder/2878124ad5fc35cb71a65a38e2950583 OK, where did you read that --with-pgsql would work with HTTPD's configure? If it's anywhere at apache.org, we have a docs bug. You need to build apr-util with pgsql! Or use your distro package. This confusion is one more argument for unbundling apr/apr-util from httpd! -- Nick Kew
Re: APr Utils and PostgreSQL
On Sat, 2017-04-08 at 10:45 -0500, Tom Browder wrote: > Well, i tried again and NO pgsql build (although I do have it > available with the Deb apr-util pgsql package install). > > I would love to help debug or fix this if I can, but I'm out of ideas. > > Best regards, > > -Tom What does your config.log say about it? If it's all greek to you, post it along with your config.nice to a pastebin and bug me or someone to take a look. -- Nick Kew
Re: APr Utils and PostgreSQL
On Fri, 2017-04-07 at 08:28 -0500, Tom Browder wrote: > I am trying to get the pqsql lib built and cannot get the config > option correct. The help says: OK, I read your mail (as I do most mail) on Debian, and I have their standard -dev package installed via aptitude. So I tested a new build, and a simple --with-pgsql worked for me: after make, I have the driver. If it had not worked, that would be a bug, and we'd want to know about it! Since you're asking on the httpd list, is it possible you're mixing httpd/apr/apu builds, and not actually reconfiguring apu when you specify your options? -- Nick Kew
Re: json, rather sooner than later
> On 31 Mar 2017, at 13:51, Stefan Eissing <stefan.eiss...@greenbytes.de> wrote: > > Hey, I'm currently evaluating some addition to httpd that needs to > parse/serialize JSON. > I know not everyone is a fan of that format, but some protocols do require it > (I try to > spare us a discussion about "better" formats :-). > > Does anyone have ideas/preferences about bringing JSON into httpd (and > eventually into > the APR at some later point in time)? I have not found a good lib that is > available > everywhere. Several have MIT license, but I think we cannot somehow bring > those under > the Apache umbrella, right? > > If anyone has a recommendation or more experience in this then me, I'd be > glad for any > guidance. > > -Stefan +1 to a JSON module. Not sure if there’s mileage in abstracting JSON parsers: when I last looked, each had its own data models at a higher level than could be usefully unified. But I’m open to persuasion. I did some work on this, albeit for a subset of JSON required by a particular project. I’ll try & dig that up, to see if there’s anything worth salvaging. — Nick Kew
Re: mod_ssl custom vhost module
On Thu, 2017-03-30 at 15:20 +, Michael Sløgedal wrote: > I looked a little on the mod_ssl source code, and it seems it does a lot of > preprocessing on config stage, and relies on a combination of VirtualHost and > ServerName / Alias directives. > I suppose this means that mod_ssl wouldn't work with grabbing certificates > based on a path stored in sql on-the-fly. I'm not familiar with the murky recesses of mod_ssl. But if I've understood you aright, I think a good startingpoint would be to see if you can hook something in to connection processing, that'll in turn run something ahead of mod_ssl getting in to a connection. Not sure if that actually leads anywhere useful. Just a thought, if you haven't already tried it. Your main problem is that you have a hack that shoehorns vhosts in where they don't belong. -- Nick Kew
Re: HttpProtocolOptions Directive
On Tue, 2017-02-28 at 09:50 +0100, Pavel Reichl wrote: > Hello, > > > I have a question regarding this new directive - HttpProtocolOptions > and in particular its parameter Unsafe. Basically the HTTP spec changed. RFC7230 outlaws behaviours that were fine under earlier HTTP versions. Apache HTTPD, along with other web software, is being updated to the new standard. Unsafe should get you legacy behaviour unless the docs and CHANGES say otherwise. If you're getting discrepancies, that's probably worth a bug report. -- Nick Kew
Re: Can byterange filter request only needed ranges from my module instead of discarding?
On Sun, 2017-02-26 at 21:00 +0300, Basin Ilya wrote: > Instead of writing data in my handler can I create there a custom bucket, let > the byterange filter split it properly and let the core filter call my custom > read function? I think so in principle. Your bucket type would have to satisfy byterange_filter, meaning it has to know its own length. In practice, it may not work as you expect. If there's any content filter between your handler and the byterange module, then that will read your bucket and defeat your purpose. Why not write a mock-up of your proposed design? Say, a bucket that serves data from a static file by seek/read, just to see how it behaves in different configurations and whether you can make the architecture work for you? -- Nick Kew
Re: Can byterange filter request only needed ranges from my module instead of discarding?
On Sun, 2017-02-26 at 15:02 +0300, Basin Ilya wrote: > However, it's inefficient to serve huge virtual files this way when only a > small part of such file requested. How to solve this? Unless your module can handle the byterange itself, you might want to consider cacheing the generated file for the benefit of future byterange requests. -- Nick Kew
Re: [RFC] ?
On Tue, 2017-02-21 at 21:58 +, Joe Orton wrote: > Any reason is a bad idea, so we can do that more cleanly > (... in a couple of decades time)? One reason it might be a very bad idea: user confusion! I'm thinking of the track record of here. Our support fora are full of users who have seen it in default/shipped config and docs, and treat it as some magic incantation they need. They end up with a problem "why doesn't Foo work?", which they bring to our fora after many hours of tearing their hair. The usual answer: Get rid of all the crap, to stop suppressing the error message you need! -- Nick Kew
Re: proxy_fcgi directives or envvars?
On Sun, 2017-01-08 at 11:31 -0500, Eric Covener wrote: I'm easy either way. > I notice that even complex proxy submodules (proxy_http) have no > directives but use envvars for config. I actually like this approach > for obscure-ish things, but directives do have some benefits (error > checking, better doc) I think (from distant memory) I may be responsible for some of that. At some point I wanted to introduce directives where there were none. But mod_proxy had, by convention, put all the directives in mod_proxy itself, including some that belonged on the same docs page as what I needed to add. I have an idea something may have been needed in both mod_proxy and mod_proxy_http. It certainly gave rise to issues with how to document it without a major reorg. Envvars offered a straightforward workaround. -- Nick Kew
Re: [users@httpd] Copyright notices in httpd source files
Cc: dev list. Looks like a catch? On Wed, 2016-12-28 at 17:44 -0500, Christopher Schultz wrote: > All, > > Is it common to have a copyright notice in httpd C source files? Not common, but neither is it unusual. I'd expect it to mean someone else copyrighted it before contributing it to apache. > [ mod_proxy_protocol.c ] > Copyright 2014 Cloudzilla Inc. If that's in our svn, it should probably have another line asserting Apache copyright alongside that one. As in, for instance, mod_proxy_html. -- Nick Kew
Re: Change the content-length header for other filters
On Wed, 2016-12-21 at 22:10 +0100, André Rothe wrote: > But after my filter completes the request processing, I'll get: > > Sending error response: The request contained fewer content data than > specified by the content-length header Sorin's reply is part of the story, and may or may not be useful in your case. The basic issue you have to consider is that the headers arrive before the body, and a handler that cares about Content-Length is likely to have read it before your filter has reset it. The alternative - read and buffer the entire body before starting to process it - becomes hopelessly inefficient for large requests. There's some discussion of the issue in the mod_proxy docs, as mod_proxy has an option to support HTTP/1.0 backends that need an explicit Content-Length. -- Nick Kew
Re: [VOTE] Release Apache httpd 2.4.25 as GA
On Sat, 2016-12-17 at 20:32 -0500, Eric Covener wrote: > I think your t/conf/extra.conf is out of date and has the +ExecCGI > masked out by . So you are getting the perl script > instead of its output setting nasty headers. [pre-breakfast post: not yet at the desktop where I ran the tests] For what it's worth, this was a pristine trunk repo of the test suite as of date about 12 hours ago. As ever, built/run in a directory newly created by lndir so as never to pollute the repo. However, there's a message earlier in the tests: CGI tests skipped because neither mod_cgi nor mod_cgid were found (nonsense: mod_cgid.so is in its expected place in /modules/ ). Looks like some gremlin in the test suite, plus running those tests when CGI tests had been skipped. -- Nick Kew
Re: [VOTE] Release Apache httpd 2.4.25 as GA
On Sat, 17 Dec 2016 18:35:22 -0600
William A Rowe Jr <wr...@rowe-clan.net> wrote:
> On Dec 17, 2016 17:22, "Nick Kew" <n...@apache.org> wrote:
>
>
> Got some test errors with the new stuff. Investigating.
>
> Test Summary Report
> ---
> t/apache/http_strict.t(Wstat: 0 Tests: 78 Failed: 5)
> Failed tests: 72-75, 77
>
>
> Really should not occur and I haven't seen this. Shoot me the -v
> (erbose) output from your t/apache/http_strict.t please?
Cut here. Bedtime now (coming up for 01:30), but if I
find time tomorrow I may play with the testcases under gdb.
They're all cases where the server returns 200 when the
testcase expects an error.
72:
# SENDING:
# GET /apache/http_strict/send_hdr.pl?OiBiYXI= HTTP/1.0\r\n\r\n
# DECODED: : bar
# RESPONSE:
# HTTP/1.1 200 OK\r\n
# Date: Sun, 18 Dec 2016 01:17:34 GMT\r\n
# Server: Apache/2.4.25 (Unix)\r\n
# Last-Modified: Sun, 18 Dec 2016 00:13:28 GMT\r\n
# ETag: "226-543e3ac24b8ae"\r\n
# Accept-Ranges: bytes\r\n
# Content-Length: 550\r\n
# DMMATCH1: 1\r\n
# Connection: close\r\n
# \r\n
# #!/usr/bin/perl\n
# # WARNING: this file is generated, do not edit\n
# # generated on Sun Dec 18 00:13:28 2016\n
# # 01: Apache-Test/lib/Apache/TestConfig.pm:961\n
# # 02: Apache-Test/lib/Apache/TestConfig.pm:1051\n
# # 03: Apache-Test/lib/Apache/TestMM.pm:142\n
# # 04: Makefile.PL:26\n
# \n
# BEGIN { eval { require blib && blib->import; } }\n
# use MIME::Base64;\n
# use strict;\n
# use warnings;\n
# \n
# print "Content-type: text/plain\\r\\n
# ";\n
# print decode_base64($ENV{QUERY_STRING}), "\\r\\n
# ";\n
# print "\\r\\n
# ";\n
# print "Hi!\\n
# ";\n
# print "SERVERNAME=$ENV{SERVER_NAME}\\n
# ";\n
# print "HTTP_HOST=$ENV{HTTP_HOST}\\n
# ";\n
#
# expecting 500, got 200
73:
# SENDING:
# GET /apache/http_strict/send_hdr.pl?RgBvbzogYmFy HTTP/1.0\r\n\r\n
# DECODED: F\x00oo: bar
# RESPONSE:
# HTTP/1.1 200 OK\r\n
# Date: Sun, 18 Dec 2016 01:17:34 GMT\r\n
# Server: Apache/2.4.25 (Unix)\r\n
# Last-Modified: Sun, 18 Dec 2016 00:13:28 GMT\r\n
# ETag: "226-543e3ac24b8ae"\r\n
# Accept-Ranges: bytes\r\n
# Content-Length: 550\r\n
# DMMATCH1: 1\r\n
# Connection: close\r\n
# \r\n
74:
# SENDING:
# GET /apache/http_strict/send_hdr.pl?RgFvbzogYmFy HTTP/1.0\r\n\r\n
# DECODED: F\x01oo: bar
# RESPONSE:
# HTTP/1.1 200 OK\r\n
# Date: Sun, 18 Dec 2016 01:17:34 GMT\r\n
# Server: Apache/2.4.25 (Unix)\r\n
# Last-Modified: Sun, 18 Dec 2016 00:13:28 GMT\r\n
# ETag: "226-543e3ac24b8ae"\r\n
# Accept-Ranges: bytes\r\n
# Content-Length: 550\r\n
# DMMATCH1: 1\r\n
# Connection: close\r\n
# \r\n
75:
# SENDING:
# GET /apache/http_strict/send_hdr.pl?RgpvbzogYmFy HTTP/1.0\r\n\r\n
# DECODED: F\noo: bar
# RESPONSE:
# HTTP/1.1 200 OK\r\n
# Date: Sun, 18 Dec 2016 01:17:34 GMT\r\n
# Server: Apache/2.4.25 (Unix)\r\n
# Last-Modified: Sun, 18 Dec 2016 00:13:28 GMT\r\n
# ETag: "226-543e3ac24b8ae"\r\n
# Accept-Ranges: bytes\r\n
# Content-Length: 550\r\n
# DMMATCH1: 1\r\n
# Connection: close\r\n
# \r\n
77:
# SENDING:
# GET /apache/http_strict/send_hdr.pl?Rm9vOiBiAWFy HTTP/1.0\r\n\r\n
# DECODED: Foo: b\x01ar
# RESPONSE:
# HTTP/1.1 200 OK\r\n
# Date: Sun, 18 Dec 2016 01:12:21 GMT\r\n
# Server: Apache/2.4.25 (Unix)\r\n
# Last-Modified: Sun, 18 Dec 2016 00:13:28 GMT\r\n
# ETag: "226-543e3ac24b8ae"\r\n
# Accept-Ranges: bytes\r\n
# Content-Length: 550\r\n
# DMMATCH1: 1\r\n
# Connection: close\r\n
# \r\n
Re: [VOTE] Release Apache httpd 2.4.25 as GA
On Sat, 17 Dec 2016 23:21:09 + Nick Kew <n...@apache.org> wrote: > Got some test errors with the new stuff. Investigating. > > Test Summary Report > --- > t/apache/http_strict.t(Wstat: 0 Tests: 78 Failed: 5) > Failed tests: 72-75, 77 The fails are: [ "R" . ": bar" => 500 ], [ "R" . "F\0oo: bar"=> 500 ], [ "R" . "F\x01oo: bar" => 500 ], [ "R" . "F\noo: bar"=> 500 ], [ "R" . "Foo: b\x01ar" => 500 ], Sorry, haven't followed strict implementation closely enough to know if these are known cases. > t/modules/http2.t (Wstat: 512 Tests: 0 Failed: 0) A visit to CPAN fixed that. Sorry for the noise. -- Nick Kew
Re: [VOTE] Release Apache httpd 2.4.25 as GA
On Fri, 16 Dec 2016 13:29:04 -0500 Jim Jagielski <j...@jagunet.com> wrote: > At long, long last, the pre-release test tarballs for Apache httpd > version 2.4.25 can be found at the usual place: > > http://httpd.apache.org/dev/dist/ > > I'm calling a VOTE on releasing these as Apache httpd 2.4.25 GA. Debian GNU/Linux here. The administrivia checks out OK, though I wonder if we really need so many variants in this day and age? I.e. do we need both .gz and .bz2, and do the non-PGP checksums really serve a useful purpose? Builds cleanly, but the tests don't run: had to hack it to load mod_session for ap_hook_session_save. That's obviously a gremlin in the test framework, but perhaps mod_session should ideally use optional symbols for dependencies? Got some test errors with the new stuff. Investigating. Test Summary Report --- t/apache/http_strict.t(Wstat: 0 Tests: 78 Failed: 5) Failed tests: 72-75, 77 t/modules/http2.t (Wstat: 512 Tests: 0 Failed: 0) Non-zero exit status: 2 Parse errors: No plan found in TAP output Files=103, Tests=2623, 123 wallclock secs ( 3.24 usr 0.39 sys + 88.52 cusr 20.56 csys = 112.71 CPU) Result: FAIL Failed 2/103 test programs. 5/2623 subtests failed. [warning] server localhost:8529 shutdown [ error] error running tests (please examine t/logs/error_log) -- Nick Kew
Re: JSON for mod_status
On Wed, 2016-11-30 at 12:54 -0500, Jim Jagielski wrote: > I'm thinking about adding JSON support to mod_status... > the "plain" version output really stinks and lacks parity > w/ the info we provide via HTML, and it would be nice > to produce a really easily parseable format. > > Thoughts...? Ideally that should be the job of an output filter. If mod_status were to generate a single standard-ish format, like JSON or XML, a general-purpose filter could generate other formats. -- Nick Kew
Re: Hackathon tomorrow?
On Thu, 2016-11-17 at 09:40 -0500, Rich Bowen wrote: > If you're around Apachecon Tomorrow please consider dropping by the > hackathon area on floor -2 to work on the items in > https://public.etherpad-mozilla.org/p/aceu-2016-hackathon > > Thanks. I'm here now, and looked around both here and in the area upstairs several times this morning. No sign of any coordinated hacking, except the inevitable infra huddle. Am I just missing some (other?) hackathon area where you're all gathered? -- Nick Kew
Re: mod_ftp segaults on rheloids
On Fri, 2016-11-11 at 08:24 +, Benjamin Lefoul wrote: > Is this list still active? > Maybe it was not the right place to ask about mod_ftp? I guess noone picked up the baton. Too many blanks to fill. Did you build mod_ftp yourself on a distro-provided httpd? What toolchain did you use for that? What happens if you build everything yourself, or alternatively use everything from the distro? Where's the traceback from your segfault? -- Nick Kew
Re: svn commit: r1768036 - in /httpd/httpd/branches/2.4.x-merge-http-strict: ./ CHANGES include/ap_mmn.h include/http_core.h include/httpd.h modules/http/http_filters.c server/core.c server/protocol.c
On Fri, 2016-11-04 at 10:47 -0400, Eric Covener wrote: > Too strict? Be conservative in what you send. An Absolute URL is never going to be the wrong thing to send. > https://tools.ietf.org/html/rfc7231#section-7.1.2 Another change from the HTTP RFCs we learned, where Location MUST be absolute. Though in practice, there was a lot of confusion, with the CGI spec - and hence serverside apps - permitting the relative semantics. Seems recent HTTP came into line with CGI on this one. -- Nick Kew
Re: EC to audit Apache HTTP Server
On Fri, 2016-07-22 at 18:59 +0200, Steffen wrote: > See https://joinup.ec.europa.eu/node/153614 > > Steffen Thanks for the heads-up! Anyone we might've heard of likely to be involved with that? Could this be time to start inserting comments? /* Note to auditors: an EALLOC here is a crash, * and error handling won't help. */ foo = apr_palloc(pool, sz); -- Nick Kew
Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached
On Tue, 2016-05-31 at 11:15 +0300, Ivan Zahariev wrote: > Hello, > > I got no feedback. Am I posting this suggestion at the right mailing > list? Sorry, I see your original post marked for attention in my mail folder, but languishing hitherto unattended. Just now opened your link in a browser to take a look. There could be others who have done something similar. As a general reply to this question, yes, this is the best available mailinglist. The other place to post it would be as an enhancement request in bugzilla (issues.apache.org). The keyword "PatchAvailable" there may help by marking it as low-hanging fruit. -- Nick Kew
Re: Module Development - Advice Needed
On Thu, 2016-05-26 at 22:44 +, Van Ulden, Joost (NRCan/RNCan) wrote: > I approached a couple of folks at ApacheCon in Vancouver about some > work that I am involved with that would benefit from an httpd module. > I am sending this message to the list to provide more information, and > to seek advice on how we should proceed. > > We are developing a format we call “Map Markup Language”, or MapML. > The objective of this format is to describe Web map semantics – that > is scale / zoom, projection, extent, features, styles, licenses, > legends in the context of a hypermedia media type that can be > manipulated / interacted with through the uniform interface alone. We > would like to eventually register the format as text/mapml. This is seriously interesting. I need to take a look, and will be well pleased if I can find time to take an active interest. > A Java servlet prototype for this functionality is available: I have yet to look at that. Would you plan the module and servlet to grow in parallel as alternative implementations, or each to take its own path? Presumably there is something to be gained with the two alternative implementations? > We would like some advice on how to proceed with the development of a > module. For instance should we pursue this as a third-party module? > Can we tap into the community for development assistance if we don’t > have the required skills in-house? Any advice would be appreciated. You've had a couple of suggestions from Bill: I would agree that either github or the apache incubator make sense and are options worth considering (the latter could lead to it eventually becoming a sub-project of httpd itself). I would think, let the servlet be your guide. If it's thriving then you have a good model. If not, you learn lessons from it. Either way, thanks for posting to tell us about your project! As Jacob points out, the modules-dev list is a good resource for technical help. It's low-traffic, but still populated by a fair few people willing and able to help. -- Nick Kew
Re: [Patch] Ensure HTTP1 filters are only added on HTTP1 requests
On Tue, 2016-03-15 at 22:57 -0500, William A Rowe Jr wrote: > My concern is that this can't and shouldn't change on 2.4.x. > Thoughts? If someone is independently using a different protocol, or has just tweaked HTTP handling for their own reasons, they could be using hacks/workarounds this'll break. We've seen similar cases in the lifetime of 2.4 where PHP introduced its own workarounds that our bugfixes then broke. Basic lesson: any such patch must be switchable and default to off. On the other hand, mod_h2 being marked as pre-stable means we haven't promised stability to anyone who's hacked it. So mod_h2 could safely deal with protocol filters. Or by extension could explicitly set a flag to trigger a core change as per this patch. Does this merit that level of hack? -- Nick Kew
Re: AddOutputFilterByType in Apache 2.4 inserts filters as AP_FTYPE_RESOURCE
On Wed, 2016-01-13 at 17:59 +0100, Micha Lenk wrote: > Hi, > > The directive AddOutputFilterByType can be used to insert filters to the > output filter chain depending on the content type of the HTTP response. > So far so good. > > PROBLEM DESCRIPTION This is probably worth a bugzilla entry. I think I can clarify a little of what's happened. AddOutputFilterByType was something of a hacked afterthought to filtering back in the days of httpd 2.0. On the one hand, it met a need. On the other hand, it worked only in a very limited range of situations where the content type was known at the time the filter was to be added. It had no capacity to respond to other aspects of the content, or indeed the request/response. And there were other issues. Then came mod_filter and a generalised framework. AddOutputFilterByType was now obsolete, but too widely-used to dispense with entirely. As a compromise it was re-implemented within mod_filter, where it could co-exist with other dynamic filter configuration. Your observation tells us the semantics aren't quite compatible. And your patch looks good - thanks. -- Nick Kew
Re: reverse proxy wishlist
On Thu, 3 Dec 2015 12:23:24 -0600 William A Rowe Jr <wr...@rowe-clan.net> wrote: > On Thu, Dec 3, 2015 at 10:32 AM, Nick Kew <n...@apache.org> wrote: > Yup, and I'm not proposing to eliminate the mechanism, I'm proposing > that the existing 'core' subset be codified in fewer, still > lightweight modules. Fairy nuff. At this level of detail I can't really tell if you're proposing anything that would concern me. > I'm looking, none of these seem like huge hacks, wondering > which of them trigger your concern? Well, your talk of refactoring config led me to wonder whether you were proposing another tilt at the whole directory walk stuff. While that may have merit, past experience tells us it's not going to be easy! -- Nick Kew
Re: reverse proxy wishlist
On Thu, 3 Dec 2015 10:09:08 -0600 William A Rowe Jr <wr...@rowe-clan.net> wrote: > Most stock/core implementations shouldn't > change if a user wants to plug in 'yet another' option, but there is > really no excuse for us to map so many ldobjects and text pages into > the memory map of a given process, when the actual program text page > of each is a few hundred opcodes, max. Extensibility. Flexibility. They're our biggest single strength when compared to the likes of nginx/tengine. > (You can submit that the user could want to replace the bytraffic > implementation, for example, but I'd counter that they can certainly > rebuild any mod_lbmethod_core module with the others still using > stock sources, and deploy their alternative, or they can give their > custom fork of a provide yet another provider name.) Someone wanting different functionality shouldn't have to take on the maintenance headache of hacking into one of our modules. We have our API/ABI promise to make life easy for them introducing their own modules. > I'm not asking anyone to coalesce these, though. It was already > on my rather long list of 'nice to have' along with supporting > multiple conf mergers per module (effectively allowing 'multiple > modules' to be a single module and splitting our monstrous core > server and dir configs into related digestible pieces that rarely > need to be merged, and optimizing away modules with no conf merge at > all). And along with cleaning up the httpd 2.next API, and i18n of > error output which I am continuing on first once the mod_ssl issues > for mod_ftp are resolved (my current project). Hmmm. There's some nice-to-have in there, but it also sounds like a big hack. > Last thought, you might want to share your question with users@? +1 -- Nick Kew
Re: Moderations for modules.apache.org
On Wed, 2015-11-11 at 11:27 +, Daniel Gruno wrote: > Does this sound like a good idea, or complete overkill? I have long thought we might employ an alternative scheme akin to a "planet" aggregator. Make the module index an aggregator from module authors providing and maintaining their own descriptions as DOAP files. That way an author doesn't have to go through any manual process or moderation to update entries, and the admin burden is reduced. We still have to bootstrap new authors wanting us to aggregate their DOAP URLs. We could fully automate it for committers by auto-approving apache.org URLs, leaving a much reduced space for manual moderation and still vulnerable to spam attacks. Then we can reduce that further by requiring oauth as you suggest. And once the OpenMiracl podling has a TA up-and-running, we can deploy that to help open a wider circle of strong trust. Just a thought. -- Nick Kew
Re: Tracking sent responses
On Fri, 6 Nov 2015 09:12:40 -0500 Julien FROMENT <julien.from...@sagemcom.com> wrote: > Here is the pseudo code: > > -- Client send a request > > -- Apache processes the request and send it to the backend server > > ... > > -- The backend server returns the response > > -- Apache sends the response to the client > > -- Apache calls Async API with the number of bytes sent Hmm, that sounds like something I wrote (not, alas, open source) for a client many years ago. But if you're only looking for bytes sent, it's probably easier than that. Before investing in new development, consider: - Could you hook your notification into regular piped logging? - Would regular logging through an API like syslog or spread serve (there are third-party modules for those). - Would a security-oriented tool like Ironbee be complete overkill? -- Nick Kew
Re: Improve Apache performance on high load (prefork MPM) with multiple Accept mutexes (Patch attached)
On Mon, 2015-10-26 at 08:45 +, Yehezkel Horowitz wrote: > Any chance someone could take a short look and provide me a feedback > (of any kind)? A patch posted here may get lost, especially if it's not simple and obvious enough for instant review and understanding. Posting it as an Enhancement request in Bugzilla would leave a record of it. > 1. Do you think this is a good implementation of the suggested > idea? If a threaded MPM really isn't an option (for most users the obvious solution), then the question is what works for you. > 3. Would you consider accepting this patch to the project? > If so, could you guide me what else needs to be done for acceptances? > I know there is a need for configuration & documentation work - I’ll > work on once the patch will be approved… Unlikely it would get in to a future 2.2 release unless it fixed something much more than an arcane performance problem (arcane because because it only happens when you reject conventional ways to boost performance, like another MPM). How well does your patch apply to trunk? If you don't want to go in that direction, you could post somewhere always available for anyone interested. Our bugzilla would serve, as would somewhere else you publish from, like github or a personal site. -- Nick Kew
Re: Is Apache getting too patchy?
On Mon, 26 Oct 2015 09:51:43 -0700 Jacob Champion <champio...@gmail.com> wrote: > I'm somewhat on the outside looking in, as an httpd newbie. But my > standard experience (it's happened two or three times now) is this: > > 1) Non-trivial patch is proposed to the list with calls for > discussion/debate. > 2) Nothing happens. > 3) List is pinged for comments. > 4) Silence. > 5) Patch author either gives up or pings relentlessly until... > 6) ...patch is applied directly to trunk without discussion. Thanks for the perspective. Yes, it's sometimes uncomfortable. > I'd rather not feel like I'm just annoying dev@ until you submit my > stuff -- I want to *talk* about it, and improve the server. That may not be easy. You need to find someone who'll be interested in an idea or patch, and has the time to review it. Plus, the community as a whole to agree it's a good idea, or at least not actively oppose it. I wonder if workflow would be improved if we had named maintainers for particular parts of the codebase - for example individual modules? Not necessarily to do all the work, but to take primary responsibility to see that your ideas don't just fall through the gaps and get ignored? -- Nick Kew
Re: [Bug 57785] REDIRECT_URL is not suitable for use in server-generated pages
On Fri, 23 Oct 2015 15:22:27 -0400 Eric Covener <cove...@gmail.com> wrote: > My opinion is to make it opt-in before the next 2.4, but I am not > committed to that. Hehe. I was leaning towards introducing separate vars for full URL and fragment. But your patch looks good to me. -- Nick Kew
Re: Apache Module Development Query on character encodings.
On Wed, 21 Oct 2015 07:04:27 +0100 "John Dougrez-Lewis" <jle...@lightblue.com> wrote: > Hi Nick, > > > Hi, are you by any chance the Raving Loony I once knew at Cambridge? > > Yes indeed - that must be 35 years ago now - these days I'm a bit more > sensible (although the legacy of the OMRLP lives on). OMLRP? It was ∇ ∇ back then (CURLS, if email screws that up). > > Basically there are three parts to working with character encodings: > > * Detecting them in incoming data. > > * Converting them to order. > > * Correctly labelling outgoing data. > > > mod_xml2enc will do all that for libxml2-based filters, and could easily > be tweaked to drop the libxml2-specific optimisations for general- > > purpose use. Alternatively the charset-detection from mod_xml2enc could > probably be folded into mod_charset_lite. > > So basically mod_xml2enc will detect the incoming encoding (whatever it may > be)? I suggest instead of debating here, take a look at it. Start with the docs, and then move on to the code if necessary. -- Nick Kew
Re: Apache Module Development Query on character encodings.
On Tue, 20 Oct 2015 20:23:02 +0100 "John Dougrez-Lewis" <jle...@lightblue.com> wrote: > Hi, Hi, are you by any chance the Raving Loony I once knew at Cambridge? > I need to be able to service and respond to requests as follows: Basically there are three parts to working with character encodings: * Detecting them in incoming data. * Converting them to order. * Correctly labelling outgoing data. mod_xml2enc will do all that for libxml2-based filters, and could easily be tweaked to drop the libxml2-specific optimisations for general-purpose use. Alternatively the charset-detection from mod_xml2enc could probably be folded into mod_charset_lite. > The input and output buffers appears to be 8-bit char* based but I can't see > any references to specific encodings. > > > > How do I go about massaging the input & output into UTF-8 and fixed width > 16-bit Unicode? > > > > Are there any good references on how to achieve this? It's a bit of a mess, because there are several different standards (HTTP, XML and HTML), and in real life those are sometimes in conflict. The detection in mod_xml2enc has been fine-tuned over the years and test-driven on a wide range of scripts, including non-Latin charsets such as Russian/Cyrillic and Arabic. -- Nick Kew
Re: XSLT filter for httpd
On Mon, 2015-10-19 at 13:49 +0200, Graham Leggett wrote: > Hi all, > > In the light of the move to simple services that talk XML/JSON and HTML and > Javascript based clients that become more and more capable, I find myself > wanting an XSLT filter quite often these days to sit in the middle and > translate between the two. > > As a complement to mod_xmlenc, would it make sense to include an XSLT filter > in httpd out the box? There are several old modules: for example mod_transform. I expect they still serve for sites without i18n requirements. One option would be to overhaul that. Note, mod_transform is GPL. Originally my decision when I released its earlier predecessor, before I was part of the dev@httpd team. I'd be happy to re-license it as Apache, and I don't think any of my co-developers would object. -- Nick Kew
Re: XSLT filter for httpd
On Mon, 19 Oct 2015 15:39:06 +0200 Graham Leggett <minf...@sharp.fm> wrote: > > Note, mod_transform is GPL. Originally my decision when I released > > its earlier predecessor, before I was part of the dev@httpd team. > > I'd be happy to re-license it as Apache, and I don't think > > any of my co-developers would object. > > I’ve been using mod_transform v0.6.0 for a while, and have wanted to develop > it further. It would be a great starting point. I have a distant recollection of consulting Paul and Edward about re-licensing, then dropping the ball on it. IIRC the outcome was, they were both happy to re-license, but there had also been one or two third-party patches raising a questionmark over whether we should consult anyone else. Cc: Paul. Do you recollect that? You still in contact with Edward? -- Nick Kew
Re: Thx and merit
On Wed, 14 Oct 2015 08:58:48 -0400 Jim Jagielski <j...@jagunet.com> wrote: > Once again, I'd like to thank them personally! +1 to all that, with one small addition. Apache is about the individuals who participate. So the chief thanks go to Stefan, who is of course now one of us. And of course honourable mentions to other developers such as your good self. -- Nick Kew
Re: GitHub (mirror) pull requests notifications
On Fri, 2015-10-09 at 11:12 +0200, Yann Ylavic wrote: > Could they somehow be sent to dev@, or is there legal issues? Bugzilla activity doesn't get sent to dev@, even when it involves PatchAvailable (kind-of equivalent to a Pull Request, yesno)? If we were to propagate pull requests, we should perhaps review bugzilla workflow (IMHO a weakness in how we work now) at the same time? -- Nick Kew
Re: Expression Parser: search and replace with s/PATTERN/REPLACEMENT/FLAGS
On Thu, 2015-10-01 at 12:26 +0200, Rainer Jung wrote:
> Since it gets more common to use the expression parser for string
> operations and not only for boolean checks, I think it would be useful
> (and powerful) to support
>
> s/PATTERN/REPLACEMENT/FLAGS
>
> and allow back references in REPLACEMENT. The operation would not try to
> do the replacement in place but create a new string according to the
> given PATTERN and REPLACEMENT.
Are you mixing two things? That's well-established regexp syntax,
but you're looking at applying it to a different class expressions.
I think the most interesting issue is to define your behaviour.
> Header set X-USER "expr=%{REMOTE_USER} =~ s/([^@]*)@.*/$1/"
Aha! In terms of the expression parser, that looks like
capturing a side-effect (as opposed to a true/false result).
Maybe it would work with something like C comma-list syntax?
But I expect the line of least resistance would be to use
plain regexp rather than expr.
--
Nick Kew
Re: Expression Parser: search and replace with s/PATTERN/REPLACEMENT/FLAGS
On Thu, 2015-10-01 at 13:32 +0100, Nick Kew wrote:
> > Header set X-USER "expr=%{REMOTE_USER} =~ s/([^@]*)@.*/$1/"
> But I expect the line of least resistance would be to use
> plain regexp rather than expr.
Come to think of it, using regexp that looks a lot like:
${REMOTE_USER} =~ s/([^@]*)@.*/$1/;
%header{X-USER} = $_;
I haven't tested the mod_perl syntax, but since it's
bog-standard perl, it's sure to be straightforward.
--
Nick Kew
Re: Expression Parser: search and replace with s/PATTERN/REPLACEMENT/FLAGS
On Thu, 1 Oct 2015 15:25:38 +0200
Rainer Jung <rainer.j...@kippdata.de> wrote:
> Am 01.10.2015 um 15:03 schrieb Jim Jagielski:
> > Doesn't mod_lua do this for you?
>
> But would it be the right tool to use whenever you want to apply a
> s/.../.../?
Some might say so (as some might say mod_perl or mod_rewrite).
> >>>> Header set X-USER "expr=%{REMOTE_USER} =~ s/([^@]*)@.*/$1/"
One further thought there that might be easier to work,
or a staging post on the way to your goal:
X-USER = $_
# (or whatever backref syntax you prefer).
--
Nick Kew
Re: svn commit: r1697322 - /httpd/httpd/branches/2.4.x/STATUS
On Mon, 24 Aug 2015 07:40:33 -
rpl...@apache.org wrote:
--- httpd/httpd/branches/2.4.x/STATUS (original)
+++ httpd/httpd/branches/2.4.x/STATUS Mon Aug 24 07:40:33 2015
@@ -238,8 +238,15 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
+1: rpluem, ylavic
niq: 1. the if(worker-s-retries) {} and comment at line 2917
don't seem to make any sense.
+ rpluem: This is just taken over from existing code. It is just indented
+ differently hence part of the path I think it should be marked
+ as TODO section. But this should be subject to another
+ patch.
Agreed it comes from existing code, but I wonder if its meaning and purpose
haven't got lost since it was originally written?
I'm +1 if that's either removed or if the comment clarifies what
actual case the TODO would be dealing with.
--
Nick Kew
Re: HSTS Header Duplication
On Thu, 13 Aug 2015 20:28:40 + Houser, Rick rick.hou...@jackson.com wrote: Some time back, I turned on HSTS for our sites with something like this: Header always set Strict-Transport-Security max-age=### I think you're misunderstanding mod_headers and the headers structure. In general terms, HTTP permits duplicate headers, which may have different values. For example,.multiple cookies. So mod_headers lets you set them, regardless of whether they're already set. If that's not what you want, you can of course configure mod_headers to unset an existing header before setting a new one. Or other configuration variants. -- Nick Kew
Developer Book
[note crosspost and followup-to] With the coming of mod_h2 and talk of EOL for 2.2, my modules book is getting a bit long in the tooth. Is it time for either a new edition or a new dev-oriented book? I'm not about to motivate myself to the level that got my modules book onto the shelves. But if there's sufficient interest, I'd certainly play my part in a multi-author effort, and offer to coordinate it if noone else wants to take the lead. In the case of a multi-author second edition to my book, I think I'd contractually have to offer first refusal to Prentice Hall, and I'd suggest making any royalties payable to the ASF. In the case of an entirely new work, there'd be no such strings attached, but it would probably be inappropriate for me to take a coordinating role or to contribute material from the old book. Anyone interested? -- Nick Kew
Re: svn commit: r1684900 - in /httpd/httpd/trunk: CHANGES modules/filters/mod_substitute.c
On Wed, 2015-06-24 at 16:03 -0400, Eric Covener wrote: For 2.4, do we need some kind of global property for people who were already working around the merge? My commit was to trunk (after someone prodded us on-list). For 2.4 we should not change existing behaviour. At least, not as a default. New directive as per Yann's suggestion? Maybe, but perhaps OTT. If so, it would be good to make it 2.4-only and keep trunk pure. And add it to upgrade notes! -- Nick Kew
Re: documentation issues for mod_authn_socache
On Mon, 22 Jun 2015 16:10:19 -0400 Helmut K. C. Tessarek tessa...@evermeet.cx wrote: On 2015-06-17 18:03, Helmut K. C. Tessarek wrote: Hello, http://httpd.apache.org/docs/2.4/mod/mod_authn_socache.html There are 2 things I want to mention: 1) cacheing is not a word. It should be replaced with caching on the entire page 2) AuthnCacheSOCache Directive: If not set, your platform's default will be used. There's no indication whatsoever what the platforms' defaults actually are. Can someone please tell me what the platforms' defaults are? I'm mostly interested in Linux and MacOSX, but a list would be nice in the docs. Does anyone have any input on this? Or is this the wrong list for doc issues? We don't know the platforms' defaults. They're up to the packagers, and those sysops who make an active decision. What you're asking is kind-of like asking us to concern ourselves with the nuances of ext2/3/4 vs XFS vs JFS vs ZFS vs NTFS, etc. We are agnostic. -- Nick Kew
Re: SSI for CGI output ...
On Sat, 20 Jun 2015 12:01:36 + Marcos Camargo mcama...@broadcom.com wrote: Is it possible to have SSI parse CGI output? Yes. If you want a longer answer, you'll have to explain what leads you to suppose there might be a problem with it. Unless you've been reading tutorials for 1998 (or earlier) Apache releases? -- Nick Kew
Re: Signal-safe way to start a worker thread in each child process?
On Tue, 02 Jun 2015 14:15:18 -0500 Jacob Champion jacob.champ...@ni.com wrote: We could just call apr_setup_signal_thread() ourselves -- and doing that does fix the problem -- but that means that modules which are initialized after us will get the global protection too, which doesn't feel clean. So, a few questions: I don't know a clean answer: it's not a problem I've ever tackled. But if you don't find a better solution, you can improve a little on your existing one by running your child_init after other modules have done theirs with APR_HOOK_LAST. -- Nick Kew
Re: 2.2 and 2.4 and 2.6/3.0
On Wed, 2015-05-27 at 22:42 +0200, Stefan Eissing wrote: Not wanting to boast, but maybe mod_h2 for httpd 2.4 can play a role in motivating people to migrate away from 2.2. I've just looked at your internals page (which seems to me an excellent piece of work), and it tends to support the gut feeling that mod_h2 might be better targeting 2.6/3.0 than 2.4. We should be joining forces to address the issues you've encountered, from minor tweaks to core to more fundamental issues like bucket alloc across threads (or a suitable alternative). Time for me to download and take a proper look at your work, and put my money (or at least timeeffort) where my mouth is! -- Nick Kew
Re: 2.4 release backport list
On Tue, 26 May 2015 10:37:28 +0200 Stefan Eissing stefan.eiss...@greenbytes.de wrote: Sorry, if this question has an obvious answer which I was unable to find: where would I find a list of the changes that will be backported to the 2.4.13 release in order to see if a change has received enough votes? The file STATUS in 2.4.x shows current votes. (See also CHANGES, which documents what has already made it). -- Nick Kew
Re: Platform specific CTR/RTC?
On Fri, 22 May 2015 01:51:49 -0500 William A Rowe Jr wr...@rowe-clan.net wrote: It might be worth mentioning that it's been in production for about 3-4 years or so, and only was delayed in 2.2 due to the unavoidable drift between trunk/2.4 and 2.2 flavors. We already included the ported-afterwards functionality in the previous 2.4.12 release, with apparently no issues. The patch below is actually the origin of the enhancement. In those circumstances it seems not so much CTR or RTC but rather commonsense to go ahead. Don't we have a bit of a history of struggling to meet RTC criteria on Windows-specific backports? I wonder if there's a case for formally adopting a lazy-consensus policy based on what wrowe is doing here? If a proposal has sat in STATUS for a qualifying period, without attracting comment/ reservations, but also without attracting sufficient review +1s, should it be eligible for lazy-consensus backport? The proponent posts here on a speak now or forever hold your peace basis, and goes ahead if no discussion calls it into question. -- Nick Kew
Re: Version check idea
On Tue, 21 Apr 2015 16:00:05 +0200 Dirk-Willem van Gulik di...@webweaving.org wrote: On 21 Apr 2015, at 15:55, Jim Jagielski j...@jagunet.com wrote: For comment: What do people think about adding the capability that when httpd is started, it tries to access http://httpd.apache.org/doap.rdf to check its version number with the latest one referred to in that file and, if a newer one exists, it prints out a little message in the error.log (or stderr)… First reaction - Eek Evil ! Breaks about every assumption I have about well defined unix programmes. However if it where something like ‘httpd C / --configcheck’ or similarily for apachectl then I could easily image that to be rather useful. +1 (to Dirk). Capability, OK. Even, configuration option, perhaps (that would allow packagers to reference their own URL). But no way would I want it calling home in a default startup behind my back! Suggestion 1: make it a module. Suggestion 2: consider something lighter-weight than RDF. e.g. an X-latest: header in our own server response to HEAD / . -- Nick Kew
