Re: Concurrent access to request body by multiple modules
> On 1 Dec 2020, at 14:29, Tim Wetzel wrote: > > Is there any possibility of having the module use, for example, a bucket > brigade or anything I am not aware of that is already in use by WebDAV and > thus operate on the request body of a file from a PUT request in > parallel/concurrently to WebDAV? How does mod_request relate to your needs? If you can't use it (e.g. because you have to deal with arbitrary-sized requests), your choices would then be to adapt it (e.g. provide for it to use a file bucket to cache requests) or to go a level down and write your own input filter module. That is, if I've understood you aright? -- Nick Kew
Re: mod_proxy_spawn: Review request
> On 29 Nov 2020, at 14:48, Florian Wagner wrote: > > Hi everyone! > > I was wondering if someone with a better understanding of httpd and > mod_proxy could review my module idea and prototype implementation and > warn me of any unforeseen pitfalls looming ahead before I commit to > implementing this fully. The main thing that springs to mind is security when you spawn a process. CGI and fastcgi are battle-hardened by decades of operational use. > Wanting to switch to mod_proxy_http for deploying backend applications > (and opening the way for WebSockets through mod_proxy_wstunnel) I'm > missing the process management provided by mod_fastcgi [1]. fastcgi is an unusual model, with a very specific purpose. It's more usual for backends to be self-managing. > While fully understanding that one can be of the opinion that process > management is best kept out of httpd, I personally like the convenience > and more importantly clarity offered by having the complete command, > arguments and environment required to run the backend application in > the httpd configuration. Authentication, URL rewriting and whatelse > will already be setup there, anyway. If it's in the config then at least it's [probably] coming from a trusted source, but then why run per-request? > So I took a shot at seeing if I could implement a module to do just > that. My current idea/prototype: > > 1. Register a hook to run before mod_proxy.c:proxy_handler and have a >look at the request filename and handler to see if they start with >"proxy:spawn://". Big red flag for security there. Hope you're paying very careful attention to your input: there's nothing to that effect in what you attached. Also I'd consider hooking it earlier in the request cycle, or into mod_proxy instead. How does mod_proxy_fcgi fit your vision? -- Nick Kew
Re: Which programming language should be used for newly developed modules?
On Thu, 20 Aug 2020 21:59:50 +0200 Simon Studer wrote: > Hi everyone, > > Recently, I was wondering which programming language should be used > for new Apache httpd modules. Whatever suits the module's developer and task. The C API has a stability promise: if you use it, your module will continue to work with (at least) future 2.4.x releases. That give you C or any language with C linkage. If you deviate from the API, you're on your own. Alternatives that (broadly speaking) wrap the C API are also possible: see for example mod_perl and mod_lua. -- Nick Kew
Re: How to read data in a request handler and then return DECLINED without consuming the data in the bucket brigade?
On Mon, 4 Jun 2018 10:23:59 -0700 Paul Callahan wrote: > Thank you for your replies. > > I did try with input filters. The reason I'm trying to do this in a > handler is because I want to return 403 to the browser if the request > body has something unsavory in it. With reverse proxied requests, > it appears the input filter fires too late and if I try to send a > bucket with 403 in it, it is ignored and a 400 goes back to the > browser. In the debugger, I see the fixup call back being hit > before my input filter. If I could get the input filter to trip > sooner without consuming the request, I could go with that. If I > call ap_get_brigade() in an earlier handler to trip the input filter, > it appears the request body is consumed. OK, that's actually quite a complex task, especially if you need to deal with larger requests. It is, however, as task that's been done in open source code you can look at, or perhaps use instead of reinventing their wheel. Either Ironbee or mod_security will scan a request body for you. > btw, Nick I bought your book - it was a great help :) Thanks :) -- Nick Kew
Re: How to read data in a request handler and then return DECLINED without consuming the data in the bucket brigade?
> On 4 Jun 2018, at 08:55, Sorin Manolache wrote:
>
> On 2018-06-04 08:27, Paul Callahan wrote:
>> In apache modules, my understanding is if a handler declines a request, the
>> request is passed on to the next suitable handler. I'm finding though if
>> I read the bucket_brigade/request body, and then decline the request, the
>> subsequent handler doesn't get any data. It is like the act of reading the
>> bucket brigade consumes it.
>> I would like to have a request handler read the data, do some task (in this
>> case just count bytes), and decline the request without consuming the data
>> for the next handler.
Why is that a handler? An input filter could count the data and pass them
straight down the chain, avoiding any such problem. At a glance, your code
would need very little modification to work as a filter.
Exhortation: work with the server architecture, not against it!
> Hello,
>
> As far as I know, there is no simple way to do that.
This is true, in the sense that data get consumed. To do otherwise would be
monstrously inefficient, and turn a big request body straight into a DoS.
This is why we have mod_request. It provides an input filter you can use
explicitly to buffer data which then remain available to the next module to
read them.
Use with caution: for example, if you fail to limit request size, you could find
yourself trying to buffer gigabytes of request data.
>> } while (!end && (status == APR_SUCCESS));
>> if (status == APR_SUCCESS) {
>> return DECLINED;
>> } else {
>> return HTTP_INTERNAL_SERVER_ERROR;
>> }
>> }
Minor tip there: you're turning EAGAIN into a fatal error.
--
Nick Kew
Re: Discard a brigade from filter
On Thu, 19 Oct 2017 16:30:27 +0200 m...@netbsd.org (Emmanuel Dreyfus) wrote: > Hello > > Is there a way to completely discard a brigade in an input filter, and > not pass it through filter chain? This doesn't quite make sense. An input filter pulls data into the brigade supplied by its caller, so you are in control all the time. > Removing all buckets cause an empty brigade to be sent to next filter > and that causes trouble. You mean, returned to the caller? That shouldn't matter (indeed it should be an expected behaviour in a non-blocking filter). If it is a problem (perhaps due to a bug outside your control), issue a blocking call to your own upstream and don't return anything until you have data (or EOS). Or if I were working around a bug in closed source, I might try inserting a placeholder such as an empty data bucket. -- Nick Kew
Re: HTTP_FORBIDDEN and sub-requests
On Fri, 2017-04-14 at 12:55 +0300, Donatas Abraitis wrote: > Hi folks! > > I have a such code snippet: > > char *proxy_ts = (char *) apr_table_get(r->headers_in, conf->deny_header); > if (!proxy_ts) > return HTTP_FORBIDDEN; > apr_table_unset(r->headers_in, conf->deny_header); > > This unsets the arbitrary header properly in application (phpinfo()), but > if the site is non-single page (with many images, css, js, etc.) it always > returns 403. It looks like there is some kind of sub-requests for those > resources. > > How do you solve such cases with requests? Well, I should start by figuring out where and why that's happening. On the server side, gdb works as fallback tool for that if you have no better ideas. If, once you've figured out, you're happy that it's not a symptom of some deeper bug, you just leave a "been here" breadcrumb. -- Nick Kew
Re: mod_ssl custom vhost module
On Thu, 2017-03-30 at 15:20 +, Michael Sløgedal wrote: > I looked a little on the mod_ssl source code, and it seems it does a lot of > preprocessing on config stage, and relies on a combination of VirtualHost and > ServerName / Alias directives. > I suppose this means that mod_ssl wouldn't work with grabbing certificates > based on a path stored in sql on-the-fly. I'm not familiar with the murky recesses of mod_ssl. But if I've understood you aright, I think a good startingpoint would be to see if you can hook something in to connection processing, that'll in turn run something ahead of mod_ssl getting in to a connection. Not sure if that actually leads anywhere useful. Just a thought, if you haven't already tried it. Your main problem is that you have a hack that shoehorns vhosts in where they don't belong. -- Nick Kew
Re: Can byterange filter request only needed ranges from my module instead of discarding?
On Sun, 2017-02-26 at 21:00 +0300, Basin Ilya wrote: > Instead of writing data in my handler can I create there a custom bucket, let > the byterange filter split it properly and let the core filter call my custom > read function? I think so in principle. Your bucket type would have to satisfy byterange_filter, meaning it has to know its own length. In practice, it may not work as you expect. If there's any content filter between your handler and the byterange module, then that will read your bucket and defeat your purpose. Why not write a mock-up of your proposed design? Say, a bucket that serves data from a static file by seek/read, just to see how it behaves in different configurations and whether you can make the architecture work for you? -- Nick Kew
Re: Can byterange filter request only needed ranges from my module instead of discarding?
On Sun, 2017-02-26 at 15:02 +0300, Basin Ilya wrote: > However, it's inefficient to serve huge virtual files this way when only a > small part of such file requested. How to solve this? Unless your module can handle the byterange itself, you might want to consider cacheing the generated file for the benefit of future byterange requests. -- Nick Kew
Re: Change the content-length header for other filters
On Wed, 2016-12-21 at 22:10 +0100, André Rothe wrote: > But after my filter completes the request processing, I'll get: > > Sending error response: The request contained fewer content data than > specified by the content-length header Sorin's reply is part of the story, and may or may not be useful in your case. The basic issue you have to consider is that the headers arrive before the body, and a handler that cares about Content-Length is likely to have read it before your filter has reset it. The alternative - read and buffer the entire body before starting to process it - becomes hopelessly inefficient for large requests. There's some discussion of the issue in the mod_proxy docs, as mod_proxy has an option to support HTTP/1.0 backends that need an explicit Content-Length. -- Nick Kew
Re: Tracking sent responses
On Fri, 6 Nov 2015 09:12:40 -0500 Julien FROMENT wrote: > Here is the pseudo code: > > -- Client send a request > > -- Apache processes the request and send it to the backend server > > ... > > -- The backend server returns the response > > -- Apache sends the response to the client > > -- Apache calls Async API with the number of bytes sent Hmm, that sounds like something I wrote (not, alas, open source) for a client many years ago. But if you're only looking for bytes sent, it's probably easier than that. Before investing in new development, consider: - Could you hook your notification into regular piped logging? - Would regular logging through an API like syslog or spread serve (there are third-party modules for those). - Would a security-oriented tool like Ironbee be complete overkill? -- Nick Kew
Re: Signal-safe way to start a worker thread in each child process?
On Tue, 02 Jun 2015 14:15:18 -0500 Jacob Champion wrote: > We could just call apr_setup_signal_thread() ourselves -- and doing that > does fix the problem -- but that means that modules which are > initialized after us will get the global protection too, which doesn't > feel clean. So, a few questions: I don't know a clean answer: it's not a problem I've ever tackled. But if you don't find a better solution, you can improve a little on your existing one by running your child_init after other modules have done theirs with APR_HOOK_LAST. -- Nick Kew
Re: output filter needs to redirect to 503 error status
On Thu, 16 Oct 2014 09:36:19 -0400 "Eric Johanson" wrote: > My output filter hooked function is defined as follows: >apr_status_t mts_out_filter(ap_filter_t *f,apr_bucket_brigade *bb) > > I need this function to "do something" that causes the whole request to be > redirected such that the client sees a 503 error status with no > body/content. > > Things that I've tried so far: > * Returning HTTP_SERVICE_UNAVAILABLE from the output filter function after > calling "ap_pass_brigade(f->next,bb)" HTTP_SERVICE_UNAVAILABLE isn't an apr_status_t! > * Setting f->r->status to HTTP_SERVICE_UNAVAILABLE after calling > "ap_pass_brigade(f->next,bb)" after ap_pass_brigade, you've already returned a status down the line to the client. It's too late to change it! > * calling "ap_send_error_response(f->r,HTTP_SERVICE_UNAVAILABLE)" That would have to go through the filter chain. Filters are supposed to be about your data. They can in limited circumstances change metadata (e.g. set a 503), but you'd need at least to set r->status before the first call to f->next. -- Nick Kew
Re: binding an external C library with I/O methods
On Mon, 30 Jun 2014 11:56:50 +0200 Pierre Lindenbaum wrote: > (cross posted on SO: http://stackoverflow.com/questions/24486926) > > I'd like to use an external library that is able to write files using a > custom format. The API for this library looks like: > > MyFormatPtr format= myformat_openfd(int filedescriptor,const char* > mode); > MyFormatPtr format= myformat_open(const char* filename,const char* > mode); > (...) > void myformat_close(MyFormatPtr format); You probably want to look elsewhere in your API. An apache request isn't a file, and has no file descriptor. If your library is designed to be usable in non-file applications, it'll offer some mechanism for plugging in your own I/O functions. If not, you could perhaps try some ugly hack: create some bucket of file-compatible type for it to write to. -- Nick Kew
Re: Apache module development on Mac OS X
On Tue, 17 Jun 2014 17:27:15 +0530 Sindhi Sindhi wrote: > Any help will be highly appreciated. You're trying to build the module as a program. You need to build it as a shared object (think DLL, since you seem to be from a Windows background). Apache provides a tool to do that, with all the right paths and linkage. Check apxs in the docs. -- Nick Kew
Re: tcp/ip sockets in apache module
On Fri, 2014-05-23 at 10:27 -0700, Jeremy Thompson wrote: > I'm trying to write a module for apache. I've successfully compile in a > test module that doesn't do a whole lot yet. I would like to be able to use > tcp/ip sockets in the module to talk to another server. This may or may not make sense in your application, but have you considered using the proxy framework? That might involve a mod_proxy_yourprotocol to handle the tcp requirements. Your main module would then create a proxy subrequest when it needs a connection. See mod_include for an example, where
Re: Writing a monitoring thread in apache filter moudle
On 20 Apr 2014, at 09:05, Hassan Monfared wrote: > Dear members, > I have global static array in my module What do you mean by a global static array? In normal operation, apache runs multi-process. So what looks like global static is actually per-process. Think that through and it'll probably tell you what's wrong. > I wrote a monitoring thread with boost::thread which runs in background and > updates the list every 5 seconds. You'd probably be better off with apr threads: two separate threading models will add complexity and detract from portability and maintainability. Either run the update per-process or use shared memory. In the latter case, use apache's (and apr's) mechanisms for shared memory and timed monitoring to avoid introducing new complexity. -- Nick Kew
Re: Apache module avoiding re-initialization of connection pool in every worker process
On 31 Dec 2013, at 15:37, Eric Covener wrote: > You can keep doing it once per process and share across threads. If > you don't manage them very smartly today, maybe apr_reslist would help > in the multithreaded case. To expand a little on that, take a look at mod_dbd, which uses apr_reslist to maintain a pool of database connections shared across threads. Your task looks quite similar to that! -- Nick Kew
Re: apr_hash_t and global scope
On 12 Dec 2013, at 02:41, Ingo Walz wrote: > So socache with shared memory (shmcb) is the way you would suggest? How important is the sharing in your case? If it's just about performance, you may be better-off accepting that each process will repeat the work of populating the hash, if that's a smaller overhead than IPC. If sharing is necessary, then socache will indeed serve, though you can leave the choice of socache engine to your config. I was also a little uneasy about your original description: the use of a forever-pool becomes a memory leak if it is populated with per-request data. Modules that do such things usually create their own pool (as a child of the process pool) which they can then maintain privately. -- Nick Kew
Re: How to distinguish between start and stop in a development
On Thu, 2013-09-05 at 09:20 +, markus.k...@here.com wrote: > Hi, > > I am developing a module which must do some clean-up work, when httpd is > stopped. Register a cleanup on the process pool. If you need that explained, see http://www.apachetutor.org/dev/pools -- Nick Kew
Re: Apache httpd sends "400 Bad Request" to client due to IPvFuture (RFC 3986) format IP address Hostname Host Header
On 2 Aug 2013, at 20:02, Rizzo, Christopher wrote: > The Apache HTTP server is rejecting the request (with "400 Bad Request") due > to this Hostname value in the header before a registered IPP Apache module > even sees it. A quick check confirms what you say. > Is it possible to write a hook or filter that would prevent Apache from > rejecting this request and allowing this Hostname Host Header value to pass > thru, or does the Apache base source code need to change for this? This fairly clearly wants a patch to the core code. Its comment suggest it's not trying to be strict, but just to filter out characters that couldn't feature in a Host: header (at the time the code was written) and save having to think about security of filesystem-sensitive characters. > and the Apache code that is logging this error appears to be fix_hostname() > in ./server/vhost.c Yep, that's the one. Looks like the protocol you're using is NOTIMPL. I'd suggest filing a new bug in bugzilla, referencing the protocol you want to support and what characters/patterns should be allowed but are currently blocked. Best chance of getting it dealt with quickly may be if you also supply a patch, but make sure to address the security concerns hinted at in the comments if you do that. You'd also want to move any further discussion to the dev list. -- Nick Kew
Re: Can a module control the socket transport protocol?
On 18 Jun 2013, at 22:18, Jon Leighton wrote: > I've been looking into the possibility of writing a module to support using > SCTP as an alternative to TCP. After spending some time reading "The Apache > Modules Book" and looking through the code, I have the impression that the > module architecture doesn't support controlling the types of sockets that are > opened. I'm hoping someone can either confirm or correct my impression. > Thanks very much. It's not covered by the book, and I'd need to tackle pretty much the same learning curve as you to hack it. Your startingpoint would probably be APR's network_io module. If you're thinking HTTP-over-SCTP (if indeed that makes any sense) then that may be most of what you need to do. If not, or if you want to do everything as a module, you could start by looking at protocol modules like mod_ftp or mod_smtp. -- Nick Kew
Re: Authentication/Authorization module vs. Basic Authentication
On 17 Jun 2013, at 10:24, Christoph Gröver wrote: > > Hello list, > > I thought I'd let you know, what I found out so far. > Perhaps someone will have an idea what is going on. > > In the access checking phase started by this line > > ap_hook_access_checker(SumpfAuthChecker, NULL, NULL, APR_HOOK_FIRST); > > I return with a "return(HTTP_MOVED_TEMPORARILY)". > I have set up the new location to go to by setting > the apropriate headers with > apr_table_set( r->headers_out, "Location", newlocation ); You'd want the err_headers_out to set that for an error return. > Instead of sending back to the client a 302 or a 301 the next thing > that happens the apache sends back a 401. Have you traced and/or stepped through execution of your own code? Could it be that your errordocument itself authenticates the client? > I tried to find out with "LogLevel debug". > But this actually leads to nearly no extra lines in the log files. My usual tool in that situation is gdb. -- Nick Kew
Re: C++ Apache module fails to load
On 11 May 2013, at 07:35, Sindhi Sindhi wrote: > > Could you please advice? Did you check the answer in the FAQ? -- Nick Kew
Re: Apache Buckets and Brigade
On 1 May 2013, at 14:41, Sorin Manolache wrote: > In my experience the buckets that I've seen have about 8 kilobytes. So you > will not consume too much memory if you "flatten" the bucket brigade into one > buffer and then perform the replacement in the buffer. (see > apr_brigade_flatten). However, you have to provide for the case in which > oldString is split between two brigades. Don't rely on that. The size of buckets and brigades depends entirely on what comes before your filter. 8k chunks are common from some sources (e.g. mod_proxy), but even there another filter could change all that - e.g. mod_deflate if the data arrive compressed. Bottom line: don't make assumptions, as there are no guarantees. You can of course look at existing filters that do similar things to yours. Or even read about it in my book :-) -- Nick Kew
Re: module using C++ library
On Wed, 6 Mar 2013 21:19:49 +0100 Somostetoi Kilato wrote: > The error is: > > /home/xyz/indian/conf/httpd.conf: Cannot load modules/mod_xyz.so into > server: libmod_xyzi.so: cannot open shared object file: No such file or > directory Most likely you have unresolved symbols in libmod_xyzi.so (could be standard/platform C++ libs such as libstdc++ / libg++). Use your system tools (likely ldd and nm) to find the dependencies, an load them before your lib. -- Nick Kew
Re: Stuck on DBD DSO Lock
On 5 Feb 2013, at 18:03, Reyad Attiyat wrote: > The apr_dbd_get_driver call blocks in the child process I should also note > this doesn't block when I'm running apache in debug (-X) and that I'm using > apache 2.4 with the event mpm. Hmm. > #4 0x7f59a03578fc in apu_dso_mutex_lock () at misc/apu_dso.c:46 > #5 0x7f59a034e036 in apr_dbd_get_driver (pool=pool@entry=0x21c2138, >name=0x7f599b1ea21c "mysql", driver=0x230ee40) at dbd/apr_dbd.c:167 Looks like something else has that mutex when your function runs. The mutex was changed in r659293, which shares the same mutex between dbd and ldap, and makes it available to other modules. Do you have ldap loaded, or anything else using apu_dso_mutex_lock that you're aware of? I don't like the logic of it. A mutex should only be required if the driver is not yet loaded. Perhaps file a bug against APR? > #6 0x7f599b1e4e8b in connect_database (db_pool=0x21c2138, >error_messages=0x7f599a7c7000, dbd_config=dbd_config@entry=0x2273940) >at database/dbd.c:35 Would it not make sense for your module to use mod_dbd to manage a database connection pool? -- Nick Kew
Re: how to do something in a mod when apache is shutting down?
On 5 Feb 2013, at 02:29, chary wrote: > I'm writing a mod for apache, and I need some help. > > how to do something in a mod when apache is shutting down? > > What kind of hooks could be used? Register your function as a cleanup on the process pool. -- Nick Kew
Re: "Close" HTTP connection callback/hook
On Tue, 16 Oct 2012 13:50:12 + Evgeny Shvidky wrote: > Hi, > > I am implementing a new module on C. > I need to perform some functionality when a user closes a HTTP connection > before he received any response for his request. > How can I know when a HTTP user request state has been changed/closed? > Is there any callback/hook for this functionality I can register? Is this just a cleanup, or some action you take only on error? And is it per-request or per-connection? If a cleanup, just register it on the request pool (or the connection pool, as appropriate). You can check the return status from the output filter chain. If there's an error, check for closed connection. You can do that in a handler or an output filter. -- Nick Kew
Re: best way to return the content of a file
On 18 Aug 2012, at 13:17, nik600 wrote: > Thanks for pointing me to the right direction. > > I've already tried to change order of execution i'v also tried to use > APR_HOOK_REALLY_FIRST or APR_HOOK_FIRST or APR_HOOK_MIDDLE but the > result is the same. That has nothing to do with it! Several people have tried to help you, but you need to learn a bit before you're capable of being helped. If you aren't going to learn the basics of the request processing cycle (which can also be found in the docs at apache.org, where they were recently updated), stick to something simple like CGI or PHP. -- Nick Kew
Re: best way to return the content of a file
On 18 Aug 2012, at 09:51, nik600 wrote: > Is better: > > - to read the file and then send is using ap_rprintf Bad. > - to use apr_bucket_* Acceptable. > - to simply redirect the request changing the r->uri and then return DECLINED. Yep. > The strange thing is that i get a 404 error and in error.log i get: That's because you set r-->uri too late, after it's already been mapped to the filesystem. Others have already told you to hook in earlier in the request processing cycle. I guess it's time to point you at http://www.apachetutor.org/ and the new programmer docs at httpd.apache.org. -- Nick Kew
Re: AP_INIT_TAKEn macros / gcc 4.6.3?
On Tue, 24 Jul 2012 21:17:51 +
"Cantor, Scott" wrote:
> Actually, I had this wrapped already. What broke is that this particular
> build is turning on AP_HAVE_DESIGNATED_INITIALIZER, which is based on:
>
> #if (defined(__GNUC__) && !defined(__cplusplus))\
> || (defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L)
> #define AP_HAVE_DESIGNATED_INITIALIZER
> #endif
>
> The second half of that ifdef is activating even for g++, which is
> definitely not supposed to happen. Not sure why yet.
Did you fix it?
My recollection of the problem is from several years ago (I think it
first hit me updating from 2.0 to 2.2), and I have no actual recollection
of what fixed it. My reply to you was based on a quick look at the
code in question, which uses the extern "C" { ... } and works!
--
Nick Kew
Re: AP_INIT_TAKEn macros / gcc 4.6.3?
On 24 Jul 2012, at 18:23, Cantor, Scott wrote:
> I'm still investigating, just wondering if anybody has seen this.
Yes. It's not new: C++ doesn't like those macros.
Simple fix is to enclose them in extern "C" { ... }.
Alternatively, expand them by hand as per C rules.
--
Nick Kew
Re: Broken request_rec structures passed to hook methods in Apache 2.0
On 5 Jul 2012, at 04:42, Tim Whittington wrote: Use apxs from your installed httpd to build your module. That way you'll get the right compile options and header files. And for sake, upgrade! 2.0 belongs in the museum! Then you can deal with the possibility that your module is using a subrequest or internal redirect in a way that doesn't work with 2.0. -- Nick Kew
Re: Best (safest) way to edit char string (from envvars)?
On 20 Jun 2012, at 15:35, wrote: > The client cert char string returned has the extra beginning line (-BEGIN > CERTIFICATE-) and ending line (-END CERTIFICATE-), but I need to > remove both of those lines for a call that I need to make. That's not quite the same as needing a mutable string. Can your call be modified to accept a non-null-terminated pointer together with a length? You would of course determine those by parsing rather than copying the original. -- Nick Kew
Re: output filter ordering question: mod_deflate vs mod_headers vs mod_pagespeed
On 17 May 2012, at 04:24, Joshua Marantz wrote: > Or is that insane &/or dangerous? AP_FTYPE_ values are something of a blunt instrument, not ideal for cases where you care about ordering. Running mod_headers as a filter is a bit of a hack. What you're proposing is kind-of building a hack onto a hack. I'm sure you can make it work (at least in controlled circumstances where you're not competing against other similar fudges) but IMHO you're right to be worried. Having once upon a time written mod_filter to deal with more complex configuration than the original design envisaged, I wonder if your issue might be a hint that we should revisit the API for programmatic configuration, and make your proposal less hackish? -- Nick Kew
Re: about setting r->headers_out
On Thu, 10 May 2012 12:36:54 +0200 Daniel Gruno wrote: > On 10-05-2012 12:25, Julio Carlos Barrera Juez wrote: > > Hi! > > > > I have the same problem than you, Rui Hu. I have an output filter that > > changes the body of the response, but then "content-type" header is > > changed to "text/plain", and I want to maintain the original > > "text/html". Erm, that doesn't sound like an original anything. More like the core inserting a default type when none was set. > Have you tried using ap_set_content_type(r, "text/html") instead? > I noticed that in its description, it says: That should be the usual solution. > "This function must be called to set r->content_type in order for the > AddOutputFilterByType directive to work correctly.", so it may be > related to your problem. Whoops! That's no longer true since AddOutputFilterByType moved to mod_filter. Where do the docs still say that? -- Nick Kew
Re: How can I hook exactly just before the default file handler?
On Thu, 26 Apr 2012 14:06:22 -0600 Fred Clift wrote: > Hi all, > > I see that you can specify in the 2nd and 3rd options of the ap_hook_* > calls modules that should be before and after your module. The > documentation states that this controls the sort-order of which modules > execute when. I suspect you're making your problem a whole lot more complex than it need be. > Ideally I'd like to make sure I have a hook after any filename/path > translation is done so I can add a header with reasonable confidence that > nothing will twiddle it before it gets to the default file handler. A fixups hook would be the usual solution to that kind of requirement. I'd > also like to make sure that no interpreters handled my file. That's up to the server admin, not up to any module. If a server admin sets an interpreter (such as PHP) as handler, that's their business. If your module wants to take control of a request then it can set its own handler (or unset the handler to get the default). But then it needs to document that it breaks normal configuration! -- Nick Kew
Re: How can I hook exactly just before the default file handler?
On Thu, 26 Apr 2012 14:06:22 -0600 Fred Clift wrote: > Hi all, > > I see that you can specify in the 2nd and 3rd options of the ap_hook_* > calls modules that should be before and after your module. The > documentation states that this controls the sort-order of which modules > execute when. I suspect you're making your problem a whole lot more complex than it need be. > Ideally I'd like to make sure I have a hook after any filename/path > translation is done so I can add a header with reasonable confidence that > nothing will twiddle it before it gets to the default file handler. A fixups hook would be the usual solution to that kind of requirement. I'd > also like to make sure that no interpreters handled my file. That's up to the server admin, not up to any module. If a server admin sets an interpreter (such as PHP) as handler, that's their business. If your module wants to take control of a request then it can set its own handler (or unset the handler to get the default). But then it needs to document that it breaks normal configuration! -- Nick Kew
Re: STL/Boost containers in Apache module
On Mon, 23 Apr 2012 13:25:41 +0100 Rajalakshmi Iyer wrote: > Hello, > > Is it possible and safe to use STL containers within Apache modules? If you have code that uses them, go right ahead. If you're writing new code, you may find also APR offers alternatives to many STL constructs, and serves to make your life very easy in the context of a module! -- Nick Kew
Re: How to register additional operators or functions for use in expressions?
On Tue, 27 Mar 2012 00:42:43 +0200 Waldemar Klein wrote: > Hi all, > > looking at http://httpd.apache.org/docs/2.4/expr.html, i found things like > "Modules may register additional unary operators." or > "Modules may register additional functions." > but not the slightest hint how to actually do this, also googling for > some time didn't get me anything useful either. You register your own functions for relevant parts of expression parsing. To get started, I suggest you read the extensive comments in the header file include/ap_expr.h. -- Nick Kew
Re: Using apr_hash_t within shared memory
On Wed, 21 Mar 2012 11:31:30 + Rajalakshmi Iyer wrote: > Hello, > > I want to be able to store a hash map (apr_hash_t) in the Apache shared > memory (created using apr_shm_create). This map will be created once and > will be shared by all child processes. You can't. Well, not unless you hack deep in APR to allocate from shm. But you can achieve a shared hash using mod_socache. -- Nick Kew
Re: Calling another URL from output filter
On Sun, 4 Mar 2012 10:19:53 -0800 (PST) Swaminathan Bhaskar wrote: > > Hello, > > How can I call another url from an output filter The easy way is to use a subrequest. "Include Virtual" from mod_includes is an example. -- Nick Kew
Re: one problem when calling ap_get_module_config
On 29 Feb 2012, at 10:47, Rui Hu wrote: > I use r->request_config to store module data as a substitute for global > vars. One type_checker_hook function uses ap_get_module_config at its > beginning. But what I got is NULL. Strangely, post_read_request_hook > function works fine, whose codes is basically the same. Any subrequests or internal redirects involved? Look carefully at the request object itself. Or tyops? -- Nick Kew
Re: how to best implement my own connection pool
On 18 Feb 2012, at 01:53, Sam Carleton wrote: > Basically what I am looking for is the same basic connection pooling that > is implemented in the mod_dbd. Is there an easy way to do this, or will I > simple need to get into the internals of the mod_dbd to figure out what it > is doing and do it myself. Skimming through your post, it's not clear what you want that mod_dbd doesn't provide. Why not focus on that question? Once you've identified the gap, one approach might be to enhance mod_dbd, and if anything of general interest emerges, maybe propose it for inclusion upstream? The other answer is, the easy way is to use apr_reslist, which is what mod_dbd does. -- Nick Kew
Re: NoRobot module
On 15 Feb 2012, at 08:07, Mike Baroukh wrote:
> Disclaimer :
> I'm absolutly not a C ou System developer.
> I'm a Java developer.
> And this is my first module.
> So maybe it could be made better ...
If you're asking for criticism, here goes:
1. It looks fine as far as it goes.
2. But would be much more generalisable if it were configurable on/off.
This would remove the issue of running order which you tackled with
APR_HOOK_FIRST.
3. "Be conservative in what you send". The last line of your
robots.txt is unterminated!
--
Nick Kew
Re: A few questions on Input Filters
On Wed, 11 Jan 2012 11:17:30 + Martin Townsend wrote: > Hi, > [chop] One point that Joe omitted to mention is that since input filtering is a PULL API, it's entirely up to your filter how much data to pull before returning to its upstream. If your filter can't deal with partial data, it can block for as long as necessary to read more. With just a couple of Kb to handle it's not exactly going to be a performance hit! Regarding the flush buckets, conceptually I'd expect they might feature if you were handling streaming data, or maybe multipart uploads, or some extension to the HTTP protocol. Maybe there's some such application- or ptotocol- oriented filter in your chain? -- Nick Kew
Re: Reading content of requests entering Apache
On Mon, 9 Jan 2012 10:33:53 -0500 "Pranesh Vadhirajan" wrote: > Hello All, > > I would like to know how to be able to read the content of the request > that Apache receives. What you describe is called an Input Filter. There are examples in the the standard apache distro: for example, mod_deflate and mod_sed offer input filters. mod_security and mod_ironbee are third-party examples. I could also recommend the book: see http://www.apachetutor.org/ -- Nick Kew
Re: module development suggestion request
On 28 Nov 2011, at 07:58, Reza Shadmani wrote: > Give me a clue where to start... I kindly please to receive your hints and > experience in this matter. What you describe is a protocol module. There are a few examples available: mod_ftpd is probably the most mature at apache.org implementing non-http. -- Nick Kew
Re: basic example shared memory code
On Tue, 22 Nov 2011 09:41:02 -0500 "Pranesh Vadhirajan" wrote: > Nick, can you suggest some of these higher-level abstractions, please? I > have been trying to make a module of mine work with a POSIX shared memory > implementation, but I'm going nowhere with that. Are you referring to the > apache shared memory implementation (apr_shm_...) or something else? Either > way, if you could suggest what I should look into, it would be greatly > appreciated. apr_shm is the old way of doing it. Today I'd recommend looking at mod_slotmem, and the socache modules. I used the latter for mod_authn_socache, which is a simple example. -- Nick Kew
Re: basic example shared memory code
On 22 Nov 2011, at 09:26, Oğuzhan TOPGÜL wrote: > Hi guys, I'm trying to learn shared memory and mutex concepts and i need an > example shared memory apache module code that was written in c. If you're planning to write a module, bear in mind that apache now provides easy-to-use higher-level abstractions for shared memory. Older modules had to work much harder to do the same thing, so looking at them may not be your best approach. -- Nick Kew
Fw: flush or pass filter brigade to avoid memory exhaustion
Ray, looks like this got lost in the ether. I only realised it when I got your contact from my online Form! I presume you're reading where you posted! Begin forwarded message: On Mon, 14 Nov 2011 13:00:07 -0600 Ray Morris wrote: > I would appreciate some help with splitting and passing a brigade in > an output filter, to avoid using memory proportional to the size of > the response and allow data to begin to be output prior to the > completion of the filter. Studying the apache.org docs, the book, > and other modules, I haven't been able to get this working. Trying > to merge the code from the docs with a sample module, the connection > is closed after 751,143 bytes. Not quite sure what your question is. You seem to have figured out what you're doing! > APR_BUCKET_REMOVE(b);// <-- new code > APR_BRIGADE_INSERT_HEAD(ctxt->tmpbb, b); // <-- new code If you add a flush bucket HERE it becomes less likely the next filter in the chain will buffer it. > rv = ap_pass_brigade(f->next, ctxt->tmpbb); // <-- new code > apr_brigade_cleanup(ctxt->tmpbb);// <-- new code > apr_sleep(1); // <-- new code Could that be triggering a timeout? -- Nick Kew
Re: mod_proxy retry
On Tue, 1 Nov 2011 13:09:57 -0400 Jodi Bosa wrote: > I couldn't spot anything useful in mod_proxy config, and figured should be > simple enough in an output filter - but how to trigger the resubmit of the > request? If the response is HTML, mod_includes will parse it. It'll also submit another request via #include virtual. -- Nick Kew
Re: log request before and after filters
On 16 Oct 2011, at 00:14, Jodi Bosa wrote: > Is there a module that can record requests+responses before and after other > filters have been invoked? You mean like mod_diagnostics? -- Nick Kew
Re: running a module as a different uid
On Tue, 4 Oct 2011 23:43:44 +0100 (BST) Doug Bridgens wrote: > my question is, are there any suggestions as to something like switch uid > on a module basis ? this module is only enabled for a specific You can't in general. The uid is an attribute of the process, not of some part of it. There are various workarounds, with setuid CGI (and variants on that) the most common. But take a look at mod_privileges, which would enable you to do what you want on Solaris. You might be able to hook into selinux to do something similar. -- Nick Kew
Re: Question on sub requests and output filter context.
On Thu, 15 Sep 2011 11:52:38 +0100 Martin Townsend wrote: > Should this new filter also > inherit the output filters context? Am I doing something wrong with my > use of mod_include? I've tried moving my filter so it's after > mod_include but still the same problem. This looks reminiscent of https://issues.apache.org/bugzilla/show_bug.cgi?id=17629 a bug that lurked a long time before being fixed! I suggest you read that - particularly comment 30 and later, and see if it sheds any light on your problem. -- Nick Kew
Re: mod_authnz_mysql
On 17 Sep 2011, at 20:45, Christopher Dyck wrote: > Hi, > > I tried to convert mod_auth_mysql to mod_authnz_mysql. > Don't expect too much, it's my first contact with Apache modules. Just to check ... Are you aware of the DBD framework? In 2.2 (and up) the standard way to use MySQL-based authnz is with mod_auth[nz]_dbd and the MySQL backend. Also, I don't see a license in your module. You are aware that MySQL is GPL licensed. You should at least check their terms before distributing your module under any license other than GPL. Sorry if you already knew all that: there's always room for more ways to do a job. But it wasn't clear from your post! -- Nick Kew
Re: Question about malloc / realloc in module
On Wed, 14 Sep 2011 14:01:19 +0200 Christoph Gröver wrote: > So would this result in a memory leak? (if this happens often!) Potentially yes, unless you can do an exhaustive analysis of all possible processing paths. The fix for that is to register a free as a pool cleanup immediately after the malloc/realloc. See mod_proxy_html for an example. -- Nick Kew
Re: Question about Setting Request Headers and Mod-Headers
On 11 Sep 2011, at 06:36, Suneet Shah wrote: > Hello, > > I am new to creating apache modules. I need to create a module very similar > to mod_headers where I need to add headers to a request. However, instead of > getting the headers from a config file I need to get it the details from my > database. Are you sure you need a new module? mod_rewrite can set headers for you, and RewriteMap can get them from a database. If you are writing a new module, you could look at how that works. Also, be sure to check out mod_dbd for how to access SQL databases more generally. -- Nick Kew
Re: RewriteRule question
On 25 Aug 2011, at 11:04, Denys wrote: > > Do you mean, Nick, that I have submitted the question to the wrong branch? branch? What's a branch? This is a mailing list for module developers! Sorry, my reply was OTT. I just get a bit cross at some of the stuff that gets misdirected from nabble.com. -- Nick Kew
Re: RewriteRule question
On 25 Aug 2011, at 11:04, Denys wrote: > > Do you mean, Nick, that I have submitted the question to the wrong branch? branch? What's a branch? This is a mailing list for module developers! Sorry, my reply was OTT. I just get a bit cross at some of the stuff that gets misdirected from nabble.com. -- Nick Kew
Re: RewriteRule question
On 25 Aug 2011, at 10:15, Denys wrote: > Any kind of help is greatly appreciated. Start by reading TFM. If you still can't understand, try a user support forum. > View this message in context: > http://old.nabble.com/RewriteRule-question-tp32332668p32332668.html Oh, right, a website that appropriates our list and misleads (some) users. > Sent from the Apache HTTP Server - Module Writers mailing list archive at > Nabble.com. But that looks clear enough to me. Is the website so two-faced as to be unclear about that, or do you struggle with English? -- Nick Kew
Re: My first Apache module!
On 24 Aug 2011, at 22:59, Chris London wrote: > Hey everyone, > > I don't like to bother mailing lists with beginner questions so I spent > a few hours on Google looking for info and tutorials. Good! > Here's the source code: > https://github.com/chrislondon/Dynamic-Stylesheets/blob/master/mod_dss.c Hey, I thought for a moment you were doing something with DSSSL. You got me all excited! You're at risk of leaking memory and file descriptors anytime something sends processing into an error path while they're allocated/open. The solution is to use pools. See http://www.apachetutor.org/dev/pools Less crucially, the APR abstraction of file IO is preferred over FILE*. Minor point: it's good practice to make your symbols static unless they're an exported API. Why is T_fread spilled over to global namespace? Once you're a bit more confident, you might want to re-write the parser as a filter, so it can process your data format not just from a static file but also from a dynamic or proxied source. -- Nick Kew
Re: I need some idea about one unusual module with threaded communication :
On Mon, 22 Aug 2011 07:21:25 -0600 Joe Lewis wrote: > On Sun, Aug 21, 2011 at 8:57 AM, milad rezai wrote: > > > How can I communicate between different thread (one thread per client)? > > > > Problem : I need to develop a module with persistent connections, I have > > two > > kind of clients : one of them persistently sends data on connection and > > another must use and get from first kind of connections and place on his > > socket for feeding his client? > > I didn't reply to that because the problem description confuses me. How many connections, and where do they come from? Doesn't sound like standard HTTP! > Consider using shared memory - or shm. Makes sense ... probably! >Using a search engine and "apache > module shared memory example" gives a previous thread on this list : > > http://marc.info/?l=apache-modules&m=113977658131259 > > In it, Nick gives a pointer to look at the util_ldap source. That's old! These days we have two shared memory frameworks: slotmem and socache. I'd look there first, rather than duplicate older code. -- Nick Kew
Re: Can't access module config from handler function
On 14 Aug 2011, at 03:51, Simone Caruso wrote: > >> because the config it sees hasn't been merged for the > The config is a server based confing not a per-dir one. Then maybe your configuration is attached to the wrong virtualhost. Look carefully at what gets passed in the post_config hook: you can access every vhost with care, or set up something server-wide. -- Nick Kew
Re: Can't access module config from handler function
On 13 Aug 2011, at 17:06, Simone Caruso wrote: > Hi all, > > im writing a module that stores data in a shared memory to set php conf values > and other in 'traslate_name' hook.. it works fine. > > I decided to output the shm content within an http request using an handler > (like mod_status). > > In configuration i have this: > > sethandler "module_test_output" > Using it in a would suggest it's per_dir config. > > I setup the shm in post_config hook: > ap_hook_post_config(post_config_hook, NULL, NULL, APR_HOOK_REALLY_FIRST); ... but per_dir config for your location doesn't exist at post_config time! > Within the translate_name hook the module access conf->cache correctly and > shm+rmm works well. because the config it sees hasn't been merged for the I suspect you're using the wrong configuration hierarchy for the job at hand. If not, then the fix might be as simple as a merge_config function. There's a very brief (and ancient) piece at http://www.apachetutor.org/dev/config In the book there's a whole chapter, which is both substantially expanded and updated from that. -- Nick Kew
Re: mod_proxy_fdpass + httpd-2.2.19
On 3 Aug 2011, at 10:26, Henrik Strand wrote: > Hmm, but this is how the code looks like when I get the error that I > posted. In that case, you've compiled it against the wrong headers. The best solution is usually to use apxs from the installed version to compile new modules. That error message is your module failing a test that it was compiled against the correct API version and is therefore compatible with the server that's loading it. -- Nick Kew
Re: mod_proxy_fdpass + httpd-2.2.19
On 3 Aug 2011, at 09:27, Henrik Strand wrote:
> module AP_MODULE_DECLARE_DATA proxy_fdpass = {
>STANDARD20_MODULE_STUFF,
>NULL, /* create per-directory config structure
> */
>NULL, /* merge per-directory config structures
> */
>NULL, /* create per-server config structure */
>NULL, /* merge per-server config structures */
>NULL, /* command apr_table_t */
>register_hooks /* register hooks */
> };
That should be fine, compiled against the right (2.2.x) headers.
--
Nick Kew
Re: mod_proxy_fdpass + httpd-2.2.19
On 3 Aug 2011, at 08:10, Henrik Strand wrote: > Hi, > > I'm trying to back-port Apache Module mod_proxy_fdpass to httpd-2.2.19, > and building from source, but are getting the following error message > when starting apachectl: > > httpd: Module "(null)" is not compatible with this version of Apache > (found 0, need 20051115). Please contact the vendor for the correct > version. Guessing in the dark You declare the module with module AP_MODULE_DECLARE_DATA proxy_fdpass_module; But you don't instantiate it. The macro commonly used in trunk doesn't exist in 2.2. -- Nick Kew
Re: Sharing information between threads and processes.
On 21 Jul 2011, at 11:34, Zaid Amireh wrote: > expanding on it to accommodate variable length data like queues and hashes > seems pretty complicated unless I missed something obvious. Indirection in shared memory is inherently complex! How near do the socache modules come to meeting your needs? -- Nick Kew Available for work, contract or permanent http://www.webthing.com/~nick/cv.html
Re: ap_hook_create_request vs ap_hook_insert_filter
On 4 Jul 2011, at 01:24, Jodi Bosa wrote: > Hi, > > In Apache 2.2 - what is the purpose of ap_hook_insert_filter? It seems to > get invoked after things like post_read, quick_handler, and others which is > too late to "filter" any input. > > On the other hand, ap_hook_create_request does get invoked early enough. The insert_filter hook is indeed too late to insert 'outer' input filters that will affect the protocol. But it's plenty early enough for other filters. The request_rec represents the HTTP(s) protocols. If you're using HTTPD's own protocol support then your filters won't mess with it, so they don't need to be inserted early. If you do want to mess with the protocol at a low level then yes, you'll want a create_request hook. The reason for the hook coming late in the cycle is that filter configuration may depend on the outcome of earlier phases (and in general WILL depend on at least the configuration walk). -- Nick Kew Available for work, contract or permanent http://www.webthing.com/~nick/cv.html
Re: input filters called again after Handler returns
On 30 Jun 2011, at 08:11, Sorin Manolache wrote: > On Thu, Jun 30, 2011 at 02:56, Jodi Bosa wrote: >> I'm encountering a strange interaction between modules (including my own). >> When I track it down, it appears that input filters are called after the >> handler is finished which results in 2 bodies in the response. >> >> In other words, this is what appears to be happening: >>Input filters called with AP_MODE_GETLINE (for each HTTP request >> header) >>Handler called and returns 302 >>input filters called with AP_MODE_READBYTES and 8192 >> >> >> I've placed logging in ap_internal_*_redirect() and removed all related >> modules to see if one was a problem - yet the same issue happens. >> >> Thanks for any help. >> > > > At the end of the request processing chain, apache places a function > that clears the request body. > (ap_finalize_request_protocol->ap_discard_request_body). > > Your inut filter should place an EOS bucket in the brigade or return > something else than APR_SUCCESS in this second invocation. Have you encountered the actual problem described? I haven't. I can see that failing to deal with EOS might cause such an issue, but I don't see that a filter should want or need self-awareness of a 'second invocation': rather that would be an issue for whatever is issuing the internal redirect. I'm guessing this problematic filter is one that doesn't pass its (modified) input through to its output? In which case, it should take responsibility for reading its input through to EOS and for returning EOS to its caller. If you fix that and still have the problem, please describe in detail! -- Nick Kew Available for work, contract or permanent http://www.webthing.com/~nick/cv.html
Re: per-worker-thread counter question
On 28 Jun 2011, at 03:57, Neil McKee wrote: > Hi, > > Here's an easy question for someone who knows their way around... Yeah, right :P > I want to maintain a new global counter, but for performance reasons I am > reluctant to use a mutex or atomic_increment to update it. I would rather > maintain a separate counter for every worker-thread, and only accumulate the > global counter when required. (If the per-worker-thread counter is 32-bit > then I shouldn't even need a mutex when accumulating the total across all the > current threads). Have you tried apr_atomic and found an unacceptable performance impact, or are you just assuming it? I haven't tried it, but my gut feeling wouldn't expect the overhead to be out of proportion. > Obviously I shouldn't just declare something as "__thread apr_int32_t > mycounter;" and mince it together as a linux-only hack. I'd like to find the > portable apr-library way to do it. So I think I need to find the following: > > * - a hook that is called whenever a worker thread is started > * - a hook that is called whenever a worker thread is about to die > * - a hook to find_or_create a 32-bit integer that is private to the current > worker-thread > * - a fn to iterate (safely) over all the current worker threads Absent something in the API, you're stuck. Just to make it work across standard MPMs will require care, and is unlikely to be future-proof. > It's the last one that seems particularly elusive. I could't find an ap_ or > apr_ library call that seemed to do anything like that. So your module would need to maintain its own register to iterate over. Hats off if you can hack that with less overhead than apr_atomic or ... > If this has all been done before, please can you point me to the relevant > module sources? I think it would save me a lot of time. Alternatively, if > you think I should just relax and use an atomic increment instead, then let > me know. What springs to mind (as being supported in the API) is the scoreboard. mod_slotmem (new in trunk) may be worth a look. -- Nick Kew Available for work, contract or permanent http://www.webthing.com/~nick/cv.html
Re: Kill a request nicely
On Tue, 14 Jun 2011 16:31:22 -0500 Jason Funk wrote: > I am writing an output filter module that will under some circumstances want > to send back an HTTP Error Code and kill the request without sending back > the content. You can't set an HTTP response code when one has already been sent down the wire to the client. It's in the nature of an output filter that the work is done in a callback, which (in general) only happens after the response has started, and too late to set a response code or headers. Thus the filter callback API isn't designed to set a status like a processing hook can when it determines the outcome of a request. The only way you can affect a response status is to return an error to your caller before passing anything down the chain. -- Nick Kew Available for work, contract or permanent. http://www.webthing.com/~nick/cv.html
Re: how to parse html content in handler
On Thu, 24 Mar 2011 22:58:07 +0800 (CST) "Whut Jia" wrote: > Hi, > Thank you! > But I want to parse a jsp page in my handler.How can I do it?? > Please help me! In my handler, I do a request (http://www.xxx/xxx.jsp)with > libcurl,and then parse returned response ,and draw some infomation.Please ask > how to parse this jsp response??? Last time I looked, JSP 2 insisted on XML well-formedness, and would work well under an XML parser. You could dispense with parsing altogether and just implement event handlers under an existing parser such as mod_xmlns or mod_xml2. JSP 1 was an SSI-like language. It would be a little more work, but mod_includes would be a good startingpoint. -- Nick Kew Available for work, contract or permanent. http://www.webthing.com/~nick/cv.html
Re: ordering output filters
On 14 Mar 2011, at 15:54, Joshua Marantz wrote:
> if (mod_includes was enabled in this config) {
>re-insert mod_pagespeed at the end of the AP_FTYPE_RESOURCE chain
>pass the buckets to mod_includes
> }
Not good. Modules are there to serve the server admin, not to enslave him.
In general they shouldn't touch each other (except through public APIs)
nor second-guess a server admin.
In practical terms, what about a third-party module that parses comments?
If includes get special treatment but others don't, you're making things
horribly
confusing for your users.
A traditional but only slightly less ugly hack would be to declare your filter
AP_FTYPE_RESOURCE+1. That also leaves an admin the possibility of
overriding it.
> Or can we, at init time, call server APIs to tweak the filter order? Is
> there any filter that seeks to do that somehow?
You could take a look at how mod_proxy_html inserts mod_xml2enc
if available.
> A third idea is to exploit the fact that INCLUDES adds itself to the output
> chain via
> ap_hook_fixups(include_fixup, NULL, NULL, APR_HOOK_LAST);
> where include_fixup() does ap_add_output_filter("INCLUDES", NULL, r,
> r->connection);
Why not an insert_filter hook?
That would be the right place to go, but then be sure to document exactly
how it works and what other modules will be auto-configured.
--
Nick Kew
Available for work, contract or permanent
http://www.webthing.com/~nick/cv.html
Re: apxs "DEFS"
On 13 Mar 2011, at 23:44, MK wrote: > I hope it is okay to be asking questions about apxs here. You're welcome to post here, but if you suspect a bug or would like to suggest an enhancement then dev@httpd would be the right place. > However, it can't be done with apxs :( It hadn't crossed my mind to try that with apxs (adjust by hand where necessary). But in principle, Good Idea! Patches welcome! Sorry, no new suggestions beyond wrowe's reply or "do it by hand". -- Nick Kew Available for work, contract or permanent http://www.webthing.com/~nick/cv.html
Re: Converting a 16-bit string to 8-bit?
On 3 Mar 2011, at 16:52, Ben Noordhuis wrote: > On Thu, Mar 3, 2011 at 17:12, Sam Carleton wrote: >> I am looking at using a 3rd party library that only operates on 16-bit >> strings. Is there a built in functions to convert the strings back to >> 8-bit? I am currenly on Windows and Windows has built in functions I could >> use, I would prefer to use Apache functions if they exist. > > Sam, you want either apr_xlate.h or apr-iconv / native iconv. apr_xlate would be the appropriate abstraction for cross-platform support, if it meets your needs. If not, you'd need an external library. You might not need to do anything at all: (1) You could send data out in 16-bit encoding if you send it with an appropriate charset parameter in the Content-type header. (2) If (1) is not an option, an existing module like mod_charset might be suitable to do the conversion for you. -- Nick Kew Available for work, contract or permanent http://www.webthing.com/~nick/cv.html
Re: Filter to modify request headers
On 25 Jan 2011, at 22:22, Jodi Bosa wrote: > Hi, > > What would be a good early hook to modify request headers that is _AFTER_ > mod_ssl is finished decrypting request? > > > When I do a ap_add_input_filter() from a ap_hook_insert_filter() seems to > trigger really late (e.g. after quick_handler, post_read, etc...). Don't. Use a header_parser hook to manipulate your request headers. Or if you need to catch them early, a post_read_request. -- Nick Kew Available for work, contract or permanent http://www.webthing.com/~nick/cv.html
Re: How to add referer header in external redirect?
On 16 Jan 2011, at 12:59, Eric Covener wrote: >> I need some help with a redirecting/referrer issue. If I do a 303 redirect >> in my module (by this way below: >> apr_table_setn(r->headers_out,"Location","http://idp/login.html";); > > err_headers_out to be preserved when an errordoc is sent for non-2xx > response. > Won't make any difference to someone trying to set a request header in a response. -- Nick Kew Available for work, contract or permanent http://www.webthing.com/~nick/cv.html
Re: module configuration kill
On Wed, 12 Jan 2011 11:24:36 -0800 (PST) Peter Janovsky wrote: > is it possible to hook into an event fired when the workers are shutdown, Any cleanup registered on the child pool will run when the child quits, if that's any use to you. -- Nick Kew Available for work, contract or permanent. http://www.webthing.com/~nick/cv.html
Re: Hook end of connection
On Wed, 12 Jan 2011 14:49:53 -0500 Victor Ronin wrote: > Hi, > > I need to write a module, which does something at the beginning and at > the end of each connection. Register a cleanup on the connection pool. -- Nick Kew Available for work, contract or permanent. http://www.webthing.com/~nick/cv.html
Re: module configuration kill
On Mon, 10 Jan 2011 14:16:43 -0800 (PST) Peter Janovsky wrote: > how do i go about preserving variable state specific to the individual worker? On a restart, the old pools are cleaned up before reinitialisation starts. So in your cleanups, you can reset any variables that need reinitialising from a virgin state. -- Nick Kew Available for work, contract or permanent. http://www.webthing.com/~nick/cv.html
Re: Help trying to figure out why an output_filter is not called.
On Wed, 5 Jan 2011 08:45:57 -0500 Joshua Marantz wrote: > What might be going wrong in his server to cause this to fail? Could some > other filter be somehow finding our filter and killing it? Or sending the > bytes directly to the network before our filter has a chance to run? Yes. Any of the above. Someone may be breaking modularity. PHP has a track record of that, and from your recent posts here I'd guess it's not the only suspect. Can you reproduce the problem? If so, run it under a debugger or trace and look for when your filter is inserted and whether it's removed. Does mod_diagnostics tell you anything if you run it at the same level as yours? -- Nick Kew
Re: Shared memory ?
On Mon, 15 Nov 2010 17:12:01 +0100 Rémy Sanchez wrote: > [chop] Since you're at an experimental stage, there's no reason not to work with trunk/2.3 versions. So use the mod_socache framework, which is provided for precisely this kind of thing! -- Nick Kew
Re: How to init an mmaped file?
On 26 Oct 2010, at 03:52, Mike Meyer wrote: > Anyone got some hints on how I ought to set this up? Maybe there's a > writeup somewhere I missed on config processing Possibly there's some > facility that I ought to be looking at to replace the mmaped file? One possible suggestion. post_config and child_init get passed the first server_rec. The server_rec in any later hook is the virtual host. Could your problem be setting data in one server then trying to retrieve it from another? -- Nick Kew
Re: Strange EOS bucket in filter
On 23 Oct 2010, at 11:06, r...@tuxteam.de wrote: > > Hello list, > > I'm currently developing an output filter that , dpending on some > condition either parses all data to convert it (and hence don't pass any > of the incomming buckets down the pipe) or decides to leave the data as > it is and just pushes all incomming buckets down. Now, strangely, for > the second case every second request my filter only gets an EOS bucket > (as the first bucket te filter sees) for all resources that the filter > doesn't need to touch. This only seems to happen for requests with > keep-alive true. That somehow sounds like someting I've encoutered > before but I can't recall when or where :-/ Any ideas what's going on? Are you clearing all the input once you've consumed it? -- Nick Kew
Re: Memory Pool
On Mon, 11 Oct 2010 15:14:02 +0100 Martin Townsend wrote: > Hi, > > I have created a pool from the child pool for storing warning messages > that can live across requests, the final request will insert the > warnings into the response. How do I ensure that this pool is cleared > at the end of the final request? That doesn't really make sense. What is "the final request"? If it's requests in a connection, use the connection pool. Otherwise, you're looking at a time-based solution such as garbage collection. -- Nick Kew
Re: httpd filters to record the client network time
On Sat, 2 Oct 2010 08:11:33 -0700 (PDT) alin vasile wrote: > I would like to write an httpd module with output filters to record when a > client sends the first byte of an HTTP request and when send the last. Your problem is that you're trying to use filters at a level in the chain where there is no request, only a connection. If you want to measure requests, you'll have to work inside the HTTP protocol filter. (That is, inside as in between the protocol filter and the handler). Otherwise for a big overhead you could record it as a datestamped bytestream, and then reconstruct HTTP requests from the bytestream. That could be done outside apache. -- Nick Kew
Re: Filters before mod_include
On Sat, 2 Oct 2010 07:44:47 -0700 (PDT) Travis Bassetti wrote: > Is it possible to register an output filter that processes the request before > mod_include? I'm trying to build a filter which inserts extra markup around > each "#include virtual" SSI directive. I can only seem to register an > output > filter after mod_include as mod_include declares APR_HOOK_REALLY_FIRST in its > configuration. The order of output filters is ordinarily a matter of configuration, subject to the broad category (AP_FTYPE_***) types. See the user docs. The APR_HOOK_* stuff has nothing to do with the filter. If you want to program ordering into your module, you can have it insert its own and other filters in your chosen order. See mod_proxy_html for an example: it may insert mod_xml2enc before and/or after itself according to the configuration options. But note that it still relies on configuration: if you really want to force it, you would also have to undo any other filter configuration. -- Nick Kew
Bounces from this list
Who manages this list these days? Can someone remove this bouncing address? This is the third bounce I've had, and I expect everyone posting here is getting them. -- Nick Kew Begin forwarded message: Date: Fri, 24 Sep 2010 16:05:17 GMT From: postmas...@sprint.blackberry.net To: n...@apache.org Subject: Delivery Status Notification(Failure) Your message: To: caroline1mu...@sprint.blackberry.net Subject: Re: modules architecture issue Sent Date: 52:52 + has not been delivered to the recipient's BlackBerry Handheld. Final-Recipient: RFC822;caroline1munoz@sprint.blackberry.net Action: Failed Status: 5.0.0
Re: modules architecture issue
On Wed, 15 Sep 2010 14:57:25 -0300 Fabio Kaminski wrote: I'm not certain I understand the question, but here goes ... > and at least for data, i thought use something like [input filter -> > handler -> output filter] > > where the input filter receives a formated string and parse into a internal > object struct.. the handler get it > and using a rest mechanism (put,delete,get,post) in the handler.. An input filter doesn't strictly speaking asynchronously with your handler, but conceptually you should think of it as asynchronous. If that fits your application well, go right ahead. Otherwise it's probably going to be easier to do all that within your handler. The exception to that is if you have functionality that becomes re-usable across multiple applications if you put it in a filter. > the handler then call a db api, for the data operation.. and if is something > like get.. the output filter decode our internal object struct into > the formated string.. That sounds like it should be just fine running asynchronously. But using a filter may still call for more groundwork. Ideally you should create a new bucket type for your internal objects to pass them down the chain. > i saw the smart filter, but couldnt fire it.. since i didnt see a example > module using it.. Smart filtering is a matter for configuration. If your application requires the filters, it should probably manage its own configuration. If the filters are optional then by all means leave it to the sysop and recommend using mod_filter in your documentation. -- Nick Kew
Re: Apache mods - possible to send request on?
On 14 Sep 2010, at 18:33, Paul Donaldson wrote: > Hello, > > I am completely new to Apache and Apache Modules, and so I'd be grateful if > someone could tell me whether it might be possible to write a module that > takes > elements of the request and makes a request to another web server at an > arbitrary URL? Yes of course you can! You can run a subrequest that'll use mod_proxy. Or you can use your choice of HTTP client code. -- Nick Kew
Re: Peek at request from within Connection input filter
On 14 Sep 2010, at 04:08, Jodi Bosa wrote: > Is there a way that I can "peek" at the request from within a Connection > filter? In other words, I need to examine the actual HTTP request in order > to affect something in another Connection filter. A constraint is that I > cannot modify this other Connection filter. No. There isn't a request object in a connection filter. A connection may be handling multiple requests which can't be deterministically mapped. Or there may be no Request at all if the protocol is not HTTP. If you can ensure there is a meaningful Connection-Request correspondence in your app, use the connection's configuration record to pass information to/from the request(s). -- Nick Kew
Re: responsehandler in C
On 18 Aug 2010, at 16:05, Peter Janovsky wrote: > my question - is it possible to write a responsehandler within C? i'm > assuming > it is but i am only finding responsehandler examples in Perl. Of course! If you're using Perl then you're also using the response handler from a C module (such as mod_cgi or mod_perl) to run your script. Chapter 5 of my book takes you through writing a response handler, from Hello World through to all you need to switch a CGI script to a module. http://www.apachetutor.org/ -- Nick Kew
Re: Modify the body of a post request Multipar/form-data?
On 10 Jun 2010, at 07:34, Eddy wrote: > How modify (decrypt data) the body content before all module ? That's what filters do. See mod_ssl for secure encryption. Or in your case since it's only the body content, see for example mod_deflate for a comparable task. -- Nick Kew
Re: Can an Apache module inject configuration in runtime?
On 1 Jun 2010, at 10:39, Andrew Godziuk wrote: > I'm wondering if it's possible for an Apache module to change global > config structures. Basically, no, not without a restart. Though there are some aspects of configuration you can change on the fly. > What I want to achieve is injecting new vhosts without Apache restart. > Of course I'm aware that the changes would fully take effect after all > workers have recycled, but for me - it's still better than a restart. There are some similar modules around. Someone already said mod_vhost_alias, but I'd also point you at mod_vhost_dbd as an example that may be nearer to what you want if your needs are sufficiently complex to demand a new module. -- Nick Kew
Re: simple mapping module example?
On 29 Apr 2010, at 04:56, Mark Harrison wrote: > I would like to do a simple mapper: > > - given some inputs, calculate a file path > - let apache serve out that file for me > > Is there a simple example of how to do this? mod_rewrite > is pretty heavy-duty, and I'd like to get a simpler > model to follow. Maybe mod_rewrite will do all you need anyway! The simplest example to look at in the apache code would be mod_alias. Otherwise, the end of Chapter 6 of the modules book[1] leads you through developing a mapper module. [1] http://www.apachetutor.org/ -- Nick Kew
Re: Issuing a client side HTTP request from a module
On 22 Apr 2010, at 15:14, Some Guy wrote: > I don't really need keepalive. Just a really basic http request. However, > this would be in a non request handler thread, so I'll have no initial > request_rec to create a subrequest from. It would also be in the parent > process via a monitor hook. In that case, you're probably the exception to the general advice to use mod_proxy. Simplest would probably be DIY with your choice of HTTP client library. -- Nick Kew
