Re: After-request hooks or asychronous modules
Hi Samuel, I am very curious about your solution. Have you satisfied with what you are doing. I am also facing a problem to asynchronous do some job related to the served page, but the served page do not rely on what I am doing. Would you please share your experience? Thanks, Libo On Sat, Feb 27, 2010 at 7:50 PM, Samuel ROZE wrote: > Hello, > > I'm writing an Apache2 module which takes some time to be executed > (500ms avg) because it is communicating with another server for each > request. What is it doing haven't any consequence on the served page > or file, so it can be run after that the connection with the client > was closed. > > Now, it is working as a basic module, with libcurl and each request > take 500ms because it is executed after sending the result page, even > if it's not needed... Is it possible to execute a module after that > the request was done? Or is it possible to create a function which > will be called at this time using a hook? > > Thanks in advance. > Regards, > Samuel. >
Re: After-request hooks or asychronous modules
On Sun, 28 Feb 2010 00:50:52 +0100 Samuel ROZE wrote: > Hello, > > I'm writing an Apache2 module which takes some time to be executed > (500ms avg) because it is communicating with another server for each > request. What is it doing haven't any consequence on the served page > or file, so it can be run after that the connection with the client > was closed. > > Now, it is working as a basic module, with libcurl and each request > take 500ms because it is executed after sending the result page, even > if it's not needed... Is it possible to execute a module after that > the request was done? Or is it possible to create a function which > will be called at this time using a hook? [this should really be on modules-dev] All you *should* need to do is send an EOS bucket at the end of your response before running backend stuff. Oh, and be sure to have consumed all input: possibly call ap_discard_request_body yourself to munch any unexpected input if your module doesn't already ensure it's fully consumed. Have you tried that? If that fails (maybe some filter eats it), you could run the backend stuff asyncronously - e.g. give it a dedicated thread and communicate via an APR queue. That's complicated by resource management: your module will need pools that outlive a request but are cleaned up after the backend stuff. -- Nick Kew
Re: After-request hooks or asychronous modules
The trick with any such thing is in protecting yourself from attack. Consider...you could fork, let the parent continue on its business, disown the child, and do your work there. Easy in terms of lines of code, but expensive (forks) and potentially hazardous. You could create a thread pool to do the job, and only block the response when you had to wait for a worker. More complex to code, but you control how much of your resources you're willing to expend. Samuel ROZE wrote: A dedicated process or thread just for my module, or still linked to the Apache request? The better thing is to create a thread unlinked with the Apache request, it is possible? I'm like a Newbie with Apache module development, is there somewhere an example or a sample of code? Like I said, the better thing is that I create a thread (more efficient than process I thing) which could run 10s if it want without interfering with the initial request and with others. I thing it isn't easy, that's why I need a few help :-) Thanks a lot for your reply, Samuel. 2010/2/28 Eric Covener : > On Sun, Feb 28, 2010 at 5:27 AM, Samuel ROZE wrote: >> Yes, the log_transaction hook is what I expect, thanks! :-) >> >> But I've still a problem: if in my hook function I put a sleep function, >> which sleep for 10 seconds for example, the first page is served >> correctly (few ms) but for the next, it will wait for these 10s (not >> really 10s but the rest time), is it because I should have threads? > > You could start a dedicated thread or process in e.g. post_config and > just post work to it asynchronously in log_transaction. > > > -- > Eric Covener > cove...@gmail.com >
Re: After-request hooks or asychronous modules
On Sun, 28 Feb 2010 11:27 +0100, "Samuel ROZE" wrote: > Yes, the log_transaction hook is what I expect, thanks! :-) > > But I've still a problem: if in my hook function I put a sleep > function, which sleep for 10 seconds for example, the first page is > served correctly (few ms) but for the next, it will wait for these 10s > (not really 10s but the rest time), is it because I should have > threads? That's probably due to keep-alive. With keep-alive on, the same connection is used for subsequent requests from the same client, which means the same thread has to receive and process the request. But I'm not recommending turning off keep-alive as a fix. Creating worker threads is one solution. Just keep in mind that many APR functions, like pool management, are not thread-safe, so you'll need to carefully protect their use with mutexes or the equivalent. Dan
Re: After-request hooks or asychronous modules
A dedicated process or thread just for my module, or still linked to the Apache request? The better thing is to create a thread unlinked with the Apache request, it is possible? I'm like a Newbie with Apache module development, is there somewhere an example or a sample of code? Like I said, the better thing is that I create a thread (more efficient than process I thing) which could run 10s if it want without interfering with the initial request and with others. I thing it isn't easy, that's why I need a few help :-) Thanks a lot for your reply, Samuel. 2010/2/28 Eric Covener : > On Sun, Feb 28, 2010 at 5:27 AM, Samuel ROZE wrote: >> Yes, the log_transaction hook is what I expect, thanks! :-) >> >> But I've still a problem: if in my hook function I put a sleep function, >> which sleep for 10 seconds for example, the first page is served >> correctly (few ms) but for the next, it will wait for these 10s (not >> really 10s but the rest time), is it because I should have threads? > > You could start a dedicated thread or process in e.g. post_config and > just post work to it asynchronously in log_transaction. > > > -- > Eric Covener > cove...@gmail.com >
Re: After-request hooks or asychronous modules
On Sun, Feb 28, 2010 at 5:27 AM, Samuel ROZE wrote: > Yes, the log_transaction hook is what I expect, thanks! :-) > > But I've still a problem: if in my hook function I put a sleep function, > which sleep for 10 seconds for example, the first page is served > correctly (few ms) but for the next, it will wait for these 10s (not > really 10s but the rest time), is it because I should have threads? You could start a dedicated thread or process in e.g. post_config and just post work to it asynchronously in log_transaction. -- Eric Covener cove...@gmail.com
Re: After-request hooks or asychronous modules
Yes, the log_transaction hook is what I expect, thanks! :-) But I've still a problem: if in my hook function I put a sleep function, which sleep for 10 seconds for example, the first page is served correctly (few ms) but for the next, it will wait for these 10s (not really 10s but the rest time), is it because I should have threads? Regards, Samuel. Sent from Rennes, France. Le samedi 27 février 2010 à 22:14 -0500, Eric Covener a écrit : > On Sat, Feb 27, 2010 at 6:50 PM, Samuel ROZE wrote: > > Hello, > > > > I'm writing an Apache2 module which takes some time to be executed > > (500ms avg) because it is communicating with another server for each > > request. What is it doing haven't any consequence on the served page > > or file, so it can be run after that the connection with the client > > was closed. > > > > Now, it is working as a basic module, with libcurl and each request > > take 500ms because it is executed after sending the result page, even > > if it's not needed... Is it possible to execute a module after that > > the request was done? Or is it possible to create a function which > > will be called at this time using a hook? > > How about the log_transaction hook? >
Re: After-request hooks or asychronous modules
On Sat, Feb 27, 2010 at 6:50 PM, Samuel ROZE wrote: > Hello, > > I'm writing an Apache2 module which takes some time to be executed > (500ms avg) because it is communicating with another server for each > request. What is it doing haven't any consequence on the served page > or file, so it can be run after that the connection with the client > was closed. > > Now, it is working as a basic module, with libcurl and each request > take 500ms because it is executed after sending the result page, even > if it's not needed... Is it possible to execute a module after that > the request was done? Or is it possible to create a function which > will be called at this time using a hook? How about the log_transaction hook? -- Eric Covener cove...@gmail.com
After-request hooks or asychronous modules
Hello, I'm writing an Apache2 module which takes some time to be executed (500ms avg) because it is communicating with another server for each request. What is it doing haven't any consequence on the served page or file, so it can be run after that the connection with the client was closed. Now, it is working as a basic module, with libcurl and each request take 500ms because it is executed after sending the result page, even if it's not needed... Is it possible to execute a module after that the request was done? Or is it possible to create a function which will be called at this time using a hook? Thanks in advance. Regards, Samuel.
