Re: Sending data to client
Hracek, Petr wrote: I am migrating apache module from version 1.3.41 to version 2.2.3 All seems to be OK but only one thing remains as not working and I have no idea how to solve it in Apache 2.2. In the version 1.3.41 I used to following code for sending data to client: ap_soft_timeout ("TEXT MESSAGE",r); ap_rwrite(MESSAGE_TO_CLIENT, strlen(MESSAGE_TO_CLIENT),r); ap_kill_timeout(r); How can send data to client on Apache 2.2? Apache v2.0 (and v2.2) introduced the concept of an input and an output filter stack to handle the problem of reading from and writing to the network. The input filter stack is a chain of code that reads data in from the network. Filters in the stack along the way might uncompress the data, decrypt the data, or do any of a number of things to the data before handing the data to you. The output filter stack is a chain of code that writes data to the network, and is the one you want to care about. The output filter stack might encrypt, compress, chunk or otherwise process the data you send, and you don't need to know or care how this is done. The filter stacks solved a long standing problem in Apache 1.3, which if you wanted to process data that was entering or leaving the server, you couldn't really without hacking away at the core. So, to come round and actually answer your question, in Apache v2.2, you want to write your data to the output filter stack. The best examples are in the webserver itself. The least confusing module to use as an example would probably be the status_handler function inside mod_status, which uses ap_rputs() and ap_rvputs() to write strings to the network, which sounds like what you are doing. If you want to have more control over the data you are sending, you can call ap_pass_brigade() to "write" some data, in the form of a bucket bridge, to the network in your response. This second option allows you to send any number of things to the network, from static areas of memory, to files on disk, in a very efficient way. For more information about bucket brigades, read the docs for the bucket brigade API in APR: http://apr.apache.org/docs/apr-util/trunk/group___a_p_r___util___bucket___brigades.html Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
RE: Sending data to client
Thanks for the question about replacement of ap_*_timeout. When the ap_soft_timeout and ap_rwrite are called in Apache 1.3 I've never inserted HTML tags like etc. They were inserted automatically? Is it necessary to include HTML tags to buffer which is sent to the Client side? And do you know how can I monitored if the date were successfully sent / delivered to the client side? All comunication is done over https. Thank you in advance regards / S pozdravem Petr Hráček -Original Message- From: Graham Leggett [mailto:minf...@sharp.fm] Sent: Friday, January 30, 2009 12:29 PM To: dev@httpd.apache.org Subject: Re: Sending data to client Hracek, Petr wrote: > I am migrating apache module from version 1.3.41 to version 2.2.3 > All seems to be OK but only one thing remains as not working and I have no > idea how to solve > it in Apache 2.2. > > In the version 1.3.41 I used to following code for sending data to client: > ap_soft_timeout ("TEXT MESSAGE",r); > ap_rwrite(MESSAGE_TO_CLIENT, strlen(MESSAGE_TO_CLIENT),r); > ap_kill_timeout(r); > > How can send data to client on Apache 2.2? Apache v2.0 (and v2.2) introduced the concept of an input and an output filter stack to handle the problem of reading from and writing to the network. The input filter stack is a chain of code that reads data in from the network. Filters in the stack along the way might uncompress the data, decrypt the data, or do any of a number of things to the data before handing the data to you. The output filter stack is a chain of code that writes data to the network, and is the one you want to care about. The output filter stack might encrypt, compress, chunk or otherwise process the data you send, and you don't need to know or care how this is done. The filter stacks solved a long standing problem in Apache 1.3, which if you wanted to process data that was entering or leaving the server, you couldn't really without hacking away at the core. So, to come round and actually answer your question, in Apache v2.2, you want to write your data to the output filter stack. The best examples are in the webserver itself. The least confusing module to use as an example would probably be the status_handler function inside mod_status, which uses ap_rputs() and ap_rvputs() to write strings to the network, which sounds like what you are doing. If you want to have more control over the data you are sending, you can call ap_pass_brigade() to "write" some data, in the form of a bucket bridge, to the network in your response. This second option allows you to send any number of things to the network, from static areas of memory, to files on disk, in a very efficient way. For more information about bucket brigades, read the docs for the bucket brigade API in APR: http://apr.apache.org/docs/apr-util/trunk/group___a_p_r___util___bucket___brigades.html Regards, Graham --
Re: Sending data to client
Hracek, Petr wrote: Thanks for the question about replacement of ap_*_timeout. When the ap_soft_timeout and ap_rwrite are called in Apache 1.3 I've never inserted HTML tags like etc. They were inserted automatically? Is it necessary to include HTML tags to buffer which is sent to the Client side? They are treated purely as strings. The decision on what to output is made by your code. In the case of mod_status, mod_status chooses to output data as HTML, which is why it goes about sticking in tags. And do you know how can I monitored if the date were successfully sent / delivered to the client side? All comunication is done over https. It depends on the level of guarantee you need. If you are just worried that the data you just wrote must be sent to the client right now, I suspect the bucket brigade API might help you with flush buckets. If you send buckets instead of strings, you have more control over what you want to send. You might put a string in a bucket, and then another string in a bucket, followed by a flush bucket, followed by another string in a bucket. The flush bucket tells the networking code "send what you have right now, don't wait for any more". Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
RE: Sending data to client
Because of I am a begginer with bucket and brigades Is there any short example how to send simple string to the client? I've found some examples but it were a pretty complicated for me. E.g. http://www.apachetutor.org/dev/brigades regards / S pozdravem Petr Hráček -Original Message- From: Graham Leggett [mailto:minf...@sharp.fm] Sent: Friday, January 30, 2009 12:59 PM To: dev@httpd.apache.org Subject: Re: Sending data to client Hracek, Petr wrote: > Thanks for the question about replacement of ap_*_timeout. > > When the ap_soft_timeout and ap_rwrite are called in Apache 1.3 > I've never inserted HTML tags like etc. > They were inserted automatically? > > Is it necessary to include HTML tags to buffer which is sent to the Client > side? They are treated purely as strings. The decision on what to output is made by your code. In the case of mod_status, mod_status chooses to output data as HTML, which is why it goes about sticking in tags. > And do you know how can I monitored if the date were successfully sent / > delivered to > the client side? All comunication is done over https. It depends on the level of guarantee you need. If you are just worried that the data you just wrote must be sent to the client right now, I suspect the bucket brigade API might help you with flush buckets. If you send buckets instead of strings, you have more control over what you want to send. You might put a string in a bucket, and then another string in a bucket, followed by a flush bucket, followed by another string in a bucket. The flush bucket tells the networking code "send what you have right now, don't wait for any more". Regards, Graham --
Re: Sending data to client
Hracek, Petr wrote: Because of I am a begginer with bucket and brigades Is there any short example how to send simple string to the client? I've found some examples but it were a pretty complicated for me. E.g. http://www.apachetutor.org/dev/brigades Go through the underlying bucket brigade API here: http://apr.apache.org/docs/apr-util/trunk/group___a_p_r___util___bucket___brigades.html Essentially from your perspective, you want to create a brigade with apr_brigade_create(), and then create buckets, and put those buckets into the brigade. When you want to flush the data, create a flush bucket, add it to the brigade, and give the brigade to the network using ap_pass_brigade(). You can call ap_pass_brigade as many times as you like (calling it excessively is inefficient though). To create a flush bucket, use apr_bucket_flush_create. There are convenience functions that allow you to add strings to a bucket brigade without having to create buckets from scratch: apr_brigade_write apr_brigade_writev apr_brigade_printf ...etc A note about what a bucket brigade is: a bucket_brigade is basically a linked list of pieces of data. "Pieces of data" can be quite broad a definition. A piece of data might be a simple string, stored in a heap bucket. A "piece of data" might be a file on disk, stored in a file bucket. The "piece of data" can also not be data at all, but rather a signal to do something special. A flush bucket is one example, and the EOS (end-of-stream) bucket is another that says "this is the last bucket in this response, you're done now". The bucket brigade API is a bit mind bending at first, but it is a very valuable tool to use. If you want to write an Apache v2 filter, you'll need to know how the bucket brigades work, so they are definitely worth getting to understand. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
RE: Load memory dyanamically using handler.
Hai all, Iam loading huge hash map in the shared memory, and the hash map can be reloaded again by hitting the handler without server restart. I use the apr_shm to create the shared memory and the apr_rmm to manage the memory.. I create around 1 GB of shared memory which i manage. Because of this implementation i find the apache server to be very slow.. I use the worker mpm.. Please guide me how to reduce the apache startup time. I guess the shared memory reduces the apache performance when we create huge shared memory. But when i read some books which shows the shared is the fastest IPC approach. Is that correct? Is there any method to faster the processing speed with the Shared memory?. Please guide me.. Thanks in Advance, Jaysingh Samuel. From: jayasingh.sam...@hotmail.com To: modules-...@httpd.apache.org Subject: RE: Load memory dyanamically using handler. Date: Fri, 12 Dec 2008 18:52:39 +0530 Hai all, Thanks a lot for your inputs and the mod_shmget source code. I have tried the apr_shm and apr_rmm api's for shared memory and dynamic allocating space for member elements and was working fine.. I did something like this. My struture is. typedef struct { char *name; }shared_struct_t; Where the shared_struct_t is a array of 100. and the name member element have to dyanamically malloced on the code flow. I found it difficult first only by getting a junk of memory from arr_shmcreate which i was not able to malloc further for space for the member element. The rmm (relocated memory management is the apache api which helps to manage any junk of memory). So i first create a Junk of memory which is shared and then i initialize with the rmm and further i was able to use this Junk of memory to allocate memory dyanamically to different elements i want.. http://apr.apache.org/docs/apr-util/0.9/group__APR__Util__RMM.html Thanks, Jaysingh. > Date: Thu, 11 Dec 2008 22:16:42 +0100 > From: sor...@gmail.com > To: modules-...@httpd.apache.org > Subject: Re: Load memory dyanamically using handler. > > On Thu, Dec 11, 2008 at 22:08, Sorin Manolache wrote: > > On Thu, Dec 11, 2008 at 19:55, Ray Morris wrote: > >> On 12/10/2008 02:09:19 AM, Sorin Manolache wrote: > >> > >>> I would propose the following approach: > >>> > >>> Hook post_config. This one is executed once by the parent process > >>> before any child is spawned. Therefore, any resource opened there is > >>> inherited by all child processes. Thus, I would create _shared_ > >>> memory > >>> structures (see apr_shm.h, not malloc). > >> ... > >>> Next, in child_init, _attach_ to the shared memory segment (see > >> ... > >>> I hope this helps. You can contact me for a code sample > >>> if you want. I don't have it handy for the moment, but > >>> I can find it. > >> > >> I've seen several variations of this same > >> question posted, so any sample code you may be > >> able to locate may be very helpful to a lot of > >> people. I'm sure I'll need it at some point, > >> so I'd file it for future use. With your permission > >> I'd post it somewhere for Google to find when people > >> want to know about ways of sharing data in Apache. > > > > Attached. > > Also, please note that in this example the shared data is trivial > enough (an integer) to not use protection against shared data > corruption through concurrent access. > > For more complicated scenarios, where some sort of protection is > needed, see apr_thread_rwlock.h. > > >> > >> On 12/10/2008 02:09:19 AM, Sorin Manolache wrote: > >>> On Wed, Dec 10, 2008 at 06:00, Jayasingh Samuel > >>> wrote: > >>> > > >>> > > >>> > I have 100's of files stored in different directories and i load > >>> all > >>> these in the memory in arrays and hash table. and i want to reload > >>> the > >>> memory automatically using some handlers. When i do this in only the > >>> particular child thread is having the updated one and the other > >>> request are showing me the old Datas. > >>> > > >>> > 1. The shared memory to store all the 100's of files in array and > >>> hash table which is dynamically malloced and stored will be too > >>> costly > >>> also the synchronization. > >>> > IS there any other way we can overcome this. > >>> > > >>> > 2. Is there any way a handler can directly access the parent > >>> process, updating the memory and removing the child process which has > >>> the old Data and spawning the new child process. Can we use the > >>> mod_balancing technique of blocking the request to the server and > >>> then > >>> update the parent and kill the child and spawn new childrens with the > >>> updated memory. > >>> > > >>> > Please guide me with you ideas. > >>> > >>> Killing processes at each update is not efficient. > >>> > >>> I would propose the following approach: > >>> > >>> Hook post_config. This one is executed once by the parent process > >>> before any child is spawned. Therefore, any resource opened there is > >>> inherited
Re: Catching apache restart signal
which apr function will return me the appropriate signal that is given to httpd? On Thu, Jan 29, 2009 at 3:31 PM, Arnab Ganguly wrote: > surely.That would be the best. > -A > > On Thu, Jan 29, 2009 at 3:27 PM, Paras Fadte wrote: >> >> Can APR be used ? >> >> On Thu, Jan 29, 2009 at 2:40 PM, Arnab Ganguly >> wrote: >> > Apache restart means,child process are killed by the process.So you have >> > to >> > write a program perhaps apache module would be the best.Catch the >> > required >> > signal and do the job.The module can be loaded from the httpd.conf. >> > >> > Writing a application putting it under usr/bin and putting the path in >> > httpd.conf can also make you achieve this. >> > Thanks >> > -A >> > >> > On Thu, Jan 29, 2009 at 1:13 PM, Paras Fadte wrote: >> >> >> >> Hi, >> >> >> >> How can one catch a signal indicating apache restart in an external >> >> program specified in httpd.conf file . Example would be rotatelogs >> >> utility or some other utility to which logs are piped . >> >> >> >> Thank you . >> >> >> >> -Paras >> > >> > > >
Re: Sending data to client
BTW, I've found this book to be excellent on how filters work, and Apache 2 modules in general: Nick Kew, _The Apache Modules Book_ Prentice Hall, 2007
Re: Using gzip and CustomLog
Hi Rainer, Can you please help me out with this ? I am using the rotatelogs utility from apache 2.0.55 Thanks. -Paras On Wed, Jan 28, 2009 at 11:20 AM, Paras Fadte wrote: > Hi , > > I have somewhat modified the rotatlogs utility to support compression > . Although it creates files in compressed format (.gz) and rotates > them properly the issue that i am facing is that when apache is > restarted (graceful or stop/start way ) the last created compressed > file doesn't seem to get closed . Is there a way to rectify this ? > For compression I am using zlib . > > Please help and thanks in advance. > > -Paras > > On Fri, Jan 23, 2009 at 1:41 PM, Paras Fadte wrote: >> Thanks Rainer, >> >> yeah.. me not a pro at development . >> >> On Fri, Jan 23, 2009 at 1:30 PM, Rainer Jung wrote: >>> On 23.01.2009 08:45, Paras Fadte wrote: Can you please tell me in which file ? >>> >>> I assume you are building rotatelogs from within the httpd sources. >>> >>> There is a file support/Makefile, which contains a line >>> >>> $(LINK) $(rotatelogs_LTFLAGS) $(rotatelogs_OBJECTS) $(PROGRAM_LDADD) >>> >>> Simply add "-lz" at the end of the line: >>> >>> $(LINK) $(rotatelogs_LTFLAGS) $(rotatelogs_OBJECTS) $(PROGRAM_LDADD) -lz >>> >>> In case you don't know what a Makefile is and how it basically works, you >>> need to read about how to do C software development. >>> >>> Regards, >>> >>> Rainer >>> On Fri, Jan 23, 2009 at 1:09 PM, Rainer Jung wrote: > > On 23.01.2009 07:55, Paras Fadte wrote: >> >> Hi, >> >> I get following error when I try to use "compress" function of zlib in >> "rotatelogs.c" . I have included "zlib.h" in rotatelogs.c . >> >> /home/paras/httpd-2.0.55/support/rotatelogs.c:294: undefined reference >> to `compress' >> collect2: ld returned 1 exit status >> >> Is it linking error ? where should I make the changes to eliminate this >> error? > > Add -lz to the linking flags. >>> >> >
[PATCH] mod_dbd with more than one pool
Hello. Once again I want to propose patch for mod_dbd module. This patch make possible to use more than one database pool. https://issues.apache.org/bugzilla/show_bug.cgi?id=45456 Also, new configuration option was added DBDInitSQL, which enables user to execute some SQL command after connecting to DB server. It can be codepage choosing command or anything else. Configuration example: DBDriver mysql DBDParams "host=172.20.30.65, port=..." DBDPersist On DBDriver mysql DBDParams "host=172.20.30.65, port=..." DBDPersist On DBDriver mysql DBDParams "host=172.20.30.65, port=..." DBDPersist On dbdtest, dbdtest2 and dbdtes3 are pool names. API is only slightly changed (new argument 'pool name' added). I am looking forward to comments and instructions how to add this to mainline http-2.2.x. Thanks in advance. -- Marko Kevac
Re: [PATCH] mod_dbd with more than one pool
Kevac Marko wrote: API is only slightly changed (new argument 'pool name' added). I am looking forward to comments and instructions how to add this to mainline http-2.2.x. You can't change the ABI, let alone the ABI, within 2.2.x. That would break the contract with third-party developers. To introduce a new API, you'd have to use a wrapper. You can propose changes in trunk, which will then feed into 2.4. Your biggest hurdle there is to get developer attention and review. Take a look at how an API changes. Contrast http://svn.apache.org/viewvc?view=rev&revision=730296 http://svn.apache.org/viewvc?view=rev&revision=732583 and see how trunk changes ABI (but preserves API) while 2.2.x preserves the ABI too. -- Nick Kew
Re: Profiling Apache
On Wed, Jan 28, 2009 at 9:25 AM, Paras Fadte wrote: > > Which tool is the most suitable for profiling Apache ? gprof ? Oprofile ? Used valgrind and gpt. -- Marko Kevac
Re: svn commit: r739150 - /httpd/httpd/trunk/os/unix/unixd.h
On 01/30/2009 03:48 AM, field...@apache.org wrote: > Author: fielding > Date: Fri Jan 30 02:48:08 2009 > New Revision: 739150 > > URL: http://svn.apache.org/viewvc?rev=739150&view=rev > Log: > revert r711228: the ap_unixd_setup_child prototype needs to go somewhere. > > Modified: > httpd/httpd/trunk/os/unix/unixd.h > > Modified: httpd/httpd/trunk/os/unix/unixd.h > URL: > http://svn.apache.org/viewvc/httpd/httpd/trunk/os/unix/unixd.h?rev=739150&r1=739149&r2=739150&view=diff > == > --- httpd/httpd/trunk/os/unix/unixd.h (original) > +++ httpd/httpd/trunk/os/unix/unixd.h Fri Jan 30 02:48:08 2009 > @@ -80,6 +80,8 @@ > } unixd_config_rec; > AP_DECLARE_DATA extern unixd_config_rec ap_unixd_config; > > +AP_DECLARE(int) ap_unixd_setup_child(void); /* mod_cgid needs this */ > + Hm. This break compilation of trunk as this symbol is now added to exports.c again with no implementation in the core (only in a module). Where did it break with -Werror prior to this change? Regards Rüdiger
Re: svn commit: r739150 - /httpd/httpd/trunk/os/unix/unixd.h
On Jan 30, 2009, at 3:07 PM, Ruediger Pluem wrote: On 01/30/2009 03:48 AM, field...@apache.org wrote: Author: fielding Date: Fri Jan 30 02:48:08 2009 New Revision: 739150 URL: http://svn.apache.org/viewvc?rev=739150&view=rev Log: revert r711228: the ap_unixd_setup_child prototype needs to go somewhere. Modified: httpd/httpd/trunk/os/unix/unixd.h Modified: httpd/httpd/trunk/os/unix/unixd.h URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/os/unix/ unixd.h?rev=739150&r1=739149&r2=739150&view=diff = = --- httpd/httpd/trunk/os/unix/unixd.h (original) +++ httpd/httpd/trunk/os/unix/unixd.h Fri Jan 30 02:48:08 2009 @@ -80,6 +80,8 @@ } unixd_config_rec; AP_DECLARE_DATA extern unixd_config_rec ap_unixd_config; +AP_DECLARE(int) ap_unixd_setup_child(void); /* mod_cgid needs this */ + Hm. This break compilation of trunk as this symbol is now added to exports.c again with no implementation in the core (only in a module). It doesn't break my compilation in trunk -- I've done two complete builds since then with no problems. Which platform and MPM? I have been testing OS X 10.4.11 with prefork. Where did it break with -Werror prior to this change? Where it is first defined: modules/arch/unix/mod_unixd.c 306:AP_DECLARE(int) ap_unixd_setup_child(void) Note that the function is only called in a few old MPMs: server/mpm/experimental/leader/leader.c 1023:if (ap_unixd_setup_child()) { server/mpm/experimental/perchild/perchild.c 862:return ap_unixd_setup_child(); server/mpm/experimental/threadpool/threadpool.c 1246:if (ap_unixd_setup_child()) { Can we move the function into those MPMs and just make it static? Or maybe mpm_common.h? Roy
Re: svn commit: r739150 - /httpd/httpd/trunk/os/unix/unixd.h
On Jan 30, 2009, at 3:30 PM, Roy T. Fielding wrote: On Jan 30, 2009, at 3:07 PM, Ruediger Pluem wrote: On 01/30/2009 03:48 AM, field...@apache.org wrote: +AP_DECLARE(int) ap_unixd_setup_child(void); /* mod_cgid needs this */ + Hm. This break compilation of trunk as this symbol is now added to exports.c again with no implementation in the core (only in a module). It doesn't break my compilation in trunk -- I've done two complete builds since then with no problems. Which platform and MPM? I have been testing OS X 10.4.11 with prefork. Let me know if r739487 fixes it. My guess is that the three old MPMs (leader, perchild, threadpool) still don't work, but I don't want to fix them if we are just going to delete them. It bothers me that I really have no idea if this code even works, given that it is supposedly the heart of a seteuid switching child (or some such). I would really appreciate it if someone who actually uses this stuff could decide what should be deleted *before* we release it. Roy
Re: svn commit: r739150 - /httpd/httpd/trunk/os/unix/unixd.h
Roy T. Fielding wrote: It bothers me that I really have no idea if this code even works, given that it is supposedly the heart of a seteuid switching child (or some such). I would really appreciate it if someone who actually uses this stuff could decide what should be deleted *before* we release it. We aren't going to release leader/perchild/threadpool. If that ever changes, it has to be because at least one committer is doing real work on it. Including sorting out issues like dropping privileges. -- Nick Kew
Re: [PATCH] mod_dbd with more than one pool
Kevac Marko wrote: Once again I want to propose patch for mod_dbd module. This patch make possible to use more than one database pool. One of the things I needed to do earlier today was determine whether mod_dbd could support more than one database pool, this patch answers that question. A definite +1 for httpd v2.3/2.4. API is only slightly changed (new argument 'pool name' added). I am looking forward to comments and instructions how to add this to mainline http-2.2.x. If you can find a way to achieve this without the ABI changing, then it can be backported to v2.2. Otherwise, it would be another reason to move on to v2.4. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: svn commit: r739150 - /httpd/httpd/trunk/os/unix/unixd.h
Roy T. Fielding wrote: > On Jan 30, 2009, at 3:30 PM, Roy T. Fielding wrote: >> On Jan 30, 2009, at 3:07 PM, Ruediger Pluem wrote: >>> On 01/30/2009 03:48 AM, field...@apache.org wrote: +AP_DECLARE(int) ap_unixd_setup_child(void); /* mod_cgid needs this */ + >>> >>> Hm. This break compilation of trunk as this symbol is now added to >>> exports.c again >>> with no implementation in the core (only in a module). >> >> It doesn't break my compilation in trunk -- I've done two complete >> builds since then with no problems. Which platform and MPM? I have >> been testing OS X 10.4.11 with prefork. > > Let me know if r739487 fixes it. My guess is that the three > old MPMs (leader, perchild, threadpool) still don't work, but > I don't want to fix them if we are just going to delete them. > > It bothers me that I really have no idea if this code even > works, given that it is supposedly the heart of a seteuid > switching child (or some such). I would really appreciate > it if someone who actually uses this stuff could decide > what should be deleted *before* we release it. imo, no one uses these mpms, we should delete them.
Re: svn commit: r739150 - /httpd/httpd/trunk/os/unix/unixd.h
Paul Querna wrote: imo, no one uses these mpms, we should delete them. ISTR having this conversation ahead of 2.2. We deleted them from 2.2 but left them in trunk for posterity, and in case anyone felt like picking it up. They look like obvious candidates for an attic now. -- Nick Kew