RE: Load memory dyanamically using handler.
Hi all, I tried this. struct check { int data; } *strucSup; shmid = shmget(key, 32768 * sizeof(strucSup), IPC_CREAT | 0666)) 0); This was working and the value inserted in one process was seen in other process. But when i tried this. struct check { int *data; Where data is integer array.. Dynamically we need to allocate so that we can store data. } *strucSup I tried to shmget for each data pointer but this is not good idea since it creates a lot of small Shared memory. and we need to have the key for calling all the data. for (i = 0; i MAX_IDXFILE; i++) { testshm[i].data2 = shmget(i, 5 * sizeof(int), IPC_CREAT | 0666)) 0); testshm[i].data2 = shmat(shmid, NULL, 0)) == (int *) -1); for (j = 0; j 5; j++) { strucSup[i].data[j] = j; } } I think this is not good idea since it can lead to segmentFault. My question is: 1. Is that we need to create our own library function to fix this, which takes chunk of shared memory and allocates space in that memory region got from the shmget. 2. Is there any apache api which supports similar task. Please let me know with your ideas. Thanks in advance, Jaysingh. From: [EMAIL PROTECTED] To: modules-dev@httpd.apache.org CC: [EMAIL PROTECTED] Subject: RE: Load memory dyanamically using handler. Date: Thu, 11 Dec 2008 13:16:48 +0530 Thanks for your input. Here is my problem i have 100's of files in different directories. and we store these files. ie the files in an Array and further hash table. Every time we do graceful the server to refresh the contents and that's why i need to dynamically reload the memory using some handlers . I need to dynamic create space for these structure and store the hash entries and datas. typedef struct abc{ hash_t *table; } gInfoDB; typedef struct xyz{ gInfoDB *ctDB; } InfoDB; InfoDB[1]; The server memory will have some max 1 of InfoDB array in which the ctDB is malloced dynamically and store the contents of the files in the hash table. My question is: 1.The shared memory with which we get a chunk of memory. Can we further malloc and store the data's given by the shared memory. or we need to create keys and shared memory for every arrays. I have tried for arrays of structure and its works fine. But i was not able to do this. struct check { int *datum; } *strucSup; Here i was trying to malloc the datum and was able to store the data but getting segFault when tring through some other process. I can use further shmget the datum pointer but i don't want to do because this may create so many shared memory in my systems. Is there any work around to do this. Please guide me with the coding if you get. Thanks in advance, Jaysingh. Date: Wed, 10 Dec 2008 09:09:19 +0100 From: [EMAIL PROTECTED] To: modules-dev@httpd.apache.org Subject: Re: Load memory dyanamically using handler. CC: [EMAIL PROTECTED] On Wed, Dec 10, 2008 at 06:00, Jayasingh Samuel [EMAIL PROTECTED] wrote: I have 100's of files stored in different directories and i load all these in the memory in arrays and hash table. and i want to reload the memory automatically using some handlers. When i do this in only the particular child thread is having the updated one and the other request are showing me the old Datas. 1. The shared memory to store all the 100's of files in array and hash table which is dynamically malloced and stored will be too costly also the synchronization. IS there any other way we can overcome this. 2. Is there any way a handler can directly access the parent process, updating the memory and removing the child process which has the old Data and spawning the new child process. Can we use the mod_balancing technique of blocking the request to the server and then update the parent and kill the child and spawn new childrens with the updated memory. Please guide me with you ideas. Killing processes at each update is not efficient. I would propose the following approach: Hook post_config. This one is executed once by the parent process before any child is spawned. Therefore, any resource opened there is inherited by all child processes. Thus, I would create _shared_ memory structures (see apr_shm.h, not malloc). Beware, post_config is run every time the server configuration is reloaded. Therefore, such shared memory structures would be created after each config reload. In order to avoid eating up the system's resources, you'll have to make sure that each created structure is also destroyed. In order to do this, _register_ a function (see apr_pool_register_cleanup) that is called when the conf pool is destroyed. This function would destroy the shared memory structures. Next, in child_init, _attach_ to the shared memory
Re: Load memory dyanamically using handler.
On 12/10/2008 02:09:19 AM, Sorin Manolache wrote: I would propose the following approach: Hook post_config. This one is executed once by the parent process before any child is spawned. Therefore, any resource opened there is inherited by all child processes. Thus, I would create _shared_ memory structures (see apr_shm.h, not malloc). ... Next, in child_init, _attach_ to the shared memory segment (see ... I hope this helps. You can contact me for a code sample if you want. I don't have it handy for the moment, but I can find it. I've seen several variations of this same question posted, so any sample code you may be able to locate may be very helpful to a lot of people. I'm sure I'll need it at some point, so I'd file it for future use. With your permission I'd post it somewhere for Google to find when people want to know about ways of sharing data in Apache. -- Ray B. Morris supp...@bettercgi.com Strongbox - The next generation in site security: http://www.bettercgi.com/strongbox/ On 12/10/2008 02:09:19 AM, Sorin Manolache wrote: On Wed, Dec 10, 2008 at 06:00, Jayasingh Samuel jayasingh.sam...@hotmail.com wrote: I have 100's of files stored in different directories and i load all these in the memory in arrays and hash table. and i want to reload the memory automatically using some handlers. When i do this in only the particular child thread is having the updated one and the other request are showing me the old Datas. 1. The shared memory to store all the 100's of files in array and hash table which is dynamically malloced and stored will be too costly also the synchronization. IS there any other way we can overcome this. 2. Is there any way a handler can directly access the parent process, updating the memory and removing the child process which has the old Data and spawning the new child process. Can we use the mod_balancing technique of blocking the request to the server and then update the parent and kill the child and spawn new childrens with the updated memory. Please guide me with you ideas. Killing processes at each update is not efficient. I would propose the following approach: Hook post_config. This one is executed once by the parent process before any child is spawned. Therefore, any resource opened there is inherited by all child processes. Thus, I would create _shared_ memory structures (see apr_shm.h, not malloc). Beware, post_config is run every time the server configuration is reloaded. Therefore, such shared memory structures would be created after each config reload. In order to avoid eating up the system's resources, you'll have to make sure that each created structure is also destroyed. In order to do this, _register_ a function (see apr_pool_register_cleanup) that is called when the conf pool is destroyed. This function would destroy the shared memory structures. Next, in child_init, _attach_ to the shared memory segment (see apr_shm.h). From then on, the shared memory structures are shared among all child processes. Each change is visible instantly in all processes. Obviously, think of synchronisation. When not every atomic change results in a consistent state of the shared data, you'll have to protect the shared data with mutexes. See apr_thread_mutex.h and apr_thread_rwlock.h and maybe others. Mutexes should be created in post_config too. I hope this helps. You can contact me for a code sample if you want. I don't have it handy for the moment, but I can find it. S -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail?
Re: Load memory dyanamically using handler.
On Thu, Dec 11, 2008 at 19:55, Ray Morris supp...@bettercgi.com wrote: On 12/10/2008 02:09:19 AM, Sorin Manolache wrote: I would propose the following approach: Hook post_config. This one is executed once by the parent process before any child is spawned. Therefore, any resource opened there is inherited by all child processes. Thus, I would create _shared_ memory structures (see apr_shm.h, not malloc). ... Next, in child_init, _attach_ to the shared memory segment (see ... I hope this helps. You can contact me for a code sample if you want. I don't have it handy for the moment, but I can find it. I've seen several variations of this same question posted, so any sample code you may be able to locate may be very helpful to a lot of people. I'm sure I'll need it at some point, so I'd file it for future use. With your permission I'd post it somewhere for Google to find when people want to know about ways of sharing data in Apache. Attached. Sorin On 12/10/2008 02:09:19 AM, Sorin Manolache wrote: On Wed, Dec 10, 2008 at 06:00, Jayasingh Samuel jayasingh.sam...@hotmail.com wrote: I have 100's of files stored in different directories and i load all these in the memory in arrays and hash table. and i want to reload the memory automatically using some handlers. When i do this in only the particular child thread is having the updated one and the other request are showing me the old Datas. 1. The shared memory to store all the 100's of files in array and hash table which is dynamically malloced and stored will be too costly also the synchronization. IS there any other way we can overcome this. 2. Is there any way a handler can directly access the parent process, updating the memory and removing the child process which has the old Data and spawning the new child process. Can we use the mod_balancing technique of blocking the request to the server and then update the parent and kill the child and spawn new childrens with the updated memory. Please guide me with you ideas. Killing processes at each update is not efficient. I would propose the following approach: Hook post_config. This one is executed once by the parent process before any child is spawned. Therefore, any resource opened there is inherited by all child processes. Thus, I would create _shared_ memory structures (see apr_shm.h, not malloc). Beware, post_config is run every time the server configuration is reloaded. Therefore, such shared memory structures would be created after each config reload. In order to avoid eating up the system's resources, you'll have to make sure that each created structure is also destroyed. In order to do this, _register_ a function (see apr_pool_register_cleanup) that is called when the conf pool is destroyed. This function would destroy the shared memory structures. Next, in child_init, _attach_ to the shared memory segment (see apr_shm.h). From then on, the shared memory structures are shared among all child processes. Each change is visible instantly in all processes. Obviously, think of synchronisation. When not every atomic change results in a consistent state of the shared data, you'll have to protect the shared data with mutexes. See apr_thread_mutex.h and apr_thread_rwlock.h and maybe others. Mutexes should be created in post_config too. I hope this helps. You can contact me for a code sample if you want. I don't have it handy for the moment, but I can find it. S -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail? -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail? #include httpd.h #include http_log.h #include http_config.h #include unistd.h #include apr_errno.h #include apr_pools.h #include apr_shm.h typedef int shared_struct_t; static apr_shm_t *shm; static shared_struct_t *shared_object; static apr_status_t conf_pool_cleanup(apr_shm_t *shm) { return apr_shm_destroy(shm); } static int post_config(apr_pool_t *conf_pool, apr_pool_t *log_pool, apr_pool_t *temp_pool, server_rec *s) { apr_status_t r; r = apr_shm_create(shm, sizeof(shared_struct_t), NULL, conf_pool); if (APR_SUCCESS != r) return (int)r; shared_object = (shared_struct_t *)apr_shm_baseaddr_get(shm); apr_pool_cleanup_register(conf_pool, shm, (apr_status_t (*)(void *))conf_pool_cleanup, NULL); return OK; } static int handler(request_rec *r) { if (NULL != r-args) *shared_object = atoi(r-args); else ap_rprintf(r, %d\n, *shared_object); return OK; } static int log_transaction(request_rec *r) { ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, PID: %d, getpid()); return OK; } static void register_hooks(apr_pool_t *p) { ap_hook_post_config(post_config, NULL, NULL,
Re: Load memory dyanamically using handler.
On Thu, Dec 11, 2008 at 22:08, Sorin Manolache sor...@gmail.com wrote: On Thu, Dec 11, 2008 at 19:55, Ray Morris supp...@bettercgi.com wrote: On 12/10/2008 02:09:19 AM, Sorin Manolache wrote: I would propose the following approach: Hook post_config. This one is executed once by the parent process before any child is spawned. Therefore, any resource opened there is inherited by all child processes. Thus, I would create _shared_ memory structures (see apr_shm.h, not malloc). ... Next, in child_init, _attach_ to the shared memory segment (see ... I hope this helps. You can contact me for a code sample if you want. I don't have it handy for the moment, but I can find it. I've seen several variations of this same question posted, so any sample code you may be able to locate may be very helpful to a lot of people. I'm sure I'll need it at some point, so I'd file it for future use. With your permission I'd post it somewhere for Google to find when people want to know about ways of sharing data in Apache. Attached. Also, please note that in this example the shared data is trivial enough (an integer) to not use protection against shared data corruption through concurrent access. For more complicated scenarios, where some sort of protection is needed, see apr_thread_rwlock.h. On 12/10/2008 02:09:19 AM, Sorin Manolache wrote: On Wed, Dec 10, 2008 at 06:00, Jayasingh Samuel jayasingh.sam...@hotmail.com wrote: I have 100's of files stored in different directories and i load all these in the memory in arrays and hash table. and i want to reload the memory automatically using some handlers. When i do this in only the particular child thread is having the updated one and the other request are showing me the old Datas. 1. The shared memory to store all the 100's of files in array and hash table which is dynamically malloced and stored will be too costly also the synchronization. IS there any other way we can overcome this. 2. Is there any way a handler can directly access the parent process, updating the memory and removing the child process which has the old Data and spawning the new child process. Can we use the mod_balancing technique of blocking the request to the server and then update the parent and kill the child and spawn new childrens with the updated memory. Please guide me with you ideas. Killing processes at each update is not efficient. I would propose the following approach: Hook post_config. This one is executed once by the parent process before any child is spawned. Therefore, any resource opened there is inherited by all child processes. Thus, I would create _shared_ memory structures (see apr_shm.h, not malloc). Beware, post_config is run every time the server configuration is reloaded. Therefore, such shared memory structures would be created after each config reload. In order to avoid eating up the system's resources, you'll have to make sure that each created structure is also destroyed. In order to do this, _register_ a function (see apr_pool_register_cleanup) that is called when the conf pool is destroyed. This function would destroy the shared memory structures. Next, in child_init, _attach_ to the shared memory segment (see apr_shm.h). From then on, the shared memory structures are shared among all child processes. Each change is visible instantly in all processes. Obviously, think of synchronisation. When not every atomic change results in a consistent state of the shared data, you'll have to protect the shared data with mutexes. See apr_thread_mutex.h and apr_thread_rwlock.h and maybe others. Mutexes should be created in post_config too. I hope this helps. You can contact me for a code sample if you want. I don't have it handy for the moment, but I can find it. S -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail? -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail? -- A: Because it reverses the logical flow of conversation. Q: Why is top-posting frowned upon? A: Top-posting. Q: What is the most annoying thing in e-mail?
Re: mod_fcgid incubation?
On Dec 10, 2008, at 8:05 PM, Roy T. Fielding wrote: On Dec 10, 2008, at 4:51 AM, Jim Jagielski wrote: On Dec 9, 2008, at 4:56 PM, Nick Kew wrote: It's unfortunate there's no clear copyright statement, but would it not be reasonable to assume Copyright Pan Qingfeng and deal with him? Contact other contributors as a courtesy, but not let it worry us if some of them prove uncontactable, only if someone actually objects? Submitting a patch or code does not imply copyright assignment. So we cannot assume copyright Pan for the whole thing. Right, but not all code has a separate copyright. Repairs, for example, do not. So it is actually feasible that Pan can determine who needs to be contacted based on the change log and only contact those whose contributions were more than repairs to Pan's work. +1
ap_expr.h interface review
typedef enum { TOKEN_STRING, TOKEN_RE, TOKEN_AND, } token_type_t; } token_t; } backref_t; ... all lack namespace-safety. as do: typedef const char *(*string_func_t)(request_rec*, const char*); typedef int (*opt_func_t)(request_rec*, ap_parse_node_t*, string_func_t); which are also undocumented and lacking parameter names. What should such callbacks do/return? AP_DECLARE(ap_parse_node_t*) ap_expr_parse(apr_pool_t *pool, const char *expr, int *was_error); the was_error parameter seems to be redundant with the function returning NULL to indicate error? The grammar of the expression being parsed is undocumented. (well, I could guess that it might happen to match the grammar documented in the mod_include docs, but could everybody guess that?) The purpose of the the *_func_t callbacks is pretty unclear without reference to the grammar. The naming of the TOKEN_ACCESS constant seems to be a throwback to the use of this API in mod_include? AP_DECLARE(apr_status_t) ap_expr_init(apr_pool_t *pool); this is a purely internal interface so needn't be exposed to modules AFAICT. Regards, Joe
mod_fcgid license questions
Hi -- I believe Pan Qingfeng (潘庆峰), the developer of mod_fcgid, has joined this list for the time being while the possibility of mod_fcgid becoming project in the Apache incubator is discussed. I'll use his English name of Ryan Pan from here on. I asked Ryan to join so that he could answer the couple of questions regarding the origin of the mod_fcgid code and his licensing intentions which have come up so far, and also to thank him publicly for being willing to consider contributing mod_fcgid to the ASF. Ryan, the main question which has come up in the last couple of days seems to be this one: When you wrote mod_fcgid, was there any code which you borrowed from mod_fastcgi? The other questions I had related to the existing license for mod_fcgid: Your current intention is for mod_fcgid to be available under the GPL version 2.0, correct? Could you confirm that you wanted the GPL to apply to all the mod_fcgid code? (I ask because the LICENSE file in mod_fcgid contains the GPL 2.0, however, the .c and .h files don't also include the usual GPL text.) Finally, Ryan, would you mind re-stating for the record your interest in the idea of mod_fcgid becoming an Apache project? I'd like to personally extend my thanks to Ryan for developing mod_fcgid in the first place, for his interest in the idea of contributing it to the ASF, and for being willing to work through the licensing issues that will involve. Many thanks! (I'd also like to thank my colleague Sharon or Xiaomei Ma (笑梅), an excellent developer in her own right, for her help translating some of the communications Ryan and I have already had.) Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: Authz directives
On Dec 9, 2008, at 10:45 AM, Chris Darroch wrote: This is all fairly simple, I think, especially if MatchNotAny/RequireNone is removed as well so that Require retains its apparent meaning everywhere. See if the patch below meets your expectations; if so, I'll commit it and update the docs. Sorry, here's a slightly updated one without any whitespace issues; it should apply cleanly against the latest SVN trunk as of a few minutes ago. I don't see a problem with RequireNone inverting the logic, and I think it would actually be useful for blocking a set of bad clients. Is it difficult to include that without MatchNotAny? In any case, I'd prefer that we clean this up before the next attempt at an alpha. I wanted to do it this week but have run out of time (I don't want to commit and fly). Can you commit this or something like it? Roy
Re: Authz directives
Roy T. Fielding wrote: I don't see a problem with RequireNone inverting the logic, and I think it would actually be useful for blocking a set of bad clients. Is it difficult to include that without MatchNotAny? Not at all difficult; trivial, in fact. The only reason I took it out as well was the name inversion issue. I'll put it back in and commit tomorrow. Might take a few days to deal with the doc adjustments, but otherwise a simple job. Thanks for the comments! Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: mod_fcgid license questions
Hi, guys Nice to meet you :) I hope I can help to clarify the questions. When you wrote mod_fcgid, was there any code which you borrowed from mod_fastcgi? No. I didn't borrow any code from mod_fastcgi. Your current intention is for mod_fcgid to be available under the GPL version 2.0, correct? Could you confirm that you wanted the GPL to apply to all the mod_fcgid code? Yes, I confirm I wanted the GPL version 2.0 apply to everything. Finally, Ryan, would you mind re-stating for the record your interest in the idea of mod_fcgid becoming an Apache project? Actually I didn't care too much about which license I should choose. The reason I chose GPL is that all other license descriptions are in English and are tricky. I didn't want to spend much time on studying those terms so I just followed most of people to choose GPL. I intended to put it as open source so people can share. I don't really care which license it is under. (Thanks Miss Xiaomei for this native translation) I was single when I developed this project, but now I have to spend more time with my family, and I am now actually interesting on LinuxTV things(So I can spend time on something both I and my wife need :). But I would like to see someone else take over my mod_fcgid project and continune to make it better. I get a lot from the others(that why I like open source and share), and I would like to share to the other if I have a chance. It's my pleasure if ASF willing to take it over. Anything I can help please let me know :) Thanks - Original Message - From: Chris Darroch chr...@pearsoncmg.com To: dev@httpd.apache.org Sent: Friday, December 12, 2008 6:15 AM Subject: mod_fcgid license questions Hi -- I believe Pan Qingfeng (潘庆峰), the developer of mod_fcgid, has joined this list for the time being while the possibility of mod_fcgid becoming project in the Apache incubator is discussed. I'll use his English name of Ryan Pan from here on. I asked Ryan to join so that he could answer the couple of questions regarding the origin of the mod_fcgid code and his licensing intentions which have come up so far, and also to thank him publicly for being willing to consider contributing mod_fcgid to the ASF. Ryan, the main question which has come up in the last couple of days seems to be this one: When you wrote mod_fcgid, was there any code which you borrowed from mod_fastcgi? The other questions I had related to the existing license for mod_fcgid: Your current intention is for mod_fcgid to be available under the GPL version 2.0, correct? Could you confirm that you wanted the GPL to apply to all the mod_fcgid code? (I ask because the LICENSE file in mod_fcgid contains the GPL 2.0, however, the .c and .h files don't also include the usual GPL text.) Finally, Ryan, would you mind re-stating for the record your interest in the idea of mod_fcgid becoming an Apache project? I'd like to personally extend my thanks to Ryan for developing mod_fcgid in the first place, for his interest in the idea of contributing it to the ASF, and for being willing to work through the licensing issues that will involve. Many thanks! (I'd also like to thank my colleague Sharon or Xiaomei Ma (笑梅), an excellent developer in her own right, for her help translating some of the communications Ryan and I have already had.) Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B