Re: mod_perl and IPC
On Mon, 22 May 2000, DeWitt Clinton wrote: The problem had to do with large numbers of objects in the cache. ... Right now, things are in a holding pattern because I'm finding a limit on the number of objects I can put in the cache (less than 100, so it is an issue). Hopefully Sam will offer some insight here. I ran into this with IPC::SharedCache a couple of months ago and had some discussion with Sam about it. The problem is a lack of shared memory segments and/or semaphores compiled into the kernel. IIRC, the default for both under Linux is 128. The BSD system I was trying to use only had 32 segments and something like 10 semaphore identifiers. In order to scale a system, you'll need to recompile the kernel with higher limits. - Matt
Re: mod_perl and IPC
Hi there, On Mon, 22 May 2000, Reilly, Thomas wrote: I have to find out what the best method is for having mod_perl have bi-directional communication with a unix daemon process which will receive a string (for simplicity!) from mod_perl, (extracted from the client request), talk to existing c-code which accesses and uses the string to update a database. The database will trigger a response which has to get back to the client via mod_perl! I think you should consider incorporating the program into a Perl module instead of having it run as a separate daemon if you think it's at all possible. Your daemon could be a bottleneck and you could find that the extra layer of complexity gives you grief you don't need. There are easy ways to talk to databases from inside Perl modules. Look in the Guide, which is at http://perl.apache.org/guide I want to also ensure my machine doesn't fall over after receiving multiple requests Yup, copy that. How much RAM in your machine? Any help would be greatly appreciated as i'm kinda feeling around in the dark at the moment Hey, join the club! 73, Ged. Thomas Reilly Software Consultant, Distributed Software Consultancy Ltd., Ballybrit Business Park,Galway,Ireland. Tel: +353 (91) 760541 Fax: +353 (91) 760542 e-mail: [EMAIL PROTECTED] web: http://www.dscie.com
RE: mod_perl and IPC
IPC::Cache has worked like a champ for me on a heavily loaded webserver. It's simple and (I'm lead to believe) it's a compiled module. You interact with it like a hash file, but if you're clever you can work it out for your usage without semaphores. It also behaves itself very nicely (no memory leaks that I have run into). Good luck! -Original Message- From: Ged Haywood [mailto:[EMAIL PROTECTED]] Sent: Monday, May 22, 2000 11:47 AM To: [EMAIL PROTECTED] Subject: Re: mod_perl and IPC Hi there, On Mon, 22 May 2000, Reilly, Thomas wrote: I have to find out what the best method is for having mod_perl have bi-directional communication with a unix daemon process which will receive a string (for simplicity!) from mod_perl, (extracted from the client request), talk to existing c-code which accesses and uses the string to update a database. The database will trigger a response which has to get back to the client via mod_perl! I think you should consider incorporating the program into a Perl module instead of having it run as a separate daemon if you think it's at all possible. Your daemon could be a bottleneck and you could find that the extra layer of complexity gives you grief you don't need. There are easy ways to talk to databases from inside Perl modules. Look in the Guide, which is at http://perl.apache.org/guide I want to also ensure my machine doesn't fall over after receiving multiple requests Yup, copy that. How much RAM in your machine? Any help would be greatly appreciated as i'm kinda feeling around in the dark at the moment Hey, join the club! 73, Ged. Thomas Reilly Software Consultant, Distributed Software Consultancy Ltd., Ballybrit Business Park,Galway,Ireland. Tel: +353 (91) 760541 Fax: +353 (91) 760542 e-mail: [EMAIL PROTECTED] web: http://www.dscie.com
Re: mod_perl and IPC
On Mon, May 22, 2000 at 12:06:20PM -0700, Karyn Ulriksen wrote: IPC::Cache has worked like a champ for me on a heavily loaded webserver. It's simple and (I'm lead to believe) it's a compiled module. You interact with it like a hash file, but if you're clever you can work it out for your usage without semaphores. It also behaves itself very nicely (no memory leaks that I have run into). Hi, I'm the author of IPC::Cache and frankly, I'm a little surprised it's worked so well for you. :) I've noticed some performance issues with it. The good news, however, is that Sam Tregar, author of IPC::SharedCache, and I have been working on a major upgrade. In the next release, IPC::Cache will be back-ended against IPC::SharedCache module. It will now scale and perform much better. The other alternative is to use File::Cache (also available on CPAN). File::Cache actually has better performance characteristics than IPC::Cache. Since switching between the two usually entails simply replacing the string "IPC::Cache" with "File::Cache" it may be worth checking out. I could use a few beta testers of the new IPC::Cache module. If anyone wants to help, please let me know. -DeWitt
RE: mod_perl and IPC
Great News! But it is true. It works great. I monitor the servers closely because I have quite a lot packed into them and the PHP module likes to get ugly once in a while. I use the IPC::Cache module to simply stash information I retrieve from a database about VirtualHost configurations. Solaris 7 Sparc is happy with it. Hope when you team up with the IP::SharedCache that I don't run into the build issues that I ran into when I looked into using it before! Cheers! -Original Message- From: DeWitt Clinton [mailto:[EMAIL PROTECTED]] Sent: Monday, May 22, 2000 11:36 AM To: Karyn Ulriksen Cc: '[EMAIL PROTECTED]' Subject: Re: mod_perl and IPC On Mon, May 22, 2000 at 12:06:20PM -0700, Karyn Ulriksen wrote: IPC::Cache has worked like a champ for me on a heavily loaded webserver. It's simple and (I'm lead to believe) it's a compiled module. You interact with it like a hash file, but if you're clever you can work it out for your usage without semaphores. It also behaves itself very nicely (no memory leaks that I have run into). Hi, I'm the author of IPC::Cache and frankly, I'm a little surprised it's worked so well for you. :) I've noticed some performance issues with it. The good news, however, is that Sam Tregar, author of IPC::SharedCache, and I have been working on a major upgrade. In the next release, IPC::Cache will be back-ended against IPC::SharedCache module. It will now scale and perform much better. The other alternative is to use File::Cache (also available on CPAN). File::Cache actually has better performance characteristics than IPC::Cache. Since switching between the two usually entails simply replacing the string "IPC::Cache" with "File::Cache" it may be worth checking out. I could use a few beta testers of the new IPC::Cache module. If anyone wants to help, please let me know. -DeWitt
Re: mod_perl and IPC
Karyn Ulriksen wrote: Great News! But it is true. It works great. I monitor the servers closely because I have quite a lot packed into them and the PHP module likes to get ugly once in a while. I use the IPC::Cache module to simply stash information I retrieve from a database about VirtualHost configurations. Solaris 7 Sparc is happy with it. Hope when you team up with the IP::SharedCache that I don't run into the build issues that I ran into when I looked into using it before! Could I ask what the problems you were having? I can't use IPC::SharedCache currently because I get the error "Couldn't find free segment" or something like that when trying to initialize the cache. The system engineers are supposed to be changing this kernel parameter, but I haven't heard back from them yet. -- Drew Taylor Vialogix Communications, Inc. 501 N. College Street Charlotte, NC 28202 704 370 0550 http://www.vialogix.com/
Re: mod_perl and IPC
DeWitt Clinton wrote: Hi, I'm the author of IPC::Cache and frankly, I'm a little surprised it's worked so well for you. :) I've noticed some performance issues with it. The good news, however, is that Sam Tregar, author of IPC::SharedCache, and I have been working on a major upgrade. In the next release, IPC::Cache will be back-ended against IPC::SharedCache module. It will now scale and perform much better. Cool! I have successfully used IPC::SharedCache on our Linux based servers and it worked well. I will be making more use of IPC memory in the future to spare the DB lookups for common info like customer data, etc. I could use a few beta testers of the new IPC::Cache module. If anyone wants to help, please let me know. I'd be interested in helping. Fast shared memory is going to be my good friend once my project begins to scale. :-) What do you need from beta testers? -- Drew Taylor Vialogix Communications, Inc. 501 N. College Street Charlotte, NC 28202 704 370 0550 http://www.vialogix.com/
Re: mod_perl and IPC
Drew Taylor wrote: Karyn Ulriksen wrote: Great News! But it is true. It works great. I monitor the servers closely because I have quite a lot packed into them and the PHP module likes to get ugly once in a while. I use the IPC::Cache module to simply stash information I retrieve from a database about VirtualHost configurations. Solaris 7 Sparc is happy with it. Hope when you team up with the IP::SharedCache that I don't run into the build issues that I ran into when I looked into using it before! Could I ask what the problems you were having? I can't use IPC::SharedCache currently because I get the error "Couldn't find free segment" or something like that when trying to initialize the cache. The system engineers are supposed to be changing this kernel parameter, but I haven't heard back from them yet. To clarify, the problem is with Solaris 2.6 on sparc hardware. Oops. -- Drew Taylor Vialogix Communications, Inc. 501 N. College Street Charlotte, NC 28202 704 370 0550 http://www.vialogix.com/
RE: mod_perl and IPC
I could use a few beta testers of the new IPC::Cache module. If anyone wants to help, please let me know. Count me in.
Re: mod_perl and IPC
On Mon, May 22, 2000 at 04:35:54PM -0400, Drew Taylor wrote: Could I ask what the problems you were having? I can't use IPC::SharedCache currently because I get the error "Couldn't find free segment" or something like that when trying to initialize the cache. The system engineers are supposed to be changing this kernel parameter, but I haven't heard back from them yet. The problem had to do with large numbers of objects in the cache. Rather than rephrase what Sam wrote to me, I'll just excerpt it here: In case you're wondering why I'm saying IPC::Cache needs help, I remember discussing two points. First, it's storing every cached object in the same giant IPC segment and copying it into local memory on every cache access. This essentially means that you are gaining *no* memory-usage advantage by using shared memory. Also, it wastes a lot of time copying large hunks of memory on every access. Second, you're doing reads inside an exclusive lock. This means that no two programs can be reading at the same time. This wastes the potential for efficient multi-processing with shared memory. He was absolutely correct about this. Since he is actively maintaining and improving IPC::SharedCache, I figured it makes sense to simply provide the straightforward IPC::Cache style interface around his code. Right now, things are in a holding pattern because I'm finding a limit on the number of objects I can put in the cache (less than 100, so it is an issue). Hopefully Sam will offer some insight here. -DeWitt
Re: mod_perl and IPC
On Mon, May 22, 2000 at 03:46:41PM -0700, Karyn Ulriksen wrote: I know what beta testing is, so I hope you didn't think I was trying to yank your chain! I knew that if it got ugly I could easily take that server offline and easily back out the code. Ack! I'm sorry for what I think is a miscommunication. I think the cordial tone was lost in e-mail. :) I was being sincere, I definitely appreciate your testing the code. As it was, it got even uglier since I last wrote in. Again this is FYI... for input on the beta and all... The load balancing service I'm using masked the more serious problem of the data failing with get passed somewhere in the bowels of Apache/mod_perl where it was before. I'll tinker with it in the wee hours to see if it is failing on the set or the retrieve. I reviewed the syntax and there doesn't appear to be any changes between new and old (did I miss anything?... it's a little late in the day and I had a long weekend :) and it's possible I did and I didn't review for any caveats... ). I'm pretty sure you are doing exactly the right thing. It is certainly possible that the new version doesn't work at all. I'd love to figure out a way to make this version work, because it does address some of the concerns that Sam pointed out. However, if there is degraded performance (which may be an understatement) then I won't release version 0.04 like this. Again, thanks for the highly valuable feedback! -DeWitt
