Re: mod_perl in larger scale environments
In data 15 aprile 2010 alle ore 05:11:15, Brad Van Sickle ha scritto: LVS does sound interesting but in your infrastructure layout aren't your single LVS load balancers single points of failure? I simplified a bit too much :) Every LVS machine has a hot-spare, and you can perform manual or automated failover. Automated failover is said to keep your connections running while migrating them over to the backup lvs. We have never had a failure, just manual failover due to upgrades, etc... I'm ignorant there. What advantages exactly? I'l be brief here because this is a mod_perl list :) [...] Thanks :) -- Cosimo
Re: mod_perl in larger scale environments
LVS does sound interesting but in your infrastructure layout aren't your single LVS load balancers single points of failure? Especially if they are running on older hardware? Maybe that isn't important in your environment? However, it seems like that negates a lot of the "high availability" goal of load balancing. It still may be a possibility for me, possibly running on the same host as my existing web layer apache instance and using a localhost connection... I will definitely look into it. Again, I'm used to working in web applications with a full blown app server. I love working with mod_perl, but I do find myself missing the advantages an app server gives me sometimes. I'm ignorant there. What advantages exactly? I'l be brief here because this is a mod_perl list :) The specific product I'm used to working with is IBM Websphere, which allows you to cluster your individual app servers and then manage them all from one administration tool. So settting or config changes, code deployments, etc... are snynced across all nodes. It makes managing app "clusters" extremely easy.It also provides a plugin to IBM's http server that handles proxying back to the application servers and provides load balancing/high availability, Those are the two advantages that address my original questions directly. App servers provide a lot of other benefits such as allowing you to leverage things like shared memory and shared DB and messaging connections/buses... many of these can be simulated in mod_perl. (Apache::DBI, etc...) On 4/14/2010 6:27 PM, Cosimo Streppone wrote: In data 14 aprile 2010 alle ore 22:57:06, Brad Van Sickle ha scritto: My first question relates to quality of service and load balancing: Hi Brad, we're using LVS (http://en.wikipedia.org/wiki/Linux_Virtual_Server), and I find it very useful and reliable. Our infrastructure for the modperl application, simplifying, consists of: a) 1 main "front" lvs load balancer b) 2 web frontends c) 1 "back" lvs load balancer d) 12 apache/modperl backends e) 5 db servers f) ...other stuff... :) a) load balances incoming traffic between the 2 web frontends. b) rewrites backend requests to a single address, "wlb" (web load balancer) which is handled by c) c) takes incoming requests for several different "virtual" hostnames, as in: wlb.domain.com => {weighted round robin to} => (back1,back2,...,back12) mlb.domain.com => {wrr} => (db1, db2, ..., db5) (mysql load balancer) s-mlb.domain.com => {wrr} => (search-db1, search-db2, ...) (search mysql lb) d) app servers are "stateless", so we don't need sticky sessions This architecture can be simplified, and we're trying to do it. So, I'm not saying this is the best practice or not even sane. :) LVS performs health checking via HTTP requests, with or without md5 checksum of the responses, or direct TCP connections to the port you specify (f.ex. for db servers). I'm currently using mod_proxy on the web layer, and I know I can set that up to load balance requests to multiple app layer nodes, but to the best of my knowledge mod_proxy is not able to provide any quality of service. So if a node in the app layer had a problem (or was shut down for maintenance) mod_proxy would be unaware of that and would still send requests to that node. That's where LVS is useful. LVS can do direct routing or tcp handoff IIRC, and we're using it. The client and servers talk directly to each other, without taking up too much resources on the LVS machine itself. I'd rather not use a hardware load balancer here if I can avoid it. LVS usually runs on our older less powerful machines. My second question deals with management of multiple mod_perl nodes: At some point, if you have enough app layer nodes, managing the code deployments, apache configs and server restarts becomes very cumbersome if you're doing it all manually. We're using a simple but limited in-house tool that basically uses rsync, ssh, and keeps list of hosts w/ roles. Currently for a pilot project I used puppet for config management and fabric as "last mile" deployment tool. So far I'm happy with the result. Are there any tools that can make these tasks easier or give me one management view? I don't know. Everything we've done is command line based, so it's not very friendly. Actually I'm currently looking into higher level tools to integrate what we've done. I also looked at ControlTier, but it feels too heavy for me. Needs its own (powerful) machine. I'd be glad to hear experiences on it. Again, I'm used to working in web applications with a full blown app server. I love working with mod_perl, but I do find myself missing the advantages an app server gives me sometimes. I'm ignorant there. What advantages exactly?
Re: [RELEASE CANDIDATE] Apache-Test-1.32 RC1
On 2010-04-13, at 3:41 PM, Fred Moyer wrote: > Please take a couple minutes to test this release candidate [1] for > Apache::Test 1.32 and report back success or failure. Thanks! +1 Tests pass on OS X 10.6.3, Apache/2.2.11 (prefork MPM), and v5.10.0 for darwin-thread-multi-2level. -- best regards, Randy
Re: mod_perl in larger scale environments
I concur with LVS. I have LVS running on a $10 piece of hardware (300 MHz CPU, 128MB memory) that acts as a load balancer for 15+ web servers. I use keepalive to monitor the systems. Dzuy Cosimo Streppone wrote: In data 14 aprile 2010 alle ore 22:57:06, Brad Van Sickle ha scritto: My first question relates to quality of service and load balancing: Hi Brad, we're using LVS (http://en.wikipedia.org/wiki/Linux_Virtual_Server), and I find it very useful and reliable. Our infrastructure for the modperl application, simplifying, consists of: a) 1 main "front" lvs load balancer b) 2 web frontends c) 1 "back" lvs load balancer d) 12 apache/modperl backends e) 5 db servers f) ...other stuff... :) a) load balances incoming traffic between the 2 web frontends. b) rewrites backend requests to a single address, "wlb" (web load balancer) which is handled by c) c) takes incoming requests for several different "virtual" hostnames, as in: wlb.domain.com => {weighted round robin to} => (back1,back2,...,back12) mlb.domain.com => {wrr} => (db1, db2, ..., db5) (mysql load balancer) s-mlb.domain.com => {wrr} => (search-db1, search-db2, ...) (search mysql lb) d) app servers are "stateless", so we don't need sticky sessions This architecture can be simplified, and we're trying to do it. So, I'm not saying this is the best practice or not even sane. :) LVS performs health checking via HTTP requests, with or without md5 checksum of the responses, or direct TCP connections to the port you specify (f.ex. for db servers). I'm currently using mod_proxy on the web layer, and I know I can set that up to load balance requests to multiple app layer nodes, but to the best of my knowledge mod_proxy is not able to provide any quality of service. So if a node in the app layer had a problem (or was shut down for maintenance) mod_proxy would be unaware of that and would still send requests to that node. That's where LVS is useful. LVS can do direct routing or tcp handoff IIRC, and we're using it. The client and servers talk directly to each other, without taking up too much resources on the LVS machine itself. I'd rather not use a hardware load balancer here if I can avoid it. LVS usually runs on our older less powerful machines. My second question deals with management of multiple mod_perl nodes: At some point, if you have enough app layer nodes, managing the code deployments, apache configs and server restarts becomes very cumbersome if you're doing it all manually. We're using a simple but limited in-house tool that basically uses rsync, ssh, and keeps list of hosts w/ roles. Currently for a pilot project I used puppet for config management and fabric as "last mile" deployment tool. So far I'm happy with the result. Are there any tools that can make these tasks easier or give me one management view? I don't know. Everything we've done is command line based, so it's not very friendly. Actually I'm currently looking into higher level tools to integrate what we've done. I also looked at ControlTier, but it feels too heavy for me. Needs its own (powerful) machine. I'd be glad to hear experiences on it. Again, I'm used to working in web applications with a full blown app server. I love working with mod_perl, but I do find myself missing the advantages an app server gives me sometimes. I'm ignorant there. What advantages exactly?
Re: mod_perl in larger scale environments
In data 14 aprile 2010 alle ore 22:57:06, Brad Van Sickle ha scritto: My first question relates to quality of service and load balancing: Hi Brad, we're using LVS (http://en.wikipedia.org/wiki/Linux_Virtual_Server), and I find it very useful and reliable. Our infrastructure for the modperl application, simplifying, consists of: a) 1 main "front" lvs load balancer b) 2 web frontends c) 1 "back" lvs load balancer d) 12 apache/modperl backends e) 5 db servers f) ...other stuff... :) a) load balances incoming traffic between the 2 web frontends. b) rewrites backend requests to a single address, "wlb" (web load balancer) which is handled by c) c) takes incoming requests for several different "virtual" hostnames, as in: wlb.domain.com => {weighted round robin to} => (back1,back2,...,back12) mlb.domain.com => {wrr} => (db1, db2, ..., db5) (mysql load balancer) s-mlb.domain.com => {wrr} => (search-db1, search-db2, ...) (search mysql lb) d) app servers are "stateless", so we don't need sticky sessions This architecture can be simplified, and we're trying to do it. So, I'm not saying this is the best practice or not even sane. :) LVS performs health checking via HTTP requests, with or without md5 checksum of the responses, or direct TCP connections to the port you specify (f.ex. for db servers). I'm currently using mod_proxy on the web layer, and I know I can set that up to load balance requests to multiple app layer nodes, but to the best of my knowledge mod_proxy is not able to provide any quality of service. So if a node in the app layer had a problem (or was shut down for maintenance) mod_proxy would be unaware of that and would still send requests to that node. That's where LVS is useful. LVS can do direct routing or tcp handoff IIRC, and we're using it. The client and servers talk directly to each other, without taking up too much resources on the LVS machine itself. I'd rather not use a hardware load balancer here if I can avoid it. LVS usually runs on our older less powerful machines. My second question deals with management of multiple mod_perl nodes: At some point, if you have enough app layer nodes, managing the code deployments, apache configs and server restarts becomes very cumbersome if you're doing it all manually. We're using a simple but limited in-house tool that basically uses rsync, ssh, and keeps list of hosts w/ roles. Currently for a pilot project I used puppet for config management and fabric as "last mile" deployment tool. So far I'm happy with the result. Are there any tools that can make these tasks easier or give me one management view? I don't know. Everything we've done is command line based, so it's not very friendly. Actually I'm currently looking into higher level tools to integrate what we've done. I also looked at ControlTier, but it feels too heavy for me. Needs its own (powerful) machine. I'd be glad to hear experiences on it. Again, I'm used to working in web applications with a full blown app server. I love working with mod_perl, but I do find myself missing the advantages an app server gives me sometimes. I'm ignorant there. What advantages exactly? -- Cosimo
Re: mod_perl in larger scale environments
On Wed, Apr 14, 2010 at 3:15 PM, Brad Van Sickle wrote: > So it sounds like Apache is simply not going to meet my needs. In the event > that I do need to replace Apache, hopefully you can save me some research > time and recommend me one of the listed options that fulfills my needs (or > confirm that perlbal does) You may want to try Apache with the event mpm using mod_proxy. I haven't used that, I ended up using Perlbal mostly so I could customize the code without having to deal with intermittent segfaults like I did when I patched apache and got something wrong. > I need the following features: > 1) provides support for named virtual hosts > 2) supports SSL to the client > 3) supports URL rewriting (similar to mod_rewrite) > 4) knows the availability of pool members and provides high availability. > 5) the ability to serve static content itself I think perlbal does all of this. Getting it configured to do all of this may not be straightforward, but the perlbal list is very helpful for that. That being said, I've heard really great things about Varnish, so I'd try that if I didn't have perlbal. > I guess 5 isn't strictly neccessary, but it would be nice to serve static > content (css/js/images/etc...) from the same piece of technology without > proxying those requests to another Apache instance running on the same host > (or something) > > Thanks for all the help! > > On 4/14/2010 5:48 PM, Perrin Harkins wrote: >> >> On Wed, Apr 14, 2010 at 5:33 PM, Brad Van Sickle >> wrote: >> >>> >>> I didn't find much info on perlbal after a quick glance, I'll certainly >>> give >>> it a closer look, but my inital reaction is that I'm leary of replacing >>> Apache on my web layer. I'm doing a few things with a few other modules ( >>> mod_rewrite for example) in addition to mod_proxy, and from what I was >>> able >>> to find in my initial look, I didn't see any support for some of those >>> types >>> of things. >>> >> >> There are many full-featured proxy servers these days. There's even >> mod_proxy_balancer for apache, but that doesn't do high-availability, >> which you're looking for. >> >> Check out some of these for reverse-proxying if you don't like perlbal: >> - nginx >> - lighttpd >> - varnish >> - pound >> >> All of those can serve as mod_perl frontends. >> >> - Perrin >> >
Re: mod_perl in larger scale environments
So it sounds like Apache is simply not going to meet my needs. In the event that I do need to replace Apache, hopefully you can save me some research time and recommend me one of the listed options that fulfills my needs (or confirm that perlbal does) I need the following features: 1) provides support for named virtual hosts 2) supports SSL to the client 3) supports URL rewriting (similar to mod_rewrite) 4) knows the availability of pool members and provides high availability. 5) the ability to serve static content itself I guess 5 isn't strictly neccessary, but it would be nice to serve static content (css/js/images/etc...) from the same piece of technology without proxying those requests to another Apache instance running on the same host (or something) Thanks for all the help! On 4/14/2010 5:48 PM, Perrin Harkins wrote: On Wed, Apr 14, 2010 at 5:33 PM, Brad Van Sickle wrote: I didn't find much info on perlbal after a quick glance, I'll certainly give it a closer look, but my inital reaction is that I'm leary of replacing Apache on my web layer. I'm doing a few things with a few other modules ( mod_rewrite for example) in addition to mod_proxy, and from what I was able to find in my initial look, I didn't see any support for some of those types of things. There are many full-featured proxy servers these days. There's even mod_proxy_balancer for apache, but that doesn't do high-availability, which you're looking for. Check out some of these for reverse-proxying if you don't like perlbal: - nginx - lighttpd - varnish - pound All of those can serve as mod_perl frontends. - Perrin
Re: mod_perl in larger scale environments
On Wed, Apr 14, 2010 at 5:33 PM, Brad Van Sickle wrote: > I didn't find much info on perlbal after a quick glance, I'll certainly give > it a closer look, but my inital reaction is that I'm leary of replacing > Apache on my web layer. I'm doing a few things with a few other modules ( > mod_rewrite for example) in addition to mod_proxy, and from what I was able > to find in my initial look, I didn't see any support for some of those types > of things. There are many full-featured proxy servers these days. There's even mod_proxy_balancer for apache, but that doesn't do high-availability, which you're looking for. Check out some of these for reverse-proxying if you don't like perlbal: - nginx - lighttpd - varnish - pound All of those can serve as mod_perl frontends. - Perrin
Re: mod_perl in larger scale environments
Fred: Thanks for the recommendations. Puppet/cfmanager look like they are along the lines of what I'm looking for. I didn't find much info on perlbal after a quick glance, I'll certainly give it a closer look, but my inital reaction is that I'm leary of replacing Apache on my web layer. I'm doing a few things with a few other modules ( mod_rewrite for example) in addition to mod_proxy, and from what I was able to find in my initial look, I didn't see any support for some of those types of things. On 4/14/2010 5:02 PM, Fred Moyer wrote: On Wed, Apr 14, 2010 at 1:57 PM, Brad Van Sickle wrote: My first question relates to quality of service and load balancing: I'm currently using mod_proxy on the web layer, and I know I can set that up to load balance requests to multiple app layer nodes, but to the best of my knowledge mod_proxy is not able to provide any quality of service. So if a node in the app layer had a problem (or was shut down for maintenance) mod_proxy would be unaware of that and would still send requests to that node. How are situations like this normally handled? Is there something I can use other than mod_proxy that is intelligent enough to mark a host as down? I'd rather not use a hardware load balancer here if I can avoid it. Check out perlbal - http://search.cpan.org/dist/Perlbal/ The load balancing is really nice, and you can handle 10k's of clients with junkyard hardware. My second question deals with management of multiple mod_perl nodes: At some point, if you have enough app layer nodes, managing the code deployments, apache configs and server restarts becomes very cumbersome if you're doing it all manually. Are there any tools that can make these tasks easier or give me one management view? Puppet and cfmanager are general purpose systems administration tools that serve well here. In addition, you can manage the rest of the system with these tools.
Re: accessing environment variables set by other modules
On Wed, Apr 14, 2010 at 5:17 PM, Chris Datfung wrote: > What I'm looking for is a way to access the environment variables stored in > the internal Apache structure. Any leads of how to access these environment > variables is much appreciated. The subprocess_env info that Adam sent you should have worked. Can you show us what you tried? Can you try it in a response handler to make sure it's not an odd bug with filters? - Perrin
Re: accessing environment variables set by other modules
On Tue, Apr 13, 2010 at 9:57 PM, Chris Datfung wrote: > On Tue, Apr 13, 2010 at 6:34 PM, Fred Moyer wrote: > >> Correct me if I'm wrong, but don't you need to do this: >> >> PerlPassEnv TE >> > Hi Fred, After a bit more research, It seems that PerlPassEnv is just for passing shell environment variables, Apache maintains a separate environment, as documented here: http://httpd.apache.org/docs/2.1/env.html What I'm looking for is a way to access the environment variables stored in the internal Apache structure. Any leads of how to access these environment variables is much appreciated. Thanks, Chris
Re: mod_perl in larger scale environments
On Wed, Apr 14, 2010 at 1:57 PM, Brad Van Sickle wrote: > My first question relates to quality of service and load balancing: > I'm currently using mod_proxy on the web layer, and I know I can set that up > to load balance requests to multiple app layer nodes, but to the best of my > knowledge mod_proxy is not able to provide any quality of service. So if a > node in the app layer had a problem (or was shut down for maintenance) > mod_proxy would be unaware of that and would still send requests to that > node. How are situations like this normally handled? Is there something I > can use other than mod_proxy that is intelligent enough to mark a host as > down? I'd rather not use a hardware load balancer here if I can avoid it. Check out perlbal - http://search.cpan.org/dist/Perlbal/ The load balancing is really nice, and you can handle 10k's of clients with junkyard hardware. > My second question deals with management of multiple mod_perl nodes: > At some point, if you have enough app layer nodes, managing the code > deployments, apache configs and server restarts becomes very cumbersome if > you're doing it all manually. Are there any tools that can make these tasks > easier or give me one management view? Puppet and cfmanager are general purpose systems administration tools that serve well here. In addition, you can manage the rest of the system with these tools.
mod_perl in larger scale environments
Hello I have a lot of experience in large scale web applications using Java and Websphere, but I now find myself needing to scale a web application built on mod_perl, and I have some questions about best practices for doing that since I don't have any sort of deployment manager or an intelligent HTTP plugin. I currently have an application set up in the standard 3 tiered model: Apache web layer -> Apache/mod_perl app layer -> MySQL DB layer. Right now I have one app layer node, but traffic is dictating that I need to expand capacity there soon and I plan on adding more hosts to that layer. My first question relates to quality of service and load balancing: I'm currently using mod_proxy on the web layer, and I know I can set that up to load balance requests to multiple app layer nodes, but to the best of my knowledge mod_proxy is not able to provide any quality of service. So if a node in the app layer had a problem (or was shut down for maintenance) mod_proxy would be unaware of that and would still send requests to that node. How are situations like this normally handled? Is there something I can use other than mod_proxy that is intelligent enough to mark a host as down? I'd rather not use a hardware load balancer here if I can avoid it. My second question deals with management of multiple mod_perl nodes: At some point, if you have enough app layer nodes, managing the code deployments, apache configs and server restarts becomes very cumbersome if you're doing it all manually. Are there any tools that can make these tasks easier or give me one management view? Again, I'm used to working in web applications with a full blown app server. I love working with mod_perl, but I do find myself missing the advantages an app server gives me sometimes. Hopefully someone can offer me some suggestions here. Thanks